Does SpiderOak backup open files

This article has moved to our new Help Center and will no longer be updated on this page. Please see the Help Center for the newest version.

SpiderOak uses a combination of techniques to Do The Right Thing with regard to handling open, changed, or busy files. First, SpiderOak does not try to get a lock on the file -- we want SpiderOak to operate without ever interfering with regular file system access or altering the filesystem in any way. SpiderOak makes an initial reading-only pass over the file and calculates some cryptographic checksums. Then, it begins to build (encrypt and covert to blocks) a new version of the file.

If SpiderOak notices that the checksums of the new version it built are different than the checksums it created during the first pass, it will assume that the file was changed while being built. SpiderOak will try another couple of times to get a consistent read/build of the file. If the file continues to change, that file will be added to a sort of quarantine and ignored for a period of time. If there is still a problem after the file comes out of quarantine, the length of quarantine time is doubled and the process repeats (we call this "exponential backoff.")

In any case, SpiderOak keeps all historical versions of each file so if the file it builds on a given pass happened to be in an internally inconsistent (mid-change) state, you could simply download the previous version. Also, since the file has changed, SpiderOak would of course store another version in the very next filesystem pass.

We think this approach to dealing with busy files creates the best combination of non-intrusiveness (not disturbing other applications or modifying the file system) and reliability. In our testing, it is also very rare that any of these issues ever occur.