Can you guess which one is mine?
Can you guess which one is mine?
In my own experience there’s been very little relationship between the
quality of a resume and the eventual usefulness of a developer. I’ve seen guys
with great work history, references, advanced degrees, numerous publications,
and so on, and yet their presence proved less valuable than their absence.
Meanwhile some of the most rewarding engineers I’ve worked with introduced
themselves with nothing more than a simple letter.
At a previous company I worked with in the dot-com era, we created an epic
test for long distance interviews for a Perl programmer/ Linux sysadmin role.
It consisted of questions that a veteran hacker would maybe know 80 or 90% off
the top of his head, and exactly which man pages to lookup for another 10%.
Cute stuff like “How can you rm a file named -rf?” and “Name
3 things you can accomplish at a GRUB prompt.” We would arrange a designated
time and email the applicant the test. They had one hour (which we would pay
them for) to return it. The test was so long and specific there was no hope of
completion if you needed Google’s help for a large portion of the answers. The
feedback from many applicants was elaborately negative.
These days our process is more to the point. If we’re considering brining
someone on staff, we start by giving them some work to do. We find detachable
development tasks that will further the SpiderOak cause, send them a minimal
set of instructions, and let them run with it. It’s usually something
smallish, 1 – 3 days at most. As an all telecommute team, we’re already
accustomed to giving code feedback. When they’re done, they send us a bill and
we send them a review.
Sometimes we give several people the same task. The results often show an
obvious contrast of strengths and weaknesses across several applicants, and it
conserves the (sometimes scarce) resource of development tasks that don’t
require detailed knowledge of core SpiderOak source code. Sometimes we’re not
sure after the first task so we give more.
I’m sure there are big corporate HR departments who would be astonished to
learn that the best predictor of a developer’s usefulness might be an ability
to complete development tasks.
My (future) children might one day know of their grandmother by the orchard
of black walnut trees she planted, even knowing the long growth times meant
they were unlikely to yield within her lifetime. “For the next generation,” she
Have you ever considered the modern day scenario for how you may be
remembered by the next generation? If one of your great grandparents had kept a
blog, even if they had only updated once every few months, would you read it at
some point in your life and be glad it was there?
When one of my university classmates died unexpectedly in a tragic accident,
his friends and family renewed his domain and prepaid his hosting arrangements
for a long time.
The challenge of long term archival of digitally stored data is not simple. If
you discovered that one of your ancestors had left you an important message,
saved on an 80s era floppy disk, could you conveniently read it today? Even if
you could find the necessary hardware, the disk is unlikely to be error free.
Even magnetic media can be damaged by time. Some commonly used optical media
often develop data errors in just a few years.
For these reasons, SpiderOak is now offering ‘forever’ accounts. You make a one
time payment, and your account remains active in perpetuity. Your SpiderOak
share rooms stay online, your data is forever accessible. The account does not
expire due to inactivity, or the death of the owner.
When the world transitions from one storage technology to the next (holographic
drives, protein memory, who knows?) SpiderOak administrators will have already
migrated your data to new devices during the normal course of everyday routine
maintenance and hardware upgrades.
Some may wonder how this is a viable business model. Is SpiderOak simply trying
to collect advance payments for impossible services, ignoring the cost of
rendering those services long term? Would this not represent a significant and
unwise liability for SpiderOak to accept?
The answer to this question is in Moore’s Law, which predicts that the capacity
of technologies double every two years. This prediction has been accurate for
decades now, and it shows no signs of changing. For example, this means that a
100 gigabyte SpiderOak account you purchased today will cost the company half
as much to maintain in two years. In four years, one forth as much, and 1/32
in 10 years. In short: the cost of fulfilling the permanent service we sell
today approaches zero after many years.
That makes sense thinking backwards also. In 1998, how much personal data might
a typical individual have accumulated? Add up all the emails, word processing
documents, low resolution (by today’s standards) digital photographs, and the
entirety of everything digital a person might have. A few gigabytes would have
been considered a rather large collection. In 1998, a 20 gigabyte hard drive
might have cost what a 500 gigabyte drive does now. Today we give 2 gigabyte
SpiderOak accounts away for free!
So, what’s a reasonable price for something timeless? There are established
formulas for the time value of money. Let’s consider a model for a 20 gigabyte
account, which we currently charge $10/mo for. That represents a payment stream
of $10 a month for two years, followed by $5 per month for two years, and $2.50
per month in years 5 and 6, and so on, all the way out to 30 years when the
projected annual cost is less than $0.01. Then I added a minimum $5/year cost
into the model, because regardless of the effects of Moore’s law, there are
still some human costs, such as providing customer service, that are never
Using this model, at a 6% rate, the present value of the 20 gig forever
account about $500. For our 100 gigabyte accounts, the present value is $2118,
and for right now we’re giving a volume discount and offering these at $1000.
You can upgrade a ‘forever’ account when you need more space just by paying the
difference in cost between what you’ve already paid, and the larger plan you
wish to purchase.
I have started inviting everyone in my family to send me genealogy records,
journals, pictures, or anything else they think would be interesting to
descendants and future historians, and I’m putting it online in my own
SpiderOak account, in share rooms known to our family. I recommend that they
save in open formats like plain text, jpeg, or HTML whenever possible, and add
as much information as they can, like who are the people in the pictures, or
what the occasion was and any story they can contribute.
It is my hope that SpiderOak is remembered by history not as just a
contemporary software company, but also as a link from one era to the next, and
from this others will know a little more about the walnuts they’re eating.
Last week we released the 1.4x series builds of SpiderOak out of beta.
Download here. Existing installs
on Windows and OS X should self-update, and the newest Linux packages are in
the apt. Most users seem to have upgraded already.
In addition to many internal improvements, users are likely to notice a huge
responsiveness and speed boost: we estmiate 4x faster than the predecessor.
There’s also several new command line options, and SpiderOak now supports
backup queues larger than physical memory, so go ahead and install on machines
with hundreds of thousands of files.
A new web feature you may have noticed: each SpiderOak share room now has an
associated RSS feed which will indicate when items in a share room are changed
or updated with newer versions. So, for example, everytime you add new
pictuers to a shared folder, or you make changes and a new version of a
document in one of your shared folders is archived, subscribers to the RSS feed
will automatically notice without you having to do anything extra.
See the complete release
notes for more information.
The newest released version of SpiderOak supports --batchmode
scheduled operation may be useful to command line users and “GUI only” people
alike. The command line version is considerably faster for most tasks (3-4x by
my estimation), and uses drastically less memory (For me on OS X, an average VM
size of 32meg, peak at 64.)
This is supported in versions 1.0.3753 and newer (released today.) On
Windows and OS X, an existing SpiderOak install should automatically upgrade
the next time it connects to the server. On Ubuntu or Debian, the apt upgrade
process should get the newest version.
Here’s what you can do (so far) from the command line:
Alan@Alan ~ $ /Applications/SpiderOak.app/Contents/MacOS/SpiderOak --help Usage: SpiderOak basic command line usage: Options: -h, --help show this help message and exit --print-selection Print a list of selected and excluded backup items --reset-selection Reset selection (but preserve excluded files) --exclude-file=EXCLUDE_FILE Exclude the given file from the selection --exclude-dir=EXCLUDE_DIR Exclude the given directory from the selection --include-dir=INCLUDE_DIR Include the given directory in the selection --force Do in/exclusion even if the path doesn't exist --headless Never start the GUI --batchmode set the config option exit_when_nothing_to_do to true
Most of these are self explanatory. --headless and
--batchmode are the ones I use most often. We’ll be adding support
for much more command line control in the future — send mail to cmdline at
spideroak.com if you want to suggest other options.
--headless just runs SpiderOak with no GUI at all. It just runs,
without printing anything to the console, so there’s no interactiveness or
activity indicators (except what’s written to the spideroak.log.) This
is suitable for use on servers or other environments where you want something
to run continuously, using as few resources as possible, without any user
By the way, one of the benefits to a fault tollerant application design, is
that you don’t have to be nice to it. Feel free to force quit or kill (even
-9) at any time, and SpiderOak will rollback any uncommitted transactions, and
resume uploading or building where it left of — without corruption — the next
time you start. If you need all the available bandwidth to your first person
shooter, Skype, or you’re just trying to make your battery last as long as
possible, just killall SpiderOak and restart it when you want backups
The next option is --batchmode (which implies --headless).
This means that SpiderOak will do all available work (i.e. scan the filesystem,
then build and upload everything in the queue, download and replay transactions
from other devices), and then exit. This is a good option for scheduled use.
You can add this to a cron job, or just run it yourself periodically whenever
you want to update your backup set.
SpiderOak is also careful not to start more than one instance of itself at a
time. For example, if you schedule SpiderOak to run in --batchmode
each night, and for the first few days, SpiderOak has so much to upload that it
does not finish before the next scheduled startup time, you don’t need to worry
about coming back to find several instances running.
In the next major release of SpiderOak, we’re restructuring the user
interface to be equally or more efficient as the command line version is now.
So, we expect the 1.5.0 series GUI versions to be several times faster than the
1.0.0 series GUI versions are today.