Protecting Against Data Breaches; Living With Mistakes

http://cacm.acm.org/blogs/blog-cacm/107800
April 27, 2011 (updated April 30, 2012)

Just to recount, here are some of the more prominent breaches in the past 18 months:

• HBGary Federal, a beltway computer security firm, had all of its email stolen and made available on BitTorrent. This incident also raised a lot of questions about the ethics of the work it was being paid to do in the first place.
• Comodo Group had its systems breached, and several fake browser certificates were created along the way.
• Databases used to maintain RSA SecurID tokens were breached using a combination of a spear-phishing attack and a zero-day Flash exploit.
• The Epsilon mailing list service, which maintains mailing lists for many corporations, had its databases hacked, quite possibly through a phishing attack.
• The PlayStation Network was hacked, with more than 65 million accounts compromised, including names, street addresses, email addresses, and purchase histories stolen. It is not clear from Sony’s public statement whether credit card and password data was stolen. [Update: Credit card and password data was stolen, but Sony says the credit card info was protected by industry standard encryption.]
• In a self-referential loop, members of the Anonymous group released recordings of a conference call between the FBI and Scotland Yard discussing Anonymous.
• Global Payments Inc. had a data breach that may involve up to 1.5 million credit and debit card accounts.

The scary part is, it is only going to get worse.

There has been a rise in the sophistication of attacks over the past years. These attackers, ranging from script kiddies to criminals to state-sponsored cyberwarriors, have been all too successful in breaking into online systems.

There are two interesting observations here. The first is that many of these attacks have shifted from just directly attacking a computer system, an attack model that computer security specialists are somewhat good at defending against, toward also exploiting the human vulnerabilities in these systems.

By human vulnerabilities, I mean all of the misunderstandings, laxness, and cognitive and social biases that arise with the people who use computer systems. The list of human vulnerabilities here are numerous: poor interfaces that are difficult to understand, interfaces that are easy to misconfigure, guessable passwords, reused passwords, tricking people into installing malware, tricking people into opening up documents (which might use zero-day exploits), and on, and on, and on.

These human vulnerabilities are clearly a major weakness, but are also a puzzling blind spot from both a research perspective and an industry perspective. Well, actually, it is not that puzzling. People are messy. We all have a wide range of experiences, knowledge, and motivations, and so it is natural and tempting to just buy that "magic black box" that claims to solve all your security problems and avoid having to actually deal with the messiness. Just go to any industry conference, like RSA, and you will see what I mean—vendors selling row after row of magic black boxes. And by doing so, computer professionals can rest comfortably in the conventional wisdom that users are just stupid, users are to blame, and not actually force ourselves to adapt to smarter attackers.

The other observation is that none of these hacks are actually new or innovative (and, no, most zero-day attacks are not that innovative unless you count buffer-overflow attacks as innovative). We do not know yet about the Sony PlayStation Network attack, but I am willing to bet it did not use any advanced techniques either. [Update: Sony has described the attack as an "external intrusion," but appears to have provided no other details.]

Instead, what is fascinating here is the sophistication of the execution. The attackers are more patient, adept at using a wide range of tools, and very capable of progressively exploiting smaller vulnerabilities into larger ones. It is analogous to the transition in martial arts from single schools of thought to cross-training and mixed-martial arts. These recent attacks have been highly creative, flexible to the situation, and make full use of a combination of techniques.

For example, in the Epsilon case, it looks like the attackers were using spear-phishing attacks for several months, trying to bait low-level employees at several mailing list companies. While there are few details, there are several interesting questions. How did the attackers know which employees to target? Did the attackers know how to format the spear-phishing email so it looked appropriate? Did they know when to send the malware for maximum effect? I suspect the answer to all these questions is yes.

The HBGary case is also an impressive case study. It started with a SQL injection on the company’s Web site, which led to stealing the password file, running a standard password crack, exploiting password reuse to remotely wipe the (now former) CEO’s iPad and access his Twitter account, and a spearphishing attack to get the password for another site. Again, all standard techniques, but just well executed.

These kinds of security breaches are a long-term problem that we will be struggling with for the next 10–20 years. The only effective strategy I see is to take a three-pronged approach. First, get research and industry to develop new best practices, create better tools, and have better training of software developers. Second, complement these technical approaches with a stronger legal structure that can properly incentivize companies to take stronger measures in protecting customer data. Finally, develop new ways of actually addressing the human vulnerabilities, in the form of simpler and better-designed user interfaces, more research to gain a deeper understanding of human biases and social influences in decision-making, and better ways of motivating and training people so they are effective in the face of these ongoing attacks.

Back to Top

Reader Comments

The single common thread to the above (and most) breaches is the failure to architecturally separate data. Everything in a lump with access rules is more convenient than designing systems that put what is needed where it is needed. Significant breaches come when someone (or system) has access to more data than they need. Rules can be broken, architectural separation is harder–you have to hop domains, not just abuse the rules. It is still possible, it is just a lot more work.

—James Byrd

Back to Top

Greg Linden "Embracing Noise or Why Computer Scientists Should Stop Worrying and Learn to Love the Errors"

http://cacm.acm.org/blogs/blog-cacm/107864
April 28, 2011

When people talk about differences between computers and the brain, they often emphasize the massively parallel processing of the brain. There are hundreds of billions of neurons in the brain firing at a relatively slow rate compared to just a few but very fast processors in a modern computer.

There is another very important difference between the brain and computers: tolerance of error. Computers are derailed if even a couple bits are flipped during a computation. Computers expect everything to be perfect. Error recovery is an afterthought, a protection against what is expected to be a rare event of a data read error, and perfection is assumed.

By contrast, our wetware embraces error. The brain is a cacophony, a battle of competing patterns, from which understanding emerges. It is a system built on noise, one that expects constant error and recovery from error. The brain abandons precision and thrives on approximation.

There are examples of embracing noise and error in computing. Machine translation has made great strides by building likely translation rules from patterns in large amounts of data, then expecting, welcoming, and correcting frequent errors in the translated text. Recommender systems make no assumptions that they are perfect, but only that they might be helpful, and only expect to be judged by how useful they are compared to alternatives for lists of movies or books you might like. Web search spreads a query across hundreds or thousands of machines and simply ignores those that fail to respond quickly, relying on getting a good enough answer out quickly rather than getting a complete answer out slowly.

Often we find estimates are good enough. Recent work on data mining in large clusters, such as Hadoop, has found orders of magnitude speed-ups can be gained if only an approximate answer—such as +-1% with 95% probability—is required. Some computer security researchers are embracing the idea that classifying what is malware is not black or white, but a gray area, and are starting to use techniques, such as slowing suspicious processes, rather than expecting to be able to identify and kill all wrongdoers.

Of the lessons we take from biological systems, one should be that wetware is sloppy. And that is okay. Precision is not required in everything or even most things. Failures are best handled by expecting them all the time, not treating them as exceptions. We should expect errors and handle them routinely. We should stop worrying and learn to love the noise.

Back to Top