A Tool for Hardening Java Crypto

Identifying cryptographic vulnerabilities in today's million-line programs has become a critical endeavor. Because of the increasing sophistication of cybercriminals, programmers can no longer afford to test for vulnerabilities using only traditional debugging techniques, followed by releasing software, collecting bug reports and patching.

The new frontier being pursued by government, industry, and academia are automated tools that are capable of culling vulnerabilities before releasing source code into the wild. When run on existing software, such as the open-source Apache programs managing the world's servers, these tools also are finding a surprising number of vulnerabilities in software that is decades old.

Most open-source automated vulnerability checkers are still finding their way, but a team of researchers at the Virginia Polytechnic Institute and State University (Virginia Tech) claim to have vulnerability-checking software that is mature, and approaching deployment. Called CryptoGuard, the software automatically identifies cryptographic vulnerabilities in Java (and soon Python) source code. Funded by the U.S. Navy's Office of Naval Research (ONR) and the National Science Foundation (NSF), CryptoGuard is poised to hacker-proof large-scale computer programs, according to CryptoGuard: High Precision Detection of Cryptographic Vulnerabilities in Massive-sized Java Projects.

Said Michael Hicks, professor in the department of computer science at the University of Maryland in College Park, and in the University of Maryland's Institute for Advanced Computer Studies. "The automated discovery of errors in programs that use cryptography is an extremely important problem. Such programs tend to rely on third-party cryptography libraries, which is good, but as numerous studies have shown, programmers misuse those libraries' application programming interfaces [APIs] in a way that introduces unforeseen vulnerabilities.

He added that Cryptoguard "is able to find many such misuses automatically, by scanning program code during development, prior to deployment."

Hicks, who was not involved in Cryptoguard's development, said the tool is "noteworthy for its combination of scale and precision. In comparison, other tools that work on large codebases are often imprecise, leading to a deluge of false alarms, while tools with few false alarms either miss bugs or don't scale to large codebases."

He said the CryptoGuard tool "can persuade industry teams to start scanning and improving the security of their code now, plus it can inspire researchers to employ its techniques to begin developing other precise, scalable analysis tools."

Said Cristina Cifuentes, director of Oracle Labs Australia (Brisbane), which specializes in program analysis as it applies to finding vulnerabilities and enhancing productivity, "Much code today needs to make use of cryptographic APIs. However, developers using crypto APIs are not necessarily crypto experts, and as such it is not always clear to them how to best use crypto APIs, leading to unexpected crypto mistakes in the code despite the fact that the code compiles correctly and the compiler does not emit a crypto misuse error."

Cifuentes identified a couple of the main issues with static software testing tools in general, including the accuracy of the tool (both precision and recall), as well as the runtime performance of the tool. CryptoGuard achieves both high precision (most of the reports are correct true positives) and recall (it doesn't miss many real mistakes from being reported).

Oracle Labs Australia tested CryptoGuard extensively, eventually improving its own vulnerability checking software by incorporating CryptoGuard's open-source algorithms into its own proprietary tools, according to Cifuentes.

CryptoGuard specializes in large Java programs because Java historically has been beset by cybersecurity vulnerabilities, especially for very large programs with millions of lines of code (LoC). Fledgling testing software resulted in too many false alarms (often called false positives) when inspecting for cryptographic vulnerabilities. CryptoGuard has been tuned to reduce false alarms to a very small number, according to Virginia Tech's Danfeng Yao, a co-author of the study.

"Building deployment-quality tools is drastically more complex than writing prototypes designed to expose vulnerabilities," said Yao. "Deployed open-source solutions just do not exist today for crypto coding." The CryptoGuard tool, she said, systematically shows "how various program-analysis design choices impact the detection quality, in terms of precision, recall, and scalability."

Yao claims that in testing, CryptoGuard achieved 98.6% precision by confirming 1,277 vulnerabilities out of 1,295 alarms on Apache Java programs.

Cryptographic algorithms are meant to guarantee security against hacker assailants, but new code is often riddled with vulnerabilities unknown to the programmers writing it. Yao blames the programming community in general for not providing the rigorous training required to hone cryptographic skills. Also at fault, she said, are Internet forums giving out misleading "tips," and the fact that many programming tools are "hard-to-use and have poor certificate-verification methodologies and/or cross-language encryption/decryption algorithms," resulting in poor performance.

While static analysis tools (which observe code while it is not running) scale up well, cover a wider range of security rules, and miss fewer vulnerabilities than dynamic tools (that observe code while it is running), the advantage of the dynamic method, according to Yao, is that it produces fewer false alarms and is better at detecting complicated vulnerabilities. Her team's goal was to achieve the best of both worlds in a static tool.

"Our goal was a static analysis tool that has no, or at least as few as possible, false positives," said Yao.

Unfortunately, static methods—historically—do not scale well to programs with millions of lines-of-code, and have hitherto not been able to detect complicated vulnerabilities. On the other hand, "CryptoGuard was designed to be a higher accuracy static analysis solution that scales well when detecting all types of cryptographic vulnerabilities," said Yao.

CryptoGuard refines the program-slicing method of spotting errors, improving efficiency by identifying and eliminating from the slice all the language-specific irrelevant parameters, benign constants, harmless arguments, and bookkeeping values, which eliminates 70% to 80% of false alarms, said Yao.

Yao said she and her team now are working to create a set of tools to automatically create programs that can detect vulnerabilities specifically targeted by a software developer, as gleaned from their natural language descriptions. Yao said this could enable non-experts to roll out detection solutions for new vulnerabilities as they are discovered.

"Our current focus is on developing a compiler that can read natural language-like statements describing detection policies, and then automatically output software that people can run to screen their code," said Yao.

R. Colin Johnson is a Kyoto Prize Fellow who ​​has worked as a technology journalist ​for two decades.