Artificial Precognition ­ses Data to See the Future

Scientists are mining large datasets to figure out all sorts of useful information, from previously unrecognized drug interactions to methods of identifying people in photographs. Now one company is asking, why not use the power of data analytics to predict the future?

London-based startup Massive Analytic has developed a technology it is calling "artificial precognition," which can look at large sets of data from various sources, apply machine learning to recognize patterns, and make predictions about future events ranging from surges in demand for particular products to imminent cyberattacks. "It’s not crystal ball gazing," says George Frangou, founder and president of Massive Analytic. "The root of all this is understanding what’s going on and making realistic models of the world."

The company released its Oscar AP platform to the Microsoft Azure marketplace this summer, and companies such as Lockheed-Martin are testing the product. It runs on Hadoop and works with a browser, a design Frangou says is aimed at making the platform accessible to people who are not data scientists.

The software combines various statistical methods used in artificial intelligence, such as random forests, decision trees, and both Bayesian and fuzzy logic, with methods such as regression and time series, and textual analytics such as sentiment analysis. It takes a two-step approach, first applying machine learning algorithms to sets of both structured and unstructured data from multiple sources; once it has processed the data to create simplified datasets, it then applies predictive analytic techniques.

For instance, a company such as Amazon wants to streamline its supply chain, stocking plenty of the products it knows customers will want but not overstocking products for which there will be less demand, and holding merchandise convenient to where it will be delivered to speed up shipping. "You have to have a pretty accurate idea of what people want to buy, when they want to buy it, and how they want it shipped," Frangou says. The company has its own database of historic customer behavior, of course, but with Oscar AP it can also examine other data, such as what is trending on social media, and how warm or wet the weather is forecast to be, to come up with predictions of which products are about to become hot, thus shortening decision times.

A health service might look at its data on patients with respiratory illnesses, and combine that with information about weather and pollution measurements to predict when those patients will be coming back to the doctor. Alternatively, the system, applying machine learning to thousands of hours of video of crowds, could learn patterns of behavior and then be able, when looking at a live video feed of a crowd, to predict when a fight was about to break out. In the financial world, it might tell analysts when commodity prices were likely to rise or fall.

Lockheed Martin is validating the performance of Oscar AP in its Virtual Technology Cluster, a project designed to help small U.K. companies develop cybersecurity technologies. The idea, says Bradley Pietras, vice president for engineering and technology at Lockheed Martin U.K., is to incorporate Oscar AP into some of the company’s own products aimed at law enforcement or military customers; if it works, the computer should be able to pick up correlations between events that might not be identified any other way, and alert law enforcement to, say, patterns of movements by drug traffickers or terrorist groups. By focusing on patterns in the data, and not on any specific question, the software might be able to see warning signs for which law enforcement never thought to look.

The ability to make predictions could come in handy in other areas, Pietras points out. For instance, software might be able to look at how people with different genetic profiles respond to one drug, and use that data to predict how they will react to another. It could be used by electrical utilities to predict peaks in energy demand and plan accordingly. "It’s a very general technology that can be applied to a number of different problems, both in the scientific community as well as in the commercial community," Pietras says.

Two other companies make systems with similar capabilities, Frangou says: IBM, which is developing its Jeopardy-winning Watson computer for healthcare applications, and Palantir, which makes software for business analytics, cybersecurity, and law enforcement applications. However, Frangou says, those systems require a level of expertise by the users that is not required by Oscar AP, which can be useful for data scientists, but also has simple-to-use browser-based commands that allow a non-expert to pose queries while the platform automatically analyzes various data sources. "We’re basically offering ease of use and this ability to automate as our main selling points," he says.

Neil Savage is a science and technology writer based in Lowell, MA.