Looking for Insights from Artificial Intelligence

'AI disentangles protein folding' was one of the headlines in the selection of breakthroughs achieved during the year 2020 by the magazine Science last December. An artificial intelligence (AI) program called AlphaFold, developed by Alphabet subsidiary DeepMind, had succeeded in making a great leap in one of biology's grand challenges: how to predict the three-dimensional shape of a protein when its amino acid sequence is known.

This breakthrough is likely just the beginning of how AI is going to change scientific discovery.

In that light, it is no coincidence that this year's 8th Heidelberg Laureate Forum (HLF) featured a panel discussion on the question of whether future discoveries can be made by AI. The Heidelberg Laureate Forum is an annual conference, this year organized online because of the pandemic, where 200 young researchers in mathematics and computer science spend a week interacting with laureates of the Abel Prize, the ACM A.M. Turing Award, the ACM Prize in Computing, the Fields Medal, and the Rolf Nevanlinna Prize.

In September, on the last day of this year's HLF, moderator and science journalist Volker Stollorz discussed the topic of using AI for scientific discovery with three scientists: Harry Collins, a professor of Social Science at Cardiff University (U.K.); Jeffrey A. Dean, senior fellow at Google Research (U.S.) and recipient, with Sanjay Ghemawat, of the ACM Prize in Computing for 2012, and Dafna Shahaf, associate professor of data science in the Department of Computer Science and Engineering at Hebrew University of Jerusalem (Israel).

Asked to predict the role AI will play in scientific discovery, Dean said there are two types of scientific problems that are well suited for AI-driven discoveries. One variety is the type of problem for which we have traditionally used computational simulators, like molecular simulations in chemistry. Explained Dean, "What has happened over the last five to 10 years is that we can use these computationally very expensive simulations as training systems for machine learning approximations of the entire simulation problem. These approximations are computationally much more efficient, sometimes 300,000 times as fast as the original simulator. You then have a tool that changes the way you do science."

The second type of problem, he said, includes those in which you can have a system generating a set of candidate solutions for a scientific problem. "You can then create an entirely computational-driven loop in which you generate solutions, evaluate how good they are, and use reinforcement learning to give feedback to the system. In this way, it generates better and better solutions. This approach is successful in a wide variety of scientific domains. A machine can thus run in a weekend 100,000 experiments, whereas a human maybe can do a few hundred."

Cardiff's Collins, who has spent 45 years as a social scientist working with researchers on gravitational wave detection, warned that the social aspect of the scientific process should not be underestimated. "During the decades I spent in the gravitational wave community, people resigned, people got sacked, people got furious at others, people didn't believe other people, people did believe other people, and so on.

"This process eventually led to the detection of gravitational waves. It worked through a series of small group meetings where people learned to trust each other. If you want to make a discovery of that craziness, people have to learn to trust each other in order to invent completely new things. Can you emulate this with computers? Certainly not now. The best deep learning machines we have are essentially individualistic."

While Collins looks at scientific discoveries with a sociological eye, Shahaf takes a computational approach. She is interested in how reasoning by analogy can help in making discoveries. "Analogy has been a force of innovation already since the ancient Greeks," Shahaf said. "Take the example of NASA hiring experts in origami-folding to design a new generation of folding spacecrafts. Reasoning by analogy has been one of the holy grails in AI for a long time."

In one of her projects, Shahaf said, she wrote an algorithm that takes the natural language text of an invention and tries to extract a representation of the purpose of the device and its mechanism. "You can then ask the system the question: 'find me another product that solves a similar problem, but in a completely different domain'. If you let people use our system, they get a boost in their own creativity. We also built a prototype of a search engine type of machine where you type in your problem, and the machine gives you some inspiration from far-away domains that are similar on an abstract level."

Shahaf said she is working to find analogies between a large set of scientific papers and a system that suggests which scientists elsewhere in the world might be interesting collaborators on a specific problem.

All three panelists agreed that bringing together different varieties of expertise is important for scientific discovery. Collins stressed that determining what is relevant and trustworthy information and what is not is very difficult for computers; in current scientific practice, he said, that happens in groups with social interaction.

Dean said that determination is also difficult for humans, adding that computational tools that can bring together different types of expertise already exist. "There is a neural network technique called 'mixture of experts', where multiple experts are used to divide up a problem into smaller pieces and the system learns to route new examples and questions to the appropriate expert. If you scale this vision up, you have different experts for different things."

The next step in AI contributing to science, Dean said, will come from multimodal AI models that integrate all human languages, human speech, audio, images, and videos. "This will give machines a much broader understanding of the world, in the same way that humans have different senses. Using AI in science, in a partnership between humans and machines, we'll be able to achieve more."

Asked whether science in the future will need fewer and fewer researchers once AI learns to contribute more to science, Shahaf noted that while scientists perform a broad spectrum of research-related tasks, "Some things will be taken over by computers, and that might even be a good thing.

"For me, the most interesting question is how we can get computers to develop the same type of intuition that human researchers have. What are the scientific questions that are really worth pursuing?"

Bennie Mols is a science and technology writer based in Amsterdam, the Netherlands.