Can Neuroevolution Change Machine Learning?

Many of the concepts associated with machine learning have been around for decades. However, over the last few years, enormous advances in computing power have allowed researchers to explore algorithms and approaches that have languished. One idea that is suddenly grabbing attention in the AI world: neuroevolution. The approach mimics natural evolution by winnowing neural pathways within artificial neural nets. Through mutations, it identifies the pathways that are the most efficient to address a particular task.

In the coming years, neuroevolution could impact fields as diverse as robotics, medicine, and logistics by allowing systems to adjust and adapt more dynamically—and more intelligently. Although the origins of the concept date back to the 1980s, today's computational power is allowing researchers to breathe new life into old algorithms.

"Neuroevolution combines evolutionary algorithms with artificial neural networks. The result is an ability to train systems in a similar way to the evolution of brains on earth," explains Kenneth Stanley, a senior research scientist at Uber AI Labs and a professor on leave from the University of Central Florida.

Natural (Code) Selection

Neuroevolution is inspired by nature. Conventional deep learning uses an approach called stochastic gradient descent (SGD); it improves an algorithm over time by gradually reducing errors through ongoing training. Although SGD is highly effective for addressing many computing tasks—including difficult activities such as speech and image recognition—it struggles more with the harder challenge of "reinforcement learning," which means learning how to act with only infrequent feedback. For example, a robot with a damaged or disabled leg has to learn a different way to walk without anyone telling it exactly how it should adapt to that unique situation.

"Typically, neural networks are used in domains where you have an existing dataset," explains Risto Miikkulainen, a professor in the computer science department at the University of Texas at Austin and vice president of research at Sentient Technologies, an artificial intelligence software firm. "They are useful when you have existing data points that tell you how to map inputs to outputs. "This includes things such as understanding the behavior of the stock market or the underlying dynamics of weather conditions, which both display known patterns over time. However, "If you do not know what the right outputs are for given inputs, you cannot use this approach," Miikkulainen adds.

Neuroevolution aims to bypass this barrier. Several researchers have dabbled with the methodology over the last few years, including members of the non-profit research company OpenAI and groups at DeepMind and Google. Stanley and Jeff Clune, a senior research scientist at Uber AI Labs and associate professor on leave from the University of Wyoming, have demonstrated that neuroevolution is also a powerful algorithm for creating deep neural networks that can solve hard reinforcement learning problems. A genetic algorithm (GA) they and an Uber AI lab team developed trained deep convolutional networks with upwards of 4 million parameters.

Machine Mutations

The research has already demonstrated the practical viability of neuroevolution. Stanley and Clune, who together have co-authored five academic papers on the topic at Uber, tapped the genetic algorithm to train a neural net to play Atari video games from pixels. The system outperformed conventional machine learning methods in 6 of 13 games and it avoided some dead ends that SGD could not. What's more, in three games, the system in outperformed humans.

Stanley says a key factor is the ability of the neural net to spontaneously mutate. He, Clune, and Joel Lehman, another research scientist at Uber, found a way to introduce "safe" mutations that helped the neural net to grow dramatically and produce better results. This technique enabled them to evolve deep neural networks (DNNs) with over 100 layers, which is far beyond what was previously shown possible through neuroevolution.

At Uber, the practical benefits might include developing improved pricing models or more efficient scheduling or routing systems. But that is only the part of the allure of neuroevolution, Clune says. The GA, especially when combined with other machine learning algorithms, could lead to all sorts of improvements in robotics, drones, autonomous vehicles, smart cities, drug design and cyber-security. These systems could adapt dynamically and intelligently to changes, whether it's structural damage to a drone or a sudden spike in traffic in a neighborhood.

"The goal is to create a framework where it's possible to explore and test possibilities quickly and select the best approach," Miikkulainen explains.

In fact, Miikkulainen believes the future of neuroevolution is bright. It adds a potentially powerful tool for tackling complex computational problems. Others, including the OpenAI group, are pursuing research in the field and eying the possibilities.

Concludes Clune: "Neuroevolution has always been an intriguing concept. Modern computation is allowing researchers to begin to truly harness the idea in much the same way that Leonardo Da Vinci envisioned a helicopter long before it was possible to actually build and fly one."

Samuel Greengard is an author and journalist based in West Linn, OR, USA.