The field of artificial intelligence (AI) is rife with misnomers and machine learning (ML) is a big one. ML is a vibrant and successful subfield, but the bulk of it is simply "function approximation based on a sample." For example, the learning portion of AlphaGo—which defeated the human world champion in the game of GO—is in essence a method for approximating a non-linear function from board position to move choice, based on tens of millions of board positions labeled by the appropriate move in that position.a As pointed out in my Wired article,4 function approximation is only a small component of a capability that would rival human learning, and might be rightfully called machine learning.
Tom Mitchell and his collaborators have been investigating how to broaden the ML field for over 20 years under headings such as multitask learning,2 life-long learning,7 and more. The following paper, "Never-ending Learning," is the latest and one of the most compelling incarnations of this research agenda. The paper describes the NELL system, which aims to learn to identify instances of concepts (for example, city or sports team) in Web text. It takes as input more than 500M sentences drawn from Web pages, an initial hierarchy of interrelated concepts, and small number of examples of each concept. Based on this information, and the relationships between the concepts, it is able to learn to identify millions of concept instances with high accuracy. Over time, NELL has also begun to identify relationships between concept classes, and extend its input concept set.
The following paper describes the NELL system, which aims to learn to identify instances of concepts in Web text. The NELL project is important and unique for a number of additional reasons:
As is often the case with outstanding research, the work raises many open questions including:
The paper articulates both the key abstractions underlying NELL and its limitations, which suggest avenues for future work in its concluding discussion section.
In a world that has become obsessed with the latest deep neural network mechanism, and its performance on one benchmark or another, NELL is an important reminder of the power another style of research: exploratory research that seeks create new paradigms and substantially broaden the capabilities and the sophistication of machine learning systems.
2. Caruana, R. Multitask learning: A knowledge-based of source of inductive bias. In Proceedings of the 10th International Conference on Machine Learning. (San Mateo, CA, USA, 1993). Morgan Kaufmann, 41–48.
3. Craven, M. DePasquo, D., Freitag, D., McCallum, A., Mitchell, T.M., Nigam, K. and Slattery, S. Learning to extract symbolic knowledge from the World Wide Web. In Proceedings of the AAAI/IAAI, 1998.
4. Etzioni, O. Deep learning isn't a dangerous magic genie. It's just math. Wired (June 15, 2016); https://www.wired.com/2016/06/deep-learning-isnt-dangerous-magic-genie-just-math/.
5. Etzioni, O., Cafarella, M.J., Downey, D. Popescu, A-M, Shaked, T. Soderland, S., Weld, D.S. and Yates, A. Unsupervised named-entity extraction from the Web: An experimental study (2005); http://bit.ly/2F5MZlf
6. Gibley, E. Google AI algorithm masters ancient game of Go. Nature (Jan. 27, 2016); http://www.nature.com/news/google-ai-algorithm-masters-ancient-game-of-go-1.19234.
a. Of course, this is an oversimplification but it suffices for our purposes here. See AlphaGo in Nature6 for an in-depth presentation.
To view the accompanying paper, visit doi.acm.org/10.1145/3191513
The Digital Library is published by the Association for Computing Machinery. Copyright © 2018 ACM, Inc.
No entries found