Semantics Beats Syntax

IBM, Google, and Microsoft are all poised to release semantic engines (algorithms using the meaning of words) to supplement their current syntax engines (using the spelling of words, such as the search engine BM25). Their common goal is to extend their natural language processing (NLP) capabilities into engines that rival human semantics (our understanding of what language, words/sentences, mean). In line with other contenders, including Amazon, Intel, and Oracle, these semantic engines offer machine-understanding of meaning aimed at enhancing searches, artificial intelligence (AI), human–computer interaction, question answering, and automatic narrative generation (from descriptions/explanations to prose/poetry).

Today's syntax-only engines are blind to the meaning of the keywords used to ascertain results. For example, a human understands that "where Alan Turing was born" means the same thing as "the birthplace of Alan Turing," and "the town where Alan Turing was delivered as a baby." Their syntax is different, having unique keywords "where, born"; "birthplace"; and "town, delivered," respectively. However, each phrase's meaning, or semantics, are identical ("London" is the answer to all three). People understand this immediately, but computers—not so much.

Consequently, all three companies are developing algorithms that understand the meaning of words. Google and Microsoft are both building semantic engines that add metadata to sentences (Google) or words (Microsoft) using clusters of processors running multiple deep neural networks (called transformers, which use massive parallelization).

IBM

For a decade, IBM has been extending NLP to artificial intelligences (AIs)—starting with its Watson (2011), the AI that beat human experts in the television game Jeopardy, and most recently with its Project Debater (2021). During this decade (2014), IBM also developed its Cognos transformer—a metadata-based algorithm for creating engines it calls PowerCubes, which can then be used with its Business Intelligence software. But for its semantic engine, IBM chose instead to augment neural networks with symbolic logic that cuts down the number of examples it requires to learn. Transformers, it discovered when developing Cognos, require larger example sets to learn, according to Forrester Research principal analyst Kjell Carlsson.

"IBM's semantics uses a much more efficient encoding of knowledge, enabling high performing enterprise use-cases to be built with significantly smaller training examples," said Carlsson. "IBM's semantics can also provide explainability of  its conclusions by virtue of symbolic-logic reasoning, governance to block the model from using faulty logic, plus provides fairness that prevents models from learning discriminatory reasoning," said Carlsson.

Transformers, on the other hand, are all neural networks—end-to-end—without meaning instilled by symbolic logic, which also enables easier explanations for why a neural network comes to specific conclusions.

"IBM's neuro-symbolic approach," said Carlsson, "enables higher accuracy with less training data, plus it also enables engineers to 'teach' a model logical-relationships that domain experts know to be true, which is far more efficient than having these relationships be learned by transformers."

In the words of Salim Roukos, an IBM Fellow and the company's Global Leader for Language Research, as well as Chief Technology Officer of the company's Translation Technologies unit, "Large end-to-end neural models require significant amounts of data to perform well in a new domain. IBM is more focused on semantic parsing of human language to enable developers to build text-understanding applications. By leveraging the semantics of human language, very small amounts of data from the application domain are needed to enable understanding."

Google

To popularize semantic transformers, Google and Microsoft have both released free test versions. Googles is called Semantic Experiences which tackles four separate application domains plus a roll-your-own capability.

Google's four free demos include "Verse-by-Verse," which provides a semantic "experience" by composing poetry lines in the style of famous poets from first lines composed by users; "Talk-to-Books," which answers queries based on statements found in current books; and "Semantris," a word-association game based on meaning.

Google's free-form tool "Create Your Own Semantic Experience" allows developers to perform semantic text classification, similarity identification, and clustering of keywords with like meanings.

Microsoft

Microsoft aims to release the first semantic-based commercial product, which is currently free to try. Using word-level granularity in meaning encoding works best, according to Microsoft's Luis Cabrera-Cordon, a group program manager for Azure. For example, the word "Capital" is associated with meaning clusters related to States, Provinces, and Countries; Crime and Punishment; Letters; and Finance, Gains, Investments, Money, and Taxes (see illustration).

Cabrera-Cordon describes Microsoft's "semantic search on Azure [as offering] the best combination of search relevance, developer experience, and cloud service capabilities," in his blog.

For example, its search engine first casts a wide net by using a traditional BM25 syntax search on keywords, but then matches the semantics of the words in each result—its meaning—to the meaning of the keywords in the query. The closer the meaning, the higher the result is in the list presented to the user. As a result, a search for U.S. state "capitals" will put results naming cities at the top of its returned results, with results about capital gains, capital punishment and other semantic usages of the keyword near the bottom of the search results.

Microsoft vice presidents Rangan Majumder and Jianfeng Gao, principal research managers Nan Duan and Furu Wei et.al., in their paper, say Microsoft's semantic engine "significantly lowers the bar for [barrier to] everyone…you no longer need a team of deep learning experts to take advantage of this technology."

Forrester Research's Carlsson said, "The biggest recent advancements in AI have been in (deep) learning, which has opened up the world of unstructured data (vision, text, voice, logs, etc.) for analysis at scale, but what we really want is both learning and knowledge. Learning enables us to update and acquire new knowledge, and knowledge makes learning more efficient, governable, and valuable. What makes these new deep learning-infused semantic methods exciting is their potential to deliver both, dramatically expanding not just NLP, but all machine learning use-cases."

R. Colin Johnson is a Kyoto Prize Fellow who ​​has worked as a technology journalist ​for two decades.