Sign In

Communications of the ACM

ACM News

Your Favorite AI Language Tool is Toxic

View as: Print Mobile App Share: Send by email Share on reddit Share on StumbleUpon Share on Hacker News Share on Tweeter Share on Facebook
A representation of natural language processing.

Pretrained language models are prone to generating racist, sexist, or otherwise toxic language, which hinders their safe deployment, according to a new research paper from the Allen Institute for AI.


The business world has been captivated by A.I. that can craft sentences that seem, at least superficially, like they've been written by humans. 

But these so-called pretrained language models have a major problem: They "are prone to generating racist, sexist, or otherwise toxic language, which hinders their safe deployment," according to a new research paper by The Allen Institute for AI (AI2), a non-profit research lab founded by the late Microsoft co-founder Paul Allen.

Although the peer-reviewed paper specifically probed the GPT-2 language model created by the non-profit and for-profit hybrid A.I. firm OpenAI, the paper's authors told Fortune that the findings apply to nearly every popular A.I. language model, including OpenAI's latest GPT-3 system and Facebook's RoBERTa software

The findings, which have been accepted for the upcoming Empirical Methods in Natural Language Processing A.I. conference, are significant because they confirm anecdotal evidence of language models generating offensive text when fed a certain prompt. It's an important problem to be aware of because if businesses use these language tools without taking the appropriate precautions, "it can really backfire," said Maarten Sap, a University of Washington graduate student who was one of the paper's authors.


From Fortune
View Full Article



No entries found

Sign In for Full Access
» Forgot Password? » Create an ACM Web Account
ACM Resources