The Drunken Plagiarists

Dear KV,

After more than a year of hearing people talk about artificial intelligence (AI) and co-pilots, I finally tried one on a small project. I even paid for the privilege of doing so, figuring that the paid version would be superior to the free one. But what I have found confuses me, and I am wondering if you too have tried any of these tools. From your previous columns, it seems you might not be focused on the latest tools in our industry. So, maybe you have just continued to use vim and Makefiles. Have you tried these things, and do you have any words of wisdom for the rest of us who are looking at them now?

Co-Piloted

Dear Co-Piloted,

It may shock KV’s readers to learn I am a bit of a tools fanatic, and in fact whenever some new tool comes out that supposedly will help me to create or understand software better, I have been willing to try it. This applies not just to tools but also techniques. I am even a certified Scrum Master, but that is a story for another time and/or column.

I have tried many editors, several IDEs, various debuggers, and all manner of new and interesting tools in my career, and continue to do so. Like you, I had held off on trying the tools based on large language models (LLMs) and even now continue to feel discomfited whenever I am forced, in conversation, to refer to these as AI. Being able to spit out passable marketing doggerel has about as much in common with intelligence as does an American presidential election. In fact, those two are clearly, deeply related.

Before trying to use these tools, you need to understand what they do, at least on the surface, since even their creators freely admit they do not understand how they work deep down in the bowels of all the statistics and text that have been scraped from the current Internet. The trick of an LLM is to use a little randomness and a lot of text to guess the next word in a sentence. Seems kind of trivial, really, and certainly not a measure of intelligence that anyone who understands the term might use. But it is a clever trick and does have some applications.

If you are typing a suicide note, for example (and the corpus of text contains thousands of these), it is quite possible the code will be able to guess which word you might use next since thousands of people who came—and went—before you typed it as well.

Code is a an even more constrained environment than prose, in a way, because code must be run through a process that has a strict syntax—one that is far stricter than any human language. It is thought this narrowness facilitates the process of guiding the creation of code, with templating being cited as an early use case for these technologies. And who would not want help with such drudgery? Many pieces of code that are written, especially for the visual web, are just copied-and-pasted versions of other pages, and the same might be said for other areas of coding.

While help with proper code syntax is a boon to productivity (consider integrated development environments that highlight syntactical errors before you find them via a compilation), it is a far cry from semantic knowledge of a piece of code. Note that it is semantic knowledge that allows you to create correct programs, where correctness means the code actually does what the developer originally intended. KV can show many examples of programs that are syntactically—but not semantically—correct. In fact, this is the root of nearly every security problem in deployed software. Semantics remains far beyond the abilities of the current AI fad, as is evidenced by the number of developers who are now turning down these technologies for their own work.

Guessing the next word used by a cohort of morons, which is what co-pilots actually do, leads to incredibly incisive text such as:

server.mtx.Lock() // Lock the cache

Yes, thank you, that is the mutex Lock method. But why do we lock the cache? And what do we do about it later? This is akin to a comment such as:

i++ // Increase i by 1

The only reason anyone is impressed by this is that it is written in a form that is more palatable to those who wish to anthropomorphize their machines, something Dijkstra warned about in the 1960s.

Another classic from our new Robot Master:

// Get retrieves the value for a given key if it exists and is not expired
//Parameters:
//- ctx: context for the request.
// - request: contains the key to retrieve.

Wow! Really? If a cursory glance at the code was not enough to tell me this, I should not be here at all.

Finally, my favorite feature of co-pilot programs is the abject plagiarism. We already know that the text and code being typed out by these things comes from scanning billions of lines of text and source code available in GitHub, but they can even be helpful in unintended ways. A colleague who was taking a night class in distributed systems showed me what happened when his professors suggested the students “use the new tools” in order to become more modern developers. As he accepted more and more of the co-pilot’s suggestions, he noticed a pattern: It was as if someone was typing in another file from somewhere else. The coding style itself was one of the clues, but eventually the co-pilot gave itself away completely by saying, “You know there is a file just like this over in this other repo?” In a way, this makes sense. But as part of a homework exercise, it is just hilarious.

The more I have used these tools in my projects, the more I have realized that co-pilots are nothing more than drunken plagiarists, sitting behind you and your code whispering semantic nothings in your ear. They are not a boon to your work, they are a rubber crutch—one that will cruelly let you down when you need it most. Now, we all just need to get real work done while we wait for this latest hype cycle to die a justified and fiery death.

KV

Related articles

Aversion to Versions
Kode Vicious
https://queue.acm.org/detail.cfm?id=3458680

Is There Another System?
Kode Vicious
https://queue.acm.org/detail.cfm?id=3639446

Taking Flight with Copilot
Christian Bird, Denae Ford, Thomas Zimmermann, Nicole Forsgren, Eirini Kalliamvakou, Travis Lowdermilk, and Idan Gazit
https://queue.acm.org/detail.cfm?id=3582083