ChatGPT Is Nothing Like a Human, Says Linguist Emily Bender

"You Are Not a Parrot and a chatbot is not a human" - an interview/profile of Emily M. Bender, the "computational linguist at the University of Washington ... [who] co-wrote the octopus paper... to illustrate what ... LLMs ... can and cannot do"

The paper: "Climbing Towards NLU: On Meaning, Form, and Understanding in the Age of Data'... NLU = natural-language understanding... LLMs... are built on statistics... looking for patterns in huge troves of text ... to guess the next word ... great at mimicry and bad at facts...beguiling, amoral, and the Platonic ideal of the bullshitter... worse than liars. They don’t care whether something is true or false. They care only about rhetorical power"

Bender: "do not conflate word form and meaning. Mind your own credulity... [we've made] machines that can mindlessly generate text... we haven’t learned how to stop imagining the mind behind it.”

Why are we building them this way, deliberately "blurring the distinction between what is human and what’s a language model?" - risks unravelling society. ChatGPT is believed to have been trained on most/all "Wikipedia, pages linked from Reddit, a billion words grabbed off the internet", but not books, protected by copyright. Problem: those words were written overwhelmingly by white people, with men and wealth overrepresented.

"A stochastic parrot ... haphazardly stitching together sequences of linguistic forms … according to probabilistic information about how they combine, but without any reference to meaning. Google initially approved 'On the Dangers of Stochastic Parrots', then "told the Google co-authors to take their names off it. Several did, but ... Timnit Gebru ... Margaret Mitchell" didn't and were fired. "stochastic parrot entered the tech lexicon", but tech execs and programmers loved it, describing themselves as stochastic parrots - do they "want to believe so badly that these language models are actually intelligent that they’re willing to take themselves as a point of reference and devalue that to match what the language model can do"?

"Creating technology that mimics humans requires that we get very clear on who we are... the safe use of artificial intelligence requires demystifying the human condition

• "dehumanization ... [is] failing to perceive another human as fully human" - or being that person perceived in that way.
• "the computational metaphor... that the human brain is a computer, and a computer is a human brain... affords the human mind less complexity than is owed, and the computer more wisdom than is due.

Dennett: "Counterfeit money has been seen as vandalism against society ever since money has existed... Counterfeit people is at least as serious... [they have] less at stake ... makes them amoral... They are sort of immortal.”

This technology creator's are "on the verge... of creating very serious weapons ... against ... society. They should take that as seriously as the molecular biologists have taken the prospect of biological warfare or the atomic physicists have taken nuclear war... We want smart machines, not artificial colleagues.”

But what happens when we habituate "people to treat things that seem like people as if they’re not”? Won't we all start treating real humans worse? "a world in which grown men... posit thought experiments about raping talking sex dolls, thinking that maybe you are one too."

Some "legal scholars suggesting ... just adopt Roman slave law and apply it to robots and AI.”

Conclusion: "LLMs are tools made by specific people ... stand to accumulate huge amounts of money and power, people enamored with the idea of the singularity. The project threatens to blow up what is human in a species sense" - we might "lose a firm boundary around the idea that humans... are equally worthy", . This borders on fascism: "The AI dream is governed by the perfectibility thesis... a fascist form of the human.