The True Threat of Artificial Intelligence

One of my favourite writers in this area on AGI risks. "It is the rise of artificial general intelligence, or A.G.I., that worries the experts... Yet a nascent A.G.I. lobby of academics, investors and entrepreneurs counter that, once made safe, A.G.I. would be a boon to civilization... beholden to an ideology that views this new technology as inevitable ... no better alternatives for fixing humanity ... But this ideology — call it A.G.I.-ism — is mistaken. The real risks of A.G.I. are political... A.G.I.-ism distracts from finding better ways to augment intelligence".

It's another version of the Thatcherites' "no alternative to the market" neoliberalism, the main biases of which AGI reinforces:

• market bias: that private actors outperform public ones
• adaptation bias: adapting to reality beats transforming it
• efficiency bias: efficiency trumps social concerns.

"we already know what to expect of an A.G.I. rollout... two phases:

• "charm offensive of heavily subsidized services...
• overdependent users and agencies shouldering the costs of making them profitable....

"solutionism ... reframes social problems in light of for-profit technological solutions... concerns that belong in the public domain are reimagined as entrepreneurial opportunities... A.I. is already pitched as an all-purpose hammer for many real and imaginary nails"

Using the Uber story as an example: "if you dislike your town outsourcing public transportation to a fragile start-up, would you want it farming out welfare services, waste management and public safety to the possibly even more volatile A.G.I. firms? "

Using AGI to "fix" our institutions - the university, the hospital, the newspaper - will "be like handing them over to ruthless consultants [brandishing] data-bolstered “solutions” for maximizing efficiency". But the data focus ensures such solutions "fail to grasp the messy interplay of values, missions and traditions at the heart of institutions".

In fact this "refusal to grasp reality at a deeper level, beyond the data’s surface" is inherent to the technology: "Good luck discerning the meaning of the Hippocratic oath by observing hospitals that have been turned into profit centers."