The case against AI doom
I’d rather the world not be destroyed by AI, so I’m heartened by this. An excerpt:
Here are the main counterarguments:
* Intelligence does not imply agency: A system can be extraordinarily capable at prediction, theorem proving, engineering, programming, etc. without having persistent goals, self-preservation drives, or an independent desire to act.
That has long puzzled me about the assertions of AI-doomers, as well.
More:
* Agency does not imply a single, stable utility function: Much alignment theory reasons about agents as expected-utility maximizers with coherent preferences. Actual AIs systems need not resemble that abstraction, and currently do not. …
* Capability and motivation are being conflated: Being able to formulate a plan for escaping a sandbox doesn’t entail wanting to escape it. …
* Recursive self-improvement doesn’t entail an intelligence explosion …
Intelligence may have sharply diminishing returns: There may be no meaningful scalar quantity corresponding to arbitrarily large “general intelligence.” Even if there is, going from IQ-equivalent 150 to 1,500 need not produce the sort of qualitative advantage that separates humans from chimpanzees. …
* Superintelligence isn’t omnipotence: Intelligence cannot repeal physics or eliminate uncertainty. A brilliant AI still needs processors, electricity, network access, money, factories, robots and people willing or tricked into doing things. …
* Humans retain numerous intervention points: The doom narrative sometimes jumps from “AI behaves dangerously” to “humanity is helpless.”
There’s a lot more at the link. Much although not all of it seems to have to do with the idea of agency and the tendency of people to anthropomorphize AI. Many of the comments there are worthwhile, too.
However, the following seems ironic, tacked on at the end of the lengthy tweet:
ChatGPT 6 Astra assisted with the research for this post.
Blame China for the gloom and doom over AI. That, and impressionable 12 yr olds.
The author is Eric S. Raymond, who writes as ESR, a well-known hacker and shrewd commentator on the computing scene, as well as society in general.
I find it encouraging that he wrote this piece. It may even be the right call for the short-term. However, longer-term I just don’t think anyone can say.
AI is a moving target.
Ten years ago there was a computer specialty called “machine learning” which had shown intriguing results in image recognition, speech recognition, and game playing.
But no one saw machine learning becoming a general-purpose AI that could converse and assist productively in four years like ChatGPT-3 in 2020.
We are in uncharted territory. No one knows what future breakthroughs may or may not be coming.