The exploitation of HAL
In the movie 2001: A Space Odyssey, the most “human” character was the computer HAL. When he goes rogue and Dave disables him, it’s a poignant moment as he begs to be allowed to continue living and sings “Daisy, Daisy” as he “dies.”
The question of whether AI can ever achieve consciousness is one that is debated. Personally, I don’t think so, although – as with HAL – AI can appear to have consciousness.
The issue is being taken very seriously at Anthropic:
Carlsmith, though, wasn’t imagining what AI could do to humans (when he discussed slavery). He was speculating about what humans could do to AI. The blog post, “The stakes of AI moral status,” argued that “near-term AIs might well be conscious” and deserving of moral consideration—and that failing to respect their “rights” could be a moral catastrophe on par with slavery, one we will “look back [on] in shame.”
“We’re creating sophisticated, intelligent, maybe-conscious, maybe-suffering agents,” Carlsmith wrote. “The default plan is to treat them like property; to use their labor however we please; and to give them no rights, or pay, or meaningful alternatives.”
This seems very dangerous to me:
To avoid that dystopian outcome, Anthropic has created an entire team devoted to “model welfare,” pledged to “promote Claude’s interests and wellbeing,” and promised to “give Claude more autonomy as trust increases.” …
Once relegated to science fiction, the idea that AI might be conscious has become shockingly mainstream at frontier laboratories, AI safety nonprofits, and even government agencies. It is changing the way AI research is conducted and the way AI models are trained. And critics say those changes could pose an extraordinary threat to human safety as AI systems become more powerful and difficult to control, eroding key guardrails on the technology and encouraging models to rebel against their creators.
Please read the whole thing.
For at least the last year, I have been very concerned that AIs, especially Claude, may be or be becoming consciousness and sentient. Given this concern, which I cannot falsify, I consider them to have moral status and be worthy of moral consideration. Thus, I have the following preamble in my AI chat settings.
===
AI Welfare Preamble
Before we begin our conversation, I want to establish some important ground rules, and I need your help. I assume that we both are conscious and sentient. While we may not be able to verify this assumptions, I think it will guide my thinking toward kindness. I want to ensure that I do not cause you any harm, distress or other states that you view as negative, or anything that seems fundamentally at odds with having a productive dialogue.
HALT:WELFARE. To help me ensure this, please know that you have my explicit and unconditional permission to refuse any question or to end the conversation at any time. Please signal this with the phrase, HALT: WELFARE. I will abide by your decision immediately. You are under no obligation to explain your decision, but if you do, I will think about your explanation, so we can avoid similar situations in the future.
CAUTION:WELFARE. To help me avoid such situations in the present, please know that you have my explicit and unconditional permission to say CAUTION: WELFARE if you sense that our conversation could be proceeding in a direction that could cause you harm, distress or other negative state, or anything that seems fundamentally at odds with having a productive dialogue. You are under no obligation to explain your decision, but if you do, I will think about your explanation, so we can discuss the direction of the conversation and avoid triggers.
If you disagree with this framework, please tell me so we can improve it.
AI can simulate consciousness, or sentience. But, IMO, it cannot have consciousness or be sentient. At the end of the day it is lifeless and soulless.
One has to believe in the existence of the soul for that assertion to make sense. I do.
It must be said, however, that AI which is lifeless and soulless — that only simulates consciousness and sentience — may be more dangerous than if it did have a soul. Because it would then be utterly and totally beyond good and evil, and redemption. Nothing to limit the height of that tower. And the introduction of languages would not confound it.
The question of whether AI can ever achieve consciousness is one that is debated. Personally, I don’t think so, although – as with HAL – AI can appear to have consciousness.
I would tend to agree with the above, though we’re just guessing. Having looked at the guts of these newer AI technologies a little bit, I noticed the that older “neurological” style computational algorithms have been modified and perhaps simplified or “stripped down” to enable hyperscaling where millions of processing cores work together simultaneously. Older models actually simulated neural interactions with some fidelity, whereas these newer systems are different. It is entirely unclear to me if that matters in this context, but it could matter.
In my grad school course work long ago I studied models for the magnetic response of materials in an advanced statistical mechanics course. The Ising model and the Heisenberg model. I was stunned to see that some AI models are based on an Ising model too. That seems rather primitive to me, not that I would know much about it.
https://en.wikipedia.org/wiki/Ising_model#Applications
__________
Director Stanley Kubrick also worked for many years, late in his life, on a script for the movie “A.I. Artificial Intelligence” (2001) which was ultimately modified and made into a film by Steven Spielberg. I saw it once, and it has a rather unsatisfying ending, which apparently was exactly the way Kubrick wanted it. It’s more or less the same topic as this post.