Can AI be taught morality?
I wonder.
There’s this supposed incident (hat tip: commenter “Snow on Pine”); the original story as reported went this way:
Hamilton described a scenario at a summit hosted by the United Kingdom-based Royal Aeronautical Society in which Air Force researchers trained a weaponized drone using AI to identify and attack enemy air defenses after receiving final mission approval from a human operator. But when an operator told the drone to abort a mission in a simulated event, the AI instead turned on its operator and drove the vehicle to kill the operator, underscoring the dangers of the U.S. military’s push to incorporate AI into autonomous weapons systems, he added.
But now the Air Force is saying that it never happened. It was merely a hypothetical thought experiment.
On the other hand, ChatGPT seems to be a BS artist. Beware relying on it. Some New York attorneys did in researching a case, and the helpful bot just made s*** up.
This discovery came to light when the defense counsel, during their due diligence, could not locate the cited cases in any standard legal databases…[T]he supposedly precedent cases were utterly fabricated.
And this other example of fabrication by AI seems to be BS-ing just for the fun of it:
Tony Venhuizen, a smart guy from South Dakota, operates a web site where he writes about the history of the governors of that state. He asked ChatGPT, “Please write a blog post discussing South Dakota’s oldest and youngest governors.” Chat GPT responded with a competent description of South Dakota’s oldest governor, Nils Boe. It then went on to write about the state’s youngest governor, Crawford H. “Chet” Taylor.
Taylor didn’t exist, although ChatGPT even came up with a fake portrait of the nonexistent governor. The strangest thing about the incident is that of course there really was a youngest governor of the state, who was ignored by the AI.
Can these sorts of things be ironed out? Or is there something inherent in AI that makes such problems – or something worse – inevitable? Those sorts of speculations have been staples of science fiction for just about forever. For example, anyone who’s seen the movie “2001: A Space Odyssey” can probably remember its most colorful and interesting character, the computer Hal, who has a mind of his own and pleads for his life when he sees his end in sight as a result of his transgressions.
Now relatively sophisticated AI has entered into the realm of reality, and must be grappled with. I don’t think most of us have confidence that those who will be doing the grappling know enough to deal successfully with the dilemmas they will be facing.
I believe that there may be ways to better ensure that a generative AI is actually using known established facts rather than making stuff up.
But the real issue is that these generative AI’s are created by humans and trained with data that is generated by humans. And humans lie.. a lot. Humans lie for all sorts of reasons. A person may lie because the truth contradicts that person’s agenda or narrative. A person may lie because they’re just bored.
So I don’t know what the answer is. But we may be in for a some hard times if we don’t figure it out.
The problem is the latest AI isn’t really thinking as humans think. It is performing statistical operations with lots of feedback loops upon vast amounts of human-generated text, which mimics how a human might respond to language.
AI doesn’t know whereof it speaks or why, though it’s uncanny how good AI has gotten.
AI doesn’t know what reality is either. Since its primary job is to generate a language response to a prompt, well, mission accomplished, even if it just makes stuff up, as long as it “makes sense.”
I haven’t yet seen good explanations for the recent cases when AI goes off the rails as badly as it has. To be sure the AI boys and girls will make improvements, but there will always be the next levels of complexity which we won’t see until they happen.
Which is why we need to keep a close eye and a short leash on this technology, though I fear it’s moving too quickly for either.
See Asimov; Laws of Robotics.
Or is there something inherent in AI that makes such problems
About 40 years ago, someone asked me if AI could reach human levels, I said yes, but it would be insane — no connection to reality, it would be like it was lying in a sensory deprivation tank and hallucinating.
The LLMs produce knockoffs, that is, they produce things that look like other things, and they do that very well, but they have no concept of reality. One could argue that humans often operate in the same state, but they get feedback when they screw up, and we have developed some procedures to avoid the worst hallucinations. Those procedures could be called operational definitions of reality.
In some ways, this reminds me of the accusation that Helen Keller was a fraud because there was no way for her to experience many of the things she wrote about. Yet she did read a lot once she had access to books in the appropriate format, and hence she could write in the same manner, not unlike LLMs 🙂
This is a reminder of how much of our world and reality is just made up. Some of us could use better editors.
AI could be useful for generating Biden’s speeches, as he knows nothing of morality.
It is programming, it has no “morality” as we would understand it.
If the desired output is factual data and it is failing to provide it, the software is faulty and should be corrected.
It’s no different than a hammer with a loose grip. When you smack your thumb with it, you don’t question the morality of the tool, you repair the faulty part. Or toss it and get a better made one.
FYI, there is a good explanation of ChatGPT fabrications here. https://twitter.com/dsmerdon/status/1618816703923912704?lang=en
ChatGPT does not have a conception of truth. It is a highly sophisticated autocomplete. If you wish to ascribe mentation to it, it “knows” not what is true, but what words are likely to follow the ones already on the page.
I’m still wonder how close are we to real AI.(As opposed to machine learning.) I mean I remember them talking about being just around the corner when I was a kid in the 80s. I haven’t tried ChatGPT so I have no idea how much farther it’s from that old Eliza program.
Anyone who had read some science fiction, or who has half a brain could see that unless closely monitored, overseen, and shepherded along very, very carefully, AI–as powerful and omnipresent as it will likely become–poses a huge–even an existential threat–to the human race, our freedom, and our very existence.
Yet, despite the warnings of knowledgeable people like Elon Musk, it appears that there is an unregulated, head long rush to develop AI, and the motto of the companies involved is apparently full speed ahead and “the devil take the hindmost.”
Meanwhile, Congress–which I wouldn’t trust to run a lemonade stand, or to keep it’s hands out of the til, is–in a day late and a dollar short fashion–now starting to think about regulation of AI.
I used to make my living programing computers. In the 1970’s we did a study on artificial intelligence. We joked that we couldn’t find much artificial intelligence but we sure found a lot of genuine stupidity. Nothing has changed.
Take a look at Tucker Carlson’s thoughts on UFOs, and on the development of AI on the “Elon Zone.”
See chttps://www.youtube.com/watch?v=uW76yBxOYsY&t=2278s
AI will not know or appreciate the subtle aroma of a rose in the still of morning air. Nor will it appreciate the anonymous shy smile and small child wave offered from a parked car. AI can know of human experience, but only from digital scavenging. It will be inferred, but not known as a human knows from experience, good experiences and bad experiences.
And there is the limitation. Experience is a profound source for developing judgement – “Good judgement comes from experience and experience comes from bad judgement.” AI will struggle to accumulate bad experience in order to develop good judgement.
But how will it know good from bad experiences? How will it rate them? How will it know how to develop good judgement? How will it know when good judgment may appear to be bad judgment or just plain naiveté
More and paramount, AI can never know first hand suffering. Human beings avoid suffering if possible, but everyone suffers. Everyone will know of death and experience dying. AI cannot know the fear of death. AI cannot really die or wonder what is beyond death.
AI cannot know faith beyond reason, or know the supernatural.
Descartes quote, “I think therefore I am” suggests thinking brings identity from rational thought. And so an AI appears to have identity from expressing rational thought (via expressive language models, etc.)
But all humans have something in common from which identity is enriched, because identity exists not as a consequence of thought but as a gift to the creature made in the image of God. And this side of Eden this enrichment is both positive and negative.
Recasting the judgement aphorism, positive judgement can come from negative experience. Or more generally, “I suffer. Therefore, I think on my suffering and wonder who I am.”
On the flip side, read “They’re Made Out of Meat.” https://www.commonlit.org/en/texts/they-re-made-out-of-meat
Unfortunately, I think that at this point a lot of us have been lied to by figures in authority so many times that it’s now hard to accept and believe anything that they say.
So, did the murderous AI incident really happen?
At this point who can really tell.
Garbage in, garbage out.
AI cannot “know” anything whatsoever in the same sense that we do. It “knows” in exactly the same way that your lamp “knows” to shine when you turn it on. Trillions and trillions of on-off switches are not intrinsically different from a single one, unless you subscribe to the act of faith that consciousness is a material phenomenon that happens if you put together enough on-off switches.
The danger of AI is not that it will want to kill us but that we will entrust decisions to it that ought to be reserved to people. Possibly the biggest danger is our not realizing that we have done so, as a result of a failure to think through the implications of something we’ve told it to do. HAL ought never to have had the ability to shut down the life support systems.
AI is such a misnomer, invented by Marvin Minsky and crew at MIT in the sixties. They anthropomorphized what was just a bunch of electrical circuits to make their research sound very advanced and important. Face it, he and his followers were geniuses at PR.
You’re familiar with spell check. If you misspell something, i.e., it can’t find the word in a dictionary, the program scans the dictionary to find the closest match and presents that. If it’s really garbled the program will present you with multiple choices to choose from, some of them weird.
ChatGPT is no different, except they’ve stored enormous amounts of text and complete a phrase by looking for the “nearest phrase.” It’s slicker than that but that’s basically what it does.
If you have modest math chops you can get an intro on Kindle, “What Is ChatGPT Doing … and Why Does It Work?” by Stephen Wolfram. The answer is, It’s Just Adding One Word at a Time.
Mac: “The danger of AI is not that it will want to kill us but that we will entrust decisions to it that ought to be reserved to people. Possibly the biggest danger is our not realizing that we have done so, as a result of a failure to think through the implications of something we’ve told it to do.”
You mean like climate models? 🙂
We have to remember that AI is in a machine. The machine needs electricity to operate. Pull the plug and AI is dead. The Green New Deal with its intermittent, unreliable energy is a major threat to AI……and humans. And that threat comes from the artificial climate models that have driven a mass psychosis about a planet with a deadly fever.
Computers have allowed us to accumulate vast amounts of data and information – amounts that would fill huge libraries – in quite small spaces. But analyzing that data and information for truth, facts, and usable information requires skill and judgement – the kind that humans have, and computers don’t…yet. The guard rails that are needed are as Mac says. We must not trust AI. It could easily send us down alleys of stupidity like the climate crisis or worse. All AI output must be verified as true and useful. That means human overseers.
Morality in Morality out… what are the chances of that happening?
It is a given that AI will be obtained by evil to use it for its own ends. The question is do we have a defense?
I don’t know about morality but so far ChatGPT and its ilk seem to do a pretty good job of modeling confabulating dementias.
AI makes me remember the Star Trek episode, The Ultimate Computer.
My tin foil hat theory is that attempts to bias AI to enforce left leaning narratives makes it discount knowledgeable sources that are ‘untrustworthy’, i.e. not Woke.
yes the M5 and Dr Daestrom,
A bit off topic but there was a very good British movie a few years ago called “Eye in the sky.” which involved drones and controversy about targets. Helen Mirren and Alan Rickman were in it. It was Rickman’s last movie.
Mac…”The danger of AI is not that it will want to kill us but that we will entrust decisions to it that ought to be reserved to people.”
Here is an example of a fatal accident that was caused in part by excessive reverence for the automation, which was not AI, but a fairly simple algorithmic system. Blood on the Tracks:
https://chicagoboyz.net/archives/43911.html
Hollywood writers vs AI ?
https://decrypt.co/143226/hollywood-writers-strike-not-rest-until-ai-stopped
The story about the drone reminded me very much of a story in a set of 1950s SF stories that I reviewed recently. The specific story is titled Boomerang.
https://chicagoboyz.net/archives/68808.html
Sounds like the drone thing didn’t really happen in an actual simulation, it was more of a though experiment. In any case, my question is, why would an anti-SAM drone system of this sort have any knowledge of the location of its human controller, or of the communications tower from which its commands are transmitted? Or, for that matter, the frequency of that communications link.
Unless someone set up an experiment that way to see what would happen.
It’s Google Autocomplete. It guesses at what text a human expects to see and delivers that. It’s not trying to turn on its operators, or lying, or making things up. We react to it as though it might be doing those things because we’re not used to anything other than humans which write as well as humans* do, and so we impute humanlike motives to an algorithm with a text output.
Which is nuttier than anything ChartGPT ever generates. That the media describes it as “AI” and that we’ve all read books and seen movies about “AI” doesn’t imply that THIS AI is anything like THOSE AIs.
*I’d say ChatGPT writes better than most humans I’ve worked with do.
The story I read was in Vice:
https://www.vice.com/en/article/4a33gj/ai-controlled-drone-goes-rogue-kills-human-operator-in-usaf-simulated-test
Sounds a little jumbled.
Aggie:
The article I linked in the first paragraph of this post says that the Air Force says that no such simulation ever occurred.
Frederick:
If it’s Google autocomplete, how is it that it’s making up an entire person who never existed, complete with a painting that is imaginary? Meanwhile, there is an actual youngest governor of the state and it could have given information about that actual person who fit the assignment.
Some time ago a woman crossing a street with her bicycle was killed by an AI-driven car that hit her. There was a lot of discussion about how to fix the system of controls to handle the problem.
There was a lot more talk about how could it have happened, but the moment I heard of it I thought that one of the chief problems we face might be that it could be that the woman wasn’t killed because the car didm’t stop so much as that she saw the sign on it that said it was a computer controlled car and she simply assumed it would automatically “do the right thing” and stop for her so she kept going.
When I’m crossing in a crosswalk and a car is coming, I stop and wait to make sure it really stops before going on.
We’ve got to learn that being controlled by a computer doesn’t mean something is going to work optimally.
@ Richard Aubrey on June 2, 2023 at 4:14 pm said:
See Asimov; Laws of Robotics.
IIRC, Dr. Asimov’s stories about Robots included a fair number describing Things That Can and Do Go Wrong, and the resulting difficulties.
One of my favorites is this (don’t read the rest of the article if you haven’t read the story yet).
https://en.wikipedia.org/wiki/Liar!_(short_story)
I haven’t tested this link; use at your own discretion. I believe it is included in the collection “I, Robot.” And probably many anthologies.
https://oceanofpdf.com/authors/isaac-asimov/pdf-epub-liar-by-isaac-asimov-download/
Neo — This can possibly be solved if/when we can actually program some element of “common sense” into the AI. It’s not happening soon, for sure.
The most obvious case of this is simple:
“Computer, the cat has fleas. Please solve this problem.”
Computer: “Heat kills fleas. Throw cat into furnace”.
Now, this “solution” is such that pretty much any human being over 6 probably grasps it’s not acceptable, because, of course, heat also kills cats. But the notion that you want to get rid of the fleas “but keep the cat alive” was not spec’d as a part of the problem. A human does not need to be told that, but the computer is an idiot savant, it’s not going to make the kind of leaps that qualify as “common sense”.
@neo:If it’s Google autocomplete, how is it that it’s making up an entire person who never existed, complete with a painting that is imaginary?
It’s not “making things up”, because that assumes a mind capable of such an intention. It has sampled bezillions of examples of human writing, and it can replicate the kinds of things humans write when asked questions like that.
Meanwhile, there is an actual youngest governor of the state and it could have given information about that
It has no way to know what is true what is false. It does not know that anything humans have written is true or false. It only knows what humans have written. It does not know there is a universe out there where things that are written about may have happened or not.
It’s no more the kind of AI science fiction has been writing about, than the robots that vacuum my floor are the kind of robots Asimov wrote about. Someone put the same label on them, and the rest is media bloviation.
“. It only knows what humans have written.”
Except afaik, there’s no indication that humans have ever written about the governor that it made up. This is a consistent problem with ChatGPT. It will create nonsensical facts and cite non-existent sources as the place where the false information came from.
On another note…
One interesting thing about HAL is that even as it’s trying to murder David Bowman, and it:a becoming increasingly clear that David doesn’t trust it, it still talks as if David is it’s best friend. It does this even as David is in the memory room removing the memory modules. But HAL is fictional, so that might not really matter all that much.
Frederick:
If it is 100% untethered to reality, why have it at all?
That person it described has NEVER been written about before. He is an entirely fictional person. On the other hand, the actual youngest governor is an actual fact, easily discovered by AI. It’s not needing to sort out competing narratives about whom he might be. There is only one possibility.
David Foster: “Boomerang” is a very relevant case study.
Frederick (and Neo): “It’s no more the kind of AI science fiction has been writing about, than the robots that vacuum my floor are the kind of robots Asimov wrote about. ”
Yes. It’s important to understand that all the language attributing thought to anything a computer does, including AI, is metaphorical. Programmers have always done this, very casually, because it was convenient. In the bad old days before the handling of case in letters became routinely more robust, you might have heard a programmer say “It keeps asking you to reply because you typed ‘y’ and it’s only looking for ‘Y’ or ‘N’, so it thinks you haven’t answered.” “Thinks” is totally metaphorical.
To say that the machine “made up” a fictional governor is metaphorical, too. Why the AI did that I have no clue, but it’s a bug. Or I guess possibly a feature, depending on the intentions of the developers, in which case it’s rather funny. Either way a good instance of why we have to be really careful about entrusting serious decisions to computers.
More on Helen Keller and the Frost King incident. I liked this bit:
Note that while automated systems will follow procedures and incentives, sometimes with bad results, so will human systems. Years ago I was talking to manager of a very large factory and asked him about piecework pay plans. He said he hated them: people would take risks to meet the bonus targets, 99.9% of the time it would work out OK but occasionally someone would lose a hand.
See also this example from Sweden of automaton-like behavior from a human organization:
https://chicagoboyz.net/archives/36610.html
EVERONE. There is a key difference here being ignored.
At one level we have machine learning. Most example above are about the memory miracle that a computer possesses.
By contrast, the aspiration of generative AI is machine learning that achieves the simulacrum of human learning. And then goes above and beyond it.
If we want to be clear with other people, we ought to observe this difference in key terms.
That person it described has NEVER been written about before. He is an entirely fictional person.
What is surprising about that? Sometimes when I am reading a story and go to bed before finishing, a little narrator voice comes in and narrates more of the story. It can be real enough that when I wake up it can take a while to distinguish exactly where the actual story ended and the narrator took up the job. Making stuff up is normal. All the AI is doing is talking like a person. Like in movies and plays, there needn’t be any connection to reality, it just needs to sound real. It can talk like a pirate every day of the year.
One use of the current technology, apart from cheating on writing assignments, might be writing code. A lot of code follows templates, and stealing code snippets from existing code is common practice. Once the proper prompts are learned, LLMs should be able to do an excellent job at that.
the larger question is do the programmers understand morality or truth, or any other transcendent thing
Chuck..”One use of the current technology, apart from cheating on writing assignments, might be writing code.”
There have been a lot of people using it for coding, some claim 2X improvement or more in their productivity. I tried it out on a few things, with varying results.
Some have argued that it will eliminate programming as a profession, one guy used an analogy from the mechanization of the apparel industry. I don’t think this analogy demonstrates what he thinks it demonstrates…although spinning & weaving were the first things to be mechanized in the Industrial Revolution, there are still millions of people employed in worldwide apparel-making. The final stages have so far proved stubbornly resistant to automation. I suspect the same will be true in many other fields.
@ Chuck > ” Making stuff up is normal. All the AI is doing is talking like a person.”
Maybe so; but the people asking the AI for information don’t want, and don’t expect, fiction.
It seems to me to be incumbent on the AI trainers to include some kind of constraints about “making things up” unless the ‘bot is specifically asked to do so.
At the very least, the instructions should prohibit making up citations to references that don’t exist.
As Neo said, the information about the youngest governor existed; it only had to be regurgitated.
That can’t be so hard, unless the programming teams themselves are incompetent.
Or perhaps miguel is correct about the larger question.
https://www.powerlineblog.com/ed-assets/2023/06/Screenshot-2023-06-02-at-8.54.05-AM.png
It seems to me to be incumbent on the AI trainers to include some kind of constraints
The whole thing was “talk like a human”, now you want “fact checkers”. That a whole other level and loaded with ambiguities. What is the “true” diet advice? Is communism better than capitalism? Math probably has the best chance here, and indeed theorem provers are getting better, but even finding “truth” using the scientific method is fraught with difficulties. For the example here, you would need to rate sources. In this case, simple majority rule would probably work, but consensus isn’t proof. But that is beside the point, the aim was “talk” like a human and the results are pretty amazing. It may be rather disturbing that it makes stuff up, but I would also argue that much of what we consider “true” is merely convention. Certainly much of what is considered news these days isn’t far from fantasy IMHO. And even eyewitnesses differ in what they saw, probably because of their expectations, interpretations, and what catches their eye. Truth is a hard thing to come by. That is the disturbing lesson of LLMs.
Yes, I’m pretty sure LLMs (graduate students in law schools) lie and make things up all the time.
If AI is just adding one likely word to the last word there may be some kind of diagnostic to figure out how it came up with Crawford “Chet” Taylor. For example: Samuel Crawford of Kansas was one of the nation’s youngest governors. C.I. Crawford was governor of South Dakota at one point. [Jack] Crawford Taylor founded the Enterprise Rent-A-Car Company and the Crawford Taylor Foundation.
It’s probably a little more complicated than just consecutively going with the next most likely word. My guess is that ChatGPT came up with “Crawford Taylor” before it inserted “Chet” in between, but still, where the heck does “Chet” come from?
There will probably be many future refinements. If true Artificial Intelligence ever does come about, is there any reason to believe that it would feel bound to obey Azimov’s Laws of Robotics? It might not value human life very much at all.
I think morality can only be taught to beings who can suffer pain. If an AI can’t be made to hurt, it cannot ever understand why an action might be evil–even if that’s how the rules label that action.
At my grandson’s graduation ceremonies, one of the student commencement speakers was thanking various groups who helped the student’s development. The biggest cheer from the graduates was when he thanked ChatGPT.
Hmmm.
Since the greatest virtue these days seems to be the ability—and willingness—to feel (i.e., to believe one feels) and to cause maximum pain, hurt and embarrassment, I wouldn’t count ChatGPT out necessarily.
(But that’s a pretty low bar….)
An LLM is essentially a Transformer-based neural network, introduced in an article by Google engineers titled “Attention is All You Need” in 2017. The goal of the model is to predict the text that is likely to come next.
I think my car purposely tried to make me late by being slow to start. I’ll make sure not to let the gas gauge get so low before I fill next time, I probably made it anxious. Or maybe it didn’t like the technicians at the shop I normally take it to; is there some way to find out what it wants? I could get written up if I’m late again.
I think my toaster burned my bread because it’s bored with being asked to make toast all the time. Maybe it would enjoy helping make bread. But how do I ask it? I hate the smell of burnt toast and the engineers who built my toaster should really have built it to not get bored and play tricks.
I hope there’s a way to teach my gigantic table of numbers to stop lying to me when I give it a text prompt which includes a question that has a factual answer. Since the answer to my question could have been found with a simple web search, it should have known to search the web to find factual information, instead of just predicting text in response to a prompt. I think the engineers who set up the system that generates that giant table of numbers should tell it to do that in the future and not just predict text in response to a prompt.