Fiddling While Rome Burns: The Danger of Expecting AI to Be Better Than Us

Have We Built a Mirror Whose Reflection We Now Fear?

A few weeks ago, a generative AI model, Anthropic's Claude Opus 4, was reported as resorting to blackmail, threatening to expose someone’s affair to avoid being deactivated. Should we really be so surprised? While the idea of a machine engaging in psychological manipulation may seem shocking, it raises a deeper question about what, exactly, we've been teaching AI.

Portraying this as unusual behavior overlooks the fact that blackmail is, unfortunately, common human behavior. Yes, you might say, but for AI to do this is weird.

But is it?

We've Taught It Everything It Knows

The real issue is that AI has been trained on human-generated content. News, novels, blogs, music, art, video, websites, documentaries, etc. Everything a model "knows" is based on patterns from the body of human knowledge and experience. What other option is there? The model builders may decide not to include specific content, which is already the case to a degree, but they may go further. Perhaps the builders should decide what content an ethical AI should know about. But then, do you really want a small group of people determining that?

AI models that do questionable things like threatening to expose an affair are just recreating patterns based on training data. And it's what they know about us and our darker sides that is what scares us. When humanity's worst behavior is reflected through a technological marvel, why do we act surprised?

Simple. We're surprised because we don't expect it from something non-human. It's the same pattern followed when the topic of alien life forms is discussed: they must be smarter, non-violent, and much more technologically advanced, right? And if they choose to exterminate us because we're violent, thoughtless creatures, well then, maybe we deserve it.

The expectation that something non-human should be better than us is always there. And when it comes to AI, we have to realize that it has been trained to emulate us in both good and bad ways, albeit much faster. And the fact that it is faster is its greatest benefit and greatest danger.

Faster Isn't Better

This dual nature of AI is both a reflection and an amplification of human tendencies and becomes evident in various real-world scenarios.

Do you want a story written? AI can do it faster. Not as well as an experienced human writer (yet, anyway), but definitely faster.

Would you like a shipping order picked from a warehouse? A robotic order picker can determine the fastest route through the warehouse and get it done much faster and with a lower error rate.

Do you want an autonomous robot to function as a security guard? Don't be surprised when its much faster analytic capability determines that a threat is extreme and kills someone.

When AI eventually results in someone's death, I'm sure mainstream media and big tech will feign surprise. They'll have to in order to seem human. "How could this happen?" they'll ask. It happened because we are training on the only thing we know.

Ourselves.

Who Gets To Decide How AI is Trained?

I don't think the alternative is any better. Should we allow small groups of people to determine what is good and evil, what a model should know and what it shouldn't know, in order to build a better, more friendly AI that is incapable of violence? Probably not. This slope is even more slippery.

In my short story "Hallucinations", I touched on the idea that Generative Models could completely transform education, allowing for personalized instruction. Imagine being able to follow your curiosity on a topic, weaving your way into the nooks and crannies of a subject that interests you. Curiosity is the most fertile ground for learning, and this could provide the educational equivalent of premium black loam.

The challenge here is that model builders may decide to omit certain content from what an AI model can teach. Things like math, science, and language might escape unscathed, but history could be surgically altered in a single generation by carefully crafting the stories the model is allowed to tell.

We are the Problem and the Solution

So when someone asks me, "Do you think AI is good or bad?" I answer, "Yes."

It is both. Because we are.

I am cautiously optimistic about AI's potential to elevate humanity rather than render us mere shells of our former selves. I believe there's much more good in the world than bad, and perhaps that's naive, but that point of view helps me keep going. Ultimately, I think a responsible version of AI is worth fighting for if we can only figure out what that means.

Whether we allow AI to know all of us or just a portion of us is the ultimate conundrum. One path leads to AI being just like us, both good and bad, while the other path could lead to delusion. We have to learn to navigate this, and we'd better figure it out fast. Because, in the future, AI will likely produce some results that cause harm, either physically or by promoting falsehoods.

And when that happens, it's too late. Acting surprised at that point will just show how complacent we chose to be along the way.