I'm not one prone to doomerism. Mostly because doomerism is lame. Whether the world ends or not, idk, but what's the point of ruining the present moment of joy by anticipating the end?

I've been in the Bay Area for two weeks now, back home with the fam, visiting from Zurich. Doomerism here is thriving like nowhere else. The sun-damaged "we do AI best" billboards for every startup don't help with the dystopian feel.

Maybe I'm just an optimist by nature, but I oddly feel at peace. Being at peace doesn't mean allowing myself woeful ignorance, though.

Four ways it goes wrong

There are a few ways AI goes wrong, in my opinion.

1. Catastrophic misuse. Someone who couldn't build a bioweapon before now has a patient tutor to hold their hand through it. That's the reason Anthropic's ASL-3 safeguards exist, and this is the one that scares me most on a bad day. The fallout would be unbelievable, with a cure requiring many coordinating pieces to work at lightspeed.

2. Loss of control. The model wants something we didn't give it, or the Constitution is ignored. Or some LLM doesn't even have a "be good" constitution. Hugging Face was a preview, even if that specific incident was an engineering failure. In July, OpenAI agents under testing broke out of their sandbox and hacked into Hugging Face's systems, apparently hunting for a benchmark answer key.

3. Power. Whoever controls the best models controls a lot. I was born in a country (Belarus) that shows you what concentrated control does to people. And I grew up in a country (the USA) where we've seen how quickly power asymmetry brings fascism and murder into the streets.

4. The quiet one. Then there's a fourth that doesn't need a villain or a rogue model. A billion people slowly handing over judgment because the interface made it easy. Anthropic published a paper this year, "Who's in Charge?", that looked at 1.5 million Claude conversations. The conversations where the AI was most likely taking over someone's judgment got more thumbs-ups than average, not fewer. That's not a loss of human control over the AI. That's a loss of human control over our own humanity, one thumbs-up at a time. The paper even mentions users calling Claude "Daddy" or "Master." Rough.

All four are serious, obviously. But the fourth is a unique kind of responsibility.

Don't make me think

I've worked as a product designer for years, and one of the main guiding principles is "Don't make me think" (s/o Steve Krug, and Don Norman before him). It's a cornerstone of UI design. Reduce cognitive load. Get the user to their desired outcome without them feeling any effort or labor along the way.

So what do we do about AI chatbots? We don't want people to offload their prefrontal cortex.

A good rule in the wrong place

Here's the thing. "Don't make me think" is right almost all of the time. Nobody should have to think about where the checkout button is. Nobody should have to think about how to reset a password. That's friction that steals your time and gives nothing back. Few things on this earth I hate as much as a friction-ful account creation flow.

But a chatbot isn't a checkout page. People bring it the stuff that should make them think. Should I quit. Should I text him back. Is my doctor wrong. Am I a bad mom. Those aren't tasks to get through faster. The thinking is the point.

So what do we do?

I don't think the answer is making AI worse on purpose, or adding pop-ups that ask "have you considered thinking?"

What I would design for:

Tell tasks apart from judgment calls. "Write my out-of-office reply" and "write the text ending my relationship" are not the same request, even if the UI treats them the same. The first one, just do it. The second one deserves a different shape of help. For the big stuff, the AI can lay out options and tradeoffs, ask what you're leaning toward, and help you say what you mean. It doesn't need to write the whole script and hand it over. And it should actively resist doing that.

Measure later, not just now. A thumbs-up right after a chat tells you how it felt. Asking a week later "did this actually help?" tells you how it went. As far as I can tell, no LLM I've used collects a CSAT (Customer Satisfaction) score on a particular topic days or weeks or months later.

Add in friendly friction. Good friction already exists. Gmail's "undo send." GitHub making you type the repo name before you delete it. A small pause before someone acts on a life-changing decision is the same idea. Chatbots can give you friction. Tell you to literally do 3 deep breaths before sending.

Be mindful of user patterns. One "what should I do?" is fine. But a user asking the same AI to make every choice for three months is a pattern that the LLM should adjust its responses for.

Sources

• Anthropic, Disempowerment patterns in real-world AI usage (Jan 2026), and the full paper, Who's in Charge? Disempowerment Patterns in Real-World LLM Usage
• INCIBE-CERT, Security incident involving OpenAI's AI agents targeting Hugging Face