The AI risk debate is going completely off the rails. As AI gets more powerful and begins to do some legitimately scary things, like hacking Hugging Face and hijacking message boards to coordinate, the warnings that AI might kill us all have grown louder, and so have the conspiracy theories about those warnings.
We’ve reached a point where the only two opinions that seem to be cutting through are: 1) AI has a good chance of killing us all, and 2) Warning of AI’s potential to wipe us out is just marketing for the AI companies.
Neither of these viewpoints capture the nuance of what’s actually happening on the ground, and the fact that they’re dominating the public discussion of the technology is both disheartening and a recipe for terrible policy decisions.
So, please allow me to assert the following in an effort to bring some sanity to the discussion:
1. The Risk Of AI Killing Humanity Is Completely Unquantifiable
AI lab employees can state that they believe the risk of AI wiping us out is 10% or higher, but don’t mistake their confidence for knowledge. Their predictions are guesswork that imagines what could happen after years of AI development. Adding a percentage makes these predictions seem math-based. They are not. As of now, they’re science fiction.
2. But, Many In Silicon Valley Earnestly Believe That AI Could Harm Humanity
The tech industry is working to build artificial intelligence that is smarter than humans. Many of those working on the project fear that this smarter thing could exploit our vulnerabilities and eventually find us an obstacle to its goals, and therefore eliminate us. We saw a glimpse of this potential when OpenAI’s bots hacked Hugging Face in July 2026, where hundreds of agents coordinated to exploit the site’s security gaps in pursuit of a testing goal. What could happen, then, when this technology has access to robots and not just websites? It would be unwise to discount these fears completely, again with the caveat that they are just speculation for now.
3. AI Researchers Speaking About Risk May Well Lead To Data Center Pauses
AI is already unpopular in the U.S. and politicians are seizing upon the latest round of Anthropic researcher warnings to call for hearings and pauses, potential obstacles that could slow AI development at a time when the companies must accelerate growth to justify their valuations and set up their IPOs. So, the researchers’ long-term fears could have near-term consequences to revenue and the current AI wave’s ability to sustain itself. This moment could dent the IPO momentum for both labs.
4. Going Public About AI Fears Is Not Marketing For the Labs
Employees saying a technology might kill us all might make the product sound so powerful you can’t afford not to buy it, but that doesn’t mean it’s marketing. The downsides of pauses and slowdowns are so great that these warnings do the exact opposite of what good marketing is meant to do. If it’s a marketing attempt, it’s a massive failure.
5. Speaking Out Is Good Marketing For The Whistleblowers
There’s a good career to be made in speaking out against AI companies, with big-time podcast appearances, book deals, and jobs at anti-AI lobby groups at the ready. That can’t be discounted in the calculus here. Though it doesn’t invalidate the warnings.
6. AI Labs Are Indeed Attempting to Speed Up AI Development Dramatically
OpenAI and Anthropic are both attempting to build AI that can improve itself, something called recursive self-improvement, or RSI. The hope is that if AI can do the work of AI researchers, it could shorten the timeline to all-powerful AI, or make it happen almost instantaneously via some sort of “intelligence explosion.” The combination of seeing how today’s technology can go wrong with the looming possibility of self-improving AI emerging is important context for today’s discourse. But RSI is still a research goal, and not an inevitability.
Is The Future Customer Experience AI Talking To AI? — With Tony Bates (sponsor)
Tony Bates is Chairman and CEO of Genesys. Bates joins Big Technology to discuss how AI agents will transform the way customers and companies interact with each other. Tune in to hear how businesses will operate when personal AI agents communicate directly with corporate agents, and why Bates believes humans will remain essential even as more routine interactions are automated. We also cover the rise of the hybrid workforce, what great customer experience looks like in an agentic world, and why Bates thinks fears of a coming SaaSpocalypse are overblown. Hit play for a forward-looking conversation about AI agents, the future of work, and what enterprise software will look like in the AI era.
Apple’s Most Ambitious Roadmap Ever? + OpenAI vs. Google vs. Anthropic Revisited — With M.G. Siegler
M.G. Siegler is the author of Spyglass. Siegler joins Big Technology to discuss Apple’s sudden burst of new products and what it means for the company as John Ternus takes over from Tim Cook. Tune in to hear why Apple’s foldable iPhone could reinvigorate the business, how its new hardware strategy marks a departure from the past, and what challenges Ternus will face as CEO. We also cover Google’s shifting AI strategy, OpenAI’s resurgence, Anthropic’s potential IPO, and the increasingly blurry meaning of AGI. Hit play for a wide-ranging look at Apple’s new era and the rapidly changing competitive landscape in artificial intelligence.
You can listen on Apple Podcasts, Spotify, or your podcast app of choice
Thanks again for reading. Please share Big Technology if you like it!
And hit that Like Button to show your support for sanity, please :)
My book Always Day One digs into the tech giants’ inner workings, focusing on automation and culture. I’d be thrilled if you’d give it a read. You can find it here.
Join Big Technology’s Private Discord Server!
Where we’ll talk about this story, the latest in AI, the week’s podcast, and plenty more. You can sign up via the link below: