The Token Reckoning is Here and It’s Not What You Think

Token waste in service of productive use, not waste for the sake of waste, could be the biggest problem in AI right now.

In June 2025, OpenAI CEO Sam Altman wrote that “Intelligence too cheap to meter is well within grasp,” predicting that as “datacenter production gets automated, the cost of intelligence should eventually converge to near the cost of electricity.”

Exactly a year later, AI costs are starting to look less like cheap intelligence and more like an AC bill in the middle of a heat wave. Stories of companies blowing through their annual token budgets in months abound, and at least some companies are rethinking (or cancelling) major AI initiatives.

But while recent headlines about ‘tokenmaxxing’ and runaway budgets have caused concerns that AI’s momentum has been inflated, the panic might be misplaced.

Token waste in service of productive use, not waste for the sake of waste, could be the biggest problem in AI right now. For every $1 in AI spend, 82 cents never make it to production, according to the startup EntelligenceAI, which estimates only 18 cents turn into shipped product, with the rest going to bug fixes, rewriting, or reworking code, and review processes.

If even directionally accurate, EntelligenceAI’s numbers are a major warning sign for the AI boom. Tokenmaxxing can be fixed by removing leaderboards or monitoring budgets closely. Unproductive uses of the technology can only be solved by finding productive uses.

This is why Uber COO Andrew Macdonald saying last week that it was tough to justify all the AI spending, because it wasn’t leading directly to productivity, went viral. If Uber’s experience is representative, that would signal an era of slowing growth ahead.

To be sure, negative AI stories seem to travel further than positive ones, and many companies are finding productive uses of the technology. So some caution is warranted. But ultimately, the industry will have to sort out the ROI question or risk a significant comedown.

The Right Way To Build AI Agents — With NVIDIA’s Adel El Hallak and ServiceNow’s Joe Davis (Sponsor)

Joe Davis is the EVP of AI Engineering, and Delivery at ServiceNow. AdelEl Hallak is the VP of Product Management, Agentic AI at NVIDIA. Both join for an episode on the inner workings of building AI agents. Tune in to hear how both are working together to make autonomous AI agents safer and governable, how ServiceNow's L1-AI-IT specialist has automated 90% of internal support tickets, and what the next few years hold for agents. Hit play for a behind-the-scenes look at what it actually takes to deploy AI agents in the enterprise.

The Intelligence Report

• Microsoft Build is this week on June 2nd and 3rd and will likely include a range of new AI-related news and other updates for hardware and software. Expected news includes new homegrown AI models and updates for making Copilot a superapp that could help it compete with ChatGPT, Claude, and Gemini.

• In a speech today at a WAN-IFRA conference, New York Times publisher A.G. Sulzberger escalated the media industry’s fight with AI companies, warning that OpenAI, Google, Meta and Perplexity are undermining journalism by repackaging publishers’ work, siphoning off traffic and revenue, and making it harder for news organizations to fund original reporting.

• Also today, Florida’s attorney general filed a lawsuit against OpenAI and Sam Altman over alleged chatbot harms, claiming ChatGPT is too dangerous and addictive for minors, especially without adequate safeguards. As Politico notes, it’s the first time OpenAI has been sued in this way by a specific state.

• Yesterday, NVIDIA announced a new superchip called the RTX Spark, which will power Windows PCs in the era of personal AI agents. It also puts NVIDIA in more direct competition with Intel, Qualcomm, AMD, and Apple, which also power laptops, desktops and tablets across the PC market.

• Last week, OpenAI published a Frontier Governance Framework about its safety and security practices and how they align with regulations like California’s new frontier AI law and the EU AI Act’s GPAI Code of Practice. The OpenAI Foundation also committed $250 million to help workers, communities and economies navigate AI-driven labor disruption.

Join The Big Technology AI Summit! June 18 at The Commonwealth Club

Learn more at: summit.bigtechnology.com

Anthropic’s funding, Opus 4.8 and a post-Vatican halo effect

Even if AI never builds a Tower of Babel, Anthropic’s tower of cash might still reach Heaven. Days after co-founder Chris Olah was at the Vatican with Pope Leo XIV, the company made several big financial and product announcements:

• Anthropic raised a $65 billion Series H round at a $965 billion post-money valuation, which the company expects will help expand compute, advance safety and interpretability research and scale products and partnerships. It also said annualized run-rate revenue passed $48 billion earlier in May.

• Anthropic also launched Claude Opus 4.8, featuring improvements in coding, agentic tasks, financial analysis, writing and knowledge work. It claims Opus 4.8 is now three times cheaper and 2.5x faster in fast mode.

• It also claims Opus 4.8 has lower rates of misaligned behaviors, with rates now similar to the Claude Mythos preview and much lower than Opus 4.7 and Sonnet 4.6.

• Early testimonials include clients like Shopify, Cursor, Harvey, BrowserBase, Bridgewater, Thomson Reuters and Databricks. Early reception has been positive, with Every’s Dan Shipper joking that Opus 4.8 “should’ve rounded it up to Opus 5,” but he also noted OpenAI’s Codex remains a stronger daily coding harness.

• Anthropic also announced new dynamic workflows in Claude Code that allow Claude to write orchestration scripts that run tens to hundreds of parallel sub-agents in a single session. Examples of what it helps include conducting bug hunts, migration and stress testing.

• Anthropic also filed to go public on Monday.

Big Technology Podcast Friday Edition: Warning Signs For The AI Boom, Anthropic Passes OpenAI, Robinhood’s AI Trading

Ranjan Roy from Margins is back for our weekly discussion of the latest tech news. We cover: 1) Companies are reconsidering their AI spend after token consumption explodes 2) Is this a widespread issue or a big deal made out of a few companies? 3) The bigger problem: only 18% of tokens are spent on things that ship. 4) Are investment decisions being made due to unrestrained tokenmaxxing? 5) The circular investment problem is real 6) A look at the memory chip boom 7) Anthropic passes OpenAI as the world’s most valuable startup 8) Robinhood let’s your favorite chatbot trade for you 9) Should you connect your gmail to ChatGPT? 10) Would you get your house cleaned for free if the cleaner videotaped it for training data?

You can listen on Apple Podcasts, Spotify, or your podcast app of choice