As AI Takes His Readers, A Leading History Publisher Wonders What’s Next
World History Encyclopedia CEO Jan van der Crabben saw his site show up in Google's AI Overviews and ChatGPT. Then traffic dropped 25%.
Late last year, Jan van der Crabben’s AI fears materialized. His World History Encyclopedia — the world’s second most visited history website — showed up in Google’s AI Overviews, synthesized and presented alongside other history sites. Then, its traffic cratered, dropping 25% in November.
Van der Crabben, the website’s CEO and founder, knew he was getting a preview of what many online publishers may soon experience. His site built a sizable audience with plenty of help from Google, which still accounts for 80% of its traffic. But as AI search and bots like ChatGPT ingest and summarize the web’s content, that traffic is starting to disappear. Now, his path forward is beginning to look murky.
“There used to be this implicit agreement between publishers and Google that basically, Google could scrape, analyze, process, and do whatever they wanted with publishers’ content and in return, they would send traffic to the publishers, send them readers,” he told me. “Now, this unspoken contract is kind of breaking.”
World History Encyclopedia is just one site and Van der Crabben’s anecdote is just one story, but the migration of readers from the web to AI summaries will likely continue. The internet has always favored ease. And using generative AI to find the most valuable parts of the web’s evergreen content — like recipes, personal finance, and history content — can be a better experience than poking through sites one by one. Along the way, these systems will likely rewrite the economics of the web, and perhaps reshape the internet itself.
Though a non-profit, World History Encyclopedia sustains itself largely through advertising, which makes up 70% of its revenue. It employs nine people, publishes approximately ten articles per week, and translates its content into 40+ languages. To run this operation, it places ads next to its articles and depends on search to bring visitors to its pages. As traffic drops, so does revenue, and that throws the site’s ability to keep churning out history articles into question.
Van der Crabben is well aware that Google has helped him scale in the first place, but what’s different now, he said, is that the arrangement is no longer mutually beneficial. “Now it's just Google that benefits,” he said. “And the same is true, let's be honest, for ChatGPT, for Anthropic, for many of the AI chatbots that gobble up the content. And, for most of them, unless you sue them — which only a big corporation like Reuters can do — they are not going to pay you for it.”
A Google spokesperson said the company sends billions of visits to the web each day and that people using AI Overviews are more satisfied with their results. “Site traffic can change for many reasons, including seasonal demand, interests of users, and regular algorithmic updates to Search,” the spokesperson said. “More broadly, the way people want to consume information is changing, and we're evolving Search to make it easy for people to get a quick answer and also dive deeper on the web.”
It’s not just Google that’s featuring World History Encyclopedia’s content. The site has a partnership with Perplexity, which does offer a revenue share model. But others including OpenAI — whose ChatGPT features the site’s content — won’t respond to his emails. OpenAI did not reply to a request for comment.
As more people interact with AI products, the quality of AI-generated content could decline if primary sources like World History Encyclopedia can’t fund new content creation. Van der Crabben imagined one scenario where, instead of websites, content creators get paid directly by AI companies to write training content.
“We will no longer be publishing to readers, but we will just be feeding the machine, and the readers read what the machine gives them,” he said, with a degree of concern. “Who controls the machine? What biases does the machine have? How does this affect the world?”
Van der Crabben, meanwhile, is working to lessen the site’s reliance on search traffic. He’s set up a membership program for instance, which currently has more than 2,000 paying subscribers. He’s thinking about other formats that should be resilient to AI, like book publishing and an app. And he’s not giving up hope. “We are taking steps, but nevertheless, the business model that we've had for the last 15 years is — I don't want to say destroyed — but it's being questioned. And so that does make us nervous.”
Quote Of The Week (sponsor)
"In service, the only outcome that matters is resolution. Your customers do not care how many steps you eliminated; they care about one thing: did you resolve my problem or find an opportunity to assist me quickly and effectively?
Zendesk CEO Tom Eggemeier announced the company's new Zendesk Resolution Platform will help companies build AI customer service agents and charge based on resolution, not volume. Eggemeier stopped by my YouTube page this week to talk about AI's role in customer service.
What Else I’m Reading, Etc.
Alibaba chair Joe Tsai thinks there may be a AI data center “bubble” [Bloomberg]
CoreWeave violated the terms of a $7.6B loan from BlackStone [Financial Times]
ChatGPT-4o’s Studio Ghibli-esque characters prompt copyright concerns [Tech Crunch]
SoftBank and OpenAI are finalizing a world record breaking $40 billion funding round [Bloomberg]
Judge blocks injunction from record labels suing Anthropic over copyright infringement [WSJ]
Many Chinese AI data centers are sitting unused [MIT]
This Week On Big Technology Podcast: AI's Rising Risks: Hacking, Virology, Loss of Control — With Dan Hendrycks
Dan Hendrycks is the director and co-founder of the Center for AI Safety, and an advisor to Scale AI and xAI. He joins Big Technology Podcast for a discussion of AI's growing risk profile, and what to do about it. Tune in to hear Hendricks explain why virology expertise in AI models is an immediate concern and how these systems might soon enable devastating hacks. We also cover intelligence explosion scenarios, the geopolitical implications of AI development, and why an international AI arms race could lead to faster development than the world can handle. Hit play for an insider's perspective on how governments and AI labs are wrestling with unprecedented technological power that could reshape global security.
You can listen on Apple Podcasts, Spotify, or your podcast app of choice
Thanks again for reading. Please share Big Technology if you like it!
And hit that Like Button if you prefer reading Big Technology in human-written form
My book Always Day One digs into the tech giants’ inner workings, focusing on automation and culture. I’d be thrilled if you’d give it a read. You can find it here.
Questions? News tips? Email me by responding to this email, or by writing alex@bigtechnology.com Or find me on Signal at 516-695-8680
Join our Discord!
Paid subscribers get access to Big Technology’s Discord server, where we talk about the latest in AI every day, you can join here: