Scale AI Field CTO: AI Isn’t Taking Your Job Anytime Soon
AI isn’t yet measuring up to benchmarks Scale set for human-level adeptness in fields like law, finance, and education.
Scale AI is working to bring artificial intelligence up to human-level proficiency by having deeply skilled humans teach it what they know. The company’s contributor network is made up of 12% PhDs and 40%+ masters and professional degree holders who are training the AI in their areas of expertise.
And yet Scale, at least publicly, wants you to know that your job is probably not at risk from artificial intelligence.
The company’s Field Chief Technology Officer, Vijay Karunamurthy, reached out after reading my story last week about AI being able to do parts of my job and asked to give a counterpoint. He told me, in short, that modern day AI isn’t measuring up to the benchmarks the company set for human-level performance. And even if it approaches that level, the AI will likely be a complement to human workers and not neccesarily a replacement.
“Even the most powerful AI models still lack the creativity, emotional intelligence, and nuanced decision-making that people bring to their work,” Karunamurthy said.
Scale sets benchmarks for human-level capabilities in certain professions and found a sizable gap between the humans and machines so far, Karunamurthy said. In finance, for instance, the company tested AI’s ability to give retirement advice and found that it lacks the empathy to adeptly handle high stress situations. “We know from these benchmarks, the models can't yet replace a really good wealth manager,” he said. When AI prepares investment advisors in advance of a call though, the company found customer satisfaction scores tend to rise meaningfully.
Last week I argued that AI can replace a number of tasks I do as a journalist, and not the job itself, but Karunamurthy’s point is well stated. Often, the narrative of AI replacing humans assumes that companies are happy with their workers’ output and would reduce their workforce if AI could do human tasks. But in reality, AI can often only do some things better than people. And when humans work alongside AI they can do a better job than when working alone, leading to better overall performance.
In medicine, for example, someone might be able to speak with a chatbot about an issue they’re having, but that likely doesn’t measure up to a doctor or nurse who comes to a conversation with a patient’s medical context synthesized and presented by AI. And in law, we’ve seen ChatGPT try, and fail, to argue legal cases. But lawyers working with legal bots like Harvey can sort through documents more efficiently, leading to better performance. “The legal domain is a creative field where you learn over time how to leverage your knowledge, your thinking, to solve really challenging problems,” Karunamurthy said, “AI is still not able to replicate that or replace that.”
Still, as Karunamurthy spoke, he mentioned “really good” lawyers and doctors and not average ones. And as AI capabilities improve — and they are moving fast — it’s unlikely to be a painless transition for everyone. Even as he waxed optimistically about our AI future, Karunamurthy allowed that customer service is one area where average employees could be replaced.
“If you have a customer support call center where people are waiting on the phone for 10 minutes, 20 minutes to try getting an answer, and suddenly replace that with an agent that can get you an answer right away, customer satisfaction scores go through the roof,” he said.
Should a robot takeover of work be en route, it at least doesn’t appear imminent. Scale, for instance, worked with the Center for AI Safety to design Humanity’s Last Exam, a compendium of about 2,700 difficult questions submitted by more than 1,000 professors, researchers, and advanced degree holders. The top models today are only able to answer about 20% of the questions, showing a wide gap between AI and experts in their fields.
The 20% result is reassuring, but just a few months ago the AI was barely reaching 10%. With technology moving this fast, predictions about its labor implications are difficult to get right. The one thing that’s certain though, is big change is on the way.
The Future of Building Construction? (sponsor)
Housing and building construction are big issues today, in a trillion-dollar market with a shortage of homes, there is a real need for a solution. And we think BOXABL has the solution.
BOXABL has raised $170M since 2020 to make quality housing affordable for everyone. Their plan is scaling factory mass production. Join 40,000 investors, shares are $0.80 each and the minimum investment is just $1,000.
Time is running out to invest in BOXABL on StartEngine. Don’t miss out on investing in BOXABL for $0.80/share. Invest in BOXABL today.
Disclosure: *This is a paid advertisement for Boxabl’s Regulation A offering. Please read the offering circular here. This is a message from Boxabl
Advertise with Big Technology?
Reach 150,000+ plugged-in tech readers with your company’s latest campaign, product, or thought leadership. We have availability starting in May. To learn more, write alex@bigtechnology.com or reply to this email.
What Else I’m Reading, Etc.
Gruber: Something is rotten in Cupertino [Daring Fireball]
Google’s new single GPU Gemma 3 accelerator is beating DeepSeek v3 and o3-mini in performance [The Verge]
Alibaba’s Qwen and Manus AI team up [Reuters]
It may be over for Roomba [Bloomberg]
French artists sue Meta over “monumental looting” by the company’s AI models [Reuters]
Meta is reportedly testing its AI models on in-house chips built in collaboration with TSMC [Reuters]
OpenAI releases new APIs and tools to help devs create more agents [Venture Beat]
Even Home Depot is dabbling with AI [Marketplace]
Read MIT Tech Review’s first impression of Manus [MIT]
OpenAI proposes letting AI train on copyrighted material and classified datasets [CNBC]
Scarlett Johansson will not take a photo with you [InStyle]
I joined CNBC to talk about Apple Intelligence’s failures [YouTube]
Number of The Week
$3.5B
That’s how much the Saudi Public Investment Fund-backed gaming company, Scopley, purchased Pokemon Go and some lesser owned games for from Niantic on Thursday.
Quote of The Week
To create an Einstein in a data center, we don't just need a system that knows all the answers, but rather one that can ask questions nobody else has thought of or dared to ask. One that writes 'What if everyone is wrong about this?' when all textbooks, experts, and common knowledge suggest otherwise.
Hugging Face co-founder Thomas Wolf in a post explaining why current LLM systems aren’t making scientific discoveries.
This Week On Big Technology Podcast: Roblox CEO: We Want AI To Generate Full Games — With David Baszucki
David Baszucki is the CEO of Roblox. He joins Big Technology Podcast to discuss how the company envisions generative AI playing into game creation on Roblox. Tune in to hear his vision for a future where users can prompt entire game experiences and modify them in real time using AI. We also cover child safety concerns, Hindenburg Research allegations, and whether parents should give their kids money to spend on Roblox. Hit play for an insightful conversation about the intersection of gaming, AI, and the future of creative digital experiences.
You can listen on Apple Podcasts, Spotify, or your podcast app of choice
AI Hype May Be Driving the Most Expensive Domain Sale Ever
By Owen Lavine, Big Technology Contributor
Larry Fischer and his son Jeff, made a big splash earlier this month when they set off to sell one of the web’s most sought after domain names: AI.com.
The Fischers are top domain brokers and have inked some of the largest domain deals in history such as Chat.com (recently bought by OpenAI) and Home.com. Now, they are looking to close the largest domain deal in history.
AI.com is currently owned by…