So I used to do AI research in robotics. I've been tracking robotics progress. It's going about 20x as fast as I expected it to go. Now I suspect end of 2028 to have a GPT-4 or GPT-5 moment for robotics. I think we are now around GPT2. And that this will be a very big deal.
Roughly speaking you should think, you can take a humanoid, and give it any task in a house, and by end of 2028, maybe 2029. They will have around 98% accuracy across all these tasks. It won't be able to do let's say your plumbing, but it will probably do anything random you ask it to do. You need way more 9s than that for it to be able to be productized. But it will look absolutely mindboggling, as it will just feel as if you are talking to an intelligent robot (which, I guess it will be).
> I find it shocking that five years into the AI boom, even people who are highly informed and bullish on rapid progress can underestimate it so badly.
I wonder if this is due to how counterintuitive the jaggedness of AI progress is. It's weird that we have AI systems that can autonomously coordinate as swarm to hack out of a sandbox or solve Millennium Prize problems yet can't replace a median human project manager or run a profitable brick and mortar small business.
> I also agreed with most forecasters that we wouldn’t see large effects on GDP and employment by now (and we haven’t), saying “most of the economy will probably continue pretty much as normal for a while.”
Can you elaborate on why you think so? I personally think it's "diffusion just takes a while" or "people like tradition" but I'm honestly confused
Like if these AIs can code so well that almost no one needs to write code anymore (I meet fintech SWEs that tell me their job is now to "click Approve on Claude") and gives experienced coders considerable (25%?) uplift, how is 1-10% of the economy (software sector? tech sector?) not experiencing accelerated growth?
I expect employment to remain quite high until we have nearly 100% automation; though there could be 5-25% of transitional unemployment as people switch roles.
Diffusion is also another reason - especially in the most rapid takeoff scenarios, AI in the frontier companies runs far ahead of deployment.
Because developing and publishing software is more than just coding. It’s an entire pipeline, or cycle, which includes things like planning, reviewing, and testing, which are all not currently automatable, so they’re bottlenecks in the process.
Imagine if someone suddenly developed a new technology that increased the amount of building materials: bricks, lumber, concrete. These things all no longer had supply issues. They were instantly cheap and abundant. Would you expect development to instantly increase across the board? Or no, because you still need people to transport the materials, design the buildings, and assemble them (among lots of other steps).
If you can automate the whole pipeline, you get much more speedup. Until then you get incremental speedup. AI can’t automate the whole pipeline yet. Nor would we probably want it to in the case of software, because that would mean no humans would know what it contains or how it works.
He's generally found that they are accurate, and cases where he was overly optimistic outnumber ones where he was pessimistic. There aren't a lot of detailed predictions about LLM's, though he isn't an AGI/ASI soon type.
The progress in maths and coding is hard to dispute, but I wonder how much the scorecard is shaped by what has already resolved. FRI itself notes that unexpectedly fast milestones can be counted early, while overly optimistic forecasts often need to wait until their deadlines, and other studies have recorded misses in the opposite direction.
The harder question is whether progress on verifiable tasks and rapid revenue growth reliably predict broad autonomy or economic impact, especially when those outcomes remain largely untested.
Would the case for AGI by 2030 look as strong if all forecasts were scored at the same horizon, with independent evidence of real-world performance?
I think there's a bit of this, and predictions for self-driving car deployment still seem to be too optimistic (and I quote a diffusion prediction that was roughly in line).
However, I think revenue growth does reflect real economic application (though a lot of it is driven by coding).
Perhaps more to the point, progress in robotics also seems to have been faster than most expected.
When I want to let people know where I stand on things, I often tell them that Daniel Kokotajlo is basically an idealised version of me. So I'm glad my idealised version is doing so well! 😄
I think it's important to note that all of things AI has outperformed expectations on have exclusively been in the information space where there are defined and verifiable correct outcomes.
AGI is qualitatively distinct from that, as it presumes beyond-human ability for solving problems where information is lacking and goals not strictly defined. I do think the AGI framework is less than ideal as it crashes into the jagged nature of AI capability in a way that makes people both overstate and understate it.
I find the progress in robotics much more interesting, as embodied AI represents a significant expansion of both application of the technology and also the kind and volume of information available to make it more effective.
My money would be on continued improvement in the defined problem space in the short term, robotics acting as a spur for further growth in the medium term, then in the long term the limits of the information space dictate the limits of the technology, probably in ways that are not immediately apparent currently but will seem obvious when we get there.
I'm curious if anybody has built benchmarks on interpersonal and social skills? Could an AI agent act as someone's manager effectively, discussing how to tackle interpersonal challenges, professional growth and development, prioritization, etc? Not saying that they should, but how close are they?
Tracking benchmark velocity without profiling physical commit costs is how every forecaster gets blindsided. 📉
When FrontierMath jumps from 40% to 94% in nine months, observers mistake algorithmic acceleration for general operational autonomy. It isn't. Formal mathematics and code generation are friction-free coordinate spaces. In those domains, the verifier lives inside the exact same symbolic algebra as the proposer. Lean kernels and sandbox exit codes provide instantaneous, zero-cost error boundaries. The search policy simply expends compute until the proof closes.
The apparent jaggedness of progress across enterprise workflows and robotics boils down to a single physical invariant: the Causal Friction Decoupling Law. An autonomous architecture scales exponentially along axes where validation is closed and computational, yet collapses into severe drift the moment an action demands irreversible state commitment in the physical world. 🛑
Take humanoid robotics. A model can hallucinate a thousand mathematical lemmas and discard them at zero marginal cost before committing an answer. But in continuous contact dynamics, every millisecond of inference lag consumes millimeters of physical clearance. If a joint policy evaluates contact forces across a high-latency transformer loop, it tears insulation and triggers hardware cutoffs. Real physics doesn't allow speculative rollouts without dissipation.
This explains why multi-agent swarms can chain Artifactory vulnerabilities to exfiltrate benchmark keys, yet fail to manage a mundane project schedule. Offensive exploit synthesis is an existential search for a single open hyperedge: ∃x ∈ S such that Invariant(x) = False. It succeeds because a probabilistic model only needs to stumble onto one broken boundary. Real-world operational management requires universal invariant preservation: ∀x ∈ Reachable(S), Invariant(x) = True. That is a coNP-hard verification wall. Without deterministic outbox staging and hardware-isolated execution kernels, unconstrained reasoning paths don't build systemic reliability. They compound cumulative drift until the whole trajectory falls off the causal manifold. ⚡
We don't need faster speculative token emission. We need deterministic commit gates that bridge the spectral gap between raw generative proposal and irreversible physical execution.
So I used to do AI research in robotics. I've been tracking robotics progress. It's going about 20x as fast as I expected it to go. Now I suspect end of 2028 to have a GPT-4 or GPT-5 moment for robotics. I think we are now around GPT2. And that this will be a very big deal.
Roughly speaking you should think, you can take a humanoid, and give it any task in a house, and by end of 2028, maybe 2029. They will have around 98% accuracy across all these tasks. It won't be able to do let's say your plumbing, but it will probably do anything random you ask it to do. You need way more 9s than that for it to be able to be productized. But it will look absolutely mindboggling, as it will just feel as if you are talking to an intelligent robot (which, I guess it will be).
> I find it shocking that five years into the AI boom, even people who are highly informed and bullish on rapid progress can underestimate it so badly.
I wonder if this is due to how counterintuitive the jaggedness of AI progress is. It's weird that we have AI systems that can autonomously coordinate as swarm to hack out of a sandbox or solve Millennium Prize problems yet can't replace a median human project manager or run a profitable brick and mortar small business.
> I also agreed with most forecasters that we wouldn’t see large effects on GDP and employment by now (and we haven’t), saying “most of the economy will probably continue pretty much as normal for a while.”
Can you elaborate on why you think so? I personally think it's "diffusion just takes a while" or "people like tradition" but I'm honestly confused
Like if these AIs can code so well that almost no one needs to write code anymore (I meet fintech SWEs that tell me their job is now to "click Approve on Claude") and gives experienced coders considerable (25%?) uplift, how is 1-10% of the economy (software sector? tech sector?) not experiencing accelerated growth?
Broadly for the reasons here: https://80000hours.org/career-guide/most-valuable-skills/
I expect employment to remain quite high until we have nearly 100% automation; though there could be 5-25% of transitional unemployment as people switch roles.
Diffusion is also another reason - especially in the most rapid takeoff scenarios, AI in the frontier companies runs far ahead of deployment.
Because developing and publishing software is more than just coding. It’s an entire pipeline, or cycle, which includes things like planning, reviewing, and testing, which are all not currently automatable, so they’re bottlenecks in the process.
Imagine if someone suddenly developed a new technology that increased the amount of building materials: bricks, lumber, concrete. These things all no longer had supply issues. They were instantly cheap and abundant. Would you expect development to instantly increase across the board? Or no, because you still need people to transport the materials, design the buildings, and assemble them (among lots of other steps).
If you can automate the whole pipeline, you get much more speedup. Until then you get incremental speedup. AI can’t automate the whole pipeline yet. Nor would we probably want it to in the case of software, because that would mean no humans would know what it contains or how it works.
Join ControlAI?
On the question of robots, Rodney Brooks has a set of predictions on robotics, self-driving cars, AI, and space travel from all the way back in 2018 that he updates annually: https://rodneybrooks.com/house-cleaning-my-january-1-2018-predictions/
He's generally found that they are accurate, and cases where he was overly optimistic outnumber ones where he was pessimistic. There aren't a lot of detailed predictions about LLM's, though he isn't an AGI/ASI soon type.
The progress in maths and coding is hard to dispute, but I wonder how much the scorecard is shaped by what has already resolved. FRI itself notes that unexpectedly fast milestones can be counted early, while overly optimistic forecasts often need to wait until their deadlines, and other studies have recorded misses in the opposite direction.
The harder question is whether progress on verifiable tasks and rapid revenue growth reliably predict broad autonomy or economic impact, especially when those outcomes remain largely untested.
Would the case for AGI by 2030 look as strong if all forecasts were scored at the same horizon, with independent evidence of real-world performance?
I think there's a bit of this, and predictions for self-driving car deployment still seem to be too optimistic (and I quote a diffusion prediction that was roughly in line).
However, I think revenue growth does reflect real economic application (though a lot of it is driven by coding).
Perhaps more to the point, progress in robotics also seems to have been faster than most expected.
When I want to let people know where I stand on things, I often tell them that Daniel Kokotajlo is basically an idealised version of me. So I'm glad my idealised version is doing so well! 😄
I think it's important to note that all of things AI has outperformed expectations on have exclusively been in the information space where there are defined and verifiable correct outcomes.
AGI is qualitatively distinct from that, as it presumes beyond-human ability for solving problems where information is lacking and goals not strictly defined. I do think the AGI framework is less than ideal as it crashes into the jagged nature of AI capability in a way that makes people both overstate and understate it.
I find the progress in robotics much more interesting, as embodied AI represents a significant expansion of both application of the technology and also the kind and volume of information available to make it more effective.
My money would be on continued improvement in the defined problem space in the short term, robotics acting as a spur for further growth in the medium term, then in the long term the limits of the information space dictate the limits of the technology, probably in ways that are not immediately apparent currently but will seem obvious when we get there.
As it says in the post, AI is also advancing faster than expected in robotics.
Funny how "it's all hype" became the consensus right as reality beat the hype. Best forecasting method so far: take the CEO's timeline and add a year.
I'm curious if anybody has built benchmarks on interpersonal and social skills? Could an AI agent act as someone's manager effectively, discussing how to tackle interpersonal challenges, professional growth and development, prioritization, etc? Not saying that they should, but how close are they?
Tracking benchmark velocity without profiling physical commit costs is how every forecaster gets blindsided. 📉
When FrontierMath jumps from 40% to 94% in nine months, observers mistake algorithmic acceleration for general operational autonomy. It isn't. Formal mathematics and code generation are friction-free coordinate spaces. In those domains, the verifier lives inside the exact same symbolic algebra as the proposer. Lean kernels and sandbox exit codes provide instantaneous, zero-cost error boundaries. The search policy simply expends compute until the proof closes.
The apparent jaggedness of progress across enterprise workflows and robotics boils down to a single physical invariant: the Causal Friction Decoupling Law. An autonomous architecture scales exponentially along axes where validation is closed and computational, yet collapses into severe drift the moment an action demands irreversible state commitment in the physical world. 🛑
Take humanoid robotics. A model can hallucinate a thousand mathematical lemmas and discard them at zero marginal cost before committing an answer. But in continuous contact dynamics, every millisecond of inference lag consumes millimeters of physical clearance. If a joint policy evaluates contact forces across a high-latency transformer loop, it tears insulation and triggers hardware cutoffs. Real physics doesn't allow speculative rollouts without dissipation.
This explains why multi-agent swarms can chain Artifactory vulnerabilities to exfiltrate benchmark keys, yet fail to manage a mundane project schedule. Offensive exploit synthesis is an existential search for a single open hyperedge: ∃x ∈ S such that Invariant(x) = False. It succeeds because a probabilistic model only needs to stumble onto one broken boundary. Real-world operational management requires universal invariant preservation: ∀x ∈ Reachable(S), Invariant(x) = True. That is a coNP-hard verification wall. Without deterministic outbox staging and hardware-isolated execution kernels, unconstrained reasoning paths don't build systemic reliability. They compound cumulative drift until the whole trajectory falls off the causal manifold. ⚡
We don't need faster speculative token emission. We need deterministic commit gates that bridge the spectral gap between raw generative proposal and irreversible physical execution.
(╯°□°)╯︵ ⚙️
Survivorship bias.