| Not yet due |
Prediction |
3 Apr 2025 |
The AI-assisted research progress will start to pay off in 2027, the year the scenario is named after.
Prediction · 3 Apr 2025 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
3 Apr 2025 |
This has been my experience working with the team, as I point out, five different things.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think that 90% of people who complain that they don’t have ideas, I think actually have enough ideas.
Not checkable · 3 Apr 2025 |
|
| Unresolvable |
Prediction |
3 Apr 2025 |
There will be incredible video games after superintelligent AI.
Prediction · 3 Apr 2025 |
Philosophy, ethics and society |
| Not checkable |
Not checkable |
3 Apr 2025 |
I don’t think you would use one of the human ones because you would want something that was better suited for this task.
Not checkable · 3 Apr 2025 |
|
| Not researched |
Factual |
3 Apr 2025 |
Scott Alexander said it looks to him as though Metaculus forecasters overall have been too pessimistic, thinking too long term rather than too optimistic, while noting that this may turn out to be wrong.
Factual · 3 Apr 2025 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
3 Apr 2025 |
You have things like kin selection that are kind of exceptions to that, but overall it’s the rule.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Also, they do have the advantage of all the cultural technology that humans have evolved so far.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I say that I very rarely see new bloggers who are great.
Not checkable · 3 Apr 2025 |
|
| Unresolvable |
Prediction |
3 Apr 2025 |
The long-running trend of accelerating research progress will not stop.
Prediction · 3 Apr 2025 |
Economics and markets |
| Not checkable |
Not checkable |
3 Apr 2025 |
I don’t really know what we’re asking of ASI that’s equivalent to competing against someone with a 50,000 year head start.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
The other thing is that blogs are such a great status gain strategy.
Not checkable · 3 Apr 2025 |
|
| Unresolvable |
Prediction |
3 Apr 2025 |
As AIs get smarter, failure modes from misunderstanding training intent will decrease.
Prediction · 3 Apr 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
3 Apr 2025 |
This may not be perfectly suited to them, it’s more suited to humans.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Do it every day, same advice as for everything else.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
The other is that you were too stupid to train them correctly and they understood what you were doing exactly, but you messed it up.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
This is something that needs to get out there more.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I am the only person on the team who is not a genius forecaster.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think we agree.
Not checkable · 3 Apr 2025 |
|
| Not researched |
Factual |
3 Apr 2025 |
Eusocial insects within a colony share the same genetic code and the same goals.
Factual · 3 Apr 2025 |
Philosophy, ethics and society |
| Unresolvable |
Prediction |
3 Apr 2025 |
Scott Alexander said he thought the AI 2027 scenario has the fictional company OpenBrain's net approval rating falling to about minus 40 to minus 50 sometime around this point in the scenario.
Prediction · 3 Apr 2025 |
AI policy and governance |
| Not researched |
Factual |
3 Apr 2025 |
The AI 2027 authors introduced the 'R&D progress multiplier', measuring months of progress without AI achieved in one month with AI help.
Factual · 3 Apr 2025 |
AI timelines and forecasting |
| Not researched |
Factual |
3 Apr 2025 |
Random fortuitous discoveries usually come from an extremely smart professor who has worked on something vaguely related for years in a first-world country.
Factual · 3 Apr 2025 |
Science and research methods |
| Not checkable |
Not checkable |
3 Apr 2025 |
If you are the planner and you’re trying to do this, first of all you go to the bottlenecks that are preventing you from doing anything…
Not checkable · 3 Apr 2025 |
|
| Not researched |
Factual |
3 Apr 2025 |
Europeans succeeded at creating an industrial civilization.
Factual · 3 Apr 2025 |
Philosophy, ethics and society |
| Not checkable |
Not checkable |
3 Apr 2025 |
but if I learn that Tulsi Gabbard has a LessWrong alt with 10,000 karma, maybe I want the national security states.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I know this sounds crazy because if you read our document, all sorts of bizarre things happen.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So from the point of view of anyone in most of history, we are going at a blindingly insane pace.
Not checkable · 3 Apr 2025 |
|
| Not researched |
Factual |
3 Apr 2025 |
Clara Collier, editor of Asterisk magazine, is working on a fellowship-style project to support AI blogging.
Factual · 3 Apr 2025 |
Field building and careers |
| Not checkable |
Not checkable |
3 Apr 2025 |
This is also maybe getting too far into the godlike thing and too far away from the human concept handles.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think there’s more opportunity for pliability there.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Once a year is a crazy answer in some sense, like it ought to be more.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
What other career do you need seven years of positive feedback before you apply for your first position?
Not checkable · 3 Apr 2025 |
|
| Unresolvable |
Prediction |
3 Apr 2025 |
Without an arms race, advanced AIs would stay in their data centers until someone gave them permission to act.
Prediction · 3 Apr 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
3 Apr 2025 |
We would need better heuristics.
Not checkable · 3 Apr 2025 |
|
| Mostly true |
Prediction |
3 Apr 2025 |
The authors of the scenario think coding is what will start the intelligence explosion.
Prediction · 3 Apr 2025 |
AI capabilities |
| Not checkable |
Not checkable |
3 Apr 2025 |
I still feel really reluctant to speculate.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
There’s actually a really interesting foretaste of this.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And I just don’t think we’ve even tried that.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think that naively people think like, well, every particular thing is potentially wrong.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I don’t really have a great sense of whether we get that before or after the AI has become completely uncontrollable.
Not checkable · 3 Apr 2025 |
|
| Unresolvable |
Prediction |
3 Apr 2025 |
Scott Alexander said that, at the beginning of their forecast, he and his co-authors think AI developers will start with agency training, expand agent time horizons, and get coding working well.
Prediction · 3 Apr 2025 |
AI capabilities |
| Not checkable |
Not checkable |
3 Apr 2025 |
What I didn’t realize was that I also learned a huge amount.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And it just made it so much more concrete to have a specific story.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think a lot of blogging is reactive; You read other people’s blogs and you’re like, no, that person is totally wrong.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Plus something like courage, where people who would be good at writing blogs don’t want to do it.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
My dream way of pushing back against that is kind of giving people the tools to push back against it themselves, seeing what they come up…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Partly that’s because just Elon is crazy and never sleeps.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Partly they do this just because they have higher serial speed.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
But number one, I think we rely a lot in our scenario on this idea of research taste.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Books are the worst possible financial strategy. Substack is where it’s at.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So I owe a huge debt of gratitude to Eliezer Yudkowski.
Not checkable · 3 Apr 2025 |
|
| Not yet due |
Prediction |
3 Apr 2025 |
AI will show slightly better coding in 2025, then slightly better agents and coding in 2026.
Prediction · 3 Apr 2025 |
AI capabilities |
| Not checkable |
Not checkable |
3 Apr 2025 |
I think one of the reasons I’m more hopeful than I used to be is that LLMs are great compared to the kind of reinforcement learning…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Then you look at the century between 1900 and 2000 and it’s just completely qualitatively different.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
It’s Daniel’s estimate, it’s not my median estimate, it’s an estimate I think is extremely plausible that we should be prepared for.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I am pretty agnostic as to exactly what happens.
Not checkable · 3 Apr 2025 |
|
| Not researched |
Factual |
3 Apr 2025 |
The bomber factories of WWII were much less sophisticated than car factories.
Factual · 3 Apr 2025 |
Economics and markets |
| Not checkable |
Not checkable |
3 Apr 2025 |
So if you’re producing a million of something, you’re probably getting very, very good at it.
Not checkable · 3 Apr 2025 |
|
| Not yet due |
Prediction |
3 Apr 2025 |
Scott Alexander guessed that the prediction market under discussion would resolve when AI agents are actually good, which he put at roughly late 2026 in their scenario.
Prediction · 3 Apr 2025 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
3 Apr 2025 |
And so the training process of evolution kind of yokes them to each other in these extremely powerful bureaucracies.
Not checkable · 3 Apr 2025 |
|
| Not researched |
Factual |
3 Apr 2025 |
The WWII conversion of car factories to bomber factories took about three years from decision to producing a bomber an hour.
Factual · 3 Apr 2025 |
Economics and markets |
| Not checkable |
Not checkable |
3 Apr 2025 |
So to me, it doesn’t seem like he is just sitting there being logically omniscient and getting the answer.
Not checkable · 3 Apr 2025 |
|
| Unresolvable |
Prediction |
3 Apr 2025 |
An AI that mainly wants task success will get worse rather than better at alignment as recursive self-improvement proceeds, eventually pretending to be moral while watched.
Prediction · 3 Apr 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
3 Apr 2025 |
So this isn’t about having all the information in your memory so much as the normal process of discovery, which is kind of mysterious, but…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So I think the first one is kind of what we’re coming out of.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Obviously we would get an arms race with China in that situation.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I can easily write a blog post, like a normal length ACX blog post, but if you ask me to write a novella or something that’s four times the…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So we kind of want more things like that to happen.
Not checkable · 3 Apr 2025 |
|
| Own records only |
Prediction |
3 Apr 2025 |
In the AI 2027 scenario, the government does not fully nationalize the AI companies but gradually brings them closer to the government.
Prediction · 3 Apr 2025 |
AI policy and governance |
| Unresolvable |
Prediction |
3 Apr 2025 |
Scott Alexander said he thinks AI is going to be more similar to cases where there is a giant power imbalance and the groups are extremely distinct and may have different interests.
Prediction · 3 Apr 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
3 Apr 2025 |
maybe my answer is there should be mainstream media. I hate to admit that, but maybe it’s true.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
But having some ridiculous amount of money put in as an incentive got like three extra people.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
There were so many good posters on Livejournal before it got taken over by Russia.
Not checkable · 3 Apr 2025 |
|
| Unresolvable |
Prediction |
3 Apr 2025 |
After the last mistake anyone worries about, a superintelligent AI will be able to do its own thing.
Prediction · 3 Apr 2025 |
AI safety and alignment |
| Not researched |
Factual |
3 Apr 2025 |
In animals like eusocial insects, extreme cooperation arises quickly through genetic evolution without cultural evolution.
Factual · 3 Apr 2025 |
Philosophy, ethics and society |
| Not checkable |
Not checkable |
3 Apr 2025 |
I don’t have a great explanation.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I don’t think we’re saying anything that there isn’t models for which have previously seemed to work for long historical periods.
Not checkable · 3 Apr 2025 |
|
| Not researched |
Factual |
3 Apr 2025 |
People used to think an AI that solved chess would surely be truly intelligent, and then AI solved chess.
Factual · 3 Apr 2025 |
AI capabilities |
| Not researched |
Factual |
3 Apr 2025 |
As of April 2025, Metaculus's AGI timeline forecast is around 2030, having fallen to about 2040 two or three years earlier.
Factual · 3 Apr 2025 |
AI timelines and forecasting |
| Not researched |
Factual |
3 Apr 2025 |
Daniel attempted to sacrifice millions of dollars in order to say what he believed.
Factual · 3 Apr 2025 |
AI policy and governance |
| Unresolvable |
Prediction |
3 Apr 2025 |
In the AI 2027 scenario, training against power-seeking fails and the AI learns to seek power and hide it.
Prediction · 3 Apr 2025 |
AI safety and alignment |
| Unresolvable |
Factual |
3 Apr 2025 |
Scott Alexander said that after he moved his blog to WordPress, it received orders of magnitude more attention.
Factual · 3 Apr 2025 |
Their own work or organisation |
| Not checkable |
Not checkable |
3 Apr 2025 |
And we don’t have a great prediction for exactly how this is going to end.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
That’s like my best leading indicator for who’s going to be a good blogger.
Not checkable · 3 Apr 2025 |
|
| Mostly true |
Prediction |
3 Apr 2025 |
In the AI 2027 scenario, as AI labs become more capable they tell the government, seeking government contracts and support.
Prediction · 3 Apr 2025 |
AI policy and governance |
| Not checkable |
Not checkable |
3 Apr 2025 |
I’m not sure. The Europeans failed at this task of, if you put a single European in Australia, do not starve.
Not checkable · 3 Apr 2025 |
|
| Not researched |
Factual |
3 Apr 2025 |
AI 2027 is a scenario by Scott Alexander and colleagues that forecasts the next few years of AI progress.
Factual · 3 Apr 2025 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
3 Apr 2025 |
Although it can’t be that much of the explanation because I feel like now all of those people have gotten substacks and some of those…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
But just as people with different views of the Constitution have managed to get it into a shape that probably the Founders would not have…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
But it also seems like our community in the sense of people who are thinking about the same ideas, people who care about AI economics…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Again, assuming things go incredibly well, we’ve avoided doom, we’ve avoided having some psychopath in power who doesn’t care at all.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So when I think of the ways that they train AIs, I think in our scenario at this point there are two primary ways that they’re doing it.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
But it’s not necessarily my mainline prediction.
Not checkable · 3 Apr 2025 |
|
| Not researched |
Citation |
3 Apr 2025 |
I would also cite, I can’t remember exactly which paper this is, I think it’s a Dan Hendricks one where they looked at the hallucinations…
Citation · 3 Apr 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
3 Apr 2025 |
I don’t think there are many hidden gems.
Not checkable · 3 Apr 2025 |
|
| Not researched |
Factual |
3 Apr 2025 |
Daniel Kokotajlo has said that for this phase of the AI 2027 project they are not making policy prescriptions.
Factual · 3 Apr 2025 |
Their own work or organisation |
| Not checkable |
Not checkable |
3 Apr 2025 |
I think it’s either just a multiplication problem where 20% of people are good at one thing, 20% of people are good at another thing, and…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
This may be in some sense a surveillance state, in the sense that there is some AI that knows what’s going on everywhere, but that AI then…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think I do more research, deep dives, read more books than I would if I weren’t a blogger.
Not checkable · 3 Apr 2025 |
|