| Not checkable |
Not checkable |
19 Oct 2026 |
I may have more of these in the upcoming week.
Not checkable · 19 Oct 2026 |
|
| Not checkable |
Not checkable |
19 Oct 2026 |
Still trying to figure out if we can do something for the Bay Area
Not checkable · 19 Oct 2026 |
|
| Not checkable |
Not checkable |
19 Oct 2026 |
Check it out, and install the browser extension, here.
Not checkable · 19 Oct 2026 |
|
| Not checkable |
Not checkable |
19 Oct 2026 |
I would once again like people who are well-versed in the AI/grantmaking worlds to take a survey on how much they value various things.
Not checkable · 19 Oct 2026 |
|
| Not checkable |
Not checkable |
19 Oct 2026 |
This is the weekly visible open thread.
Not checkable · 19 Oct 2026 |
|
| Not checkable |
Not checkable |
5 Oct 2026 |
I’m pretty busy and will try to make up for it after that changes.
Not checkable · 5 Oct 2026 |
|
| Not checkable |
Not checkable |
5 Oct 2026 |
This is the weekly visible open thread.
Not checkable · 5 Oct 2026 |
|
| Not checkable |
Not checkable |
5 Oct 2026 |
And thanks again to Ana for handling the parts that required Portuguese profiency.
Not checkable · 5 Oct 2026 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
We know that in general, the median outcome for a forecast like this is being totally humiliated when everything goes completely…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
To me, the differences between an autocratic centralized three-letter agency and an autocratic centralized corporation aren’t that exciting…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I do think courage has to be some part of the explanation.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
But I think that our policy prescription is let’s try as hard as we can to make it happen.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Humans have such good heuristics that probably most of the things that show up even in our conscious mind, rather than happening on the…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I don’t understand why they can write good magazine articles without being good bloggers.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And the limiting factor is basically hours in Elon’s day in the sense that he cannot be doing that with everybody’s.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
If you have a physics simulation available, then all of these real world bottlenecks don’t matter as much.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
If people have questions about quantum computing, they ask Scott Aronson or he is like the authority.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
You tend to get more of a diffusion of power democracy where there are many different groups and none of them can really act on their own.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
We’re not sure about that, actually.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
If you can create a country of geniuses in a data center, as I think Dario Amodei put it, then you no longer have this population bottleneck
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And I think that once a thousand of these natural consequences of training add up, the AI is evil, in the same way that once the AI can do…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
If our timeline was 2040, then I would have these broad heuristics about is government good? Is private industry good? Things like this.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
The problem isn’t that they are dumber than the Aborigines exactly, it’s that the Aborigines have a vast head start.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And here not only is society inherently chaotic, but there are so many things that we could be leaving out.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
In another phase we may make policy suggestions, and one of the ones that Daniel has talked about that makes a lot of sense to me is to…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think, read the document and decide which of us is right.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
They already understand that superintelligence is kind of the key to power in the future.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And I think that’s like the closest thing we have to a neutral aggregator where we’re not cherry picking things.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So that 80% of scenarios where we survive contains a lot of really bad things that I’m not happy about.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
But only like 1% of these people who are good at short and medium form ever go to long form.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I do just think it’s true that the good blogging space is undersupplied and there is a strong power law.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I absolutely reject the premise that everybody has always been too optimistic.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I see the AI is lying, and it’s obviously just an artifact of the training process.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
they actually have a huge amount of things to say.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
We have something which is smarter than Elon Musk, better at optimizing things than Elon Musk.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I only like certain bloggers, there are many people who I’m sure are great that I don’t like.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Part of what they want is security, because they know that if China steals some of this and they get these superhuman hackers, and part of…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Yeah, we don’t know how this will work at the limit of all these different training methods, but we’re also not completely making this up.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I mean there are probably hundreds of other professors who do quantum computing things but nobody knows who they are because they don’t…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And there my answer is just, “I don’t know. Let’s ask the super intelligent AI oracle. Maybe it has good ideas”.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Your blog posts are actually very good.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Yeah.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I can also count on a pretty small number of figures the number of people who had great blog posts but weren’t that prolific.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I do think it’s possible to unbundle liberalism in this sense.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think with blogs and I mean this is self-serving, maybe I’m an arrogant person, but that doesn’t seem to be the case.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
There are many, many times more people who have written good books than who are actively operating great bloggers right now, I think.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Works in Progress is a great magazine.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
we’re trying to take almost in some sense a conservative position where the trends don’t change, nobody does an insane thing, nothing that…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Partly you get this by bumbling about and having accidents and some of those accidents are productive.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think there are two reasons why they might fail to do what you want, kind of reflecting how they’re trained.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
This has been my experience working with the team, as I point out, five different things.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think that 90% of people who complain that they don’t have ideas, I think actually have enough ideas.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I don’t think you would use one of the human ones because you would want something that was better suited for this task.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
You have things like kin selection that are kind of exceptions to that, but overall it’s the rule.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Also, they do have the advantage of all the cultural technology that humans have evolved so far.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I say that I very rarely see new bloggers who are great.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I don’t really know what we’re asking of ASI that’s equivalent to competing against someone with a 50,000 year head start.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
The other thing is that blogs are such a great status gain strategy.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
This may not be perfectly suited to them, it’s more suited to humans.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Do it every day, same advice as for everything else.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
The other is that you were too stupid to train them correctly and they understood what you were doing exactly, but you messed it up.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
This is something that needs to get out there more.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I am the only person on the team who is not a genius forecaster.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think we agree.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
If you are the planner and you’re trying to do this, first of all you go to the bottlenecks that are preventing you from doing anything…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
but if I learn that Tulsi Gabbard has a LessWrong alt with 10,000 karma, maybe I want the national security states.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I know this sounds crazy because if you read our document, all sorts of bizarre things happen.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So from the point of view of anyone in most of history, we are going at a blindingly insane pace.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
This is also maybe getting too far into the godlike thing and too far away from the human concept handles.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think there’s more opportunity for pliability there.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Once a year is a crazy answer in some sense, like it ought to be more.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
What other career do you need seven years of positive feedback before you apply for your first position?
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
We would need better heuristics.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I still feel really reluctant to speculate.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
There’s actually a really interesting foretaste of this.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And I just don’t think we’ve even tried that.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think that naively people think like, well, every particular thing is potentially wrong.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I don’t really have a great sense of whether we get that before or after the AI has become completely uncontrollable.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
What I didn’t realize was that I also learned a huge amount.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And it just made it so much more concrete to have a specific story.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think a lot of blogging is reactive; You read other people’s blogs and you’re like, no, that person is totally wrong.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Plus something like courage, where people who would be good at writing blogs don’t want to do it.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
My dream way of pushing back against that is kind of giving people the tools to push back against it themselves, seeing what they come up…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Partly that’s because just Elon is crazy and never sleeps.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Partly they do this just because they have higher serial speed.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
But number one, I think we rely a lot in our scenario on this idea of research taste.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Books are the worst possible financial strategy. Substack is where it’s at.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So I owe a huge debt of gratitude to Eliezer Yudkowski.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think one of the reasons I’m more hopeful than I used to be is that LLMs are great compared to the kind of reinforcement learning…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Then you look at the century between 1900 and 2000 and it’s just completely qualitatively different.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
It’s Daniel’s estimate, it’s not my median estimate, it’s an estimate I think is extremely plausible that we should be prepared for.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I am pretty agnostic as to exactly what happens.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So if you’re producing a million of something, you’re probably getting very, very good at it.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And so the training process of evolution kind of yokes them to each other in these extremely powerful bureaucracies.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So to me, it doesn’t seem like he is just sitting there being logically omniscient and getting the answer.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So this isn’t about having all the information in your memory so much as the normal process of discovery, which is kind of mysterious, but…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So I think the first one is kind of what we’re coming out of.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Obviously we would get an arms race with China in that situation.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I can easily write a blog post, like a normal length ACX blog post, but if you ask me to write a novella or something that’s four times the…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So we kind of want more things like that to happen.
Not checkable · 3 Apr 2025 |
|