| Not checkable |
Not checkable |
3 Apr 2025 |
maybe my answer is there should be mainstream media. I hate to admit that, but maybe it’s true.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
But having some ridiculous amount of money put in as an incentive got like three extra people.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
There were so many good posters on Livejournal before it got taken over by Russia.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I don’t have a great explanation.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I don’t think we’re saying anything that there isn’t models for which have previously seemed to work for long historical periods.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And we don’t have a great prediction for exactly how this is going to end.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
That’s like my best leading indicator for who’s going to be a good blogger.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I’m not sure. The Europeans failed at this task of, if you put a single European in Australia, do not starve.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Although it can’t be that much of the explanation because I feel like now all of those people have gotten substacks and some of those…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
But just as people with different views of the Constitution have managed to get it into a shape that probably the Founders would not have…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
But it also seems like our community in the sense of people who are thinking about the same ideas, people who care about AI economics…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Again, assuming things go incredibly well, we’ve avoided doom, we’ve avoided having some psychopath in power who doesn’t care at all.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So when I think of the ways that they train AIs, I think in our scenario at this point there are two primary ways that they’re doing it.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
But it’s not necessarily my mainline prediction.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I don’t think there are many hidden gems.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think it’s either just a multiplication problem where 20% of people are good at one thing, 20% of people are good at another thing, and…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
This may be in some sense a surveillance state, in the sense that there is some AI that knows what’s going on everywhere, but that AI then…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think I do more research, deep dives, read more books than I would if I weren’t a blogger.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And I get lots of praise when I read a book and often lots of money, and that’s a really good incentive.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And I think right now if we tried it, we would run into the combinatorial explosion.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think it’s actually better at the style on a word-to-word, sentence-to-sentence level, than it is at planning out a blog post.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think one thing that you didn’t mention in there was the data efficiency.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
But imagine that you have to make a business out of you and your hundred closest friends who you agree with on everything.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And the way that the AI intelligence explosion dynamics work are just so weird that in order to have nothing happen, you need to have a lot…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think it might be a crazy outcome to make a lot of money as a blogger.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
But nothing we’re saying is that different from what has already happened.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
First of all, I agree if there’s only one way to do something that makes it much harder, and maybe that one way takes very long, we’re…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I’m not completely convinced that we don’t get something like alignment by default.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I kind of want to call you out on the claim that groups of humans don’t plot against other groups of humans.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
One is that they’re too stupid to understand their training.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And I think that that has been the most consistently wrong prediction of all.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
There’s also cases where it’s very obvious who’s part of what group.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
The literal Cambrian explosion, the agricultural revolution, the industrial revolution, has phase changes.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I don’t buy that as a real limiting factor for most people.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think with race, it’s very easy to know whether you’re black or white, and so there have been many cases of one race kind of conspiring…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And the answer is: combination of that speed up and the speed up in serial thought speed.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And I’m going to say we all believe it, although I can imagine people getting really conspiratorial about it and this not working.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
It just seems like applying those two things.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And then the example I gave was that humans also can’t do this.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Once we can really understand the weights of an AI on a deep level, then we have a lot of alignment techniques open up to us.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
It’s probably the weirdest couple of years that have ever been.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Kind of looks like you asked ChatGPT to summarize the past five years of AI progress, and you got something with a couple of…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And also you can correct people when they’re wrong, which is a very important side benefit.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
One of them is just continuing the next token prediction work.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So for me, the most salient example is the etymology of words.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
But we know the people involved, we know who’s in the government, we know who’s leading all of these labs.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
It seems like he’s a genius, he’s thought about this for years, probably at some point, he heard a couple of Indian words and a couple of…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
This is actually part of our misalignment story, is that if the AI is sufficiently misaligned, then yes, we can tell it it has to follow…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So a regulation saying there has to be whistleblower protection.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Like, I think in order to have nothing ever happen, you actually need a lot to happen.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So one of these models is just a hyperbola. Everything is along the same curve.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
We could have a different copy of the superintelligence optimizing every single part full-time.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think it is plausible to imagine a future society that works the same way.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Everything we’ve talked about has happened before.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
But I do think that we have a pretty good chance of surviving.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
You get more lottery tickets for these discoveries when you are intelligent, when you have good technology, when you’re doing good work.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
If you actually want to understand an issue in depth, you can’t use deep research. You gotta do it on your own.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I am also worried about UBI from a different perspective.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Right now, I just write about it. I think writing about it is important, but I don’t know.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Yeah, I think it might be useful to think of our timelines as being like 2070, 2100.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I admit that my p(doom) is literally just p(doom) and not p(doom or oligarchy).
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And so after dozens of viral hits, then you have a fan base.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
Would all of those journalists have become bloggers if there was no such thing as mainstream media? I’m not sure.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And like I feel like the company leaders have so far made slightly better noises about caring about alignment than the government leaders…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And if you look at why this happens, why this doesn’t happen, it tends to happen in cases where one group has an overwhelming advantage.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
But just running them through these environments again and again, training on the successes, training against the failures, kind of…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
When it is one smooth process, the smooth process is often a hyperbola that shoots to infinity in weird ways.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I have always wanted to get more involved in the actual attempt to make AI go well.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
To me it does not seem like the same kind of problem as inventing all human institutions from the Paleolithic onward.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
There are lots of ways that cheating and doing bad things can improve your success.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think of superintelligence as partly when they become so much more data efficient than humans that they are able to build on cultural…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I feel like if it’s literally the superintelligent AI that is never wrong, then we have solved some of these coordination problems.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think that’s just a really big speed up.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So these AIs will have access to all human knowledge, they will have read management books in some sense, they’re not starting blind.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
My guess is it’s a lot worse than for startup founders.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think I was the most boring normie liberal in the world before encountering LessWrong.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
So like we said before, I think there’s a big difference between discontinuous and very fast.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And if it says like, “by the way, try to manipulate the government into doing this or that”, then we know that something bad has happened…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I realize that’s a much weaker thing than we’re positing, but it was done just with the human brain with a lot fewer resources than we’re…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think if you were to try this, you would get something that was so obviously an AI house style that it would use the word ‘delve’ or…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
that team of ethnobotanists would succeed in fewer than 50,000 years.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think if you wrote a really stupid blog post, it could point out the correct objections to it.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I’m a little bit concerned that AI will make it much easier to break anonymity.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think you can either be the sort of person who can go on a Dwarkesh podcast and get the New York Times to write about you, or you can do…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think this is just not that hard a problem.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think you could think of this as like a chess engine.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
And the way to get the President on their side is to really play up the national security implications.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
It’s interesting because like a lot of areas of life are selected for arrogant people who don’t know their own weaknesses because they’re…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think that it’s a question of what is your default option or what are you comparing it to.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I think that’s kind of why all of our policy prescriptions are things like more transparency, get more people involved, try to have lots of…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
That’s why I say that maybe the limiting factor for so many of these people is courage because everybody I talk to who blogs is within 1%…
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
I feel like Larry Summers, the President would just say “I don’t trust him”.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
3 Apr 2025 |
There is going to be something like: predict how Bill Gates would complete this next character or something like that.
Not checkable · 3 Apr 2025 |
|
| Not checkable |
Not checkable |
10 Dec 2024 |
Why is it so hard to fill these positions?
Not checkable · 10 Dec 2024 |
|
| Not checkable |
Not checkable |
10 Dec 2024 |
I previously believed this, but this post makes me more skeptical.
Not checkable · 10 Dec 2024 |
|
| Not checkable |
Not checkable |
10 Dec 2024 |
My uneasy synthesis is that criminals have nominated themselves for the short end of a tradeoff.
Not checkable · 10 Dec 2024 |
|
| Not checkable |
Not checkable |
10 Dec 2024 |
You laugh, but this is exactly how our health care system works.
Not checkable · 10 Dec 2024 |
|
| Not checkable |
Not checkable |
10 Dec 2024 |
Everyone was right to criticize me, because I have a confused and muddled middle position here.
Not checkable · 10 Dec 2024 |
|
| Not checkable |
Not checkable |
10 Dec 2024 |
I think if first shoplifting arrest is a warning, this would be a good way of dealing with the second and third, and then after the Xth…
Not checkable · 10 Dec 2024 |
|
| Not checkable |
Not checkable |
10 Dec 2024 |
But we shouldn’t weigh criminals’ at zero, and we definitely shouldn’t punish them just for fun.
Not checkable · 10 Dec 2024 |
|