| Not checkable |
Not checkable |
6 Apr 2023 |
And you don't ever want to have chronic fatigue syndrome on your medical record because that just tells doctors to give up on you.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
We’re guessing much more blindly here.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
But from my perspective, the future is just really hard to predict.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I agree it’s possible to imagine things being even worse.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I don’t think I’m unusual in looking around myself in that highly multidimensional space and not finding a ton of neighbors ready to take…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
No, they are not.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
The alignment field failed to gel.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
There’s an established preference for people going to some lengths to make their kids healthier, not necessarily via the options that they…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Usually when you’re trying to build a rocket and your model of rockets is lousy, it doesn’t cause the rocket to launch using half the fuel…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Humanity hasn’t really existed for very long.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
But as soon as your sentence has more than like five words in it, its probability has dropped to basically zero because of all the extra…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
We saw these people get caught utterly flat-footed on the Internet.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Don’t have that be the plan. That does not sound like a good plan.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I’m worried that they’re pretty distant.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I mean you definitely want to use the alien observation of 10,000 planets like this one prior for what you get after training on, like…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Because as you get smarter, you get new options that are further from the options that you are faced with in the ancestral environment that…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It makes everything a lot more grim.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
These are also quite dangerous things, but they do not have the utter lethality of artificial intelligence.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
We just suck at alignment, and that’s why our stuff blew up.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I think that’s about the level where a human could start to work on alignment, more or less.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
But I think the smarter they are, the more likely they are to go for it if it’s credible.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Let’s offer $100 billion in prizes for Interpretability.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I think that a lot of the people who want kids would go for this new offer that just offers them so much more of what it is they want from…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
There’s no established preference for four eyes.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Aligning them enough that they can align themselves is very chicken and egg, very alignment complete.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I think you’re wrong.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It depends on the exit plan.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
They’re made out of all these complicated desires.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
The hinge of this argument is the capabilities constraint.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It’s on the podcast, it goes on the Internet.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
The jump to superintelligence does strike me as being significant in the same way as the first self-replicator.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It violates the laws of physics.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Because we haven’t yet had options as far enough outside of the ancestral distribution that in the course of choosing what we most want…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
But you would. You just look down at your fellow humans. You have no confidence in their ability to tolerate weirdness, even if they can.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Your entire analogy fails.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
So being very unsure about the future looks like predicting with probability nearly one that the humans are all gone
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It’s hard. It is just easier to predict the endpoint than it is to predict the path.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I don’t think alignment is actually so easy that you just get whatever you want.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
You’re saying all lottery numbers are equally likely instead of winning and losing are equally likely.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
This is not an IQ 100 AI we’re talking about here.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Reality is just quite large.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
And the smarter you are, the more out of distribution you get.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It is very clearly understood which systems spit out low quantities of gold and the qualitatively different systems that don’t actually…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I think it’s much easier to get a mask like that than to get a shoggoth like that.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Trying to play this game against things smarter than you is a fool’s game.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It is always going to be at an early stage relative to the super intelligence that can actually kill you.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
No, no. What I’m saying is that as smart as the people it’s pretending to be are, it’s got planning that powerful inside the system…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
is that the existence of this technical array of hope, if you do just the right things, is not the same as expecting that the world…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I’m here on podcasts instead of working on them, aren’t I?
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Look at the thing with ice cream, look at the thing with condoms, look at the thing with pornography, see where this is going.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
But my point is to get to like 90% chance of winning, which is pretty hard on any exit scheme, you want a fast exit scheme.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
This kind of happened with humans. Well, life generally.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Weird is relative to intelligence. The smarter you are, the more you can move around in the space of abstractions and not have things seem…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
So large language models will change their preferences as they get smarter.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
What? We get a black box output, then we get another black box output.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I’ve seen no evidence, but who knows what the alternate Everett branches of Earth are like?
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I’m not trying to violate the laws of physics, even as you probably know them.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
And this is very much not the situation we have with nuclear weapons.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
All kinds of interesting things started happening with GPT 3.5 and GPT-4 that were not in GPT-3.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
If you try to make people who are inhumanly nice, who are nicer than anyone has ever been before, you’re going outside the space that human…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
What will the future be? I don’t know. It could be paperclips, it could be staples.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It doesn’t have to be a very sharp threshold
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
So magic in the sense is I can tell you exactly what I’m going to do.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
So the problem I see is that your typical human has a great deal of trouble telling whether I or Paul Christiano is making more sense.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The most success I’ve had so far is, well, imagine that the humans are running at very high speeds compared to very slow aliens.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Oh, we are so far behind on interpretability compared to capabilities.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I’m not actually an expert on that. And the experts don’t know either cause it’s kind of hard to predict the future.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It’s all the things that I value about it and maybe all the things that I would value if I understood it better.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It’s not smart enough to prevent you from pressing the off switch and probably not smart enough to want to prevent you from pressing the…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
And it’s replacement by nothing interesting or worthwhile, even from a very cosmopolitan perspective on what counts as worthwhile.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
If I am wrong and there is that kind of public outcry, then somebody in high school could be ready to be part of that.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think the human alignment problem is a terrible phrase because it is very, very different to like try to build systems out of humans
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think you’re kind of like projecting the aliens being like humans, and like thinking of a human in a human society rather than a human in…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I mean, there’s the question of to what extent it is thereby being made more human-like versus to what extent an alien actress is learning…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think even GPT-2 probably has like very stupid fragments of alien actress in it.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Reverse stupidity is not intelligence.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I would say, find prediction markets that matter to you and bet in the prediction markets.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Because, no, we are not in the shape to frantically at the last minute do decades’ worth of work.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think one should only be satisfied with solving the entire problem.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
And if you have something that is generally smarter than a human, it’s probably also generally smarter at building AI systems.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think the mask is a slice through you. It may even be the slice that’s in charge of you.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
And indeed, something smarter than us would also be much smarter than natural selection, so it doesn’t just automatically carry over.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think they could be answered in 50 years with unlimited retries, the way things usually work in science.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It’s definitely a big leap from GPT-3.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think that for there to be a person who I care about looking out at the universe and wondering at it and appreciating it, it’s not enough…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It’s smarter than natural selection, stupider than the upper bound.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
So there’s an upper bound, but it’s very, very, very far up above here.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The whole notion of open sourcing, this was always the wrong approach, the wrong ideal.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It has changed my understanding of what kind of work can be performed by which kind of processes and by which means.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
And if you train an AI system to make people press thumbs up, maybe you get these long, elaborate, impressive papers arguing for things…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
You do not need life to be finite to be meaningful. It just has to be life.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
If you can’t verify it, you can’t ask the AI for it.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I do worry that like most of the aliens who are like humans or like a modern human civilization, I kind of worry that the super vast…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think that the first AIs you’re trying to build, not a million years later, the first ones look more like narrowly specialized biologists
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Subgenic fitness is the implicit loss function of evolution, which it cannot change.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The thing you would do at this point if there were massive public outcry pointed in the right direction, which I do not expect, is shut…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Are Earth’s billionaires going to put up the giant prizes that would maybe incentivize young hotshot people who just got their physics…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The first time you’re building a rocket, being wrong is, in a certain sense, quite easy.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The thing that it ends up wanting most is a thing that even from a very embracing cosmopolitan perspective, we think of as having no value.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Well, how smart is it?
Not checkable · 30 Mar 2023 |
|