| Not checkable |
Not checkable |
Feb 2023 |
You can get more people aware that we're walking directly into the whirling razor blades, because even if only 10% of the people can get…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
When you use the hill-climbing paradigm and optimize for one single extremely pure thing, this is how much of it gets inside.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
If natural selection had been a foresightful, intelligent kind of engineer that was able to engineer things successfully, it would have…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
but they do burn any money literally at all.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
And I think that the actual catastrophe scenario for AI looks like big advancement in a research lab, maybe driven by them getting a giant…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
I expect that's much easier for some alien species than others.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
they're making up arguments rather than deeply considering things that are held to any standard of rigor
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
Its actions are the most efficient actions for accomplishing its goals. If you think you see a better way to accomplish its goals, you're…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
I mean, I could say something like shut down all the large GPU clusters.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
And a couple of days later, everybody on earth falls over dead in the same second.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
This is a *lot* harder than nuclear weapons.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
And 50 years later, you'd have an aligned AGI.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
An AI is like that, but instead of needing to refine plutonium, you can make nuclear weapons out of a billion tons of laundry detergent.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
If I have buy-in from a major government that can run actual security precautions and more than just $10 billion, then you could run a…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
How the hell would I know?
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
because they're too far away
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
ChatGPT is a commercialization of the actual AI-in-the-lab giant leap forward.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
This is not what I actually predict. I think that we are hearing the last winds start to blow, the fabric of reality start to fray.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
There's no way you can make it easy to align by making it stupid.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
Well, no. You can run ChatGPT any number of times without igniting the atmosphere.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
So ChatGPT is less general than a human, but it's genuinely ambiguous, I think, whether it's more or less general than (say) our cousins…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
I don't have really good answers. It depends on what your talents are.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
You're probably wrong.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
The reason you're not in its utility function is that the programmers did not know how to do that.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
I sure hope that this is where it saturates — this or the next generation, it goes only this far, it goes no further.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
And we're not even that good at learning from relatively minor disasters!
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
maybe you work with Redwood Research or something.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
You're just doing a pretense of research that you can show off to a funding agency.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
So the *efficient market hypothesis* is of course not generally true.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
But I think that probably some of the vast quantities of money being blindly and helplessly piled into here are going to end up actually…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
The saner outfits do have uses for money.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
ChatGPT itself? Zero. It's not smart enough to do anything really wrong. Or really right either, for that matter.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
They have not run into the actual problems of alignment.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
There is not a very good way to simplify the argument, any more than there is to simplify the notion of zero-knowledge proofs.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
Back in the day, we had more time. Humanity squandered it.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
I would give myself and MIRI credit for why the world isn't just walking blindly into the whirling razor blades here, but it's not clear to…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
It's not that they don't exist in principle.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
And then the rocket ignites the atmosphere, is the problem there.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
I mean, every organ in your body is a reproductive organ.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
Humans are not *fully* general. If humans were fully general, we'd be as good at coding as we are at football, throwing things, or running.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
And then, gradient descent, probably may be even worse.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
"Everybody falls over dead" just doesn't inspire the monkey political parts of our brain somehow.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
I think that the current systems are actually very weak.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
at a billion dollars, I might try to bribe people to move out of AI development, that gets broadcast to the whole world
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
Well, unless it's limited, narrow superintelligence, I think you mostly don't get to observe it because you are dead, unfortunately.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
It doesn't hate you.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
I think the state of affairs is we have eminent scientific voices making fun of this possibility, but not engaging with the arguments for…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
I hope they can't fix it. I hope we get, like, a 10-year AI winter after this.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
If you can think of an alternative to the giant inscrutable matrices, then, you know, don't tell the world about that.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
A superintelligence is something that can beat any human and the entire human civilization at all the cognitive tasks.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
You know, Marvin Minsky, John McCarthy back in 1955, they weren't idiots.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
there is no physical law, computational law, mathematical/logical law, saying when you optimize using hill-climbing on a very simple, very…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
It looks like me being wrong.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
I did not see ChatGPT coming, I did not see Stable Diffusion coming, I did not expect that we would have AIs smoking humans in rap battles…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
I can predict it will be in the class of states that are winning positions for black or white or whichever color Stockfish picked, because…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
And if you got to try over and over again, you could find it.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
I'm not going to say that it was luck, because the calculations were actually pretty solid.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
We have not done those things that it would take to earn a good outcome, and this is not a case where you get a good outcome by accident.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
This thing alone cannot end the world.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
It's not that the youngsters are stupid.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
The AI doesn't hate you, neither does it love you, and you're made of atoms that it can use for something else.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
Maybe there's ones in universes with alternate physics where you just can't do that.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
How the heck would I know?
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
If it's smart, it doesn't announce itself.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
But, you know, it's hard to see what's coming in advance of experimental evidence hitting you over the head with it.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
And that's similarly where I'm getting the prediction about everybody being dead, because if everybody were alive, then there'd be some…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
And MIRI has that property.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
That's the disaster scenario if it's as smart as I am.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
We could figure out how to align AI in 50 years given unlimited retries.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
It's surprising if the thing that you're wrong about causes the rocket to go twice as high on half the fuel you thought was required and be…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
If you're doing research that you know will work, you're probably not really doing any research.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
Some of them might go into breaking AI systems instead, because that's where you learn anything.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
Any fool can build a crypto[graphy] system that they think will work.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
we lost a million people and we sure did not learn very much as far as I can tell for next time.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
Problem is that the world got destroyed in the meanwhile. And that's why, you know, that's the problem there.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
So, every now and then, somebody figures out how to put a new prompt into ChatGPT.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
if you don't have this very deep form of the security mindset, you will end up producing fake research and doing more harm than good
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
I'm not sure if they can scale using more money
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
Oh, well, supposing that they got killed by their own AIs doesn't help at all with that because then we'd see the AIs.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
That's an irreducible source of uncertainty with respect to superintelligence, or anything that's smarter than you.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
ChatGPT shows a very wide range of generality compared to the previous generations of AI.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
I would say probably Redwood Research has that property.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
We have sex, but we wear condoms.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
Redwood Research being the only ones I really trust at this point, but they're tiny
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
We try to be real.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
I don't really have a good organized answer to how in general somebody goes down fighting with dignity.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
But then what do they do? I don't know. Maybe they'll be able to do something later.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
Can you get all the people? Can you get all the politicians?
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
the space of minds is *very wide*.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
This is merely conventional science and merely the conventional understanding of the world.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
You cannot predict exactly what move it would make, because in order to predict exactly what move it would make, you would have to be at…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
Now, this does intrinsically take a powerful AI.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
It doesn't matter who gets it first. Everybody falls over dead.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
so I would not tell all the successful cryptocurrency people to run off and start their own research outfits.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
We are not built to want ice cream.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
put them all on a large island, and try to figure out some system for filtering the stuff through to me to give thumbs up or thumbs down on…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
I'll keep on fighting until the end, which I wouldn't do if I had literally zero hope.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
Caring is easy to fake.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
the chance of them producing a *secure operating system* is effectively zero.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
But if it's better at you than everything, it's better at you than building AIs. That snowballs.
Not checkable · Feb 2023 |
|