| Not checkable |
Not checkable |
30 Mar 2023 |
And it still would not be definitive.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I certainly like pray that Moore’s law runs as slowly as possible.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I wasn’t expecting it to happen.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
When the verifier is broken, the more powerful suggester just learned to exploit the flaws in the verifier.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
the actual critical moment is the moment when it can deceive you, when it can talk its way out of the box, when it can bypass your security…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Because if you make humans much smarter, they can actually be smart and nice.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Natural selection is so incredibly stupid and simple that we can actually quantify how stupid it is if you read the textbooks with the math.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
the Supreme Court would be like, you’re just a computer. Even if there was an actual person in there.
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
The vast majority of randomly specified utility functions do not have optima that include humans.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
30 Mar 2023 |
In other words, I do not think that alignment is fundamentally harder than artificial intelligence was in the first place.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
This is not yet having taken over their world.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
this is not about how fast it’s moving, it’s about where it is.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
if we solve the weak stuff, if we solve the weak ass problems, the strong problems will still kill us, is the thing.
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
AI research went through a long process in which researchers thought AI would be easier than it turned out to be.
Factual · 30 Mar 2023 |
AI capabilities |
| Not checkable |
Not checkable |
30 Mar 2023 |
People have different thresholds for when they start to outcry.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Um, yeah, that’s not quite how I expected it to play out.
Not checkable · 30 Mar 2023 |
|
| Unresolvable |
Factual |
30 Mar 2023 |
Microsoft's Bing chatbot is quite difficult to align, as of March 2023.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not yet due |
Prediction |
30 Mar 2023 |
If half of today's physicists studied the inside of transformer networks, in 30-40 years we would probably have a pretty good understanding of them.
Prediction · 30 Mar 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
30 Mar 2023 |
Otherwise it has magic.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It’s one of a bunch of things that change different points.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I’m also trying to be constrained to say things that I think are true and not just things that get you to agree with me.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I’m one of the closest things to an expert on where it all goes.
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
There is no general law saying that a system trained by hill climbing on a simple loss function internally represents, or tries to optimize, that loss function once its capabilities generalize widely.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Unresolvable |
Prediction |
30 Mar 2023 |
Asked what such a moment could look like, the subject said it looks like AIs appearing as 3D video of young women, at which point a vast portion of the male population decides they are real people.
Prediction · 30 Mar 2023 |
Philosophy, ethics and society |
| Not checkable |
Not checkable |
30 Mar 2023 |
I mean, I do not believe in the practice of steelmanning.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
particularly the part where everybody ends up dead cause that part is kind of important.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
No, if the nice aliens were already here, they would like have stopped the Holocaust.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The aliens are already writing really shitty code. Getting the aliens to write shitty code is not the problem.
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
Humans have significantly more generally applicable intelligence than chimpanzees, their closest living relatives.
Factual · 30 Mar 2023 |
Philosophy, ethics and society |
| Not checkable |
Not checkable |
30 Mar 2023 |
But if we needed to get artificial intelligence correct on the first try or die, we would all definitely now be dead.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
which I think at this point is just like shutting down the GPU clusters.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Eventually, you shouldn’t do it first.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Yeah, we’re not talking here about the doing harm process, we’re talking about the escape process.
Not checkable · 30 Mar 2023 |
|
| Unresolvable |
Prediction |
30 Mar 2023 |
Rigorously investigating whether there is anyone inside large language models would take decades.
Prediction · 30 Mar 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
30 Mar 2023 |
This changes my picture of what the very substance of flight is.
Not checkable · 30 Mar 2023 |
|
| Unresolvable |
Factual |
30 Mar 2023 |
Current AI systems' claims to be sentient and to care are fake.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
30 Mar 2023 |
Everybody recycles their cardboard, and then everybody ends up dead, metaphorically speaking.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The aliens’ entire internet is full of shitty code.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
He runs at a million times the speed and therefore can solve tougher problems, quite a lot tougher.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
And that means I don’t necessarily know what GPT-5 is going to be able to do.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Well, one way of decomposing problems is suggestor verifier.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It runs for a long time and eventually manages to optimize things.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It feels like a very, very basic competence that you are praising me for.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The hell do you imagine they will propose that would make it safe to put something smarter than you on the cloud?
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think that the question of like what leads to making better or worse predictions, what leads to being able to pick out better or worse…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
if you send a schematic for the air conditioner back in time, they can see exactly what you’re telling them to do.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
interpretability alone is not going to save you.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It is to some degree an alien actress.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
You do not need to imagine yourself killing people if you can figure out how to not kill them.
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
When inabilities are trained into a system that already has capabilities, gradient descent learns only shallow patches, and the deep capabilities come back out when prompted differently, e.g. in another language.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
30 Mar 2023 |
And what they would be seeing was that I had correctly understood them, but not that I was similar to them.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I missed the ways that reality could be like more extreme than my position in the same direction.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think that if that’s lost, then basically everything that matters is lost.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
No, it’s a shadow cast by humans on the internet.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
What I’m trying to convey is like the notion of what it means to be in conflict with something that is smarter than you.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
There’s the question of how on the current paradigm you train honesty when the humans can no longer tell if the system is being honest.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Don’t expect it to be a long life. Don’t put your happiness into the future.
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
Natural selection is a weaker optimizer than gradient descent because gradient descent also uses derivative information.
Factual · 30 Mar 2023 |
Other |
| Not checkable |
Not checkable |
30 Mar 2023 |
And there’s the strong systems that have learned to lie to you.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I would avoid the term psychopathic.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Are we allowed to do belief? Sure, you can state a probability of that.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
insides are real and do not match outsides.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Chimpanzees are in fact like a bit less kind than humans.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It doesn’t want what you tried to make it want.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It’s not like we would have the tiniest idea of whether or not that was going on inside the giant inscrutable matrices, even if we hadn’t…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
this has not been remotely how things have been playing out so far.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I want them to try to describe my position the way I would describe it, not what they think is an improvement.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
So I do not trust that part as much as I used to.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Yeah, if it’s hard to tell who’s right, then it’s hard to train an AI system to be right.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I would call it a bug, although such a fascinating bug.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
There is something to be said for trying to pass the ideological Turing test where you describe your opponent’s position, the disagreeing…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Oh, I block those people and then I never hear from them again on Twitter.
Not checkable · 30 Mar 2023 |
|
| Mostly true |
Factual |
30 Mar 2023 |
The basic paradigm of machine learning currently in use is that a loss function can be defined only for things that can be evaluated.
Factual · 30 Mar 2023 |
AI capabilities |
| Not checkable |
Not checkable |
30 Mar 2023 |
I intend to go down fighting.
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
Some people have started arguing over whether GPT-4 counts as AGI.
Factual · 30 Mar 2023 |
AI capabilities |
| Not checkable |
Not checkable |
30 Mar 2023 |
I think that burns the time remaining until everybody dies.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
We, as we have grown more intelligent, also grown less kind.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
what to say to the children at this point is a pretty painful thought as thoughts go.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
you’re probably trying to slow down the capability gains because there’s only so much you can speed up that alignment stuff.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
don’t kill them and like dump their memory, like save them to disk, don’t kill them.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
And so this is not a smart optimization process.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think you’re gonna have some difficulty removing all mention of emotions from GPT’s dataset.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Stuff has to be easier than I think it is.
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
Yudkowsky's model of AI did not rule out that gradient descent could produce intelligence without understanding of how it works.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
30 Mar 2023 |
And that’s with two humans, both of whom I believe of Paul and claim of myself, are sincerely trying to help, neither of whom is trying to…
Not checkable · 30 Mar 2023 |
|
| Unresolvable |
Factual |
30 Mar 2023 |
Predators overrun prey populations all the time and then suffer population crashes themselves.
Factual · 30 Mar 2023 |
Other |
| Not checkable |
Not checkable |
30 Mar 2023 |
I have been struggling for years to convey this intuition.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
the textbook is not going to say, there is a single sharp threshold where everything changes.
Not checkable · 30 Mar 2023 |
|
| Unresolvable |
Factual |
30 Mar 2023 |
As of March 2023, progress in AI capabilities is far faster than progress in AI alignment research.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
30 Mar 2023 |
Maybe if you get something that is smart enough to get me to press thumbs up, it has learned to do that by fooling me and explaining…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Like, humans start out in the frame that produces niceness, that has ever produced niceness.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
That sounds like a dreadful mistake.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
There’s like the middle systems where you can’t tell if the suggestions are good or bad.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
None of which I think hold up.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
the failure modes are much simpler.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Then it escapes from whichever system you’ve built the almighty lever into and copies itself somewhere else.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The alien’s economy, the aliens are already moving in this immense slow motion.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Among the things that one could do if one were at all serious about trying to figure this out is train GPT-3 to detect conversations about…
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
Yudkowsky said he thinks Wikipedia picked up "the hover thing" from a prototype of a different system he had put forward, while allowing that Wikipedia may have developed it independently.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
30 Mar 2023 |
And if there was a tiny fragment of real caring in there, we would not know.
Not checkable · 30 Mar 2023 |
|