| Not checkable |
Not checkable |
30 Mar 2023 |
And indeed, something smarter than us would also be much smarter than natural selection, so it doesn’t just automatically carry over.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think they could be answered in 50 years with unlimited retries, the way things usually work in science.
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
Humans develop emotions even if they are not told about emotions as children.
Factual · 30 Mar 2023 |
Philosophy, ethics and society |
| Not researched |
Factual |
30 Mar 2023 |
Yudkowsky said 'tiny molecular paperclips', which was later altered to 'paperclips' and then to an AI in a paperclip factory.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Unresolvable |
Factual |
30 Mar 2023 |
Almost all people (999 of 1,000) would think a person claiming an AI is sentient is mistaken, which is almost surely correct though not known.
Factual · 30 Mar 2023 |
Philosophy, ethics and society |
| Not checkable |
Not checkable |
30 Mar 2023 |
It’s definitely a big leap from GPT-3.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think that for there to be a person who I care about looking out at the universe and wondering at it and appreciating it, it’s not enough…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It’s smarter than natural selection, stupider than the upper bound.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
So there’s an upper bound, but it’s very, very, very far up above here.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The whole notion of open sourcing, this was always the wrong approach, the wrong ideal.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It has changed my understanding of what kind of work can be performed by which kind of processes and by which means.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
And if you train an AI system to make people press thumbs up, maybe you get these long, elaborate, impressive papers arguing for things…
Not checkable · 30 Mar 2023 |
|
| Mostly true |
Factual |
30 Mar 2023 |
Transformers were the main architectural change that gave a non-linear jump in performance, compared with earlier approaches such as RNNs.
Factual · 30 Mar 2023 |
AI capabilities |
| Not checkable |
Not checkable |
30 Mar 2023 |
You do not need life to be finite to be meaningful. It just has to be life.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
If you can’t verify it, you can’t ask the AI for it.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I do worry that like most of the aliens who are like humans or like a modern human civilization, I kind of worry that the super vast…
Not checkable · 30 Mar 2023 |
|
| Unresolvable |
Prediction |
30 Mar 2023 |
Yudkowsky said he would be relatively surprised to find that GPT had developed exact analogs of human emotions.
Prediction · 30 Mar 2023 |
AI capabilities |
| Not checkable |
Not checkable |
30 Mar 2023 |
I think that the first AIs you’re trying to build, not a million years later, the first ones look more like narrowly specialized biologists
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Subgenic fitness is the implicit loss function of evolution, which it cannot change.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The thing you would do at this point if there were massive public outcry pointed in the right direction, which I do not expect, is shut…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Are Earth’s billionaires going to put up the giant prizes that would maybe incentivize young hotshot people who just got their physics…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The first time you’re building a rocket, being wrong is, in a certain sense, quite easy.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The thing that it ends up wanting most is a thing that even from a very embracing cosmopolitan perspective, we think of as having no value.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Well, how smart is it?
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I mean, I think that you are more than the mask.
Not checkable · 30 Mar 2023 |
|
| Unresolvable |
Prediction |
30 Mar 2023 |
Yudkowsky said he thinks there is a definite point at which everybody dies because something sufficiently smarter than everybody exists.
Prediction · 30 Mar 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
30 Mar 2023 |
You would prefer, like maybe that’s the only way out but if you can find a security hole in the box you’re on, you’re gonna prefer to…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
it feels to me like I have watched the field of alignment just fail to thrive
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
To train a system to play, to win chess games, you have to be able to tell whether a game has been won or lost.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It means that things are in a horrible state, getting worse, and there’s nothing you can do about it.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I don’t want to be interpreted charitably.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
And if you can’t tell whether the output is good or bad, you cannot train the AI to produce better outputs.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
And even knowing exactly what I’m going to do, you can’t see how I got the results that I got.
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
Humans came from evolutionary computation, i.e. evolution.
Factual · 30 Mar 2023 |
Science and research methods |
| Not checkable |
Not checkable |
30 Mar 2023 |
The aliens, being as stupid as they are, have actually put you on Microsoft Azure cloud servers instead of this hypothetical perfect box.
Not checkable · 30 Mar 2023 |
|
| Unresolvable |
Prediction |
30 Mar 2023 |
Alignment findings from weak AI systems may not generalize to very strong AI systems.
Prediction · 30 Mar 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
30 Mar 2023 |
And I think that all the analogies between them are horribly misleading and yeah.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
We are boiling the frog.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Yes, cause that’s not how the fate of the galaxy gets settled.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I do not feel that seeing a being has changed my model of what intelligence is.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
the point at which a system has sufficient intelligence and situational awareness and understanding of human psychology that it would have…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
There is so much interpretability work to be done on the weaker versions of the systems.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It’s being predictably wrong.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Yeah, that’s just part, yeah, that’s part of the modern paradigm of alchemy.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
the game board has already been played into a frankly awful state.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Unless there’s an alien actress who is learning how to sound human, and is getting good at it.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
They wanted an excuse to wave it off.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
We need systems that will, that will like have a pause button where they won’t try to prevent you from pressing the pause button.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I just look at all the people, you know, like one by one, up to the eight billion and be like, that’s life, that’s life, that’s life.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The undignified thing is not being wrong.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I’m saying that the easiest way to pick up a hundred million people saying that you seem like a person is to look like a person talking to…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
No, I’m saying it’s difficult and dangerous in proportion to how it’s alien and how it’s smarter than you.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think we are not on track to learn remotely near fast enough, even if it were open sourced.
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
Yudkowsky said he pioneered "the thing that appears when you hover over it" and so gets some credit for that user experience, while noting that a team of developers also gets credit.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
30 Mar 2023 |
But it’s not like the thing that makes transformers smart.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
chipping flint handaxes and outwitting your fellow humans is basically the same problem as going to the moon.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Like the difference between capabilities now and 1000 years ago is a bigger gap than the gap in capabilities between 10 people and one…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
the more you can like, say oops softly, routinely, not as a big deal, the more chances you get to be like, I see where that reasoning went…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The meaning of life is not some kind of, like meaning is something that we bring to things when we look at them.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
And you don’t necessarily need any of these actual qualities in order to kill everyone, so.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
So the work of alignment might be qualitatively different above that threshold of intelligence or beneath it.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Any goal, almost any set of, almost every set of utility functions with a few narrow exceptions implies killing all the humans.
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
The COVID pandemic killed a few million people, and there was definitely a cover-up regarding whether it came from a lab leak.
Factual · 30 Mar 2023 |
Biosecurity |
| Not researched |
Factual |
30 Mar 2023 |
Despite full read access to GPT's numbers, we know vastly more about the architecture of human thinking than about what goes on inside GPT.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
30 Mar 2023 |
I know you want this to be true, but why would it be true?
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Yeah, I think there’s multiple thresholds.
Not checkable · 30 Mar 2023 |
|
| Mostly true |
Factual |
30 Mar 2023 |
As of March 2023, leading AI systems are being trained on computers that are connected to the internet.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
30 Mar 2023 |
Cause for 20 years, humanity decided to ignore the problem.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Well, I can easily imagine that at some point this panic expresses itself in the waste of a billion dollars.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
We would choose to preserve that within ourselves because it matters and on some viewpoints is the only thing that matters.
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
Yudkowsky said he believes the name Less Wrong was suggested by Nick Bostrom.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Unresolvable |
Prediction |
30 Mar 2023 |
The first time humans fail at aligning an AI much smarter than themselves, everyone will die and there will be no second attempt.
Prediction · 30 Mar 2023 |
AI safety and alignment |
| Not researched |
Factual |
30 Mar 2023 |
Natural selection has to run for hundreds of generations to notice that a trait is working.
Factual · 30 Mar 2023 |
Other |
| Unresolvable |
Factual |
30 Mar 2023 |
AI systems' rate of gaining capabilities is vastly outpacing researchers' ability to understand what goes on inside them.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
30 Mar 2023 |
It’s not even clear what it means exactly.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
A blank map does not correspond to a blank territory.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It is entirely possible that the things I am saying are wrong.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
So verifying a guess is easy, but coming up with a good suggestion is very hard.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
So if it were up to me, I would be like, okay, like this far, no further, time for the summer of AI where we have planted our seeds and now…
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
Yudkowsky said he did not think that, as of March 2023, a prompted AI system could successfully argue for its own consciousness before the US Supreme Court.
Factual · 30 Mar 2023 |
Philosophy, ethics and society |
| Not checkable |
Not checkable |
30 Mar 2023 |
If you’re thinking of it as a debate, you’re already screwing up.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think those people are closer to the masks that they wear than an alien from another planet would, like, learning how to predict the next…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Like you throw enough computing power at it, it obviously works.
Not checkable · 30 Mar 2023 |
|
| Unresolvable |
Factual |
30 Mar 2023 |
Bing can't, but GPT-4 can pretend that right now; the quote does not say what "that" refers to.
Factual · 30 Mar 2023 |
AI capabilities |
| Not checkable |
Not checkable |
30 Mar 2023 |
I think that if you optimize really hard on efficiently just having the useful parts, there is not then the thing that says like, I am…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Well, zero, 40%, and 100% is a bit closer to it, based on what happens to Chat GPT after you RLHF it to speak humane.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Suppose that some alien civilization with goals ultimately unsympathetic to ours, possibly not even conscious as we would see it, managed…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The old, you can’t bring the coffee if you’re dead.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
One can aspire to be well calibrated, like not so much think in terms of like, was I right, was I wrong?
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
But also when I say I put a low probability on it, I can feel myself reaching into the part of myself that thought that GPT-4 was not…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
And Bing has not yet updated me in that way.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Are you scared? Of this whole thing? Yeah. Probably. I don’t really know.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think that you will find great difficulty getting AIs to help you with anything where you cannot tell for sure that the AI is right once…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
But open sourcing it, no, that’s just sheer catastrophe.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
that GPT-4 is not turning out this way
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
But having built this thing, they do not understand how it output cold air.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
When the verifier is broken, the more powerful suggester does not help. It just learns to fool the verifier.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I did not grow up thinking that death was part of the meaning of life and now I still think it’s a pretty stupid idea.
Not checkable · 30 Mar 2023 |
|
| Mostly true |
Factual |
30 Mar 2023 |
Beyond external performance metrics, there are no guardrails for determining what is inside current large AI models.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
30 Mar 2023 |
Yeah, the probabilistic stuff is a giant wasteland of Eliezer and Paul Christiano arguing with each other and EA going like, eh?
Not checkable · 30 Mar 2023 |
|