| Unresolvable |
Prediction |
6 Apr 2023 |
Yudkowsky said he would be quite surprised if AI progress slowed down a lot after reaching slightly above human level because the systems are trained on human text.
Prediction · 6 Apr 2023 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
6 Apr 2023 |
He did not actually have those options. You are not pointing out to me a lack of preference on Oppenheimer’s part. You are pointing out to…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Well, I hesitate to use the term midwit but maybe this was all just a midwit thing.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It seems foolish and to lack dignity to not even try to say what ought to be done.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
And you don't ever want to have chronic fatigue syndrome on your medical record because that just tells doctors to give up on you.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
We’re guessing much more blindly here.
Not checkable · 6 Apr 2023 |
|
| Unresolvable |
Prediction |
6 Apr 2023 |
Yudkowsky said he feels it is probably over 1% technically feasible to build something that applies general intelligence narrowly to the neuroscience of augmenting humans, while saying the world is far from trying to do so in a way that could work.
Prediction · 6 Apr 2023 |
AI capabilities |
| Not checkable |
Not checkable |
6 Apr 2023 |
But from my perspective, the future is just really hard to predict.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I agree it’s possible to imagine things being even worse.
Not checkable · 6 Apr 2023 |
|
| Not researched |
Factual |
6 Apr 2023 |
Interpretability researchers are working on models smaller than GPT-2.
Factual · 6 Apr 2023 |
AI safety and alignment |
| Not researched |
Factual |
6 Apr 2023 |
Yudkowsky has not solved the AI alignment problem.
Factual · 6 Apr 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
6 Apr 2023 |
I don’t think I’m unusual in looking around myself in that highly multidimensional space and not finding a ton of neighbors ready to take…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
No, they are not.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
The alignment field failed to gel.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
There’s an established preference for people going to some lengths to make their kids healthier, not necessarily via the options that they…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Usually when you’re trying to build a rocket and your model of rockets is lousy, it doesn’t cause the rocket to launch using half the fuel…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Humanity hasn’t really existed for very long.
Not checkable · 6 Apr 2023 |
|
| Unresolvable |
Prediction |
6 Apr 2023 |
Once AIs can build AI systems better than humans, rapid recursive self-improvement (foom) will definitely begin.
Prediction · 6 Apr 2023 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
6 Apr 2023 |
But as soon as your sentence has more than like five words in it, its probability has dropped to basically zero because of all the extra…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
We saw these people get caught utterly flat-footed on the Internet.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Don’t have that be the plan. That does not sound like a good plan.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I’m worried that they’re pretty distant.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I mean you definitely want to use the alien observation of 10,000 planets like this one prior for what you get after training on, like…
Not checkable · 6 Apr 2023 |
|
| Not researched |
Factual |
6 Apr 2023 |
Current RLHF-trained language models ended up as a weird corporate-speak, left-leaning-sounding chatbot persona.
Factual · 6 Apr 2023 |
AI capabilities |
| Not checkable |
Not checkable |
6 Apr 2023 |
Because as you get smarter, you get new options that are further from the options that you are faced with in the ancestral environment that…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It makes everything a lot more grim.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
These are also quite dangerous things, but they do not have the utter lethality of artificial intelligence.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
We just suck at alignment, and that’s why our stuff blew up.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I think that’s about the level where a human could start to work on alignment, more or less.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
But I think the smarter they are, the more likely they are to go for it if it’s credible.
Not checkable · 6 Apr 2023 |
|
| Unresolvable |
Prediction |
6 Apr 2023 |
Trying to use a very smart AI system that understands biology is unsafe enough that you will probably die.
Prediction · 6 Apr 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
6 Apr 2023 |
Let’s offer $100 billion in prizes for Interpretability.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I think that a lot of the people who want kids would go for this new offer that just offers them so much more of what it is they want from…
Not checkable · 6 Apr 2023 |
|
| Mixed |
Factual |
6 Apr 2023 |
MIRI ran a Visible Thoughts Project that did not get enough funding or personnel and was progressing too slowly.
Factual · 6 Apr 2023 |
Their own work or organisation |
| Not researched |
Factual |
6 Apr 2023 |
Neither Yudkowsky nor the signatories of the open letter say that current AI systems are already at a dangerous point.
Factual · 6 Apr 2023 |
AI policy and governance |
| Unresolvable |
Prediction |
6 Apr 2023 |
Yudkowsky suspects that, as the people a language model is pretending to be get smarter, at some point a new coherence will emerge within the system and something strange will start happening.
Prediction · 6 Apr 2023 |
AI capabilities |
| Not checkable |
Not checkable |
6 Apr 2023 |
There’s no established preference for four eyes.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Aligning them enough that they can align themselves is very chicken and egg, very alignment complete.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I think you’re wrong.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It depends on the exit plan.
Not checkable · 6 Apr 2023 |
|
| Mostly true |
Factual |
6 Apr 2023 |
In principle, the task of minimizing loss on predicting human text does not stop being improvable once a system is as smart as a human.
Factual · 6 Apr 2023 |
AI capabilities |
| Not checkable |
Not checkable |
6 Apr 2023 |
They’re made out of all these complicated desires.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
The hinge of this argument is the capabilities constraint.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It’s on the podcast, it goes on the Internet.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
The jump to superintelligence does strike me as being significant in the same way as the first self-replicator.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It violates the laws of physics.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Because we haven’t yet had options as far enough outside of the ancestral distribution that in the course of choosing what we most want…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
But you would. You just look down at your fellow humans. You have no confidence in their ability to tolerate weirdness, even if they can.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Your entire analogy fails.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
So being very unsure about the future looks like predicting with probability nearly one that the humans are all gone
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It’s hard. It is just easier to predict the endpoint than it is to predict the path.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I don’t think alignment is actually so easy that you just get whatever you want.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
You’re saying all lottery numbers are equally likely instead of winning and losing are equally likely.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
This is not an IQ 100 AI we’re talking about here.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Reality is just quite large.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
And the smarter you are, the more out of distribution you get.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It is very clearly understood which systems spit out low quantities of gold and the qualitatively different systems that don’t actually…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I think it’s much easier to get a mask like that than to get a shoggoth like that.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Trying to play this game against things smarter than you is a fool’s game.
Not checkable · 6 Apr 2023 |
|
| Not researched |
Factual |
6 Apr 2023 |
Simply stacking more layers in neural networks achieved general capabilities first, without specialized systems.
Factual · 6 Apr 2023 |
AI capabilities |
| Not checkable |
Not checkable |
6 Apr 2023 |
It is always going to be at an early stage relative to the super intelligence that can actually kill you.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
No, no. What I’m saying is that as smart as the people it’s pretending to be are, it’s got planning that powerful inside the system…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
is that the existence of this technical array of hope, if you do just the right things, is not the same as expecting that the world…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I’m here on podcasts instead of working on them, aren’t I?
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Look at the thing with ice cream, look at the thing with condoms, look at the thing with pornography, see where this is going.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
But my point is to get to like 90% chance of winning, which is pretty hard on any exit scheme, you want a fast exit scheme.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
This kind of happened with humans. Well, life generally.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Weird is relative to intelligence. The smarter you are, the more you can move around in the space of abstractions and not have things seem…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
So large language models will change their preferences as they get smarter.
Not checkable · 6 Apr 2023 |
|
| Not researched |
Factual |
6 Apr 2023 |
Researchers understood AlphaZero's goals much better than they understand large language models' goals.
Factual · 6 Apr 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
6 Apr 2023 |
What? We get a black box output, then we get another black box output.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I’ve seen no evidence, but who knows what the alternate Everett branches of Earth are like?
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I’m not trying to violate the laws of physics, even as you probably know them.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
And this is very much not the situation we have with nuclear weapons.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
All kinds of interesting things started happening with GPT 3.5 and GPT-4 that were not in GPT-3.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
If you try to make people who are inhumanly nice, who are nicer than anyone has ever been before, you’re going outside the space that human…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
What will the future be? I don’t know. It could be paperclips, it could be staples.
Not checkable · 6 Apr 2023 |
|
| Unresolvable |
Prediction |
31 Mar 2023 |
Yudkowsky expects that, now that things have been reduced to this last resort, everyone will die.
Prediction · 31 Mar 2023 |
AI safety and alignment |
| Resolved true |
Factual |
31 Mar 2023 |
Yudkowsky's TIME piece did not suggest nuclear strikes against countries refusing to sign a global agreement against large AI training runs.
Factual · 31 Mar 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
30 Mar 2023 |
It doesn’t have to be a very sharp threshold
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
So magic in the sense is I can tell you exactly what I’m going to do.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
So the problem I see is that your typical human has a great deal of trouble telling whether I or Paul Christiano is making more sense.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The most success I’ve had so far is, well, imagine that the humans are running at very high speeds compared to very slow aliens.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Oh, we are so far behind on interpretability compared to capabilities.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I’m not actually an expert on that. And the experts don’t know either cause it’s kind of hard to predict the future.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It’s all the things that I value about it and maybe all the things that I would value if I understood it better.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It’s not smart enough to prevent you from pressing the off switch and probably not smart enough to want to prevent you from pressing the…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
And it’s replacement by nothing interesting or worthwhile, even from a very cosmopolitan perspective on what counts as worthwhile.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
If I am wrong and there is that kind of public outcry, then somebody in high school could be ready to be part of that.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think the human alignment problem is a terrible phrase because it is very, very different to like try to build systems out of humans
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think you’re kind of like projecting the aliens being like humans, and like thinking of a human in a human society rather than a human in…
Not checkable · 30 Mar 2023 |
|
| Unresolvable |
Factual |
30 Mar 2023 |
Yudkowsky said he did not think the AI system under discussion was as smart as a human yet.
Factual · 30 Mar 2023 |
AI capabilities |
| Not checkable |
Not checkable |
30 Mar 2023 |
I mean, there’s the question of to what extent it is thereby being made more human-like versus to what extent an alien actress is learning…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think even GPT-2 probably has like very stupid fragments of alien actress in it.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Reverse stupidity is not intelligence.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I would say, find prediction markets that matter to you and bet in the prediction markets.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Because, no, we are not in the shape to frantically at the last minute do decades’ worth of work.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think one should only be satisfied with solving the entire problem.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
And if you have something that is generally smarter than a human, it’s probably also generally smarter at building AI systems.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think the mask is a slice through you. It may even be the slice that’s in charge of you.
Not checkable · 30 Mar 2023 |
|