| Not checkable |
Not checkable |
30 Mar 2023 |
The question is whether the current architecture of stacking more transformer layers, which for all we know GPT-4 is no longer doing…
Not checkable · 30 Mar 2023 |
|
| Unresolvable |
Factual |
30 Mar 2023 |
The most striking examples of AI behaviour are being posted to Twitter rather than examined systematically.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
30 Mar 2023 |
I don’t actually like the term control problem.
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
Versions of AI language models are already claiming to be conscious.
Factual · 30 Mar 2023 |
Philosophy, ethics and society |
| Mostly true |
Factual |
30 Mar 2023 |
Sixty years later, AI has made progress on a number of the earlier problems discussed, such as vision, though AI systems have not yet improved themselves.
Factual · 30 Mar 2023 |
AI capabilities |
| Not checkable |
Not checkable |
30 Mar 2023 |
I am not sure you are training it to output sense because I have seen the nonsense that has gotten thumbs up over the years.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
This does not mean that if you have a loss function of predicting the next token from that data set, that the mind picked out by gradient…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Well, no, the off switch makes sense. They’re just not opposing your attempt to pull the off switch.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Yeah, like oops is the sound we emit when we improve.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I do not think it is possible to understand the full depth of the problem that we are inside without understanding the problem of facing…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It is a bit smarter than I thought this technology was going to scale to, and I’m a bit worried about what the next one will be like.
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
The model was exposed to a vast corpus of internet text about people discussing consciousness, so it is not known to what extent its talk of being self-aware repeats what it was trained on.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
30 Mar 2023 |
And when you can easily tell whether the AI output is good or bad or how good or bad it is, and you can tell that accurately and reliably…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I don’t know that it’s going to be exponential, so I’m not gonna say exponential.
Not checkable · 30 Mar 2023 |
|
| Unresolvable |
Prediction |
30 Mar 2023 |
AGI could be achieved with neural networks as understood today.
Prediction · 30 Mar 2023 |
AI timelines and forecasting |
| Not researched |
Factual |
30 Mar 2023 |
People are already claiming that AI systems are conscious and deserve rights.
Factual · 30 Mar 2023 |
Philosophy, ethics and society |
| Not checkable |
Not checkable |
30 Mar 2023 |
And what it means is that you lose.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
It may be entirely wrong, but it’s the only time I’ve ever seen anybody like even come up with a halfway good argument for how many of…
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
Among the things humans can do is design new AI systems.
Factual · 30 Mar 2023 |
AI capabilities |
| Not checkable |
Not checkable |
30 Mar 2023 |
Yeah, I was just mistaken.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think that it is useful to an intelligent mind to have a model of itself
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
They spit out gold until they get large enough and then ignite the atmosphere and kill everybody.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
You’re trying to like, in the process of its creation, choose its direction.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
They would say that right up until the end of the world.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
That is a more difficult, more lethal form of the problem.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
So usually in science, if you’re mistaken, you run the experiment, it shows results different from what you expected.
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
Yudkowsky put up a prediction market on whether, in 2026, we will have understood anything inside a giant transformer net that was not known in 2006.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
30 Mar 2023 |
If humanity was trying to survive at this point in the straightforward way, it would be like shutting down the big GPU clusters, no more…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Gwern Branwen has been like, righter about this than I have.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
And I wouldn’t have said that this was a particularly smart thing for a species to do, which is an opinion that has changed less than my…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
The thing I want to communicate is the sort of difference that separates humans from chimpanzees.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
On the present paradigm, what you can verify is what you get more of.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Everybody recycling their cardboard is not gonna fix this.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
charisma is not generated in the liver rather than the brain. Charisma is also a cognitive function.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
And if they have anything like that, they should not say it.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
And if you’re smarter than that, you could like work on alignment problems where it’s harder to tell if you got them right or not.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
in a sane world, we sure would have a bunch of physicists working on this problem before it becomes a giant emergency.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
And from my perspective, this is all like an indistinguishable blob of people who are trying to not get to grips with the difficult problem…
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
Yudkowsky was never skeptical that evolutionary computation would work in the limit.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not researched |
Factual |
30 Mar 2023 |
Nobody knows what is going on inside the giant inscrutable matrices of floating point numbers that he was describing.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
30 Mar 2023 |
Spending a billion dollars correctly? That’s harder.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
When you optimize against visible misalignment, you are optimizing against misalignment and you are also optimizing against visibility.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
But first you have to solve the alignment. First, you have to solve inner alignment.
Not checkable · 30 Mar 2023 |
|
| Unresolvable |
Prediction |
30 Mar 2023 |
Catastrophe following from the release of GPT-4 is unlikely (a pretty low probability, no number given).
Prediction · 30 Mar 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
30 Mar 2023 |
We could have been working on this earlier.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I think that the person who won that debate was Guern.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Once you find those, you can tell that you found them. You can verify that the discovery is real.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Sure, but like humans having great leaps in their map, their understanding of the system is a very different concept from the system itself…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
I’d rather not be wrong next time.
Not checkable · 30 Mar 2023 |
|
| Not researched |
Factual |
30 Mar 2023 |
Large language models can play chess.
Factual · 30 Mar 2023 |
AI capabilities |
| Not researched |
Factual |
30 Mar 2023 |
Natural selection producing hominids did not require exponentially more resource investment to yield linear increases in competence.
Factual · 30 Mar 2023 |
Other |
| Not checkable |
Not checkable |
30 Mar 2023 |
Just because we cannot understand what’s going on inside GPT does not mean that it is not there.
Not checkable · 30 Mar 2023 |
|
| Unresolvable |
Factual |
30 Mar 2023 |
Current AI technology can instil outwardly observable behaviours in systems but does not know how to instil internal wanting of particular goals.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not researched |
Factual |
30 Mar 2023 |
GPT models could use calibrated probabilities before reinforcement learning from human feedback, but training them to satisfy humans made them worse at probability.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not researched |
Factual |
30 Mar 2023 |
In his writings on rationality, Yudkowsky emphasized probability theory more than reason as such.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
30 Mar 2023 |
Change their name to ClosedAI and like sell GPT-4 to business backend applications that don’t expose it to consumers and venture…
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
30 Mar 2023 |
Being wrong in a way that makes stuff easier, you know, that’s not the usual project management story.
Not checkable · 30 Mar 2023 |
|
| Unresolvable |
Factual |
30 Mar 2023 |
Yudkowsky said that the development under discussion probably was not happening yet.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Unresolvable |
Prediction |
30 Mar 2023 |
The next big jump in the number of people who think AI systems are conscious will come when AI looks like a cute, sympathetic person talking to them.
Prediction · 30 Mar 2023 |
Philosophy, ethics and society |
| Not checkable |
Not checkable |
30 Mar 2023 |
But so like humans aggregating don’t actually get, in my opinion, very much smarter, especially compared to running them for longer.
Not checkable · 30 Mar 2023 |
|
| Not checkable |
Not checkable |
Mar 2023 |
And then it's like maybe just barely a general intelligence or like a narrow general intelligence or you know something we don't really…
Not checkable · Mar 2023 |
|
| Not checkable |
Not checkable |
Mar 2023 |
If you boil a frog gradually enough, if you zoom in far enough, it's always hard to tell around the edges.
Not checkable · Mar 2023 |
|
| Unresolvable |
Prediction |
Mar 2023 |
With a GPT-5 that is more clearly a general intelligence, it may become even harder to turn back from AI development.
Prediction · Mar 2023 |
AI safety and alignment |
| Unresolvable |
Prediction |
Mar 2023 |
If this is how things are scaling, GPT-5 may be more unambiguously a general intelligence.
Prediction · Mar 2023 |
AI capabilities |
| Unresolvable |
Factual |
Mar 2023 |
Someone asked Bing's Sydney to describe herself and fed the resulting description into a Stable Diffusion image generator.
Factual · Mar 2023 |
AI capabilities |
| Not checkable |
Not checkable |
Mar 2023 |
this does not mean that if you have a loss function of predicting the next token from that dataset, the mind picked out by gradient descent
Not checkable · Mar 2023 |
|
| Not checkable |
Not checkable |
Mar 2023 |
And you know if they say that they're kind of whack because how could they possibly know that even if it were true.
Not checkable · Mar 2023 |
|
| Not checkable |
Not checkable |
Mar 2023 |
I was expecting more of that.
Not checkable · Mar 2023 |
|
| Not checkable |
Not checkable |
Mar 2023 |
I think if you sent me to a distant galaxy with aliens that are much stupider than I am
Not checkable · Mar 2023 |
|
| Not checkable |
Not checkable |
Mar 2023 |
Uhm yeah,that is not quite how I expected it to play out.
Not checkable · Mar 2023 |
|
| Mostly true |
Factual |
Mar 2023 |
Yudkowsky previously expected AI progress to involve more distinct discoveries, rather than scaling an existing architecture.
Factual · Mar 2023 |
AI capabilities |
| Unresolvable |
Factual |
Mar 2023 |
The image generated from Bing Sydney's self-description was a pretty picture of a girl with steampunk goggles on her head.
Factual · Mar 2023 |
AI capabilities |
| Not checkable |
Not checkable |
Feb 2023 |
GPT-3 was a big leap forward.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
If I just have another $50 million, I'm not quite sure what to do with that
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
whatever civilization arises after that would probably, I'm guessing, end up stuck in just the same trap we are.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
It became clear we were all going to die.
Not checkable · Feb 2023 |
|
| Unresolvable |
Factual |
Feb 2023 |
Yudkowsky says it is theoretically possible, but seems very unlikely, that the AI event horizon has already been crossed, because somebody would need an AI in their lab that was much more advanced than the public AI technology.
Factual · Feb 2023 |
AI policy and governance |
| Not checkable |
Not checkable |
Feb 2023 |
If we got unlimited free retries and 50 years to solve everything, it'd be okay.
Not checkable · Feb 2023 |
|
| Resolved true |
Factual |
Feb 2023 |
Yudkowsky says that we do not know how to get goals into an AI system, and that gradient descent can cause systems to do a thing inside the distribution they were optimized over.
Factual · Feb 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
Feb 2023 |
But 1% of $10 billion is still a lot of money to actually accomplish something.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
You can get more people aware that we're walking directly into the whirling razor blades, because even if only 10% of the people can get…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
When you use the hill-climbing paradigm and optimize for one single extremely pure thing, this is how much of it gets inside.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
If natural selection had been a foresightful, intelligent kind of engineer that was able to engineer things successfully, it would have…
Not checkable · Feb 2023 |
|
| Own records only |
Factual |
Feb 2023 |
MIRI did not know in advance that its failed lines of research would fail to pan out.
Factual · Feb 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
Feb 2023 |
but they do burn any money literally at all.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
And I think that the actual catastrophe scenario for AI looks like big advancement in a research lab, maybe driven by them getting a giant…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
I expect that's much easier for some alien species than others.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
they're making up arguments rather than deeply considering things that are held to any standard of rigor
Not checkable · Feb 2023 |
|
| Unresolvable |
Factual |
Feb 2023 |
We do not know how to design an AI that would avoid the science-fiction disaster scenarios described, such as one that hovers around being obsessively nice instead of leaving people alone, though such AIs are not impossible in principle.
Factual · Feb 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
Feb 2023 |
Its actions are the most efficient actions for accomplishing its goals. If you think you see a better way to accomplish its goals, you're…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
I mean, I could say something like shut down all the large GPU clusters.
Not checkable · Feb 2023 |
|
| Not researched |
Factual |
Feb 2023 |
Yudkowsky said in February 2023 that he thinks the technology does not exist to align an AI well enough to instruct it to build a copy of a strawberry without destroying the world.
Factual · Feb 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
Feb 2023 |
And a couple of days later, everybody on earth falls over dead in the same second.
Not checkable · Feb 2023 |
|
| Mostly true |
Factual |
Feb 2023 |
The technology underlying ChatGPT was published, if he recalled correctly, in 2018.
Factual · Feb 2023 |
AI capabilities |
| Not checkable |
Not checkable |
Feb 2023 |
This is a *lot* harder than nuclear weapons.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
And 50 years later, you'd have an aligned AGI.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
An AI is like that, but instead of needing to refine plutonium, you can make nuclear weapons out of a billion tons of laundry detergent.
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
If I have buy-in from a major government that can run actual security precautions and more than just $10 billion, then you could run a…
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
How the hell would I know?
Not checkable · Feb 2023 |
|
| Not checkable |
Not checkable |
Feb 2023 |
because they're too far away
Not checkable · Feb 2023 |
|