| Not checkable |
Not checkable |
6 Apr 2023 |
Yeah, they’re somewhat less so.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Every year up until the end of the world, people are going to max out their track record by betting all of their money on the world not…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Rationalists should win, but rationalists should not win the lottery.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
You are now 99.99% of the way to the finish line.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
the notion that you only have to be smart as a human to predict all the text on the internet, as soon as you turn around and stare at that…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Or some other exit plan.
Not checkable · 6 Apr 2023 |
|
| Not researched |
Factual |
6 Apr 2023 |
Text on the internet includes material produced not by humans conversing but by extensive work extracting experimental results from reality.
Factual · 6 Apr 2023 |
Science and research methods |
| Not checkable |
Not checkable |
6 Apr 2023 |
And if you had those for intelligence, you’d already be dead.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
You cannot configure the atom bomb in a clever way where it destroys the whole world and gives you the moon.
Not checkable · 6 Apr 2023 |
|
| Unresolvable |
Prediction |
6 Apr 2023 |
AI will not reach a distinct human level; it will be somewhere around human but not like a human.
Prediction · 6 Apr 2023 |
AI capabilities |
| Not checkable |
Not checkable |
6 Apr 2023 |
There wasn’t a galaxy-brained purpose behind it.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Then you are trusting it to explain the theorem to you informally and that the informal meaning of the theorem is correct and that’s the…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Very little.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I’m not that smart. I’m not that wise.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
First of all, let me preface by saying that for all I know of the hidden variables of nature, it’s completely allowed that GPT-4 was…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I made most of my negative updates as of five years ago.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
If you get to the point where you’re turning into gods yourselves, you’re not quite home free, but you’re sure past a lot of the death.
Not checkable · 6 Apr 2023 |
|
| Unresolvable |
Prediction |
6 Apr 2023 |
Yudkowsky said he thinks that at some point, once a system knows itself especially well and is able to rewrite itself, it crystallizes, unless one specifically chooses not to.
Prediction · 6 Apr 2023 |
AI safety and alignment |
| Not researched |
Factual |
6 Apr 2023 |
Yudkowsky said he was not familiar with anybody who predicted, in advance of the observation, the qualitative jumps in ability that corresponded to smoothly declining loss on text prediction.
Factual · 6 Apr 2023 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
6 Apr 2023 |
That sure has not been the history of the last 20 years from my perspective.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I think we’re just mainly in the territory of Hail Mary passes at this point, and human intelligence enhancement is one Hail Mary pass.
Not checkable · 6 Apr 2023 |
|
| Not researched |
Factual |
6 Apr 2023 |
Using PCR, a person can currently make a gallon jar full of copies of their own DNA.
Factual · 6 Apr 2023 |
Biosecurity |
| Not checkable |
Not checkable |
6 Apr 2023 |
You can break down anything into stages.
Not checkable · 6 Apr 2023 |
|
| Own records only |
Factual |
6 Apr 2023 |
No one in government has reached out to Eliezer Yudkowsky in a way suggesting they have the broad contours of the AI risk problem right.
Factual · 6 Apr 2023 |
AI policy and governance |
| Not checkable |
Not checkable |
6 Apr 2023 |
We are learning from the AI systems that we build and as they fail and as we repair them and our learning goes along at this pace
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
The bad effects were very legible, very understandable.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Because it’s going this fast and capabilities are going this fast.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Because Interpretability is a kind of work where you can actually see the results and verify that they’re good results, unlike a bunch of…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
And if you make up some fancy number for your chance of dying next year, there’s very little you can do with it, really.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
If you have a very smart thing that’s aligned with humanity, good, you’re golden.
Not checkable · 6 Apr 2023 |
|
| Unresolvable |
Prediction |
6 Apr 2023 |
Yudkowsky said he is pretty sure that AI systems that are smart enough will no longer need giant training runs.
Prediction · 6 Apr 2023 |
AI compute, hardware and data |
| Not checkable |
Not checkable |
6 Apr 2023 |
I don’t know.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Not good news for alignment.
Not checkable · 6 Apr 2023 |
|
| Unresolvable |
Factual |
6 Apr 2023 |
The LessWrong sequences were written primarily as an instruction manual for potential young people like Yudkowsky.
Factual · 6 Apr 2023 |
Field building and careers |
| Own records only |
Factual |
6 Apr 2023 |
GPT-4 got further than Eliezer Yudkowsky had expected simply stacking more layers would get.
Factual · 6 Apr 2023 |
AI capabilities |
| Not checkable |
Not checkable |
6 Apr 2023 |
you can have almost any kind of self consistent utility function in a self consistent mind.
Not checkable · 6 Apr 2023 |
|
| Not researched |
Factual |
6 Apr 2023 |
Yudkowsky said the LessWrong sequences were part of an effort he "really, really tried" to make, and added that they also "had other purposes".
Factual · 6 Apr 2023 |
Field building and careers |
| Not checkable |
Not checkable |
6 Apr 2023 |
It looks like you can verify it and then it kills you.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
To me it looks like people are not dense in the incredibly multidimensional space of people.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I can print out all my answer sheets for it and each time I do that, it gets a little bit harder to make the case next time.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Like continuing to play out a video game you know you’re going to lose.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I am no longer willing to say that GPT-6 does not end the world.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
As you apply optimization pressure, the correlations that were found naturally come apart and aren’t present in the thing that gets…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
This is like literally my AI alignment fantasy from 2003, though not with RLHF as the implementation method or LLMs as the base.
Not checkable · 6 Apr 2023 |
|
| Mostly true |
Factual |
6 Apr 2023 |
Eliezer Yudkowsky has consistently refused for many years to publish AI timelines with detailed probabilities.
Factual · 6 Apr 2023 |
Their own work or organisation |
| Unresolvable |
Factual |
6 Apr 2023 |
Nate Soares has written a post about the common mistakes everyone makes when coming into AI alignment.
Factual · 6 Apr 2023 |
Their own work or organisation |
| Mostly false |
Prediction |
6 Apr 2023 |
Paul Christiano will probably win his forecast disagreement with Yudkowsky on an AI getting IMO gold by 2025, with prediction-market odds then running around 30%.
Prediction · 6 Apr 2023 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
6 Apr 2023 |
You zoom in and look at the mechanics and not the grand scale view, because the grand scale view just never gives you the right answer.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
But look far enough and there should be some species of nice aliens that were smarter or better at coordination and built their happily…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
You could ask GPT-4 to generate 10,000 approaches to alignment and that does not get you very far because GPT-4 is not going to have very…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I believe that Paul is honest.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Making people smarter has a chance of going right in a way that making an extremely smart AI does not have a realistic chance of going…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
So the quantum multiverse adds nothing.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Everybody makes the same mistakes.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
So many of the predictions like that are really anti-predictions.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I think that many of them don’t have qualia.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
But it’s a small ray of hope.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
most people do not happen to get a handful of cards that contain the writing card, whatever else their other talents.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I don’t think that GPT-3 to 3.5 to 4 was all that smooth.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I would be shocked if they did.
Not checkable · 6 Apr 2023 |
|
| Resolved false |
Prediction |
6 Apr 2023 |
The current AI developers will not try the alignment approach he described (filtering training data to nice examples and training on it).
Prediction · 6 Apr 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
6 Apr 2023 |
I think they’re enacting the ritual of the young optimistic scientist who charges forth with no ideas of the difficulties
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
And they invent ice-cream. It doesn’t just get coughed up at random, they are searching the space of things that they want and generating…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Humans are just running faster than natural selection, it didn’t actually halt.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
If GPT-5 is out next year all hell is broken loose and I don’t know.
Not checkable · 6 Apr 2023 |
|
| Unresolvable |
Prediction |
6 Apr 2023 |
AI systems will never be exactly human, imitating humans imperfectly in some ways while exceeding them in speed.
Prediction · 6 Apr 2023 |
AI capabilities |
| Not checkable |
Not checkable |
6 Apr 2023 |
If anything, things have been taking longer to play out than I thought they would.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I do feel like GPT-4 is already kind of hanging out for longer in a weird, near-human space than I was really visualizing.
Not checkable · 6 Apr 2023 |
|
| Not researched |
Factual |
6 Apr 2023 |
In 2019 and 2020 AI progress slowed somewhat, leaving more time than Yudkowsky had feared.
Factual · 6 Apr 2023 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
6 Apr 2023 |
It would have to rewrite itself from scratch and if it wanted to, just upload a few kilobytes yes.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Okay, so then you expect most possible molecular configurations of the solar system to be equally probable.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
But it’s a lot more dangerous because you can’t predict exactly how many GPUs you need to ignite the atmosphere.
Not checkable · 6 Apr 2023 |
|
| Not researched |
Factual |
6 Apr 2023 |
MIRI did not advertise the Visible Thoughts Project to people as a way to save the world, but as a tiny shred of hope.
Factual · 6 Apr 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
6 Apr 2023 |
Basically no.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
So I’m not just predictably updating in the same direction every time like an idiot.
Not checkable · 6 Apr 2023 |
|
| Not researched |
Factual |
6 Apr 2023 |
In 2015, 2016 and 2017 Eliezer Yudkowsky noticed he had repeatedly been surprised by AI progress moving faster than he anticipated.
Factual · 6 Apr 2023 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
6 Apr 2023 |
The technical contribution I made is specifically, if you look at it carefully, a way that a malicious actor could use to poke a super…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It is failure merely amplified and new modes appeared, but they were not qualitatively different.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Having AI do your AI alignment homework for you is like the nightmare application for alignment.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
there’s actually no difference between extreme optimism and extreme pessimism
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I don’t actually know what GPT-5 is going to be like.
Not checkable · 6 Apr 2023 |
|
| Unresolvable |
Prediction |
6 Apr 2023 |
Yudkowsky worries that if researchers understood what goes on inside GPT-4, they would know how to rebuild it much smaller.
Prediction · 6 Apr 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
6 Apr 2023 |
I am not a good mentor.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
We are so far behind right now.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It still wouldn’t slow down forever.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
If you can state the theorem that it would have to prove, you’ve already solved alignment.
Not checkable · 6 Apr 2023 |
|
| Unresolvable |
Factual |
6 Apr 2023 |
Yudkowsky wrote something about the circuitry of general intelligence when he was 22.
Factual · 6 Apr 2023 |
Their own work or organisation |
| Not researched |
Factual |
6 Apr 2023 |
As AI programs have become simpler, humans have less insight into them and their content is more opaque, as with AlphaZero.
Factual · 6 Apr 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
6 Apr 2023 |
I know people who have a billion dollars.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
There is no rule about one of them is bound to work.
Not checkable · 6 Apr 2023 |
|
| Own records only |
Factual |
6 Apr 2023 |
CFAR (named in the surroundings as the "Center For Applied Rationale") did not get as far as buying an fMRI machine, and it had no funding.
Factual · 6 Apr 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
6 Apr 2023 |
And Goodhart’s law applies, Goodhart’s curse applies.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Probably at that point the AI rewrote the AI and the losses declined.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I thought that this was something very unlikely for governments to adopt and then all of my friends kept on telling me
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
As long as you don’t have the weird options, you’re not going to make the weird choices.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It has been very hard to call the rate at which these systems acquire capability as they are trained to larger and larger sizes and more…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
No. We don’t need a bunch of stuff, we need one.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I have watched this field fail to thrive for 20 years with narrow exceptions for stuff that is more verifiable in advance of it actually…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
This is a simple demonstration of how you can go on getting lower losses by throwing a hypercomputer at the problem.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Pencil and paper to GPT and it got smarter, right?
Not checkable · 6 Apr 2023 |
|