| Not checkable |
Not checkable |
7 Apr 2023 |
To threaten these things, but end up not having to do them, is not very morally distinct—I would say—from doing them.
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
If there was a plan for Earth to survive, if only we passed a six-month moratorium, I would back that plan.
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
We are not prepared. We are not on course to be prepared in any reasonable time window. There is no plan.
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
This 6-month moratorium would be better than no moratorium.
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
Hearing this gave me a tiny flash of hope, because it’s a simpler, more sensible, and frankly saner reaction than I’ve been hearing over…
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
Visualize an entire alien civilization, thinking at millions of times human speeds, initially confined to computers—in a world of creatures…
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
If you can’t be sure whether you’re creating a self-aware AI, this is alarming not just because of the moral implications of the…
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
It shows an overwhelming gap between how seriously we are taking the problem, and how seriously we needed to take the problem starting 30…
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
This is a stupid state of affairs, and an undignified way for Earth to die, and the rest of humanity ought to step in at this point and…
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
The moratorium on new large training runs needs to be indefinite and worldwide.
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
Shut down all the large GPU clusters (the large computer farms where the most powerful AIs are refined).
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
There’s no proposed plan for how we could do any such thing and survive.
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
Key thresholds there may not be obvious, we definitely can’t calculate in advance what happens when, and it currently seems imaginable that…
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
This is not how the CEO of Microsoft talks in a sane world.
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
That kind of caring is something that could in principle be imbued into an AI but we are not ready and do not currently know how.
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
I agree that current AIs are probably just imitating talk of self-awareness from their training data.
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
Progress in AI capabilities is running vastly, vastly ahead of progress in AI alignment or even progress in understanding what the hell is…
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
I have respect for everyone who stepped up and signed it.
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
None of this danger depends on whether or not AIs are or can be conscious; it’s intrinsic to the notion of powerful cognitive systems that…
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
Make it explicit in international diplomacy that preventing AI extinction scenarios is considered a priority above preventing a full…
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
We are not ready. We are not on track to be significantly readier in the foreseeable future.
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
And the thing about trying this with superhuman intelligence is that if you get that wrong on the first try, you do not get to learn from…
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
It’s not that you can’t, in principle, survive creating something much smarter than you; it’s that it would require precision and…
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
we have no idea how to determine whether AI systems are aware of themselves—since we have no idea how to decode anything that goes on in…
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
The key issue is not “human-competitive” intelligence (as the open letter puts it); it’s what happens after AI gets to smarter-than-human…
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
7 Apr 2023 |
If we held anything in the nascent field of Artificial General Intelligence to the lesser standards of engineering rigor that apply to a…
Not checkable · 7 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Yeah I think in that case, we should believe that the dates on the calendars will never show 2024.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Maybe I will be wrong.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
But that doesn’t mean that you can do this with arbitrary minds and arbitrary mind space because all the different motivations hold…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
We’ve got different weirdnesses over time.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I’m actually getting into specifics here, which is the opposite of the point I originally meant to make
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
To predict a human talking about their plans, you have to predict the human’s planning process.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Yeah, they’re somewhat less so.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Every year up until the end of the world, people are going to max out their track record by betting all of their money on the world not…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Rationalists should win, but rationalists should not win the lottery.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
You are now 99.99% of the way to the finish line.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
the notion that you only have to be smart as a human to predict all the text on the internet, as soon as you turn around and stare at that…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Or some other exit plan.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
And if you had those for intelligence, you’d already be dead.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
You cannot configure the atom bomb in a clever way where it destroys the whole world and gives you the moon.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
There wasn’t a galaxy-brained purpose behind it.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Then you are trusting it to explain the theorem to you informally and that the informal meaning of the theorem is correct and that’s the…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Very little.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I’m not that smart. I’m not that wise.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
First of all, let me preface by saying that for all I know of the hidden variables of nature, it’s completely allowed that GPT-4 was…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I made most of my negative updates as of five years ago.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
If you get to the point where you’re turning into gods yourselves, you’re not quite home free, but you’re sure past a lot of the death.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
That sure has not been the history of the last 20 years from my perspective.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I think we’re just mainly in the territory of Hail Mary passes at this point, and human intelligence enhancement is one Hail Mary pass.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
You can break down anything into stages.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
We are learning from the AI systems that we build and as they fail and as we repair them and our learning goes along at this pace
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
The bad effects were very legible, very understandable.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Because it’s going this fast and capabilities are going this fast.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Because Interpretability is a kind of work where you can actually see the results and verify that they’re good results, unlike a bunch of…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
And if you make up some fancy number for your chance of dying next year, there’s very little you can do with it, really.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
If you have a very smart thing that’s aligned with humanity, good, you’re golden.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I don’t know.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Not good news for alignment.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
you can have almost any kind of self consistent utility function in a self consistent mind.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It looks like you can verify it and then it kills you.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
To me it looks like people are not dense in the incredibly multidimensional space of people.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I can print out all my answer sheets for it and each time I do that, it gets a little bit harder to make the case next time.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Like continuing to play out a video game you know you’re going to lose.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I am no longer willing to say that GPT-6 does not end the world.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
As you apply optimization pressure, the correlations that were found naturally come apart and aren’t present in the thing that gets…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
This is like literally my AI alignment fantasy from 2003, though not with RLHF as the implementation method or LLMs as the base.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
You zoom in and look at the mechanics and not the grand scale view, because the grand scale view just never gives you the right answer.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
But look far enough and there should be some species of nice aliens that were smarter or better at coordination and built their happily…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
You could ask GPT-4 to generate 10,000 approaches to alignment and that does not get you very far because GPT-4 is not going to have very…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I believe that Paul is honest.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Making people smarter has a chance of going right in a way that making an extremely smart AI does not have a realistic chance of going…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
So the quantum multiverse adds nothing.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Everybody makes the same mistakes.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
So many of the predictions like that are really anti-predictions.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I think that many of them don’t have qualia.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
But it’s a small ray of hope.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
most people do not happen to get a handful of cards that contain the writing card, whatever else their other talents.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I don’t think that GPT-3 to 3.5 to 4 was all that smooth.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I would be shocked if they did.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I think they’re enacting the ritual of the young optimistic scientist who charges forth with no ideas of the difficulties
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
And they invent ice-cream. It doesn’t just get coughed up at random, they are searching the space of things that they want and generating…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Humans are just running faster than natural selection, it didn’t actually halt.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
If GPT-5 is out next year all hell is broken loose and I don’t know.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
If anything, things have been taking longer to play out than I thought they would.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I do feel like GPT-4 is already kind of hanging out for longer in a weird, near-human space than I was really visualizing.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It would have to rewrite itself from scratch and if it wanted to, just upload a few kilobytes yes.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Okay, so then you expect most possible molecular configurations of the solar system to be equally probable.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
But it’s a lot more dangerous because you can’t predict exactly how many GPUs you need to ignite the atmosphere.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Basically no.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
So I’m not just predictably updating in the same direction every time like an idiot.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
The technical contribution I made is specifically, if you look at it carefully, a way that a malicious actor could use to poke a super…
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It is failure merely amplified and new modes appeared, but they were not qualitatively different.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
Having AI do your AI alignment homework for you is like the nightmare application for alignment.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
there’s actually no difference between extreme optimism and extreme pessimism
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I don’t actually know what GPT-5 is going to be like.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I am not a good mentor.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
We are so far behind right now.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
It still wouldn’t slow down forever.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
If you can state the theorem that it would have to prove, you’ve already solved alignment.
Not checkable · 6 Apr 2023 |
|
| Not checkable |
Not checkable |
6 Apr 2023 |
I know people who have a billion dollars.
Not checkable · 6 Apr 2023 |
|