| Not checkable |
Not checkable |
28 Oct 2020 |
I wonder if there's any correlation between modern sanity losses, and discoveries in science no longer being named after individuals.
Not checkable · 28 Oct 2020 |
|
| Unresolvable |
Factual |
27 Oct 2020 |
In summer 2020 there was a limited window to rapidly test COVID-19 vaccines using many volunteers for human challenge trials, and it was not used.
Factual · 27 Oct 2020 |
Biosecurity |
| Not checkable |
Not checkable |
23 Oct 2020 |
the Democratic Party will keep running candidates so awful that ~half the population
Not checkable · 23 Oct 2020 |
|
| Not checkable |
Not checkable |
21 Oct 2020 |
I feel that I should get credit for turning out even *this* normal
Not checkable · 21 Oct 2020 |
|
| Not checkable |
Not checkable |
17 Oct 2020 |
So many Onion articles have become reality that I now feel scared that this, too, will become real
Not checkable · 17 Oct 2020 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
I think that a lot of people in the field would agree that human-level AI, defined as “literally at the human level, neither above nor…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
A conman who just wants money and gets no innate kick out of you believing false things will cause you to believe false things in order to…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
I’m concerned that the sloppy version can be built two years earlier and that there is no non-sloppy version to defend us from it.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
Whereas endless dystopias don’t seem to me to have any use to a paperclip maximizer.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
Because I’m not sure I believe you.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
You are in a terrible, terrible situation.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
Our current methods of alignment do not scale, and I think that all of the actual technical difficulty that is actually going to shoot down…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
More toward the fairy tale.
Not checkable · 28 Feb 2018 |
|
| Mostly true |
Factual |
28 Feb 2018 |
Yudkowsky said that, as far as the rest of the world knows, the participant in an AI-box experiment later sent a PGP-signed email saying "I let Eliezer out of the box."
Factual · 28 Feb 2018 |
AI safety and alignment |
| Not checkable |
Not checkable |
28 Feb 2018 |
the unknowable is a concept you have to be very careful with.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
our only competitor in terms of general optimization—where “optimization” is that sort of narrowing of the future that I talked about—is…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
And this is the kind of thinking that does not enable you to save civilization.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
deceiving the humans into thinking that you are friendly is a very generic strategy across a wide range of utility functions.
Not checkable · 28 Feb 2018 |
|
| Unresolvable |
Prediction |
28 Feb 2018 |
Once artificial general intelligence capabilities exist, they will scale too fast for an economic framing to be a useful way to view the problem.
Prediction · 28 Feb 2018 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
28 Feb 2018 |
The first thing I’d want to say is this is an empirical question.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
The thing I’m worried about is that it’s going to end up with a randomly rolled utility function whose maximum happens to be a particular…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
I think that that example might be unconvincing to somebody who’s coming in with a suspicion that intelligence and values are correlated.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
I don’t want to put absolute faith, because there is the replication crisis
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
Now that’s not human general, but it’s a step forward in generality of the sort that we’re talking about.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
They’re looking in the wrong direction and ignoring the thing that’s actually going to kill them.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
Then my guess would be something like Kickstarter, but much better, that turned out to enable people in large groups to move forward when…
Not checkable · 28 Feb 2018 |
|
| Mostly true |
Factual |
28 Feb 2018 |
One condition of the AI-box experiment was that no one would ever say what went on inside it.
Factual · 28 Feb 2018 |
AI safety and alignment |
| Own records only |
Factual |
28 Feb 2018 |
At the time of the interview, Eliezer Yudkowsky said he did not remember whether the person being discussed, apparently Robin, had yet replied to a particular point.
Factual · 28 Feb 2018 |
Their own work or organisation |
| Not checkable |
Not checkable |
28 Feb 2018 |
Nothing can prepare you for Robin Hanson!
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
it is okay for us to reason about aligned (or even unaligned) artificial general intelligences of sufficient power as if they’re trying to…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
Maybe AlphaZero is the sort of thing that happens five years before the end of the world across most planets in the universe.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
Which means that it’s not the case that as you scale intelligence, it gets harder and harder to buy.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
That’s the little big problem, which is still a very large problem.
Not checkable · 28 Feb 2018 |
|
| Mostly true |
Factual |
28 Feb 2018 |
Even most scientists in the field had no idea how to build a nuclear bomb until the Hiroshima headlines.
Factual · 28 Feb 2018 |
Other global risks |
| Not checkable |
Not checkable |
28 Feb 2018 |
I think the model of the world is wrong. I think they’re factually wrong about what happens when you take a random AI and make it much…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
humans are not secure software.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
These two questions together form a paperclip maximizer. You don’t need anything else.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
the idea that human minds are secure is loony.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
That’s what the AI-box experiment illustrates. You don’t know what went on in there, and that’s exactly the position you’d be in with…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
You only get oughts from other oughts. That’s the Hume version of the thesis.
Not checkable · 28 Feb 2018 |
|
| Not researched |
Factual |
28 Feb 2018 |
AlphaZero does not need to first completely rewrite itself to do powerful things, making it more plausible that AIs could do dangerous or useful things without prior complete self-rewrites.
Factual · 28 Feb 2018 |
AI capabilities |
| Not checkable |
Not checkable |
28 Feb 2018 |
I would describe myself as a decision theorist.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
I’m worried that it’s not happening fast enough
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
Parenthetically, I do want to be slightly careful about using the word “rational” to describe the behavior of people stuck in the system…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
Collectively. Like, if people who slow down and do things right finish their work two years after the universe has been destroyed, that’s…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
the way I would phrase it is that it’s not that the paperclip maximizer has a different set of oughts, but that we can see it as running…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
all of the ones who are not utter idiots are past the point where they would deliberately enact Hollywood movie plots.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
We know humans are conscious, so the computations that you run to build very accurate predictive models of humans are among the parts that…
Not checkable · 28 Feb 2018 |
|
| Unresolvable |
Prediction |
28 Feb 2018 |
Very powerful and smart AI systems will redesign and improve themselves unless prevented for some reason.
Prediction · 28 Feb 2018 |
AI safety and alignment |
| Not checkable |
Not checkable |
28 Feb 2018 |
And I would say this is a computer science issue.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
So, one way to look at the book is that it’s about how you can get crazy, stupid, evil large systems without any of the people inside them…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
paperclips are not one of those things. No matter how long you stare at a paperclip, it still seems pretty pointless from our perspective.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
The original “orthogonal” means “at right angles.”
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
And I find that convincing, but other people may want additional details.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
I’d say that Neil de Grasse Tyson is failing to respect the AI’s intelligence to the point of asking what he would do if he were inside a…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
I’d say this is a thesis of capability gain.
Not checkable · 28 Feb 2018 |
|
| Resolved true |
Factual |
28 Feb 2018 |
Nobody currently knows how to build a safe, friendly ("nice") AI.
Factual · 28 Feb 2018 |
AI safety and alignment |
| Not checkable |
Not checkable |
28 Feb 2018 |
I don’t think we’re looking at an economic problem.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
A coordination problem is where there’s a better way to do it, but you have to change more than one thing at a time.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
They’re not imagining enough changing simultaneously.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
I don’t think that’s true, but I’m worried about the sentiment behind it.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
It could be 50 years away. It could be two years away.
Not checkable · 28 Feb 2018 |
|
| Unresolvable |
Prediction |
28 Feb 2018 |
No AI development will ever look like a socially agreed, undeniable fire alarm for AGI.
Prediction · 28 Feb 2018 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
28 Feb 2018 |
The people at Google are not dumb enough to build an AI and tell it to make paperclips.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
that pursues arbitrary tractable goals. Including, for example, making paperclips.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
you can have arbitrarily powerful intelligence, with no defects of that intelligence
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
I wouldn’t say disparagement. These are just initial reactions. These are people we haven’t been talking to yet.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
The main way in which I would be worried about conscious systems emerging within the system without that happening on purpose would be if…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
But the three laws, I mean, they’re deontology. Decision theory requires quantitative weights on your goals.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
I would mostly say intelligence.org.
Not checkable · 28 Feb 2018 |
|
| Not researched |
Factual |
28 Feb 2018 |
Yudkowsky said the position he defends was named the orthogonality thesis by somebody he thought was Nick Bostrom.
Factual · 28 Feb 2018 |
AI safety and alignment |
| Not checkable |
Not checkable |
28 Feb 2018 |
Instrumental convergence! Which means that a lot of times, across a very broad range of final goals, there are similar strategies (we…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
We aren’t smart enough to treat the brain as something that from our perspective is a mechanical system and just navigate it to where you…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
It’s more about how to predict which parts of society will perform poorly to the point where you as an individual can manage to do better…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
And if you don’t have a business plan for this sort of thing, then you’re obviously just using it as an excuse.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
AlphaZero seems to me like a genuine case in point.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
We’re still on the same map. I agree with most of it.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
It is certainly possible to have agents such that as they get smarter, what they would report as their utility functions will change. A…
Not checkable · 28 Feb 2018 |
|
| Unresolvable |
Prediction |
28 Feb 2018 |
Once artificial general intelligence capabilities exist, something like AlphaZero going "from 0 to 120 mph in four hours or a day" is not out of the question.
Prediction · 28 Feb 2018 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
28 Feb 2018 |
I didn’t have any super-secret special trick that makes it all make sense in retrospect. I just did it the hard way.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
I mean, sort of crowdfunding, I guess, although the hammer currently isn’t very large.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
But this is a fairy tale. This is not a prediction.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
there seems to have been a very sharp jump in generality between chimpanzees and humans.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
The thing I’m worried about is that the way that natural selection does it—it’s not stable.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
The sentiments behind this are something that I have a great deal of sympathy for.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
But broadly speaking, it’s an AI where you can say what it’s trying to do.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
Those are plot devices. If they worked, Asimov would have had no stories.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
And that in turn is an example that refutes another thesis about how artificial general intelligence develops slowly and gradually
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
they couldn’t all jump to the new system because they couldn’t jump one at a time.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
The thing that makes something smarter than you dangerous is you cannot foresee everything it might try.
Not checkable · 28 Feb 2018 |
|
| Mostly true |
Factual |
28 Feb 2018 |
Thanks mostly to the cryptocurrency boom, MIRI is no longer short of funds and is instead short of engineering talent.
Factual · 28 Feb 2018 |
Field building and careers |
| Unresolvable |
Factual |
28 Feb 2018 |
Yudkowsky said he was pretty sure he was not the one who invented the term under discussion.
Factual · 28 Feb 2018 |
Other |
| Not checkable |
Not checkable |
28 Feb 2018 |
I’m not concerned that you’re going to teach the AI to deceive humans. I’m concerned that someone somewhere is going to get to the point of…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
the real story wasn’t in how AlphaGo beat human Go players. It’s in how AlphaZero beat human Go system programmers and human chess system…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
The system didn’t fit me that well, and I’m good at self-teaching.
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
This is the special case of the system that is smart in the way that you are smart and that mice are not smart.
Not checkable · 28 Feb 2018 |
|
| Not researched |
Factual |
28 Feb 2018 |
Asked whether the thought experiment originated with Bostrom, Eliezer Yudkowsky said that, as far as he knew, it originated with him.
Factual · 28 Feb 2018 |
Their own work or organisation |
| Not checkable |
Not checkable |
28 Feb 2018 |
The problem I’m worried about is that it’s technically difficult to get the AI to have a particular goal set and keep that goal set and…
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
The way I would rephrase the fact/values thing is: We all know about David Hume and Hume’s Razor
Not checkable · 28 Feb 2018 |
|
| Not researched |
Factual |
28 Feb 2018 |
AI researchers who forecast AGI about 30 years off have not argued that, once reached, AGI will develop much more slowly than AlphaZero did.
Factual · 28 Feb 2018 |
AI timelines and forecasting |