| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
I would not knowingly swallow a pill that made me enjoy committing murder, because currently I prefer that my fellow humans not die.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Hominids have survived this long only because, for the last million years, there were no arsenals of hydrogen bombs, no spaceships to steer…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
The catastrophic scenario which stems from underestimating the power of intelligence is that someone builds a button, and doesn’t care…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
I don't know how to solve this problem.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
You trust that the authors didn't unconsciously look just a little bit harder for mistakes in equations that seemed to be leaning the wrong…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
The bystander effect is not about individual selfishness, or insensitivity to the suffering of others.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
It's harder to abuse heuristics and biases than psychoanalysis.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Ordinarily, local strategies are most plausible.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
In practice it often takes a deep technical understanding to figure out what you want.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Evolutionary biology offers both mathematics and case studies to help hammer out anthropomorphic bias.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
I can predict the outcome of a process, without being able to predict any of the intermediate steps in the process.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
If I am selective about which arguments I inspect for errors, or even how hard I inspect for errors, then every new rule of rationality I…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
A fast, nice intelligence wielding molecular nanotechnology is power on the order of getting rid of disease, not getting rid of cancer.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
If so, then by making you a more sophisticated arguer - by teaching you another bias of which to accuse people - I have actually harmed…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Reality throws at you one challenge after another, and when you run into a challenge you can’t handle, you suffer the consequences.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
The one whom you must watch above all is yourself.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
As soon as a milestone is actually achieved, it ceases to be “AI.” This can only encourage procrastination.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
If we want people who can make progress on Friendly AI, then they have to start training themselves, full-time, years before they are…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
A good book for quickly picking up a sense of the field.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
You may not drive; you may not fly; you may not walk across the street even after looking both ways, for it is still too great a risk.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
It seems like an unfair challenge.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Rather, the representativeness heuristic is what produces the intuitive sense that sequence 2 "seems more likely" than sequence 1.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
No more than Albert Szent-Györgyi could multiply the suffering of one human by a hundred million can I truly understand the value of clear…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
The folly of programming an AI to implement communism, or any other political system, is that you’re programming means instead of ends.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
With luck, future historians will look back and describe the present world as an awkward in-between stage of adolescence, when humankind…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Now it seems to me that an AI capable of language, abstract thought, creativity, environmental interaction, originality, prediction…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Each and every absence of knowledge dates back to the dawn of human curiosity; and the hole lasts through the ages, seemingly eternal…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
I strongly urge my readers not to start thinking up reasons why a fully generic optimization process would be friendly.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Because human engineers deliberately use designs that they can understand.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
the ability to self-improve does not imply the choice to do so.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
We evaluate descriptions of events, not extensions of events.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
It is acceptable to fail at AI and at Friendly AI. It is acceptable to succeed at AI and at Friendly AI.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
But obviously it was not that improbable.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
It is disrespectful to human ingenuity to declare a challenge unsolvable without taking a close look and exercising creativity.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Nature is, not cruel, but indifferent; a neutrality which often seems indistinguishable from outright hostility.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Yet before we can pass out of that stage of adolescence, we must, as adolescents, confront an adult problem: the challenge of…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Even so, I suspect that, pragmatically speaking, our alternatives boil down to becoming smarter or becoming extinct.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
I don’t know. I’m not that smart.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
But that's a technical question, not a matter of taste; no amount of psychologizing can tell you the exact slope of that curve.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
The point about underestimating the potential impact of Artificial Intelligence is symmetrical around potential good impacts and potential…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
A prospective risk going from 10,000,000 deaths to 100,000,000 deaths does not multiply by ten the strength of our determination to stop it.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Halting or relinquishing an undesirable technology tends to require an impossible unanimous strategy.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
I call this mix of anchoring and availability the logical fallacy of generalization from fictional evidence.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
If you are well-calibrated, then on a test with one hundred such questions, around 2 questions will have answers that fall outside your 98%…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
All scientific ignorance is hallowed by ancientness.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
The moral may be that once you can guess what your answer will be - once you can assign a greater probability to your answering one way…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Despite all dangers and temptations, it is better to know about psychological biases than to not know.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
It is important to understand that the authors of huge catastrophes need not be evil, nor even unusually stupid.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
So if we find that we cannot make progress on Friendly AI, then we need to figure out how to exit that regime as fast as possible!
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
The calibration research is dangerous knowledge - so tempting to apply selectively.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Pluralistic ignorance can explain, and individual selfishness cannot explain, subjects failing to react to a room filling up with smoke.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Artificial Intelligence does not belong in the same graph that shows progress in medicine, manufacturing, and energy.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
When using long, complex chains of reasoning to argue in favor of an anthropomorphic conclusion, each and every step of the reasoning is…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
To get a really good answer you must sacrifice your ability to predict the answer, albeit not your ability to say what is the question.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Vivid, specific scenarios can inflate our probability estimates of security, as well as misdirecting defensive investments into needlessly…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
The glorious thing about formal mathematical proof, is that a proof of ten billion steps is just as reliable as a proof of ten steps.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
We would last only until the probabilities caught up with us.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Such is the power of creativity.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
It is no law of Nature that physical operations must crawl at the pace of long seconds.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
“Fairly good” is not good enough to last another million years.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Sufficiently tall skyscrapers don’t potentially start doing their own engineering.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
If we suppose the g-factor metaphor, then global catastrophic risks of this scenario are relatively mild; a hostile AI is not much more of…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Anthropomorphic bias can be classed as insidious: it takes place with no deliberate intent, without conscious realization, and in the face…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
To improve the truth of our beliefs we must change our beliefs.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
A success on human intelligence enhancement would make Friendly AI easier, and also help on other technologies.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
If you throw too much money at a problem that a field is not prepared to solve, the excess money is more likely to produce anti-science…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Molecular nanotechnology fits this criterion, first because its elementary operations are fast, and second because there exists a ready…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
The prospect of AI interacts with global risk in more complex ways than that; if AI were a pure liability, matters would be simple.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
All that vast potential hinges on our survival here, now, in the days when the realm of humankind is a single planet orbiting a single star.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
EP is a powerful, still maturing technique that is intrinsically unsuited to the demands of Friendly AI.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
An optimization process steers the future into particular regions of the possible.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
But if people are liable to confidently believe that the button does something quite different from its actual consequence, that is cause…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
I don’t think this exact scenario is plausible, mostly because I don’t expect the curve of recursive self-improvement to move at a linear…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
There is a taboo against talking about humanlevel capabilities.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Nature does not match her challenges to your skill, or your resources, or how much free time you have to think about the problem.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Storytellers obey strict rules of narrative unrelated to reality.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Any technology which increases available computing power decreases the minimum theoretical sophistication necessary to develop Artificial…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Of all possible configurations for the atoms making up the Corolla, only an infinitesimal fraction qualify as a useful working car.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Yet for a futurist, disjunctions make for an awkward and unpoetic-sounding prophecy.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
The danger of confusing general intelligence with g-factor is that it leads to tremendously underestimating the potential impact of…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
The subjects in these experiments don't make mistakes on purpose; they make mistakes because they don't know how to do better.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
To build an AI that discovers the orbits of the planets, the programmers need not know the math of Newtonian mechanics, only the math of…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
The above methodology for expert elicitation looks perfectly reasonable
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
This may have been one contributing factor to the embarrassing history of AI, which stems not from the difficulty of AI as such, but from the…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
The scenario of humanity going extinct in the next century is a disjunctive event.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Since the given description of Linda was chosen to be similar to a feminist and dissimilar to a bank teller, (8) is more representative of…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
g-factor metaphors seem most common in popular futurism: when people think of “intelligence” they think of human geniuses instead of humans.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
The second threat scenario is less vivid and alarming, but the defense is more useful because it is more vague.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Natural selection isn’t friendly, nor does it hate you, nor will it leave you alone.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Often Nature poses requirements that are grossly unfair, even on tests where the penalty for failure is death.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
But human beings are not trustworthy to peer over a purported proof of ten billion steps; we have too high a chance of missing an error.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
We must execute the creation of Artificial Intelligence as the exact application of an exact art.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
People may be overconfident and over-optimistic.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Awareness of human fallibility is a dangerous knowledge, if you remind yourself of the fallibility of those who disagree with you.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
The first moral is that confusing the speed of AI research with the speed of a real AI once built is like confusing the speed of physics…
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
A majoritarian strategy only works if it is not possible for a single defector to cause global catastrophic damage.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
And when you run into a lethal challenge too difficult for you, you die.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
Every true idea which discomforts you will seem to match the pattern of at least one psychological error.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
The main advice the metaphor gives us is that we had better get Friendly AI right, which is good advice in any case.
Eliezer Yudkowsky · Not checkable · 2008 |
|
| Not checkable |
Eliezer Yudkowsky | Not checkable |
2008 |
But in forecasting, every added detail is an extra burden!
Eliezer Yudkowsky · Not checkable · 2008 |
|