| Not checkable |
Not checkable |
2008 |
Anthropomorphic bias can be classed as insidious: it takes place with no deliberate intent, without conscious realization, and in the face…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
To improve the truth of our beliefs we must change our beliefs.
Not checkable · 2008 |
|
| Resolved true |
Factual |
2008 |
Biases implicit in the availability heuristic affect people's estimates of risk.
Factual · 2008 |
Science and research methods |
| Not checkable |
Not checkable |
2008 |
A success on human intelligence enhancement would make Friendly AI easier, and also help on other technologies.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
If you throw too much money at a problem that a field is not prepared to solve, the excess money is more likely to produce anti-science…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Molecular nanotechnology fits this criterion, first because its elementary operations are fast, and second because there exists a ready…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The prospect of AI interacts with global risk in more complex ways than that; if AI were a pure liability, matters would be simple.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
All that vast potential hinges on our survival here, now, in the days when the realm of humankind is a single planet orbiting a single star.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
EP is a powerful, still maturing technique that is intrinsically unsuited to the demands of Friendly AI.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
An optimization process steers the future into particular regions of the possible.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
But if people are liable to confidently believe that the button does something quite different from its actual consequence, that is cause…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
I don’t think this exact scenario is plausible, mostly because I don’t expect the curve of recursive self-improvement to move at a linear…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
There is a taboo against talking about humanlevel capabilities.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Nature does not match her challenges to your skill, or your resources, or how much free time you have to think about the problem.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Storytellers obey strict rules of narrative unrelated to reality.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Any technology which increases available computing power decreases the minimum theoretical sophistication necessary to develop Artificial…
Not checkable · 2008 |
|
| Unresolvable |
Prediction |
2008 |
Yudkowsky said he thinks Artificial Intelligence is the road we will end up taking into "that challenge", which the surrounding text describes as the way out of the high-mortality phase of the life cycle and probably the single most dangerous risk we face.
Prediction · 2008 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
2008 |
Of all possible configurations for the atoms making up the Corolla, only an infinitesimal fraction qualify as a useful working car.
Not checkable · 2008 |
|
| Not researched |
Citation |
2008 |
Latane and Darley's 1969 experiments found the bystander effect: larger groups are less likely to act in emergencies, individually and collectively.
Citation · 2008 |
Science and research methods |
| Not checkable |
Not checkable |
2008 |
Yet for a futurist, disjunctions make for an awkward and unpoetic-sounding prophecy.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The danger of confusing general intelligence with g-factor is that it leads to tremendously underestimating the potential impact of…
Not checkable · 2008 |
|
| Unresolvable |
Prediction |
2008 |
When technology first becomes capable of uploading, it will have overwhelmingly more computing power and probably far better cognitive science than building an AI requires.
Prediction · 2008 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
2008 |
The subjects in these experiments don't make mistakes on purpose; they make mistakes because they don't know how to do better.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
To build an AI that discovers the orbits of the planets, the programmers need not know the math of Newtonian mechanics, only the math of…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The above methodology for expert elicitation looks perfectly reasonable
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
This may have been one contributing factor to the embarrassing history of AI, which stems not from the difficulty of AI as such, but from the…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The scenario of humanity going extinct in the next century is a disjunctive event.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Since the given description of Linda was chosen to be similar to a feminist and dissimilar to a bank teller, (8) is more representative of…
Not checkable · 2008 |
|
| Not researched |
Factual |
2008 |
Discussions of existential risk not uncommonly draw their categories, choices, consequences and strategies from movies, books and television shows.
Factual · 2008 |
Other global risks |
| Not researched |
Factual |
2008 |
Later research has generalized and elaborated the phenomena in the Kahneman, Slovic and Tversky volume, and its basic results still stand.
Factual · 2008 |
Science and research methods |
| Not researched |
Factual |
2008 |
Extreme confidence in wrong answers is common in the heuristics and biases literature, where it is known as overconfidence.
Factual · 2008 |
Science and research methods |
| Not checkable |
Not checkable |
2008 |
g-factor metaphors seem most common in popular futurism: when people think of “intelligence” they think of human geniuses instead of humans.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The second threat scenario is less vivid and alarming, but the defense is more useful because it is more vague.
Not checkable · 2008 |
|
| Mostly true |
Factual |
2008 |
As of 2008 there is no shield that can deflect a nuclear explosion.
Factual · 2008 |
Other global risks |
| Not checkable |
Not checkable |
2008 |
Natural selection isn’t friendly, nor does it hate you, nor will it leave you alone.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Often Nature poses requirements that are grossly unfair, even on tests where the penalty for failure is death.
Not checkable · 2008 |
|
| Unresolvable |
Prediction |
2008 |
Yudkowsky said he feared regulatory agencies would not approve human experiments, in the context of augmenting human intelligence through biology.
Prediction · 2008 |
AI policy and governance |
| Not checkable |
Not checkable |
2008 |
But human beings are not trustworthy to peer over a purported proof of ten billion steps; we have too high a chance of missing an error.
Not checkable · 2008 |
|
| Not researched |
Factual |
2008 |
Taking Piracetam or drinking caffeine will not make a person substantially smarter than Einstein.
Factual · 2008 |
Science and research methods |
| Not researched |
Citation |
2008 |
By the halting problem (Rice 1953), it is impossible to predict whether an arbitrary computational system implements any given input-output function, including simple multiplication.
Citation · 2008 |
Science and research methods |
| Not checkable |
Not checkable |
2008 |
We must execute the creation of Artificial Intelligence as the exact application of an exact art.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
People may be overconfident and over-optimistic.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Awareness of human fallibility is a dangerous knowledge, if you remind yourself of the fallibility of those who disagree with you.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The first moral is that confusing the speed of AI research with the speed of a real AI once built is like confusing the speed of physics…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
A majoritarian strategy only works if it is not possible for a single defector to cause global catastrophic damage.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
And when you run into a lethal challenge too difficult for you, you die.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Every true idea which discomforts you will seem to match the pattern of at least one psychological error.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The main advice the metaphor gives us is that we had better get Friendly AI right, which is good advice in any case.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
But in forecasting, every added detail is an extra burden!
Not checkable · 2008 |
|
| Unresolvable |
Prediction |
2008 |
A superintelligent AI whose goal system operates in an unforeseen fashion may actively resist human attempts to correct it.
Prediction · 2008 |
AI safety and alignment |
| Unresolvable |
Factual |
2008 |
It seems that people do not extrapolate from experienced small hazards to the possibility of large risks.
Factual · 2008 |
Philosophy, ethics and society |
| Unresolvable |
Factual |
2008 |
The obvious experimental manipulations of heuristics-and-biases studies have already been tried, and the results were found to be robust.
Factual · 2008 |
Science and research methods |
| Not researched |
Citation |
2008 |
In the Buehler et al. study, students took on average 55 days to complete their thesis, 22 days longer than anticipated and 7 days longer than their worst-case predictions.
Citation · 2008 |
Science and research methods |
| Not checkable |
Not checkable |
2008 |
The elapsed turnaround time would be, imaginably, on the order of a week from when the fast intelligence first became able to solve the…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Familiarity with heuristics and biases can enable us to detect a wide variety of logical flaws that might otherwise evade our inspection.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
A majoritarian strategy is sometimes workable, if you have decades in which to do your work.
Not checkable · 2008 |
|
| Not researched |
Factual |
2008 |
The two Gilovich, Griffin and Kahneman edited volumes overview the heuristics and biases field in its current form.
Factual · 2008 |
Science and research methods |
| Not checkable |
Not checkable |
2008 |
As logic, the above argument contains holes.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Few people would push a button that they clearly knew would cause a global catastrophe.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
I introduce the concept of an optimization process
Not checkable · 2008 |
|
| Not researched |
Factual |
2008 |
Yudkowsky wrote that, to the best of his ability to discern, there were no existing major works on the topic to cite as of January 2006.
Factual · 2008 |
AI safety and alignment |
| Not checkable |
Not checkable |
2008 |
Because of hindsight bias, we learn overly specific lessons.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
We would expect a true AI, an Artificial General Intelligence, to be capable of changing its empirical beliefs
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
And the stakes of existential risk extend beyond even the six billion humans alive today, to all the stars in all the galaxies that…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
But if they had said might, I would have no objection.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The raw materials for AI are already everywhere.
Not checkable · 2008 |
|
| Resolved true |
Prediction |
2008 |
Yudkowsky doubted that a thousandfold increase in AI hardware would buy anything like a thousandfold increase in effective intelligence, while noting that such doubt is not reliable without an analytical calculation.
Prediction · 2008 |
AI compute, hardware and data |
| Not checkable |
Not checkable |
2008 |
If the AI tribe wins in military conflict and wipes out the humans, that is an existential catastrophe of the Bang variety
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Substantially larger numbers, such as 500 million deaths, and especially qualitatively different scenarios such as the extinction of the…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
I do not assign strong confidence to the assertion that Friendly AI is easier than human augmentation, or that it is safer.
Not checkable · 2008 |
|
| Not researched |
Factual |
2008 |
Building dams reduces flood frequency but damage per flood becomes so much greater that average yearly damage increases.
Factual · 2008 |
Economics and markets |
| Not checkable |
Not checkable |
2008 |
It only takes a single counterexample to falsify a universal quantifier.
Not checkable · 2008 |
|
| Not researched |
Factual |
2008 |
It would be physically possible to build a brain computing a million times faster than a human brain without shrinking it, cooling it, or using reversible or quantum computing.
Factual · 2008 |
Science and research methods |
| Not checkable |
Not checkable |
2008 |
They are somewhat less accessible to a general audience.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Otherwise you become slower to react to incoming evidence.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
But one should not so quickly write off the challenge, especially considering the stakes.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Intelligence is the foundation of human power, the strength that fuels our other arts.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
When we talk about “AIs” we are really talking about minds-in-general, or optimization processes in general.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
A dead planet, lifelessly orbiting its star, is also stable.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
But the stakes are much, much higher.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Absolute power is a conservative engineering assumption in Friendly AI, exposing broken designs.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
I cannot see how nanotechnology would make it easier to develop Friendly AI.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
There is no way to obtain more truth for a proposition by bribery, flattery, or the most passionate argument - you can make more people…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Friendly AI is not about brute-forcing the problem.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Any two AI designs might be less similar to one another than you are to a petunia.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The AI might transcend the factual error of its programmers, provided that the programmers understood inference rather better than they…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
A technical failure occurs when the code does not do what you think it does, though it faithfully executes as you programmed it.
Not checkable · 2008 |
|
| Not researched |
Factual |
2008 |
Overconfidence applies strongly to planning, where it is known as the planning fallacy.
Factual · 2008 |
Science and research methods |
| Not researched |
Factual |
2008 |
Spearman's g is a statistical abstraction from individual differences in intelligence among humans, a species far more intelligent than lizards.
Factual · 2008 |
Science and research methods |
| Unresolvable |
Prediction |
2008 |
Yudkowsky said he thinks it will prove easier, in the end, to build a 747 from scratch than to scale up an existing bird or graft on jet engines, an analogy he used for building Artificial Intelligence rather than taking another road into the challenge of smarter-than-human intelligence.
Prediction · 2008 |
AI timelines and forecasting |
| Unresolvable |
Prediction |
2008 |
With molecular nanotechnology, an AI could potentially rewrite the solar system unopposed.
Prediction · 2008 |
Other global risks |
| Not checkable |
Not checkable |
2008 |
To live even a million years, as an unaging human in a world as risky as our own, you must somehow drive your annual probability of…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Someone should know that when an expert says the probability is "a million to one" without using actuarial data or calculations from a…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The first AI built cannot by itself do catastrophic damage.
Not checkable · 2008 |
|
| Own records only |
Factual |
2008 |
When he first began thinking about these matters, Yudkowsky expected the largest benefits from Artificial Intelligence yet felt no anxiety about it.
Factual · 2008 |
Their own work or organisation |
| Not checkable |
Not checkable |
2008 |
It once occurred to me that modern civilization occupies an unstable state.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Heuristics and biases are widespread in human reasoning.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Friendly AI programmers should assume that the AI has total access to its own source code.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Do-What-I-Mean is a major, nontrivial technical challenge of Friendly AI.
Not checkable · 2008 |
|