| Not checkable |
Not checkable |
28 Feb 2018 |
I think they’re wrong
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
28 Feb 2018 |
it seems to me that it is perfectly coherent to imagine an AI that is better at everything (or almost everything) than we are
Not checkable · 28 Feb 2018 |
|
| Not checkable |
Not checkable |
30 Jan 2013 |
Are you sure the same way of thinking wouldn’t prevent you from working on any important problem?
Not checkable · 30 Jan 2013 |
|
| Not checkable |
Not checkable |
30 Jan 2013 |
They didn’t work like that.
Not checkable · 30 Jan 2013 |
|
| Not checkable |
Not checkable |
30 Jan 2013 |
If you’re still like, “No, no, we must think more meta” after a year, then I don’t believe you’re the sort of person who will ever act.
Not checkable · 30 Jan 2013 |
|
| Not checkable |
Not checkable |
30 Jan 2013 |
When people constantly say “maybe we should do some other thing,” I would say, “Come to an interim decision, start acting on the interim…
Not checkable · 30 Jan 2013 |
|
| Not checkable |
Not checkable |
30 Jan 2013 |
If he then came up with some particular intervention which he thought was higher priority, I’d ask about the specific case.
Not checkable · 30 Jan 2013 |
|
| Not checkable |
Not checkable |
30 Jan 2013 |
Work on whatever your current priority is, after an hour of meta reasoning but not a year of meta reasoning.
Not checkable · 30 Jan 2013 |
|
| Not checkable |
Not checkable |
30 Jan 2013 |
Real computers don’t work very much like the formalism, but Turing’s work was useful anyway.
Not checkable · 30 Jan 2013 |
|
| Not checkable |
Not checkable |
30 Jan 2013 |
Nick Bostrom isn’t making this mistake, either.
Not checkable · 30 Jan 2013 |
|
| Not checkable |
Not checkable |
30 Jan 2013 |
No, this issue of trying to have an AI write a similar AI seems like a fairly fundamental one, and the Löb Problem blocks it.
Not checkable · 30 Jan 2013 |
|
| Not checkable |
Not checkable |
30 Jan 2013 |
But if you’re the person who always tries to go meta and only thinks meta because there might be some better thing, you’re not ever going…
Not checkable · 30 Jan 2013 |
|
| Not checkable |
Not checkable |
30 Jan 2013 |
I’d start by having them read a description of a particular technical problem we’re working on, for example the “Löb Problem.”
Not checkable · 30 Jan 2013 |
|
| Not checkable |
Not checkable |
30 Jan 2013 |
The fact that we can’t figure out how to do these things — even given infinite computing power — is alarming.
Not checkable · 30 Jan 2013 |
|
| Not checkable |
Not checkable |
30 Jan 2013 |
Are you sure you wouldn’t be going back in time and telling Alan Turing to not invent Turing machines because who knows whether computers…
Not checkable · 30 Jan 2013 |
|
| Not checkable |
Not checkable |
30 Jan 2013 |
I don’t have much objection to that.
Not checkable · 30 Jan 2013 |
|
| Not checkable |
Not checkable |
2008 |
I would be pleasantly surprised to be proven wrong about this.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
No system is secure that searches for ways to defeat its own security.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Wason's 2-4-6 task is a "cold" form of confirmation bias; people seek confirming but not falsifying evidence.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
it would be nice if we knew how to build a nice AI.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
This statement now seems to me like complete gibberish.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
This does not imply the AI has the motive to change its own motives.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
I have given you another opportunity to fail each time you face the challenge of changing your mind.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Such competence is not historically typical of human institutions, no matter how hard they try.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
However, ranking (8) as more probable than (6) violates the conjunction rule of probability theory which states that p(A & B) ≤ p(A).
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
By far the greatest danger of Artificial Intelligence is that people conclude too early that they understand it.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Thus, all else being equal, I would greatly prefer that Friendly AI precede nanotechnology in the ordering of technological developments.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Moore’s Law of Mad Science: Every eighteen months, the minimum IQ necessary to destroy the world drops by one point.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
I should not have had to stumble across that reference by accident.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The first AI must be Friendly.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
If an AI will hurt you given magic, the Friendliness architecture is wrong.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Lacking the space for exposition, I can only plead with the reader to consult the primary literature.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
We should recognize this as a literary criticism, not a scientific one; it is about good or bad novels, not good or bad hypotheses.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
All else being equal, not many people would prefer to destroy the world.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
I am skeptical that a scholar of heuristics and biases, unable to read physics equations, could check the work of physicists who knew…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Artificial Intelligence is not settled science; it belongs to the frontier, not to the textbook.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
There is no guarantee whatsoever that, just because we can’t manage a risk, the risk will obligingly go away.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
If we postulate that future minds exhibit the same mixture of foolishness and wisdom, the same mixture of heroism and selfishness, as the…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
We might survive for another decade, even another century, but not another million years.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
But if combinatorics could not correctly predict that a lottery ticket has a 10-8 chance of winning, ticket sellers would go broke.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
It would be, not physically impossible, but cognitively impossible.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The same thing could as easily happen to the whole human species, if the human species runs into an unfair challenge.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
It is a risky intellectual endeavor to predict specifically how a benevolent AI would help humanity, or an unfriendly AI harm it.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Philosophical failure is trying to build the wrong thing, so that even if you succeeded you would still fail to help anyone or benefit…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
minds do not blindly wander into attractors, they have motives.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
If you want to know why others aren't responding to an emergency, before you respond yourself, you may have just answered your own question.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Friendly AI, as I have proposed it, requires repeated cycles of recursive self-improvement that precisely preserve a stable optimization…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
If the AI wants to modify itself to be no longer Friendly, then Friendliness has already failed, at the point when the AI forms that…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
I would not knowingly swallow a pill that made me enjoy committing murder, because currently I prefer that my fellow humans not die.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Hominids have survived this long only because, for the last million years, there were no arsenals of hydrogen bombs, no spaceships to steer…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The catastrophic scenario which stems from underestimating the power of intelligence is that someone builds a button, and doesn’t care…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
I don't know how to solve this problem.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
You trust that the authors didn't unconsciously look just a little bit harder for mistakes in equations that seemed to be leaning the wrong…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The bystander effect is not about individual selfishness, or insensitivity to the suffering of others.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
It's harder to abuse heuristics and biases than psychoanalysis.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Ordinarily, local strategies are most plausible.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
In practice it often takes a deep technical understanding to figure out what you want.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Evolutionary biology offers both mathematics and case studies to help hammer out anthropomorphic bias.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
I can predict the outcome of a process, without being able to predict any of the intermediate steps in the process.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
If I am selective about which arguments I inspect for errors, or even how hard I inspect for errors, then every new rule of rationality I…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
A fast, nice intelligence wielding molecular nanotechnology is power on the order of getting rid of disease, not getting rid of cancer.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
If so, then by making you a more sophisticated arguer - by teaching you another bias of which to accuse people - I have actually harmed…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Reality throws at you one challenge after another, and when you run into a challenge you can’t handle, you suffer the consequences.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The one whom you must watch above all is yourself.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
As soon as a milestone is actually achieved, it ceases to be “AI.” This can only encourage procrastination.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
If we want people who can make progress on Friendly AI, then they have to start training themselves, full-time, years before they are…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
A good book for quickly picking up a sense of the field.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
You may not drive; you may not fly; you may not walk across the street even after looking both ways, for it is still too great a risk.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
It seems like an unfair challenge.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Rather, the representativeness heuristic is what produces the intuitive sense that sequence 2 "seems more likely" than sequence 1.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
No more than Albert Szent-Györgyi could multiply the suffering of one human by a hundred million can I truly understand the value of clear…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The folly of programming an AI to implement communism, or any other political system, is that you’re programming means instead of ends.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
With luck, future historians will look back and describe the present world as an awkward in-between stage of adolescence, when humankind…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Now it seems to me that an AI capable of language, abstract thought, creativity, environmental interaction, originality, prediction…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Each and every absence of knowledge dates back to the dawn of human curiosity; and the hole lasts through the ages, seemingly eternal…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
I strongly urge my readers not to start thinking up reasons why a fully generic optimization process would be friendly.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Because human engineers deliberately use designs that they can understand.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
the ability to self-improve does not imply the choice to do so.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
We evaluate descriptions of events, not extensions of events.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
It is acceptable to fail at AI and at Friendly AI. It is acceptable to succeed at AI and at Friendly AI.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
But obviously it was not that improbable.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
It is disrespectful to human ingenuity to declare a challenge unsolvable without taking a close look and exercising creativity.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Nature is, not cruel, but indifferent; a neutrality which often seems indistinguishable from outright hostility.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Yet before we can pass out of that stage of adolescence, we must, as adolescents, confront an adult problem: the challenge of…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Even so, I suspect that, pragmatically speaking, our alternatives boil down to becoming smarter or becoming extinct.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
I don’t know. I’m not that smart.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
But that's a technical question, not a matter of taste; no amount of psychologizing can tell you the exact slope of that curve.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The point about underestimating the potential impact of Artificial Intelligence is symmetrical around potential good impacts and potential…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
A prospective risk going from 10,000,000 deaths to 100,000,000 deaths does not multiply by ten the strength of our determination to stop it.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Halting or relinquishing an undesirable technology tends to require an impossible unanimous strategy.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
I call this mix of anchoring and availability the logical fallacy of generalization from fictional evidence.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
If you are well-calibrated, then on a test with one hundred such questions, around 2 questions will have answers that fall outside your 98%…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
All scientific ignorance is hallowed by ancientness.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The moral may be that once you can guess what your answer will be - once you can assign a greater probability to your answering one way…
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Despite all dangers and temptations, it is better to know about psychological biases than to not know.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
It is important to understand that the authors of huge catastrophes need not be evil, nor even unusually stupid.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
So if we find that we cannot make progress on Friendly AI, then we need to figure out how to exit that regime as fast as possible!
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
The calibration research is dangerous knowledge - so tempting to apply selectively.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Pluralistic ignorance can explain, and individual selfishness cannot explain, subjects failing to react to a room filling up with smoke.
Not checkable · 2008 |
|
| Not checkable |
Not checkable |
2008 |
Artificial Intelligence does not belong in the same graph that shows progress in medicine, manufacturing, and energy.
Not checkable · 2008 |
|