| Not researched |
Factual |
Sep 2025 |
In his assessment, GPT-4 does not have the obvious mode of building a smarter version of itself and exploding in capability, so it is not the kind of AI meant by "if anyone builds it, everyone dies."
Factual · Sep 2025 |
AI capabilities |
| Not researched |
Factual |
Sep 2025 |
Yudkowsky said he was, as far as he knew, the person who originally made the argument under discussion, and that it was his position in 1996.
Factual · Sep 2025 |
Their own work or organisation |
| Not researched |
Factual |
Sep 2025 |
A reactor at prompt critical does not simply melt down; it explodes and vaporizes.
Factual · Sep 2025 |
Other global risks |
| Not researched |
Factual |
Sep 2025 |
Ribosomes can synthesize any kind of protein, not just those found in grass.
Factual · Sep 2025 |
Biosecurity |
| Not researched |
Factual |
Sep 2025 |
GPT-4 is not an AI whose building would cause everyone to die.
Factual · Sep 2025 |
AI capabilities |
| Not researched |
Factual |
Sep 2025 |
Most proteins in biological tissue are held together by relatively weak forces.
Factual · Sep 2025 |
Science and research methods |
| Not researched |
Factual |
Sep 2025 |
There are many historical cases of people who had an early grasp of a scientific principle and correctly predicted where it would eventually lead.
Factual · Sep 2025 |
Science and research methods |
| Not researched |
Factual |
Sep 2025 |
Starting in 2000, Eliezer Yudkowsky obtained enough funding to work full-time on AI alignment.
Factual · Sep 2025 |
Their own work or organisation |
| Not researched |
Factual |
Sep 2025 |
Eliezer Yudkowsky has sometimes said that the keynote question of rationality is what do you think you know, and how do you think you know it.
Factual · Sep 2025 |
Field building and careers |
| Not researched |
Factual |
Sep 2025 |
In 2000 Eliezer Yudkowsky believed that a very smart AI would automatically end up very nice.
Factual · Sep 2025 |
Their own work or organisation |
| Not researched |
Factual |
Sep 2025 |
AI systems that are very good at chess have not destroyed the world.
Factual · Sep 2025 |
AI capabilities |
| Not researched |
Factual |
Sep 2025 |
The written book does not cover all objections to its argument; some additional objections are considered in an online supplement.
Factual · Sep 2025 |
Their own work or organisation |
| Not researched |
Factual |
Sep 2025 |
The speaker says that the person he is describing, who "saw through from there to nuclear weapons", saw that he should not publish his idea because it was a bit dangerous.
Factual · Sep 2025 |
Science and research methods |
| Not researched |
Factual |
Sep 2025 |
Yudkowsky said the very first place he carved out an exception to a general view of his was in the discussion about nanotechnology; the surrounding text does not say what that view was.
Factual · Sep 2025 |
Their own work or organisation |
| Not researched |
Factual |
Sep 2025 |
A delayed-critical nuclear reactor receiving 0.65% more neutrons would literally explode, not just melt down.
Factual · Sep 2025 |
Other global risks |
| Not researched |
Factual |
Sep 2025 |
AI has been used to optimize social media algorithms for outrage and clicks, and arguably contributed to political shifts involving a collapse of the use of expertise in government institutions.
Factual · Sep 2025 |
AI policy and governance |
| Not researched |
Factual |
Sep 2025 |
Leo Szilard foresaw that Hitler was likely to be a problem.
Factual · Sep 2025 |
Science and research methods |
| Not researched |
Factual |
Sep 2025 |
Leo Szilard did not foresee that the first atomic bomb would be dropped in August 1945.
Factual · Sep 2025 |
Science and research methods |
| Not researched |
Factual |
Sep 2025 |
Diamond is made out of carbon.
Factual · Sep 2025 |
Science and research methods |
| Not researched |
Factual |
6 Apr 2023 |
Text on the internet includes material produced not by humans conversing but by extensive work extracting experimental results from reality.
Factual · 6 Apr 2023 |
Science and research methods |
| Not researched |
Factual |
6 Apr 2023 |
Yudkowsky said he was not familiar with anybody who predicted, in advance of the observation, the qualitative jumps in ability that corresponded to smoothly declining loss on text prediction.
Factual · 6 Apr 2023 |
AI timelines and forecasting |
| Not researched |
Factual |
6 Apr 2023 |
Using PCR, a person can currently make a gallon jar full of copies of their own DNA.
Factual · 6 Apr 2023 |
Biosecurity |
| Not researched |
Factual |
6 Apr 2023 |
Yudkowsky said the LessWrong sequences were part of an effort he "really, really tried" to make, and added that they also "had other purposes".
Factual · 6 Apr 2023 |
Field building and careers |
| Not researched |
Factual |
6 Apr 2023 |
In 2019 and 2020 AI progress slowed somewhat, leaving more time than Yudkowsky had feared.
Factual · 6 Apr 2023 |
AI timelines and forecasting |
| Not researched |
Factual |
6 Apr 2023 |
MIRI did not advertise the Visible Thoughts Project to people as a way to save the world, but as a tiny shred of hope.
Factual · 6 Apr 2023 |
Their own work or organisation |
| Not researched |
Factual |
6 Apr 2023 |
In 2015, 2016 and 2017 Eliezer Yudkowsky noticed he had repeatedly been surprised by AI progress moving faster than he anticipated.
Factual · 6 Apr 2023 |
AI timelines and forecasting |
| Not researched |
Factual |
6 Apr 2023 |
As AI programs have become simpler, humans have less insight into them and their content is more opaque, as with AlphaZero.
Factual · 6 Apr 2023 |
AI safety and alignment |
| Not researched |
Factual |
6 Apr 2023 |
As of April 2023, AI developers are not being careful and deliberate in designing AI systems.
Factual · 6 Apr 2023 |
AI safety and alignment |
| Not researched |
Factual |
6 Apr 2023 |
AI algorithms are continuing to improve.
Factual · 6 Apr 2023 |
AI capabilities |
| Not researched |
Factual |
6 Apr 2023 |
Eliezer Yudkowsky said he tried very hard to replace himself.
Factual · 6 Apr 2023 |
Field building and careers |
| Not researched |
Factual |
6 Apr 2023 |
Interpretability researchers are working on models smaller than GPT-2.
Factual · 6 Apr 2023 |
AI safety and alignment |
| Not researched |
Factual |
6 Apr 2023 |
Yudkowsky has not solved the AI alignment problem.
Factual · 6 Apr 2023 |
Their own work or organisation |
| Not researched |
Factual |
6 Apr 2023 |
Current RLHF-trained language models ended up as a weird corporate-speak, left-leaning-sounding chatbot persona.
Factual · 6 Apr 2023 |
AI capabilities |
| Not researched |
Factual |
6 Apr 2023 |
Neither Yudkowsky nor the signatories of the open letter say that current AI systems are already at a dangerous point.
Factual · 6 Apr 2023 |
AI policy and governance |
| Not researched |
Factual |
6 Apr 2023 |
Simply stacking more layers in neural networks achieved general capabilities first, without specialized systems.
Factual · 6 Apr 2023 |
AI capabilities |
| Not researched |
Factual |
6 Apr 2023 |
Researchers understood AlphaZero's goals much better than they understand large language models' goals.
Factual · 6 Apr 2023 |
AI safety and alignment |
| Not researched |
Factual |
30 Mar 2023 |
Humans develop emotions even if they are not told about emotions as children.
Factual · 30 Mar 2023 |
Philosophy, ethics and society |
| Not researched |
Factual |
30 Mar 2023 |
Yudkowsky said 'tiny molecular paperclips', which was later altered to 'paperclips' and then to an AI in a paperclip factory.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not researched |
Factual |
30 Mar 2023 |
Humans came from evolutionary computation, i.e. evolution.
Factual · 30 Mar 2023 |
Science and research methods |
| Not researched |
Factual |
30 Mar 2023 |
Yudkowsky said he pioneered "the thing that appears when you hover over it" and so gets some credit for that user experience, while noting that a team of developers also gets credit.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not researched |
Factual |
30 Mar 2023 |
The COVID pandemic killed a few million people, and there was definitely a cover-up regarding whether it came from a lab leak.
Factual · 30 Mar 2023 |
Biosecurity |
| Not researched |
Factual |
30 Mar 2023 |
Despite full read access to GPT's numbers, we know vastly more about the architecture of human thinking than about what goes on inside GPT.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not researched |
Factual |
30 Mar 2023 |
Yudkowsky said he believes the name Less Wrong was suggested by Nick Bostrom.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not researched |
Factual |
30 Mar 2023 |
Natural selection has to run for hundreds of generations to notice that a trait is working.
Factual · 30 Mar 2023 |
Other |
| Not researched |
Factual |
30 Mar 2023 |
Yudkowsky said he did not think that, as of March 2023, a prompted AI system could successfully argue for its own consciousness before the US Supreme Court.
Factual · 30 Mar 2023 |
Philosophy, ethics and society |
| Not researched |
Factual |
30 Mar 2023 |
The vast majority of randomly specified utility functions do not have optima that include humans.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not researched |
Factual |
30 Mar 2023 |
AI research went through a long process in which researchers thought AI would be easier than it turned out to be.
Factual · 30 Mar 2023 |
AI capabilities |
| Not researched |
Factual |
30 Mar 2023 |
There is no general law saying that a system trained by hill climbing on a simple loss function internally represents, or tries to optimize, that loss function once its capabilities generalize widely.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not researched |
Factual |
30 Mar 2023 |
Humans have significantly more generally applicable intelligence than chimpanzees, their closest living relatives.
Factual · 30 Mar 2023 |
Philosophy, ethics and society |
| Not researched |
Factual |
30 Mar 2023 |
When inabilities are trained into a system that already has capabilities, gradient descent learns only shallow patches, and the deep capabilities come back out when prompted differently, e.g. in another language.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not researched |
Factual |
30 Mar 2023 |
Natural selection is a weaker optimizer than gradient descent because gradient descent also uses derivative information.
Factual · 30 Mar 2023 |
Other |
| Not researched |
Factual |
30 Mar 2023 |
Some people have started arguing over whether GPT-4 counts as AGI.
Factual · 30 Mar 2023 |
AI capabilities |
| Not researched |
Factual |
30 Mar 2023 |
Yudkowsky's model of AI did not rule out that gradient descent could produce intelligence without understanding of how it works.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not researched |
Factual |
30 Mar 2023 |
Yudkowsky said he thinks Wikipedia picked up "the hover thing" from a prototype of a different system he had put forward, while allowing that Wikipedia may have developed it independently.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not researched |
Factual |
30 Mar 2023 |
Versions of AI language models are already claiming to be conscious.
Factual · 30 Mar 2023 |
Philosophy, ethics and society |
| Not researched |
Factual |
30 Mar 2023 |
The model was exposed to a vast corpus of internet text about people discussing consciousness, so it is not known to what extent its talk of being self-aware repeats what it was trained on.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not researched |
Factual |
30 Mar 2023 |
People are already claiming that AI systems are conscious and deserve rights.
Factual · 30 Mar 2023 |
Philosophy, ethics and society |
| Not researched |
Factual |
30 Mar 2023 |
Among the things humans can do is design new AI systems.
Factual · 30 Mar 2023 |
AI capabilities |
| Not researched |
Factual |
30 Mar 2023 |
Yudkowsky put up a prediction market on whether, in 2026, we will have understood anything inside a giant transformer net that was not known in 2006.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not researched |
Factual |
30 Mar 2023 |
Yudkowsky was never skeptical that evolutionary computation would work in the limit.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not researched |
Factual |
30 Mar 2023 |
Nobody knows what is going on inside the giant inscrutable matrices of floating point numbers that he was describing.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not researched |
Factual |
30 Mar 2023 |
Large language models can play chess.
Factual · 30 Mar 2023 |
AI capabilities |
| Not researched |
Factual |
30 Mar 2023 |
Natural selection producing hominids did not require exponentially more resource investment to yield linear increases in competence.
Factual · 30 Mar 2023 |
Other |
| Not researched |
Factual |
30 Mar 2023 |
GPT models could use calibrated probabilities before reinforcement learning from human feedback, but training them to satisfy humans made them worse at probability.
Factual · 30 Mar 2023 |
AI safety and alignment |
| Not researched |
Factual |
30 Mar 2023 |
In his writings on rationality, Yudkowsky emphasized probability theory more than reason as such.
Factual · 30 Mar 2023 |
Their own work or organisation |
| Not researched |
Factual |
Feb 2023 |
Yudkowsky said in February 2023 that he thinks the technology does not exist to align an AI well enough to instruct it to build a copy of a strawberry without destroying the world.
Factual · Feb 2023 |
AI safety and alignment |
| Not researched |
Factual |
Feb 2023 |
ChatGPT does not get smarter each time it is run.
Factual · Feb 2023 |
AI capabilities |
| Not researched |
Factual |
Feb 2023 |
Enrico Fermi said nuclear fission chain reactions were 50 years off, if possible at all, two years before he built the first nuclear pile.
Factual · Feb 2023 |
Science and research methods |
| Not researched |
Factual |
Feb 2023 |
A number of the CEOs running major AI projects are making fun of the idea that AI could ignite the atmosphere, i.e. cause catastrophe.
Factual · Feb 2023 |
AI policy and governance |
| Not researched |
Factual |
Feb 2023 |
The Wright brothers said heavier-than-air flight was 50 years off shortly before they built the first Wright Flyer.
Factual · Feb 2023 |
Science and research methods |
| Not researched |
Factual |
Feb 2023 |
Yudkowsky's view on AI extinction risk is not the consensus view in the wider scientific field.
Factual · Feb 2023 |
AI safety and alignment |
| Not researched |
Factual |
Feb 2023 |
MIRI has pursued many lines of research that have not panned out.
Factual · Feb 2023 |
Their own work or organisation |
| Not researched |
Factual |
28 Feb 2018 |
AlphaZero does not need to first completely rewrite itself to do powerful things, making it more plausible that AIs could do dangerous or useful things without prior complete self-rewrites.
Factual · 28 Feb 2018 |
AI capabilities |
| Not researched |
Factual |
28 Feb 2018 |
Yudkowsky said the position he defends was named the orthogonality thesis by somebody he thought was Nick Bostrom.
Factual · 28 Feb 2018 |
AI safety and alignment |
| Not researched |
Factual |
28 Feb 2018 |
Asked whether the thought experiment originated with Bostrom, Eliezer Yudkowsky said that, as far as he knew, it originated with him.
Factual · 28 Feb 2018 |
Their own work or organisation |
| Not researched |
Factual |
28 Feb 2018 |
AI researchers who forecast AGI about 30 years off have not argued that, once reached, AGI will develop much more slowly than AlphaZero did.
Factual · 28 Feb 2018 |
AI timelines and forecasting |
| Not researched |
Factual |
28 Feb 2018 |
Economists already know about market failures and have that concept.
Factual · 28 Feb 2018 |
Economics and markets |
| Not researched |
Factual |
28 Feb 2018 |
MIRI is the original AI alignment organization and today works primarily on the technical parts of the alignment problem.
Factual · 28 Feb 2018 |
Their own work or organisation |
| Not researched |
Factual |
28 Feb 2018 |
Eliezer Yudkowsky wrote two to three years of daily blog posts on human rationality, which were collected into the book Rationality: From AI to Zombies.
Factual · 28 Feb 2018 |
Their own work or organisation |
| Not researched |
Factual |
28 Feb 2018 |
Eliezer Yudkowsky conducted the experiment that became known as the AI-box experiment, to demonstrate human limits in dealing with smarter minds.
Factual · 28 Feb 2018 |
AI safety and alignment |
| Not researched |
Factual |
28 Feb 2018 |
Yudkowsky said that AlphaZero is not reflecting on itself, is not learning things, does not have a general model of the world, and is not operating in new contexts or making new contexts for itself to be in.
Factual · 28 Feb 2018 |
AI capabilities |
| Not researched |
Factual |
28 Feb 2018 |
Two years after the Wright Flyer, people could still be found saying heavier-than-air flight is impossible.
Factual · 28 Feb 2018 |
Other global risks |
| Not researched |
Factual |
2008 |
Subjects in Wason's 2-4-6 task try to confirm their hypotheses rather than falsify them.
Factual · 2008 |
Science and research methods |
| Not researched |
Factual |
2008 |
Flaws have occasionally been discovered in previously accepted informal human mathematical proofs.
Factual · 2008 |
Science and research methods |
| Not researched |
Factual |
2008 |
Information known to be irrelevant still anchors people's judgments and contaminates their guesses.
Factual · 2008 |
Science and research methods |
| Not researched |
Factual |
2008 |
The book, which is aimed at an intelligent general audience, explains Bayes's Theorem rather than assuming it, in only a few pages.
Factual · 2008 |
Science and research methods |
| Not researched |
Citation |
2008 |
A 1982 review of fourteen papers covering thirty-four experiments on human calibration found overwhelmingly that people are overconfident.
Citation · 2008 |
Science and research methods |
| Not researched |
Factual |
2008 |
The Y2K problem, which was very expensive to fix but not globally catastrophic, arose from failing to foresee future design requirements.
Factual · 2008 |
Technology |
| Not researched |
Factual |
2008 |
The effect of many cognitive biases has been found to increase with time pressure, cognitive busyness, or sparse information.
Factual · 2008 |
Science and research methods |
| Not researched |
Factual |
2008 |
Human brains are not designed to be improved by outside neuroscientists or by internal recursive self-improvement.
Factual · 2008 |
Science and research methods |
| Not researched |
Factual |
2008 |
Guns were developed centuries before bulletproof vests.
Factual · 2008 |
Other global risks |
| Not researched |
Factual |
2008 |
Critical times for elementary molecular interactions are measured in femtoseconds, sometimes picoseconds.
Factual · 2008 |
Science and research methods |
| Not researched |
Factual |
2008 |
No curriculum exists that has prepared a large pool of researchers to make progress on Friendly AI.
Factual · 2008 |
Field building and careers |
| Not researched |
Factual |
2008 |
In planning, reality usually delivers results somewhat worse than people's stated worst case.
Factual · 2008 |
Science and research methods |
| Not researched |
Factual |
2008 |
In the anthropomorphism experiments discussed, subjects who consciously believed an AI is unlike a human still visualized scenarios as if the AI were anthropomorphic.
Factual · 2008 |
Science and research methods |
| Not researched |
Factual |
2008 |
For decades the U.S. and the U.S.S.R. avoided nuclear war, but with close calls such as the Cuban Missile Crisis in 1962.
Factual · 2008 |
Other global risks |
| Not researched |
Factual |
2008 |
Hindsight bias is important in legal cases where judges or juries decide whether a defendant was negligent in failing to foresee a hazard.
Factual · 2008 |
Philosophy, ethics and society |
| Not researched |
Factual |
2008 |
Instructing juries to avoid hindsight bias has no significant effect on that bias.
Factual · 2008 |
Philosophy, ethics and society |
| Not researched |
Citation |
2008 |
Latane and Darley's 1969 experiments found the bystander effect: larger groups are less likely to act in emergencies, individually and collectively.
Citation · 2008 |
Science and research methods |
| Not researched |
Factual |
2008 |
Discussions of existential risk not uncommonly draw their categories, choices, consequences and strategies from movies, books and television shows.
Factual · 2008 |
Other global risks |