| Not checkable |
Not checkable |
28 Feb 2025 |
poorly presented evidence of risk is actually the worst enemy of mitigating risk.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
Through some mixture of voluntary convince and laws, we can get ourselves to slow down if the models are too dangerous.
Not checkable · 28 Feb 2025 |
|
| Resolved true |
Factual |
28 Feb 2025 |
Anthropic trained Claude 3.7 to focus more on real-world tasks, as distinct from competition coding.
Factual · 28 Feb 2025 |
AI capabilities |
| Unresolvable |
Factual |
28 Feb 2025 |
The models Anthropic has released so far were not that expensive to train.
Factual · 28 Feb 2025 |
Their own work or organisation |
| Not checkable |
Not checkable |
28 Feb 2025 |
So we've kind of taken a big step in that direction, but we're not to where we want to be yet.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
But if their enforcers are no longer human, that starts painting some very dark possibilities.
Not checkable · 28 Feb 2025 |
|
| Unresolvable |
Factual |
28 Feb 2025 |
Despite a climate of worrying less about AI risks, the underlying risks from AI have been increasing.
Factual · 28 Feb 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
28 Feb 2025 |
The technology has immense military value.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I worry more about them from a national competition and national security perspective.
Not checkable · 28 Feb 2025 |
|
| Unresolvable |
Promise |
28 Feb 2025 |
Anthropic will release bigger, more expensive models than those released so far.
Promise · 28 Feb 2025 |
Their own work or organisation |
| Not researched |
Factual |
28 Feb 2025 |
Dario Amodei has never used the term 'P-Doom'.
Factual · 28 Feb 2025 |
Their own work or organisation |
| Resolved true |
Prediction |
28 Feb 2025 |
By the end of 2025, AI will do very serious things in coding.
Prediction · 28 Feb 2025 |
AI capabilities |
| Not checkable |
Not checkable |
28 Feb 2025 |
I don't know that innovations are necessarily copied exactly.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
But I think that all gets thrown out the window in the world of international competition.
Not checkable · 28 Feb 2025 |
|
| Resolved true |
Factual |
28 Feb 2025 |
Anthropic's approach to reasoning models in Sonnet 3.7 emphasized different things from competitors' reasoning models.
Factual · 28 Feb 2025 |
AI capabilities |
| Not researched |
Factual |
28 Feb 2025 |
Reasoning models offered by AI companies have generally been separate from the regular model, as a regular model plus a distinct reasoning model.
Factual · 28 Feb 2025 |
AI capabilities |
| Not checkable |
Not checkable |
28 Feb 2025 |
I recognize the 3.7 new was a misstep.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
some basic critical thinking, some basic street smarts is maybe more important than it has been in the past
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
But in the background, like I said, as the models have gotten more powerful, the amazing and wondrous things that we can do with them have…
Not checkable · 28 Feb 2025 |
|
| Not researched |
Factual |
28 Feb 2025 |
Amodei said that they found that with "CLAWD" (as transcribed, apparently Claude), the task, described as "a bunch of statistical analysis", can be done in three days.
Factual · 28 Feb 2025 |
AI capabilities |
| Not checkable |
Not checkable |
28 Feb 2025 |
So I think there will be a lot by ordinary standards.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
Yeah, you know we are already seeing some of it.
Not checkable · 28 Feb 2025 |
|
| Not yet due |
Promise |
18 Feb 2025 |
Anthropic has a goal of reaching the point where interpretability can reliably detect most model problems by 2027.
Promise · 18 Feb 2025 |
Their own work or organisation |
| Not checkable |
Not checkable |
18 Feb 2025 |
We can’t stop the bus, but we can steer it.
Not checkable · 18 Feb 2025 |
|
| Unresolvable |
Prediction |
18 Feb 2025 |
If truly compelling evidence of the dangers of autonomous AI emerged, a slowdown in AI development would be just barely possible.
Prediction · 18 Feb 2025 |
AI policy and governance |
| Unresolvable |
Factual |
18 Feb 2025 |
Chris Olah was unusual in proposing and seriously pursuing a comprehensive effort to understand everything neural networks do.
Factual · 18 Feb 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
18 Feb 2025 |
If you are a competitor and you don’t want this to happen, you too should invest more in interpretability!
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
Our long-run aspiration is to be able to look at a state-of-the-art model and essentially do a “brain scan”
Not checkable · 18 Feb 2025 |
|
| Mostly true |
Factual |
18 Feb 2025 |
The lack of understanding of how AI systems work is essentially unprecedented in the history of technology.
Factual · 18 Feb 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
18 Feb 2025 |
interpretability should function like the test set for model alignment, while traditional alignment techniques such as scalable…
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
A serious moral accounting on AI can't trust their self-reports, since we might accidentally train them to pretend to be okay when they…
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
I believe that democratic countries must remain ahead of autocracies in AI.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
interpretability acts as an independent check on the alignment of models, uncontaminated by the training process which might incentivize…
Not checkable · 18 Feb 2025 |
|
| Unresolvable |
Prediction |
18 Feb 2025 |
Interpretability, meaning understanding the inner workings of AI systems, could be achieved before AI models reach an overwhelming level of power.
Prediction · 18 Feb 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
18 Feb 2025 |
Honestly I can sympathize with both reactions, and this might be a clue as to why the debate over this risk has become so polarized.
Not checkable · 18 Feb 2025 |
|
| Not researched |
Factual |
18 Feb 2025 |
For some applications such as mortgage assessments, where decisions are legally required to be explainable, the inability to see inside AI models is a legal blocker to adoption.
Factual · 18 Feb 2025 |
AI policy and governance |
| Unresolvable |
Prediction |
18 Feb 2025 |
Amodei said that multiple recent breakthroughs had convinced him that interpretability research is on the right track and has a real chance of success in its goal of fully revealing the inner workings of AI models.
Prediction · 18 Feb 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
18 Feb 2025 |
We therefore must move fast if we want interpretability to mature in time to matter.
Not checkable · 18 Feb 2025 |
|
| Unresolvable |
Prediction |
18 Feb 2025 |
Better interpretability could greatly improve the ability to set bounds on the range of possible AI errors.
Prediction · 18 Feb 2025 |
AI safety and alignment |
| Not researched |
Factual |
18 Feb 2025 |
For decades the conventional wisdom was that understanding AI models was impossible and that they were inscrutable black boxes.
Factual · 18 Feb 2025 |
AI safety and alignment |
| Not researched |
Factual |
18 Feb 2025 |
The field of AI as a whole is further ahead than interpretability efforts and is advancing very quickly.
Factual · 18 Feb 2025 |
AI safety and alignment |
| Not researched |
Factual |
18 Feb 2025 |
There is also evidence of real-world behavior that looks vaguely like “cheating on the test”, though it’s more degenerate than it is…
Factual · 18 Feb 2025 |
AI safety and alignment |
| Mixed |
Factual |
18 Feb 2025 |
People outside the AI field are often surprised and alarmed to learn that AI creators do not understand how their AI systems work.
Factual · 18 Feb 2025 |
Philosophy, ethics and society |
| Not yet due |
Prediction |
18 Feb 2025 |
Amodei said that, on its current trajectory, he would bet strongly in favor of interpretability becoming a sophisticated and reliable way to diagnose problems in even very advanced AI within 5-10 years.
Prediction · 18 Feb 2025 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
18 Feb 2025 |
I am very concerned about deploying such systems without a better handle on interpretability.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
It is the latter point where seeing inside the models could help a lot.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
Fighting the market head-on like this feels like trying to stop a freight train with your toe.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
I strongly encourage them to allocate more resources.
Not checkable · 18 Feb 2025 |
|
| Not researched |
Factual |
18 Feb 2025 |
there are a lot of experiments (many of which were done by Anthropic) showing that models can lie or deceive under certain circumstances…
Factual · 18 Feb 2025 |
AI safety and alignment |
| Not researched |
Factual |
18 Feb 2025 |
Anthropic's interpretability team immediately found basic mechanisms in a language model for copying and sequential pattern-matching.
Factual · 18 Feb 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
18 Feb 2025 |
If we find that the computation they perform is similar to the brains of animals, or even humans, that might be evidence in favor of moral…
Not checkable · 18 Feb 2025 |
|
| Not researched |
Factual |
18 Feb 2025 |
What there isn’t is evidence of dangerous behaviors emerging in a more naturalistic way, or of a general tendency or general intent to lie…
Factual · 18 Feb 2025 |
AI safety and alignment |
| Mostly true |
Factual |
18 Feb 2025 |
When Anthropic was founded, its founders made mechanistic interpretability a central part of the company's direction, focused on LLMs.
Factual · 18 Feb 2025 |
Their own work or organisation |
| Mostly true |
Factual |
18 Feb 2025 |
Amodei said that "we" were able to provide a strong theoretical basis for having conviction that superposition was the right hypothesis to pursue; the surrounding text does not say who "we" refers to.
Factual · 18 Feb 2025 |
AI safety and alignment |
| Mostly true |
Factual |
18 Feb 2025 |
It became clear that superposition would be a central problem for language models, much worse than in vision models.
Factual · 18 Feb 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
18 Feb 2025 |
Interpretability gets less attention than the constant deluge of model releases, but it is arguably more important.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
Open-weights models present additional dangers in that guardrails can be simply stripped away.
Not checkable · 18 Feb 2025 |
|
| Not researched |
Factual |
18 Feb 2025 |
we’ve never seen any solid evidence in truly real-world scenarios of deception and power-seeking
Factual · 18 Feb 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
18 Feb 2025 |
Powerful AI will shape humanity’s destiny, and we deserve to understand our own creations before they radically transform our economy, our…
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
it should be clear why it doesn’t make sense to regulate or mandate that companies conduct them, at least at this stage
Not checkable · 18 Feb 2025 |
|
| Mostly true |
Factual |
18 Feb 2025 |
Modern generative AI systems are opaque in a way that fundamentally differs from traditional software.
Factual · 18 Feb 2025 |
AI safety and alignment |
| Unresolvable |
Factual |
18 Feb 2025 |
Amodei said he does not think truly compelling evidence of the dangers of autonomous AI exists today, contrary to the claims of advocates.
Factual · 18 Feb 2025 |
AI safety and alignment |
| Unresolvable |
Prediction |
18 Feb 2025 |
If it were possible to look inside models, it might be possible to systematically block all jailbreaks and characterize what dangerous knowledge models have.
Prediction · 18 Feb 2025 |
AI safety and alignment |
| Unresolvable |
Prediction |
18 Feb 2025 |
Amodei said he is quite skeptical that any slowdown in AI development to address risk is possible, even among companies within democratic countries, given AI's economic value.
Prediction · 18 Feb 2025 |
AI policy and governance |
| Not researched |
Factual |
18 Feb 2025 |
One year before this essay, neural network thoughts could not be traced and millions of concepts inside them could not be identified; as of the essay, they can.
Factual · 18 Feb 2025 |
AI safety and alignment |
| Resolved true |
Factual |
18 Feb 2025 |
Anthropic created "Golden Gate Claude", a version of one of its models with the Golden Gate Bridge feature artificially amplified, making it obsessed with the bridge.
Factual · 18 Feb 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
18 Feb 2025 |
We are thus in a race between interpretability and model intelligence.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
If we believe them to be superficial pattern-matchers, it seems unlikely they warrant moral consideration.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
we recommend that in assessing official, high-stakes production models, we treat interpretability analysis with the same care we would…
Not checkable · 18 Feb 2025 |
|
| Not researched |
Factual |
18 Feb 2025 |
A very popular mechanistic interpretability workshop at the ICML conference was reportedly rejected on seemingly pretextual grounds.
Factual · 18 Feb 2025 |
Field building and careers |
| Not checkable |
Not checkable |
18 Feb 2025 |
we should be very hesitant to directly train or optimize on interpretability outputs (features/concepts, circuits) in production, as this…
Not checkable · 18 Feb 2025 |
|
| Not researched |
Factual |
18 Feb 2025 |
When a generative AI system performs a task, researchers do not know at a specific or precise level why it makes the choices it does.
Factual · 18 Feb 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
18 Feb 2025 |
Many of the risks and worries associated with generative AI are ultimately consequences of this opacity, and would be much easier to…
Not checkable · 18 Feb 2025 |
|
| Not researched |
Factual |
18 Feb 2025 |
Anthropic found over 30 million features in Claude 3 Sonnet, a medium-sized commercial model.
Factual · 18 Feb 2025 |
AI safety and alignment |
| Mostly true |
Factual |
18 Feb 2025 |
Circuits, groups of features, allow researchers to trace an AI model's thinking steps.
Factual · 18 Feb 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
18 Feb 2025 |
Interpretability would have a crucial role in determining the wellbeing of AIs in such a situation.
Not checkable · 18 Feb 2025 |
|
| Not researched |
Factual |
18 Feb 2025 |
AI models use superposition because it lets them express more concepts than they have neurons, enabling them to learn more.
Factual · 18 Feb 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
18 Feb 2025 |
If true, this behavior is shortsighted and self-defeating at exactly a time when academics in AI are looking for ways to maintain relevance.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
I don’t intend to imply that interpretability is our only risk mitigation tool.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
generative AI systems are grown more than they are built
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
because deceit is precisely the behavior we’re trying to find, external behavior is not reliable.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
Our inability to understand models’ internal mechanisms means that we cannot meaningfully predict such behaviors, and therefore struggle to…
Not checkable · 18 Feb 2025 |
|
| Not researched |
Factual |
18 Feb 2025 |
Even a small AI model may contain a billion or more concepts.
Factual · 18 Feb 2025 |
AI capabilities |
| Not checkable |
Not checkable |
18 Feb 2025 |
it’s important not to “use” the diagnostic test signal too many times in one production run to inform changes to the training process, as…
Not checkable · 18 Feb 2025 |
|
| Unresolvable |
Prediction |
18 Feb 2025 |
The most likely route to "smoking gun" evidence of AI danger is interpretability itself.
Prediction · 18 Feb 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
18 Feb 2025 |
it’s possible to have great positive impact by doing so.
Not checkable · 18 Feb 2025 |
|
| Not researched |
Factual |
18 Feb 2025 |
Without interpretability, the only way to discover that a jailbreak of an AI model exists is to find it empirically.
Factual · 18 Feb 2025 |
AI safety and alignment |
| Not checkable |
Not checkable |
18 Feb 2025 |
There are, in fact, already some mildly concerning signs from this perspective.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
All of these—accelerating interpretability, light-touch transparency legislation, and export controls on chips to China—have the virtue of…
Not checkable · 18 Feb 2025 |
|
| Not researched |
Factual |
18 Feb 2025 |
Research papers from the preceding few months have shown that interpretability can help humans understand patterns predicted by AI in DNA and protein sequence data.
Factual · 18 Feb 2025 |
Science and research methods |
| Not checkable |
Not checkable |
29 Jan 2025 |
My thoughts on China, export controls and two possible futures https://t.co/LZAANEcZJW
Not checkable · 29 Jan 2025 |
|
| Resolved true |
Factual |
11 Nov 2024 |
Anthropic's Responsible Scaling Policy uses an if-then structure: if models pass a certain capability, Anthropic imposes a set of safety and security requirements.
Factual · 11 Nov 2024 |
AI safety and alignment |
| Not checkable |
Not checkable |
11 Nov 2024 |
I think they're two sides of the same coin.
Not checkable · 11 Nov 2024 |
|
| Not researched |
Factual |
11 Nov 2024 |
Dario Amodei said he believes Claude Haiku 3.5, the smallest new model, is about as good as Opus 3, the largest old model.
Factual · 11 Nov 2024 |
AI capabilities |
| Not checkable |
Not checkable |
11 Nov 2024 |
Anthropic’s mission is to kind of try to make this all go well.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think there’s an inflection point around a thousand and we want to be much more careful how we grow.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
My strong instinct would be that there’s no ceiling below the level of humans.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I feel good about our RSP. It’s not perfect. It needs to be iterated on a lot.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
But even if it were completely unaligned and could get around all these human obstacles it would have trouble.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
If you scale up one, not the others, you run out of the other reagents and the reaction stops.
Not checkable · 11 Nov 2024 |
|