| Not checkable |
Not checkable |
28 Feb 2025 |
We go to great pains to say, we're not worried about the present.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
There are ways to surgically and carefully address the risks without slowing down the benefits very much, if at all.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
we've been successful at shaping the behavior of these models, but the models are unpredictable, right?
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
We actually see Anthropic as almost a dry run for how will society handle these issues in a sensible and humanistic way.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
Once things get polarized, once it's like we're going to cheer for this set of words and boo for that set of words, nothing good gets done.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
So the thing you describe is absolutely correct,
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
we are reserving quad four, sonnet, and maybe some other models in the sequence for things that are really quite substantial leaps.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
So I think our model is kind of a step towards this.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
But I don't want the autocratic governments to have a military advantage.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I think the optimistic vision is one that often deals in abstractions
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
Yeah, I think you definitely should be, although it's not clear what direction that will land in.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
We don't care about that at all.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I've now probably increased my confidence that we are actually in the world where things are gonna happen.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
the market is more segmented than you think it is.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I'm not saying they're not impressive, but they're sometimes less relevant to tasks in the real world or the economy.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
the models are actually relatively different from each other and that creates differentiation.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
It means that a new risk exists in the world, a new threat vector exists in the world,
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
And if it does, that doesn't mean we're all gonna die of the plague tomorrow.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
the worst outcome of course is if people get fired because of a model.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
my worry is, if the US is a couple years ahead of China, we can use that couple years to make things safe.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I'm a fan of seizing the opportunities.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
We care about this kind of esoteric, high, uncommon knowledge that say only a virology PhD or something has.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
These people are gonna regret what they've said.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I worry less about DeepSeek from a commercial competition perspective.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
the default state of nature is that things go at maximum speed.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I have to tell you I was deeply disappointed in the summit.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
then it might just say, well, I don't want my error corrected, these are my values, and do completely the wrong thing.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
We have kind of a plan for how to make them safe, but it's not a plan that's going to reliably work yet.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
anyone who's involved in industries where this risk exists should take a precaution against that risk in particular.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
if you if you don't talk about the benefits, you can't inspire people, no one's going gonna be on your side if you're all gloom and doom.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I'm definitely focusing quite a lot on making sure that I have the best impact I can these two years in particular.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I'm just worried that it'll be a shock to people when it happens.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
There's often not a lot of specific things to point to.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I'd give numbers more like 70 and 80% and less like 40 or 50%.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I think at least at this stage, it's not a huge change to things.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
There was no one with the authority to enforce any agreement between the US and China, even if one were to be made.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
Look, bringing AI benefits to everyone, like curing previously incurable diseases, that's not a partisan issue.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
the models, they're inherently somewhat difficult to control.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
The risk is increasing up to some critical point, whether you're paying attention or not.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
Like it's wrong not to tell people that that is coming or to try to sugar coat it.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
It's not that there aren't present dangers and there are always kind of normal tech risks,
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
but we wanted the focus to be a little bit different.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
And so things like the export controls, which I discussed in that post, are one of the things we can do to prevent that.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I looked at the Paris summit and being there, it kind of made me angry.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
Whatever the intention of those companies, particularly in China, there's a governmental component to this.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I'm not trying to deny them the benefits of the technology.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
If you think about repressive governments, the limits to how repressive they can be are generally set by what they can get their enforcers…
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I'm actually not sure a lot of the economic value is there.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
Even within coding, there's really a difference between competition coding and doing something in the real world.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I think that requires nuance.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
That is what Anthropic is mostly worried about.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
Which is not the same as like an AI killing everyone, which people sometimes mean by P-Doom.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I'm more worried about the dangers that we're going to see as models become more powerful.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I'm supportive of trying to do that, but it cannot be the plan A.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
So you could take it as a reassuring sign, and in some ways I do.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
poorly presented evidence of risk is actually the worst enemy of mitigating risk.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
Through some mixture of voluntary convince and laws, we can get ourselves to slow down if the models are too dangerous.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
So we've kind of taken a big step in that direction, but we're not to where we want to be yet.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
But if their enforcers are no longer human, that starts painting some very dark possibilities.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
The technology has immense military value.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I worry more about them from a national competition and national security perspective.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I don't know that innovations are necessarily copied exactly.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
But I think that all gets thrown out the window in the world of international competition.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
I recognize the 3.7 new was a misstep.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
some basic critical thinking, some basic street smarts is maybe more important than it has been in the past
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
But in the background, like I said, as the models have gotten more powerful, the amazing and wondrous things that we can do with them have…
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
So I think there will be a lot by ordinary standards.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
28 Feb 2025 |
Yeah, you know we are already seeing some of it.
Not checkable · 28 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
We can’t stop the bus, but we can steer it.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
If you are a competitor and you don’t want this to happen, you too should invest more in interpretability!
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
Our long-run aspiration is to be able to look at a state-of-the-art model and essentially do a “brain scan”
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
interpretability should function like the test set for model alignment, while traditional alignment techniques such as scalable…
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
A serious moral accounting on AI can't trust their self-reports, since we might accidentally train them to pretend to be okay when they…
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
I believe that democratic countries must remain ahead of autocracies in AI.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
interpretability acts as an independent check on the alignment of models, uncontaminated by the training process which might incentivize…
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
Honestly I can sympathize with both reactions, and this might be a clue as to why the debate over this risk has become so polarized.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
We therefore must move fast if we want interpretability to mature in time to matter.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
I am very concerned about deploying such systems without a better handle on interpretability.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
It is the latter point where seeing inside the models could help a lot.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
Fighting the market head-on like this feels like trying to stop a freight train with your toe.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
I strongly encourage them to allocate more resources.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
If we find that the computation they perform is similar to the brains of animals, or even humans, that might be evidence in favor of moral…
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
Interpretability gets less attention than the constant deluge of model releases, but it is arguably more important.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
Open-weights models present additional dangers in that guardrails can be simply stripped away.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
Powerful AI will shape humanity’s destiny, and we deserve to understand our own creations before they radically transform our economy, our…
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
it should be clear why it doesn’t make sense to regulate or mandate that companies conduct them, at least at this stage
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
We are thus in a race between interpretability and model intelligence.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
If we believe them to be superficial pattern-matchers, it seems unlikely they warrant moral consideration.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
we recommend that in assessing official, high-stakes production models, we treat interpretability analysis with the same care we would…
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
we should be very hesitant to directly train or optimize on interpretability outputs (features/concepts, circuits) in production, as this…
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
Many of the risks and worries associated with generative AI are ultimately consequences of this opacity, and would be much easier to…
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
Interpretability would have a crucial role in determining the wellbeing of AIs in such a situation.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
If true, this behavior is shortsighted and self-defeating at exactly a time when academics in AI are looking for ways to maintain relevance.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
I don’t intend to imply that interpretability is our only risk mitigation tool.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
generative AI systems are grown more than they are built
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
because deceit is precisely the behavior we’re trying to find, external behavior is not reliable.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
Our inability to understand models’ internal mechanisms means that we cannot meaningfully predict such behaviors, and therefore struggle to…
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
it’s important not to “use” the diagnostic test signal too many times in one production run to inform changes to the training process, as…
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
it’s possible to have great positive impact by doing so.
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
18 Feb 2025 |
There are, in fact, already some mildly concerning signs from this perspective.
Not checkable · 18 Feb 2025 |
|