| Not checkable |
Not checkable |
18 Feb 2025 |
All of these—accelerating interpretability, light-touch transparency legislation, and export controls on chips to China—have the virtue of…
Not checkable · 18 Feb 2025 |
|
| Not checkable |
Not checkable |
29 Jan 2025 |
My thoughts on China, export controls and two possible futures https://t.co/LZAANEcZJW
Not checkable · 29 Jan 2025 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think they're two sides of the same coin.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
Anthropic’s mission is to kind of try to make this all go well.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think there’s an inflection point around a thousand and we want to be much more careful how we grow.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
My strong instinct would be that there’s no ceiling below the level of humans.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I feel good about our RSP. It’s not perfect. It needs to be iterated on a lot.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
But even if it were completely unaligned and could get around all these human obstacles it would have trouble.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
If you scale up one, not the others, you run out of the other reagents and the reaction stops.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
having this capability may make it scarier once it has the cognitive capability to do something at the ASL-3 and ASL-4 level, this may be…
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
That kind of cycle doesn’t solve the key problems that I’m talking about here.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I want them to really understand that this isn’t Doomers versus Accelerationists.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
These models are new artifacts that no one really understands and so getting experience playing with them.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
computer use isn’t a fundamentally new capability like the CBRN or autonomy capabilities are.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
the worst enemy of those who want real accountability is badly designed regulation.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
But the truth is that the scale up is very quick.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I don’t think I was a better programmer or better at coming up with research ideas than any of the hundreds of people that I worked with.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
You cannot just reach in there and say, "Oh, I want the model to apologize less."
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
It’s yet another way in which the science of how these models work is very poorly developed.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
Everyone agrees that we don’t want models to present these CBRN risks.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
That’s what we might need for gene therapy for certain diseases.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
That is the thing I worry about most, the concentration of power, the abuse of power, structures like autocracies and dictatorships where a…
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
So I am optimistic about meaning.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
It takes a long time to produce faster hardware.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
We don’t internally have the resources to try all these different things.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think it’s very important that some jurisdiction, whether it’s California or the federal government and/or other countries and other…
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
So you put all those things together and we’re not going change the world and upload everyone in five minutes.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think those were mostly nonsense, but there are better arguments against regulation.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
It’s basically a form of self-play. You’re training the model against itself.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
There’s just a smooth exponential.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
It’s not about being the good guy, it’s about setting things up so that all of us can be the good guy.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
You have these exponential increase in computer intelligence for linear increase in ability to predict.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
To be clear, this itself is like a failing by us and by the other model providers that the models are just often sensitive to small changes…
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
But the issue is just that if every time someone super talented looks around, they see someone else super talented and super dedicated…
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
It’s because if we can get to the other side of these risks, if we can run the gauntlet successfully, to put it in stark terms, then on the…
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I’m fully expecting to well, definitely be wrong about all the details.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
So it’s one tool in the toolkit, but I think it is a very important tool.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think that’s probably a bad idea during training because the model can be changing its policy, it can be changing what it’s doing and…
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
it’s been a good forcing function for getting the company to take these risks seriously
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I might be just spectacularly wrong about the whole thing and people will laugh at me for years.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think the number one quality, especially on the research side, but really both, is open mindedness.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
But, if AGI is some discrete or separate thing, which is the way people often talk about it, then it’s a meaningless buzzword.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
ASL-3 is going to be the point at which the models are helpful enough to enhance the capabilities of non-state actors
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
It’s usually some boring matter of practice and trade craft.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
And so we can have more impact by trying to mitigate the risks.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
Maybe we’re stuck with the terms and my efforts to change them are futile.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
So our thinking was different companies produce large and small models, better and worse models.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
They can be boring or they can be very distinctive like Golden Gate Claude was.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I just hope we can get it right and make it real.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think it’s probably better to learn and explore this capability before the model is super capable
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
The point isn’t to be virtuous, the point is to get the system into a better equilibrium than it was before.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
If you only talk about risks, your brain only thinks about risks.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think they just neglect the laws of physics.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
They're like ghosts, but they're coming at us so fast because the models are improving so fast.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
A customer service agent behaves very differently from a lawyer and obeys different principles.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think there are still worlds where it doesn’t happen in 100 years. The number of those worlds is rapidly decreasing.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I’m amazed at how clean it’s been.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think evaluations, we’re still very early in our ability to study evaluations, particularly for dynamic systems acting in the world.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think my number one piece of advice is to just start playing with the models.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
And yeah, we want to get up to the human level reliability of 80, 90% just like anywhere else.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
Yes, it’s very frightening. It’s very frightening.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
Beyond that, it gets very uncertain and there our goal is generally for the models to be more neutral, to not espouse a particular point of…
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
The barriers are going to fall apart gradually and then all at once.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
It’s like RLHF bridges the gap between the human and the model.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think this is a science like the safety of airplanes or the safety of cars or the safety of drugs. I don't think there's any big thing…
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
We have something that we think a better and more responsible way of doing things. It’s also a competitive advantage.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
It’s not about us in particular. It’s not about having one particular good guy.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
nothing that this model is doing inherently increases the risk from an RSP perspective
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think the ASL-3 measures are, I won't say straightforward, they're rigorous, but they're easier to reason about.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
my view is that there’s so much more we can still do there.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
The idea that the model can write the code means that the model can then run the code and then see the results and interpret it back.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
Now, honestly, the current model has, it leaves a lot still to be desired and we were honest about that in the blog.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
One way I think of it is the thing we’re all afraid of is the race to the bottom and the race to the bottom doesn’t matter who wins because…
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
we think it’s a way to make models safer.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
But we have found, and you can talk in much more detail about this to Chris, that when we open them up, when we do look inside them, we…
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I’m very worried about that.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think releasing the model while the capabilities are still limited is very helpful in terms of doing that.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
we also need to address the risks because those risks are in our way. They’re landmines on the way from here to there, and we have to…
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
We've seen lots of examples of demagoguery in our life from humans, and there's a concern that models could do that as well.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
AI is not God
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
It’s about shaping the incentives to point upward instead of shaping the incentives to point downward.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
In many ways, the manner and personality of these models is more an art than it is a science.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
probably going forward, we might release new ones multiple times a year because it's hard to get these policies right technically…
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think that's like a present day analog of future control problems in AI systems that we can start to study today.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think containing bad models is a much worse solution than having good models.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I just saw these as independent dials that you could turn.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
That in itself I think can overcome almost every other disadvantage.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
we have a theory of change called Race to the Top. Race to the Top is about trying to push the other players to do the right thing by…
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I’m not at all confident.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
It seems like the process is what matters and how it shows who you are as a person along the way and how you relate to other people and the…
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
So I actually do believe that you can gain benchmarks, but I think if we get to 100% on that benchmark in a way that isn’t over-trained or…
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
But the system as a whole, I think makes some obvious trade-offs that are very far from maximizing human welfare.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
In particular, linear scaling up of bigger networks, bigger training times and more and more data.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I could point you to a hundred people here who are better at that than I am.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
generally, you want to preserve mechanistic interpretability as a verification set or test set that's separate from the training process of…
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
If you have a strong pre-trained model, I feel like you’re halfway to anywhere in terms of the intelligence space.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think the time scale is just too long and I can see it.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
I think that’s a pretty useful direction.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
these systems are actually quite unpredictable. They're actually quite hard to steer and control.
Not checkable · 11 Nov 2024 |
|
| Not checkable |
Not checkable |
11 Nov 2024 |
The perfect organization doesn’t exist.
Not checkable · 11 Nov 2024 |
|