| Not checkable |
Not checkable |
13 Oct 2025 |
I think another thing I’m worried about is that the wrong, even if we manage to make a super human AI safe or an AGI safe, then it might be…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I want to keep working on the research end all the way until we get AGI whenever that is.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
You actually don’t need any kind of formal education.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
It didn’t seem like what they were describing was actionable
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think of this as a precaution.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I’m not sure you can sell stuff if you’re nonprofit.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
Yoshua Bengio’s Group is pretty well-known for doing a lot of good stuff.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
The safety work that I’m most excited about, I think, it sounds obvious but the two things you most need are an extremely strong background…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
It’s not a variable that I have a lot of control over.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
It’s just a statistical learning system. It doesn’t know about racism, it doesn’t know about racial slurs.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I felt like maybe it was starting communication the best way to understand intelligence would be to actually directly work on building…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think ML work is so hot right now that anyone who goes into it, particularly on the fundamental research side, it’s easy to transition to…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
It actually could turn out that this is mostly just ML work and the final systems we build are different enough that for whatever reason…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
Also if you are at the forefront of AI, you have a better ability to implement AI’s safety in the final system that’s built.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think the important thing is that we shouldn’t go around with every other word we say being AGI in particular like the research itself…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think the attempt was to come up with some conceptual problems that relate to both that have long term and short term versions.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think safety shouldn’t be anything different from reinforcement learning.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think openness is intended to indicate the idea that these benefits should accrue to everyone.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I thought there were a number of really talented researchers here and it was a good environment in which to think about safety in the…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
Installing the fire alarm is the right course of action.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
Again, I’m pretty happy in that world.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
My view is we’re relatively early in the game and I think there’s a substantial probability that the gloomy analyses are really…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think it’s actually complicated.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think pure safety considerations tell us that it’s always good to have more time
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think it’s urgently important research area.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
But that’s not enough, we need them 99.999% safe.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think that’s an area that’s already getting a decent amount of attention but probably should get more like everything in safety should…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
We certainly wouldn’t want that to happen.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think generally our view
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
Generally, we’re working on following the gradient to a more general artificial intelligence and making it safe.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
Reinforcement learning is the only paradigm we have that even close to capturing this,
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
Most adults living in the developed world can afford a thousand dollars but most 17-year-olds might not be able to.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
If we did build them, I would definitely worry about their safety.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I don’t really have a good answer on the kind of like nerds think AI can solve everything question.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
My general view is I’m much less confident in our ability to identify those problems.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
The summary is we don’t know very much because it’s something we haven’t done yet.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
Whereas in the world where it does matter, it’s really great and really impactful to get a headstart on it.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
The lesson of that is that it’s very easy to make small changes in the reward space and have that lead to big differences in the behavior…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
If I take a complicated set of goals like that and I try and forced it into the framework of a hard-coded reward function, it’s going to…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
Maybe it’s turns out to be really exciting and that’s like a bet that I’m perfectly happy to take.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
If we don’t understand those, I don’t know how we can build them.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
That’s just wasteful and pointless.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
Supervised learning has this static quality where it’s like a one off.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think the kind of safety work that we’re doing has many of the same skills as any other area in machine learning, even though the subject…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
But I think going forward, if you’re a young student, I don’t particularly see a case for doing a PhD in another field if what you want to…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
We call that “reward hacking”.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think we’re better off if AGI is developed in a stable political environment with leaders who are intelligent and have reasonable views.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I’m not the expert on it.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think most of the people there are smart people who either had or could have really great careers as software engineers.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I wasn’t particularly happy with the way they were phrasing things.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
Tensorflow is pretty easy to learn.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
one thing that I found about neural nets is that they’re very powerful but they’re very unevenly powerful.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think they’ve been winning the talent war recently.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
When people at the major organizations are friends with each other and work to actively collaborate, then that reduces the probability of…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think actually that’s the case in any area of machine learning where you’re trying to do original research.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
at the same time no one’s defined what successful work looks like.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think that there’s a magic of empiricism because it’s very easy to engage in long chains of reasoning about a topic that don’t get tied…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
What I’d rather do is have a thread running from long term to short term things where you identify what the fundamental problems are.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
It’d be extremely illegal.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
it’s clear that at the very least we’re going to have a big challenge in making sure that something doesn’t go wrong.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
It’s really pretty self-contained and you don’t need that much help.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I’m really not against using formal reasoning methods and using mathematics but I think it’ll be possible to do that work more productively…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
This works on the reward hacking, scalable supervision side of things.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I get pretty nervous when we don’t have a strong background in machine learning because even if they think that a machine learning system…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think we better understand the principles by which the system operates.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
Someone could sabotage a self-driving car with adversarial examples.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think those efforts have been successful and so I feel like it’s been good to cause things to happen that wouldn’t have caused otherwise.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think generally our view of AI works and what to build in AI and the focus on safety are actually pretty similar between the two…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
Generally, when we build engineering systems, that silent, random failure, it’s not something that we see as a desirable property in…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
Personally, I’ve no interest in getting rich from AGI.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think the limiting factor is just a very strong machine learning talent.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think many people here don’t know if there’s a way to work on safety right now.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think that’s the basic reason to work on AI.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
it seemed like very few people were actually working on it.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think that they largely thought AI safety was about fears that AIs would malevolently rise up and attack their creators.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I generally tend to give people the advice to come to industrial labs.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I was actually really excited about this because I saw this as a test bed to study safety.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think the most important thing is being able to keep up with the literature and make, creative, original discoveries that are novel and…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
Yeah, there’s a huge amount of leverage to be had.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
If they don’t already have access to one it might be a good way to get people started early.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
It’s just one little bit of progress on one.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I myself want to stay at OpenAI and work on safety.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
By no means does this solve all safety problems.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
The level of technical ability you need to show is not lowered, it’s even higher when you don’t have the educational background, but it’s…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I’m pretty suspicious of anything where you can’t get that type empirical feedback loop because I think it’s really easy for people to fool…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
If people are worried about progress being slowed down, I don’t believe anything I do is close to that.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
If it has the wrong objective function, if you don’t specify its goal correctly, it can do something unpredictable and therefore dangerous.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think there are almost certainly things that are very specific to powerful systems.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I can’t see any reason and principle why that couldn’t happen.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
The same kind of thing could go wrong with an AGI as could go wrong with a very simple agent playing a video game or robot cleaning your…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
People can do it in high school like, if you meet a 17 year old, how do you get them into machine learning.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
Yes, thanks for having me.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I think that’s a pretty important problem.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I wouldn’t say we’ve gone away from software engineering so much as we’ve been experimenting with how to produce tools and takes a few…
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
We’re still laying out what the problems are and what the work is that needs to be done.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I don’t think it will be a barrier.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I’m starting to wonder that in the last year we’re not in such a peaceful geopolitical state.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
13 Oct 2025 |
I wouldn’t recommend by going back and doing another PhD in machine learning.
Not checkable · 13 Oct 2025 |
|
| Not checkable |
Not checkable |
5 Jun 2025 |
But over the last few months, I have become convinced that fully addressing the risks requires even more prudence
Not checkable · 5 Jun 2025 |
|
| Not checkable |
Not checkable |
5 Jun 2025 |
Slowing the rate from “extremely fast” to “only somewhat fast” gives up relatively little strategic advantage, while potentially greatly…
Not checkable · 5 Jun 2025 |
|