I was confused, but I gather by "pro-safety" you mean "that AI is or will be safe" (not "in favor of working on AI safety").
I basically agree with your assessments of how (in)correct these arguments are.
And I think it's very helpful to assess them over multiple time-scales, as you do.
This aligns very well with my thinking and the point of this blog...
For instance, "friction" arguments seem to apply very well to current LLM agents. But with the real AI, we will have cracked teams of AI consultants, salespeople, and lobbyists who could take all the friction out of deployment -- "the real AI deploys itself", but maybe not by going rogue.
I saw you on the Nonzero podcast on YouTube. I agree with your conclusion about Real AI (or advanced super intelligent AI) and just finished Yudkowsky & Soares’ book. If the resistance to Real AI doesn’t become effective soon, we will go extinct. In addition to the utopian dreams about extended lifespans and miraculous tech advances, another factor driving the tech titans is simply wealth addiction. I’m also disheartened by the near complete lack of concern or even awareness that we are so close to an extinction event by 99% of people I encounter. I wish you luck. For my part I plan to write my Senator and Congressman, but the chance of Congress doing anything in the face of AI lobbying pressure is slim to none, and Slim just rode out of town.
Thanks! I’m not quite so pessimistic. It’s not in anyone’s interest (almost) to build something this dangerous. Politicians are people too, and want to survive and protect their loved ones. We still have a long way to go but the direction of travel is pretty clear. People are waking up. Let’s keep at it!
An interesting read. I am less alarmed than you are, but I agree that, unless we change the way we’re building AI, there’s a real danger that we’ll end up creating incomprehensibly alien intelligences whose motivations we won’t understand. I think we can still mitigate it by incorporating natural features into the AI paradigm, and actively using the lessons of evolutionary biology and developmental psychology to shape the inherent nature of the AI agents we create.
If interested, you can read more detailed pieces on my substack “Bounded Alignment”.
One problem I have with all such proposals is that I don't think they're anywhere near mature enough. Another is that they typically don't do much to address gradual disempowerment (https://gradual-disempowerment.ai/).
P.S.: I wouldn't say I'm alarmed; I've been watching this unfold in slow motion for the last 15 years, and can't say I've had many surprises.
You’re right that these proposals are not mature enough, but that can mature over time. But not enough people in AI think in these terms, or engage with research in psychology and neuroscience. The entire paradigm of how we think of AI agents will need to change, so we recognize them as living entities and treat them accordingly.
The alternatives are all either extremely unlikely (stopping AI work) or insufficient (control, preference alignment).
I love the way you speak clearly about this. No bull-shit, no cope, no "trying to seem sensible". Just plain honesty based on what you have learned so far.
Glad you're writing on this, and looking forward to reading, David. Most of the pro-safety arguments seem naive to me (wrote about that here: https://liminalera.substack.com/p/is-ai-optimism-just-hope-and-handwaving)
Thanks for sharing!
I was confused, but I gather by "pro-safety" you mean "that AI is or will be safe" (not "in favor of working on AI safety").
I basically agree with your assessments of how (in)correct these arguments are.
And I think it's very helpful to assess them over multiple time-scales, as you do.
This aligns very well with my thinking and the point of this blog...
For instance, "friction" arguments seem to apply very well to current LLM agents. But with the real AI, we will have cracked teams of AI consultants, salespeople, and lobbyists who could take all the friction out of deployment -- "the real AI deploys itself", but maybe not by going rogue.
My latest post builds on this response: https://therealartificialintelligence.substack.com/p/the-real-ai-deploys-itself
I saw you on the Nonzero podcast on YouTube. I agree with your conclusion about Real AI (or advanced super intelligent AI) and just finished Yudkowsky & Soares’ book. If the resistance to Real AI doesn’t become effective soon, we will go extinct. In addition to the utopian dreams about extended lifespans and miraculous tech advances, another factor driving the tech titans is simply wealth addiction. I’m also disheartened by the near complete lack of concern or even awareness that we are so close to an extinction event by 99% of people I encounter. I wish you luck. For my part I plan to write my Senator and Congressman, but the chance of Congress doing anything in the face of AI lobbying pressure is slim to none, and Slim just rode out of town.
Thanks! I’m not quite so pessimistic. It’s not in anyone’s interest (almost) to build something this dangerous. Politicians are people too, and want to survive and protect their loved ones. We still have a long way to go but the direction of travel is pretty clear. People are waking up. Let’s keep at it!
An interesting read. I am less alarmed than you are, but I agree that, unless we change the way we’re building AI, there’s a real danger that we’ll end up creating incomprehensibly alien intelligences whose motivations we won’t understand. I think we can still mitigate it by incorporating natural features into the AI paradigm, and actively using the lessons of evolutionary biology and developmental psychology to shape the inherent nature of the AI agents we create.
If interested, you can read more detailed pieces on my substack “Bounded Alignment”.
Thanks!
One problem I have with all such proposals is that I don't think they're anywhere near mature enough. Another is that they typically don't do much to address gradual disempowerment (https://gradual-disempowerment.ai/).
P.S.: I wouldn't say I'm alarmed; I've been watching this unfold in slow motion for the last 15 years, and can't say I've had many surprises.
You’re right that these proposals are not mature enough, but that can mature over time. But not enough people in AI think in these terms, or engage with research in psychology and neuroscience. The entire paradigm of how we think of AI agents will need to change, so we recognize them as living entities and treat them accordingly.
The alternatives are all either extremely unlikely (stopping AI work) or insufficient (control, preference alignment).
I love the way you speak clearly about this. No bull-shit, no cope, no "trying to seem sensible". Just plain honesty based on what you have learned so far.