Let's think about the black swan event. The black swan event is when you get a model that's good enough to automate a bunch of labor, but not good enough that it can build new drugs and prevent cancer and make you live for 200 years and all of this other stuff. So there's a gap. And if you can stop it here and it doesn't get to there, now you do have the worst of all worlds. You have this thing that kind of displaces labor. No new things come after it because we stop innovating.

That's not really scary with unregulated AI. Like you could have snuff porn.

By the way, the main problem with social with AI that I see is not so much that they will bring calamity on their own, it's that they would drive people to do crazy stuff. So they will manipulate the human mind in ways that will make us the robots. It will not need access to the physical world. It will control the minds of people in a way that will create a lot of damage.

You're sitting there saying this will free us from our problems, but it's still humans that program the Nexus 0.7 so it could actually amplify the problem. I'm saying we should be wary of all of them.