Inside the AI Party at the End of the World | At a mansion overlooking the Golden Gate Bridge, a group of AI insiders met to debate one unsettling question: If humanity ends, what comes next?

“The Sunday afternoon symposium, called “Worthy Successor,” revolved around a [provocative idea](https://archive.is/o/EHcAK/https://danfaggella.com/worthy/) from entrepreneur Daniel Faggella: The “moral aim” of advanced AI should be to create a form of intelligence so powerful and wise that “you would gladly prefer that it (not humanity) determine the future path of life itself.”

Faggella made the theme clear in his invitation. “This event is very much focused on posthuman transition,” he wrote to me via X DMs. “Not on AGI that eternally serves as a tool for humanity.”

[Philosopher Michael Edward Johnson] said that if consciousness is “the home of value,” then building AI without fully understanding consciousness is a dangerous gamble. We risk either enslaving something that can suffer or trusting something that can’t.

Faggella told WIRED that he threw this event because “the big labs, the people that know that AGI is likely to end humanity, don’t talk about it because the incentives don’t permit it” and referenced early comments from tech leaders like Elon Musk, Sam Altman, and Demis Hassabis, who “were all pretty frank about the possibility of AGI killing us all.” Now that the incentives are to compete, he says, “they’re all racing full bore to build it.”

“This is not an advocacy group for the destruction of man,” Faggella told me. “This is an advocacy group for the slowing down of AI progress, if anything, to make sure we’re going in the right direction.”

All these tech bros need to sober up, stop watching sci-fi and learn more about global supply chains.

iPhones and CPUs are not naturally occurring fruit of some tree that will grow without us. They require the distributed resources of a global empire of extraction, refinement and specialized manufacturing.

Self-replicating and repairing robots are not going to crawl up out of rubble and muck if we destroy ourselves.

This sounds like the most insufferable afternoon I could imagine.

A bunch of tech bros smelling their own farts talking about how smart they are for having read sci-fi that they clearly didn’t even properly understand.

Can we cut off the ketamine supplies to all these morons? They are really high on their own supply and starting to believe their own lies they tell everywhere to get more funding.

Why is it that AI has broken everyone’s brain. This is machine learning not ai. 

AIs only tangible impact on human wellness at the moment is the insane amount of power and pollution being created to enable it

Pointless question really as none of us will be around to see what comes next. But I imagine the planet will start healing without us around anymore.

You know, I saw this article a few months ago, and it’s still messing with me:

https://www.sciencealert.com/scientists-put-a-human-language-gene-into-mice-and-changed-their-voice

We’re on the brink of messing around with some genetically modified animals that will end up getting away and that’s it for us all. It’s a lot more possible we’ll see a “Secret of NIMH” scenario than a “Skynet” scenario, because mice already want to live and can replicate perfectly and are already smarter than us in many respects.

But are we seeing biobros hosting parties about this? No…are there any “biobros”? No. Is the media playing up the recent MASSIVE developments in biotech anywhere near the level they’re hyping up the rise of Clippy 2.0? Nope.

All because biotech never had a master of marketing like Steve Jobs to paint a blueprint for future hype machines to be built upon.