A family plays midway up a stone staircase, with wreckage below and an emerald gate above

It was serendipitous that I finished listening to Eliezer Yudkowsky and Nate Soares’s If Anyone Builds It, Everyone Dies the day Mark Zuckerberg published The Future is for Everyone. And last Wednesday, Bill Gates published his own long essay about AI.

All of them seem to agree that AI is one of the great filters and is coming for humanity.

The filters we’ve already passed

The Fermi paradox says: if our galaxy is old and enormous, there must have been time for life to appear many times over, and for at least one of those civilizations to spread across it. However, we look, and we find nothing. So, where’s everybody?

The Great Filter is one answer to it. Getting from dead chemistry to a civilization spreading between stars takes a long series of steps, and if the sky is empty, at least one of those steps has to be almost impossible. Something stops nearly everyone. That step is the filter.

For humanity to get here, we must have passed multiple of those great filters. Life arising on Earth at all, out of dead chemistry. Single cells becoming multicellular organisms. Multicellular life producing a species intelligent enough to ask the question. Any one of those could have stopped us, and we cleared it.

What we don’t know is whether that great filter is behind us or in front of us. If it is behind us, we are rare and the sky is empty because almost nothing else got this far. If it is ahead, the sky is empty because everyone who got this far ran into something, and we are about to meet it.

Three views of the same danger

Yudkowsky and Soares start with how AI is made. Their central claim is that AI is grown, not crafted, and nobody can read the weights of a trained system and derive its goals. They believe an artificial superintelligence is imminent and will very likely kill everyone. Their remedy is a coordinated global halt on large-scale general AI training.

Zuckerberg’s argument begins with the balance of power. Humanity isn’t a monoculture, so a single superintelligence cannot represent everyone’s opposing values and interests. His conclusion:

“There is no such thing as a singular benevolent superintelligence.”

He wants personal superintelligence distributed broadly, with individual empowerment, invention and a balance of power as the path to safety.

For Gates the starting point is the transition itself. He says there is no plan to ease our entry into the AI era, and he names three places to begin: new national and international institutions, work kept “Human Reserved,” and a tax on AI tokens and robots. He would support a credible global slowdown, but he doesn’t think one is going to happen.

Each proposed solution is the danger the others warn about. Yudkowsky and Soares’s global halt requires worldwide enforcement, which is the concentration Zuckerberg warns against. Zuckerberg’s distribution puts powerful capability into more hands, which is the proliferation Yudkowsky and Soares fear. Gates reaches for institutions and taxes, more centralized than Zuckerberg wants and slower than the extinction scenario accepts.

Optimistic nihilism

“If we are all going to be destroyed by an atomic bomb, let that bomb when it comes find us doing sensible and human things—praying, working, teaching, reading, listening to music, bathing the children, playing tennis, chatting to our friends over a pint and a game of darts—not huddled together like frightened sheep and thinking about bombs. They may break our bodies (a microbe can do that) but they need not dominate our minds.”

That is C.S. Lewis in 1948, on living in an atomic age.

Most of us cannot pull any of the levers proposed by Yudkowsky, Zuckerberg or Gates, on any given Tuesday. We don’t decide whether frontier training stops, who gets access to superintelligence or whether new international institutions are built. Furthermore, not even them with their fancy levers can control whether an ASI rises and wipes us off the face of this planet.

I wrote a post in May, AI Will Never Have Shower Thoughts, arguing that what separates us from AI is our ability to exist in the world, to embody these meat bodies, what makes us human.

And now, if there’s a possibility that an ASI is coming for us, either for our jobs and livelihood or as one of the great filters that ends humanity, it’s the right time to protect that same humanity because that’s what makes us different from it. We need to protect the workplace chit-chat, the interactions with our neighbors, the walks in the forest, playing with our children.

Optimistic nihilism means nothing cosmic is guaranteed. If the universe has no purpose assigned to us, we get to decide what purpose matters. Because I don’t know if there’s a filter ahead. If there is, we should do this because our time is running out. If there isn’t, we should still do this because AI is supposed to make us more efficient with the things a machine is good at and give us more time to enjoy being humans.

The human interaction, our humanity, should still be there and should be protected.