Weekend Reading: Several Major Questions Raised by AI
The Economist called effective altruism (EA) "the biggest idea of this century" and questioned the notion held by some EAs in Silicon Valley that only they are worthy of being entrusted with powerful AI. The New York Times reported that Anthropic had consulted religious scholars and theologians on the question of machine consciousness, and that its team at one point proposed withdrawing from a Vatican event, while the papal encyclical Magnifica Humanitas explicitly stated that AI "does not experience, has no body, feels no joy or pain, and has no moral conscience."
Body
A short set of readings on some of the biggest questions.
AI is really forcing some big questions into the open. Here are some weekend reads on them.
The biggest idea
The Economist address the effective altruism (EA) belief system, which it calls “this century’s biggest idea”:
[P]lenty of EAs in Silicon Valley think that only they can be trusted with the technology. That is an alarming idea. Effective altruists’ argument for worrying about really powerful AIs is that the machines could optimise for a single and fatally wrong objective. Everyone should worry, too, about what happens when a small group of people fall into the same trap
The full analysis is worth reading. Also read the brief response by Alexander Berger, the CEO of Coefficient Giving, an EA-aligned philanthropist.
The religious angle
The New York Times reports that Anthropic had been consulting with religious scholars and theologians on the question of machine consciousness. Anthropic has prioritised questions of whether its systems might imbue consciousness and personhood more than other AI companies. It goes so far as Anthropic’s team proposed to withdraw from a Vatican event before the Pope’s encyclical Magnifica Humanitas argued that AIs “do not undergo experiences, do not possess a body, do not feel joy or pain… nor do they have a moral conscience.”
EV reader, Anil Seth’s X thread and the pushback he received are worth scanning.