Concerns Over AI’s Rapid Development
The latest warnings from artificial intelligence researchers regarding the swift pace of its advancement have sparked widespread apprehension about the potential consequences for humanity. These alerts stem from the notion that, driven by fierce competition and rivalry between the United States and China, companies are racing towards the creation of superintelligent models whose capabilities will far surpass those of humans.
However, Father Paolo Benanti, a priest who advises the Catholic Church on matters related to AI, argues that this controversy diverts attention from a crucial discussion about how to impose limits on the companies developing these technologies. By framing the issue as one so intricate that only they can address it, he contends that major laboratories effectively exclude others from the conversation.
“Ethics cannot belong solely to the boardroom of a company. It must be balanced and debated within the public sphere,” Benanti explains. “We need democratic regulation.”
Calls for External Oversight
In an essay released following the recent safety warnings, Dario Amodei, CEO of Anthropic, advocated for mandatory external evaluations of new AI models prior to their public release. He also proposed some restrictions on the practice of “distillation,” where rival laboratories train their own AI using the outputs of proprietary models. Others have called for a complete ban on “recursive self-improvement,” a process in which AI utilises its programming capabilities to develop future iterations of itself.
While Benanti does not aim to restrict the methods used for AI development—fearing that such actions could stifle innovation—he is keen on establishing guidelines to ensure that these models remain under human control. Regardless of the form these guidelines may take, he asserts that it should not be the laboratories themselves that create them.
“The narrative from pioneering companies reflects their vested interests in gaining or maintaining market dominance,” Benanti states. When a small group dictates the rules, he emphasises, “it has a very clear name: a cartel.”
The Complexity of AI Developments
Benanti highlights the challenge of taking seriously the grandiose claims regarding the capabilities of the latest AI models, partly due to the economic, political, and personal incentives involved. While organisations like OpenAI and Anthropic support calls to slow down development, they are also preparing for initial public offerings, creating a multi-billion dollar incentive for market leadership. The tensions between the US and China, where some of the most advanced models are being developed, further complicate matters.
Anthropic declined to comment on this subject, while OpenAI and Google DeepMind did not respond to requests for input.
The Myth of Superintelligence
Benanti also critiques the manner in which certain sectors of the industry discuss “superintelligence” with a reverence typically reserved for deities. This perception positions AI as something so sophisticated that it eludes human comprehension, almost omnipotent. For instance, Sam Altman, CEO of OpenAI, once expressed ambitions to create a “magical intelligence in the sky.” Benanti argues that this perspective threatens to skew the discourse.
“Are we discussing something or someone? That is the real distinction. It is not a matter of faith, it is a matter of mathematics,” he asserts. “We are not creating the next divinity. We are merely deceiving ourselves into believing that there is something more than a statistical calculation at play.”
The Role of Religion in AI Discourse
As society grapples with the boundaries between humans and machines, and how to safeguard future generations from the adverse effects of technology, the emergence of AI presents an opportunity for religions to assume a new role in public discourse, according to Benanti. “Living through the challenges of our times is part of the Church’s mission.”
While Benanti is a member of the clergy, he does not officially represent the Vatican, the governing body of the Catholic Church. Instead, he serves as an advisor on philosophy, theology, and technology. He notes that he employs AI to assist in creating tools and materials for his university students, but refrains from disclosing how Pope Leo XIV relates to technology, if at all, citing a “confidentiality agreement.”
Papal Warnings Against Technocratic Logic
In May, Pope Leo published his first encyclical, a letter that applies religious teachings to contemporary issues and clarifies the Church’s stance. He chose the topic of artificial intelligence as its focus. Within this document, the Pope cautioned against the technocratic logic that permeates the AI sector, which presents “efficiency as the ultimate measure of productivity” and reduces humans to “a project to be optimised.” He also called for international collaboration among rival superpowers and AI companies, urging them to move beyond the “desire to secure geopolitical or commercial dominance” and to consider the common good.
In one of the more prophetic sections, he warned of the “risk that humanity may become a victim of its own achievements.” Benanti believes that the debate surrounding superintelligence risks overshadowing the aspirations laid out in the encyclical, encouraging individuals to relinquish their voices to a select group of mathematicians, computer scientists, and executives from large corporations.
“What we need first is brakes and a steering wheel,” Benanti declares. “That means humans must have meaningful control over the machine.”