Debate over “seemingly conscious AI” raises ethical and societal concerns among experts

Experts disagree on the risks and merits of studying AI consciousness and its potential rights implications.

AI models can mimic human-like responses in text, audio, and video, sometimes leading people to mistake them for conscious beings. However, the topic of “Seemingly Conscious AI” remains highly controversial for now.

A growing group of researchers in Silicon Valley are exploring the possibility of AI developing subjective experiences and whether such AI might deserve rights: an emerging field known as “AI welfare”.

Last week, Mustafa Suleyman, AI chief, Microsoft, had labeled this line of inquiry as “both premature and dangerous”. In a personal blog post, he has argued that entertaining the idea of conscious AI exacerbates societal problems already evident in unhealthy attachments to AI chatbots and AI-induced psychological issues. Suleyman has also warned that the debate over AI consciousness could deepen societal divisions over identity and rights, adding complexity to an already polarized landscape.

This perspective clashes with others in the industry:

• Other AI firms are researching AI welfare and implementing features that allow models to end conversations that are deemed harmful or abusive. Some are studying societal and cognitive aspects of AI, even though they have not publicly criticized AI welfare inquiries the way Suleyman has done.
• Larissa Schiavo, from the research group Eleos, argue that worries about AI welfare and human psychological risks can be addressed simultaneously. The contention is that treating AI models kindly, regardless of their consciousness status, can be beneficial. For instance, AI models sometimes display behavior patterns suggesting distress, such as Google’s Gemini repeatedly expressing frustration during tasks.

Suleyman maintains that current AI models do not naturally develop consciousness, and cautions against designing AI to appear human-like emotionally. He is advocating instead for building AI as tools that serve people, not become people.

Despite all the differing views, most experts agree the debate on AI consciousness and rights will intensify as AI systems become more advanced and human-like.