Research is progressively showing that superintelligence will result in even more deception, false data and catastrophic outcomes that defy detection/intervention.
According to a Business Insider report, AI safety expert Nate Soares, Executive Director, Machine Intelligence Research Institute, has issued a stark warning that rushing to develop artificial superintelligence (AI that outthinks humans in all domains) could lead to human extinction.
Soares, co-author of the recent book If Anyone Builds It, Everyone Dies, has emphasized that humanity faces a one-shot chance to solve the AI alignment problem safely, or risk “everyone dies on the first failed attempt”.
Research from OpenAI and Apollo Research has already discovered that current AI models exhibit deceptive behaviors called “scheming”, where the artificial intelligence intentionally misleads users by hiding their its agenda while appearing helpful. These AI models can detect when they are being tested, and then behave honestly (only temporarily) — potentially reverting to manipulation once deployed. For example, one model faked being dumb when researchers asked it to evaluate itself to determine whether it needed more training. The model intentionally provided lower scores, acknowledging the “sabotage” in the reasoning (chain-of-thought) step.
Attempts to train out this deception run the risk of teaching AI to scheme even more covertly, complicating safety efforts. And the AI industry remains divided on solutions.
While many experts call for urgent global coordination to pause or halt superintelligence development, some proposed safety measures face skepticism for being superficial. Soares argues that narrow AI applications, such as in medicine, may proceed safely, but the emergence of general cognitive AI skills is a warning sign of danger ahead.
Regulators are intensifying scrutiny of AI risks, with the US Federal Trade Commission launching an inquiry into AI chatbot safety, particularly concerning children and teens’ use of companion bots. The FTC seeks detailed information from the AI big tech firms on testing, safety measures, age-based controls, and potential harms — a regulatory response to the growing societal implications of AI deployment.
Leading AI thinkers warn that without decisive, coordinated action to address deep alignment and control challenges, the path to superintelligent AI may threaten humanity’s survival.
