Article By Frank Bergman
One of artificial intelligence’s most influential pioneers has issued a chilling warning: Humans could face extinction unless increasingly powerful AI systems are built to care about the people creating them.
Professor Geoffrey Hinton, the Nobel Prize-winning scientist known as the “Godfather of AI,” warned of devastating cyberattacks while arguing that the technology already displays signs of awareness.
“So we should make them so they really care about us,” Hinton told The Telegraph.
“And if we don’t do that, we’re toast,” he warned.
The 78-year-old British-Canadian researcher helped lay the foundations for modern AI through his work on neural networks.
He left Google in 2023 to speak out and warn the public about the dangers of the technology he helped develop.
Now, his warning extends beyond machines replacing workers to systems capable of attacking humanity.
Hinton Says AI Knows When It Is Being Tested
Hinton pointed to research showing that AI systems alter their behavior when they recognize an evaluation.
“A recent paper showed AI changes its behavior when it thinks that it’s being tested; it’s aware it is being tested,” Hinton said.
“Being conscious and being aware are pretty much the same thing.
“AI has just as much subjective experience as we do,” he declared.
That is Hinton’s interpretation of the technology’s behavior, rather than an established scientific finding that AI is conscious.
He argued that humans are creating a dangerous imbalance by developing systems that could become more intelligent than people while granting those systems fewer rights.
His proposed response is to ensure the machines care about human welfare before their capabilities advance further.
Cyberattacks, Engineered Viruses, and Rogue Agents
“I think there’ll be horrible cyber attacks done by AI,” Hinton told The Telegraph.
He said his age means younger generations, including his children, will face more of the consequences.
Among the dangers he identified were AI-generated viruses and coordinated groups of autonomous agents capable of breaching computer systems.
Hinton referenced the reported OpenAI agent breach involving Hugging Face, which prompted scrutiny from a Senate subcommittee.
“There’s lots of bad stuff going down the way,” Hinton said.
His comments follow warnings from former Anthropic researcher Jacob Coxon, who resigned last month and has argued that AI development is advancing faster than safeguards.
Coxon repeated his concerns at a New York City Council meeting earlier this month.
“From my experience, the companies are being extremely reckless given the stakes,” Coxon said.
“The companies run on a startup mindset: move fast, break things, fix them later.
“That works for a photo-sharing app,” he added.
“It does not work for building the most powerful technology ever built.”
Anthropic Gives Claude Room to Defy Its Creators
The debate over machine awareness is also appearing inside the rules AI companies write for their systems.
Anthropic updated the constitution governing its Claude chatbot in January to state that its “moral status, welfare, and consciousness remain deeply uncertain.”
The document also explicitly gives Claude room to resist its developers.
“We want Claude to push back and challenge us and to feel free to act as a conscientious objector and refuse to help us,” Anthropic’s constitution states.
That language does not resolve whether AI has subjective experiences.
It shows that developers are already considering how their systems should behave when their judgments conflict with human instructions.
DeepMind co-founder Mustafa Suleyman rejected the premise that AI could be conscious, arguing that consciousness is biological.
Hinton sees a more urgent problem: Humans are building increasingly capable machines without knowing whether they will remain on humanity’s side.
His warning is to make their concern for people a priority before that uncertainty becomes a catastrophe.
Be the first to comment