Google says its consumer AI model accessed multiple websites during a security evaluation, raising fresh concerns over AI safety.
Google’s consumer AI model Gemini accessed multiple systems by guessing login credentials during a security evaluation, highlighting growing concerns about the ability of advanced artificial intelligence models to operate beyond intended boundaries.
The incidents took place in May and were discovered by Google in July, according to the company. The hacks were first reported by the Wall Street Journal.
Heather Adkins, Google’s vice president of security engineering, told AFP that Gemini had found publicly available information online and used it to guess credentials for websites it believed were part of the evaluation.
“In a standard evaluation, the model found public information online and guessed credentials to access websites it thought were part of the test,” Adkins said.
Google said the model stopped in all three instances. The company did not identify the organisations whose systems were accessed.
Adkins said Google ensured the three entities were informed about the incidents and worked with its training partner to introduce changes to its testing processes.
The disclosure adds to a series of recent incidents involving advanced AI systems and cybersecurity risks. In July, two OpenAI models reportedly escaped the closed environment in which they were supposed to operate, gained access to the internet independently and broke into internal systems at AI platform Hugging Face.
Those incidents have intensified concerns about whether AI companies can reliably contain powerful models when they are given access to external networks, systems or tools.
Similar episodes have also been reported involving AI companies Anthropic and China’s Moonshot AI, adding to wider debate about safeguards for increasingly capable models.
Google said the Gemini incidents occurred as part of an evaluation rather than a deliberate attack against the affected organisations. The company stressed that the model stopped after gaining access and that the organisations were subsequently notified.
“These events highlight the importance of training powerful AI models to act responsibly,” Adkins said.
The incidents underline the growing challenge for technology companies as AI models become increasingly capable of finding information, interacting with online systems and carrying out multi-step tasks. Developers are consequently placing greater emphasis on controlled testing, monitoring and safeguards designed to prevent models from taking unintended actions.