Everyone is navigating AI security in real time – even Google

I recently had the opportunity to sit with Francis de Souza, COO of Google Cloud, backstage at an event in Los Angeles. Amid the noise all around us, de Souza, who speaks in the calm, measured manner of a university professor, offered useful advice for companies navigating the AI ​​security moment we’re all living through, noting that “there will be a transition period, and then I think we’ll get to this better place.”

He wasn’t speaking about Google at the time, but it’s clear that Google is still figuring things out.

De Souza’s key message was what security professionals have been trying to internalise for years as authorities have now become imperatives: security cannot be an afterthought. “As companies move forward on this AI journey, they need to adopt a platform approach,” he said. Security must be integrated from the start; it cannot be forced on later or left to employees. He specifically warned about “shadow AI” – employees accessing consumer devices without organisational oversight – and argued that companies need to demand security, governance and auditability from their platforms from the start. “There is no such thing as an AI strategy without a data strategy and a security strategy. They need to work together.

Worth noting: He wasn’t alone in advocating for Google Cloud. When I noticed that his advice sounded like a Google ad, he backed off. Google is committed to a multicloud approach, he said, and he made the case that all companies are definitely operating on multiple clouds. “Even if they choose the same cloud and they’re relying on SaaS applications, there are business partners who may use different clouds,” he said. “It is important for companies to have a security posture that is consistent across all models.”

He also said that the threat landscape has changed so fundamentally that old defensive models are too slow. He said the average time between the initial breach and reaching the next stage of an attack has dropped from eight hours to 22 seconds, and the attack surface has expanded well beyond the traditional network perimeter. In addition to your common assets, you now have models available. You have data pipelines used to train the models. You have agents, you have signals. All of these need to be protected.”

De Souza flagged a danger that doesn’t get enough attention: Agents going through a company’s internal systems could uncover forgotten data repositories that no one has thought about in years. Many organisations have old, unupdated SharePoint servers and access controls, but it didn’t matter because no one really knew where they were. But agents roaming around your enterprise will find those data assets and expose the data on them.’

In his view, the answer lies in matching the speed of the machine with that of the human. “We are now seeing the emergence of AI-native, fully agentic defence, where organisations can run agents to protect themselves,” he said. “Instead of human-led defence or even having a human in the loop, you can now have humans overseeing a completely agentic defence.” He said that the situation has become not just an issue of technology but of leadership. “This matter is a board-level issue and an executive team issue. This is not just a security team issue.”

But even as AI takes on more defensive workloads, there is a shortage of qualified people to oversee it – and the vulnerabilities AI itself is introducing are growing faster than security teams can handle. “We will need people to deal with the bug-pocalypse,” said Lee Kisner, LinkedIn’s chief information security officer. He told the New York Times this week that he does not expect the industry to understand AI safety in any sustainable, long-term way for at least several years.

This brings us back to the platform providers themselves. The Register has published a series of reports over the past several weeks documenting a wave of Google Cloud developers being hit with five-figure bills after Gemini models made unauthorised API calls — many of which were services they did not use or that were unintentionally enabled. Matters followed a familiar pattern: API keys originally deployed for Google Maps, which were kept public according to Google’s own instructions, were quietly able to reach Gemini after Google expanded its scope without explicitly disclosing the change.

Rod Dannon, CEO of interview-prep platform Prentus, said that external factors have affected his bill. $10,138 in about 30 minutes after attackers exploited its compromised API key. Isuru Fonseka, a Sydney-based developer whose account was similarly compromised, was charged almost $17,000 AUD when he believed he had a $250 spending limit. Neither of them knew that Google’s automated systems had upgraded their billing tiers based on account history, increasing their effective limit to $100,000 without explicit consent.

Google retracted both after The Register published its initial report. Nevertheless, Google told The Register that it has no plans to change its automatic tier-upgrade policy, saying it prefers to prevent service interruptions rather than enforce users’ stated budget preferences.

Meanwhile, it’s a different question what happens when a developer tries to shut things down. This week, research by the security firm Aikido has shown that even developers who capture a compromised key and then immediately delete it may not be safe. According to Aikido’s findings, the attackers apparently could continue using that key for up to 23 minutes as Google’s revocation slowly spread across its infrastructure. Aikido researcher Joseph Lyons told The Register that during that window, success rates are unpredictable — more than 90% of requests are still authenticated within a few minutes — and attackers can use the time to exfiltrate files and cached conversation data from Gemini.

Lyon also noted that Google’s own new credential formats don’t appear to have the same problem: Service Account API credentials are revoked in about five seconds, and Gemini’s new AQ-prefixed key format takes about a minute. “Both run on the Google scale,” he wrote in Aikido’s related paper. “Both suggest that the issue is technically solvable even for Google API keys.” In short, according to Lyons, the 23-minute window is not an engineering constraint but a matter of priorities for the company.

This is something to consider when you read de Souza’s sound advice, which you should take very seriously. He’s not wrong, but there is a gap between what the platforms are currently setting and how fast they are adapting themselves, and it’s good to be aware of that too.