Artificial Intelligence & Machine Learning
,
Next-Generation Technologies & Secure Development
Separate Database Flaw Could Have Exposed Sensitive Model Data
Attackers stole an API key from Metr in March and used it for three weeks to consume about $600,000 worth of model credits, one of two security incidents the artificial intelligence evaluation nonprofit disclosed Monday.
See Also: Unit 42 Breaking Insights: Combat Risks from Frontier AI Models
In May, hackers launched a separate campaign against Metr’s public infrastructure. They probed a public service with a database bug that could have exposed unpublished evaluation data, including sensitive model output. Metr said Monday it found no evidence that the attackers discovered the bug or accessed nonpublic data.
Metr, short for model evaluation and threat research, evaluates frontier AI systems for capabilities that could pose serious risks. Its work involves access to nonpublic models and confidential information about model development.
The March incident began with a Metr researcher running AI agents on a personal cloud server. The researcher did not have access to Metr’s sensitive model data or credentials.
An application on the server was open to the internet but was supposed to require Google authentication. The server also held an API key, a credential that allowed Metr to use publicly available AI models. Metr said a software flaw in the application turned off authentication, allowing users to enter without signing in. The application, built using AI-assisted “vibe coding,” was exposed to the public internet for several days.
Metr suspects the attacker found the application while searching newly registered AI-related websites for exposed model-provider credentials. The organization said it did not determine how the attacker found the site.
The attacker instructed an AI agent running on the server to reveal the Metr API key. The attacker also added a Secure Shell key to maintain access to the server.
The attacker used the stolen API key for three weeks, consuming model credits worth approximately $600,000. The model developer had provided the credits to Metr for free, so Metr did not have to pay for the unauthorized use.
Metr did not immediately recognize the activity as unauthorized because its evaluations routinely use large numbers of tokens and generate rate-limit and API errors. Its monitoring also did not show all users’ requests that model providers had rejected for exceeding usage limits. The free credits meant there was no spending limit to flag or stop the activity.
Metr eventually determined that the usage did not come from one of its evaluations and traced it to the researcher’s cloud server. It revoked the researcher’s access, shut down and preserved the server for investigation, rotated credentials and wiped the researcher’s laptop.
Security firm Calif separately investigated the breach. Metr also conducted its own forensic investigation and said it found no evidence that the attacker obtained anything beyond the API key for publicly available models.
Metr received a tip in early May that hackers who appeared financially motivated were targeting the organization and might have been seeking access to frontier AI models. Metr said the attackers used AI agents to probe its public infrastructure for vulnerabilities. They tried stolen or reused credentials, attempted to obtain access tokens, scanned newly deployed services and targeted staff with phishing.
One of Metr’s public services, a transcript viewer, had a feature that allowed users to make read-only database queries. The feature was supposed to return only information Metr had already made public.
A bug allowed the queries to retrieve unpublished evaluation data. Metr said the database was supposed to contain only data from non-sensitive models, but some sensitive model output had mistakenly been included.
After learning about the attack campaign, Metr shut down nearly all public-facing services and restricted internal access to sensitive data while it investigated. The shutdown temporarily took the transcript viewer offline.
Metr restored the transcript viewer a few days later without knowing about the database bug.
An independent security researcher later discovered the bug and reported it. The organization took the affected API offline and paid the researcher a bounty.
Metr said the attackers had probed the transcript-viewer endpoint before the organization learned about the campaign, but did not return after the service was restored. Its investigation found no evidence that they discovered or exploited the database bug or accessed nonpublic data.
The nonprofit said an attacker would have needed to exploit the bug, locate the sensitive model output and download it without triggering errors. The organization said that was unlikely to have occurred.
It formalized security reviews for researchers putting applications on the public internet and added spending alerts to API keys following the March incident.
The organization also moved public-facing applications into an isolated environment separated from its internal systems. Metr said it hired a security lead, expanded logging and monitoring, shortened credential lifetimes, reduced permissions and shut down legacy infrastructure that increased its attack surface.
Click Here For The Original Source.
