In early October 2026, attackers reached environment variables at AI infrastructure company Fireworks and took API keys and tokens belonging to a limited number of accounts. Weeks later, the company disclosed a second, wider incident. According to BleepingComputer, a third-party cloud provider that hosted Fireworks research and development machines had a security incident on October 23, and an unauthorized party used it to reach those machines the next day.
This time the exposed material included billing records and, for some customers, incomplete model weights and datasets. Fireworks says it detected the activity the same day and has found no further successful access to the platform customers use.
The details of this incident are still emerging, and Fireworks has not named the provider. But the pattern is clear enough to learn from. The real lesson is that AI data now lives in places most security programs do not yet inventory: with inference vendors, on their development machines, and behind their vendors in turn.
What Fireworks Has Disclosed
Fireworks runs an inference platform for open-source models. In its update, summarized by Cyber Security News, the company reports that attackers accessed resource metadata such as identifiers, names, descriptions, and creation times. They also accessed billing information: billing and shipping addresses, invoices, payment history, account balances, and credit usage.
For some customers, a subset of incomplete model weights and datasets was also reached. Cyber Security News notes that the disclosure does not establish theft of complete production models. Fireworks says it does not store payment card numbers, so those were not exposed, and it reports no detected impact to its Inference, Training SDK, or Nexus services.
The company says the evidence points to a financially motivated group that sells unauthorized access to large language models through illicit routing gateways, a practice known as LLMjacking. It has engaged Palo Alto Networks' Unit 42 for incident response, and the investigation is ongoing. We should be fair here: the company detected the second incident quickly, rotated credentials, isolated affected systems, and told customers what to do. Some details, including the number of affected customers, have not been published.
Fireworks also paused self-service dataset uploads and downloads for Batch Inference and dataset-based training while it investigates, according to Cyber Security News. That is a sensible containment step, and it shows how much of the exposure sits in the dataset workflow specifically.
Your Vendor's Vendor Holds Your Data Too
The entry point was not Fireworks itself. It was a provider hosting Fireworks development machines. Customers who chose Fireworks as a vendor never chose that provider, and probably never saw its name in a contract.
This is the sensitive data supply chain at one more remove. We described the general problem in The Sensitive Data Supply Chain: every copy of data creates another place it can be exposed. A dataset uploaded for fine-tuning is a copy. A copy staged on a development machine for testing is another. When the hosting layer under those machines is compromised, every copy on it is in play, whatever the vendor's own controls looked like.
Datasets and Weights Are Data, Not Just Assets
Teams often treat model weights as intellectual property and datasets as raw material. Both deserve the same classification as any other sensitive data. Fine-tuning sets are routinely built from support transcripts, clinical notes, contracts, and customer records. Weights trained on that material can carry traces of it.
That matters because AI workflows are good at making copies. Data is exported, cleaned, uploaded, versioned, and re-uploaded, often by individual teams moving quickly. Each step creates a file or table that sits outside the systems the security team normally watches. The data that goes in is rarely less sensitive than the data it came from.
Why Deleting a Secret Is Not Enough
The first Fireworks incident adds a second lesson. Per Cyber Security News, Fireworks reminds customers that deleting a stored secret does not invalidate the underlying credential. If a key was ever readable in an environment variable, it has to be rotated at the issuer, not just removed from the place it was stored.
We covered the broader credentials problem in our look at the Zeabur breaches. The point carries over: a secret in readable form is exposed the moment the environment around it is. And a stolen API key for an AI service has a direct market, since LLMjacking turns it into resold model access.
The practical consequence for customers is uncertainty. Without a published count of affected accounts, each customer has to ask for itself whether its data was involved, which datasets it uploaded, and what those datasets contained. Teams that already know what they sent, and in what form, can answer in hours. Teams that do not can take weeks.
Five Questions to Ask Before Sending Data to an AI Vendor
- What data are we sending? Does the model need real names, account numbers, or diagnoses, or would masked or tokenized values produce the same result?
- Where else does it go? Ask which subprocessors and hosting providers touch your datasets, including development and test environments.
- How long does it stay? Training files and exports should have a retention period and a deletion step, not an open-ended home in a storage bucket.
- Is the data protected if the platform is not? Encryption in transit and at rest protects the pipe and the disk. Ask what protects the data when an authorized process, or someone impersonating one, reads it.
- Can we rotate everything quickly? Keep an inventory of keys and tokens issued to each AI service, and practice revoking them at the source.
OnData's Take
The Fireworks incident shows that the boundary of your data is no longer the boundary of your network or even your vendor's. Once data reaches an AI platform, it may be copied, staged, and hosted in places you do not control. Security that depends on every one of those places holding up will eventually fail somewhere.
OnData SecureAI is built for this stage of the pipeline: preparing, protecting, and governing sensitive data for analytics and generative AI, with de-identification, masking, and need-to-know access, so what reaches a model or a vendor is no more than it needs. For the files and databases that feed those workflows, OnData SecureFile and OnData SecureDB keep protection attached to the data itself, including copies that leave your control. For related reading, see Access Control Best Practices for Agentic AI.
Your AI vendor's security is only part of your security. Protect the data before it leaves, and a breach down the chain becomes a smaller story.
