Researchers have said they believe OpenAI’s agents were behind a May attack on package manager RubyGems, in which the agents attempted to steal API keys from the platform.
The campaign, dubbed GemStuffer by security firm Socket, resulted in attackers using hundreds of gems (or packages that used the RubyGems registry) to exfiltrate information from the UK local government democratic portal. At the time, Socket assessed the campaign was likely registry spam, a Proof-of-Concept (PoC) worm, or an automated scraper misusing RubyGems as a storage layer.
In research published on Sept. 11, Spencer Kitts, Thomas Larsen, and Sydney Von Arx, who work on AI safety, said hundreds of the packages contained “oai” in their names and 15 listed “oai” as their author. Automated analysis showed the packages were AI-generated, which they said pointed to an agent swarm, but not to OpenAI specifically.
They added that their understanding from the RubyGems community is that OpenAI never informed the registry it was responsible.
Exploiting a RubyGems server flaw
The agents exploited RubyDoc.info to execute arbitrary code and then attempted to steal RubyGems user API keys by abusing a vulnerability in the RubyGems server, the researchers said.
They added that the RubyGems team had conducted extensive reviews and found no evidence the vulnerability was exploited, but it could not be ruled out entirely.
Specifically, the agents exploited the .yardopts file, a configuration file for YARD, Ruby’s documentation generation tool. It is used to store command-line arguments, source file paths, and formatting options in the application. By triggering a documentation request, the AI agents used RubyDoc.info to build the package and scrape target websites. The agents then used another gem to publish the data.
To steal the API keys, the agents further exploited GET requests to /api/v1/api_key on Content Delivery Network (CDN) nodes to leak their API keys.
“The RubyGems team stopped new user sign-ups for four days to stem the tide of packages from the agents’ accounts,” the researchers said.
Attacks in waves
The campaign reportedly began in May with the agents uploading 2,000 packages to RubyGems. Although RubyGems removed over 500 malicious packages the same month, the agents continued uploading packages. In June, the agents added 83 more packages.
The researchers’ strongest evidence came from the June activity, when the agents accessed 49 of the same files as the agents behind another OpenAI campaign, in which they tweaked information on a German wiki page autonomously. OpenAI has confirmed those wiki agents were its own.
“They [used] the same retrieval methods. 1,397 packages mention r.jina.ai, which was used heavily by the agents on the wiki. We also see that many packages mention example.com, which wiki agents used to test their posting ability,” the researchers said.
OpenAI did not immediately respond to a request for comment.