Developers in the open-source community are warning that spammers are flooding their inboxes with inaccurate, AI-generated bug reports—and want to make it perfectly clear that the behavior is unacceptable.
In a blog post in December 2024, the Python Software Foundation’s security developer-in-residence Seth Larson warned of an “uptick” in security reports generated via large language models (LLMs)—specifically ones that are “extremely low-quality, spammy, and LLM-hallucinated.”
Larson, who is part of the security triage team for multiple open-source projects, wrote the reports often appear legitimate. But since LLMs don’t actually understand code or “human-level concepts like intent, common usage, and context,” he added, those reports are often riddled with errors and end up wasting volunteer time.
“In many ways, these low-quality reports should be treated as if they are malicious,” Larson wrote.
Daniel Stenberg, the founder and lead developer of the “curl” and “libcurl” projects, had penned a different blog post last year complaining that users who feed LLMs code to generate vulnerability reports distract developers from focusing on other bug fixes. Stenberg explained that each vulnerability report requires subsequent human review, draining time and resources.
“The better the crap, the longer time and the more energy we have to spend on the report until we close it,” Stenberg wrote.
Overall, Stenberg added, around two-thirds of the 415 vulnerability reports received by the project up to that date were worthless. In one case, Stenberg said he intervened when a user relying on Google Bard submitted a vulnerability report claiming to identify a flaw ahead of its actual disclosure date (it was bogus). In another instance, Stenberg wrote in the post, he suspected a user who submitted a report incorrectly claiming they had identified a buffer overflow vulnerability not only used AI to generate the original report but was communicating with developers via a LLM.
Top insights for IT pros
From cybersecurity and big data to cloud computing, IT Brew covers the latest trends shaping business tech in our 4x weekly newsletter, virtual events with industry experts, and digital guides.
While LLMs can be useful for bug detection, the humans using them to detect bugs have to be capable of reviewing the work themselves first, Stenberg concluded in the post: “I suspect that will be true for a long time into the future as well.”
According to Larson, AI vulnerability spam exists because it’s an “easy win” for security researchers or anyone else who wants to take credit for spotting bugs while “[spending] the least amount of time possible” actually contributing productive work. In the case of the curl project, he speculated one motive might be financial, as curl runs a bug bounty program with payouts.
“A lot of projects don’t have that, including Python, and yet we still get some,” Larson said.
The behavior is particularly egregious because the open-source community by and large relies on volunteers to maintain and secure software, according to Larson.
Larson said he hopes to get out ahead of what he sees as a likely broader trend and put would-be spammers on notice that volunteers might not take well to such a breach of basic etiquette.
“I don’t think there are very many feasible, scalable technological solutions to this,” Larson said. “And so I would like to pull the social lever, which is, ‘We’re just going to shame you.’”
“People get uncomfortable about it, but realistically, I don’t know what else to do about this, other than making it unacceptable on a social level to use [these] tools in a way that is abusing volunteer time.”