Software development is going through something of a revolution due to the rising importance of AI—as with any revolution, risk isn’t far behind.
Generative AI in particular has become a focal point for developers. The technology’s ability to be deployed for testing software makes it “a Swiss Army knife for tech,” Cars24 VP of Engineering Deepak Gupta wrote for Forbes in July.
Trusted tool. That Swiss Army knife usage of generative AI is most clearly shown in the software engineering testing process, BlinqIO co-founder and CEO Tal Barmeir told IT Brew. Developing code can lead to bottlenecks because of the need for testing—automating as much of that process as possible can lead to a quicker and more efficient software supply-chain. AI test engineers, with human oversight, allow for a better testing regime, Barmeir said.
“There is a human in the loop just to oversee stuff—similar to if you had employees that would write the code, you’d have a manager that would sort of code review it before it’s submitted into the main testing process,” Barmeir said. “But basically, all the work is done by the AI test engineers.”
BlinqIO isn’t the only company deploying generative AI for software testing. In March, DataCebo—a company started at MIT—said its Synthetic Data Vault is helping to utilize generative AI for software testing by bringing its data to bear on the process.
“You need data to test these software applications,” company founder Kalyan Veeramachaneni told MIT News. “Traditionally, developers manually write scripts to create synthetic data. With generative models, created using SDV, you can learn from a sample of data collected and then sample a large volume of synthetic data (which has the same properties as real data), or create specific scenarios and edge cases, and use the data to test your application.”
Top insights for IT pros
From cybersecurity and big data to cloud computing, IT Brew covers the latest trends shaping business tech in our 4x weekly newsletter, virtual events with industry experts, and digital guides.
Builders only. As IT Brew has reported, AI’s role in software development is growing but has yet to completely revamp how the industry operates. One expert urging caution is Gartner analyst Philip Walsh, who noted that getting people to change the way they do things is easier said than done.
“Developers are creatures of habit, and we all have our muscle memory in terms of how we get work done,” Walsh told IT Brew. “You can’t just give someone a new tool and say, ‘Oh, you’re going to be 50% more productive now.’”
Jim Routh, chief trust officer for Saviynt, told IT Brew AI in the software industry, especially in testing, is “monumental in its upside potential for business benefit.” But he cautioned that the upside should be understood in the same way that all AI integrations are—specifically warning about the possibility of inaccurate outputs and the need for filtering and human control throughout the process.
“That’s generative AI done well, that’s responsible generative AI,” Routh said. “But what I’ve just described is not mainstream.”
Barmeir told IT Brew that AI testing of the kind her company does avoids threat complications in part by ensuring that the automated systems cannot engage in curiosity or creativity.
“We basically reduce the creativity and flexibility level to zero, which makes sure that the AI engine provides stable and consistent results,” Barmeir said, adding, “no code is actually submitted into the main branch before a human actually goes through it.”