Orgs are investing in generative AI, but like your duct-taped Shark Tank idea, many projects aren’t quite ready for prime time.
One major reason: It’s really hard to do AI without data, Jim Rowan, principal and head of AI at pro-services firm Deloitte, told IT Brew.
“Do you understand where the data is coming from, how it’s being used, but also, are you using any other third-party information? And what are the usage rights associated with that? And are you allowed to do that, and what are the ramifications—all those different components that you don’t normally have to think about when you see a great demo,” Rowan said.
According to an August 2024 quarterly report from Deloitte, 67% of surveyed organizations said they are increasing investments in the intriguing output machines known as generative AI. Seven out of 10 respondents also admitted their organization has moved 30% or fewer of their GenAI experiments into production.
Rowan spoke more with IT Brew about why data keeps great demos from seeing full-scale, production-level usage.
Responses below have been edited for length and clarity.
How is an IT pro’s job challenged as a project moves from proof of concept to production?
If an organization is starting from the beginning and saying, “We’re going to have a strategy around generative AI; we’re going to form a cross-functional team, and that team is going to include the right people: IT, legal, risk, compliance, key business owners,” it can scale pretty quickly because they’ve got the right team around the table. What I’ve seen and where IT struggles is when [IT teams say], “We’ve tried out a solution in a department, but we didn’t really involve all the other stakeholders. We didn’t think we needed to tap into the legal team because it was just going to be used within this one department.”
How has the testing process challenged IT teams?
Top insights for IT pros
From cybersecurity and big data to cloud computing, IT Brew covers the latest trends shaping business tech in our 4x weekly newsletter, virtual events with industry experts, and digital guides.
If you’ve deployed one version of a model—and we see new models coming out every quarter, every month, depending on the company—you’ve got to figure out how you’re going to be managing your large language model operations and then pulling and pushing different models into production. And as you’re doing that: what are the implications to the end solution you’re creating? The IT professional is now thinking through not just their traditional, “Hey, do I have the data to source this model?” but “Do I have the right to use the data to source this model? What’s the direct trajectory of the models that I’m using? What’s the right model fit for the use case that I have? Are there multiple models in play for a series of processes that I’ve connected on this use case?” It’s a fairly complex set of steps.
Do you imagine a majority of IT pros facing the complexity, the involvement of legal, for example, and saying, “Let’s just forget about this”? Or “We’ll keep it in the department, but we won’t even put it in production”?
This is going to highlight the importance of data in the organizations, because it’s the question of: Why are you involving data? “Well, because data is an asset for us.” If we just use generative AI without thoughtful processes, we could be eroding the value of our company; we could be putting personal, confidential information at risk. I think it’s going to highlight the importance of some of the foundational things that IT has been trying to drive for a while. When I talk with chief data officers, it’s: “Finally, people are interested in investing in data management, and they are because they can see the value finally and why this is really important for the organization.” They get excited about it.