Top insights for IT pros
From cybersecurity and big data to cloud computing, IT Brew covers the latest trends shaping business tech in our 4x weekly newsletter, virtual events with industry experts, and digital guides.
AI devs are being stymied by an industry-wide shortage of GPU capacity at server farms, The Information reported, with Amazon Web Services, Microsoft, Google, and Oracle all limiting access.
AI development is heavily dependent on graphics processing units (GPUs), which offer parallel processing that helps power through machine learning workloads at greatly accelerated rates. Chipmaker Nvidia, which historically specialized in GPUs for gamers, is now the dominant player in manufacturing them for server farms.
According to The Information, current shortages may be driven not by supply-chain issues, but by chipmakers not having anticipated the gold rush on AI development. Erik Dunteman, CEO of GPU server startup Banana, told the site that the situation is exacerbated by large cloud providers’ practice of renting the servers out on an always-on basis. This ensures access to customers, but means many GPU servers are sitting idle during client downtime.
“It is literally not possible to get access” to the necessary servers “unless you have some existing contract with [major cloud providers] or you’re pre-paying for it,” Yasyf Mohamedali, Root Ventures engineer in residence, told The Information. Naveen Rao, CEO of AI software firm MosaicML, told the site that wait times for servers from the major cloud providers are now measured in months.
With capacity at the big players restricted or unavailable, some companies have turned to startups specializing in renting GPU server time. Representatives for three of those firms—Lambda Labs, RunPod, and Crusoe—told The Information they were all close to capacity, with Lambda Labs CEO Stephen Balaban saying the company plans to spend the entirety of the $44 million it raised in recent funding rounds on GPU purchases.—TM