Before deploying patches, IT specialist Roman Shain often runs them in a virtual environment on his machine. He installs the update, reboots, and then sees how the patch interacts with applications—mainly, to answer the question: Is everything still working?
Shain, of Nero Consulting, may open Word, for example, and try to type, start Outlook, or maybe go to YouTube to test video and audio. With one click, a virtual machine (VM) can be restored to its previous state. A VM is completely isolated from its host system, so any breaks caused by a bad patch are contained.
But you can’t really replicate a client system virtually, according to Shain. “I don’t know what people do, day to day,” Shain told IT Brew. “So, I can’t test everything.”
At some point you just have to roll out the changes…slowly.
Shain divides his firm’s approximately 50 customers into five days: Monday through Friday. Customers with fewer clients get patched first. If an issue—like loss of connectivity to a VPN or a printer—is found with the smaller sample size, problems are investigated and updates may be halted.
“We block the updates that we know caused our 10-user computers problems, [so as] not to affect our 250-user computer problems,” Shain told IT Brew.
A tool from Microsoft, released in July, phases patch rollouts according to “rings” of priority. The “Autopatch” automates updating of Windows 10/11, Microsoft Edge, and Microsoft 365 software and aims to automate a strategic deployment of fixes as IT professionals face the common patch challenge of balancing functional risk and security risk.
“Microsoft will continue to release updates on the second Tuesday of every month, and now Autopatch helps streamline updating operations and create new opportunities for IT pros,” said Microsoft in a July 11 post announcing the Autopatch arrival.
Top insights for IT pros
From cybersecurity and big data to cloud computing, IT Brew covers the latest trends shaping business tech in our 4x weekly newsletter, virtual events with industry experts, and digital guides.
The service creates four testing groups in Azure Active Directory that are progressively tested:
- A “test” group intended for IT administrators
- A “first” group of 1% of devices that get early dibs on the patch
- Then, a “fast” group (9% of devices)
- And, finally, the broad group (90%)
Kev Breen, director of cyber threat research at the cybersec company Immersive Labs, is a big fan of the design and its concentric rings, arranged by priority.
“You can set up a ring of devices that are maybe less risky and say, ‘Hey, deploy these patches over here first,’ and then if there’s success and we don’t get any feedback, then start to roll them out,” said Breen, on a recent panel.
Windows Autopatch uses the Windows Update for Business tool to update devices.
A slow rollout still can’t afford to be too slow, given how quickly attackers pounce on a vulnerability. According to a May 2021 threat report from Palo Alto Networks, attackers typically scan for flaws within 15 minutes of a CVE announcement.
“Most people will take a functional risk over a security risk, pretty much every time,” said Jeffrey Martin, VP of product at the application-security company Mend.
For many enterprises, especially those with a small IT department, the Autopatch option will make sense, said Dustin Childs, senior communications manager at the Trend Micro bug-bounty program known as the Zero Day Initiative.
“Especially for those folks who are really understaffed for patch management, they’re gonna love it, provided that it works,” Childs told IT Brew.—BH
Do you work in IT or have information about your IT department you want to share? Email [email protected] or DM @BillyHurls on Twitter.