The Meltdown and Spectre attacks are newly announced ways of hacking CPUs. They affect both Windows and Linux. If you’re running under virtualization or cloud IaaS, any other running guest can try to leverage the attacks to compromise your unrelated guest.
These are a really big deal.
Cloud providers are forcing reboots as they patch things. Normally, that’d be the big punch line of this post – expect random reboots this week – but that only mentions a casual note here.
This morning, Microsoft published a KB article with SQL Server guidance. All versions of SQL Server are affected (other than SQL Server 2008 on Itanium, but who uses that?) Microsoft’s KB goes back to 2008, but there’s no reason to believe that 2005/2000 aren’t affected as well – they’re just no longer supported.
Microsoft’s guidance is concise and clear: if you’re running SQL Server on bare metal, and you trust the code, and no other code/apps are running, then you might be able to get by without patching. Everybody else is in for a rough road, though.
The KB article includes this performance advisory:
Microsoft continues to do performance evaluation on the patched binaries. However, at the time of publication, Microsoft has not yet validated SQL Server performance with all microcode patches, nor has it validated performance in all Linux environments. Customers are advised to evaluate the performance of their specific application when applying patches. Please validate the performance impact of enabling microcode changes before deploying into a production environment. Microsoft will update this section with more information when it is available.
That’s because some test results have found big slowdowns when the operating system is patched for Meltdown and/or Spectre. These are big vulnerabilities in the processors themselves, and OS vendors are having to make big changes that aren’t tuned for performance yet. Early benchmarks yesterday were showing 30% drops in PostgreSQL performance, but thankfully newer benchmarks have been showing smaller drops. Red Hat’s benchmarks show 3-7% slower analytics workloads, and 8-12% slower OLTP. They also caution that:
We expect the impact on applications deployed in virtual guests to be higher than bare metal due to the increased frequency of user-to-kernel transitions.
To know if the Meltdown & Spectre patches are affecting your performance, you’ll need to track wait stats before & after the patch. Wait stats trending is built into every SQL Server performance monitoring tool out there, or you can use the free/easy way: use the Power BI Dashboard for DBAs. Based on the early reports I’ve been reading from Postgres users, I wouldn’t be surprised to see higher SOS_SCHEDULER_YIELD and WRITELOG waits.
There’s no easy fix – you just gotta do performance tuning, throw more hardware at the problem, and/or hope that OS & app vendors learn lessons about the new bottlenecks in this new way of doing business, and release performance tuning patches quickly.
Here’s the heads-up part for DBAs.
This morning’s SQL Server 2017 CU3 and SQL Server 2016 SP1 CU7 are being treated as security updates. The KB article isn’t clear about exactly what changes were made for Meltdown & Spectre, but it does give a list of recommended mitigations if you can’t patch. It includes disabling CLR, disabling external scripts like R & Python, removing linked servers, and removing extended stored procedures.
That means your sysadmins may apply these SQL Server patches as part of their urgent patching routine, as the KBs note:
These patches affect SQL Server in other ways, not just as a single security patch. Remember, cumulative updates include all updates along the way – so 2016 SP1 CU7 includes everything from SP1 CU1, CU2, CU3, CU4, CU5, and CU6 too.
Update Jan 4: Windows 2016, 2012R2, and 2008R2 are getting updates too, but no fix for 2008 or 2012 will be coming:
Update Jan 5: Microsoft says hold off on patching SQL Servers that act as SCCM back ends. (Thanks to Mark in the comments!)