This month, to mark the 20th anniversary of BrentOzar.com, I’m stepping back and looking at the big picture rather than blogging about database problems & solutions like I usually do.
Twenty years ago, I tried to switch to Linux.
That wasn’t my first attempt, nor was it my last – there were so many more – and every time, I had a terrible experience. I struggled to get the right combination of hardware, software, and configuration settings. But you know what the worst part was? Trying to get support for all of it.
I saw myself as a decently skilled sysadmin and developer with a good amount of experience and troubleshooting skills, but I had one hell of a bad time getting support. Things people said would work, didn’t, and nobody seemed terribly interested in helping me get it right. Documentation, forums, email lists – it was a byzantine mess, nobody was accountable for getting things right, and I felt firsthand the impact of gatekeepers and condescending answers. The more I tried to make it work, the more burned out I got.
In 2006, I gave up and got a Mac.
I gave up the idealistic hope of a complete open source stack, configuration via simple text files, and the feeling of being a wizard. I wasn’t an operating system wizard. I just wanted something to boot up, connect to WiFi and a VPN, install applications, and sleep & resume, and when something broke, I wanted to get support from a company that would be personally accountable for getting it working again. I was fine with handing over money and permissions in order to get simplicity.
Today, I maintain the open source
First Responder Kit.
It has SQL Server scripts like sp_Blitz that help you assess your SQL Server’s health and performance. They’re the same scripts I use in my own consulting work, and are used by data professionals around the world. Back when I used to track downloads, I was tickled pink to see that even Microsoft’s own support staff downloaded them.
Today, as an open source maintainer, I see the other side of the story.
Many users don’t read the documentation. They try to use the scripts in completely inappropriate and unsupported ways. They complain when the scripts don’t do what they want. They see all this as my own responsibility, saying things like:
- “You should make it be able to do ____”
- “I’m very disappointed that it doesn’t ____”
- “It needs to ____ or else it’s worthless”
- “Please do the needful”
When I point out the readme and the documentation, which clearly states that if you want a new feature, you need to start coding for it – it almost always falls on deaf ears. I’m endlessly thankful to the folks who actually do contribute code, documentation, and support, because they’re so few and far between compared to those who see open source as free consulting & development.
This is hilarious timing, but as I was writing this, my watch vibrated with a direct message in the SQL Server community Slack from someone who wants help. I responded asking them to try in the #FirstResponderKit channel as the readme says so that other folks can help too. (I checked back an hour later – they never did.)
I totally understand that a lot of the First Responder Kit users who need help are just as qualified as I was when I was trying to get Linux help. I wasn’t an amateur, and neither are these users. But it’s just an endless flood: there are always way, way more inexperienced users than there are experienced users. There just aren’t enough experienced users with free time to help, and many of the people who see themselves as experienced…aren’t.
It happens with Microsoft, too:
try Azure Data Studio.
It’s one of Microsoft’s newest open source products for database developers, a SQL Server Management Studio replacement for people who write T-SQL rather than manage servers. When you run into a problem with Azure Data Studio, you’re expected to file it in Azure Data Studio’s Github repo.
When you browse that list, it’s hard to determine if the software is buggy, if the users need better support, or if it’s a combination of both. Pro tip: it’s always a combination of both.
There are almost 1,000 open requests for enhancement, and many aren’t trivial requests: they’re monsters, like adding support for entirely new database platforms. Users feel free to ask for a pony because there’s no cost to them for making the request. Any bozo can ask for a pony by writing up a Github issue, and they can gather thousands of upvotes by posting it on social media or forums.
I don’t blame Microsoft for how this is going down. This is what happens when you build complex cross-platform software that runs on multiple operating systems and targets multiple databases. It’s the same story with Visual Studio Code’s issues, where there are also over 1,000 open bugs going back to 2016.
These aren’t open source problems.
They’re software problems overall.
It’s a problem with $1 phone apps, where users resort to leaving bad reviews and developers try to do tech support by responding to the review.
It’s a problem with Microsoft SQL Server, one of the most expensive database platforms in the world. You would think that when you hit a SQL Server problem and you think it might be a known bug issue, that it’d be easy to go to a web page, search for the symptoms, and see a known list of bugs and the status of whether they’ve been fixed or not.
For example, say you’re hitting issues with CMEMTHREAD waits. You go to support.microsoft.com, put CMEMTHREAD in the search box, and you get this wall of text:
There’s no sorting and filtering for database versions. You’re left to click on every single post and figure out whether it’s relevant to your patch level. There’s no commenting or questions, either. At the bottom of each post, there’s a “Need more help?” search box – which just takes you back to this search result list.
And these are only the published, fixed issues.
Not the known broken issues. Those are hidden away in secret for only Microsoft employees to access.
If you’ve been working with SQL Server for a while, you might know about feedback.azure.com, a place where you can post known bugs with SQL Server in the vain hopes that Microsoft will do something about it. In rare cases, they do – but in most cases, the requests go unaddressed. And just like its predecessor, Microsoft Connect, this feedback site is going away in 2021.
Closed source software only seems like it doesn’t suffer from the extensive bugs problem because the bug list is kept hidden by the manufacturer. We users thought the software was reliable, but if you talk to the really experienced pros in the industry, they’re often furious by just how broken common features are.
The way developers and users interact
is completely broken.
I’m hopeful that someone’s going to solve these problems during my lifetime.
Stack Overflow might be one way to solve it, but they’ve got a lot more work ahead of them. Right now, the newest incoming questions feed is an insane firehose, and the DBA.StackExchange.com new questions, while lower in volume, is still too challenging for the volunteers to solve alone. Maybe Stack could solve it by doing a better job of walking users through crafting good questions, and then recruiting (and paying) vendor employees to answer questions about their products.
Microsoft’s acquisition of Github might be another way to solve it because there’s no way they’re happy with the way this is going down, either. Microsoft could tune the new-issue process at Github and start handling paid support requests through Github issues. I’m sure I wouldn’t be the only open source maintainer who would gladly charge $10 for folks to open a support case just to cause ’em to think more before asking for a pony, and offer a higher-paid issue tier with a higher service level agreement.
Or maybe it’ll be an all-new Software-as-a-Service provider, but…I don’t think that’s going to be the case, given how Microsoft experimented with UserVoice and then pulled back to their own in-house products.
I don’t know what the right answer is, but the way we’re doing it now is completely busted. I should be thankful, though, because the fact that SQL Server support is so completely broken happens to enable a delightful career in fixing that very product’s shortcomings.