One of the cool things about working with an established product like SQL Server is that there’s a ton of knowledge out there. The intertubes are chock full of advice about how to manage all kinds of problems.
Unfortunately, some of them are flat out wrong.
They may not have started out wrong – they might have been right when they were initially written, but over time, we’ve figured out better ways to work. When I started out working with SQL Server, everybody watched the Perfmon counter for Average Disk Queue Length in order to determine whether their storage subsystems were overloaded. These days, we know that’s not a good idea, but somebody who’s just getting started with SQL Server might stumble across a lot of that advice on sites that aren’t kept up to date. I know Brad (the author who originally wrote that article) and he was completely right at the time, but that web site hasn’t been updated by its current owners. Some pages never should have been published to begin with – on this Perfmon page, the author says if a server’s available MB of memory is <20-25% of its installed total, it doesn’t have enough memory. That is simply not true, period.
I run across this situation in consulting all the time – someone’s banging their head against the wall because they’re following what they think is solid advice from a good source, and they’re not getting the results they want. They truly want to do the right thing, and they’ve done at least a little research on the web to find the right solution. It just so happens that they found an incorrect answer, but it’s not their fault for trying. This week in my Consulting Lines series, I’m covering how to get past those misconceptions.
The Conversation
Me: “So I understand you’re having problems taking down the alien mother ship.”
David: “Yeah, I’ve written a virus on my Mac, and – ”
Me: “Wait, what? I thought Macs couldn’t get viruses?”
David: “I know, that’s what makes it so genius! They’ll never see it coming! So anyway, I’m trying to plug my Mac into the alien mainframe, and it’s not working. I can’t find a USB port.”
Me: “A what?”
David: “A USB port – you know, Universal Serial Bus. Alien ships are still from this universe, so they should have a USB port, right? That’s the best way to get files from one computer to another.”
Me: “Yes, that used to be a best practice, and it worked for the longest time. However, drives have gotten bigger and we’ve wanted to transfer more data, we’ve had to change our tactics. Here’s the latest trick we’re using to work around it – we’re using the cloud.”
What That Line Does
It’s almost easier to explain that line in terms of what it DOESN’T do. It doesn’t dispute the speaker’s claim, because you don’t want to go down the rathole of arguing about whether or not the point was right at the time. You could spend the entire day arguing about why USB has the word Universal as part of the name, but that’s beside the point. You need to get to the right solution as fast as possible, and sometimes that just means ignoring something that’s wrong.
It doesn’t chastise the other guy for not keeping up with the latest tips and tricks. It’s hard enough to just do our jobs, let alone keep up with blogs and training, and real life interferes. It lets the other person save face.
It doesn’t divide you against the other guy. I try to use the word “we” whenever I use this line, because I want to take the other guy along with me on the journey to fix this problem.
What Happens Next
This is one of my favorite lines because things almost always go the easy way. It lets everybody in the room continue to believe that they were doing the right thing, and that it’s not their fault things weren’t working.
If the other guy wants to pursue the original point, agree that he’s right. Agree vehemently that he’ll find lots of supporting evidence on the web from very credible people. The key is to keep saying yes, and explaining why the new technique is even better – not that the old way is bad, but the new way is better.
10 Comments. Leave new
Now that’s attention to detail! (David was the name of Jeff Goldblum’s character in Independence Day)
Thanks, sir. I had to look it up to remember it though, hahaha…
Nice! It had occurred to me yesterday that I should write a blog on the lost art of allowing others to save face. This is especially important as a consultant.
Overall great post. I wanted to point out that your characterization of http://blogs.technet.com/b/vipulshah/archive/2006/11/30/understanding-perfmon-counters-while-troubleshooting-sql-server-performance-issues.aspx was a bit off I think. It actually says: A consistent value of less than 20 to 25 percent of installed RAM is an indication of insufficient memory. My take on this is seeing a consistent value at this level provides sufficient reason to look at memory deeper.
Will – thanks, but that’s not what the author says. He clearly says “an indication of insufficient memory.” A new DBA will read that, look at the server’s memory, and say, “We need more memory.” They buy more, install it, and wonder why they’re right back up to 98% memory usage the next day. The author doesn’t even MENTION how max server memory is configured. That’s shameful.
It is EXACTLY what he says because I copy/pasted directly from his page.The problem is you need a discerning reader to understand that the phrase “… an indication…” implies there is probably more study needed. If a new DBA reads that and convinces management that the server needs more memory then they deserve what they get because they misinterpreted what the advice suggests which is that you’ve found a potential issue and you need to look deeper. The author also mentioned that this level should be CONSISTENTLY observed AND points out that the stat is point-in-time. Sorry but this doesn’t look like bad advice to me…..
Sure a more thorough treatment of the topic should mention memory config details but is not mentioning it really “shameful”?
Will – we’re just going to have to agree to disagree there. I simply don’t believe that someone “deserve what they get” if they read that page and come to the same conclusion I did.
I think the message the author was trying to convey is pretty clear and doesn’t really suggest the point you’re trying to motivate in your blog post. I also think there are plenty of examples on the Internet of exactly what your post was about, but in this case you happened to pick a bad example.
I love this series of posts! There are a lot of sources of technical information out there, but it’s very hard to find practical, real world information on soft skills. Technical skills will only take you so far. Whether you’re a consultant or you’ve got a full time gig somewhere, it’s the soft skills that will decide your career ceiling. Kudos on a great series of posts, and keep ’em coming!
I worked as a consultant for many years, and saw a lot of code that was not the best.
To begin with I would get annoyed or feel my predecessor was an idiot.
As time went on, some of the lousy code I stumbled over was my own. Perhaps 5 years old, but now I was less ignorant.
Sometimes the code was done as as a poc in 5 minutes, but life happened and suddenly it was part of business critical systems.
So when I got young developers under my wing, one of my lectures to them would be to just accept that some times good people make crap code.
Either fix it, leave it in, or explain to the customer that the code base needs to be redesigned.