Blog

We had a comment recently on Five Things That Fix Bad SQL Server Performance that got me thinking. The comment came from a frustrated system administrator, who wrote:

Chucking resources at a problem is not solving that problem, it’s just temporarily masking the symptoms of that problem.

Funnily enough, I’ve heard the exact same thing from developers who are facing a tough problem on a dramatically undersized SQL Server. The basic gist of what they ask is:

Doesn’t it make me a bad developer if I can’t solve this problem with code?

Everybody’s worried that adding memory is somehow cheating.

Performance Problems and Memory

There are three general categories of performance problems relating to memory:

1. Problems you can solve with memory OR code / indexes
2. Problems you can solve only with code / indexes and NOT just memory
3. Problems you can solve only memory and NOT just code / indexes

For category #1, solving the problem with memory is often cheaper than changing code and indexes: the operational cost of the hours of development and testing can be quite high.

But I specifically started thinking about category #3. This is the category that the frustrated sysadmin and proud developer think doesn’t exist!

Here are three examples of problems that you can solve with memory, and not simply with code and indexes:

1. Memory Reduces IO On a Read-Hungry Workload

You’ve got a critical OLTP application. You’re using shared storage. Most of the time, your storage throughput is pretty good. But sometimes, other users of the storage get busy, and your read and write latencies go through the roof. Your queries slow down, blocking increases, and timeouts start hitting the application. You’ve tuned your indexes, but parts of the application rely on realtime reports that need to scan large tables.

SQL Server is designed so that in this case you can add more memory and reduce read IO to the storage, giving you more stable performance. It’s absolutely not cheating to give the server more memory — speeding up the storage would be much more expensive.

2. Some Queries Need Lots of Workspace Memory

In another database, you do lots of reporting and aggregation. You’ve got large tables. Your queries frequently join them, do sorts, and use parallelism. All of these operations need memory, and the more data you’re aggregating and querying, the more memory each of these queries can need.

Before a query starts running, it figures out how much of this workspace memory it needs, and looks at what is available given other queries that are running. SQL Server has to balance the memory used for data pages, execution plans, and this workspace memory: it doesn’t want to let one query take over! If not enough memory is available, your query has to wait.

You’ve optimized your queries and indexes and made sure the memory estimates are realistic, but when lots of people run reports, your queries can’t even get started because of a memory crunch. Solving this problem by adding more memory isn’t cheating: it’s helping the SQL Server do what it’s designed to do.

3. Some Features Need Lots of Memory Periodically

You’re managing a data warehouse where you’re using table partitioning to implement a sliding window. This has helped your users a lot: you can add and remove large amounts of data with minimal locking and blocking.

But for query performance reasons, you’ve had to add one or two non-aligned indexes that are present during the day. To do partition switching, you have to disable these indexes to do your sliding window data load and archive each night. Creating the partitioned index can consume large amounts of memory.

Using Enterprise features can require investing in Enterprise levels of memory.

Adding Memory Isn’t Cheating

Memory is an important tool for performance in SQL Server: it isn’t just something that covers up bad code, it’s often a solution to many different types of problems. Memory isn’t the only tool you need to help your queries go fast in SQL Server, but it’s certainly NOT a tool you should ignore.

Tune in here to watch our webcast video for this week! To join our weekly webcast for live Q&A, make sure to watch the video by 12:00 PM EST on Tuesday, September 16! Not only do we answer your questions, we also give away a prize at 12:25 PM EST – don’t miss it!

Curious how you can give a compelling technical presentation? Join Kendra to learn five important tips on how to select the right topic for your talk, write an effective abstract, construct a coherent presentation, and make it to the podium to give your first presentation.

Have questions? Feel free to leave a comment so we can discuss it on Tuesday!

http://youtu.be/g1P5UAgoIpU

Until now, SQL Server training usually meant registering for a course, showing up, hearing a lecture from a podium (often from someone who’s only done demos on their laptop, not production systems), working through a few workshops, and then going home with a bunch of ideas. It was like trying to ride a rollercoaster by jumping on as it came speeding toward the platform, having a great time, and then jumping off at high speed again. No graceful transitions, just a shocking experience.

We wanted to do better.

We wanted to ease students into the experience, get them more involved with the material ahead of time, connect the dots between the demos and their real environment, and empower them to help each other with their studies. We wanted to mix up the training so that it was less about theory, and more about their real world environment. Finally, as students left the classroom, we wanted to ease them back into the real world in a way that empowered them to succeed.

We couldn’t accomplish all of those goals immediately – doing this kind of practical, realistic, and comprehensive training is really hard work, so we’ve been gradually experimenting with more and more tactics this year. Here’s what we’ve come up with so far.

What It’s Like to Attend Our Training

We start mixing things up as soon as someone registers for one of our courses. For example, our How to Be a Senior DBA students were assigned a series of private pre-requisite training videos as soon as they registered for the course. They get started learning about RPO, RTO, and SQL Server’s options to meet their availability goals. (Behind the scenes, it took us years of work to get our e-commerce setup and training videos to this point, but the integration is definitely paying off now.)

On the first day of the in-person class, we started by discussing a couple of real-life case studies – then turned their own environment into a real case study. Each attendee used worksheets to describe their current production environment’s requirements to the person sitting next to them, and that person acted as a consultant. The team worked together to sketch out the right long-term solution for a company’s needs, and then they switched roles. Each student was able to sit on both sides of the issue – describing a problem, and building a solution for someone else.

In just one hour, this combination of pre-learning plus collaborative exercises accomplished three goals: it reinforced the student’s knowledge, helped them get a second opinion about their systems, and built a relationship with other people in the room. The relationship-building paid off throughout the course as the students carried on lively discussions at every break, over every lunch, and hanging around the flipcharts to talk shop. I wish I could say this was part of our giant master plan, but it was just a happy discovery that we made back in San Diego, and we’ve been using it ever since.

Another discovery this year was that students learned better when we assigned them script homework before the course. For example, before the SQL Server Performance Troubleshooting class, students were assigned scripts to run on their production servers and copy the results into Excel.

In Chicago, one of the students came up to me and said:

“I’m so glad you gave us the wait stats homework before class. I’d run the queries on several of my slow production servers, and I didn’t understand the results at the time, but now it all makes perfect sense. I understand why the data warehouse has PAGEIOLATCH and the web app has CXPACKET, and I know exactly what to do next.”

As we’d been covering wait stats, she’d been comparing her results with what we were discussing, and she was able to make a simple set of steps for her next maintenance window – all while sitting in class. Some of our students even used our lab scripts on their live production servers, fixing issues as we discussed them. (That might be taking the concept a little too far!)

Our Next Round of Training Experiments: Seattle

In our 2-day Seattle class, Make SQL Server Apps Faster, we’re running a couple more experiments.

We’re starting off with a day of lectures on performance tuning. We cover how to think like SQL Server, using wait stats to know why SQL Server is slow, tuning with the plan cache, beating anti-patterns, building the best indexes, and a challenge for attendees.

We weren’t content with that. People keep asking for more hands on sessions like “Watch Brent Tune Queries.” You really want to see our thought processes at work. We’ve done just that. Attendees think through the problem and sketch out their answers. Then they spend the day watching us solve it – they watch me doing query tuning, Kendra doing index tuning, and Jeremiah working through execution plans. At the end of the day, we judge their answers together.

We’re also offering a package deal – some attendees have said they want to watch recordings of the sessions later to reinforce what they’ve learned. (They already get the demo scripts, but sometimes it’s better to watch us doing it.) For an extra $200, the attendees get our 7-hour Developer’s Guide to SQL Server Performance videos that cover about half of what we’re teaching in-person, and we’ve extended similar deals to our 2014 training class attendees too.

For 2015, our new week-long courses represent another new experiment. This year’s attendees kept telling us, “As long as I’m out of the office, I might as well be out for the whole week.” Sounds good! So we’re taking more time to dive deeper into more topics.

We’re excited about how we’re doing SQL Server training differently than anybody else on the market, and we can’t wait to share it with you.

You love Q&A sites like StackOverflow.com and DBA.StackExchange.com, but sometimes it’s hard to find interesting questions that need to be answered. So many people just sit around hitting refresh, knocking out the new incoming questions as soon as they come in. What’s a database person to do?

Use the power of the SQL.

Data.StackExchange.com lets you run real T-SQL queries against a recently restored copy of the StackExchange databases. Here’s my super-secret 3-step process to find questions that I have a shot at answering.

Step 1. Find out how old the restored database is.

Run this query to get a list of site databases and the newest post date in each one:

I take a glance at StackExchange.Dba and StackOverflow to make sure I’m dealing with a relatively recent database restore. These database restores are done weekly, but you know how it goes – sometimes scheduled jobs fail.

Step 2. Find questions everybody’s talking about.

Run this query to find questions with many comments, but no accepted answer. Note that for this one, you do have to pick the site you want to focus on – this doesn’t run across all databases.

If you click on the Post Link, you’re taken to the live question. If the database restore was a few days ago, keep in mind that the live question may have changed or been answered by now.

The other columns help me see at a glance if this is a question I’m interested in. In the above screenshot, the third question, “New database server hardware,” has 2 answers, but none of them have been upvoted, and there hasn’t been any comments since 2014-05-13. It’s old, and the questioner has probably moved on, but you can revive the question by posting a really good answer. You can also take your time, knowing nobody else is really active on it right now. These are great opportunities to post a really in-depth answer for other folks to find as they search the web later.

Which brings me to my next favorite query:

Step 3. Find questions that people keep looking at.

Run this query to find questions with many views, but no accepted answer. This one is filtered by a specific tag because the SQL Server ones don’t usually show up in the top views. Use tags from the DBA.se tag list or the StackOverflow tag list.

Obscure error numbers are usually going to pop up here because there’s not much public information about ’em. Now’s your chance to write an answer explaining how to troubleshoot that error number, and presto, you’re the next Jon Skeet.

We always like to innovate — not just with the solutions we design for our consulting customers and in how we teach, but in our free videos, too.

Our YouTube channel has become super popular. Lots of folks watch the recordings of our live webcasts. We stopped recently and asked, “How can we make this even better for the folks who attend our live event?” And we realized: we can give you more time to ask questions about that week’s training topic!

Here’s your mission:

1. Watch the video below today. We won’t be presenting this live this week or re-covering the material from the video, we’re doing more advanced QA for the folks who’ve already watched it.
2. Note down questions or comments you have on this post. (This is totally optional, but it means you won’t forget your question and it’s more likely we have time to talk about it with you.)
3. Attend the live webcast on Tuesday at the normal time (11:30 am Central). Register here.
4. During the first 10 minutes of the webcast, we’ll give away a prize– but you must be present to win!

https://www.youtube.com/watch?v=kYYYymJ0tcc

The live discussion of the video and Q&A won’t be recorded and published, and you also need to be present to win the prize. See you on Tuesday!

The only thing you ever need to use for database identity is an IDENTITY, right? Well, maybe. There are a lot of different options and they all have different pros and cons.

IDENTITY columns

The default way to identify objects in SQL Server is to use an INT or BIGINT column marked as an IDENTITY. This guarantees relatively sequential numbers, barring restarts and failed inserts. Using identity columns put the responsibility for creating and maintaining object identity in the database.

SQL Server will cache IDENTITY values and generate a new batch of identity values whenever it runs out. Because identity values are cached in memory, using identity values can lead to jumps in the sequence after SQL Server is restarted. Since identities are cached in memory in large batches, they make it possible to rapidly insert data – as long as disks are fast enough.

Sequences

Sometimes the application needs more control over identity. SQL Server 2012 added sequences. A sequence, unlike an identity value, is a separate object in the database. Both application and database code can read from the sequence – multiple tables can share a sequence for an identity column or separate sequences can be created for each table.

Developers using a sequence can use the CACHE value to cache a specific number of sequence values in memory. Or, if the application should have minimal gaps in the sequence, the NOCACHE clause should be used.

The Problem with Sequential Identities

Both IDENTITY and SEQUENCE values keep identity generation squarely in the database and, by using integral values, they keep the value narrow.

You can run into problems with sequential inserts on very busy systems – this can lead to latch contention on the trailing pages of the clustered index. This issue can be resolve by spreading inserts across the table by using a GUID or some other semi-random clustering key. Admittedly, most systems are never going to run into this problem.

GUIDs for Object Identity

Some developers use GUIDs as a way of managing object identity. Although database administrators balk at this, there are good reasons to use GUIDs for object identity.

GUIDs let the application generate object identity. By moving object identity out to the application layer, users can do work in memory and avoid multiple round trips to the database until they’re ready to save the entire set of data. This technique gives tremendous flexibility to application developers and users.

There’s one other thing that a well designed application gets from this technique – independence from the database. An application that generates its own identity values doesn’t need the database to be online 24/7; as long as some other system is available to accept writes in lie of the database, the application still function.

Using GUIDs for object identity does have some issues. For starters, GUIDs are much wider than other integral data types – 16 bytes vs 4 bytes (INT) or 8 bytes (BIGINT). This is a non-issue for a single row or even for a small database, but at significant scale this can add a lot of data to the database. The other issue is that many techniques for generating sequential GUIDs in the application (see NHibernate’s GuidCombGenerator) can still run into GUID collisions.

Integral Generators

What if you could get the best of both worlds? Applications generating unique identities that are also sequential?

The point of identity generation is to abstract away some portion identity from data attributes and provide an independent surrogate value. GUIDs can provide this, but they aren’t the perfect solution. Identity generators like flake or rustflakes promise roughly sequential identity values that are generated in the application layer and are unique across multiple processes or servers.

The problem with an external identity generator is that it is an extra piece of code that developers need to manage. External dependencies carry some risk, but these are relatively safe items that require very little effort implement and maintain.

The Solution

There’s no right solution, there’s only a solution that works for you. You may even use each solution at different points in the lifecycle of the same product. It’s important, though, for developers and DBAs to be aware of how identity is currently being handled, the issues that can arise from the current solution, and ideas of how to handle it going forward.