You're looking at a wide table with 100-200 columns.
Years ago, it started as a "normal" table with maybe 10-20, but over the years, people kept gradually adding one column after another. Now, this behemoth is causing you problems because:
Folks sometimes ask me, "When a table has more indexes, and SQL Server has more decisions to make, does that slow down execution plan generation?"
Well, maybe, but the table design choice that really screws you on compilation time is partitioning. If you choose to partition your tables, even tiny simple queries can cause dramatically higher CPU times. Even worse, as the famous philosopher once said, "Mo partitions, mo problems."
How's that for a niche topic? You're probably never going to know this, but since I had to figure it out the hard way, I'm writing this down so I don't forget it: queries that get trivial optimization may not get partition elimination. This post is going to be a little on the long side…
In the last episode, I talked about knowing when it's time to scale out: identifying when our data is getting to be so large that we have to split it across multiple servers, and I explained why that is so challenging. But what about table partitioning - SQL Server's ability to break up a single table into smaller ones on the same server?
It's not that I don't like partitioning
It's just that most of my time talking about it is convincing people not to use it.
They always wanna use it for the wrong reasons, and I can sort of understand why.
A really odd choice
I've talked a bit about my annoyance with hardware limits for Standard Edition here and here.
If you're a bunch of SQL-savvy developers who understand the Enterprise Edition features that are now available in 2016 SP1, great. You're a very small minority of shops. I'm talking about places like StackOverflow and kCura.
This is not about table partitioning If you want to learn about that, there's a whole great list of links here, and the Best Blogger Alive has a tremendous post on why table partitioning won't make your queries any faster over here. With that out of the way, let's talk about partitioned views, and then…
This post is mostly a thought experiment The thought was something along the lines of: I have a table I want to keep temporal history of. I don't plan on keeping a lot of data in that table, but the history table can accumulate quite a bit of data. I want to partition the history…
It's tempting to think that table partitioning will improve query performance. After all, it's an Enterprise Edition feature-- it must have a lot of magic, right? Table partitioning does have magic for the right situations. It shines when you want to add a large amount of data to a table or remove a large amount…
It was a dark and stormy… Oh, wrong story. It was actually a warm, sunny afternoon in Charlotte, NC. I was presenting “Index Methods You’re Not Using” at PASS Summit. In this talk, I discussed how indexed views, filtered indexes, and compressed indexes can improve your query performance by reducing I/O.
From stage right, an intrepid audience member raised his hand and asked, “Can you think of an example of when you would use filtered indexes instead of partitioning?”
One of the many best practices for SQL Server's table partitioning feature is to create "extra" empty partitions around your data. This is explained in SQL Server Books Online in the page on altering a Partition Function: Always keep empty partitions at both ends of the partition range to guarantee that the partition split (before…
There's a secret to learning about SQL Server's table partitioning feature: you need to get your hands on some code and really play with it in a test environment.
In this tutorial, Kendra will walk you through test scripts that set up a sample table partitioning environment. These scripts demonstrate gotchas to look out for and the cool features that can dramatically change your database's performance.
Getting started with table partitioning is difficult; there are many decisions to make after you've decided to partition data. Correctly configuring partitioning is critical to the long term performance and stability of the database from both a querying and data modification standpoint. There are three key considerations that database architects should devote time to before embarking on a partitioning implementation: the design of the partitioning key, a thorough study of querying practices, and an examination of data modification practices.
At first glance, SQL Server’s partitioning seems like it should be an easy way to solve problems inserting data into busy tables. If the writes are spread across many partitions it only makes sense that we can avoid write hot spots in SQL Server, right? The truth is much more complicated than it appears. Asking…
Great volumes have been written about table partitioning. It’s a complex feature and you can read for days to just understand how you might apply it. But will it improve performance for you? Table partitioning produces great benefits for some applications, but causes giant headaches for others.
How do you know if you should invest your time in table partitioning?
When SQL 2005 was in beta, I clapped my hands and jumped for joy the first time I played with partitioning. I've done a lot of data warehouse work, and partitioning lets the DBA break down massive tables into smaller, easy-to-manage chunks. We can put heavily accessed parts of the table onto fast storage, and…
