This week, Erik, Richie, Doug, and Angie discuss a script for getting more information on a specific index, using partition tables, shrinking data and log files, stress testing SQL servers, as well as some entertaining things to ask Siri!
Office Hours Webcast – 2016-06-29
Angie Rudduck: All right, it’s 15 after the hour. We’ll read some questions, maybe. Tammy has got a question that I think Doug had referenced once. “Has anyone installed R for SQL 2016 yet? Have you had any problems? Do you have any idea how to uninstall?” It didn’t install correctly but she thinks it installed when she tried to reinstall.
Doug Lane: Oh, boy. No, I haven’t but I can tell you if you’re planning on going to the PASS Summit this year, there are like five intro to R sessions. It’s bonkers.
Erik Darling: None of them are on installing R.
Angie Rudduck: I was just about to say that. Good luck installing it.
Erik Darling: You’d really think that R would get more pirate jokes out of it.
Richie Rump: [Mimicking a pirate] Arrr!
Angie Rudduck: There is one pirate…
Doug Lane: Yeah, there’s one session that makes a pirate joke in the title.
Erik Darling: Yeah, you’d think they’d all do it because it’s like the most obvious thing. Whatever.
Richie Rump: Next.
Angie Rudduck: Next.
What script do I use to get more information about an index?
Angie Rudduck: All right. James wants to know what script can he use to get more information on a specific index.
Erik Darling: Perhaps you’ve heard of sp_Blitzindex by chance.
Richie Rump: No, I haven’t.
Erik Darling: When you run it, there’s literally a column called “more info” with a script to get more info about an index. Read columns to the right. Your job from now on: read columns to the right. Scroll, my friend.
Doug Lane: There’s also Mode 2 which will tell you a lot of particulars about the indexes on a given table if you want to specify a table. That will give you about as much information as you could possibly need to make whatever decisions you’re trying to make about that index.
What do logical reads mean in STATISTICS IO output?
Angie Rudduck: Rizwan says, “Hi guys, I have a question about SET STATISTICS IO. What does logical reads mean? Does it mean—does data being read from memory or is it being copied to memory?”
Erik Darling: Logical reads is being read in memory. If you see the physical reads when you set STATISTICS IO there are a whole bunch of reads, so you’ll see like physical and read ahead and lob and all the other ones. Physical reads are the ones that you’re reading from disk. Logical reads are the one where you’re reading from memory.
Richie Rump: But won’t it count it for both? Because isn’t the physical reads from disk you would read that and then it would have to read it from memory again?
Erik Darling: You know what? I actually don’t do enough looking at that to tell you. If you know from your lovely website Statistics Parser, then share it with the class.
Richie Rump: I don’t know, that’s what I’ve noticed, right? So when you’re reading from disk, you’ll get the same amount or less, or more from the logical side.
Erik Darling: Right, but if physical reads is zero, then you’re just reading stuff in memory.
Richie Rump: Memory, yeah. Correct.
When should I start using partitioned tables?
Angie Rudduck: Steve wants to know if there’s a rule to start using partitioned tables. Are you concerned with size or performance first?
Erik Darling: Partitioning is not a performance enhancement. Don’t drink that Kool-Aid my friend. Partitioning is a manageability enhancement so that you can swap your data in and out without locking the whole table and doing anything crazy. It also allows you to archive stuff back. You can make stuff read only, put stuff on different filegroups. It allows you a lot of great management things, not really a lot of great performance things. Kendra, over on her new home, littlekendra.com, has a great roundup of why partitioning is not making your queries faster. If you want to go over there, there’s a video and links to several other things where she goes into detail about why partitioning is not a performance enhancement.
Doug Lane: If I remember right, there’s a rule of thumb. Was it like 100 million rows or something like that?
Erik Darling: See the thing is, I wouldn’t want to start partitioning at a table that big. I would want to like—like if I know that I’m going to have a table like an audit table or something that could potentially get big, I would want to do it at the beginning. Because when you start partitioning at a large table size, it’s just obviously some overhead to get everything lined up.
Doug Lane: Yeah, or anything where you know you’re not going to be deleting from the table, you’re just going to be adding as time goes on and you’re going to be adding a lot of rows. That’s a time where you might want to think about partitioning by say year.
Erik Darling: For sure.
Richie Rump: So what about partition elimination?
Erik Darling: What about it?
Richie Rump: Is that a performance enhancement?
Erik Darling: When it happens. Good luck getting it to happen.
Richie Rump: Use your partition key I guess.
Erik Darling: Yeah, all your misaligned, non-clustered indexes. Watch what happens.
Is there a safe way to shrink files?
Angie Rudduck: Let’s reward Richard for asking this question early. “Shrinking data files is not recommended because it disorganizes the indexes. Is there less disorganization if you use EMPTYFILE to move the data in a multi-file filegroup? If the table has varbinary (max) blobs, does that add any other wrinkles to a file shrink/move besides the size?” Why are you shrinking, Richard? Do you have to shrink?
Erik Darling: I get EMPTYFILE, you’re moving stuff around. I’m not sure that that’s the way I’d choose to move things around though. I’d need to know a little bit more about what you’re trying to do and just why you’re trying to do it. Because if it’s just like a one-time data move, then there’s some options. As far as the fragmentation and stuff goes though, I’m actually not sure. I would assume it uses a fairly similar mechanism to EMPTYFILE—or to SHRINKFILE. The only difference is that you’re not actually shrinking anything, right? Like you’re kind of just pushing data out of the one file so I’m not sure that it is as aggressive as just sort of mashing pages together as an actual file shrink would be.
Doug Lane: One of the reasons that we tend to as a group not recommend shrinking data files is because it grew that big for a reason and if you’re going to shrink it, what we’re afraid of when we say don’t do it is that you’re going to get in a cycle where you keep growing and shrinking it. That’s what we want to avoid. But if it got really big for some reason and you know that’s not going to happen on a recurring basis, then we’re actually okay with shrinking data files.
Erik Darling: Even log files.
Angie Rudduck: I did just tell my clients yesterday that I had faith in them shrinking one giant large file, large log file. Actually, I used one of the default ones where it says shrink it to one meg and then grow 25 percent of the database size. Their log file was 30 gigs. Their data file was six.
Erik Darling: That’s an interesting choice.
Doug Lane: Yeah and the log file was almost completely empty.
Angie Rudduck: Yeah.
How can I get started using CLR in SQL Server?
Angie Rudduck: Nate has a question about using CLR in SQL. Do you have a quick how-to or if it’s too much to cover on air, a blog post or article to point him to?
Erik Darling: Richie.
Richie Rump: I think Adam Machanic has some stuff up there as well as there’s a video from Tech Ed a few years back that he kind of goes into some of that stuff. I may—that’s actually a good post to do—I know I’ve got a demo on my site that kind of goes into it a little bit. I have to remember what presentation that was a part of. But it’s actually not too bad. It’s not as hard as you think it is. Testing it is a little bit of a pain but once you get the hang of it, it’s not bad at all.
Erik Darling: Adam Machanic does have a cool blog post where he was writing a CLR string splitter to try and beat out some SQL, like [inaudible 00:07:39] SQL implementations of it. There’s a link there to a pretty cool thread on SQL Server Central where it goes through all the feedback and like figuring out how to do stuff. I’ve done it once and like [DBA compiling 00:07:52] DLL was like me just mashing knuckles on the keyboard. It’s horrible.
Richie Rump: Similar to me, by the way.
Erik Darling: There we go.
How should I stress test a SQL Server that keeps failing?
Angie Rudduck: Dennis wants to know what the best way is to stress test a large SQL Server with 60 cores, a terabyte of RAM, SAN, SSD, xyz, etc. “Our IT claims the server randomly reboots due to heat but has no other information available.” He wants to try and recreate the problem.
Erik Darling: You want to try to overheat that thing?
Doug Lane: I hope this isn’t a production server, although based on the 60 cores I would hope that that’s not sitting around in development either. So for something like this, there are a few tools you can use. There’s OSTRESS and there’s also one called HammerDB but HammerDB is a little bit harder to configure. It’s a little older too. So those would be two stress testing tools. There’s also just for the pure IO side of things, there’s SQL IO which probably don’t want to run on a production box just because it can hammer the SAN to the point where it shuts down. So I would probably start with OSTRESS if you’re looking for a tool that will simulate a heavy production load.
Erik Darling: With a server like that, it sounds like you have some money. So, if you want to do something a little bit fancier, Dell has—well at least for now, until software gets sold off to whoever—has a tool called Benchmark Factory.
Doug Lane: Oh, yes.
Erik Darling: Really throw some heat at your server if you’re interested in spending money on it.
How do I move replication publishers around?
Angie Rudduck: We’ll see if we can answer this question without Tara. From Andrew, “We’re on 2008 R2 SP3.” They’re doing log shipping to another 2008 R2 SP3 server. The log shipping server is also the replication publisher. Is there any way to change the primary to the secondary permanently and have replication follow or do you need to redo replication on the new primary?
Erik Darling: Nope, I don’t know.
Angie Rudduck: Test it in dev and find out. Sorry, Tara is not here. Replication question denied.
Doug Lane: I’m still trying to make sense of the question. Change the primary to secondary, so change the log shipper to the log shippee? Is that what you’re saying?
Angie Rudduck: I think so.
Doug Lane: Yeah, at some point, you’d have to have a cutover where you know you freeze what happens on the primary. Then you’d have to kind of burn all the log shipping down, set it up again. Yeah, replication and log shipping are two completely different features so I don’t think there’s anything you can do with log shipping to make replication magically flip sides. I think you’re going to have to stop everything, tear down what you’ve got, and then reconfigure it. Both for log shipping and for replication.
Angie Rudduck: I think you can repoint log shipping, but replication is scary. So without Tara, test on your own.
Doug Lane: I’d be a little nervous to repoint log shipping exactly in the other direction without knowing that everything was stopped.
Angie Rudduck: Yeah.
How can I find out if I have page contention?
Angie Rudduck: Gordon wants to know if there’s a DMV that will give him information on which page types other than PFS, GAM, and SGAM are causing contention.
Erik Darling: If you’re running sp_WhoIsActive, it will tell you which pages are in trouble if you’re in fact waiting on pages. So if you’re running stuff and you think there’s contention on a particular page, you can run that and it will tell you where it’s happening if it is page contention. So like it will tell you for tempdb it will show you like page latch and it will give you some more information about the pages that it’s hitting. I would go right to sp_WhoIsActive and try to run that and see what’s going on rather than trying to roll your own query. If you want to look at what sp_WhoIsActive is hitting to give you that information, you can do that and you can try to maybe reverse engineer a small part of it to get some page type waits.
How can I copy SSRS permissions from one server to another?
Angie Rudduck: Tommy wants to know do we have a script or process that can be used to copy permissions, users, etc. for SSRS to a new SSRS server? Doug.
Erik Darling: Doug Lane.
Doug Lane: We don’t. There’s an older script that is—yeah, it’s a script—that is sort of retired but it’s still out there if you can find it, RS Scripter. That’s a tool that will allow you to basically just take an instance of a reporting server database and move it somewhere else. The other thing that you can do is reporting services, at least before 2016, and 2016 may not have changed, but it was conveniently compartmentalized to everything being stored in the database. So there are a few configurations settings in like XML files, .config files, but the stuff like users is all stored in just straight up database tables. So it’s the kind of thing where if it’s a production instance you want to be careful, try this out in dev first where you basically take the existing reporting services database, that would be the report server database probably. Then take the settings out of there from like the users and security tables and move them over to the new instance, somewhere in dev, turn it on. Make sure everything is the way it’s supposed to be. But one thing you might run into is when you do that if you try to go to the site on the new server you may get like a permission denied, access is denied error. When that happens you just have to open Internet Explorer, run it as administrator, and go in and assign your user or group permissions to see the home folder. It’s a common problem, even when you aren’t moving databases around. So just be aware of that too.
Angie Rudduck: I’ve run into that a few times before. It was like, “Why does this work only in ‘run as admin?’”
Doug Lane: Yeah.
Angie Rudduck: Speaking to Doug, Tim wants to know when your new video is going to be released.
Doug Lane: Oh, fairly soon. I would expect before the end of July. I don’t have a date but it’s coming pretty soon.
Angie Rudduck: Yay.
Erik Darling: They look awesome though. So get those credit cards ready, or PayPal, I don’t know, your mom’s checkbook.
Richie Rump: Don’t tell your wife how much you really paid for it.
Should I use different drive letters for data, logs, and TempDB?
Angie Rudduck: Mike wants to know with SAN drives if it’s important to keep data files and log files and tempdb all on different drives if using a SAN, non SSD.
Erik Darling: Not really. I mean it’s all working off the same sort of spread out shared pool of stuff anyway so you don’t have to sweat it as much. That was a good rule of thumb back when you had direct attached storage. The only time I’d really break from that is if you’re in a non VM environment and you do have sort of a higher grade of drives for certain things. Then I would of course put tempdb on SSD or flash but if it’s all the same pool of spinning disks, then it doesn’t really matter so much.
Doug Lane: Yeah. You just don’t want tempdb to get promoted and demoted all the time. You want it to stay in the top tier if you have tiering.
How should I handle recursive relationships?
Angie Rudduck: Okay, Wes wants to know if there is hierarchyid data type is better to use than a string in a parent field and a child field. Erik pointed him to Adam Machanic’s blog about “Re-Inventing the Recursive CTE.”
Erik Darling: Yeah, buddy.
How can I copy data quickly between servers?
Angie Rudduck: Now on to Mike who wants to know if anybody knows of a “fast” copy tool. Some of his users continuously want db refreshes from prod to QA and sometimes they are several hundred gigabytes in size and take too long to copy.
Doug Lane: Replication.
Angie Rudduck: Ugh, log ship maybe. With standby mode, right?
Erik Darling: You could try that. You could take a database snapshot. If you’re on a SAN, you could take a SAN snapshot and just pop that on over pretty quick. Aside from that, you’re looking at setting up some, like Angie said, probably like log shipping so that you could have an existing copy and just log ship over to it and catch things up when developers need a refresh.
Richie Rump: Right, there’s also some development methodologies saying that you maybe don’t ever want production in outside of production. So essentially you’re testing QA environments have their own test data that’s [inaudible 00:16:22] and it’s not production data. So that’s something to consider as well. It’s a security risk as well because your QA environments aren’t usually as locked down as your production environments. So if you’ve got sensitive data there, you may want to rethink that.
Erik Darling: One really cool thing you can do is if you don’t want to go through like the hard process of doing this stuff is you can actually just script out all the statistics for your database. So if you script out all the statistics you get a copy of your data that mimics how the data reacts to queries and whatnot without actually having any data having to exist on the disk itself. I believe a fella named Thomas LaRock has a post on how to script out your database statistics, with pretty pictures and screen caps and whatnot so you can head on over there.
Angie Rudduck: Richie, FYI, someone couldn’t hear you so you’re going to have to get closer to that mic.
Richie Rump: Someone couldn’t hear me? [Makes noises close to his mic]
Angie Rudduck: Much better.
Erik Darling: Get inside the mic.
Why is CLR enabled on my SQL Server?
Angie Rudduck: Richie, Kyle wants to know, they have CLR enabled on their server but he doesn’t know why. Is there an easy way to find out if anything is using it?
Richie Rump: Not that I know of but that sounds really interesting and I already made a note of that because I saw the question earlier about investigate and how we would do that.
Doug Lane: Turn it off and listen for the screams.[Laughter]
Erik Darling: One thing you can do is some of the DMVs that record query execution stuff record CLR time. So you can just check those out and see if there’s any CLR time being recorded on your server. I believe there’s also if you look at specifically at functions I think there’s a flag in one of those DMVs that tells you if it’s a CLR function so it could be like in CLR or something. I don’t have a more specific answer because I’ve never really gone searching for it.
Angie Rudduck: He said he’s considering the try pull the plug method.
Erik Darling: You could do that.
Can I automate adding new databases in my Availability Group?
Angie Rudduck: I don’t know if this question is direct enough but Paige wants to know if there’s a way to automate AlwaysOn Availability Groups to all replicas when a new database is created on the primary.
Erik Darling: In 2016 there is. You can use a feature brand new to Availability Groups called direct seeding. That’s S-E-E-D-I-N-G not S-E-A-T-I-N-G. So that’s pretty cool. Before that, no. There are a bunch of questions on dba.stackexchange.com that have sort of like the beginnings of scripts but the problem is that it’s really hard to just write one that works for everyone because you do have to add the database, take a backup, restore it, or something like that to make it all work. So it’s really hard to just write a one-off script that works for everyone. But you can get some examples of beginning scripts if you search on dba.stackexchange.com for Availability Group automating or some form thereof.
Seriously, I want to shrink a file.
Angie Rudduck: All right, do you guys want to answer a question about shrinking or a mysterious missing file?
Erik Darling: Yes.
Doug Lane: Sure.
Angie Rudduck: So Monica has a database which was sized for future growth at 800 gigs but they’re only using half that. However, other databases have now grown and their data file disk is running out of space. Until they add more disk space, what would be the best way for her to shrink the database file to avoid any downtime?
Erik Darling: You could just shrink it and specify the size to like 50 or 100 gigs larger than what you have now. Does truncate only work on data files?
Angie Rudduck: I don’t know.
Erik Darling: Because if truncate only works on data files, you could throw the truncate only flag on there and it would just lop off the free space.
Angie Rudduck: Would you use the GUI or would you script it out?
Erik Darling: I would script it out because I’m weird like that.
Angie Rudduck: Paranoid.
Erik Darling: Because I spent a lot of money on…
Doug Lane: Truncate used to only work on logs but I think they got rid of that.
Erik Darling: No, truncate only works as a command on logs, it doesn’t work as a backup command on log shipping.
Doug Lane: Oh, okay.
Erik Darling: Yeah, so that doesn’t work. But it still works as a DBCC SHRINKFILE command. I’ve never tried it on a data file, so I don’t know.
Doug Lane: Back when I was a web developer, I used to run into log file problems all the time and I’d just backup log with truncate only, whee! Good-bye. That’s how I solved every problem I had.
Why are my permissions messed up after doing a Windows update?
Angie Rudduck: Let’s try to tackle Mike’s mystery. He’s on SQL 2012 with a two-node cluster. They have log shipping to a reporting server. They did a Windows update reboot and the shared folder on the cluster where log shipping copies from became invisible to the reporting server. They couldn’t re-add it or share it, they had to create a new share network name to get it going again. Any idea what may have caused that?
Doug Lane: Nope.
Angie Rudduck: Yeah, me neither. I was going to say, “Talk to your sysadmin.”
Erik Darling: Permissions maybe? Are you sure it just wasn’t a permissions issue? Like you could temporarily try flipping the security thing to everyone and see if they can see it then. If not then … we’re getting a Lifestyles of the Rich and Famous tour of Doug’s room.
Why is Doug broadcasting from an undisclosed location?
Doug Lane: Clinton tells me, “To buy a poster, man” because I’m behind a giant blank wall here. Yeah, it is kind of blank but…
Richie Rump: Buy more posters, Doug.
Doug Lane: I do have other stuff in the room, I’m not being interrogated.
Angie Rudduck: Except for by you, Clinton.
Richie Rump: Get an X-wing behind you, Doug. Come on, man.
Angie Rudduck: Richie, what’s the shuttle behind you? One of the shuttles, maybe.
Richie Rump: That is a Lego shuttle that they came out with about five years ago along with Slave 1, the TIE fighter, and an X-wing. There you go.
Angie Rudduck: What Star Wars shuttle space thing is that, Doug?[Doug’s spacecraft model makes noise]
Angie Rudduck: Oh, it has sound.
Doug Lane: Your move, Rump.[Laughter]
Richie Rump: I could break mine into 1,000 pieces, but I’m not going to, because I don’t want to put it back together.
Erik Darling: Don’t you have a model of the Challenger back there?
Angie Rudduck: Yeah, what is back there?
Richie Rump: Too soon, man. Too soon. Too soon.
Angie Rudduck: Didn’t that happen before I was born?
Erik Darling: Probably, I was in kindergarten.
Doug Lane: Oh, ouch.
Richie Rump: Stop it.
Doug Lane: Ouch.
Is SQL Server 2016 ready for production use?
Angie Rudduck: Speaking of new things. Richard says, “Microsoft seems to be touting that SQL Server 2016 is production ready now” and that shops are using it in production since prerelease. “Given its many performance and other features, do you feel it’s solid enough to deploy in production now or would you wait for a particular number of CUs or service packs?”
Erik Darling: I would just roll that sucker out. No, so it is, it is pretty much production ready. It is in RTM. You’re going to have your problems but the things that I would want to do is start running everything on it in production immediately because Microsoft changed the way they’re supporting things. So like you’re not going to get a service pack, there’s no guarantee of a service pack. You’re going to get cumulative updates every so often when they feel like it. So who knows how long you’d be waiting. If you want to upgrade, the only thing I’d be really careful of is the new cardinality estimator stuff. Which I think Doug looks like he wants to talk about.
Doug Lane: That hasn’t really changed since 2014. So if you’re on 2014 then you already know what the cardinality estimator is going to do. But there’s also features like the Query Store that will allow you to go back and look at what happened, which is pretty exciting because we never had the ability to do that in SQL Server before without using a monitoring tool. So that’s pretty exciting. There’s a bunch of other stuff obviously, like R, that are new in 2016. So if that sort of thing excites you, then you can rush out to that too. I know a lot of shops will go like one CU behind basically. Microsoft is committed to doing more frequent CU cycles now, well, so they say. As opposed to going like a year plus between full blown service packs. So updates will be happening supposedly quicker. So if you did want to wait, you wouldn’t have to wait as long if your old policy was to wait for the first service pack.
What database diagramming tools do you recommend?
Angie Rudduck: All right. I think we have time for one more question. I’m hoping Richie can help us with this one. James wants to know what database diagraming/ERD tool or tools you recommend.
Richie Rump: I think there’s two—there’s three big ones out there. One I’ve never used, I’m not even going to mention it. The other two is ERwin and it’s now Idera’s ER/Studio. Both are really good. Both do comparable things. Take a look at them. Both are fairly expensive per seat. So if you’re trying to get it for the entire team, it’s going to be pretty pricey. So I would talk to either of those and take a look at it. My personal preference is ER/Studio but that’s only because I’ve used it longer than ERwin. So take the demo, take them for a whirl. They both are very similar and check them out.
Erik Darling: There’s also Oracle SQL Developer if you—just kidding.
Doug Lane: No love for Visio?
Richie Rump: Diagraming tools, that’s not really a tool. That’s something else.
Erik Darling: It’s SSIS that doesn’t do anything.
Richie Rump: They bought Visio for BizTalk and only BizTalk.
Erik Darling: All right, that’s two more mentions of BizTalk than I want to deal with.
Angie Rudduck: Agreed. Let’s get out of here.
Erik Darling: We’re at something 46 past the hour, so let’s go back to work.
Angie Rudduck: Bye, guys.
Erik Darling: Bye, everyone
Doug Lane: Thanks, everyone.
Erik Darling: Thanks for coming.
Wanna attend the next webcast live? Register here.