[Video] Office Hours 2017/06/07 (With Transcriptions)

This week, Brent, Erik, and Richie discuss running SQL Server 2017 CTP on Linux, receive side scaling on servers and VMware, query tuning, nested roles vs. assigning roles to users, hierarchical data type, apps and scripts for automating restores, and learning Azure and AWS.

The audio’s kinda rough this week because Brent & Erik were in the same room, using a webcam microphone instead of our usual high-quality headset-type microphones. We feel sorry for our transcriptionist on this one, and we didn’t even try to make edits on it because reading it makes us chuckle as-is.

Here’s the video on YouTube:

You can register to attend next week’s Office Hours, or subscribe to our podcast to listen on the go.

Enjoy the Podcast?

Don’t miss an episode, subscribe via iTunes, Stitcher or RSS.
Leave us a review in iTunes

Office Hours Webcast – 2017-06-10

 

Have you worked with SQL Server on Linux?

Brent Ozar: Michael says, “have either of you installed or configured SQL Server 2017 CTP, like the community previews of SQL Server 2017; any initial thoughts or impressions?

Erik Darling: There’s a follow up down there, it says [crosstalk]

Brent Ozar: Oh have you done it on Linux, oh god. Oh that changes everything, I liked your original question better.

Erik Darling: I don’t I could hang out with this Linux…

Brent Ozar: So why haven’t you?

Erik Darling: I’m just not a Linux guy. I feel like I would not learn anything that I want to learn by installing SQL Server on Linux. I’ve used Linux before, I’ve installed it. I’ve tried to install Arch Linux once, it didn’t go well. I’m not a fan of just all the weird command line stuff you have to do on Linux. It doesn’t do it for me – not want to spend time fixing that.

Brent Ozar: Richie, how about you? Out of your array of laptops, do you got any VMs that are running Linux or any ones running Linux [inaudible 0:00:57.5]?

Richie Rump: No, no. I mean I’ve got plenty of other stuff in the cloud that are running stuff, but everything that I run, I can run on Windows. And it’s not really about the OS at that point, it’s about the language and the application, for me. And the question, for me, with SQL Server on Linux is, why? Right, is there a reason for you running Linux in SQL Server, this brand new product, essentially, from the ground up? And if the answer is, because that’s the only knowledge we have is Linux, I’d be like, well why not Postgres? Right, why not MySQL, why not MariaDB, right? [crosstalk] it’s alright.

 

Should I use receive side scaling in VMware?

Brent Ozar: Let’s see here. So next up we’ll take – Gregg says, “what’s the recommendation on enabling receive side scaling on both servers and VM network adapters? This is VMware, I’ve got SQL Server 2008 R2.” If you’re playing around with receive side scaling, make sure you Google for receive side scaling and SQL Server and VMware, because there was a huge bug that performance just died with SQL Server, which I should have remembered on that client the other day. But I don’t know what the symptoms are or anything like that, I just remember reading a blog post and going ooh wee. I’ve never had a problem that I went back and went, this, this is the answer here, you know, receive side scaling. But I would also argue that most of the problems that I run into are either CPU and memory, that they surface bigger that way and it’s easier to play around within – I have no problem with receive side scaling, I just haven’t used it to solve a problem yet.

 

Why do Excel queries use more resources than SSMS ones?

Brent Ozar: Let’s see, Michaela asks – I’m just going to pronounce your name different every single time, so this time I’ll say Michae-la. Michaela asks, “why do Excel queries seem to use more CPU and memory than the same query in Management Studio?” That’s a great question, which one of you wants to answer it? No, come on… There’s the same post that we – Michaela says, “got my name right”. So it is maybe Michae-la, maybe I got that right.

So my thing would be probably parameter sniffing, so there’s this thing called slow in the app, fast in SSMS. And you know what’s funny, somebody said the other week, why don’t you guys show the blog post up when you’re talking about it? You know, like it made a huge difference with them with Office Hours. And you know what, that’s exactly what I’ll do. So if you search for slow in the app, fast in SSMS; if you want, for bonus points, if you want to spell Earland Sommarskog’s name out, you totally don’t have to. But this thing is an epic monster post, it’s huge, it has a table of contents and it lists out the different things that can cause your query to be slow in one app and then fast in another, and vice versa.

So that’s the place I would start is looking at the two different execution plans on them and see if you can get them to be the same just by blowing the plan cache, using the trick techniques that he shoes inside there. But otherwise it shouldn’t make a difference – no, that’s not true. Okay, so there’s one other scenario where it could make a difference. Let’s say that your Excel user is in, like, Tijuana and you SQL Server is in Alaska, whatever the network pipe is across there, if you’re facing ASYNC network I/O waits, meaning it’s taking a long time to push the data across the…

Erik Darling: It’s asking pretty specifically, why does Excel use more CPU and memory…

Brent Ozar: That’s true. [crosstalk]

Erik Darling: It seems like they’re hitting some sort of resource there. I’d be curious about what they’re actually pulling, because…

Brent Ozar: How many rows…

Erik Darling: Just because I, for a very short time, gave project managers access to SQL Server via Access, because they were incapable of getting it any other way. So I would just set up Access as a data provider to SQL Server and they would be off to their tracks, you know, looking at their various supports and stuff. And writing access queries is awful, so even if you’re hooking it into SQL Server, it’s not fun. So at this point I would just want to know what’s going on with it, why Excel’s the choice for expressing the data.

Brent Ozar: And then if you want to share, if you can share, the execution plan to see what the differences are between the SSMS query and the Excel query, there’s this tool that Richie Rump built, Paste The Plan. So if you go to PasteThePlan.com; Richie, tell them what they’ve won…

Richie Rump: You’ve won a website that will actually format the crap for you. Actually that’s another website, oh my gosh, you just got me all mixed up with the different websites. No, this will actually – you paste in your XML and it will then give you a link, and then with that link you can then share that with other people. I went Statistics Parser on you, oh my gosh.

Brent Ozar: They’re totally different. So it shows the plan – this one’s not a good example because it’s a WHERE condition. [crosstalk]

Richie Rump: Actually, we’ll have a release here pretty soon that will remove that and will format all that cursor-y stuff. It will actually show you a plan for all that.

Brent Ozar: Don’t you go breaking my heart. That sounds amazing; I want this. Let’s see here – oh, and Michaela asks – Michae-la says, “report people love Excel and charts.”

Erik Darling: Ah, there you go.

Brent Ozar: I’m with them on that.

Erik Darling: You use a graph database.

Brent Ozar: Whoa, a graph database, that’s good.

 

I have a normalization argument with a developer…

Brent Ozar: Wes says, “I’m writing a back end database for a coworker, moving things from Access to SQL Server. I have it in perfect Boyce-Codd normal form and he wants to make changes to break data integrity in normalization. What’s the best way to pushback?” Richie, that’s got you written all over it.

Richie Rump: You could roll up tables for that, I mean, if you’re against it. I’m not one of those guys who says that everything needs to be in perfect third normal form, or fourth or fifth normal form. I’m one you go to third and if your performance or your needs need to back off and de-normalize, then go ahead and de-normalize it. It depends on the scenario that you need. And the normal forms are kind of primarily to save disk space, because disk space was extremely expensive back in the day. We don’t really have that problem now. So if you need to have some sort of roll up tables where or reporting tables that the data is duplicated or you actually, you know, break out into second normal form on some of your tables; I think that’s perfectly fine. You just need to take a look and say, okay, what’s our options between to fix this problem in the application, and if it is, means we de-normalize for speed, then you de-normalize. I mean, that’s just the way I look at it.

Brent Ozar: He might get excited by views too, if there’s – if he wants to pull data out more easily, you can write him some views to do easier queries.

Erik Darling: Lewis Davidson’s book.

Brent Ozar: [crosstalk] We’ll pull that up. Good, keep talking…

Richie Rump: Yeah, the cool thing about Lewis Davidson’s book is that when I was initially reading it, I didn’t know Lewis at all. I had never met him, and I had a question about the book, and so I Tweeted him the question about the book as I was reading it, and we started having a conversation about whatever question I had in real time, over Twitter, while I’m reading the book.

Brent Ozar: And, I got [inaudible 0:08:23.4]. most authors are probably excited to do that too because it’s like, oh my god, someone’s actually reading my book.

Erik Darling: We’ll pay you a fifth [inaudible 0:08:33.0]…

Brent Ozar: Yeah, and hopefully you didn’t tell him you downloaded it off of BitTorrent either. I mean, that’s always heartbreaking for authors.

Richie Rump: Yeah, this guy, Brent Ozar, send me a PDF, I don’t know where I got it from, it’s crazy.

Brent Ozar: Ys, he’s got a few of those in there. The most recent one, the very one on the top of the list is the newest one, you don’t need the older versions.

Erik Darling: I brought the newer version. It’s an interesting read; there’s some good stuff in there about storing names that even I thought was interesting, as someone who used to store a crap-ton of names, never thought of. When I read it, I thought oh it’s so good. But I’m not going to tell you what it is because that would ruin it. No spoilers from me.

Brent Ozar: If you’re the kind of person who likes saying the name Boyce-Codd normal forms, stuff like that, you’ll probably enjoy this book.

Erik Darling: Absolutely.

Richie Rump: There was another book that I used to go to all the time, it was the Data Model Design Patterns, and there was like three volumes of this thing. The cool thing about that book is it had different verticals. So if you were in healthcare, it gave you a bunch of models for that, if you were in certain order taking type stuff, it gave you certain patterns for that. That was pretty awesome; I’ll see if I can find the title of it and I’ll put it into the cat window.

Brent Ozar: There’s a few in there. Oh, there’s a volume three; data model resource book?

Richie Rump: I’ve got to look it up. You know, it’s usually on the top of my head, but because I’m on the spot and the book is in the library, I’ve got to – it’s not here, it’s over there somewhere, behind my wall.

Brent Ozar: behind my wall… Behind door number one…

 

Can I change passwords in a transaction on multiple servers?

Brent Ozar: J.H. says, “Will creating a stored proc with begin and end, will that rollback a user changing their own password if it succeeds…” Oh my goodness, so you’re trying to do a transaction across multiple servers through linked servers, probably, in one stored proc changing passwords. I can’t imagine that’s going to get honored.

Erik Darling: No, just begin and end [inaudible 0:10:29.4] I think you would have to do some sort of explicit BEGIN TRAN and…

Brent Ozar: Yeah, or even beyond that, what I would do is I’d like create a table of, here’s all the servers I’ve got to change, and not shucks I’m going to need their passwords, yeah it doesn’t work. You know, have you heard of Windows logins?

Erik Darling: Yeah, that would solve a much bigger problem for you.

Brent Ozar: Yeah, or just do all the changes on just one server and take the logins from there and replicate it out to others.

Erik Darling: You know what I would do, I would use a registered server query for that, because that way you can put all the servers that you need to change the password on into a group [inaudible 0:11:06.1] and do all your password changes from one place.

Brent Ozar: I like that.

Erik Darling: I mean, it doesn’t work for a stored procedure, but…

Brent Ozar: Yeah, but if it’s still like changing and enlarging. Yes, I like that a lot. So if you search for either registered servers or a central management server, just lets you execute queries across lots of servers. They may fail on one, it may still fail on one, one times out, whatever, but then at least you’ll know which one it was and you can go retry it.

Erik Darling: Would agent jobs be a good fit for that? Like if you had an agent job that [inaudible 0:11:33.3] to other servers?

Brent Ozar: The problem I worry about is, if someone’s changing their password, this has got to be like a parameter for the stored proc, and if you’re going to retry it, you’re going to write it to a database somewhere and like…

Erik Darling: Yeah, because you’re executing a stored procedure with someone, then that’s in the plan cache maybe and…

Brent Ozar: Cool… [crosstalk]

Erik Darling: Weird stuff could go on with that…

Brent Ozar: Don’t do that. So instead, if you want to sync logins across multiple servers, Robert Davis Sync Logins; Robert Davis wrote the book, literally, on database mirroring, and so he’s got scripts out there on how to copy your logins to a database. And the same trick works for even just standalone SQL Servers.

 

Should I have nested roles or lots of roles?

Brent Ozar: let’s see what we got next here. Deborah says, “is it better to have nested roles or assign many roles to users?”

Erik Darling: It’s much better to have an empty nest.

Brent Ozar: Yeah, much more peaceful, says the guy with kids [crosstalk]…

Richie Rump: Ten more years they go to college, yeah.

Brent Ozar: You’re going to be so lonely. I don’t think any of us have an opinion on that one.

Erik Darling: No, we don’t have a lot of security opinions in general.

Richie Rump: Do security, that’s my opinion, do security.

Brent Ozar: Someone else do security. Look Securing SQL Server by Denny Cherry. It’s now, I think, in its third edition. So just check to make sure you’re getting the most recent version – yeah, third edition. Normally I say SQL Server security book and people are like, whoa. It’s great, really well written, it’s fun to read, it’s a really enjoyable digest for reading. And if you know Denny Cherry, you’ll think that someone else probably wrote it, because there’s not a typo in the book as far as I remember seeing. It’s really nice and cleanly spelled. I kid, because I love Denny, he’s wonderful.

Richie Rump: No, you kid because it’s kind of true.

Brent Ozar: It is completely true, yes. When Denny said his wedding vows to his wife, he couldn’t even get I do correctly. It’s like yeah….

Erik Darling: Do we, do I?

 

Have you used the hierarchical data type?

Brent Ozar: Wes Palmer says, “have you ever used the hierarchical data type, what are your views on it?” We just had a client question about that…

Erik Darling: Yeah, it’s okay, there are…

Brent Ozar: Gleaming endorsement.

Erik Darling: It’s okay, it does solve the hierarchical problem pretty well. There are some great articles out there that, I think, are much better choices. Jeff Moden, over on SQL Server Central, has some cool hierarchy on steroids articles, where he goes through how to calculate left and right bowers and traverse trees. And then Adam Machanic has a really good one, using CTEs and some crazy CLR code [inaudible 0:14:27.9] and that was a good read. The funny thing about that article was, one of my first blog posts when I was on your site was about – just a simple thing about common table expressions and how if you join a copy table expression it re-executes the syntax, blah, blah, blah. There was a comment on it from Adam, because this thing, this monster dropped like the same day or the day after or the day before, and he left a comment on my post, like I guess we both blogged about CTEs this week, and I was like yes, I guess…

Brent Ozar: I was so proud. And then he does the big monster post, of course.

Erik Darling: Yes, and so he does a lot of cool stuff. And [inaudible 0:15:10.6] kind of thing with the same author, the T-SQL querying book [crosstalk] there’s a pretty good chunk of space dedicated to hierarchies in there as well. So those are the – I would explore those choices before committing to the hierarchy ID type, just because, you know, it can be a little bit more clean and you don’t have to have all those crazy functions in there. I’ve seen, not CLR waits, but weird sleep task waits that can kind of dominate query execution when you’re waiting on those hierarchy IDs parent of, and whatever you calculate and you come back and forth.

Brent Ozar: Just saw that…

Richie Rump: Yeah, with everything, I think I say this ever week, but test it. You know, just go ahead, try one, try the other, test it, write a CTE. I’ve written a ton of recursive CTEs; if you know what you’re doing, you can get it to be fairly fast. Just go ahead and try it, test it and see what works in your environment.

Erik Darling: The one thing that I will say is cool about the hierarchy ID type is that the functions that you can use on that to, like, calculate [crosstalk] you can make computed columns from those and index those. So you can help yourself out a little bit in those situations. Where that’s a little bit harder to do, like persist, with a recursive CTE.

Richie Rump: And it really depends what you’re doing in a hierarchy. You could actually pre-build your trees and kind of do that, if you’re not going to touch it very much. But if your trees are always, kind of, moving in flex and if you kind of want to go from one node down to another one, or move up. There’s a ton of stuff that you could do with the tree, but do you need to do those things with the tree is really the question.

Erik Darling: Yeah, I played around with it a little bit just using the stack overflow database, because in the post table [crosstalk] and then the comments table, you can track that back. So you can make these kind of cool lists. You know, here’s like the initial question, here are answers to the question, here are comments to the answers and – so you can do kind of cool stuff like that.

Brent Ozar: Sum up like total score for a question [crosstalk]

 

Brent Ozar: let’s see, Jacqui says, “I bought your online everything classes and looking forward to getting to review them, hope to learn a lot.” Awesome, cool, I hope you learn a lot as well. Erik’s been adding performance classes too, and added some new T-SQL classes.

Erik Darling: Yeah, I’m going to start rolling out some of the triage-y stuff. [crosstalk] Yeah, closer to DBA stuff and performance stuff. Sort of how some DBA fundamentals work.

James says, “your T-SQL level up, level two was awesome.”

Erik Darling: Oh thanks, you’re the one who watched it [crosstalk] You know, I didn’t give my mom a log in because I didn’t want her to be that one view. So thank you for being the one.

 

Any good apps to automate database restores for testing?

Brent Ozar: Dorian says, “are there any good apps or scripts that can automate database restores for testing and development. [crosstalk]

Erik Darling: Yeah, it’s awesome for that, they have a crap-ton of automation built in. they even have a really cool one that Jason [inaudible 0:18:27.8] was telling us about at DBA Days, where they automate the backup restore, CHECKDB and drop of databases to a secondary server, if you want to get really crazy and offload your CHECKDB somewhere else. That is a really cool [problem 0:18:41.5] with a lot of good stuff built into it.

Brent Ozar: People often say, you know, like does anyone still buy backup utilities, you know, what’s the use now that compression is in the product. Stuff like that, adaptive compression, wizards for log shipping are better and they have cool extended stored procedures you can call for log shipping; some neat stuff.

Erik Darling: They also – the stuff that always blows me away about Litespeed though is, not to get too product endorse-y, you can do object double restores, you can restore a table, they’re really good at reading – you can read through the logs, to a certain point, behind the transaction you want. So like, if you have log backups every 15 minutes, you need to restore to a point within a log backup, it makes it dead simple to bring you to like the seven, eight, nine minute mark to a certain transaction. Like the stuff that the DBA [inaudible 0:19:31.5] if you had to do that on your own at 3am, you would be [inaudible 0:19:36.5].

Richie Rump: I have a question about that question. Why are we restoring production to test? Why are we restoring to dev? Our dev and test environments should be something completely different so that the developers can test, not just the 80% case that is typically in production, but the case that is kind of out there that usually doesn’t happen when I got data, kind of, all messed up. So our test databases need to be different than production and our development databases need to be different than test. So I’ve never been a fan of just restoring production into test and calling that a good test, because typically, you’re going to get the same thing in production and not the oddball whack-a-do problems that we need to be testing in test.

Erik Darling: But production’s a good thing to base that on. If you restore production data, you can then mangle that in some way to get your edged cases in, or you can, you know, do some scrubbing.

Brent Ozar: You can sell it to the Chinese…

Erik Darling: Yeah, Russians…

Richie Rump: But at that point, put it into a stage environment and test your deployment that way. [crosstalk] You should have certain test cases, be testing out your tests in a test environment.

Brent Ozar: Yeah [inaudible 0:20:42.4] You should, dear readers, also know that we will often advocate for things we don’t do here. So like, I think our dev databases actually are different here; in this case, they’re all with Stack Overflow. Ooh, the other thing I was going to say was – mainly on CHECKDB – if you’re the kind of shop where you can’t buy tools and you’re doing the restores in order to offload CHECKDB, the Minion Ware folks have this free tool out called Minion CHECKDB. And this is so neat to me because they built in all this load balancing, and they’ve also built in a popup that I can’t seem to close. Okay hold on [crosstalk] there we go. So this thing will let you, kind of, round robin your CHECKDBs. If you only have one server to do restores, it will restore different production boxes each day. This isn’t the kind of, like Litespeed has a GUI, click your things and a wizard. This isn’t that; you’re going to be 100% table-driven here, so it’s just more work, but it has a lot of cool capabilities.

 

Should I start learning Azure?

Brent Ozar: Alright, let’s see, next one, Ronnie says, “based on your experience in the field, do you think it’s necessary to start learning Azure or support cloud databases to the backups restores? Is that the direction of the big money jobs or is that a specialty skill set?” Well, we should do our round robin, so…

Erik Darling: You should learn R, I don’t know.

Brent Ozar: Oh yeah, sure.

Erik Darling: No, it’s – I think it’s always worth it to know this stuff and be prepared if, you know, someone wants you to start doing it. But would I make that my primary job focus? Probably not, you know, just looking at specifically Azure adoption rates. I’m not sure that that’s where, like, the big money – you could probably find some niche work in there. I don’t think you’re going to find, you know, the end of your career in it.

Brent Ozar: And so we should explain that, because other people out there are probably like is everyone doing Azure? So the work that you do out of there, what percentage of the work is like normal [inaudible 0:22:41.7] versus Azure versus AWS?

Erik Darling: Oh I think I’ve worked on maybe one or two Azure boxes in my time consulting with you. I work on a ton of AWS boxes. I haven’t hit a Google Cloud box yet aside from when we did the white paper work, but the majority is on-prem and on-prem virtualized.

Brent Ozar: Richie, how about you?

Richie Rump: I’ve kind of never been a fan of learning any knowledge without an understanding on where you want to go with it, right. So if you don’t have a plan of what I’m going to do with this cloud knowledge, then it’s probably going to be wasted time and wasted knowledge. I’ve always preferred a, kind of, multi-step approach to learning knowledge. I want to be able to know enough about the cloud so that I can have an intelligent conversation about it and maybe not necessarily how to implement it. And if I get into a gig where I need to implement it, then I’m going to take that next step and know a little bit more on how it actually works and maybe how to implement it and how – what the nuts and bolts are. And then maybe after that, I take another step to where I’m actually teaching about it, where I’m really intimate with the product and I can kind of show others how to do it.

The second part of your question is, well is the jobs going to the cloud? Kind of, maybe. I think a lot of companies are struggling with that question, especially, you know, we’ve got CIOs saying, we’ve got to go to cloud because we’re going to save all this money; and that’s, kind of, really out for debate about what you’re trying to do in the cloud. I think those companies that are, you know, doing Greenfield development in the cloud are saving money, and those that are putting Legacy stuff into the cloud are probably not saving as much. That’s just kind of my gut feeling for it. And the whole Azure versus AWS thing, I try to be agnostic, but I do a ton of Amazon work here, and I kind of really learned to love it, and as a Microsoft geek. And 20 years of my career has all been at Microsoft, I kind of feel bad about that, but then I yell at the Microsoft PMs in Azure and I just say make it better, you know. Because this is what AWS does, and now make this better; and it kind of makes me feel good that I’ve actually, you know, unloaded stuff off my chest and say Microsoft, you need to be more like AWS because I don’t know what Azure does, because it does like four or five different things, but it’s the exact same thing.

Erik Darling: One other thing you might run into is if you – say you spend the next three to six months and you learn a whole bunch of Azure stuff, and then you put it down and you come back to it in a year, year and a half, or even six months later, a lot of what you know might not be all that relevant. It probably won’t be as true as it was when you were using it, you know, it just changes so drastic.

Richie Rump: No, it’s totally going to change, and here’s a perfect example. So Lambda when it was first released, and it was only released what, two years ago, not even that?

Brent Ozar: [inaudible 0:25:28.5].

Richie Rump: Yeah, so when it was first released, there was a certain way you needed to go and end a function, right, and the way that you would end a function, you had to call certain things and it’s done. And that is still in the documentation, because it technically still works, but now they have a new way to do it. Essentially you call a callback and it’s done, and that old way is now essentially saying we’re deprecating it. It works but at some point, we’re going to pull it. And, you know, if you kind of knew that and then you came to it now and you start writing this old way, then a few months later they actually pull it out of the product, then you’re like – well one, what happened? And two, how does it actually work now? And three, oh boy, I was way behind the curve. Or even worse, you come back to it a year later and it doesn’t even work [crosstalk]

Erik Darling: Then their apps stop working because you wrote the function.

Richie Rump: It’s so crazy on how the cloud is just constantly changing; it is maddening. Then you get to an environment like Javascript that is also constantly changing as well, right. So moving target on moving target, and you know, trying to say that this is where we’re going to be for the next five, ten years, it’s not going to happen. This isn’t like we’re writing Cobalt code and it’s not going to change for 30 years, it’s a whole different animal.

Brent Ozar: Yeah, I would concur. Especially that you don’t want to waste your time learning something you’re not using. Literally, we wrote the Google white papers, they’re out of date within three or four months, yes, things start changing. I do either an Amazon or an Azure blog post about all the stuff around backup speeds. It’s going to be out of ate within a short number, a couple months, so just change [crosstalk]

Erik Darling: Like a lot of the stuff I wrote in the perf white paper is now not exactly true anymore because AD couple memory and CPUs from being tied together. Like usually have X amount of GBs per CPU and it has to even it out and we actually need more CPUs and more RAM and stuff. I should go back and revisit…

Brent Ozar: You asked one thing that was interesting, is that the direction of the big money jobs? It’s always supply and demand. So when there’s a huge demand and low supply, all of a sudden, those people will be raking in the big bucks, but it may not be for a long period of time. So for example, R and Python right now, there’s a huge demand, there’s very little supply. Those people are raking in the bucks right now. That will change as more R and Python people just simply come out of college, because those technologies are free.

Richie Rump: I mean that was true with Hadoop too right, a couple of years ago? And now, where’s Hadoop? I mean, you don’t hear about it anymore.

Brent Ozar: We blogged about it, like Jeremiah was doing Hadoop work, and really quickly realized that, oh my god, all these college kids are coming in and they’re making, you know, $25-$30 an hour, why would you want to compete with that? It’s a bad direction. Alright, well thanks everybody for hanging out with us this week at Office Hours, and we will see you guys next week. Adios.