I’m finishing up a bit of a summer tour – Techorama Belgium, SQL Saturday Dallas, SQL Saturday South Florida, and this week, SQL Intersection in Orlando – and I’ve gotten the same question from several attendees. It goes something like this:
“My company hasn’t moved to the cloud yet, but at every conference session, I keep hearing Microsoft talk about the cloud. Is everybody else really doing it? And if they do, what happens to my job?”
To answer it, I usually start with a show of hands – asking the other attendees how many of them run their production databases in the cloud. I don’t have exact numbers here, since I do show-of-hands, but I’ll give you a rough rundown:
- Most attendees aren’t running production databases in the cloud yet
- Of the ones who are, most of them are using infrastructure-as-a-service (VMs)
- Relatively few (5-10%) are hosting their production databases in platform-as-a-service (Azure SQL DB, Amazon RDS)
Microsoft sessions are about the future.
The 5-10% Azure SQL DB ratio is very much the opposite of the content ratios that you see from Microsoft presenters at conferences: those can sometimes seem like 90-95% Azure services. However, it makes sense: there’s already an army of community presenters who can teach you about on-premises servers or VMs – those are the same as they’ve always been. Microsoft needs to tell you information you can’t get anywhere else, and that’s Azure. They need to tell you about the future, not the present.
It’s no different than SQL Server versions. You don’t see a lot of Microsoft presenters talking about how to get the most performance out of SQL Server 2012 and 2014. To them, that’s old hat. They’re focused on telling you about what’s new in SQL Server 2019, a version we don’t even have a release date for yet – and again, that makes sense too. Microsoft already gave presentations about how to manage SQL Server 2012 and 2014 – they did it back in 2012 and 2014, hahaha. They’re moving on – and that’s fair.
That can seem odd to attendees who are only just now adopting 2014 or 2016, but have no fear: there are still plenty of other conference sessions for you. They’re just not taught by Microsoft employees, and that’s totally okay.
Even in Microsoft’s vision of the future,
there are still production DBAs.
Let’s run with the Microsoft presentations about SQL Server 2019 for a minute. Even in 2019 – the version you don’t have yet, and you lust after with hungry eyes – it’s still a plain ol’ boxed product that you download and install. In the cloud, that means infrastructure-as-a-service – but it’s just plain ol’ VMs, only more complicated. Microsoft’s talking a lot about running it on Linux and in Kubernetes clusters – again, things you have to set up yourself, things not exactly known for their ease of use.
You still have to figure out patching.
You still have to figure out high availability and disaster recovery.
You still have to figure out troubleshooting on that Big Data Cluster.
You still have to manage permissions. And Agent jobs. And corruption checking. And restores. I could go on and on, but you get the point: there is still a lot of administration for that database.
If your app fits into a few profiles, you can outsource some of that production database administration work to Microsoft if you move to Azure SQL DB. Today, Azure SQL DB, Hyperscale, and Managed Instances are great fits for a lot of databases that, frankly, never had production database administrators to begin with. I love that – I’d rather have someone taking care of a database than no one taking care of it.
And in platform-as-a-service,
development DBAs are hella valuable.
There’s a big difference between development DBAs and production DBAs. I know, you think you do all of those things, but you only kinda sorta halfway do all of ’em. As you gain seniority, it’s time to specialize in one of those two:
The development DBA job role is incredibly valuable in the cloud. See, on premises, when I tune a query, the company doesn’t get a refund for those unused CPU cycles. They can’t instantly downgrade from Enterprise Edition to Standard Edition and get their licensing money back.
In the cloud, they can.
One of my favorite recent examples was a company who came to me saying, “We’re spending about $2M per year in the cloud just on our databases alone. Can you help us reduce those costs?” Absolutely: with just a couple of days spent query & index tuning, we chopped their biggest database expenses in half while increasing performance.
At the end of that engagement, the CTO told me, “I thought I’d save money in the cloud by not having a DBA, but what I’m learning is that in the cloud, I actually get a return on my DBA investments.”
Bingo.
In the cloud, a good development DBA pays for herself.
It’s up to you to communicate that.
If you’re a DBA and your company uses the cloud – whether it’s infrastructure-as-a-service or platform-as-a-service – it’s time for you to start learning more about:
- How much your company is spending on your databases
- What your workloads look like on each of those
- How you could do performance tuning to cut the costs
- How to make sure to reduce your instance sizes after your tuning work completes
- How to follow up with the end users to make sure they’re still happy after the cost cuts
- How to take the credit for your hard work, and prove that you’re paying for yourself
This is how you’ll get your raise this year, and how you’ll be the obvious no-brainer candidate next time you’re looking for work. The cloud isn’t killing your job – the cloud is making you the hero of the executive suite.
12 Comments. Leave new
when I first learned about IT stuff, 1GB RAM was a lot for a server and lots of documentation dealt with performance tuning of making small changes to squeeze every kilobyte of performance you could.
With the cloud being pay as you go and bad sql code resulting in higher bills, a DBA who can redesign indexes to get a tiny bit of better performance will save the company his salary in cloud costs.
After having worked in multiple types of cloud workloads, I have come to learn that moving to the cloud will generally eliminate the need for most high level administration and engineers, while radically increasing the amount of low and mid level administration time. And absolutely never saving any money for core LOB infrastructure, it just amortizes it more evenly, which I have found to be the single largest boon and headache cure for IT, in organizations that have a hard time budgeting.
The statement that the development DBA role is incredibly valuable in the cloud is correct, but the reality isn’t any different from on-prem – other than when on prem, you are only going to be dealing in cash hits of tens or hundreds of thousands of dollars once per year, instead of hundreds or thousands of dollars per month.
Sure, a good developer can allow you to reduce or avoid use SQL resources for your DB as a service, service, but that isn’t any different than on-prem. From what I have found in every organization I have worked with so far, is that there is always a shortage of developer DBA time and the organization ends up buying that additional server with 12 or 16 cores, the 12 or 16 cores of SQL licensing, the server licensing, the SAN space, etc, while the developers drown in a pile of work months or years long. Just because the cost adjustments hit once per year instead of once per month, doesn’t change the fact that the return on investment on developer DBAs is or isn’t there within the man hours constraints.
I would expect most production DBAs to become busier, just for the fact that many things that are easy to do on-prem, require work arounds in the cloud. Sure, you wont have to do patching anymore (which isn’t a big deal in an organization that recognizes you use AAGs for a reason and that they don’t need to be patched off hours), but you wont have control over it either, and in the rare case that a bad cloud patch happens, you will have no idea it is coming and woops, there goes your plans.
My favorite was the multiple day Azure AD outage in the last couple years where the geo-redundancy customers had paid for didn’t have the capacity to failover, IT staffs everywhere got called on over the weekend, to sit helpless and wait for 5-6 days while Microsoft worked on a problem that CyrusOne, right down the street had absolutely no issue with.
LOL. Your #1 task as a SQL Azure DBA is evaluating and prototyping the benefits of getting off SQL Azure. You #2 task is keeping an eye on what is going wrong today *on* SQL Azure, because Microsoft has fifty guys mucking with the system every day, not the three guys you work with in your DBA group. I like SQL Azure, its greatest problems are dynamic scaling (it doesn’t) and the linear “scaling” of cost (should be log), but you’d *better* have a DBA or two keeping an eye on prod much less supporting dev and QA activities also on Azure.
I was hired as my current employer’s first DBA, and almost everything had been built in Azure SQL Databases from the start, with just a few SQL instances in VMs. They hired me, in part, because they felt they were spending too much on Azure, and specifically on the databases. They were certainly right about that! By doing some index tuning and working with the development groups to fix some of their code and database design issues, I was able to bring those costs down significantly.
Any time you can scale down one level (in the DTU model), you generally cut the monthly costs for running that database in half, and they now have over 300 Azure SQL Databases with most of the applications using them being continually enhanced, so it’s a “target-rich environment”. Making an application more scalable and preventing the need to scale a VM or database up over time is great (and I do a lot in that area), it is much more tangible to management when you can cut an existing cost. It’s also a lot more fun for me to be able to have an immediate impact.
While Microsoft continually comes up with more ways to automate some DBA work, such as backups and even index tuning (the latter being a great way to hide code bugs), they’ll never eliminate the need for a DBA to work on schema design and performance tuning. Someone has to find the non-optimal data types, implicit conversions, incomplete compound joins, key lookups on index scans, parameterization problems and other performance-killers that the “full stack” developers don’t have the expertise to uncover and that things like Automatic Tuning can’t find or fix.
On the other hand, I’m the only DBA here and it is unlikely that they’ll hire another one in the near future. If we had started out on-prem with a group of 4 DBAs and then migrated 300+ databases to Azure SQL Database, some of those DBAs would have needed to branch out into other areas (for example, learning PowerShell and Azure Automation to compensate for the lack of Job Agent) change from Production DBAs to Development DBAs, or been laid off. While migrating to Azure isn’t the employment doomsday scenario that scare some DBAs, some may need to focus on different areas than they have in the past, and most will need to learn some additional skills.
Love reading your articles, this is def the best quote:
“I thought I’d save money in the cloud by not having a DBA, but what I’m learning is that in the cloud, I actually get a return on my DBA investments.”
Oracle is claiming that their cloud database as a service also takes care of designing indexes, monitoring performance, and tuning queries automatically.
Oh, cool. What’s the pricing on that?
Pricing? Don’t ask, just fork over the $$$ or we’ll do an audit on you and you’ll owe us even more $$$ for those extra cost features you didn’t realize cost extra.
Eeeeexactly.
[…] Brent Ozar argues that production DBAs will still be important even at cloud-only companies: […]
Over a decade ago I recall seeing a comment on Gartner or Forrester claiming that a high percentage of system performance issues were due to inefficient SQL- and those problems are more expensive to fix once they get to production systems. So here we are with supposed Cloud nirvana and really nothing has changed… (-:
[…] O has a great slide about the different kinds of DBAs you might meet in the world. If you pair that up with the features that are available in each tool, […]