You’ll notice that we’ve been blogging and writing a lot about Google here this week. I gotta tell you – it’s totally bizarre sitting in a Google Cloud Next conference session in San Francisco, watching presenters using Macs, coding with Visual Studio Code, demonstrating SQL Server.
I even talked to a SQL Server DBA attendee yesterday who said, “My employer won’t send me to PASS, but they sent me here because they’re thinking about moving everything to the cloud, and they want me to be ready.”
Yes, this stuff matters to you.
Now, with that out of the way, let’s get started. In managed platform-as-a-service databases like Microsoft Azure SQL DB, Amazon Relational Database Service (RDS), and Google Cloud Spanner, the vendor takes care of basic care and feeding like:
- Backups, restores, and corruption checking
- Hardware & storage provisioning and maintenance
- OS & database patching
- And all the really boring stuff you never really liked to do anyway
Developers love PaaS databases because, uh, they hate DBAs. I mean, they don’t hate us as people – especially when we can help make their apps go faster – but we’re often labeled as Don’t Bother Asking because all we do is say no. With PaaS databases, nobody says no. (That’s totally not true – it’s just that the vendor’s the one saying no, and it’s really clear black-and-white, as in no, that feature simply isn’t turned on for anyone in the world. It’s not just you.)
SQL Server DBAs haven’t worked much with PaaS.
In the Microsoft space – where you, dear reader, probably make your living as do I – platform-as-a-service means Azure SQL DB. That’s been a total non-starter for migrating most existing apps over because it requires so much work:
- Different T-SQL coverage area – like Azure SQL DB’s painful cross-database query setup, or different ways of scheduling Agent-type jobs
- Different (and changing) feature support – like no SSIS, or SSRS, or when Azure SQL DB removed support for SQLCLR
So you probably haven’t seen your developers racing to convert their SQL Server apps over to Azure SQL DB. (I’d argue that for building a new SQL Server-based app, though, you should default to Azure SQL DB first. Although if you’re building any app, you should look at the PaaS options out there – for example, we’re using PostgreSQL for our new development. No, don’t worry, we won’t turn this into a PostgreSQL site.)
But away from Azure SQL DB, things are different.
I have to say this because if you read SQL Server blogs, they’re likely written by SQL Server folks who spend the vast majority of their time focused on the Microsoft ecosystem. (That’s especially true of MVPs – Microsoft heavily markets Azure to them, and encourages them to blog about Azure.)
When you get away from Azure SQL DB, there’s a very different story happening in the rest of the database community. Developers who built their apps on MySQL and PostgreSQL, the two big open source databases, have all kinds of cool options on where to host their apps with dramatically fewer (or no) code changes:
- Amazon Relational Database Service is hosted/managed MySQL, PostgreSQL, MariaDB, Oracle, and yes, SQL Server
- Amazon Aurora is MySQL and PostgreSQL compatible, but actually a different storage layer underneath, giving you performance and scalability capabilities you didn’t even have with those databases (seriously, the Aurora promo video is the best explanation I’ve ever seen for PaaS databases)
- Heroku Postgres is managed PostgreSQL, got a lot of early traction because Heroku offered a lot of cool services for developers, but that’s fallen off the radar lately
Developers can just take their MySQL/PostgreSQL apps, forklift ’em up into the cloud, and run them as-is. That’s pretty doggone compelling. That’s why PaaS databases are on fire – outside of the Microsoft market.
Today, Google announced Cloud SQL, their competitor.
Today at Google Cloud Next this morning in San Francisco – it’s like their Microsoft Build – Google announced Cloud SQL, their own managed PostgreSQL database-as-a-service in beta.
Cloud vendors are racing for managed support of every database – including SQL Server. If your existing app’s database will work up in their cloud, then you can move your entire workload up to their cloud – and that’s what they want.
In the day 1 keynote, Eric Schmidt – the chairman of Alphabet – focused entirely on databases. His whole point was that if you can get your valuable data unlocked from your on-premises proprietary database, and get it up into the open cloud, then you can run all kinds of analytics and machine learning against it. He said you’ll want to use Google’s team – not just their services, but their team, touching on their acquisition of Kaggle – to help unlock the value.
He wasn’t talking about lowering costs or reducing staff. He was talking about adding value. (This is a refreshing change from earlier cloud vendor marketing that kept touting reduced costs, which frankly almost never happens when you move an existing company up to the cloud. It totally does happen for new businesses, though – I ain’t buildin’ no data center.)
So what do Microsoft SQL Server pros need to take away?
Imagine if you were a PostgreSQL DBA and you saw these things happening. You’d probably be scared for your job.
In reality, PostgreSQL people still have a huge amount of work ahead of them. (As an HSBC exec said during the day 1 keynote, the hardest part is finding people who can use all these data tools.) Their jobs won’t change tomorrow, but they will gradually change over the coming years. The crappy part of database administration – backups, recovery, patching, failovers – become the vendor’s job. That’s awesome.
Do SQL Server DBAs need to learn PostgreSQL? If you’re already a DBA, no. Your job is still safe working in the Microsoft SQL Server stack. Even if you do want to jump ship, be careful which parts you learn. I wouldn’t want to focus my career on the internal plumbing, the parts that the cloud is abstracting away anyway. Focus on things like query construction, tuning, and index design.
Do SQL Server developers need to consider PostgreSQL? Probably yes, the next time you start a project. Developers are in the business of constant learning. Every time they start a new project, they consider new frameworks, languages, and persistence layers. I don’t think you have to pick up a good PostgreSQL book yet, but I think you should at least know that the option is out there – and it’s a good one.
30 Comments. Leave new
Thats good news i’m think dba should spread His knowledge . So that book it’s really good but there are anothers for start.
I started using postgres a year and a half ago. Our application is multidatasource, the saas solution running on big sql instances (tbs of data), now we have added 6 smaller db
after a first period to find an optimal confguration, the next step was to optimize the query and the indexes. usually if a query is optimized PG also works well on sqlserver, the other way not always. There are many configuration parameters that can affect performance, for example, I changed the cost of random reads to take advantage of the performance of our san. Understand and use the “vacuum” is crucial (like upd stats in sql)
Once PaaS Databases comes bundled with wait stats related monitoring capabilities, i think it will start really taking off. AWS is working on it, very similar to Oracle OEM. Scary for the DBA job, but exciting from the technology prrspective.
Anand – good news! Amazon RDS for SQL Server has had that since day 1, and Azure SQL DB has even better wait stats than SQL Server does. In Azure SQL DB, you get database-level wait stats.
Thanks. I am checking in SQL RDS. There is host level info via cloud watch. Enhanced monitoring shows processes. Dont see info from dmvs. What am i missing :(.
sys.dm_os_wait_stats, the same one you would use in on-premises SQL Server. You can even use sp_BlitzFirst @SinceStartup = 1 to query it if you like.
Ahhhh, i have sp_blitz, whoisactive etc already running there. Was mentioning a UI with these info e.g. Like a OEM. Already RDS does backup, HA/DR (with MultiAZ/ Read replica). Add OEM style, deep perf info/recommendation to the mix and i dont know what will be a DBA R&R.
Thanks for the time Sir!!!
Btw looked at SQL Azure, nice performance related info. Have to do deep dive. Thanks for the tip.
Sure, check out RDS Performance Insights. It’s already in public beta for RDS PostgreSQL, and if you contact your AWS reps, they may be able to get you on the SQL Server preview. You can see a picture of it from the re:Invent session:
https://www.brentozar.com/archive/2016/12/7-things-learned-aurora-aws-reinvent-2016/
Or of course, get any number of monitoring tools – Quest Spotlight, Idera SQL DM, SQL Sentry Performance Advisor, etc. They all have varying levels of compatibility with RDS SQL Server, and since the underlying wait stats are the same as on-premises SQL Server….
Hi Brent,
interesting blog post and cool to read something positive about open source databases especially from your great company. My company is developing a web portal for our customers…the “old version” is based on a proprietary language and does take heavily usage of SQL Server Features in TSQL, XML agent jobs etc. However the new version we develop and migrate is written with Java and using Hibernate thus it’s basically able to run on any RDBMS. We thought about moving to open source databases for that application in the long term in order to get more flexibility with scaling databases (no license costs). Cloud probably is not an option because in Germany data privacy rules are quite strict and our customers are insurance companies who very much care about the security and privacy of their data. I’ll definitely have a look at the Postgres book you recommended…but apart from that do you have any advice on best practice setups or literature? First thing for me to start is backup and recovery. Here I have read some rather scary blog posts (promoted by your weekly mailing lists) about MySQL and probably Postgres as well which led to more confusion than answers. Furthermore I did not find any migration success stories or people who confess…yes switching the database system can be done and mustn’t lead to a complete desaster.
Your advice is greatly appreciated. It’s perfectly OK for you to say “OK I might help you get on track but that’s consulting services”. Then just get me a quote. Sorry for writing such a long text as comment…I hope that is not against any written or unwritten brentozar.com rules… please be kind 🙂
Martin
Martin – thanks for the kind words. No, I don’t have any recommendations for learning resources on other platforms. When we started doing new development in other databases, one thing was very important to me: I didn’t want our team to be managing a different database. We’re simply not experts in other platforms, and it would cost us money to get up to speed on other databases. That’s why we’re working with managed databases. Sorry about that!
Interesting, thanks! Cloud Spanner is very intriguing, will you be looking into it?
Alex – you bet. I sat in on all of the Cloud Spanner sessions, even though it’s not for me – it’s just a really interesting technology. If you like databases, and have a couple of hours to spare, I’d recommend watching the Cloud Spanner 101 and 201 videos from Next on Google Cloud’s YouTube channel. It was neat to see the design decisions they’d made along the way.
It’s not really useful to me today because it’s not compatible with anything else – for example, you have to do all DML through their API rather than SQL statements – but they’re very up-front about that. Use Cloud SQL for existing app compatibility, Cloud Spanner for new app scalability.
Ooooh awesome, I’ll watch those.
Really as a Microsoft DBA you’re job is still safe? Which world do you live in? Look at any job opening in Silicon Valley and look at what they are using as an RDBMS for their data – its either Postgres, MySQL or one of the NoSQL databases. And they have the pulse of where the industry is moving. Why would any company want to pay for huge licensing costs when they can go open source? SQL Server has gone the Oracle way, just ridiculously upping their licensing fees.
Sure thing.
I love how some of those mention MCDBA certification. Oh you want a SQL Server 2000 certification that was officially retired 7 years ago? I wonder if these companies are behind the times, out of touch, or don’t have people that know what they are talking about helping to write the job reqs. Seems like it has to be one of those and any of them make me skeptical of applying.
Note: No offense to MCDBAs out there but SQL 2000 would be in college now if it were a kid.
All crappy low paid contract/consulting jobs with dinosaur era companies.
Check out any name-worthy tech company in the Bay area, and ask them if they use SQL Server in their tech-stack?
Sure, I have name-worth tech company clients in the Bay area. Nothing I can name publicly, obviously.
@Brent – Am sure there are, the percentage of companies has to be pretty low. Why would any company spend their dollars when there are equivalent technologies which are open source and allow you to customize as they see fit? I started my career with SQL Server, and still, love it to death. Why would a startup getting off the ground use SQL Server? It’s my observation in the tech market in California. If you have only SQL Server listed on your resume, good luck finding a job.
Kind of odd that you’re spending the time to read a SQL Server site then, hmm?
Might wanna get back to work at Your Incredibly Cushy and Well-Paying Bay Area Job™. Someone might walk past your open cube and see you goofing off.
this is a pretty good rundown of why an enterprise might choose an established vendor based product over OSS.
https://www.quora.com/Why-would-a-company-choose-to-use-an-SQL-server-MySQL-Server-PostgreSQL-etc-over-open-source-databases-Is-it-because-the-company-requires-commercial-support-as-MySQL-has-commercial-support-from-Oracle-and-PostgreSQL-has-it-from-EnterpriseDB
I agree with Sharad. Microsoft is killing SQL Server by raising prices to outrageous levels (it’s even orders of magnitude more expensive than Oracle on AWS!) in an effort to drive customers to Azure. But for seasoned RDBMS people, Azure is still not-ready-for-prime-time. As a 20 year Data Architect I see with my own eyes the rapidly shrinking market share for SQL Server. Its a real shame.
Jeff – wait, what? What do you mean that SQL Server is orders of magnitude more expensive than Oracle on AWS?
Here’s the RDS pricing, for example:
https://aws.amazon.com/rds/oracle/pricing/
https://aws.amazon.com/rds/sqlserver/pricing/
There’s not a single order of magnitude difference, let alone plural orders.
Brent I’ve been updating my info on AWS SQL Server config options since there have been big changes like the support for native SQL backups you talk about here. I had to stand up a few test SQL instances and just for giggles I compared cost to create an Oracle instance ( I believe it was an “Oracle Standard 2”?) with a micro instance type and 20 gb storage to a SQL Standard version with non-prod config and it the AWS estimator for monthly costs showed something like $56 for the Oracle instance versus $600 for the SQL instance. Hey I played with quite a few instance sizes so don’t shoot me if my numbers aren’t exact. But it seemed obvious that SQL is now even more expensive than Oracle. Besides… the whole point here is how expensive MS has made SQL at a time when open source is everywhere and getting better rapidly. I made my career first on administering and then on designing and implementing SQL Server databases and for the past 5 years have seen the market share drop like a stone. Customers who actually prefer on-premises databases over cloud offerings will not pay these prices for new SQL licenses. Just like you, they are building their new products on Postgres or some other less expensive database platform, even knowing that the robustness and functionality isn’t there yet.
Jeff – you don’t really want to license SQL Server Standard Edition on a micro instance. Use SQL Server Express Edition – it’s completely free.
#techexistsoutsideofthebayarea
Not relevant to me 🙂
Cool. If you ever want a lower cost of life and better quality of life, just holler and we can hook you up.
On the other hand, if you like living in a shoebox and smelling the urine of the homeless, you’re good to go.
Hey Brent – Chill out dude. I was critiquing the product. There are some things that MS started doing right after Satya came on board, like embracing Linux and open source. I still love SQL Server, but you have to agree the pricing is out of control.
You wrote “crappy low paid contract/consulting jobs” – that’s not critiquing the product. Goodbye.