There’s a lot that goes into a Disaster Recovery plan. One of them is offsite backups. There are some businesses that don’t have backups going offsite, let alone a Disaster Recovery plan.
This conversation has taken place more than I thought it would.
Me: Do you have any HA or DR features in place?
Them: We have backups for HA, but that’s it.
Me: Do the backups go offsite?
Them: No.
Me: What happens if a major disaster hits and the data center is destroyed?
Them: <crickets>
Me: <stares into the webcam, slowly leaning in>
Them: Is there a cheap disaster plan that we could implement?
These aren’t our HA/DR clients. These are the ones that have called us to help tune their servers, but we also spend a few minutes discussing if and how they can achieve their RPO (data loss) and RTO (downtime) goals.
Is there a cheap disaster plan that we could implement?
What if you could store your backups in the cloud and then spinup the environment only when needed? Well you can!
GCE, AWS and Azure offer cloud storage. You use their tools to copy your files into a storage bucket. Their tools allow you to manage the retention, so you’re not storing everything all the time, unless that’s what you want to do.
All that you pay for until disaster strikes is the storage.
- Copy the backups to the storage bucket after the backup completes
- Lather, rinse, repeat
- Disaster strikes
- Create an instance in the cloud
- Install and configure SQL Server
- Restore the databases
- Copy backups to the storage bucket
- Disaster is over, time to failback to primary data center
- Restore from the storage bucket
- Copy backups to the storage bucket
Is this something you could quickly get online?
That’ll depend on if you’ve tested it and how complex your environment is. I can get the SQL Server online fairly quickly (depends on restores though), but what about everything else?
Remember that we are talking about a cheap disaster plan and your data center is either offline for a long time or for good. If we’ve got the data somewhere, we at least haven’t lost everything. Hopefully you’ve also made plans to store your source code and other dependent things offsite.
I do recommend that you test this scenario so that you know how to do it and how long it’ll take. You’d have to pay for the instance during your test, but it would be minimal as compared to paying for DR servers all the time.
Where can I get more information?
To learn more about this relatively affordable Disaster Recovery plan, check out the white paper that I wrote for Google. It’s in our First Responder Kit.
Brent says: And cloud storage is really cheap, like pennies per GB per month. I’ve been using Arq to back up my computers to multiple cloud vendors for years, and I don’t think my bill has ever approached $5 per month.
17 Comments. Leave new
It’s amazing how little thought companies give to their end to end disaster planning. This is a good post for folks to read on that.
The first couple of times I had this discussion with people, I about fell out of my chair. My last 3 jobs (totally around 15 years of employment) has been at medium to large companies. All 3 had dedicated DR sites. And we even failed over to those sites 1-2 times per year and ran production out of there for like 2-3 weeks at a time. Not only did they care about HA and DR, we practiced it and proved the solution worked.
One of my clients in theory has a DR site and plan. I’m responsible for the SQL side of things. It’s been a bit too long between actual failover tests, but I’m pretty confident about MY piece of it. I can’t speak for the non-SQL stuff though.
Cloud storage is cheap, but – as always – that’s not the whole picture. If Brent only pays $5 / Month, then that’s only ~200GB on S3 (2.3 cents/GB). My MP3 collection is 4x that.
A TB in S3 is ~$24/Month – and that doesn’t include egress cost (9 cents/GB), should you ever have to bring the data back on site.
Think about that from a company’s perspective. Even if they have several terabytes that they need to store, it’s relatively cheap per month. 5TB is $100/month <-- That's cheap.
I am thinking from a company’s perspective (we have clients where we manage this scenario). 5 TB is not a realistic scenario. Your average SMB organization will – over the course of time – go way beyond 50TB (~$1200/Mth) or more on backups. Add regulatory compliance to that (i.e. “keep forever”) and you’ll easily go beyond thousands of dollars per month – again, excluding egress costs.
What I’m tying to say is you’re absolutely making a valid point, I just think it’s a little bit more complex than “just use cloud storage”. For DR, for just a single SQL box, I’d probably go ASR.
But this is probably a discussion much better had at Karl Strauss or Ballast Point 😉
But remember this is for a cheap disaster plan for a company that has nothing in place. For these companies, they wouldn’t be storing forever (they’d have something else in place for that). They’d be storing just enough backups in the event of a disaster. So maybe 7 days of backups. I’d hope that any company with a massive amount of data, lots of servers, etc would have something more robust in place than just S3. The companies I’ve worked for have spent millions of dollars on disaster plans. This solution is not for them.
I run a database for a small non-profit. The database gets updated maybe once a month (and if I lose the data, it’s easy to reload the data). But even with that, it’s fairly simply to create an “off-site” copy simply be doing a backup and putting it to the cloud and making sure two other members of the organization have access to it.
It’s critical to have something! Not every plan has to be 5 9s of uptime with instant recovery. Scale to ones needs, but make sure you have something!
I am working on exactly this right now and would love to see your whitepaper, but I can’t find it on the link provided. I am may be missing something obvious, but could you provide a direct link?
Thank you for the great post!
I found it! https://www.brentozar.com/archive/2017/03/new-white-paper-build-sql-server-disaster-recovery-plan-google-compute-engine/
Thanks!
We’re currently getting our heads around this idea right now. But then we read the fine print of the Microsoft SQL Server licensing guide and we stopped dead in our tracks. For some inexplicable reason, MS requires you to license BOTH on-prem active server and PASSIVE cloud-based server for failover. (Wish I could link to the image but for anyone interested, it’s on page 27 of the 2017 Licensing Guide). Can anyone provide any insight as to why this is? This fact is currently preventing us digging any further into a cloud-based DR plan.
This was a change implemented as of SQL 2014, most likely to encourage people to pay for Software Assurance, as previously it was just included as part of your license. Brent wrote about it a bit here: https://www.brentozar.com/archive/2014/04/sql-server-2014-licensing-changes/
You don’t need to license the passive server separately as long as you have software assurance.
Technically, you can move your license to another server as long as you’re not planning to move it back within 90 days. Anything more frequent than that requires active software assurance, at least as I understand it.
Thanks for the reply, Paul. However, the figure in the SQL Licensing Guide (pg 27) shows that the Active Server requires licenses WITH SA *and* the Passive Server in the Cloud also requires the same number of licenses. The verbiage in the document states that the Failover server does not need to be separately licensed as long as it is truly passive and the Primary is covered with SA. Yet the figure completely contradicts that. Furthermore, the same figure is in both the 2016 and 2017 SQL Server Licensing Guide. I’m just looking for someone to provide some rationale behind the difference in the Cloud licensing requirement.
JP – in terms of the rationale, you would need to be talking to Microsoft about why they set their licensing rules.
Microsoft doesn’t patrol my blog comments for support questions. 😉
JP – Multi-Tenant environments have strict rules with regards to (Microsoft) licensing. As a customer, you cannot just copy your SQL ISO image to the cloud instance and install. Microsoft does not allow service providers to run customer-owned licenses on multi-tenant platforms.
For cloud environments, you have two options: 1. “rent” licensing from the cloud provider (i.e. select a SQL server VM) or 2. provide license mobility (the first option/diagram on page 28). With license mobility, you are using your SA to run a passive node.
Also keep in mind that not every Microsoft product has license mobility. For example there is no license mobility for Windows Server – you must license (read: pay for) Windows Server through the service provider.
Hence my original comment – it’s not quite as simple as “just a bit of cloud storage”
Thank You, Tara, for this article.
Thank You, Tara, for driving me here from Office Hours.
I had read this and lost track of it in the busyness.
Sorry I gave Brent credit.
Tara is the Boss!
Thanks for sharing !