Tag Archive: database mirroring

SQL Server Denali AlwaysOn Rocks

The next version of SQL Server, code name Denali, brings huge improvements when it arrives in 2011. To put these changes into perspective, let’s take a trip down memory road first and look at the history of database mirroring.

SQL Server 2005 first introduced mirroring, although it wasn’t fully supported until a service pack. In many ways, mirroring beat the pants off SQL Server’s traditional high availability and disaster recovery methods. Log shipping, clustering, and replication were known for their difficulties in implementation and management. With a few mouse clicks, database administrators could set up a secondary server (aka mirror) to constantly apply the same transactions that were applied to the production server. In synchronous mode, both servers had to commit every transaction in order for it to commit, giving a whole new level of confidence that no transactions would be lost if the primary server suddenly died. In asynchronous mode, servers separated by hundreds or thousands of miles could be kept in sync with the secondary server being a matter of seconds or minutes behind – better than no standby server at all.

Preparing to Compress the Stream

Preparing to Compress the Stream

SQL Server 2008 improved mirroring by compressing the data stream, thereby lowering the bandwidth requirements between the mirroring partners.

In one of the most underrated features of all time, Microsoft even used mirroring to recover from storage corruption. When the primary server detected a corrupt page on disk, it asked the mirror for its copy of the page, and automatically repaired the damage without any DBA intervention whatsoever. Automatic page repair doesn’t get nearly the press it deserves, just silently working away in the background saving the DBA’s bacon.

Database Mirroring’s Drawbacks

While SQL Server was able to read the mirror’s copy of the data to accomplish page repairs, the rest of us weren’t given the ability to do anything helpful with the data. We couldn’t directly access the database. The best we could do is take a snapshot of that database and query the snapshot, but that snapshot was frozen in time – not terribly useful if we want to shed load from the production server. I wanted the ability to run read-only queries against the mirror for reporting purposes or for queries that could live with data a few minutes old. Some companies implemented a series of snapshots for end user access, but this was cumbersome to manage.

Definitely Not Using High Safety Mode

Definitely Not Using High Safety Mode

Unlike log shipping and replication, mirroring only allowed for two SQL Servers to be involved. We could either use mirroring for high availability inside the same datacenter, OR use it for disaster recovery with two servers in different datacenters, but not both. Due to this limitation, a common HA/DR scenario involved using a cluster for the production server (giving local high availability in the event of a server failure) combined with asynchronous mirroring to a remote site. This worked fairly well.

Fairly.

The next problem: database failovers are database-level events. DBAs can fail over one database from the principal to the secondary server, but can’t coordinate the failover of multiple databases simultaneously. In applications that required more than one database, this made automatic failover a non-option. We couldn’t risk letting SQL Server fail over just one database individually without failing over the rest as a group. Even if we tried to manage this manually, database mirroring sometimes still ran into problems when more than ten databases on the same server were mirrored.

Database mirroring didn’t protect objects outside of the database, such as SQL logins and agent jobs. SQL Server 2008 R2 introduced contained databases (DACs), a packaged set of objects that included everything necessary to support a given database application. I abhor DACs for a multitude of reasons, but if you were able to live with their drawbacks, you could more reliably fail over your entire application from datacenter to datacenter.

Enter AlwaysOn: New High Availability & Disaster Recovery

It’s like mirroring, but we get multiple mirrors for many more databases that we can fail over in groups, and we can shed load by querying the mirrors.

That might just be my favorite sentence that I’ve ever typed about a SQL Server feature.

I am the last guy to ever play Microsoft cheerleader – I routinely bash the bejeezus out of things like the DAC Packs, Access, and Windows Phone 7, so believe me when I say I’m genuinely excited about what’s going on here. I’m going to solve a lot of customer problems with mirroring 2.0, and it might be the one killer feature that drives Denali adoption. This is the part where I raise a big, big glass to the SQL Server product team. While I drink, check out the Denali HADR BooksOnline pages and read my thoughts about the specifics.

First off, we get up to four replicas – the artist formerly known as mirrors.

Denali also brings support for mirroring many more databases. We don’t have an exact number yet – we never really got one for 2005 either – but suffice it to say you can mirror more databases with confidence.

Preparing to Demo Availability Groups

Preparing to Demo Availability Groups

DBAs set up availability groups, each of which can have a number of databases. At failover time, we can fail over the entire availability group, thereby ensuring that multi-database applications are failed over correctly.

Denali’s HADRON improvements change my stance on virtualization replication. For the last year, I preferred virtualization replication over database mirroring because it was easier to implement, manage, and fail over. Virtualization still wins if you want to manage all your application failovers on a single pane of glass – it’s easy to manage failovers for SQL Server, Oracle, application servers, file servers, and so on. However, the secondary servers don’t help to shed any load – they’re only activated in the event of a disaster.

AlwaysOn Isn’t Perfect

I need to be honest here and tell you that Denali threw out the baby with the bathwater. There’s going to be a lot of outcry because some of our favorite things about database mirroring, like extremely easy setup, are gone. Take a deep breath and read through this calmly, because I think if you see the big picture, you’ll think we’ve got a much smarter toddler.

AlwaysOn relies on Windows clustering. I know, I know – clustering has a bad reputation because for nearly a decade, it was a cringe-inducing installation followed by validation headaches. Some of my least favorite DBA memories involve misbehaving cluster support calls with finger-pointing between the hardware vendor, SAN vendor, OS vendor, and application vendor. This is different, though, because clusters no longer require shared storage or identical hardware; we can build a cluster with a Dell server in Miami, an HP server in Houston, and a virtual server in New York City, then mirror between them. Now is the right time for AlwaysOn to depend on clustering, because the teething problems are over and clustering is ready for its close-up. (One caveat: clustering requires Windows Server Enterprise Edition, but Microsoft hasn’t officially announced how licensing will work when Denali comes out.)

When you’ve got a clustering/mirroring combo with multiple partners involved, you want to know who’s keeping up and who’s falling behind. You’ll also want to audit the configurations. There’s an improved Availability Group dashboard in SQL Server Management Studio, but I’d argue that GUIs aren’t the answer here. For once, brace yourself – I would actually recommend PowerShell. I’ve given PowerShell the thumbs-down for years, but now I’m going to learn it. It’ll make HADRON management and auditing easier.

Do not try what you're about to see at home. We're what you call consultants.

Do not try what you're about to see at home. We're what you call consultants.

Summing Up Denali AlwaysOn

There’s a lot of challenges here, but as a consultant, I love this feature. It’s a feature built into the product that gives me new ways to handle scalability, high availability, and disaster recovery. There’s a lot of potential in the box, but the clustering requirements are going to scare off many less-experienced users. Folks like us (and you, dear reader, are in the “us” group) are going to be able to parachute in, implement this without spending much money, and have amazing results.

Over the next few months, I’ll be taking you along with me as I dig more into this feature. I plan to implement it in labs at several of my customers right away, and I’ll keep you posted on what we find. If it’s anywhere near as good as it looks, I’m going to be raising a lot of glasses to Microsoft.

If not, I’ll be pointing Diet Coke bottles at Building 35 until they fix the bugs, because this feature could rock.

Brent Ozar

Brent specializes in performance tuning for SQL Server, VMware, and storage. He's one of the very few Microsoft Certified Masters of SQL Server, a published author, and a Microsoft MVP. He likes travel, Jeeps, Apple gear, jokes, and writing about himself in the third person. Read more and contact Brent.

Website - Twitter - Facebook - More Posts

Getting Started with Amazon EC2 SQL Server 2005

Sometimes you need an offsite database server in case something goes wrong, but you can’t afford a full-blown disaster recovery datacenter.  Or maybe you’ve got some ideas that you’d like to try out with a big SQL Server 2005 box, but you don’t have the hardware sitting around idle.  Or maybe you’d just like to learn SQL Server 2005 – sure, it’s not the latest and greatest, but it’s still the most popular version out in the wild.  Now you’ve got a way to accomplish this for around $1 per hour.

With Amazon EC2, DBAs can “rent” virtual servers running Windows 2003 and SQL Server 2005.  In this article, I’ll explain the five big steps required to turn on your own SQL Server in Amazon’s datacenters.

Step 1: Understand what “cheap” SQL Server hosting costs.

As of this writing (10/2008), here’s the smallest SQL Server you can get at Amazon EC2, what’s known as a “Standard Large” configuration:

  • 7.5 GB of memory
  • 4 EC2 compute units (2 virtual cores with
  • 850 GB of storage
  • 64-bit Windows 2003
  • SQL Server 2005 Standard
  • $1.10 per hour, or roughly $800 per month

Yes, that’s a lot of money, and no, that does not include your bandwidth.  You’ll also incur some additional fees for bandwidth – probably nowhere near $100/mo, but you can use the Amazon EC2 pricing calculator to estimate your numbers.

Let me put it this way: $9 for this much power for one day is a heck of a deal.  I can do a lot of learning and experimenting for my $9.  Even if I use it as a lab box one day per week, that’s still around $40/mo – not a bad deal, especially if I’m a junior DBA who wants a sandbox to break stuff.  At $800/mo, well, we’re in Jaguar XF territory.

Step 2: Get an Amazon EC2 hosting account.

Sign up for an account at aws.amazon.com.  It’s free to enroll, but you have to link it to a method of payment like your credit card or checking account.  Your server usage will be deducted automatically from your account, so don’t blame me if you turn on a whole virtual datacenter and you can’t pay your rent.

After you’ve got the account set up, enable your account for EC2 (the virtual server hosting part) and S3 (where you can store your hard drives):

Step 3: Get the Elasticfox plugin for Firefox to manage your virtual EC2 SQL Servers.

Elasticfox is a browser-based way to manage your virtual army.  It’s free, it’s open source, and it’s the easiest way to get started with EC2.  There’s a guide on getting started with Elasticfox, but I’ll give you the highlight reel:

  • Download the Elasticfox plugin and install it
  • Launch Firefox and click Tools, Elasticfox.
  • Click the Credentials button and input your Amazon Web Services access key and secret key.  Click OK.
  • Create a key pair (to encrypt your Windows admin login) by clicking the KeyPairs tab and click the button to create a new keypair.  Type in anything for the key name, and save the certificate file.
  • Build a set of firewall rules by clicking the Security Groups tab.  In theory, you could skip this step and leave all your ports open, but come on.  Click the Create Security Group button and create a group named SQL Servers.
  • Click the Grant Permission button to set up a firewall rule.  For testing purposes, you can leave the CIDR (source) IPs at 0.0.0.0/0, which means the entire internet.  For production, you would want to restrict this access to your company’s subnet.  In the Protocol Details, set up each of these:
    • TCP/IP 3389 – remote desktop
    • TCP/IP 1433 – SQL connections (if you want to connect to your cloud-based server using SSMS on your desktop)

Step 4: Start up a virtual EC2 SQL server.

Go back to the AMIs and Instances tab and your Elasticfox screen will look something like the below screenshot (you can click on it to enlarge).  I’ve resized my columns to make it easier to see the instances I want:

Elasticfox AMI List

Elasticfox AMI List

In the screenshot, there’s an edit box at the right side where I typed in “sql” to help filter down the list of instances.  Amazon has a ton of servers available, and you have to pay close attention to get the right one.  Here’s a zoomed screenshot, and the highlighted one is the one I’ll be using:

Readthe filename carefully: the “Anon” means it’s not using the extra-charge Windows Authentication Services, and the “v1.01″ is the latest version available as of this writing (10/2008).  Newer versions mean newer patches of Windows and SQL, so the newer the better.

Right-click on the instance you want and click Launch Instance.  The next screen is full of pitfalls.

The Instance Type must be m1.large or greater.  The default is probably going to be m1.small, but that won’t work.  If you try to launch a SQL Server with m1.small, you’ll get this error:

Amazon EC2 Launch Error

Amazon EC2 Launch Error

The error says:

“EC2 responded with an error for RunInstances
InvalidParameterValue: The requested instance type’s architecture (i386) does not match the architec…”

The virtual image for SQL Server is a 64-bit machine, and you have to launch it with an InstanceType of m1.large or larger.  This catches me all the dang time.

For the KeyPair dropdown, choose the certificate name you generated, and under Security Groups, move SQL Server over into the “Launch In” group.

Click Launch, and if all goes well, your instance will show up in the “Your Instances” list in the bottom of the screen.  It takes a minute for the server to boot, but the Elasticfox screen doesn’t update on its own – you have to push the Refresh button manually to see if the server’s available.

Step 5: Connect to your new virtual EC2 SQL Server.

When the server’s State shows “running”, right-click on it and click Get Administrator Password.  Elasticfox will ask for the key pair certificate file that we created earlier.  I’ve had problems with it not always recognizing the file, so just try again and it’ll probably work.  The administrator password will be saved to your clipboard.  Windows doesn’t always allow pasting into the password field, so you may need to bring up Notepad, paste the password in there, and then look at that Notepad screen while you’re logging in.

Click on the server and click the Connect button in Elasticfox.  Elasticfox starts the Remote Desktop client and directs it to the server’s public DNS name, which is going to be something completely forgettable.  Don’t worry – if you’re planning to use this server for disaster recovery, you can assign it a permanent IP address and a better DNS name, and there’s plenty of instructions for that in the Amazon documentation.

When you start SQL Server Management Studio, you’ll either have to put in (local) for the server name to connect to, or start the SQL Server Browser service.

Before you create databases, go into Windows Explorer and take a look at your hard drive configuration:

800 GB, here I come!

800 GB, here I come!

In this screenshot, I’ve got two local drives, D and E, each with 420 GB.  Cha-ching!

From here, the world is your oyster.  You could set up database mirroring, and use this as a disaster recovery server.  Be aware that SQL 2005′s database mirroring is not compressed, so your bandwidth charges may be higher.  Instead, I’d suggest doing log shipping.  The advantage to using log shipping is that you can compress it with Quest LiteSpeed, plus you don’t necessarily have to be running the SQL Server at all times.  You can copy the files to a cheap non-SQL box at Amazon, and only start up the SQL Server once per day (or per week!) to apply the log files.  (I see a blog post coming on that after PASS when things die down.)

Another great use: testing software.  The whole reason I wrote this article today was that I had to test a new beta of Quest Change Director, and I needed a quick new environment to test it in.

I’ve got an upcoming project where I’m working with a European client on a SharePoint whitepaper, and both of us work for secure public companies with paranoid IT departments.  Neither company wants to give VPN access to the others’ staff, so instead we can just build our lab in Amazon EC2 from scratch and both access it from anywhere on the planet.  Everybody wins.

Just don’t forget to shut the server down when you’re done with it, or else you’ll keep paying by the hour!

Update 10/30Stephen Moore asked about SQL Express, and yes, they do have AMIs prebuilt with SQL Express and they start as low as $.125 per hour.  That’s a steal if you only have small databases.

Brent Ozar

Brent specializes in performance tuning for SQL Server, VMware, and storage. He's one of the very few Microsoft Certified Masters of SQL Server, a published author, and a Microsoft MVP. He likes travel, Jeeps, Apple gear, jokes, and writing about himself in the third person. Read more and contact Brent.

Website - Twitter - Facebook - More Posts