Chris Shaw started a new blog quiz for SQL Server DBAs: name two mistakes you’ve made in your career.  I’m glad he restricted it to two entries, or else I’d have had to start a whole separate blog category just for this.

Mistake #1: deleting without a where clause. You’ve heard that horror story before, but here’s what makes it different: I was a teenager working as a SCO Unix sysadmin for a photographer with a small chain of photo studio.

What’s that, you ask?  How does a teenager get a job as a SCO Unix sysadmin?  Two reasons: first, I was cheap, and second, I was slightly more qualified than the last sysadmin, a high school cheerleader who was hired solely for her looks. (I know this because they both told me separately.)

Thankfully, I had a tape backup running every night, and the delete was literally the first thing I did in the morning when I got in.  The moment I realized what I’d done, I put the tape back in, started the restore, and began hoping that my early-bird boss decided to drive through a nail-laden construction site on his way into work.  No such luck, of course.

When he showed up (before the restore finished, of course), I promptly explained what I did and how I was recovering.  He was pissed, but I think he’d forgotten it by the next day.  That taught me an important lesson as a DBA: get coffee before you start work.  I’ve been addicted to espresso ever since.

Mistake #2: lifting servers to the top of the rack by myself. It was a dark and stormy Saturday night a couple of short years ago, and there I was, alone in the datacenter.  I was fairly new in the company, and I wanted to make a great impression on the other admins.  I’d carefully arranged an outage window during my on-call weekend to defrag some space in our racks so that I could cram more gear in.  I was almost done manhandling a bunch of heavy servers around, and my last opponent was an old server built long before the days of tool-free rails.  In order to get the server in, you had to hold it in midair, balance it so the rails lined up, and fasten it with six of the tiniest screws you’ve ever seen in your life.  Wristwatches are built with larger screws than this server.

By that time of the night, what little strength I had was already drained, and my awareness level wasn’t that high either.  With the server balanced precariously on my shoulder, aimed at the very top slot in the rack, I started putting in the microscopic screws with one free hand when disaster struck.

The server fell from 6′ up in the air down to the floor, diagonally onto a front corner so the whole server’s weight smashed down on one tiny little area.  It missed my foot by an inch, tops, and put a big hole in the datacenter’s recessed floor tiles.  Metal tiles, mind you.

I looked up to thank God that I still had my foot attached and I saw him looking back down on me – in the form of a surveillance camera.

When my heart stopped racing a few minutes later, I swapped out the broken datacenter tile, mounted the server in a lower slot and booted it up.  Worked the first time, although the faceplate was trashed and the server wouldn’t slide all the way into the rack because of the damage.  I locked everything up and headed out of there, glancing at the security cameras every minute or so.

The facilities manager was a good old boy who’d been with the company for decades and knew where all the bodies were buried, so to speak.  First thing on Monday morning when he came in, I took him into the datacenter, showed him the damaged floor tile and told him the story.  He gave me a stern warning about the relative worth of my job and my health, and asked how I would have handled a 911 call from the datacenter about blood all over the floor from a smashed foot.  He grinned when he said it, though, and he took the floor tile and never mentioned it again.  For all I knew, he was holding the floor tile as ransom, and that was fine with me.

That taught me a bunch of lessons: never rack an old server by yourself, bring a cell phone into the datacenter when you’re working alone, and know the guys who bury the bodies (and floor tiles).

Who I’m tagging: Chris already tagged Jason Massie of StatisticsIO.com, and I’ll tag Jeremiah Peschka of Facility9.com, David Stein of Made2Mentor.com (especially because his Vegas mistake story was so hilarious) and the masked man behind SQLBatman.com.