A Gentle Reminder That Corruption Stinks



Oh. It's broken.

Oh. It’s broken.

Brent says: normally, we don’t post client screenshots on the blog because it’s hard to sanitize the sensitive data. This time, not so much. Storage took care of this for us.

Previous Post
Lessons Learned From Working in the Clouds
Next Post
[Video] Office Hours 2016/11/02 (With Transcriptions)

25 Comments. Leave new

  • Noah Engelberth
    November 4, 2016 8:33 am

    I see your client has found the new SQL Server burn-before-reading feature for truly sensitive data.

  • So, begging the question…what was root cause? Memory bit flipping, bad SSD, something else?

  • It’s actually a work of art. This should be put on a t-shirt or something. 🙂
    PS: Thankful it’s not *my* data….

  • Larry The Cable Guy
    November 4, 2016 11:16 am

    Not to be picky but who says this is corrupt and not just someone sanitizing the data. Did the DBCC actually show the error and if it did, which one. I assume it did or why the blog post but just checking.

  • My Blackjack cheat bot in action!
    Winner Winner, Chicken Dinner!

  • We went through this a couple weeks ago with a 1 TB critical production database. It was not a fun experience.

  • What can I learn from this post? What is the takeaway, besides “corruption is bad” (which I’m already aware of)?

  • What can you learn? Have good up to date backups and don’t play with DBCC Writepage! 🙂

  • I’d say the consistency check worked fine. This is the most consistent data page I’ve ever seen.

  • I think the ascii cross symbol adds a nice touch. Have you tried running DBCC DBEXORCISE? (“THE POWER OF CODD COMPELS YOU!”)

  • Lesson 1. Don’t be complacent
    Lesson 2 Check that you are CheckDB’ing
    Lesson 3. Now where did I put those backups – well at least make sure you can access them.

    Thanks Erik.

    • Thanks, Mal! I still see a lot of people running CHECKDB, but not alerting if it fails. That’s no fun!

      • Question: How to test that the alert actually works, given that in a “normal” situation there are no errors.

        • Download a corrupt database (or corrupt one yourself with a hex editor).

        • You have a couple options. You could download a hex editor and try to corrupt a database on your own, or you could head over here and download a sample corrupt database. If you follow along with the challenge, you can see a few different kinds of corruption that can strike, and how to fix them.

  • I would love to hear the generic recovery steps that were taken for this .. I presume it was restore from sound backups (if you knew which backups were sound).