Warm up your keyboard, it’s time for a contest.
The challenge: Write the creepiest sentence you can come up with. You get 10 words max.
An example to get you started:
Oh, I’m sure someone set up the backups.
The rules:
- Enter as often as you like (here on the blog, in the comments for this post)
- The contest will close on Thursday, June 4, 2015
The prize:
- Bragging rights. That’s sublime all by itself.
- One free license for Goat Simulator from Steam – we’ll send the gift certificate to the email address of your choice. ($9.99 value USD.) http://store.steampowered.com/app/265930/
Update…. the winner has been selected! We had tons of great entrants, but I had to pick a winner.
Honorable mentions:
5. “Microsoft has released details of new SQL licensing model.” … Dorsey
4. “I wrote a trigger to prevent duplicate inserts.” … Chris Bergin
3. “We just make everyone a sysadmin so it’s easier.” … John Nelson #2
2. “The server room is directly below the men’s room?” … ScalabilityDoug
And the winner of the bragging rights AND the coveted free license of Goat Simulator is:
1. “Please keep this pager on you at all times.” …Tim Cockerham
Tim, we’ve emailed you all the info to claim your prize.
1283 Comments. Leave new
The database got corrupted and we don’t have a backup.
We backed it up to Null so should be fine
T-log’s? Is that some sort of cereal?
Trust me, i’m MCDBA Certified.
“I’ve been doing it this way for 20 years, why should I change now?”
“Nooo? A column with NVARCHAR(MAX) data type in this index? No online rebuild?”
“Who set the fillfactor default to 50?”
So: your script sets sa to null in all servers
Came here to leave that exact message, and see you beat me to it, let alone with the first post. WINNER!
(And note that I’ve been responsible for this once upon a time.)
“Sure, I can provide you the sa password.”
SQL 2000 is good enough for what we’re doing.
What was that sp_whoisactive script for again.
Hey, what’s a DR plan?
“No worries, production server is definitely faster.”
Sure, here’s the SA password.
Backup concept – well, do you know how to build one?
So, you’re saying we were backing up to the SAN?
I couldn’t reach you, so I changed it myself.
Checkdb has been silently failing for weeks.
Dan, nice!
You’ve written T-SQL before so you’re the DBA now.
oh, I hear you.
Goodnight Paul, thanks for trying.
I love it!
hilarious!
Great! 🙂
Tell me it is just a bad dream :S
“An untrusted FK on the most frequently used table with 300 million datarows”
WINNER.
We’d like you to set up a merge replication.
I literally felt this one in my chest!
DBCC CHECKDB (mydb) executed by MYDOMAIN\theuser found 124521 errors.
Everybody has sysadmin rights here.
I worked for that company.
I feel like that’s a DBA rite of passage. “Deal with everyone having sa that really, really, really shouldn’t.”
You must have been in the cube next to me!
Oh, that was my brilliant sentence, you stole it ????
How do you rollback a TRUNCATE again?
it all runs on the domain controller
In spite of what many believe, TRUNCATE in SQL Server is fully transactional and can be easily rolled back.
Try it for yourself… CREATE TABLE Foo(Col1 INT); INSERT INTO Foo(Col1) VALUES (1); BEGIN TRANSACTION; TRUNCATE TABLE Foo; ROLLBACK TRANSACTION; SELECT * FROM Foo;
You’re seen a SQL 6.5 instance before haven’t you?
I think I still have the disks 🙂
It is easier to give the vendors the SA password.
we use sa as application login
All of our data is in an Access database.
Oh, sh*t, I’m on the PROD server.
jajajaja, this is the best phrase!!!!
+1…good stuff
+1 for that
Oh s$#@! Me too!!!
#Heartstops
+1!!!!
Been there done that!!
All DBA’s have done this at one point of another.
Yeah, that is a pretty awful feeling
“Why I cannot create another clustered index on this stupid table?”
The vendor says, disabling AUTOSHRINK violates their support agreement.
Ouch
I know, right? Didn’t think you could actually delete master.
We only support SIMPLE recovery.
We don’t use SA so we left the password blank.
I do not know, what is SA’s password!
Whoops…. well, that was unexpected.
DBA: I QUIT!
Sure, I can drop that contraint for you.
With our users, we shouldn’t worry about SQL injection
I removed BUILTIN\Administrators on Production and can’t login anymore.
your application needs global-admin and SA to run… no problemo
CIO (CEO’s Nephew): Move to the NoSQL Cloud because synergy.
We don’t have time to run DBCC CHECKDB.
“Let’s ask Brent what he would do”
HAHAHA
Funny!
“Brent said it was ok.”
Probably he would say “Depends”.
Coworker asked Brent about something (AO on Windows 2008, I think), the response was “you can pay me 20k to get it working, or just use 2012”
Lol he is right tho
What’s wrong with backing up to the same SAN that holds the database?
The developer needs db_owner rights in production.
Hired in as SQL Server DBA:
“We forgot to tell you that you’ll manage the mainframe.”
I’ve actually had that (well, a Vax not a mainframe but same diff)
Clustered GUID primary key? No problem!
Backup log dbname with truncate_only ohhhh scary!
Weird! I developed and tested this in production last night!
I deleted all those huge ldf and bak files.
Just write “Drop Database [DBNAME]” and press F5
(True Story… Senior Dev joking to the Jr Dev… the Jr Dev executed the query… Was 1 week full of nightmares… And no backup to restore…)
Serious? That’s scary!
I did hear someone joke to a user “try DROP ALL, F5”
We’ve decided you are now part of the accounting department.
Um, I live that every day. The IT data droid “sold” to the finance group…
No need to test, just put it right in production.
“I don’t always test, but when I do, I test in production.”
We can test in production.
Runs fine in dev, can you test live?
“We run dbcc checkdb repair_allow_data_loss on the regular”
We need 100% uptime. We have no budget.
The developers converted all the databases to MongoDB last night.
LOL… that is a very real possibility here
We’re all sa here
“Yeah, right: SSMS…, where do I undo the ‘Execute(F5)’ again…?”
Where is that foreachdb/foreachtable maintenance script again??
The auditors found all our users credit card information.
No really, I read this trick on Brent’ site…
Just offline/online the database, it should solve our deadlocks….
**cries a little, actually happened to me today…**
Lawsuit names you personally responsible for the 1,000 enterprise licenses.
Even if it blows my chance of winning? This would get my vote. YIKES.
Oh my god! DBA executed delete statement without where clause in production database and there were no proper database backups.
Ten words Kris, ten words.
I’ve got a new one “I should really stop skim reading…”
I fixed our space issue by deleting the transaction log.
I hate to say it but I got that call one time at my last job…
There’s no budget, just put TempDB on the C: drive!
And this job flushes the cache nightly to free memory.
The bug deletes random rows. Nothing important yet. No Worries.
Oops, more than one sentence. I’ll just make those semicolons… yeah…
The bug deletes random rows; nothing important yet; no worries.
Vendor is aware of problem, fixing in a future patch.
Then he asked me to destroy the backups, your honour.
Oh, my…
Nice!
We think you can really help the DBA in Yakutsk.
Trust me, I’m SQL certified.
Yes, it’s unencrypted. Yes it’s publicly available. Problem?
Do you have a copy of the Database Master Key?
CEO’s son used the production data for his school project.
CHECKDB has errors, does anyone have Paul Randal’s number?
I called Paul Randal and he told me there’s nothing that he can do.
Here, sysadmin rights are only granted to people wearing suits.
No, there aren’t any backups, you told me all the data was synced from the OLTP system every night!
We decided to duplicate every row rather than run backups.
Cluster nodes must be in sync, so reboot them simultaneously.
Latest update keeps copy of entire database on user machines
(true story! — was supposed to help concurrency)
Yes i’m the SQL DBA at FIFA..
Gooooooooaaaaaaallllllll!
Login failed for user ‘Fifa/sblatter’.[CLIENT:11.11.1.11]
We’ve been doing log shipping, so we’re safe from disaster.
You don’t need internet access to manage database servers.
You don’t need a budget with open source!
Oops, I did an update and forgot the “where” clause.
We figured the server closet was your office.
I swapped ALL the hot-swappable drives last night.
Oh god! Hahaha either you have a scary imagination or you need a new job!
Jeremiah Peschka favorited a tweet you were mentioned in
Sorry, couldn’t help it 🙂
I can’t stop laughing at this one.
It’s no “If you love a tx set it free…”
Database backups? We have RAID5.
We figured data corruption is normal maintenance, like Access
We need to cluster the 2000 instance
What were those scripts you had me run last night?
We accidentally installed developer edition in Production
“We’re giving the developers SA on all the production servers”.
The internal production server has been accessible from the internet.
Brent Ozar and Paul Randal have deleted their blogs
This one has my vote (which doesn’t count, but for what it’s worth…)
…and SQL Server Central is offline too!
You said the vendor requires SA for their app to run or they won’t support it?
All reports must update real-time with the latest data.
Fighting this battle now! 🙁
Restore? You know we’ve never tested that…
“That” developer: “Hello, how’s your day so far?”
All Production SA passwords are documented in the (public) wiki…
I have two words that’ll ruin your day: catastrophic failure.
What’s the worst that could happen?
All your data are belong to us.
Bump for upvote!
Darn it! I was going to use that! Like!
“Hey, we’dlike to link some tables froms Access so our CRM team can run reports during business hours, is that ok for you?”
Oops, 10 words rule. Rephrased, still scary :
“Hey we’d like to link some tables from Access?”
We’ve decided to use an ORM to develop code faster.
We’ve decided to use an ORM to develop faster code…
Ups!!!
We moved Bob from lead developer to SAN administrator.
Your I/O is not only taking longer than 15 seconds some of the time, it’s taking longer than 1 second most of the time- you either need to spend 50k to fix this or move your 5TB database to a diffetent server.
What do you mean Test server?
We haven’t done anything and since then it is not working anymore.
Autoshrink works great.
Oops..
Always use nolock ’cause it makes things fast
What do you mean by Database Security? (spoken by IT Manager)
We don’t do best practices here!
Why can’t we use Access to right SQL Reports?
Why should we install that SP, it is running great!
We want to use AlwaysOn Availability Group
LOL!
🙂
Why is your DBA colleague running?
“I found our HR database on a Russian torrent site”
Please keep this pager on you at all times.
Oh, nice one. This is, indeed, terrifying.
Someone just flattened the SAN.
That code was released to production yesterday.
But the CIO said I could have sysadmin access!
This is hard, let’s use cursors.
Got my vote!
Test has been pointed at production since the last upgrade.
“In our design philosophy the DB’s just a bit bucket”
I love this one- I heard this exact thing once a long time ago in a job interview. I had a really hard time figuring out if they were serious or if it was a trick question and they wanted me to argue.
I’d assume the latter and argue. If they were serious then they can hire you to fix their issue or you don’t really want to be working there.
Cannot rollback no active transaction found
The backups haven’t been running
I thought the application is deadlock aware.
we use one sql account with sysadmin rights in conn strings
We support multiple active datacenters with peer to peer replication.
The Vendor requires Max Degree of Parallelism set to one.
SharePoint!!!
Dynamics AX!
Biztalk!
happened today:
“And then the application server does the 90 updates one by one in the database.”
Don’t worry, contractors’ installed the server, I’m sure it’s fine…
How about “Our consultants installed it, there’s nothing wrong!!!”
My multi cursor query is slow, fix it
Our plan is commvault backups for DR
Your manager comes up behind you and says: “Sooooo, I was in production, and…we have backups, right?”
What do you mean by “Test Database”?
Linked servers to Access files on 10 Mbps NAS shares.
The DBA doesn’t need to worry about SAN performance.
Why is Wednesday’s differential so much smaller than Tuesday’s?
My multi cursor stored proc is slow, fix it!
My nested cursor stored proc is slow…
The DBA is on vacation and cannot be reached.
[Scary to everyone except said DBA]
what do you think of AWS ?
This table has 2 columns and ID and xml.
I think you just triggered a flashback– I found myself huddled in a corner, rocking back and forth.
Could be worse .. could have no ID field 😉
Could be even worse… we have a table like that 🙁
We had a vendor – self taught – knew EVERYTHING. Decided to build a two column table. One called ID, the other called data. Data was VARCHAR (7936). It utilised six bespoke nested delimiters (weird ASCII characters). And by the time the CEO wanted us to write reports out of it, it had 6,000,000 rows…
Ah yes, the ol’ Entity-Attribute-Value design’s ugly cousin…
No, we’ve never needed to backup Master before
Kendra bought a new mattress
A empty table is a fast table so we truncate often.
We need to encrypt the user’s credentials?
All our backups are done to this one tape.
Truncate the transaction log – VMWare snapshot recovery is our strategy
Before you go home, merge replication’s giving some funny errors
Why my SSMS connected to production?!?!?!
The data is insecure and unavailable considering the available resources.
(from a dev or qa)
We take backups right?
“The emergency-mode repair failed. You must restore from backup.”
Has anyone looked into the ‘Server is down’ emails?
All queries MUST use a cursor.
oh, we use the same instance for staging and production
why not skip dbcc checkdb and just repair it?
Maybe I should’ve added a WHERE clause to the DELETE.
Sorry, we ran out of coffee.
Winner. Right here.
yes!
Code it however you want, CPUs and Disk are cheap.
Hey Developer – can you help me please [insert *every* Dba responsibility here] – I don’t know how to do that.
“Backup what key? The work order said enable TDE!”
That optional WHERE clause is a major design flaw
I accidentally restored over production instead of development.
The last two quit because of the on-call schedule.
HAAAAAAAAAA +1
I think Ryan was one of the last two.
I thought the corruption error message was just informational.
System’s says everything’s fine, so what’s wrong with SQL Server
LAN Admin to DBA, Hey we just lost two drives in the Production server, Did you have backup. Huh!
This was my last day, and did you know production’s down?
Hold on, I thought YOU wrote down the sa password?
Backups were taking too long, so I stopped doing them.
It’s only data
Did I just drop that table in production?
I didn’t think running the script would actually DO that.
(Happened to me yesterday)
They keep the bottles of DBA tears in here.
We don’t have money in the budget for hardware. Lets load test against the production instance.
Email from a co-worker – “Where do we keep the sys database backups for production?”
Microsoft called, they would like to do an audit.
+1
+10
We fired the DBA and we need the SA password.
We don’t know how to patch SQL 2005
There was a torn page so I deleted the .mdf.
You what?
Load test on Monday. Production Go Live on Tuesday….mmmmkkk …. terrific.
The backup is on this floppy.
Just restart the MSSQLSERVER service, should be good to go.
#$@! Is the replication peer for test set for production?
We’re migrating all databases to Oracle.
SQL is SQL, right? What do you mean, “It isn’t”?!
Those deleted row messages aren’t real, right?
Ooops! Damn. Damn. Damn.
*DEV TALKING*
High severity errors are negligible right?
1) just tick the checklist box, the backups are always fine.
2) run it now and cancel if it passes 28 seconds.
3) Is max-memory in GB?, Ive set production at 128.
4) You forgot to highlight the where clause on that statement
4a) Why are all of our customers called Mrs Griffiths?
I have done 4/4a. Luckily just on a report server, but I write update/delete statements within comment blocks now so I can’t be that dumb again.
We always perform our testing in production.
The web guys had us open port 1433
Power went out, SAN UPS failed – BLEEP!
“I forgot the WHERE clause in my UPDATE/DELETE statement.”
Hey, did you see our database was just mentioned in Brian Krebs’ blog?
+1
Here’s the SA password, here’s your server list. I quit.
We don’t need a DBA, the software is self-aware.
Please file a Service Request.
GRANT CONTROL SERVER TO public
Don’t touch that server, it might not come back up
I worked at this company.
“The most important transactional process we have is on a SQL 2000 box. Don’t ever touch that box. Unless something is broken. Then restart SQL Services on it to clear the error.”
“Boss, i granted the company YOU found online full access.”
Oops. Maybe better this way
““Boss, i granted the consultant YOU found online full access.”
You’re now the sole DBA managing 50 Prod servers.
Should I be concerned that I AM this DBA?
Probably….. 😉
Only 50?
We can’t be out of memory. We still have vRam.
But it was only a store procedure with 2 or 3 “delete” sentences without a “where” clause, how bad could it be?
Begin Tran Update ImportantTable. Go to lunch.
Management wants to know why our customer’s CCs are Google-able.
We thought we should store everything in flat files.
Ugh. I know at least one University department that thinks a flat file is a database,
Just put the SharePoint database on the OLTP server.
WINDOWS cluster is working fine while SQLSERVER is down..
Identity Keys is useless for this table, don’t create it.
We always test queries in production, for accurate performance profiling.
Evaluation period is about to expire on the production server.
+5
By the way we just bought SharePoint. You’re Admin now.
Our DBA Manager was mentioned many times on TheDailyWTF.com .
You support Nosql, right?
checkdb not needed, if there is corruption, we will know!
Oh… Well that is not good.
{Dev} Ummm, I think I just ran an update against production
You’ve been chosen as the speaker for the SQL Conference!
Ha!
What do you mean by, “Did you test the backup?”
I thought YOU were on-call
I have heard this sentence. And I was terrified.
Hmmm, my Availability Group Synchronization Health on INST1 is Critical.
CIO: “Drop the database and step away from the backup… “
So, I have this Excel spreadsheet…
Ha! +1
“We take backups every night, but haven’t ever tested them…”
I know it’s mild cheating to use a contraction, but hopefully it still counts 🙂
This is from an actual coding standards doc:
“All table reads need to use the hint “WITH (NOLOCK)”. “
That is terrifying indeed!
“We, uh, haven’t actually been archiving your backup files. Sorry.” *
* Actual quote from IT
OH yeah. Had that one recently: “You know how we said we always copy your backups? Well, we haven’t the last 3 months. We’re working on it”
What? The sa password is embedded in the application code?
A classic one….
when vendor executed sqlcmd to run a truncate sql script
What do you mean, “you really shouldn’t clean the keyboard while mgmt studio is open”…?
We’ll test the new app patch in production, for sure.
Microsoft just released a new service pack for SQL Server.
(Sorry, but after the last release, I had to.)
Oooooooooo, burn.
What do you do around here anyway? It’s just data.
What’s with all those the red icons in Cluster Admin?
Seriously, can I just delete those?
:-)) Any time, Kendra!
DELETE FROM TABLE
–WHERE date < '2014-01-01'
“Winner, Brent Ozar’s Scariest DBA Sentence Award, 2014” on resume
The Trigger is using Cursors
The trigger sends e-mails.
–I’ve seen it in the wild, ugh.
You got to be kidding me!
Cursor-using triggers that call a sproc that uses cursors. 🙁
I once had to write a trigger that used the OLE xp_’s to call a component (which I also wrote) that used a vendor API to charge the credit card in the inserted row.
Don’t worry, we have backups… Oh $%#$%!, they’re corrupted!
What do you mean we’re out of disk space?
UPDATE ImportantTable SET MostImportantField = NULL
–WHERE criteria IN (‘Commented’, ‘Out’)
The C# Sql Proxy UserID was granted DB Owner permissions
Our front-end code does all the validation.
This one happened for real, I was the one doing the ask:
“Where are our lifeboat VMs? Not responding to ping/mstsc..”
Turns out they were deleted by an admin without any prior notification :D.
I just told Paul Randal our problem, he’s still laughing.
Only executes have administrator access because they are administrators.
My query has an estimated subtree cost of 7249…is that bad?
I think the new intern can setup TDE in production.
I don’t know, it works fine on my dev box!
D@mmit Jim, I am a doctor, not a DBA!
We back up to C; that’s where there’s room.
Do you happen to have a copy of the DEK?
Oh he doesn’t have the permissions? just make him a sysadmin
Ha. Or “We gave sysadmin to ‘Everyone’ so we don’t have to worry about permissions”
“Transaction log full”? Oh, just reboot. That’ll fix it.
“Brent Ozar Ultd is closing shop and moving to Fiji.”
Are the new servers ready? Our SAN admin just quit.
Sorry didn’t know my code would cause 350% data growth
Oy…
why did I click “Notify me of followup comments to this thread via email.”?!
Goodbye inbox
The drive was full, so I deleted that .ldf thing.
So….how important was that where clause in the delete statement you asked me to run?
Well, the CEO is just used to using Access and needs a little help.
No-One can connect to the accounting db, do you know where the server is….Check under Bob’s desk.
The password is the same as the user name.
The database seems “slow” today…
I added “.\EVERYONE” to SA. That fixed it!
Disk full on SQL production server – nobody can work.
Sub #1:
Just do it in Production.
Sub #2
Just add a bit column for all 200 units.
We don’t need a schema design. Let’s use AOM instead. (i.e. Adaptive Object Modeling)
drop database hiden in script
Foreign keys slow things down, so we took them out..
“Brent who?”
The SA password? Yes, it’s ‘SA’
I need two new databases, call them: ‘offline’ and ‘suspect’
We had deadlocks, so we added NO LOCK to everything.
Here’s a screenshot of the long running query.
‘So your backup tool didn’t copy open files?”
But most scary to me: “Yes, I’m sure…”
OH: Darn, I forgot the WHERE statement again.
Thought that the tape admins are backing up that folder?
I wasn’t expecting Rows Affected to be quite so high….
I’m backing up SQL! I robocopy the entire drive nightly!
I’m supposed to test the backups regularly?
We store the images in the database.
Why does this say “in recovery”, we didn’t restore this?
This actually happened to me yesterday
This is dev. Production is the other query window. Wait….
Hey, there’s nothing but red X’s in this backup history.
What ,”you can’t restore just a table from Backup “
I just texted you my password, didn’t you get it?
This is one I wish I thought of
I can trust Skynet and Azure with my data – right ?
I hope you haven’t made plans for the next 6 weekends.
Let’s just put all of our business logic in triggers.
It’s just a reporting server, so I develop there.
SAN Admin: We don’t need backups, we have snapshots every 15 minutes.
Kendra wants to use our database for scariest database blog.
Oh, did I do that delete in production?
I freed up disk space by deleting log files – ok ?
Don’t worry, no one would ever do that.
LOL!
This table holds the javascripts we use.
Due to deadlock issues we reboot this box daily.
Is 111 indexes on a single table a bad idea?
I always create 42 indexes because that’s a lucky number.
And The Answer.
When restoring to Dev; user calls, says production off-line! OOPS!
ROLLBACK!! We did not receive any e-mail about this migration.
Oops, I accidentally switched off the SAN.
What backups?
We implemented everything using Multi Statement Table Valued Functions
We’re using Compatibility Level 90 on our SQL 2014 Server.
Latin1_General_CS_AS_WS
We have a winner
Gets my vote.
That production database server also serves as our corporate fileshare.
Severity 21 Alert message says the Log drive is gone!
The previous DBA said multiple log files helps performance.
The FBI wants a data audit…today
XP_CMDSHELL and SA password is required for our application.
“We’re using the free version of VMWare.”
You have HOW MANY nested triggers?!?!
Transaction log backup? Aren’t logs junk files?
The new vendor app needs sysadmin rights.
I only run checkdb on a restored copy.
The DAC is enabled, right?
This totally didn’t happen to me yesterday in my lab.
DBA: “Who set up the sql server cluster?”
SAN Admin: “Me.”
On-call in this company? One or two calls per week.
lol I’ve heard this before…
So we want to switch to a NoSQL solution…
It was running too long; I disabled it months ago.
The developers built an ORM that doesn’t use stored procedures
Im sure we have backups of the prod server..
Define ‘Database is missing’
I was told that some random data loss is okay.
You scripted that, right?
All our primary keys are guids
I have had this discussion too many times. 🙂
My condolences 🙂
Oh, you work for the MS Dynamics CRM team?
I was waiting for the first Dynamics joke. Nicely done.
Application uses SQL Express with a database for each client
Entity Framework means we don’t have to tune queries
We don’t need backups, everything is in the cloud.
Wait.. where did those databases on production come from?
What do you mean “the SAN fell off the truck”?
This rollback is taking forever. Let’s reboot the box!!
The project manager installed it. DBA time is nonrebillable.
What do you mean that’s the UA server, not production?
We’ve added you to 24/7 retail support on call list.
My DBA wants to be a goat
We processed over one million records in ONLY ten days!
Oops, I think I shut it down myself, wrong script!
What do you mean “the data is gone”?
What do you mean I’m the only DBA left here?
We installed another application on the database server
Don’t worry! Our DBA can fix it!
I fixed it myself. I’m a Cisco A+, you know.
The app will take care of referential integrity.
This one stands out
You know, win or lose, I would very much like someone to tell me that I’m terrifying. 😉
You ARE terrifying! ????
You are terrifying! 🙂
Woohoo! Thank you! 😀
This new database was not added to backup “Maintenance Plan” !!
I saw performance issues, so I just started killing SPIDS….
The auditors have some new requirements.
We try to use cursors for all of our loads.
Holy cow, that was unexpected!!!
EXEC sp_AskBrent;
(0 row(s) affected)
I’m doomed.
Common wisdom is to leave SA blank. They’ll get in anyway.
Can you fix my Excel database?
I thought Full recovery meant we didn’t need backups?
LOL. Good one.
I really wish that line was made up. 2 TB log for a db that hadn’t been backed up in almost a year.
I cried.
SAN Admin said:
“Looks like same disk was assigned to two Database Servers”
(True story!)
Both servers of this SQL cluster are in the same rack
“Sorry, I forgot the backup step and now…”
We don’t really need a DBA for SQL Server.
Guy at front desk says he is from the FBI
We can’t find your database backup files on tape.
We store the PDFs in the database.
There are ~400k Access databases on our network.
*yes this really happened to me. They wanted to convert them to SQL.
We don’t need foreign keys on our tables – it’s all handled within the application by the developers.
Honey, I shrunk the databases 😛
We needed space so I did an “erase *.bak”
I stopped the job because it said “DEADLOCK” somewhere
“You’re smart, I know you can fix what just happened.”
Stored procedure code changes should be tested directly against production
What’s the SA password?
bReNtOzAr
We don’t need foreign keys on our tables, it’s all handled within the application by the developers.
Backup ? What backup ?
Re-write my endless correlated sub-query so it doesn’t cause blocking?
You DO know you’re connected to production, right?
The backup tapes are in the cabinet under the sink.
Oh, the primary db has been suspect for awhile now
Relation database design is irrelevant.
Application team asked me to give rights for user to access db…to save time, I gave DBO rights…
Last backup is month old. Is that OK ?
How about a scheduled job for shrinking files nightly.
Don’t be shy… drop it
SQL Server isn’t a real database. We use Oracle.
I think we’re experiencing a SQL injection attack
We don’t have anywhere to test a backup.
Why are all the production tables empty?
Sure, put Biztalk with all the other production databases.
There is a new ransomware that targets SQL Servers.
What do you mean that job hasn’t run in over two years?
As a specialist and an eminent figure certify that something is fucked up with your server
The application needs sysadmin-rights during installation to create it’s database.
our app doesn’t work unless users connect using SA
Hypothetically, would we know if we had a data breach?
What do you mean sp_Blitz returned 40K rows?
I kept forgetting the password so I coded the website to return the username and password in the login failure error message.
DEV said they need sysadmin- the app requires it apparently….
Wow! Didn’t see that coming!
Since TFS is basically a database product, the DBA team gets to administer it.
But the exam training kit example enabled autoshrink on model!
Someone just executes an ssis package by mistake.
That database server is under someone’s desk
From consulting company…
“It won’t hurt. We Promise.”
haha
Our production server is 2005 and our development server is 2008R2.
BrentOzar.com is down…..
Don’t worry, there is only 1 script with eighty-coughThousandcough lines to check for code review.
Uh oh….Did you say delete or detach?
First day on the job at major online retailer…
“What happened to those orders we asked you to drop?”
“I dropped them.”
“They’re not here any more.”
“I know. You told me to delete them.”
“NO. In retail, DROP AN ORDER MEANS “SET IT TO READY TO SHIP.””
“Oh. Huh.”
Always know your jargon… AND the jargon of your customers.
Can we get that dropped table from the recycle bin?
Too big? Just delete it. It’ll automatically create a NEW ldf, right?
After a single statement:
(10172 rows affected)
(10172 rows affected)
🙂
nice trigger…
That database runs on the Windows XP server.
Yes, you are presenting to a packed room at the PASS Summit in your underwear, and no, this is not a dream.
our app is too small to justify many security precautions
We’re moving the DBA team under Development.
End user: “Who shut down the internet server?”
nobody changed anything in the application
The CTO has decided that we can save money on SQL Server licensing by migrating to Access.
We just delete the data if the numbers look off.
I need the sa password because that’s what the application is hardcoded to use.
The systems team is already backing up the servers, so we’re going to let them manage SQL Server backups too.
With this new unstructured data storage running on RAID 5, we’ve decided we don’t even need a DBA any more.
Why we should backup database?
Will test it first, (in production). This is not reproducible.
No need to include DBA in design meetings
nice, heard this a few times…
Crash – backups failed, SAN snapshot corrupt, replica lagging, DR unreachable.
we put recompile on all stored procedures
SharePoint. nuff said
The production server is under my desk.
Script executed successfully. DBA: Am I on the production server ?!!!
Just install Enterprise Edition. We don’t worry about licenses.
Hey, far be it from us to tell the company how it spends its money if we advise them on cost.
I think I’d be a little bit giddy if my company said that to me.
“It’s on the C: drive.”
Can’t we just create an index across all 32 columns?
Ooh, that’s a good one! Even worse, “It won’t let me index all the columns I need.”
Thanksgiving will give you 4 days to do your upgrade
Backups are on network drive (yes!! from the same SAN)
Buy more RDP licenses!!! Devs need to RDP for SSMS
That column holds the sql scripts we execute.
Why is the server so slow? What’s a cross join?
… or What is a where clause?
What’s a number you can ALWAYS be reached at?
We can make that Password column as Primary key! Just don’t tell the DBA
We use our employees ssn as the primary key
We have the database server on the outside of the firewall.
What do you mean by, “There is no development database”?
You’ve been promoted to management.
*twitch*
Do I need a where clause in a Delete statement
We don’t backup our log, isn’t that our audit trail?
We can’t encrypt Social Security Number, it’s the primary key.
+10
What’s the RTO time at your company?
“You usually start with 2 weeks, and 5 sick days.”
Developers given sysadmin rights on production
What do you mean “it must be a memory leak”?
Migration will begin one day before holiday.
Have a delete statement and select “Delete From Table” to change it for a select statement, and press F5 unconsciously.
Disk space was low so I dropped all indexes.
Same time 🙂
I do development work in production.
O rly? You gone learn today!
nobody changed anything in the application.
We don’t need SQL Server backups, the SAN does snapshots of the data files.
Take the website down, we’re resnapping replication!
I Dropped the Indexes.. we Were Running Out Of Space!
We run Profiler and PerfMon on all DB Servers.
I clicked shutdown instead of restart.
Oh … uh, I guess that the drives aren’t hot swapable …
True story, word for word, referring to a critical database during a live deployment, spoken absent-mindedly by the DBA as he concentrated on his screen:
“It is not as much there as it should be.”
We’d better mirror AdventureWorks.
The C drive is big enough to store all database files
I don’t know how it works, but watch this!
It shouldn’t affect anything
You need a backup of the database? Hmm maybe call the NSA.
We will be implementing ORM across the enterprise.
Our application doesn’t support foreign keys…
Just add with(nolock) to every table selected
We want all the data kept forever!
…..and we want it to always want sub-second response times on our queries.
What do you mean “did I license the SQL Servers”?
Why is the transaction log 147GB?
I did that because Brent has taught on his blog
You’ve clearly been watching this video: https://www.brentozar.com/archive/2014/12/brents-bad-idea-jeans-video/
🙂
What do you mean, no backups?
Guys, where is the firewall?
Here’s an older one … SQL Server 7.0 goes down, and we are standing in the server room after having tried everything possible to connect to the instance from application, Enterprise Manager, etc. This is the statement from the VP of development at the time …
“What do you mean MSSQL and Sybase can’t be installed on the same machine!!??”
Sure, I’ll give the new hire SA access to the shared database server!
IT: The server rebooted and CHKDSK is running…
(DBA Wanna-be):well its right 99.9% of the time-its only 1% loss of data
Real DBA: WHAT!! Ever hear of Data Integrity?
Do I smell smoke coming from the data center?
On a web connected server: “Who put EVERYONE in the sysadmin role”? (unfortunately, this actually happened)
Its a pool of disks! We manage the hot spots.
The power went out and the data center UPS failed
Power up the data center at the main circuit breaker
The CEO wants to know who can give permission to query the tables directly
Alternate
So you think it’s a bad idea to develop against the same database on the production server and the development server at the same time?
I need you to denormalize all the OLTP tables
You mean the query connects to the DB Instance separately from the DB Explorer?
I think we’ve experienced most of these creepiest DBA statements 🙁
Yeah, me too.
We have 536 currently used Access databases in the wild.
After a data center outage caused by a faulty HVAC …
“Yeah, nobody checked the backup battery in the fail-over unit.”
If ifs and thens were cases and whens… you’re fired.
System is slow; it should be database.
Why can’t SSMS connect Excel to production? TechNet shows how!
It’s SOX time…we audit right?
What’s wrong in deploying the script to the production database manually ?
Production DB Got Corrupted….Last backup also failed
That query had too many joins, I created temp tables.
What! Wait! No, the restore would have taken longer than 38 seconds!
Give the users that security. They know what they need!
It’ll work, I googled it!
Producion Server is slow. Please restart.
No, no offsite backups. We’ve never had a disaster before.
Hey, Always On should solve all of our DR issues!
What did the Developer say to the DBA?
TLDR.
Delete query looks good, hit F5…Oh no, Undelete, undelete….
… and that was an sp_updateResume event.
Storage is mirrored, so don’t worry about your changes.
DEV: You can use linked servers..
DBA: What’s that ?
(this one is real unfortunately)
No backups needed, we have RAID0.
Server down, why do I need backups.
Vendor Meeting @ 3pm: Our software requires sysadmin rights.
Vendor Meeting @ 4pm: Our software requires sysadmin rights.
Vendor Meeting @ 5pm: Our software requires sysadmin rights.
Vendor Meeting @ 6pm: Our software requires sysadmin rights.
…
We’re the vendor, we know the best SQL setup / security.
The hardware technology has finally caught up with our advanced application.
Translation: The new servers are finally powerful enough to offset the crappy, inefficient architecture of our application.
OMG!!! That machine under the desk which crashed was a production box.
So you’re saying backups were in the burnt server room.
“Yes, production backups are stored on the dev box”
DR? We don’t need no stinkin’ DR!!!
Your UDF calls a UDF which calls a UDF which……
dev: put that data in one table – joins are too hard
Didn’t we delete that login yesterday?
Umm, what do you mean there’s no rollback plan?
I noticed our Replicated Subscription articles haven’t been filling with new data for awhile….
manager: just cluge it for now, we’ll fix it later
you’re thinking of this as a relational db but it’s not
“I didn’t see the ‘commit\rollback’ at the bottom
Ok, now check ‘Boost SQL Server priority’…
It’s a virtual machine, we don’t need to perform backups.
Indexes? Do we really need those?
You have 62 indexes on WHICH table?
All the tables, just to be safe.
Me: So, why is this synonym created and discarded at runtime?
Me investigating a peculiar issue : This vwLamas view does exist…as a table.
Me…again investigating a peculiar issue: This tblLama does exist…as a view.
Developer: So, I used float to store the decimal values.
I accidently posted our database on the internet
Developer truncated a production table and the tapes are bad.
Queries are slowing down again, need another index on the table.
We can always call Microsoft support
C:\ was full so I deleted this LDF file.
That database can’t be that important (or complex), the mdf file is only 1.25GB.
-Are my SQL databases in your backup jobs?
–What’s SQL?
If I cancel a delete, will the data come back?
Just unattach and backup on an external harddrive.
From Management-
“Create a database in production for my son, school project…”
Lets create all the indexes suggested by the Database Engine Tuning Advisor in development on production DB.
Hi, I’m the software development manager and DBA…
Hmm. Not sure if I should be insulted by this one… =)
No worry with backups, We got RAID!!!
Install SQL Server on the domain controller, sure.
“Hey guys, watch this!”
Stay close to the candles, the stairway CAN be treacherous.
“I forgot to change the database context before running this cleanup script”
Microsoft says “That error code doesn’t exist.”
We can’t afford for you to take time off.
Setup our new ERP system in NoSQL.
Setup our new financial database in NoSQL.
Our new database server was setup in the hall way.
Help our new IT department migrate the servers.
The CEO’s son will be job shadowing you.
While writing a cleanup script – “This table only stores contacts and not employees, correct?”
Sure, the SSMS script will stop if there’s an error.
Database in rollback hard reboot backup jobs failing on SQL65
We have a year of daily backups. None of it works.
Sysadmin: “I setup your sql cluster with [vendor’s] new clustering software.”
“Oh, I forgot to select the required statement before I executed the query!”
Told you we should buy cooler or at least extinguisher.
“no, this trigger with cursors is temporary”…1 year later…
I’m sure this worked on Oracle
DELETE FROM dbo.Sales
–WHERE Id = 14667890
Chuck Norris is our only hope now.
The sales manager will be DBA while you are out
Just install it on RAID 0. It’ll run faster!
Who commented out the Integrity check in the Agent Job?
Our service accounts share an id so things are easy
That’s how we’ve always done it!!!!!
32768 simultaneous user connections.
All of the important data is stored in one table.
Was it wrong to email that confidential list of data?
Setup a data transfer to this state agency…
We only accept faxes.
What is that noise coming from the server room?
A wild animal ransacked the server room.
The server room is flooded.
We’re moving all of our confidential databases to the cloud.
We are moving everything to open source.
That was supposed to go to production, right?
We’ll encrypt those SSNs when we get some time.
(´?_?`)
Just put that small change into production! No need to test this.
I heard some database corruption is actually healthy.
I’ve decided to be more of a hands-on manager
DELETE FROM Users
WHERE Name ”
I meant SELECT *!
We’ve encrypted the entire database. You have the key right?
Did I just ran the Delete statement of PRD instead of DEV?
Did I just ran the restore statement of PRD DB instead of DR DB?
Schedule a full backup on Saturday and a log backup on Thursday.
Umm, yeah… I’m going to need you to come in on Saturday.
The SQL minds went on a 3 hour tour…
We don’t need backups, we’re mirrored.
someone in shipping gave the new guy sysadmin
Before I left Microsoft, I made SHRINKDATABASE introduce random corruptions.
HAHAHAHAHAHAHAHAHA
LMAO
Great career move!
“We were getting a lot of errors in the app, so we just changed the data type from DATETIME to VARCHAR”
“What’s this script /job supposed to do?”
Nobody ever took down SQL with one line of code.
“Getting deadlocks? Use WITH (NOLOCK)”
This!
Explained my problem to Paul Randal and he told me to call Brent Ozar. (g,d&R)
“Transaction rollback in progress” – So how do we kill that?
We’re moving to AvailabiltyGroups for performance reasons.
Indexes use 3x the storage of the table data – so what!
DBA – Designated Blame Acceptor; DBA really means…Don’t Bother Asking
I would like to mention a couple :
1. We always backup to ‘NUL’ device.
2. The load test config file had PROD server details – Did we do anything wrong ?
Kudos for steering our a great discussion !
truncate table master; truncate table msdb; drop database tempdb; go
I thought I had a where clause on my update
I don’t need to test this code change…
The server room is directly below the men’s room?
Actual LOL on this one.
I once had a server room directly below a bio lab. True story.
Nothing like frantically covering your cabinets because the CEO’s private toilet overflowed. Couldn’t be sure the color of the water was due to fireproofing foam between floors or something a bit more disgusting. ????
“On-call doesn’t bother me. I put the phone on silent”
Why should we put AutoClose off ?
“We don’t really use foreign keys in our database.”
…so I set up a job that shrinks the transaction logs every night.
The subscription(s) have been marked inactive and must be reinitialized
The only possible response to that is, “ARRRRGH, NOOOOOOOOOOOO!!!!!!!”
All instances run under the same domain account.
We’re bringing in PeopleSoft.
Transaction log was taking quite a bit of disk space, so I truncated it.
Please make an adhoc backup of my 2000 database…..
My SQL is running slow, let’s start SQL Profiler!
My SQL Database is running slow, let’s start SQL Profiler!
whis is my query not working “please give me”
The application doesn’t need Foreign Keys, Views, Sprocs, or Indexes.
We do production deployments on Friday at 5:00
We lost the SAN and backups because of the flood….
“Test systems are for cowards.”
DBA actually said to me once:
We should try to keep our CPUs at 100% so they aren’t wasting cycles
Customer “IT-Architect” told me the same, “why are we not utilizing our processors to 100%, fix that”.
Dev guy: “Could you please script out all objects from prod?”
…
Yes that means he didnt have any idea what he deployed a few weeks ago …
Microsofts new killer title for the next SQL Release: “ToServeMan”
Transaction log was full, don’t worry I changed it to SIMPLE.
Its a cookbook!
We need vendor approval to use FULL recovery model
Wait, there’s no where clause in that update statement?
Kathy, I like that one…wish I’d thought of it.
What do you mean “Do I like Cricket and Curry?”
You hear?!, Bob the Access guru from accounting won MegaMillions!
lol!! i worked at a company once that went public and bunch of folks were suddenly very well-off. People were very scared who was going to not show up for work again
Who’s that Fill Factor guy?
don’t worry, eventually consistent will work fine
Whew! finally every instance now has mirroring! We’re set forever!
S is a mess? oh SSMS
That update should be done soon. Well goodnight.
I thought the drive array SCSI cable was loose…
I checked the box to be notified about followup comments, and changed my mind after so many came in…I’ve tired suspending it and finally deleting it in your subscription manger, but it’s not working, I’m still getting emails. Can you do some behind the scenes magic and get the update notifications to stop!!!!????!!!! Please and thank you!!!!
DBA: I thought I was in DEV.
You can’t rollback that statement.
I’m using Access and I’d like to…
Fix server performance! There is nothing wrong with my query!
“Our maintenance plans do index reorganize and then a shrink”
The vendor set up the database server.
The application requires sysadmin.
We don’t need error handling, this code is bulletproof.
Oracle have bought Microsoft
I heard threatening music in the background as I read these words.
SQL Mirror broke. Now I’ll have 7 years bad sp_blitz.
Ha! Nice one.
Boss: Ask Microsoft when they’ll have SQL Server for Linux.
“…Bob, a DBA for 47 years, is survived by his…”
Awwwwwwww…. this one is scary for multiple reasons.
We’ve gone the NoSQL route, it’s called Excel.
“Dad, I want to be a DBA just like you!…”
Need only one word – Opps !
The cursor in the delete trigger will be fine.
nothing ever goes wrong here we have great documentation.
Erm…we made some database changes whilst you were away.
Promise this is my last one and I actually overheard this last week:
“So it’s a SQL Server back-end with Access front-end” :o0
This is bad??? 🙂
don’t need a DBA to install SQL, just click ‘next’ — actually heard during a project startup meeting.
The transaction log grows 40GB every month, so we just set the vdisk to 4 TB thin allocation. Should be enough.
Previous DBA surfed porn on the server; downloaded a virus.
Vendor won’t support anything past SQL 2000.
What do you mean the fire alarm test tripped the breakers for the computer room?!?
– True story…
When talking to a customer who is self hosting the software (which is their line of business\crm software) and the sql database has corrupted due to a power failure: “Backups? Doesn’t your software back itself up?”
Why did the server restart during the patch window? Make it stop!
Yep – I’ve even been asked if we can just not patch the servers so the servers aren’t rebooted ever. The scary thing is that the business succeeded in that request for 6 months before someone just did it anyway. It was a cluster too, not sure why everyone was so worried!
Sadly, that is my company’s standard policy.
“We can’t afford to do OS or SQL patches. We can’t afford any downtime.”
When I looked at some of my SQL servers for the first time, the OS was literally 3-4 years behind on updates. care to guess how long THAT took to patch?
I’m just waiting for this boat to hit land!
When referring to a public facing SQL server: “Hang on, I’ll just email you my password [from my personal email account]”
“Our data loading process is to drop all indexes and foreign keys in the database, load the data and then recreate the indexes”
My first gig as a database developer, the guy who I eventually replaced thought that was the ONLY way to perform ETL. Not only indexes and FKs but a boat-load of indexed views got dropped, data loaded, then about a 3-hour process to rebuild everything including the indexed views.
Sure, the readonly account is really read only
What do you mean there isn’t an sp_undo???
“The disk was full, so I just deleted all the backups and stopped the maintenance plan”
“Database [myproductiondb] does not exist.”
No, I said never type delete *.* ever!
Why can’t I? I had sysadmin at my last job?
This is our dba, he’s done great things – terrible, yes, but great.
Is that smoke coming out of the tape drive?
and
But I deleted the table BEFORE the backups ran!
(Personally, I’m probably buying the goat simulator tonight, that’s just too weird! And I originally misread the title as Goat Stimulator, that’s a whole different game.)
we only use identity [int]s for primary keys
Interesting. Are we sure this is a horror story, or a best practice? Your mileage may vary.
Was that our main prod server? Oopss…were are our last backups?!
Eight instances? Eight processors! We’re good!
It’s supposed to do that
Cool! I found a flash drive in the parking lot!
What does “Repair_Allow_Data_Loss” mean?
Why is the Client table’s record count zero this morning?
Why are there no tables in the payroll database?
I didn’t know sp_Blitz could produce so many rows!
SQL Server has service packs? (true story)
Why is Celko turning so red?
I got this script from a Russian web site!
SQL server is similar to Access…so I reckon I can handle it
Sure the vendor can have sysadmin on production
Backups aren’t necessary with asynchronous replication.
We’ll just go pick up more storage at Best Buy.
“We didn’t make any changes, but this replication is very slow today” – 6 days, 2 hours and 23 minutes later……..
We’ll have to escalate this to Conor.
“Surely the DBAs can create a job to recompile that function every hour” – actual suggestion at a workshop concerning Server CPU Utilisation
We don’t need SQL backups, we do disk backups
Out of curiosity – did anyone check the free space on the db server before we posted that question that got nearly a thousand comments?
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 1.4T 1.4T 0B 100% /
Uh oh.
Haha, how’d you know I was hoping it would hit 1,000?
I’ve never used more than 1 database or tempdb file…..
Once we upgrade to SQL 2012, we’ll be able to have an Active/Active cluster, and we can replace replication!
What on-call schedule. You are our only DBA.
CFO: What’s wrong with SQL Express Edition? It’s free!
So, the application is running under sa?
“The reporting layer uses CLR T4 recursive dynamic SQL bitmasks”
True (current) story
T4 Text templates: https://msdn.microsoft.com/en-us/library/bb126445.aspx
used to generate C# that generates SQL objects (views, stored procs etc) that are then dropped when the data is returned.
The DW stores customerIDs using bitmasks… because
BitOr CLR aggregate functions anyone?
Oops!
They unplugged both sides of the mirror…
There is this bug where sometimes the mirroring software writes to the wrong block on disk…
Restoring a database, what does cannot find server certificate mean?
😛
DELETE FROM dbo.Users –WHERE ID=’PenTest’
Gotcha, Again freak developer is ignoring Index.
Damn, How can you perform live backup on portable HDD?!!
Because the app didn’t work with primary keys.
Oh God,I don’t want to discuss with SAN buddy.
Yep, All decaff now.
Oops, where’s the undo button in SSMS ?
Whole DBA team: “We all just use the system account to RDP”
None of our indexes has more then one field!
(saw this really – PK on the UNID, idx1 on deleted, idx2 on created by, idx3 on the customerid … and this in the whole db in the hope that (SQL 2000 to 2005 (depending on the customer)) will combine all the single column indexes)
We never use indexes, they slow down inserts and updates.
“Why wasn’t the external storage on the UPS too?” (true story)
Phone call from customer: “I made a booboo…” (thought the WHERE clause for the “DELETE FROM Customer” statement was a separate statement to run aftwards. Also a true story)
“I don’t believe in SET based data manipulation” (bit of a cheat, plenty of cursor horror already :))
“That’s not my fault! I’ve tested it! On prod…”
I am not really a DBA
The script works. I tested it in my local db
Can you quickly restore those rows I deleted last week?
Wow, Backup Execs error messages are so clear!
1: comp room on fire
2:Good thing we have backups taken to offsite
1: offsite ?
What DBA? Our databases just look after themselves…
…and this unventilated broom closet here is our data center.
You’ve got , we need you immediately
Actual event:
” that’s funny. When I pull the cables from the test server, the lights keep on burning”
The database is down! Where is the server?!?
(A huge telecom company had a Sybase SQL Anywhere running an invoice system. It was found in a small storage room for papers, pens and stuff. It had stopped because logs filled all avaliable disk.)
“all we need for our application is a means to persist our object data”
“BDOS Error on C: Bad Sector” (I’m surprised how many DBAs are old enough to remember this)
So many *clever* people that can’t count to ten.
DB Corrupted… fixed by finding users’ medical data on pastebin!
1. I’m sure you’ll love working here…it’s always so busy.
2. Get ready !! James Bond’s infiltrated the base !
3. Remember ‘Drop database good, drop table bad’
4. Grant Fritchey is coming to audit the databases…..
My database is called “master”
“Where’s the database gone?”
I’m not sure what happens with the backups, I’ve never checked.
we can “simply” just use merge replication to achieve zero-downtime
We’d never had a disaster until you joined…
Just restart the service (Switch it Off and On again)
So I need “WHERE” as well ?
DBA: I need more memory.
Manager: ask SAN Team, I approve
we don’t need monitoring. The customer will call when is bad
Is New SQL server ready? Tomorrow will know the application requirements
Yes I deleted the Transaction Log, the disk was full
That UPDATE is taking a while… Oh crap… No WHERE!
HDD full? no problem, add an .ndf on a pendrive.
“Where the hell are my PROD-Servers gone?”
http://thumbs.dreamstime.com/z/data-center-empty-racks-25622785.jpg
” Well I got the code from the internet” TeraByte database GONE!
Re-imagining SQL injection: “Our new backup strategy is Pastebin dumps.”
I’ll just reboot the server. That’ll fix it.
Sorry I didn’t highlight the Where in my delete statement
Do you think it’s because the server is a Capricorn?
I’ve never heard of primary keys, so I didn’t use them.
(This one is real!)
Our DR plan is the Zip drive I take home?
Not a DBA, but I stayed at Holiday Inn Express
Success! Your password has been changed. Congratulations on your recent move to Romania.
You can reject this one. It exceeds the 10 word limit.
Nine word version:
Password successfully changed! Congratulations on your move to Romania.
My salesman said this hardware would make SQL run fast.
“The firemen at the datacenter last night were really nice.”
“However, the firefighters at the DR site were just rude!”
Why do you need monitoring software?
You don’t need a test server
I want to be a DBA…..
Actual event:
” That’s funny. When I pull the cables from the testserver, the lights keep on burning”
I scheduled job to shrink log file every 10 minutes.
Nothing on the internet says it’s bad practice.
What is a WHERE clause?
” Not really sure what this “SP_OLAPDance_AnaLZR” did, but I am pretty we are gonna need a really good PROC to fix it?
not 10 words but I wanted to share..
I sweat it worked in my machine!
Back out plan? What’s that?
It’s industry standard and best practice to run SHRINK DATABASE on a regular basis.
No definition documentation?! I’ve just applied encryption!!
When I met you I was but AdventureWorks now I am the Master…
Sure, I know T-SQL, select * from …
So the Production DataBase is named “constoso”?
That’s not the backup you’re looking for…
I thought the Cloud took care of everything?
Finance has requested a new fee column called ‘hoodwink’
EXEC sp_Rollback_My_Most_Recent_Transaction @PrettyPlease = 1
They can learn it as they go
why do you need supplementary space on SAN, why do you want to backup database ?
Text files? Just dump your debug text into a column.
(We discovered this little vendor gem when Replication started choking on a row with 2+ GB of text in a single column.)
Deadlocks? I thought you said “Dreadlocks”!
I don’t know the vendor set that up.
Sysadmin; “I installed SQL for you”
DBA: “Oh god”
I/O problems? just create a stripped volume in windows.
We store all our Oracle backups in /dev/null.
We didn’t plan for the DBA hit by bus scenario…
I thought of another one that I don’t think has been mentioned yet.
“Good news, you’ve been promoted to manager!”
soon followed by
“Unfortunately we’re unable to backfill your role at this time.”
We don’t need Change Management…
From head developer – “What’s a clustered index?”
“The server room as a drop ceiling, roof above.”
“has”
“Oh sorry, were you just leaving?”
Application handles auditing and doesn’t shutdown server if audit fails!
Yay, compliance!
“Go ahead, I think the licensing covers that.”
OH NO!! I thought I was in DEV!
I thought a clustered index was only used in a Windows cluster, why are we using it on a standalone server?
I dont know how, our DBA just quit.
Test it here before going production. Wait, where we are ..?
I use query hints everywhere cuz I know better than SQL.
Let’s try it. What can possibly go wrong?!?
“Just use the MSDN DEV license for that.”
“Are we talking about production or real production?” …apparently there’s a difference
From Data Scientist – “The query performance runs incredibly fast on my laptop…”
What would have been scarier to say is… some guy in our company said… “I’m the Data Scientist.”
A struggling small company let the IT hardware guy go without assigning duties to anyone else. I was the report writer and this is what I heard when I walked in one morning:
“I got in here early this morning and there was an alarm going off in the server room that seemed to be coming from the AC system so I flipped the breaker for it and closed the door to keep the noise down.”
That company is no longer in business.
No need to test, let’s move it to production!
I want to use SAN snapshots for backups.
The Developers have to have SYSADMIN rights to Production.
integrate this with hadoop
After this sun storm there will be no electricity forever!
PS: I think this scares all of us.
this db controls the nuclear destruct button.
true story from team member – i restarted the db, it’s recovering, 4 day estimate.
There was a restored db (we’ll call it test) on a production instance… this is from IT Director.
“Run the query on TEST, should be done by tomorrow.”
Our vendor’s DBA is named Edward Snowden, and…
Is a fill factor of 15 OK?
I use Cursors for every query I do.
Not sure what’s wrong? did exactly as suggested on brentozar.com !!
Go and talk to those developers
Couldn’t we just move all the tables to Excel sheets?
End users have SYSADMIN rights on production.
(True story – this is 3 years later and can’t get it changed)
Couple of Junior DBAs are joining to help you out
Is anything going on right now on that server.
Change directly in production, copy the DB to test.
(That was the change control proposed)
Last time the DBA(s) did something and issue was resolved.
Performance tip: kill all the users!
I know it’s a test server but …
Just make all the columns VARCHAR(MAX)
Actually, our Marketing Director also doubles as or DBA … why?
It needs domain admin and sa access
About that SAN firmware update, um, yahhhhhh.
Tapes? We haven’t used those in years. Why?
Activate trace flag 843 in the SQL-VM to improve performance.
Correction:
Activate trace flag 834 in the SQL-VM to improve performance.
JSON in 2016 will solve all your XML performance issues.
Sure it’s o.k. to have 30 joins in a select statement……
What do you mean you won’t give that vendor sysadmin to the production Server ? They say they require it to install the latest version of their software! And no, we did not test this, they are an international company …
(true story)
I wish I had $10 for every time I’d heard *that*
Who is Brent Ozar?
Operations wants real-time analytics out of the data warehouse.
Nested views have no affect on performance
Query not running with no indexes and 100 joins, why?
“Vendors break support if you index their huge heap table”
I have a few of them that popped up in my head right away
Situation: All hell broke loose
Sentence: Are you sure you dropped the TEST database
Situation: Vendor comes to install product
Sentence: Just make the service account sysadmin, that way it works
Situation: Me asking the former DBA why the SQL 2000 servers are the RTM versions (its 2015 now)
Sentence: Oh we didn’t have time to update the servers
Situation: A developer made an adhoc change and wants to deploy it
Sentence: Oh sure just deploy right into production without testing
Situation: All hell broke loose version 2
Sentence: I’m sorry the last good backup is 4 weeks old
Situation: Missing disk on production cluster
Sentence: I’m sorry I deleted the wrong LUN
I cleared some space by deleting the transaction log file
I heared that Hadoop was going to solve our problems
+50000
“Hi”
“These features are scheduled to be removed”
— I tried to think of the 10 word authored by MS itself.
Mirroring and Trace come to mind. I’ve got AGs down but I am so attached to SQL Trace/Profiler etal. I just resist Extended Events.
Has anyone done a DBA wage comparison study by Gender?
— scary for guys and gals
Here is the list of vendor folks who need SA rights.
We don’t believe you need clustering, now that we are virtual.
We have SSD, don’t think you need any database maintenance now.
Why do we need a DBA, we can manage everything
The trigger fires a dynamic SQL cursor which cross joins…
Shards are the secret ingredient in the web scale sauce.
Just reboot the server – it’s the solution for every problem!
Hold your breath and run, that’s the halon alarm!
There’s a fire in the UPS battery room.
The sudden power loss caused corruption in the SAN.
(2/3 comments are true, you decide which.)
Tge phone rings, the GM is on the other end of the phone and says, hey did backups complete last night? Hood, then can you come to my office? Oh, and on the way, grab the tapes and bring them with… minutes later, after arriving at the Casino GM’S front office, he appears with a rather large police presence. Curious, I ask what’s up and the Sherri informs me of the bomb threat we just had. Apparently it was supposed to be located in or near the boiler room adjacent to the server room. It took me a few moments before I realized that we took our time gathering tapes, then we realized we were asked to gather tapes… not just get the hell out. Well, we know where his priorities were.
This demonstrates how scary you can be with more than ten words. Holy cow! Stephen King eat your heart out.
184000 VLF’s? It might take a moment to boot!
“We are migrating from SQL Server to Hadoop!”
“Ooops”
–Anytime that is spoken by any DBA Be afraid very afraid
This SQL script was authored by Brent Ozar.
Backups are just a waste of disk space.
Encrypt data? Why? We have insurance to cover breaches.
The invoice from Brent Ozar Ultd just arrived.
“DROP DATABASE……….., was there a back up?”
We called Brent Ozar and he couldn’t fix it either 😉
I know you’re just beginning, but I am leaving…
Priority- 1:
Hey DBAs, We are having timeouts. Please do the needful.
*and then this person never replied after you start asking questions*
From the SQL developers:
No, we fix everything directly in prod, we don’t have time to test!
That process is running too long. Let’s reboot the server.
We’ve decided to switch everything to Oracle and MySQL.
After initial, deep review of production OLTP financial database, which fed every other database in company, I had to meet with CIO to let him know that there was close to 60% data corruption. His response? “In my opinion, accuracy is overrated…”
Can I have the sa password?
Can we put a public IP on the server?
My SAN can handle 200K IOPS, far from your poor 500 IOPS, so do not complain about latencies, this is not a problem, and please do not make IO that size more than 4KB if you want good performance.
Every live database will soon taste the restore:)
When did you move the backups to a different directory?
True story – hospital – nightly tape backups backed up an empty folder for 6 months
That scripted change you ran had a DROP and CREATE.
Also true story. Scripted change from dev being put into production. Dumped ALL of the data out of the table 🙂 Followed shortly by my previous entry
DELETE FROM CUSTOMERS;
I accidentally the whole DB server and no backups.
SQL? I can write that faster in Java.
I read somewhere MongoDB can do all of this better.
How long to restore all the SAP databases…?
A server emailed me today; we decommissioned it years ago.
All true stories i’ve come across as a DBA.
– “We have backups!…from 2013….”
– “we can’t remember the SA password, but all jobs depend on it”
– “we changed the SA password, now we can’t get in”
– “let me get back to you about this query… i’m still waiting…”
– “The backup tapes are…empty ”
– “disk is full, let’s shrink the log files”
– “what’s wrong with 100 joins in a joins, I made it easy to the eye.”
and my favorite
-“you’re the DBA, make it run faster!”
If we temporarily shut down the sqlerver while being audited by MS, we’ll be ok…
~IT manager
I heard you said refresh production database from development, am I right?
Prod DB server down, backups corrupt, need SQL 7 media
At least its not asking for the 6.5 media!
Grant Fritchey is in town. 🙂
Can we turn off referential integrity for these updates? (True story)
I thought it was Dev machine…
Infrastructure: Why do you need to do backups? The VM’s back everything up every night.
+1
“We host production, acceptance, test and development databases all on the same server instance here”
It’s built on freeware. That’s got to be good, right?
Database Died Disaster Documentation Deleted DBA Dismissed
“I’m leaving next week; no, there’s no replacement.”
It’s OK, I got this script off the internet
Job advert: must have two years experience of SQL 2016!
… and at least 3 years with SQL Server 2015.
I just checked, the backups stopped working a month ago.
We don’t have that in our SLA !!
I upgraded the enterprise to Access
What do you mean I dropped the wrong database?
I deleted the tempdb files – they’re temporary anyway, right?
Did the UNIQUIFIER really hit (2^31-1)?
I’m sure the backup is there… barring no problems!
“Your data is on a big RAID 5 Array with 3 – 1TB Disks, you’ll have plenty of space.”
“We dont need a DBA any more, we can just google any problems”
All columns are varchar(max) so we never worry about truncation.
Oh that pay rate table.
I took the fk’s off, they were causing application errors
We have Excel, why do we need SQL Server?
No problem to go straight to production without testing.
Reading these gives me gas.
from developer…
I’m not sure what is causing the problem. Can you look through my code and see if you can find it?
Who’s on call next week-end?
“How far back do you have backups of production?”
How did you, John Devloper, get access to Production?
You added Corp\Everyone to the SysAdmin role?
We don’t need to back up this server; its RAID 5.
You’ve been marked SUSPECT by recovery.
The server is upstairs, in the loo.
True story, unfortunately. Conversation was between the local manager and the corporate CIO. Server was relocated posthaste.
We’re going to be hiring an additional DBA to offload some of your extra work, can you recommend what training classes we should send them on to learn about database administration?
The function should evaluate words <11, not words <10.
Script sent from a dev (true story):
Delete from Orders
Select * from OrdersHistory
where Date <'12/31/2012'
“The SA password is sa”
All the firewall rules are any:any,. Why?
Unable to create a restore plan due to break in the LSN chain.
That’s my job?
No FWRC required, that SQL Server is on the DMZ.
One Server, Two Instances, Mirrored Database, Who needs backups?
The result set keeps changing
Give the end users full admin rights, we trust them.
(That’s the CIO at a former company in 10 words too! Wooo-Hoooooo!)
I think there’s a process in SQL Server to ‘automagically’ take care of this.
(Note… Believe it or not, I just came across this word in the SQL 2016 Books online “What’s New” section, under temporal tables… https://msdn.microsoft.com/en-us/library/bb510411(v=sql.130).aspx)
LOL great technical writing
What do you mean it’s a MySQL Database.
Microsoft is going to replace SQL Server with xml datastores.
My machine crashed; you were backing up its databases, right?
The data and logs are on C:
Its just a simple delete.
I’m done with this, I’ll update my CV asap
“There are no SQL Server DBA’s out there!”
Statement actually made to me by my clueless former CIO while pointing out the window and asking me if I knew how many SQL Server DBA’s were out there!
I ran the test environment set up script in production…
Cant we just test it on live?
“The transaction logfile got too big so I deleted it.”
(True story. I know a sysadmin who actually did that. Hey, it’s only a logfile, right?)
I don’t believe in that normalization nonsense.
Hey Brent, put these comments in a DB to analyze. – Kendra
It’s Microsoft, what could go wrong?
You STILL use sp_who2?
Turns out we do store tax IDs in the database.
The DR site is located on the San Andreas Fault
Vendor recommends placing everything on the C drive.
Just restart SQL, that will fix it.
The clustered index needs all 14 columns.
SELECT * FROM
I was testing out this truncate script but forgot to change the server to test which began truncating all of the production tables as I stepped out to lunch…
I think I deleted the production database instead of the test database.
SELECT *
FROM (
SELECT *
FROM (SELECT *
FROM…
(This is a real query in our environment.)
I think I deleted the production database, instead of test.
We have never tested the backups.
The sysadmin has pointed the DNS entry for PROD to DEV.
I don’t always test, but when I do it’s in production.
I’m just going to restart our production SQL Server, o.k.?
Parameter Sniffing!!! Really, Is that a problem?
The vendor is using sa/sa and doesn’t know how to change the password in their app.
Don’t worry, I earned an ‘A’ in Data Structures.
That was the Dev box I just deleted that datafile from…..right?
Our offsite server provider will be underwater within an hours. #TexasFloods
Can’t we just compress the disk for more disk space?
We just make everyone a sysadmin so it’s easier.
I wish we had referential integrity.
SAN Team wants CheckDB disabled. Causing hundreds of 833-IO-SCOMs/hour.
Referential integrity is slow. We’ll handle it in the app.
Restarting all cluster nodes simultaneously to deploy new windows patch.
You don’t mind learning Cache, do you?
What do you mean the database doesn’t handle Feb 29th.
Relational databases are obsolete
Why are there so many tables?
That will never happen.
Uh, I think I may have screwed up.
PostgreSQL/Redit/Linux better Enterprise Solution than SQL Server Enterprise
Event viewer is showing disk errors, but that’s expected.
Who remembers the TDE certificate backup password?
We don’t have a dba.
“I used no lock, I can’t be impacting performance” – Really happened.
Vendor: “Whenever that happens just reboot the SQL Server.”
This was how one of our third party software’s help desk suggested fixing an error within the application.
I give all my users DB owner for good relations.
Using BCP is a great way to do a backup.
Manager for a related helpdesk: “We need you to take calls from the [Customer Service Helpdesk] queue.”
“Can you try running MSSQL off a network drive?”
Let’s see what happens!
What is point in time recovery?
In Production, I implemented a SQL Agent job to truncate Error Log every five seconds.
With over 1000 comments, there’s now just a 1-in-1000 chance — or worse! — that any of us will win the coveted Goat Simulator.
Which means…
http://media.giphy.com/media/ToMjGpKniGqRNLGBrhu/giphy.gif
We love SQL 2000, we’re never upgrading.
“The previous consultant said SQL Express was all we needed.”
“I believe it is working.”
“We are having some issues with the backup software.”
“Don’t worry about it, we take OS-level backups.”
“We don’t use stored procedures. Everything here is .NET datasets.”
“SharePoint has the best database architecture I’ve ever seen!”
Here we consider the first day in production as the test.
“You have to disable t-log backups before doing database backups”
I need a linked server to an Excel file.
We migrated all production DB servers without any errors.
Whoops! I left out a word. It should have read, “Strangely, …”.
“For Sale: One backup device. Never used.” (In honor of my fellow Floridian Earnest Hemmingway. http://ezinearticles.com/?The-Shortest-Story-Ever-Told&id=1305285)
BUCK WOODY!?!?!??!?! OH MY GOD I MISS YOU.
Can’t upgrade SQL 6.5 as we need sp_configure ‘allow updates’
I’m from management and I’m here to help.
LOL, that is hilarious!
Our app only uses nvarmax datatype for the best flexibility
*nvarchar(max)
We have to run datawarehouse and transactional database on the same server
Heaps are our developers best friends. Dare not add an index
Upgrade? why the current version is working fine.
I wrote a trigger to prevent duplicate inserts.
What could possibly go wrong?
nHibernate is job security for dba’s
I’ve got 2 here:
from dev: “we want to do data encryption using CLR”
from SAN admin: “the RAID 10 array has 2 physical disks in it”
I went to restore that database from last week and realized the SQL Agent had been stopped for the last 2 months…no wonder I wasn’t getting notifications from that server.
Best practice says make every field an NTEXT.
Sure, we can run your Databases on that Thumb Drive.
Stored Procedures, We don’t need them.
We lost the SAN.
——
Company Policy, Autoshrink must be turned on in all Databases.
But it worked beautifully in Test!
If you give me sa rights I can create more.
“8 Node SQL Cluster is down” DBA-Node name??..L1 –Next update in 30mins”
This is really scaryyyyyyy……
“DB Backups not Required because takes long time than snapshot.”
All I did was add a select to the stored procedure; how could that break production ?
How could stepping through the debugger in production bring the database to it’s knees?
It’s a myth that servers break down
Major Incident Management on the phone about a missing database…
It can’t be Entity Framework that’s the problem…
truncate table msdb.dbo.backupset
“I prefer Count Star to Count Dracula, because I don’t like having to consider which Draculas are NULL.”
Our SA password is very secure it’s “password” clever huh!!!
“I don’t see anything better in SQL Server 2012 than what we currently are using in SQL Serer 2005”
“Whoa, that was close!” – SysAdmin
tag! You’re it!
You were stood next to the server when it died
We need to hire (Consulting hours will not be sufficient) folks from Brent Ozar Unlimited to fix these issues!
Whose the accidental DBA again? Oh F…its me
I’ve increased performance by replicating the mirror’s snapshot to itself
sp_blitz result: Poison wait detected.
We don’t need a DBA, our developers know how to do that.
The Production database just went offline! Did you unplug the external USB drive from that computer?
We’re going to P2V your SQL servers this weekend.
The SQL Service Pack failed and corrupted the Production instance.
Hold my beer whilst I setup this bi-directional replication publication!
“Oh, I saw that CXPACKET were top 1 waitstat so I just changed maxdop to 1 to optimize performance on the production instance ! :)”
Let’s run this random script I found on the internet
“Mary brings the backup tapes home so they are off-site.”
We only hire junior DBAs and train them.
Run this for me real quick in production:
UPDATE hugeCriticalTable SET colName = CASE WHEN colName = ‘value’ THEN ‘newValue’ ELSE colName END
We’re out of ideas and have tried nothing.
“Yep, the backup tapes should be fine. We keep them in a fireproof safe in the server room” — the IT Director said after his server room burned.
I added NOLOCK to prevent deadlocks, just in case.
“…which is why we need you to start being our Oracle DBA”
I have two entries. In general:
“ooops”
and in the DB-world:
” Hey guys, how, em, do I undelete a table?”
….. Everthing is working fine . . . . . !
Our last DBA was an alcoholic mess – eventually.
I love this one. It has a single word punchline. It’s scary (and a little sad)
Replicated prodution to test, without changing vendor PO mail addresses.
(true story… vendors started calling to verify they really needed that order 5000 times)
We can’t allow an antivirus exclusion on LDF files.
Simplifying is… using identical connection strings for all OTAP parts.
Simplifying is… testing backups, but not restores.
Simplifying is… loading an active/active cluster to max, expecting HA.
Hey DBA – Can you clear that blocking?
I need “full access” to the database.
Why doesn’t CTRL-Z work on restores?
We’ll let you know when we need a DBA.
The monitor jobs produce too much email to read.
400 users, 80GB DB; we’re too small to require a DBA
Sorry, I thought I ran that on the QA server.
True story. This was said to me by the VP of engineering with no further context.
“I need an analysis of our database”.
The last DBA tried their best.
We don’t need database backups, we have system backups.
Working in a hospital: “Do you know what happened to the OR schedule?”
I´ve emailed you the changes made in prod.
Brent Ozar “Wow, I’ve never seen that before.”
Application specialist on vacation, vendor upgraded the app Sunday.
“Is there an ‘UNDO’ for dropping a database?”
True story.
We do all our performance benchmarks with recovery mode simple.
Microsoft has released details of new SQL licensing model.
Real man use only RAID 0 !!
“I prefer to do things with the mouse”
I think this is my favorite
“I’ll just make you sysadmin, it’ll be easier”
or
“I’m sure i used BEGIN TRAN….”
“Can you set up something to shrink TEMPDB every half hour?” – vendor
I heard 4 out of 5 at my last job, makes me glad I left. There was a plastic lined box on top of the mainframe in the data center because the bathroom directly above it had leaked
Put this on my resume? Why would I even need a resume? They LOVE me here! Luckily I saw this in between meetings about Sql Server licensing costs and reducing developer privileges.
There is NO contest here. Kendra was just joking 🙂
I promoted the TEST Db to PROD.
(Yep, another true one, the numbnut next to me informed me that is how he promotes his changes…Jesus wept)
oh noes…. I’m too late… AAAARGH :p
Grant Fritchey is carrying a baseball bat.
So did anybody win?
Yes! The post is updated way up above. I realize that once you get down here in the comments it’s hard to notice that though 🙂
This is not the contest you are looking for *waves hand before your eyes*
http://bit.ly/1CwxYNI
Ivan Krechetov ?@ikr wrote “Never let a colleague’s vacation go to waste. Act now! Rewrite everything!”
And he didn’t even know he was entering.
Matt Velic ?@mvelic replied “Oh, the coworker is that DBA you don’t like? DROP all the databases to make their vacation more interesting. ”
Neither did Matt.
My entries: “Look what we did on your vacation!” or “We had a slight problem while you were gone.”
HAHAHA
“Don’t worry. We fixed it.”
“Oh, by the way….”
SSMS has been authorised as an End User Tool
The database is in suspect mode.
Oh. That shouldn’t happen.
(Late, I know)
I deleted them now how do I back them up?
Auto-Shrink and Auto-Close are turned on, that’s best practice, right?
Are backup table is called backup in the same database
“Why’s it doing that?” – I overheard our our Junior DBA saying that while he was working in production, bout gave me a heart attack.
Everything here runs as \admin. You can’t change the password because it’s hard coded in most of our production applications.
That was meant to be domain\admin.
“Oh, you meant THAT server.”
We’re covered – backups are stored on the SQL server C-drive.
The database went suspect and the log file couldn’t be read. So we created a new, empty log file, using notepad.
Database has corruption! So is the database down, no, then let the business continue processing while you fix it on the test environment.
There was no change to the proc but its started failing suddenly… well we just added CTE only but that shouldn’t cause anything…
disable the optimizer is consuming a lot of space on the server
#1 it used to work on the old system, so you can just copy it
#2 its company standard so we dont need to test it
SQL Server? We use Excel and MS Access as backend!
#1 – Backups? I’m not responsible for those.
#2 – Why would you use compression? Doesn’t that make it slower?
#3 – The SQL account has had domain admin access for the last 10 years – we’re going to change that – but we’re not sure if anything will stop working…
“I deleted the old encryption key I swapped out”
This actually happened to me. The CTO of my company was working in the data warehouses on the weekend. He calls me up and asks, “Quick question: does Linux have a recycle bin like Windows?”
It smells funny around that white metal door.
“I can’t believe TRUNCATE actually does that.”
The Cloud is automated. We don’t need DBAs.
dbcc freeproccache running hourly, really fixed production performance. ?
Just change the password. It will be alright.
Since this is super old and no one will read it.
“We backup to tape!”
The reasoning:
No magic dust in those words so nope, no magic fix.
Usually rotate the same tapes.
They have used the same tapes for years to save money.
No one checks the tapes for hard read errors.
Further thoughts:
It was not configured using the proper Microsoft way so they will not troubleshoot it till the end.