Just had a server crash at a client site in Phoenix because I forgot to check the UPS battery backups

I was swapping out a failed drive in their RAID array and when I unracked the case, the UPS died and took down three VMs before I realized the backup battery had been blinking red for months, has anyone else had a silent UPS failure that almost cost them a client?

2 comments

2 Comments

miller.rowan1d ago

Wait, the UPS had been blinking red for months and nobody said anything? That's rough man, I've had a similar thing happen with a client where the battery was beeping quietly behind a rack for weeks before it finally gave out during a storm. Feels bad when something so preventable almost ruins your whole day, lol.

joel_jones1d ago

Red light on a UPS is basically a death sentence nobody reads.