Data loss didn't happen!

So I've ranted, raved, and preached ZFS for so many years, and it continues to pay off! After many years of running my main array, I finally lost a drive.

For many years now, I've been running a set of 3, 10TB drives in a RAIDZ, and I later added another 3 12TB drives, also in a RAIDZ, I was smart enough to buy 4 drives when I added those, and left the last in a a hot spare. Low and behold, I woke up to a message on the 27th stating that I had no more hot spare, and my array was resilvering. My 10TB had enough errors, and was booted from the array, and replaced with the hot spare, all while I was sleeping.

All data is safe

Not only did the hot spare mean that there was zero downtime while this all happened, there were also 2 more drives that could have died before data was lost, and even then, proper backups have been made. There is a full backup of all non replacable data as well as full machine backups on a schedule on a seperate machine locally, as well as offsite storage of critical data. RAID is NOT A BACKUP. That said, I love my uptime, and don't want the massive inconvenience of fixing things. Praise ZFS, and all it's holy glory.

Planned downtime.

While I could absolutely pull the old drive, and replace it with this new hot space that I've obtained, I'm going to take this chance to pull the machine down, dust it, and do host level upgrades. Downtime should be no more than 2 hours, and will be over this upcoming weekend as of the time of this post. Keep an eye on https://status.kdb424.xyz if you need updates on what's going on, or just want to know when things are happening. I keep a good log of when anything is happening there, as well as scheduled maintenance times.

Running all of this is getting expensive!

Self hosting is absolutely cheaper than paying for services elsewhere, and more importantly, the data stays on trusted servers. I'm not the only one using these services anymore! Friends are using immich, my git server, XMPP (which is starting to stockpile a bit of storage usage for media that people send), as well as other services. We're currently using 73.4% of the ~40TB capacity, and that's logical capacity. There's nearly 80TB of physical storage in the server, not to mention the 32TB of backup disks, not to mention the offsite storage. I'm also paying ~$13USD/month to make things available to others via VPS, and that's ignoring the electricity to power all of this storage, and an Intel GPU for transcoding video on the fly (media streaming, and compression for immich videos). If you're using these services, and would like to see me do more, consider buying me a coffee. I never like to ask, but I've been offering more and more to people, and costs are adding up! That said, don't be scared to reach out and see if you can have an account on anything. Worst you'll get is a polite no :)