[kwlug-disc] NVME failure?

[Khalid Baheyeldin]

Thanks Jeff and Chris,

Will check Backblaze ...

The server has internal backups. When the server was being
provisioned, I asked the host to
have a large spinning disk just for daily backups. Every day, in the
early hours, I have scripts
to backup the site's web root, and do a dump of the database.

This data also goes to another server at a different data centre and
different hosting company.

The web host has backup via Acronis, which is incremental and online.
I asked them to restore
the entire server, so data integrity matches (static files and
database). They then said that they
restored only the database, but I could not see the live database,
neither in the /var/lib/mysql
default location, nor /data/mysql on the NVME. Instead of wading
through tickets and phone
calls with the support staff, I decided to save time, and use the on
disk database dump.

Restoring the database from the daily backup took  almost 1.5 hours,
but all was good in
the end ...

Being a paranoid sysadmin pays off ...