[kwlug-disc] TrueNAS scale (4-5 drive configuration)

[Chris Irwin]

On Sun, Sep 22, 2024 at 12:05:37PM GMT, Charles M wrote:
>setting this system up with TrueNAS.

I had a "DIY" nas for a number of years running Fedora Server, using 
BTRFS. I switched to TrueNAS Scale a few years ago and am mostly pretty 
happy with it.

My main reason was because I was sick of dealing with the minutia of 
server admin for the "boring" parts of my setup.

>What would be the ideal drive configuration for 5 drives? 

I'm using 4x 8TB CMR NAS drives, and went for RAIDz1, which is kinda 
like ZFS's version of RAID5 -- so n-1 drives worth of storage.

That gives me 21.01TiB (8TB drives actually 7.28 TiB) of storage.

>Capacity would be nice, but I also don't want to have to re-rip 
>everything, so data integrity is important.

TrueNAS sets up a weekly scrub. Mine scrubs the entire array every 
Sunday, verifying the data integrity on-disk. It takes about 9.5 hours 
for 12.3TiB on the array.

If I have a drive failure, I'll just pop over to the store and buy a 
replacement drive >=8TB (whatever looks like a good deal for a CMR 
drive). Pop it in and everything will rebuild, and be verifiable at the 
end.

In the case of actual disaster (i.e., two drive failures, failure during 
rebuild, etc), I still have backups. That's why I didn't opt for RAIDz2, 
for example.

>But if drives are NFS mounted by another machine are there other 
>concerns? All drives are CMR NAS drives.

Do you mean performance concerns? Not really.

My NAS Server, as I said, is 4x 8TB CMR NAS drives, and has a 2.5GB 
ethernet connection.

My XCP-ng Server runs multiple VMs, all from storage on the NAS via NFS. 
One of those VMs is Plex, which can handle multiple streams.

So no, there isn't really any performance concern for regular use.

I only really experience an issue if I'm rebooting the xcp-ng server, 
and want to boot all my VMs up at once. But that's an issue that goes 
away in 4-5 minutes anyway.

Note with TrueNAS: Unlike with mdadm, lvm, or btrfs, you're stuck with 
the layout you pick. So if you have a 5-drive RAIDz1, you can't just add 
a drive to bump your capacity (ex: migrate to 6-drive RAIDz1). This 
functionality is reportedly "coming soon", though. It's not something 
that matters at the enterprise level, but is something that is 
relatively common at home-lab scale.

-- 
Chris Irwin

email:   chris at chrisirwin.ca
  xmpp:   chris at chrisirwin.ca
   web: https://chrisirwin.ca