Different size drives question

[LemonBag]

Hello all,

I understand for ZFS you have to pretty much use the same capacity drives in a vdev because it will only see the smallest drive. However, I have a 4x10tb mirror and see a 12tb drive going for cheaper than a 10tb drive by $30 USD. Does ZFS use that extra 2tb at all? Not that I need ZFS to use it, but maybe it uses it for some overhead considering ZFS doesn't like to run at more than 80% ish of capacity?

 

[LemonBag]

Basically any downsides to adding a 12tb drive to a 10tb 4 drive mirror array other than "wasted" terabytes? "Wasted" being trivial here since again they are cheaper for me here lol

 

[Arwen]

Your vDev description is vague, as you state 4x10TB mirror... which implies 2 vDevs of 2 x 10TB disks in a Mirror. Thus, it is irrelevant in what another 2 disk mirror vDev disk size is.

Please note that ZFS does not support irregular stripped Mirrors that some other RAID schemes do. Those others might have a pool of disks and mirror data on any other available disk. Thus, adding a 12TB disk may be of benefit.

ZFS uses the concept of virtual devices, (aka vDevs), for it's redundancy. Each vDev contains it's own level of redundancy, and use the smallest disk for the size cacluations. So, 4x10TB disks in a Mirror would either be 4 way Mirror in a single vDev, (a bit extreme redundancy since you only get 10TB of space). Or as I said, 2 vDevs of 2 x 10TB disks Mirrored.

If you put a 10TB & 12TB disk in a Mirrored vDev, then yes, that extra space is (temporarily) wasted.

I say "temporarily" wasted because if you later replace the 10TB disk with another 12TB disk, (or larger), you can get that 2TB of extra space.

 

[LemonBag]

Your vDev description is vague, as you state 4x10TB mirror... which implies 2 vDevs of 2 x 10TB disks in a Mirror. Thus, it is irrelevant in what another 2 disk mirror vDev disk size is.

Please note that ZFS does not support irregular stripped Mirrors that some other RAID schemes do. Those others might have a pool of disks and mirror data on any other available disk. Thus, adding a 12TB disk may be of benefit.

ZFS uses the concept of virtual devices, (aka vDevs), for it's redundancy. Each vDev contains it's own level of redundancy, and use the smallest disk for the size cacluations. So, 4x10TB disks in a Mirror would either be 4 way Mirror in a single vDev, (a bit extreme redundancy since you only get 10TB of space). Or as I said, 2 vDevs of 2 x 10TB disks Mirrored.

If you put a 10TB & 12TB disk in a Mirrored vDev, then yes, that extra space is (temporarily) wasted.

I say "temporarily" wasted because if you later replace the 10TB disk with another 12TB disk, (or larger), you can get that 2TB of extra space.

Sorry for the confusion.

I have four 10tb drives. It is 1 vdev, so they are all copies of one another.

 

[LemonBag]

Would there be any downside though to having this "cheap" 12tb drive other than the wasted 2tb? I understand a 4 way mirror is a bit overkill and it does not take the place of a backup. But doing it this way I can rationalize the data is really safe short of user error deletion, weird corruption that spreads everywhere, rogue psu, fire, etc. I only have a few tb of data I'll be putting in this pool.

 

[Arwen]

Weird to have a 4 way Mirror, but okay.

Yes, you can add a cheap 12TB drive as a 5th drive in your Mirror vDev. Having such a wide Mirror vDev will be slower on writes. But, theoretically faster on reads because ZFS should be able to queue up read requests to any sub-mirror.

In someways, it would be better to have it as a separate pool, even if inside the server. Then use it as a copy destination for on-line backups. Or even;

3 x 10TB in a Mirror for your primary pool
1 x 10TB + 1 x 12TB in a Mirror for your online backup pool


Don't forget to setup snapshot tasks. They can't take the place of a backup, but that can help in user error deletions or encrypting ransomware.

 

[LemonBag]

Weird to have a 4 way Mirror, but okay.

Yes, you can add a cheap 12TB drive as a 5th drive in your Mirror vDev. Having such a wide Mirror vDev will be slower on writes. But, theoretically faster on reads because ZFS should be able to queue up read requests to any sub-mirror.

In someways, it would be better to have it as a separate pool, even if inside the server. Then use it as a copy destination for on-line backups. Or even;

3 x 10TB in a Mirror for your primary pool
1 x 10TB + 1 x 12TB in a Mirror for your online backup pool


Don't forget to setup snapshot tasks. They can't take the place of a backup, but that can help in user error deletions or encrypting ransomware.

Snapshots are a good suggestion....I understand making a 1 wide vdev in a pool is a bad idea with multiple vdevs in that same pool. ----> lose 1 vdev, lose your whole data for the pool. But, can I do a 3 way mirror in 1 zpool and then a 2 wide mirror in another zpool where all the snapshots of the 3 wide mirror array go into?

 

[LemonBag]

Like if that a thing ZFS natively does? Or is it matter of just having it on a script to backup to said secondary zpool?

Do you have any reccomended forum threads/general readings for this subject matter you can provide please?

 

[Arwen]

Yes, having a 1 disk pool is generally not recommended. But, can be useful for backup purposes.

And yes, having 2 separate and completely independent pools, in which 1 is a primary and the other is a backup can be helpful.

Depending on what your goal is, you can have a set of snapshot tasks on the primary pool. And during backup, they can be copied automatically to the backup pool, (aka using ZFS Replication / Send & Receive). Thus, even your backup pool can go back in history.

For my own desktop, (which uses ZFS & Linux), I have my home ZFS Dataset snapshotted. I take 24, hourly snapshots, kept 24 hours, (meaning the one at 2pm is removed when a new 2pm snapshot needs to run). And I have 7 daily snapshots, kept 7 days. My scheme is complicated but suitable for me and simple recoveries of deleted files.



No, I don't have any specific recommendations on reading, except the Resources section. See the very top of any forum page for a link to the Resources.