Why Raidz?

[quasarlex]

Everywhere, everybody in every blog post looks telling that mirror vdev Is Better than raidz vdev. If that's Always the truth, why do Raidz even exists? What are the pros of using raidz over mirror ( with same number of redundant disks)

 

[danb35]

Everywhere, everybody in every blog post looks telling that mirror vdev Is Better than raidz vdev.

No, they don't. And absolute statements like that will almost always be incorrect.

If that's Always the truth,

It isn't, and nobody who has any clue of what they're talking about says otherwise.

why do Raidz even exists? What are the pros of using raidz over mirror ( with same number of redundant disks)

Better space efficiency is the biggest reason. Six disks in RAIDZ2 gives you (roughly) the capacity of four disks, and can tolerate the loss of any two disks without any loss of data. To get the same capacity and redundancy with mirrors, you'd need twelve disks--4x three-way mirrors. You'll get more IOPS with the mirrors, but those aren't always that important.

The post that's cited most often is this one:

ZFS: You should use mirror vdevs, not RAIDZ. – JRS Systems: the blog

...from nine years ago. And it does make some good points (and some less-good ones--the "OMG resilvering is so stressful" point is bunk, though it's true that mirrors do resilver more quickly), though its application is limited by its own terms, and even as so limited, I think it overstates its conclusion.

For smaller installations, mirrors make a lot of sense. For systems that require lots of IOPS, they're the only way to go (at least if you're stuck using spinners). If you want to be able to expand your pool a little bit at a time, they're a good choice. But if you're wanting to store hundreds of terabytes and don't want waste half (or more) of your disk space on redundancy, and particularly if you don't need maximum performance while doing it, mirrors don't make much sense.

 

[quasarlex]

No, they don't. And absolute statements like that will almost always be incorrect.

It isn't, and nobody who has any clue of what they're talking about says otherwise.

Better space efficiency is the biggest reason. Six disks in RAIDZ2 gives you (roughly) the capacity of four disks, and can tolerate the loss of any two disks without any loss of data. To get the same capacity and redundancy with mirrors, you'd need twelve disks--4x three-way mirrors. You'll get more IOPS with the mirrors, but those aren't always that important.

Mainly the common talk Is:

Pros of mirrors:
- Better performance
- Better scalability
- Faster and safer resilvering
- More reliable with same number of disks (at least for small vdevs like 2, 3, or 4)

Pros of raidz:
- Better storage efficiency

This VS tells that One Is an industry standard, the other Is a cheap alternative. So ?

 

[quasarlex]

For smaller installations, mirrors make a lot of sense. For systems that require lots of IOPS, they're the only way to go (at least if you're stuck using spinners). If you want to be able to expand your pool a little bit at a time, they're a good choice. But if you're wanting to store hundreds of terabytes and don't want waste half (or more) of your disk space on redundancy, and particularly if you don't need maximum performance while doing it, mirrors don't make much sense.

It makes sense. So for home usage mirror Is Better, for Enterprise grade, go with raidz, right?

 

[danb35]

- Faster and safer resilvering

Faster, yes. Safer? meh.

- More reliable with same number of disks

This belongs under RAIDZ, not under mirrors.

- Better scalability

That's going to depend on your definition of "scalability." What do you understand it to mean in this context?

- Better performance

More IOPS, yes--as a result of more vdevs. Better sequential read performance? Sequential write? Other? Performance has many aspects, and not all favor mirrors.

This VS tells that One Is and industry standard

It tells us nothing of the kind, and I'm honestly mystified as to how you'd think any pros/cons comparison would demonstrate an industry standard. Maybe--if it were as lopsided as you seem to think, and if economy were as meaningless as you also seem to think--it would demonstrate that mirrors should be an industry standard, but even then it doesn't demonstrate that they are.

the other Is a cheap alternative.

RAIDZ is usually less expensive for a given combination of capacity and redundancy than mirrors. That difference can easily run into thousands--or even tens of thousands--of dollars. To dismiss that as "a cheap alternative" is frankly nonsensical.

So for home usage mirror Is Better, for Enterprise grade, go with raidz, right?

I don't think that's at all a reasonable conclusion from what I wrote.

 

[quasarlex]

I'm not trying to mistify anything, i Just asked a question. The question hides the belief that raidz exists for a reason and i'm trying to understand that. I still don't

 

[elorimer]

I am in the shallow end of the pool (HO, not SO) so I come at this from a different perspective.

#1 priority is backup (1 local and 1 offsite, at a minimum). #2 is energy use. #3 is how fast do you need to/can get your primary server safely online. So I've approached it with my primary as a mirror because a daily snapshot/replication to the backups is ok; if I were a small law firm, I'd do a three way mirror or Z2 with a spare or two so the online access would be more durable. But otherwise I minimize my disk power usage by increasing the size of the individual disks on the primary as opposed to increasing the number of disks.

 

[Ericloewe]

Is it so hard to imagine that different people in different contexts may choose different setups? Engineering is all about compromises, you can't have everything and you'll end up with nothing if you try.

See also: the fact that parity RAID is a longstanding and popular idea outside of ZFS.

 

[danb35]

raidz exists for a reason and i'm trying to understand that.

Better storage efficiency and better redundancy aren't enough? You already know about these, and they are not small things, especially as the storage requirements increase.

Some use cases (VM storage, for example) clearly call for mirrors. Others (e.g., large-scale bulk storage) clearly call for RAIDZn. Yet others can go either way. Smaller home or office applications are likely in the third category. When I built a NAS for my parents, I put in a mirrored pair of 16 TB spinners--it's plenty of capacity for their needs (including serving as a replication target for some of my data), it's still redundant, and it can be readily expanded with another pair if needed. But a four-disk RAIDZ2 array with smaller disks would also have been a sensible configuration.

 

[Davvo]

Suggested readings:

ZFS Storage Pool Layout

This resource was originally created by user: @Davvo on the TrueNAS Community Forums Archive. https://www.truenas.com/community/resources/zfs-storage-pool-layout.201/download [1] This amazing document, created by iXsystems in February 2022 as a “White Paper”, cleanly explains how to qualify...

www.truenas.com

Assessing the Potential for Data Loss

This guide was written to be read from top to bottom without jumps, with the intent of spreading awareness to both new and experienced users; the author of this document assumes the understanding of the concepts explained in the following...

www.truenas.com

A faster resilver is a safer resilver (less chance of bad things happening). Four drives in RAIDZ2 have better resiliency than two VDEVs in 2-way mirrors; usually the tilting point is 5 drives, but lots of folks prefer the safer RAIDZ2 even with 4 drives, especially in remote systems. If the option is going 3 drives in RAIDZ1 or two (bigger) drives in 2-way mirror, usually the latter is preferred.

It's always a composite choice based on the pool's task (performance and storage requirements, including future expansion), resiliency, and budget.

A 12-wide RAIDZ3 VDEV gives you resiliency and storage space that a MIRROR pool can't realistically achieve.
Assuming 18TB drives, in order to get the same space of a 12-wide RAIDZ3 VDEV while maintaining the same level of redundancy you would need 9 VDEVs in 4-way mirror, for a total of 36 drives: 12 vs 36 is a lot of money saved; moreover, such a greater number of drives actually means an increased probability of pool failure.​

 

[danb35]

Just to expand a bit on the redundancy/efficiency point... To go back to the example I used in my first post on this topic, a six-disk RAIDZ2 vdev has (roughly) the capacity of four disks (less slop, overhead, etc., which I'm going to ignore for the sake of discussion), and can tolerate the failure of any two disks without data loss. I said in that same post that you'd need twelve disks--4x 3-way mirrors--to match that capacity and redundancy using mirrors.

You could look at that and think that I'm mistaken, because that configuration can tolerate the failure of up to eight disks (up to two in each vdev), and you're correct so far as that goes--but if the wrong three disks (i.e., all three disks in one vdev) fail, you lose your pool. A more-common arrangement would be 4x two-disk mirrors, and while that can tolerate the loss of up to four disks (one in each vdev), if the wrong two disks (both disks in one vdev) die, you've lost your pool.

So, if I wanted to make my pool out of mirrors and keep the same redundancy I have now, that'd take 48 disks, which is 12 more than my chassis can hold--that's a bit of a problem. They'd draw twice as much power--also a bit of a problem; my system already idles at 300+ watts.

OTOH, a point in favor of mirrors that I don't think has been mentioned so far is vdev removal--you can remove mirror vdevs from your pool, so long as your pool consists only of single-disk or mirrored vdevs. That does give greater flexibility.

The bottom line is that there's no one best configuration--mirrors are better for some things, RAIDZn for others, and dRAID for yet others (and I'm not familiar enough with dRAID to say what those would be).

 

[joeschmuck]

As I understand it, dRAID is for a high number of drives, typically when exceeding 100 drives. I don't think that is a good option for a home user at this time (I can't afford over 100 drives, nor would I enjoy the electric bill). But I to do not know anything about dRAID, just what little I have read which is not much.

This thread is a duplicate of so many threads here asking about clarity of Mirrors vs. RAIDZ, or most of the time just "This is what I think I am doing, What should I really do?". As clearly stated above, there are pros and cons to each and whoever is designing a system SHOULD know what the use cases are to do this effectively. Maybe a home system has two pools, one RAIDZ2 and one Mirrors, each having a purpose.

Personally, I like RAIDZ2, it makes sense for many home use cases that require redundancy and not a lot of speed (speed is relative), and the most important to a new user most of the time is cost effective for capacity. Nothing locks a person into a specific format as you can, backup your data, reconfigure (change the format), restore your data. It's an easy process, it may not be popular to do this due to the effort involved especially if you have a large amount of data, but it is an option.

I think the people who posted here have done a great job of explaining RAIDZ vs. Mirrors and the Pro's and Con's. I have relearned a bit myself with respect to some of the advantages of a Mirror VDEV. Now I want to build a mirror, just kidding.

 

[danb35]

This thread is a duplicate of so many threads here asking about clarity of Mirrors vs. RAIDZ

Pretty much; the twist here was the opening assertion that everyone in every blog post everywhere says to use mirrors, so why does RAIDZ even exist? The false premise naturally leads to false conclusions.