Raidz2 Number of disks

afmiller

Contributor
Joined
Dec 11, 2013
Messages
106
When using raidz2 what is the minimum number of drives you should use to get decent performance while not spending a fortune on drives? From what I've seen people seem to have varying opinions. I know 4 is the min, but seems like a waste, is 5+ ok?
 

JaimieV

Guru
Joined
Oct 12, 2012
Messages
742
With ZFS, you gain performance not by adding disks (it helps some for reads but not writes) but by adding vdevs (helps with both). Take a look at this link for details.

You haven't given enough parameters to get useful advice, or whether you want direct practical advice or theoretical.

If you want practical advice, give us some clues:
What is your budget?
What is your data storage need? Both purpose - commercial/domestic - and quantity.
What is your disk enclosure - how many drives will it take?
What's your system spec - mobo/HBA/CPU/RAM?
What's your network speed - single GigE? Some aggregated GigE's? 10gigE? No point carefully optimising for speed when a single disk will saturate your gigE link.
How will you be backing up your system?
 

afmiller

Contributor
Joined
Dec 11, 2013
Messages
106
Jamie,
Currently I am running 3 vdevs that are 5 disks each. 3TB drives, I am upgrading to the He10, trying to figure out if I want/need to spend $ and make them 6 drives vs keeping the 5. So in the long run saving $1k

I was looking at around 1k for drives for budget,
Home / media storage
I am currently running a supermicro 846 24bay. Looking at eventually getting the 847 that has the more drive bays.
The hardware is a older. Its an x8 supermicro board, Intel(R) Xeon(R) CPU E5620 @ 2.40GHz 12 Cores in total with 48GB ram
Network speed to switch is 10Gig fiber.
 

JaimieV

Guru
Joined
Oct 12, 2012
Messages
742
Great! Thanks for the backstory. By a useful coincidence we have pretty similar hardware in HDD/cpu/RAM/network terms.

With my pool of three vdevs, each 4x4TB RAIDZ1, I get about 900meg/sec large-file read/write speed with caching restricted to a couple of gig - it's faster and more IOPS when I let the cache go to the default of most RAM. So with three vdevs we don't need to worry overmuch about speeds - particularly if your clients aren't also 10gigE, or only one is. 10gigE doesn't practically net you much over 1gigE in general daily usage, and definitely isn't of any benefit for media streaming (until we go to 16K resolutions!) but it is good for doing the replication to a second FreeNAS and other bulk transfers.

As the link I gave explains, you don't gain anything significant by making your vdevs have more disks each, the big changes happen when adding vdevs. So whether to make your new pool out of three vdevs of 5 disks in RAIDZ2 or three vdevs of 6 disks in RAIDZ2 is a cost and capacity decision, not a speed one.

Basically, if you can afford the higher outlay now, go for it.

Consider also that your new vdevs will be much, much larger than the old, and the He10's will also be about twice as fast as the old 3TBs natively due to the head count and data density. You can take advantage of this to reduce upfront costs by building the new pool from two vdevs and add the third vdev later when you have the money and need the capacity and/or extra speed. Or even initially start with one vdev if that has enough capacity, and see whether that is quick enough to support your needs in the short term. You've got the spare slots so you can experiment with that before ordering the second batch of drives.
 
Top