Intel Xeon Supermicro X12 build

[Gaspetaahl]

Hello,
I am currently building a new system for data sharing and maybe some other applications or VMs. I set together the following list and would appreciate any recommendations:

• Motherboard: Supermicro X12STL-IF retail
• CPU: Intel Xeon E-2336, 6C/12T, 2.90-4.80GHz
• PSU: Corsair SF Series Platinum SF600 600W SFX
• Case: SilverStone Case Storage DS380, Mini-ITX
• RAM: ?
• HDD: 2x Western Digital WD Red Pro 22TB, SATA 6Gb/s

Can someone reccommend me some RAM? The list from Supermicro only reccomend this one: https://store.supermicro.com/32gb-ddr4-2933-mem-dr432l-hl01-eu29.html
which isn't available anywhere else than directly from supermicro. I would prefer to order from German online shops.

 

[Whattteva]

Just like what your motherboard specs say. You need DDR4 UDIMM.

 

[joeschmuck]

I think you should really define what type of data sharing will be done, is it a small amount of SMB sharing such as Plex Audio/Video streaming, sharing a dataset of files that 12 people will need to access, office environment. Being more specific helps us be more objective.

The storage you have selected, two 22TB hard drives? I have to assume this will be in a Mirror configuration. If the data is important then I'm not sure having a two drive mirror is wise, maybe a 3-way mirror? But I'm not a ZFS Pool design professional. If the data is not important and data loss would be inconsequential, a mirror of drives that size is fine. Geez, I have no idea how many days that would take to replace a drive if you were say 80% full. That is the run against using very large drives, long resilver times.

Also, depending on use case, the amount of RAM can be critical. If it's a home system and you do not expect much out of it, 16GB ECC RAM is fine, but I'd recommend 32GB RAM for any new system build. It provides a good buffer in case you decide to do more. You did say you wanted to do a few VM's, so add additional RAM for that. One thing you never want to do is use the SWAP Space on the hard drives, which means you ran out of memory. If you are more precise on your system use, we could be better at stating how much RAM to get.

I noticed you didn't mention a boot drive, You have lots of options, NVMe, DOM, or SATA SSD. I'm not a fan of DOM's so I'd go with the SATA SSD myself, either one is inexpensive. NVMe's are not nearly as inexpensive. you might consider saving the NVMe slot for a larger drive capacity for running VM's from. You have options.

Good luck on your build.

 

[Gaspetaahl]

Thanks for the replies so far.
The NAS is intended for a small team of 4 people in an office. We want to store a few hundred GB of data on some days and access that irregular.

I am aware that I need DDR4 UDIMMs. However I thought, that those RAM compatibility lists exist for a reason and for previous builds I selected DIMMs from that list. Can I just choose any ECC DDR4 UDIMMs? I would get at least 32GB and update later if needed.

Does anyone else have thoughts on the 2 22TB Mirror config? The data is rather important to me. My reasoning there was to start with a small number of drives that I could upgrade later as more storage is needed. If that does not make sense I would choose a raidz2 or raidz3 and buy more and smaller drives.

For the boot drive a SATA SSD seems reasonable.

 

[Etorix]

If the data is important, then a 2-way mirror is not secure enough, for the same reason that RAID5/raidz1 is advised against: After the complete loss of one drive, there would be too much data to resilver without redundancy. Go for 3-way mirror or raidz2 (≥4 drives).

You may want a pair of SSD for a VM/application pool, especially if you put data on raidz2.
You may also want to boot from a NVMe drive to save a SATA port—but you'll still be short of 2 ports to cable the 8 LFF bays (not to mention the 4 SFF cage).

I found the Silverstone DS-380 case horrible to work with and not capable of cooling HDDs while being quiet. Throwing in a hot CPU will not help.

 

[Whattteva]

If the data is important, then a 2-way mirror is not secure enough, for the same reason that RAID5/raidz1 is advised against: After the complete loss of one drive, there would be too much data to resilver without redundancy. Go for 3-way mirror or raidz2 (≥4 drives).

This is not a really good comparison as it is only true for 2 disks. It is definitely not true as the number of drives go up. RAIDZ1 is way more prone to complete loss than a bunch of striped mirrors as the number of disks go up beyond 3.

 

[Gaspetaahl]

Thanks for the feedback. I'll reconsider the drives and probably get more and do a raidz2.
Is there any other case you can recommend? I'd like to have hot-swappable bays and noise is important to me. I would also go rack mountable but there noise was my main concern.

 

[Whattteva]

Thanks for the feedback. I'll reconsider the drives and probably get more and do a raidz2.
Is there any other case you can recommend? I'd like to have hot-swappable bays and noise is important to me. I would also go rack mountable but there noise was my main concern.

Is hot swap really that much of a concern to you? I'm willing to bet 90% of home users don't touch their HDD until it needs to be replaced and can afford a 1-2 hour downtime.

 

[Gaspetaahl]

Im willing to compromise there. It's not home use but a very small office.

 

[Whattteva]

Im willing to compromise there. It's not home use but a very small office.

I was only asking because hot-swap bays would limit your selection of chassis considerably.

 

[Whattteva]

Fractal Design Node 804 is pretty popular among some forum members.
Silverstone CS380 has the hot swap bays you like, but I have heard that it has bad thermals.

You should tell us how many drives you're looking to put into this thing.

 

[Gaspetaahl]

I already have a Node 804 at home. I really dislike it's waste of space. The CS380 seems quite similar to the DS380 but with additional 5,25" bayes that I don't need.
Silverstone has some 19" ones. Do you know how the airflow and noise is with those?

 

[Whattteva]

I already have a Node 804 at home. I really dislike it's waste of space. The CS380 seems quite similar to the DS380 but with additional 5,25" bayes that I don't need.
Silverstone has some 19" ones. Do you know how the airflow and noise is with those?

No experience with Silverstone stuff. My primary system in my signature uses Fractal Design Define 7 XL. It has good thermals, good noise dampening and a ton of space for you to turn it virtually into anything you want. It just lacks hot-swap bays, which I don't care for since it's a server that just sits in a corner and collects dust for years until I need to service it.

It is, however, massive and heavy, which sucked when I initially picked it up from shipping. Once setup though, it just sits there for eons, so it's not a concern for me. Anyways, for my use case (4x 6TB drives) and a single socket Xeon Silver 4210T, it's whisper quiet enough for me to run in my bedroom. It's actually quieter than my gaming PC due to it being able to run big 140mm fans and just the sheer size of it allows me to space the drives apart that I don't need the fans running at high speeds.

 

[Etorix]

This is not a really good comparison as it is only true for 2 disks.

Going slightly off-topic, but this is an accurate comparaison based exactly on the same scenario and the same math: One drive fails in the vdev, the vdev has to be resilvered without redundancy to compensate in case an URE occurs. The probability for an URE depends only on the amount of data to be resilvered, irrespective whether this data is spread over many raidz1 disks or comes from a single huge disk in a mirror. And, for me, the risk from drives with an URE rate of 1E-14 is not acceptable. Full stop.
The additional risk of a second disk failing during resilver is not even considered.

 

[Whattteva]

You are comparing a RAIDZ1 vs mirrors. One can withstand just 1 disk failure whereas the other can withstand multiple disks as the number of disks go up as long as it isn't in the save vdev. That doesn't even take into account that resilvering a RAIDZ regardless of the the Z number is far more IO intensive operation than resilvering a simple mirrored vdev.

In terms of the risk for complete data loss, the numbers aren't even remotely comparable when you compare say... a 6-disk RAIDZ1 vs 3 mirror vdevs.

 

[joeschmuck]

4 people in an office.

This is a game changer.

My reasoning there was to start with a small number of drives that I could upgrade later as more storage is needed.

You cannot just add drives to add more storage, er, you can but it may not be the way you want it to be. If you started with a mirror set and wanted to change to RAIDZ2 for example. You would have to destroy your mirror and then create the RAIDZ2 thus loosing all your data. Or the other way to do it is to add additional hard drives to create a separate RAIDZ and then transfer the mirror pool over. It's not fun if you want to keep all the paths the same. The office environment means to me that your data is very important. This means redundancy.

A few comments for an office NAS:
1) Max out the RAM, 64GB ECC RAM if possible. you can only put two stick in this motherboard so make it count.
2) Even a small NAS can fit into a Mid Size case. Build a RAIDZ2 or RAIDZ3. Since you are looking for approx 22TB then I'd build it with seven 6TB disks in a RAIDZ3 configuration. Think about if you lost your data what that would do to your business. If this is not as important data then you can reduce by 1 drive and make a RAIDZ2. This allows you to upgrade at a later date by replacing the drives with larger ones and maintaining your pool geometry. Using 10TB drives in five years would give you about 33TB of storage.
3) Design the system correctly from the start and you will enjoy it much more.
4) Hot Swap Drive Bays are not worth it in this small system unless you still plan to shut the system down before removing/installing a drive. Never Hot Swap if possible, you are asking for trouble unless you have high quality hardware. I'm not spending over $100 on a drive bay. Not me. Spend your office money better, get the 64GB RAM.

You have a lot of comments and they really are all good ones and meant to influence you to purchase the right system for your needs. No one likes it when a person builds a system that does not meet the needs and they have a bad experience. Or in other words, do not cut corners. This isn't a home system and should be treated properly. Just like it should be burned in properly as well.

My last words: RAIDZ2 or RAIDZ3, 6 to 8 hard drives or appropriate size to reach your capacity goal. I hope that goal covers the next 3 years if possible. 5200/5400 RPM drives preferred, not 7200 RPM (due to heat generation), a High airflow case (not a loud one), max out the RAM. Proper power supply for what you end up buying. I'd also use Core, not Scale, only due to long term use and proven stability. You can upgrade to Scale in a year or two if desired.

I do with you the best of luck and hope you post the end results and how it works.

 

[Gaspetaahl]

Thanks for all the replies. Some were really helpful. However Im still struggling with the case.
As I said I want something small and quiet with good airflow, rackmountable would also be nice but those are usually not quiet.
Any recommondations?

 

[ChrisRJ]

Your case desire is unrealistic. Small and quiet directly collides with good airflow. What is the reason behind the small size? Why is it important in an office setting?

 

[ChrisRJ]

You are comparing a RAIDZ1 vs mirrors. One can withstand just 1 disk failure whereas the other can withstand multiple disks as the number of disks go up as long as it isn't in the save vdev.

If we are talking about 2-way mirrors, why is it relevant that I can theoretically loose half my drives without data loss? I cannot choose which drives goes bad next. In my view this is about a purely pessimistic scenario, where the "wrong" second drive failing will kill my pool.

 

[Etorix]

Thanks for all the replies. Some were really helpful. However Im still struggling with the case.
As I said I want something small and quiet with good airflow, rackmountable would also be nice but those are usually not quiet.
Any recommondations?

"Quiet" puts a hard limit on the number of spinning drives. How much is a personal matter. My personal limit is 6, in a Node 304; your mileage may vary…

"Rackmountable" is not what I would call "small". Stock racks are indeed not meant to be quiet but it has been suggested that it might be possible to provide sufficient cooling with large and "quiet" fans in a 4U chassis if filling only half of the HDD slots in a chequered pattern.

Overall, I'd say it's: Small. Quiet. Rackmount. Pick any two.