Need more SSD drives, more S-ATA ports. And 2.5 bays

[dnilgreb]

On my primary Truenas (13.0-U3) system, which is running inside an old HP Z420, I have 1 pool consisting of 6x 4TB WD Red drives.
So all 3.5 slots, and all 5.25 slots are taken.

For a boot device I have an SSD. Plus, recently I installed an SSD to house my VMs. Both of those are single drives, with no redundancy.
I would like them to be redundant, and since all my 8 S-ATA ports on the motherboard are taken, I need an HBA.

This should do the trick, if I have understood everything correctly:

LSI 9210-8i 6Gbps SAS HBA FW:P20 9211-8i IT Mode ZFS FreeNAS unRAID 2* SATA | eBay

9210-8I 1 FW p20 9211. 8087 SATA 2.

www.ebay.com

If I buy that, is installation as easy as sticking it in a PCI slot and it "just works" or is there more work and/or other things to consider?
Is it a good choice? The plan is to put a mirrored pool consisting of 2 SSD drives on the HBA. For VMs only. No boot devices will be plugged into it.

And then there is the question of 2.5 bays for the SSD drives. The two I have currently are taped using velcro, one to the case ceiling, and on to the side of the PSU. Doesn´t feel exactly great, so I´d like someting to stick 4 SSDs in, that will fit someweher inside my case.

A lot of questions, if someone could find the time to help sort this out, I´d be (as always) grateful.
Thanks!

 

[Arwen]

A few things:

• You want to use the correct firmware, IT mode and specific version, (I don't have the version handy)
• Some LSI controllers get hot, (which may cause disk transfer errors), so make sure it gets enough cooling. Or add a fan.
• If you have a free 5 1/4" disk bay, you can get a 4 x 2.5" SATA adapter, (some aren't cheap, but work better than cheap ones)
• And yes, adding a LSI SAS HBA should "just work", (if IT mode & specific version of firmware)

You might read the forum post about HBAs, as they sometimes mention specific E-Bay seller(s) that have supplied LSI cards very reliably. Again I don't have that information handy.

 

[dnilgreb]

Ok,

The card I mentioned above is FW P20, which seems to be the one I want.

For heatsink, I found this:

https://www.amazon.com/Gelid-Solutions-PCI-Slot-Holder/dp/B00OAPH76Q/ref=sr_1_1?crid=1XAO7QQPKMSRF&keywords=gelid+pci+fan&qid=1674216560&sprefix=gelid+pci+fan%2Caps%2C193&sr=8-1

Space is not a big problem, and I´d rather not touch the HBA...

I don´t have a free 5,25 slot. I would need to get creative about how to install the SSDs

Does that seem like a sane setup? What would you change?

 

[Arwen]

If the PCIe slot fan is positioned in the next slot over from the LSI heat sink, that may work fine.

But, it was just a warning... you may not need a fan.


Now that you mention odd PCIe devices, I have seen in the past 2.5" disk drive bay in a PCIe slot carrier. Have not looked at them recently... so I don't know what is available. Nor if you have the free PCIe slots for one.

Getting 2.5" SSDs in a workstation where you don't have slots has always been tricky. In most cases you have 2 cabled, power & data to also to route.


As for what I would change, sorry, I can't answer that.

Edit: After a quick check of Amazon, I found these, (their are more);

https://www.amazon.com/Sedna-Express-Extended-Connector-Included/dp/B07L5SPWNV/ref=sr_1_4?crid=3CX0QAX018SN9

https://www.amazon.com/Syba-SY-MRA25023-Slot-Mobile-Black/dp/B0080V73RE/ref=psdc_11548952011_t3_B07L5SPWNV

 

[dnilgreb]

Yes, the plan is to put the fan next to the HBA. And the PCI SSD holder was a great idea. Never heard of it.
Found these:

Amazon.se

I just noticed now that you found some as well. I think this will turn out nicely. THanks for the help

This is what I plan to get:

HBA: https://www.ebay.com/itm/1244651273...d=link&campid=5337967823&toolid=20001&mkevt=1
Fan: https://www.amazon.se/gp/product/B00OAPH76Q/ref=ox_sc_act_title_1?smid=A3OVN94OLIX1JJ&psc=1
pci/ssd https://www.amazon.se/gp/product/B00NOBG1DW/ref=ox_sc_act_title_3?smid=ANU9KP01APNAG&psc=1

and then some cables. Reasonable?

 

[Whattteva]

You might read the forum post about HBAs, as they sometimes mention specific E-Bay seller(s) that have supplied LSI cards very reliably. Again I don't have that information handy.

I think you are probably referring to "The Art of Server". He pre-flashes them and tests them to make sure it's functional and I believe he also puts a fan on it.

 

[Etorix]

If you have a free PCIe slot (and possibly several, possibly with bifurcation), putting the VM pool on NVMe drives may be a better move than adding a HBA for 1-2 drives.

 

[NugentS]

Yes, the plan is to put the fan next to the HBA. And the PCI SSD holder was a great idea. Never heard of it.
Found these:

Amazon.se

I just noticed now that you found some as well. I think this will turn out nicely. THanks for the help

This is what I plan to get:

HBA: https://www.ebay.com/itm/1244651273...d=link&campid=5337967823&toolid=20001&mkevt=1
Fan: https://www.amazon.se/gp/product/B00OAPH76Q/ref=ox_sc_act_title_1?smid=A3OVN94OLIX1JJ&psc=1
pci/ssd https://www.amazon.se/gp/product/B00NOBG1DW/ref=ox_sc_act_title_3?smid=ANU9KP01APNAG&psc=1

and then some cables. Reasonable?

@dnilgreb That seller is in China which brings up the concept of fake LSI cards. I have no idea if that card is real or fake. Just waving a warning at you. When you get it inspect it very carefully to make sure you got what you ordered

 

[Whattteva]

@dnilgreb That seller is in China which brings up the concept of fake LSI cards. I have no idea if that card is real or fake. Just waving a warning at you. When you get it inspect it very carefully to make sure you got what you ordered

I got my HPE H220 from a Chinese seller and it was not a fake. I'd imagine OEM's tend to be less prone to counterfeits similar to how Intel NIC's are more prone to counterfeits over rebranded OEM NIC's.

That being said, it was a very outdated firmware and don't expect to get any kind of support from the reseller. I tried inquiring about firmware version and didn't get much out of him. So... expect to struggle and to have to do your own homework as it most likely won't be a simple plug and play situation. I had to flash mine twice as the original HP firmware refused to boot unless the OPROM was disabled. I suppose disabling it isn't a big deal if you don't plan on booting from the drives, but it still isn't optimal.

 

[dnilgreb]

@dnilgreb That seller is in China which brings up the concept of fake LSI cards. I have no idea if that card is real or fake. Just waving a warning at you. When you get it inspect it very carefully to make sure you got what you ordered

Thanks! I had not thought about that. Maybe I should just follow recommendations and shop from Art of Server.
I found this one there:

Genuine LSI 6Gbps SAS HBA LSI 9211-8i (=9201-8i) P20 IT Mode ZFS FreeNAS unRAID | eBay

I have fully tested them using all 8 SAS lanes. I've used ZFS for many years. I'm familiar with ZFS on Linux, FreeNAS, and many of the technologies commonly used by other enthusiasts. They have been flashed to Avago / LSI 9211-8i IT (Initiator Target) firmware version P20 (specifically 20.00.07.00).

www.ebay.com

More expensive, but seems safer. I guess they do not include cables though, correct?

If you have a free PCIe slot (and possibly several, possibly with bifurcation), putting the VM pool on NVMe drives may be a better move than adding a HBA for 1-2 drives.

That´s a really nice idea. Hadn´t thought of that really. I will look into it, as an alternative.

 

[dnilgreb]

I got my HPE H220 from a Chinese seller and it was not a fake. I'd imagine OEM's tend to be less prone to counterfeits similar to how Intel NIC's are more prone to counterfeits over rebranded OEM NIC's.

That being said, it was a very outdated firmware and don't expect to get any kind of support from the reseller. I tried inquiring about firmware version and didn't get much out of him. So... expect to struggle and to have to do your own homework as it most likely won't be a simple plug and play situation. I had to flash mine twice as the original HP firmware refused to boot unless the OPROM was disabled. I suppose disabling it isn't a big deal if you don't plan on booting from the drives, but it still isn't optimal.

Hm. I guess you posted this as I was writing mine. Didn´t see it until now.
Seems a little risky, and possibly a lot of work, compared to the Art of Server on, right? On the other hand cheaper, of course.
I shall think on it

 

[NugentS]

The other calculation to take into account is if you are going to be using SSD's vs HDD's. SSD's if more than a couple will really push the 9200 LSI Cards. You ought to be using 9300's.

If using HDD's then 9200 is good

 

[dnilgreb]

The other calculation to take into account is if you are going to be using SSD's vs HDD's. SSD's if more than a couple will really push the 9200 LSI Cards. You ought to be using 9300's.

If using HDD's then 9200 is good

When I started this thought, the idea was to put 4 SSDs on the LSI. But maybe I could switch it around, and put my main data pool, consisting of 6x HDD on the LSI card instead, and then connect all SSD-s directly to the mainboard. Better idea?

The 9300 cards are straining my budget, and I´m just not feeling it....

 

[NugentS]

When I started this thought, the idea was to put 4 SSDs on the LSI. But maybe I could switch it around, and put my main data pool, consisting of 6x HDD on the LSI card instead, and then connect all SSD-s directly to the mainboard. Better idea?

The 9300 cards are straining my budget, and I´m just not feeling it....

Better idea - yes

 

[dnilgreb]

Ok,
This is moving slowly forward. I will put my HDD pool on the LSI card, and connect the SSDs to the main board.

Now I am wondering if my 600 w PSU will be enugh.
This will be my new setup

The power consumers will be:
CPU: Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz
RAM: 64GB (4x 16GB) / 1600MHz / DDR3 / ECC
GPU: GPU (unsure of model, but cheap card with small single fan)
LSI https://www.ebay.com/itm/163846248833?
PCI slot Fan https://www.amazon.com/Gelid-Solutions-PCI-Slot-Holder/dp/B00OAPH76Q
Boot: single SSD 256 GB
New pool: 6x SSD ~250 GB each

In time perhaps two more HDDs connected to the LSI. But we can leave that out for now.
Does this look like something that would work with a 600w PSU?

 

[Whattteva]

Should be plenty as long as you buy a name brand one. Stick to 80+ Gold from a reputable brand. Platinum is good, but probably not worth the extra cost. Probably want to research if it comes with enough SATA power cables. You don't want to be custom jerry-rig daisy-chaining non-standard power cables. It can be a fire hazard.

 

[dnilgreb]

Not planning to get a new PSU. The one I have is a 600w one, and the question is whether I can keep it and get all this going.

Well yes, I am looking at getting one of these, to get more SATA power connectors

https://www.amazon.com/Benfei-Power-Y-Splitter-Cable-Inches/dp/B07ML447FG/ref=sr_1_6?crid=172B5ATW7PS8B&keywords=sata+power+splitter&qid=1674572108&sprefix=sata+power+splitter%2Caps%2C163&sr=8-6

and one of these to put an old MOLEX to use (sorry, only found this on swedish amazon, but you get the idea)

Honglei 4-stifts Molex till 15-stifts SATA splitter strömkabel, Gelrhonr LP4 till 5 x 15-stifts SATA strömkonverteringslinje förlängning Y-kabel för hårddisk : Amazon.se: Industriella verktyg & produkter

Honglei 4-stifts Molex till 15-stifts SATA splitter strömkabel, Gelrhonr LP4 till 5 x 15-stifts SATA strömkonverteringslinje förlängning Y-kabel för hårddisk : Amazon.se: Industriella verktyg & produkter

www.amazon.se

 

[Whattteva]

Not planning to get a new PSU. The one I have is a 600w one, and the question is whether I can keep it and get all this going.

Well, you didn't specify what kind of PSU you have. We can't exactly say "Well yeah that's totally okay" cause we have no idea what it is. Whether a PSU is good or not is more than just looking at the wattage rating. What's more important is how much current the individual rails can deliver. How good is the voltage regulation at a given load, etc. etc.

Well yes, I am looking at getting one of these, to get more SATA power connectors

https://www.amazon.com/Benfei-Power-Y-Splitter-Cable-Inches/dp/B07ML447FG/ref=sr_1_6?crid=172B5ATW7PS8B&keywords=sata+power+splitter&qid=1674572108&sprefix=sata+power+splitter%2Caps%2C163&sr=8-6

I think so long as you keep the number of split to no more than 2 you should be alright. Each SATA connector probably can't handle the startup current of more than 2 HDD's without significant power risk as they're only rated up to 4.5A.

and one of these to put an old MOLEX to use (sorry, only found this on swedish amazon, but you get the idea)

Honglei 4-stifts Molex till 15-stifts SATA splitter strömkabel, Gelrhonr LP4 till 5 x 15-stifts SATA strömkonverteringslinje förlängning Y-kabel för hårddisk : Amazon.se: Industriella verktyg & produkter

Honglei 4-stifts Molex till 15-stifts SATA splitter strömkabel, Gelrhonr LP4 till 5 x 15-stifts SATA strömkonverteringslinje förlängning Y-kabel för hårddisk : Amazon.se: Industriella verktyg & produkter

www.amazon.se

Molex has a bit more wiggle room as they are rated up to 10-13A if my memory serves me right. Again, I'd err on the more conservative side here and probably would not do a split more than 4 to be safe.

 

[dnilgreb]

Ok, so new (adjustment to the) plan.

My PSU has two "sets" or cables of SATA connectors, each has three connectors. On the end of one of them is the MOLEX.
I will use these six connectors to power my original six HDDS. They will then be connected to tha LSI card (data).

On that MOLEX, I will place the MOLEX to SATA splitter in my post above. This will power my SSDs.
That is, one single SSD for boot (config backed up regularly) and three (to start) for my VM pool, in RAIDz config.
The SSDs will connect directly to the mainboard (data).

I think I have it all planned out now, any thoughts?

 

[Etorix]

For block storage, including zvols for VMs, mirrors are better than raidz.