Need a hardware suggestion for DL380 G8

[Hurmoth]

I'm sure that good answers to this question have been posted a thousand times, I did search and came up with a lot of possibilities, but honestly, I simply don't know enough about SAS controllers and HBAs to make an informed decision.

Bought a DL380 G8 cheap on eBay. It came with a P920 controller, which based on everything I've read, won't work with TrueNAS.

I have 12x4TB drives.

What's the correct card I should get for this? It came with a dual 10GB NIC card, but that's isn't something I need, so I technically have two slots open replacing the P920.

I thought about going with a HP H220 or a LSI 9201-8i. But like I said, I don't know if this is actually the correct direction though.

I really appreciate any help.

 

[jgreco]

P420? P420i? The tech docs for that class of server suggest P420i.

The P420 is a conventional 6Gbps RAID card and you should be able to replace it with an LSI 2008 or 2308 based card that is crossflashed to IT mode. You may have cable-reach issues if you do not use one of the LSI cards with connectors on the end, so you are best off looking for cards such as the LSI 9211-8i, PERC H310, HP H220, and if you buy them used off eBay, I'd say that the extra ten bucks to buy from someone who has already crossflashed the card may be worth it.

I just looked at the P420i and could not discern its method of attachment for the HDD's. If you have a P420i, we may need to make more effort to determine your cabling situation.

 

[Hurmoth]

DOH I mistyped. I'm an idiot. It is a P420. Sorry about that.

Here's a photo of the cabling inside:

 

[jgreco]

You mistyped and I'm experienced at ferreting out details. So yes that's an SFF-8087 cable. Am I seeing correctly that the mainboard has an SFF-8087 socket mounted at a 45 degree angle? Just so you're aware, that might be usable if it is hooked up to mainboard SATA or a SCU or something like that.

Further clarification needed here though.

Is your unit a 12-drive unit? Because I see the TWO SFF-8087 cables on the backplane but not a third. I also see a heatsink, which SUGGESTS that you may have SAS expander chip on the board. I also see the top row of bays outfitted with drives. Is the chassis already stuffed with drives?

You've included no pictures of the P420 but I guess that's superfluous anyways. It's a standard-ish part.

My curiosity here is what's the port on the mainboard. If you're ambitious and willing to wade through excitement and possible dashed hopes, consider trying the following:

Hook one of the SFF-8087 cables up to that 45' connector on the mainboard.

Install a copy of TrueNAS Core (not Scale). If you are UNIX-experienced you may be able to get away with this from the installer media, a little bit of singleuser mode, manual dhclient, and netcat without actually installing it.

What I want to see are the contents of /var/run/dmesg.boot

In particular if it scans your drives under the isci driver. For example

Code:

isci0: <Intel(R) C600 Series Chipset SAS Controller (SATA mode)> port 0x4000-0x40ff mem 0xe7afc000-0xe7afffff,0xe7400000-0xe77fffff irq 18 at device 0.0 on pci3
da3 at isci0 bus 0 scbus3 target 0 lun 0
da3: <ATA ST6000DX000-1H21 CC46> Fixed Direct Access SPC-3 SCSI device
da3: 5723166MB (11721045168 512 byte sectors)
da4 at isci0 bus 0 scbus3 target 1 lun 0
da4: <ATA ST6000DX000-1H21 CC46> Fixed Direct Access SPC-3 SCSI device
da4: 300.000MB/s transfers
da4: 5723166MB (11721045168 512 byte sectors)

The upside to this is that if so, this driver is 100% compatible with Core and should also be 100% compatible with Scale. The downside is that it is 3Gbps, so you only get 12Gbps out to your SAS Expander. That is somewhat less than what your drives might be able to push under optimum circumstances in a 12-drive chassis.

This is totally optional on your part. If you do this today and you decide on the H220 at a later point, it will just swap over to the new controller with no drama, and you can get full 24Gbps on each SFF-8087 that way (though you really only need one for 12 HDD drives).

 

[Hurmoth]

Hooked up one of the cables to the port on the motherboard and attempted to boot, but TrueNAS Scale never gets beyond an error message COMREST failed errno=-32.

 

[Hurmoth]

You mistyped and I'm experienced at ferreting out details. So yes that's an SFF-8087 cable. Am I seeing correctly that the mainboard has an SFF-8087 socket mounted at a 45 degree angle? Just so you're aware, that might be usable if it is hooked up to mainboard SATA or a SCU or something like that.

Further clarification needed here though.

Is your unit a 12-drive unit? Because I see the TWO SFF-8087 cables on the backplane but not a third. I also see a heatsink, which SUGGESTS that you may have SAS expander chip on the board. I also see the top row of bays outfitted with drives. Is the chassis already stuffed with drives?

You've included no pictures of the P420 but I guess that's superfluous anyways. It's a standard-ish part.

My curiosity here is what's the port on the mainboard. If you're ambitious and willing to wade through excitement and possible dashed hopes, consider trying the following:

Hook one of the SFF-8087 cables up to that 45' connector on the mainboard.

Install a copy of TrueNAS Core (not Scale). If you are UNIX-experienced you may be able to get away with this from the installer media, a little bit of singleuser mode, manual dhclient, and netcat without actually installing it.

What I want to see are the contents of /var/run/dmesg.boot

In particular if it scans your drives under the isci driver. For example

Code:

isci0: <Intel(R) C600 Series Chipset SAS Controller (SATA mode)> port 0x4000-0x40ff mem 0xe7afc000-0xe7afffff,0xe7400000-0xe77fffff irq 18 at device 0.0 on pci3
da3 at isci0 bus 0 scbus3 target 0 lun 0
da3: <ATA ST6000DX000-1H21 CC46> Fixed Direct Access SPC-3 SCSI device
da3: 5723166MB (11721045168 512 byte sectors)
da4 at isci0 bus 0 scbus3 target 1 lun 0
da4: <ATA ST6000DX000-1H21 CC46> Fixed Direct Access SPC-3 SCSI device
da4: 300.000MB/s transfers
da4: 5723166MB (11721045168 512 byte sectors)

The upside to this is that if so, this driver is 100% compatible with Core and should also be 100% compatible with Scale. The downside is that it is 3Gbps, so you only get 12Gbps out to your SAS Expander. That is somewhat less than what your drives might be able to push under optimum circumstances in a 12-drive chassis.

This is totally optional on your part. If you do this today and you decide on the H220 at a later point, it will just swap over to the new controller with no drama, and you can get full 24Gbps on each SFF-8087 that way (though you really only need one for 12 HDD drives).

Sorry, I also meant to respond to the your question, yes it is a 12-drive unit and all bays are populated.

 

[melloa]

Interesting conversation.
I got one DL380 three, four years ago to run some tests and was able to replace the original controller with a LSI. I believe, to @jgreco 's point, that I had to get longer cables. It is still running ESXi passing through the HBA to a TrueNAS VM. Was working fine until was replaced with a X10 last month, so run a good 3 to 4 years.
Good luck with your tests.

 

[jgreco]

Yeah, I can't help that much with the Linux. If you can install Core like I asked, I have a much better idea of what I'm looking at.

 

[Hurmoth]

Yeah, I can't help that much with the Linux. If you can install Core like I asked, I have a much better idea of what I'm looking at.

Scale was already installed and I thought I'd give it a shot to see what might happen. I've got Core downloaded, just need to get it installed.

 

[melloa]

Hooked up one of the cables to the port on the motherboard and attempted to boot, but TrueNAS Scale never gets beyond an error message COMREST failed errno=-32.

In my case I added a SSD inside the chassis, connected to a SATA port and used all drives for the NAS. Check for power and SATA options available for that too. (I had to get my wire cutter and soldering iron to built a power cable for mine ...)

 

[Hurmoth]

During the launch process to install Core, I got a bunch of errors immediately during boot. When both cables are plugged into the P420 I don't get this, but when one cable is plugged directly into the motherboard I do:

 

[jgreco]

Ah, see, that I grok. I'm pretty sure those are just AHCI ports, so they won't work with the SAS expander. Sorry. But as @melloa says, they could be used for boot drives with a SFF-8087 breakout cable.

 

[Hurmoth]

Ah, see, that I grok. I'm pretty sure those are just AHCI ports, so they won't work with the SAS expander. Sorry. But as @melloa says, they could be used for boot drives with a SFF-8087 breakout cable.

Appreciate the feedback. Any of those earlier cards recommended would work? Do you have one that you'd recommend over another for this?

 

[jgreco]

The H220 is least likely to be "problematic" but they should all be close matches.

 

[sybreeder]

I had DL380e G8 and integrated AHCI controller was limited to 4 drives with sas expander(another 2 directly connected to mobo). When i plugged in HBA Fans started to freakout. Because of that i sold it. It's unusable with cards other than HP branded :( That's main reason why those servers are so cheap. I also had P420 and with plain raid it worked fine in Server 2019
Now I've configured Supermicro X9DRH-7TF with ESXi with virtualized TrueNAS Scale with LSI2208 converted to 2308-IT. Works great so far.
Unless noise is not an issue for you that HO server will work just fine with HBA remember that rear cages are connected to front by default. You have to connect them to motherboard directly. connect boot ssd to rear cage.
then Put HBA and connect it to from backplane. And it will work just fine.

 

[jgreco]

AHCI controller was limited to 4 drives with sas expander

An SAS expander is incompatible with an AHCI controller.

 

[sybreeder]

An SAS expander is incompatible with an AHCI controller.

In documentation says you can connect 4 to front and 2 to back. I've tested it and it worked.
It's integrated B120i anyway HBA connected to front will work just fine

 

[jgreco]

To a SAS expander? Okay, whatevs. I don't believe it. And it didn't work for the poster above. What do you do with the other eight bays?

 

[Hurmoth]

Truly appreciate the responses. Going to pick up a HP H220 from eBay.

 

[ljvb]

I have a DL380p G8, 25 sff drive version. If you update the bios to the latest one (you have to get creative to get it since HP hides crap behind a support contract, except for critical security related updates), the fans won't freak out when you use non HP cards. At one point, I had 2 Dell and an IBM external card (I also have a 25 drive external storage chassis). I have consolidated a little, and am down to just the DL380 with a single HP H240, which you can find on ebay cheap enough. The backplane on the 380p G8 (at least in mine), supports 12Gb drives. Network is a Flexlom card, 10GB with iscsi hardware offload.. which I am trying to figure out now whether Scale takes advantage of that or not.

The onboard connector is for the built in sata controller which is junk, and generally only used for the CD drive if yours had that option (not available when you go with the 25 drive cage). mine also came with a P420, it is also junk.

I actually boot mine from the built in SD card (which uses the same built in controller, which is why I cannot disable it in the BIOS) directly into scale.

It makes a decent storage server, but it is power hungry.. like seriously power hungry.