SOLVED Find info on your LSI SAS HBA

[sfcredfox]

Greetings all,

I'm interested in any commands that can be run from CLI that display hardware/driver information regarding your SAS HBA. Searching yields a lot of info on flashing it, but not finding it/displaying it.

I have seen people running a 'sas2flash -listall', which concerns me a little because my system responds with this:
No LSI SAS adapters found! Limited Command Set Available!
ERROR: Command Not allowed without an adapter!
ERROR: Couldn't Create Command -listall
Exiting Program.

Can anyone share any available commands to find your hardware HBA info like firmware? And the driver the system is using for it?

I know from reading posts that P16 is the standard for certain LSI HBAs. I'd like to see where I'm at and see what should be running on a 3801 chip.

Thanks!

 

[cyberjock]

sas2flash is your friend. Mostly I use sas2flash -listall but there's others that give more detailed info too. ;)

 

[sfcredfox]

sas2flash is your friend. Mostly I use sas2flash -listall but there's others that give more detailed info too. ;)

Since I tried that and got a negative result, does that mean my particular card is not supported? Or on such a wrong/old firmware that sas2flash won't work with it?

I read never to mismatch the driver/firmware (I assume)? I wanted to compare the two I guess if that's a thing. Maybe the only way to see the information I seek is to reboot the system and look at HBAs BIOS?

I was looking into camcontrol devlist also, but it didn't seem to be going in the right direction.

 

[depasseg]

I used 'dmesg | grep mpr' to find the driver and firmware versions (SAS3). I think the previous SAS2 version use the mps driver so 'dmesg | grep mps'.

I got a line that looked like this:
mpr0: Firmware: 05.00.00.00, Driver: 05.255.05.00-fbsd

 

[sfcredfox]

I used 'dmesg | grep mpr' to find the driver and firmware versions (SAS3). I think the previous SAS2 version use the mps driver so 'dmesg | grep mps'.

I got a line that looked like this:
mpr0: Firmware: 05.00.00.00, Driver: 05.255.05.00-fbsd

You got me on the right track, good call.

My system is using MPT, and it listed all the drives under it.

'dmesg | grep mpt' got this:
mpt0: MPI Version=1.5.20.0

Unfortunately, the card is not specifically supported by the driver on its man page, so this is not good news.

I just flashed the 3801E up to P16, but am I right in thinking my card's firmware (FW- 1.26.00 BIOS-6.24.00) is not a correct match for the driver it's using? MPI 1.5.2?

Is there any fix if the card isn't specifically listed on the MPT/MPS/MPR/More? Maybe I'm just screwed because this card isn't a 'best choice' for FreeBSD/FreeNAS? I guess I can always go to the trusty M1015...

 

[depasseg]

That's beyond my knowledge, sorry.

 

[Ericloewe]

mpt? Oh, that's bad, that's a RAID driver.

You want mps (SAS2) or mpr (SAS3). If a card doesn't work with those drivers, it's a bad sign...

Oh, wait, that's an SAS1 card. Yeah, I doubt the driver is P16, you'll have to check that yourself. Also, SAS2flash probably won't work, you'd need the SAS1flash, or whatever it's called.

 

[cyberjock]

So the fact you are seeing mpt is a red-flag that you are using hardware RAID. So yeah, you need to look at better hardware. I'd recommend the M1015. Do note that since you are using hardware RAID you aren't going to be able to plug the drives into the M1015 and they "just work" with your data and all. You'll have to create a new pool and such (or move disks one at a time from the current controller to another controller and resilver them one at a time.

 

[sfcredfox]

It's not hardware RAID unless it's magic and set itself up, the card's flashed with P16 IT mode. Am I missing something? Are you saying because it's running that particular driver, you know that it will not export/import properly?

I'd like to do the 1015, but I'll have to waste my last slot putting in an interior/exterior converter to connect the disk enclosure, which sucks.

Do we have a 1015 type card that has external 8088 ports before I buy the M1015?

 

[Ericloewe]

It's not hardware RAID unless it's magic and set itself up, the card's flashed with P16 IT mode. Am I missing something? Are you saying because it's running that particular driver, you know that it will not export/import properly?

I'd like to do the 1015, but I'll have to waste my last slot putting in an interior/exterior converter to connect the disk enclosure, which sucks.

Do we have a 1015 type card that has external 8088 ports before I buy the M1015?

Sure, any of the LSI SAS 9207-8e, LSI SAS 9211-8e plus rebrands will work just fine. Notice the 'e's instead of 'i's.

 

[cyberjock]

It's not hardware RAID unless it's magic and set itself up, the card's flashed with P16 IT mode. Am I missing something? Are you saying because it's running that particular driver, you know that it will not export/import properly?

I'd like to do the 1015, but I'll have to waste my last slot putting in an interior/exterior converter to connect the disk enclosure, which sucks.

Do we have a 1015 type card that has external 8088 ports before I buy the M1015?

You're in a whole different realm. sas2flash *is* the appropriate tool for cards that use the LSI P16 IT mode firmware that we tout so much around here. In fact, that's one of the tools you use in reflashing it (http://www.servethehome.com/ibm-serveraid-m1015-part-4/).

Then you said things like this:

'dmesg | grep mpt' got this:
mpt0: MPI Version=1.5.20.0

I just flashed the 3801E up to P16, but am I right in thinking my card's firmware (FW- 1.26.00 BIOS-6.24.00) is not a correct match for the driver it's using? MPI 1.5.2?

And you should recognize that the hardware isn't using the firmware you think you flashed. Not sure if you are confusing cards or not, but that's not a SAS2008 based card, so it's under a different set of rules from the M1015 (which is the SAS2008). Also noticed that your firmware version is 1.26.00. That's not a firmware version that matches typical SAS2008 chips. Here's my output...

Code:

# sas2flash -listall
LSI Corporation SAS2 Flash Utility
Version 14.00.00.00 (2012.07.04)
Copyright (c) 2008-2012 LSI Corporation. All rights reserved

  Adapter Selected is a LSI SAS: SAS2008(B2)

Num  Ctlr  FW Ver  NVDATA  x86-BIOS  PCI Addr
----------------------------------------------------------------------------

0  SAS2008(B2)  16.00.00.00  10.00.00.06  No Image  00:01:00:00

  Finished Processing Commands Successfully.
  Exiting SAS2Flash.

Notice it's 4 sets of numbers, and not a 1.xx.xx. That' my hint that I'm doing SAS2008.

The 3801E is a different chipset than the M1015, uses different drivers, and is govern by a different set of rules. The SAS3Gb/sec stuff (not SAS3) is old and basically not really used anymore (or recommended because it often has limits like 2TB drives and such).

Gotta keep all of these cards in perspective.

The M1015 is a SAS2008 chip:

- Which is not the same as the 3801E (different chipset and therefore different drivers and firmware, 3Gb/sec SAS) (this uses the mpt drivers)
- Which is the same chipset as the 9207-8e and 9211-8e and 9211-8i, but has external connectors (same chipset and therefore same drivers and firmware and can be reflashed to IT mode) (this uses the mps driver, phase 16 in 9.2.x and 9.3)
- Which is not the same as the new LSI 9300 series of controllers (different chipset and therefore different drivers and firmware, the reliablity of which has been pretty questionable thus far but people keep wanting it for 12Gb SAS despite there not being a performance reason for such) (this uses the mpr driver).

The easy way to remember these drivers is to use the letters in reverse, r, s, t is 12, 6, and 3Gb/sec respectively. Only the 6Gb/sec is really "well supported" (and inexpensive to implement), which is why the M1015 is the recommended card). The 3Gb/sec can work, but there's a laundry list of reasons you wouldn't want to use it. 12Gb/sec doesn't seem to work particularly well, and considering there is no performance gain with using it, seems silly to use something with only downsides.

Hopefully I'm not saying anything wrong as I have a headache developing. :p

 

[sfcredfox]

Sure, any of the LSI SAS 9207-8e, LSI SAS 9211-8e plus rebrands will work just fine. Notice the 'e's instead of 'i's.

Sweet.

I was looking at the LSI 9207-8e, but I didn't see it specifically listed under the drivers mentioned above.

Does that matter?

 

[sfcredfox]

you should recognize that the hardware isn't using the firmware you think you flashed.

I'm confused by this. When the machine boots, that's what the card shows. I flashed it with 3801(B3).

Having said that, the whole reason I'm concerned is because the sas2flash command keeps telling me there are no cards found. That's why I figured FreeBSD hates it. It's working fine, but not the best choice it seems. Why else wouldn't sas2flash work?

I think you're saying the 9207-8e uses the same driver (mpt) and the 3801e?

 

[sfcredfox]

Crap, Ok I re-read that last post and you said the 9207 DOES use the same driver as the M1015 (rebranded LSI).

*edit* - They use the same driver because they are based on the same chip (the 2008). The 2008 is listed on the man page for MPS, so it's #1 - listed for support, and #2 - known to work well based on people's use of it.

 

[Ericloewe]

Actually, the 9207 series uses the SAS 2308, which is the 2008's big brother with PCI-e 3.0 support and a faster processor. It also takes more devices than the 2008, but it's an academic difference (256 vs 1024 or thereabouts).
The SAS 2308 is fully software compatible with the 2008, as they are both based off the same SAS2 design. In IT mode, they both use the mps driver.

 

[sfcredfox]

Thanks, looks like my next purchase will be the 9207-8e for the enclosure. I saw that we should be using P16 with this? So I will be downloading the P16 IT mode to flash it with once I get it.

 

[mjws00]

Your card DOES have IT mode. And it does have p16 IT firmware available. (I have no idea which version matches the mpt driver in bsd). It may even be p20 as the card and driver are older.

The card is based on the 1068E which was a staple 2009-2010ish. You have a pretty high chance of things being stable for a very long time and the kinks being out. The downside is you likely have a 2TB limit, and the card is fairly slow. The dirty math splits bandwidth of 8 drives to 65MB/s ish. So new fast drives can outrun it.

If it suits you, use it. If not, alternatives are cheap. It is pretty easy to build a pool. Then transfer it to another box and verify that the controller isn't adding some kind of jbod rubbish to the drives. (It won't be if things are set right.) If it passes smart data nicely, its really the size limit and possibly speed that will be the downside. Not instability or risk of loss.

The sas2flash util is a no go. mtputil? might work.

 

[sfcredfox]

Your card DOES have IT mode. And it does have p16 IT firmware available. (I have no idea which version matches the mpt driver in bsd). It may even be p20 as the card and driver are older.

The card is based on the 1068E which was a staple 2009-2010ish. You have a pretty high chance of things being stable for a very long time and the kinks being out. The downside is you likely have a 2TB limit, and the card is fairly slow. The dirty math splits bandwidth of 8 drives to 65MB/s ish. So new fast drives can outrun it.

If it suits you, use it. If not, alternatives are cheap. It is pretty easy to build a pool. Then transfer it to another box and verify that the controller isn't adding some kind of jbod rubbish to the drives. (It won't be if things are set right.) If it passes smart data nicely, its really the size limit and possibly speed that will be the downside. Not instability or risk of loss.

The sas2flash util is a no go. mtputil? might work.

Nice dude, thanks. I was able to get results with mptutil:
mpt0 Adapter:
Board Name: SAS3801E
Board Assembly: L3-01123-04C
Chip Name: C1068E
Chip Revision: UNUSED
RAID Levels: none

Basically just confirms what you said, but I didn't know about it before.

Thanks for the further info, just glad to see something positive about that adapter. I can definitely see the slowness. I can't seem to get any individual drives to go over about 30MB/s doing a DD/2048K/50K. Not sure if there are any test to tax them harder, but the card seems to be working just well enough for now. It's running both of my pools listed in sig.

 

[mjws00]

That is slow as fsck. But still enough to max a single 1Gb link in all likelihood. If you can push 110MB/s on a link... who cares.

 

[sfcredfox]

That is slow as fsck. But still enough to max a single 1Gb link in all likelihood. If you can push 110MB/s on a link... who cares.

Yeah. The pool does 400-500 MB/s running the pevious mentioned DD, only by virtue of how many old disks I have. It's probably 'eh' on handling the random workload from VMs since it's not using mirrors, but again, it slide over the finish line with a solid 'good enough'.

I don't know if the newer adapter will fix any of that though, the DL380G5 is using PCIe version 1 I think, so it's all kinda slow and stuck that way. It would be cool to compare the two adapters performance side by side. I might do that is I replace the 3801 just because I'm curious, but I don't think it will matter. Probably based on a slow PCI bus even though it's x8. Not sure.