SOLVED Adaptec 6805E and arcconf issue

[Athar]

Hi everybody,


Well, a simple (I hope) question for you guys, I just install the latest version of FreeNAS to get away from my Windows Server 2008R2 which was doing only share, so a lot of ressources was wasted.

Now, everything is running good so far, RAID card is detected, it see the array (yes, I prefer to let that array managed by the RAID card), except one little issue.

To see informations about the RAID adapter (like disk status, read Health, etc.), we need to use this command:
arcconf GETCONFIG 1

If running with the embedded version (v4.3 or v4.5), it didn't see the RAID adaptor, so I decide to download the more recent one on Adapatec website for this particular card (v1.5) and now, I can get the data.

Question is, how can I replace this binary file as it seems to be in a protected area ? otherwise, where is the best place to store any scripts and binary ?


Thanks.

 

[cyberjock]

Ok, I'm going to stop you right here. I can understand that you prefer to let the array be managed by the RAID card. But this is such a mistake I didn't even bother reading the rest of your post. I'm not sure what your long-term plans are, but if you are using a RAID card and you are determined to let it manage your disks then FreeNAS is not the product for you.

 

[Athar]

At the end, I'll have 3 "arrays".

One will remain managed by the RAID card (RAID10), I don't see the issue to let the card handle this raid, unless I didn't catch something about Freenas.

Then, a second array in RAIDZ2 with 6x2TB (which give me arround 7TB) and a last one in mirror with 2x3TB. Those eights HDD are plugged directly to the MB, so I let FreeNAS managed them.
I only have 8GB of memory, making another RAID with ZFS and the 8GB will be really too small (8 HDD of 3TB, arround 11TB in RAID10).

After, I didn't really care of what you think it's the best or worst option, I just ask on how to get arcconf replaced in the /usr/local dir in fact, but if you don't want to help me, no problem, will continue to search by myself ;) (I was just trying to get a faster answer there).

 

[cyberjock]

One will remain managed by the RAID card (RAID10), I don't see the issue to let the card handle this raid, unless I didn't catch something about Freenas.

Ok.. read our manual.. doc.freenas.org. I think I summed up what the manual says in my original post though. ;)

 

[Athar]

Ok, I didn't saw that part on the doc earlier.

For anyone that will read in the future:

NOTE:
instead of mixing ZFS RAID with hardware RAID, it is recommended that you place your hardware RAID controller in JBOD mode and let ZFS handle the RAID. According to Wikipedia:
“ZFS can not fully protect the user's data when using a hardware RAID controller, as it is not able to perform the automatic self-healing unless it controls the redundancy of the disks and data. ZFS prefers direct, exclusive access to the disks, with nothing in between that interferes. If the user insists on using hardware-level RAID, the controller should be configured as JBOD mode (i.e. turn off RAID-functionality) for ZFS to be able to guarantee data integrity. Note that hardware RAID configured as JBOD may still detach disks that do not respond in time; and as such may require TLER/CCTL/ERC-enabled disks to prevent drive dropouts. These limitations do not apply when using a non-RAID controller, which is the preferred method of supplying disks to ZFS.”

So, now, I catch your point of view, and will see how to set the Raid card in "IT"/bypass/JBOD mode and let the system handle it.

Anyways, this didn't solve the first point I ask here, about the ability to change "arcconf" :) (because it still good to catch datas about the controller itself, for health report, like temp, firmware check, etc.)

But, thanks to make me point this info about the mistake I was doing on letting the RAID controller doing the job with ZFS system (I only know basics Linux distro, not FreeBSD based systems).

 

[cyberjock]

Adaptec's don't do passthrough properly. If you search for "adaptec" in the forum you'll see that every post involving Adaptec is followed up with someone saying "you'll have to buy another card".

After you've gotten a more appropriate card SMART will handle all of the necessary monitoring for you.

 

[Zedicus]

actually the 6805E is not a raid card, it DOES do JBOD and there are a lot of people using them in HBA configurations.

adaptec slightly changed modeling after they were bought out (around the begining of the 6xxx series line) before that none of their cards could really do HBA or JBOD modes at all. the 6xxx and 7xxx series do have some HBA style cards (usually those with an E towards the end) but you do need to watch for some model variances.

rather or not the card has usable BSD support may be the hangup though.

 

[Athar]

@cyberjock: Yeah, saw some "old" post that was telling that not all drives when set in JBOD was appearing (only the first attach port were showing the disk, those on the second port wasn't).
Now, I updated the firmware and this seems to work a bit more.
The only way to know is to test it, so I'll (I try an LSI card, but not enough shared memory on the PCIe x16 port... so I'm stuck with that card for now)

@Zedicus: Sorry, don't understand the last phrase^^

 

[cyberjock]

Athar,

What happens if you do a smartctl -a /dev/da0 (substitute some disk on your controller). Last time I heard its still not supported which is literally a game killer. So unless something has changed the adaptec threads that said not to use them are still quite accurate.

Adaptec has always done "their own thing". It has worked out for them in many ways. But this is one way in which they cannot "do their own thing" and end up ahead. I sold all 4 of my adaptecs when I switched to FreeNAS. It's just how things work out.

 

[Athar]

Actually, the da0 disk is my actual RAID10 array so... I get an error and this is normal so far. Only arcconf (with the good version) can extract such data.

I'm running a copy that is taking a long time... too long in fact (source HDD is maybe failling, or has just poor performances xD), when this will be over, I'll scratch the ZFS volumes from that raid, then will check this JBOD function.

Will let you know the result (probably tomorrow).

 

[cyberjock]

Actually, the da0 disk is my actual RAID10 array so... I get an error and this is normal so far. Only arcconf (with the good version) can extract such data.

And that's precisely the problem with Adaptecs even when you do their "JBOD". SMART needs to work properly. It's how FreeNAS is designed. Without SMART you basically lose your primary indicator of a failing disk. So unless smartctl -a works you shouldn't be using that controller. ;)

 

[Athar]

Well, like I said, I'll do all the necessary test, I always prefer to check it by myself, we can have surprises.

 

[Athar]

Ok, no need to reboot, I've done it through command line.

So, smartctl work, but we need to specify "-d sat", if we don't, it refuse, example:

Normal "light" command:

[root@NAS-v2] /mnt/RAIDZ2-7T/Private_Share_2/TTmp# smartctl -a da0
smartctl 6.2 2013-07-26 r3841 [FreeBSD 9.2-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

da0: Unable to detect device type
Please specify device type with the -d option.

Use smartctl -h to get a usage summary

and with the "long" command:

[root@NAS-v2] /mnt/RAIDZ2-7T/Private_Share_2/TTmp# smartctl -a -d sat da0
smartctl 6.2 2013-07-26 r3841 [FreeBSD 9.2-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family: Western Digital Red (AF)
Device Model: WDC WD30EFRX-68AX9N0
Serial Number: WD-XXXXXXXX:)
LU WWN Device Id: 5 0013ee 0017611f5
Firmware Version: 80.00A80
User Capacity: 3,000,592,982,016 bytes [3.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-2 (minor revision not indicated)
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Tue Aug 5 00:55:56 2014 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
(pass9:aacraidp1:1:0:0): ATA COMMAND PASS THROUGH(16). CDB: 85 06 2c 00 da 00 00 00 00 00 4f 00 c2 00 b0 00
(pass9:aacraidp1:1:0:0): CAM status: CCB request completed with an error
SMART STATUS RETURN: incomplete response, ATA output registers missing
SMART overall-health self-assessment test result: PASSED
Warning: This result is based on an Attribute check.

General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (40380) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 405) minutes.
Conveyance self-test routine
recommended polling time: ( 5) minutes.
SCT capabilities: (0x70bd) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0
3 Spin_Up_Time 0x0027 178 177 021 Pre-fail Always - 6066
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 403
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0
9 Power_On_Hours 0x0032 096 096 000 Old_age Always - 3619
10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 403
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 402
193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 0
194 Temperature_Celsius 0x0022 121 104 000 Old_age Always - 29
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 100 253 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 100 253 000 Old_age Offline - 0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]


SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

EDIT: In "Storage => Volumes => Show disk", for each "daX", I add the option: -d sat
For the SMART options, that should be good so no ? :)

 

[cyberjock]

I don't know. I know in 8.3.1 adding -d sat did NOT fix it. It also doesn't fix SMART testing. So unless you want to put your data at risk with this experiment I'd say stick with what I said before.. get the right hardware. ;)

And then, even if it does work, there's no guarantee that it will work in the future.

If you are okay with the risk, go for it. Me, I'd never consider nor would I tell a friend its a good idea.

 

[Athar]

Well, as I didn't know this version, I'll not tell you if it will work, maybe I can take a disk a install on it just to check, anyways, with latest version of FreeNAS, and setting ALL disks to "JBOD", the "-d sat" is working as expected. Also, I update to the latest firmware of the card, which can make a difference.

I did a short smart test, it pass without issue.

[root@NAS-v2] /mnt/RAIDZ2-7T/Private_Share_2/TTmp# smartctl -t short -d sat da0
smartctl 6.2 2013-07-26 r3841 [FreeBSD 9.2-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION ===
Sending command: "Execute SMART Short self-test routine immediately in off-line mode".
Drive command "Execute SMART Short self-test routine immediately in off-line mode" successful.
Testing has begun.
Please wait 2 minutes for test to complete.
Test will complete after Tue Aug 5 16:21:09 2014

Use smartctl -X to abort test.
[root@NAS-v2] /mnt/RAIDZ2-7T/Private_Share_2/TTmp# smartctl -a -d sat da0
smartctl 6.2 2013-07-26 r3841 [FreeBSD 9.2-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family: Western Digital Red (AF)
Device Model: WDC WD30EFRX-68AX9N0
Serial Number: WD-XXXXXXXX:)
LU WWN Device Id: 5 0013ee 0017611f5
Firmware Version: 80.00A80
User Capacity: 3,000,592,982,016 bytes [3.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-2 (minor revision not indicated)
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Tue Aug 5 16:35:04 2014 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
(pass9:aacraidp1:1:0:0): ATA COMMAND PASS THROUGH(16). CDB: 85 06 2c 00 da 00 00 00 00 00 4f 00 c2 00 b0 00
(pass9:aacraidp1:1:0:0): CAM status: CCB request completed with an error
SMART STATUS RETURN: incomplete response, ATA output registers missing
SMART overall-health self-assessment test result: PASSED
Warning: This result is based on an Attribute check.

General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (40380) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 405) minutes.
Conveyance self-test routine
recommended polling time: ( 5) minutes.
SCT capabilities: (0x70bd) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0
3 Spin_Up_Time 0x0027 178 177 021 Pre-fail Always - 6066
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 403
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0
9 Power_On_Hours 0x0032 096 096 000 Old_age Always - 3635
10 Spin_RetryCount 0x0032 100 100 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 403
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 402
193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 0
194 Temperature_Celsius 0x0022 113 104 000 Old_age Always - 37
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 100 253 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 100 253 000 Old_age Offline - 0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed without error 00% 3634 -

SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

But I can understand that past experiences make you not like Adaptec card, but with this, at least thos versions, it work as expected, no error, SMART test running, just need to specify the "-d sat" and it's all good.
Now, the last check is to see if the built-in FreeNAS cron will work now.

So, it work NOW, and if I didn't upgrade the software, that will work untill an hardware faillure.

 

[cyberjock]

Ok, and you are 100% sure that SMART is properly monitoring with the -d sat added? How did you come to that conclusion?

Actually, don't tell me. I don't want to be complicit in this bad idea.

Good luck to you sir.

 

[Athar]

Yeah, we will stop here, you have your point of view, I've mine, and I also did a mistake, it's not "-d sat" but "-d scsi".

Anyways, thanks for your help.

[root@NAS-v2] /mnt/RAIDZ2-7T/Private_Share_2/TTmp# smartctl -H -d test /dev/da0
smartctl 6.2 2013-07-26 r3841 [FreeBSD 9.2-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

/dev/da0: Device of type 'scsi' [SCSI] detected
/dev/da0: Device of type 'scsi' [SCSI] opened