Disks attached to ASMedia UAS controller are not seen by middleware

[Cocoloco]

In my TrueNAS SCALE Bluefin system I have a couple of SSD SATA disks from SK Hynix that previously were installed and configured as two separate single drive zpools. I decided to replace them with another couple of SSD disks, but configured in RAID1, and to move the SK Hynix disks inside a SilverStone DS233 dual-SATA enclosure. To my surprise, the disks now are not seen by the TrueNAS SCALE GUI, and indeed I can't create a new zpool on them from the GUI.

Initially I thought it was because that was due to the fact the disks had still configured the scrapped zpools, so I wiped the drives with wipefs and created a new GPT partition table. Yet, the GUI still can't see them listed as available.

I first checked that the external enclosure is properly seen by the underlying Debian kernel, and yes, it is:

Code:

➜  ~ lsusb
...
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 002: ID 174c:55aa ASMedia Technology Inc. ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge, ASM1153E SATA 6Gb/s bridge
...
➜  ~ sudo lsusb -t
...
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=uhci_hcd/2p, 12M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=uas, 5000M

Then I checked that the relevant sdx devices are created:

Code:

➜  ~ sudo blkid
...
/dev/sdf1: PARTUUID="4ef3e666-34c9-4b09-a9ad-cc303e0a5ba5"
/dev/sdg: PTUUID="6bff8b0c-081d-1640-aa35-46c4ab1f8a5a" PTTYPE="gpt"
/dev/sdh: PTUUID="0b8f2787-2679-884c-9adf-e1e958d44c48" PTTYPE="gpt"

Finally I checked that the drives are recognized properly by the system, i.e. not hidden by the optional RAID mode featured in the enclosure (I have configured the box in no raid mode). smartctl confirms it:

Code:

➜  ~ sudo smartctl -a /dev/sdg
smartctl 7.2 2020-12-30 r5155 [x86_64-linux-5.15.79+truenas] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     VK001920GWTHC
Serial Number:    SJ95N8249I0106C1N
LU WWN Device Id: 5 ace42e 02501e99f
Firmware Version: HPG4
User Capacity:    1,920,383,410,176 bytes [1.92 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    Solid State Device
Form Factor:      2.5 inches
TRIM Command:     Available, deterministic, zeroed
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-4, ACS-3 T13/2161-D revision 5
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sat Apr 22 13:23:52 2023 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
...

➜  ~ sudo smartctl -a /dev/sdh
smartctl 7.2 2020-12-30 r5155 [x86_64-linux-5.15.79+truenas] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     VK001920GWTHC
Serial Number:    SJ95N8249I0106C42
LU WWN Device Id: 5 ace42e 02501e9f6
Firmware Version: HPG4
User Capacity:    1,920,383,410,176 bytes [1.92 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    Solid State Device
Form Factor:      2.5 inches
TRIM Command:     Available, deterministic, zeroed
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-4, ACS-3 T13/2161-D revision 5
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sat Apr 22 13:24:20 2023 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
...

AFAIK, the above would entitle the disks to be properly managed, but as said they are now shown in the GUI.

So I opened a shell and created manually the zpool:

Code:

zpool create -R /mnt backup-pool -o failmode=continue mirror /dev/disk/by-id/ata-<disk1> /dev/disk/by-id/ata-<disk2>
zpool export backup-pool

and then I have imported the pool into the GUI. The import selector finds the created zpool, however something strange occurs after the import completes: there is no VDEV at all (including the data VDEV) ad the disks are not there.

Not surprisingly, it is not functioning properly: atime is set to ON, but if I set to OFF the GUI still states that it is ON. ACL can be switched to SMB/NFSv4, but then setting the ACL no ACL templates are shown, and so on.

So, a couple of question arise:

1. Is the aforementioned ASMedia UAS properly supported? With my experience in Debian and the findings above, I would say yes, but...
2. And if it is supported, why the middleware completely ignores and/or misbehaves with anything related to those two disks?

Thank you for your attention, and I apologize for the long post.

 

[Samuel Tai]

Why you should avoid USB attached drives for data pool disks

This subject has been coming up quite often since about 2022. Perhaps because of TrueNAS SCALE has brought forth some attention to TrueNAS in general and ZFS in specific. Please note that this is about USB attached storage for ZFS data pools. On occasion, people use USB drives, (flash...

www.truenas.com

Unfortunately, these external UAS enclosures aren't suitable for TrueNAS, as their firmware only really supports Windows desktop access patterns.

 

[Cocoloco]

Unfortunately, these external UAS enclosures aren't suitable for TrueNAS, as their firmware only really supports Windows desktop access patterns.

Hey Samuel,

Thank you for your super-fast follow-up. I didn't know the thread you have pointed to and I have read its content. I have to say that of the twelve main points/issues listed at the beginning of the linked post, none of them apply in my specific case: the USB cable is secured, the power is drawn directly from the USB port (no attached power brick), and as you have probably read in my original post it is a UASP device, it is not running with any kind of RAID mode (it is not in JBOD mode), SMART has full access to the drives, etc.

Given the above, I fail to see why the enclosure couldn't be used for backup purposes (as you may have guessed from the zpool's name) without resorting to any kind of kludge. To begin with, if it is not supported, why you can import the pool? I understand there are reasons behind a general adoption of the many flaky UASP chip available on the market, I for one have suffered by some of them being completely unreliable. Yet, your words seems to indicate more a political decision than a technical bug, which I thought it was my case.

Thank you again.

 

[Samuel Tai]

OK, let's talk technical details. The Silverstone DS223 uses ASMedia ASM1352R A1 controllers to provide the SATA to USB 3 connection, and a ASM1542 controller to multiplex the 2 separate disks. In essence, the firmware is always multiplexing access to both disks, so TrueNAS does not have direct access to either disk, full-time.

 

[Cocoloco]

Interesting. I assumed the fact smartctl has full access to them (and I know it has, the output is the exact copy of when those disks were attached directly on the SAS/SATA internal HBA) would suffice to deal with them. If the kernel can see them, SMART can be monitored, ... it should be enough, but eventually other inner details I fail to see escape me, or due to my ignorance.

Thanks nevertheless, and let's go on with the kludge then.

 

[HarryMuscle]

Interesting. I assumed the fact smartctl has full access to them (and I know it has, the output is the exact copy of when those disks were attached directly on the SAS/SATA internal HBA) would suffice to deal with them. If the kernel can see them, SMART can be monitored, ... it should be enough, but eventually other inner details I fail to see escape me, or due to my ignorance.

Thanks nevertheless, and let's go on with the kludge then.

How did things turn out? I'm thinking of getting an IoSafe Duo that apparently uses the same chip so I'm curious to know if you got things working.

I agree with you about this being strange that it doesn't work out of the box. According to the info you posted the serial numbers are different for each drive which is usually the big issue with USB enclosures with multiple disks.

Thanks,
Harry

 

[Arwen]

Serial numbers for USB attached drives are sometimes taken from the USB device, NOT the attached SATA devices. Thus, if you have 2 or more SATA devices in an USB enclosure, the software can get confused about how many disks their really are attached. Sometimes this ends up showing only 1 of the disks in a multi-disk USB enclosure.

There was another thread about the USB disk serial number issue on SCALE, and a work around. I just read it tonight a few minutes ago, but I don't remember the details. (It's not something I need.)


All that said, I use a single drive USB enclosure for backups, (on Core), and it works perfectly fine. But, I do all the work from the command line. Never even bothered to see if the pool was seen from the GUI.

Here is the guide I wrote;

How to: Backup to local disks

How to: Backups to local disks Overview: This subject comes up often enough that I thought I would start a post of how I do it. Specifically I back up my TrueNAS to a hot-inserted, locally attached disk, then when done, hot-remove and take...

www.truenas.com

It is not meant to be a one size fits all. Just something to show what another person did, and potentially give someone else ideas.

 

[Cocoloco]

How did things turn out? I'm thinking of getting an IoSafe Duo that apparently uses the same chip so I'm curious to know if you got things working.

I agree with you about this being strange that it doesn't work out of the box. According to the info you posted the serial numbers are different for each drive which is usually the big issue with USB enclosures with multiple disks.

After months of usage, it just works, even with the latest version. Scrub is performed as expected, resilvering too, and yes: as the the real serial number of the disks is shown, I fail to understand why they still refuse to manage them in the GUI.

However I have detected a mistake in my original post: the -o ashift=12 option was missing in zpool create, hence the pool is created with a not-optimal block size. The GUI takes care of it, normally.

 

[Cocoloco]

Serial numbers for USB attached drives are sometimes taken from the USB device, NOT the attached SATA devices. Thus, if you have 2 or more SATA devices in an USB enclosure, the software can get confused about how many disks their really are attached. Sometimes this ends up showing only 1 of the disks in a multi-disk USB enclosure.

Which was not my case. At a later stage I exported the pool, extracted the disks, connected them on another system only this time using an HBA, and imported back the pool on the new system. Zero problems. The labels presented were in all the cases the real ones of the connected disks.