Switching SATA cables and plugs

[troun]

Hi,

Probably a noob question; but it is time, after almost 2 years of service, for my Freenas server to get cleaned (talking about dust not ZFS scrub ;) ).
Should I worry, when I will disconnect Hard Drives, in which plug it is, or can I remount without taking care because Freenas is gonna recognised each drives?

Bonus question; I have 'few' (but still too much) parity errors messages (let say 1 error / 10GB writing, -> Freenas retry -> OK). Any 'trick' or 'advice' to locate which of my 5 HD is the faulty 'ada2' in order to replace its SATA cable?

cheers,
Alx

 

[DrKK]

If you go to Storage->View Disks in the GUI, it should tell what the serial number is for each device, then you compare it to the number printed on your hard drive, and find the right one.

 

[gpsguy]

While you have it apart, label the serial number to the visible end of the drives, so should one fail in the future, you don't accidentally remove the wrong one.

I keep a Word document, with configuration details, notes, command line syntax, etc.

 

[troun]

Thanks both of you, that's obviously a good suggestion to identify which hard drive is "faulty".


Anyone for the first question? Are each hard drives linked to their sata port?

 

[Dusan]

Anyone for the first question? Are each hard drives linked to their sata port?

No, but make sure that you export/detach the pool before switching the cables. ZFS will figure everything out when you Auto Import the pool back even when the device names changed .

 

[DrKK]

Right. What Dusan said. There is a GPTID identifier for each of the drives (if you created the pool from the GUI), and when you re-import the pool, it shouldn't matter which is connected to what port.

 

[Dusan]

Right. What Dusan said. There is a GPTID identifier for each of the drives (if you created the pool from the GUI), and when you re-import the pool, it shouldn't matter which is connected to what port.

ZFS actually doesn't care much about the GPTIDs. Those are primarily used by the FreeNAS GUI. ZFS is able to correctly import a pool even if the device names changed and you didn't use GPTIDs. Each ZFS device is identified by a ZFS GUID and contains four copies of a vdev label that describes the configuration of the entire pool (including GUIDs of all the other devices). zpool import searches all devices in /dev, reads the labels and figures everything out (even without GPTIDs). You can find lot of additional details here: http://www.osdevcon.org/2009/slides/zfs_internals_uli_graef.pdf

 

[troun]

Thanks a lot! :)
So to summarize, I will have to;
_ In GUI, detach Zpool and then shut down the NAS
_ Disconnect and clean everything, re-assemble
_ Restart NAS and in GUI import back Zpool (auto-import may work, I guess).

This is good to know because before end of the year, I will have to transport the NAS, and it will be much more convenient and safe if I can unmount hard drives from their support and wrap separately in bubblepaper.


So I'm done, but just by curiosity, what may happen if zpool is not properly detached before you invert two Sata ports? Re-contruction? Simply pool mount fail?

 

[Dusan]

It may not mount cleanly. Example: http://forums.freenas.org/threads/ecc-vs-non-ecc-ram-and-zfs.15449/page-4#post-83743

 

[cyberjock]

@Dusan I'm really not understanding the whole reason behind your exporting a pool in case you swap cables. Last night I deliberately swapped hard drives in my system despite not doing an export. It came right back up without a hitch. This was how it was "supposed" to work in 8.x and I'm not seeing any reason why it won't work in 9.x. Especially if you are using gptids like FreeNAS uses. I think more testing and research is necessary on this subject. We've discussed this before and I didn't really push the subject, but after my test last night I'm fairly convinced that you don't really need to export the pool. For as long as I've been on this forum we've never told people to export a pool for a situation such as this. I'm really wondering what has changed.

 

[Dusan]

I also tried it in a VM -- it did work fine. I just shutdown the VM, randomly shuffled the SATA ports without exporting the pool first and it did import/mount properly. However, as @jyavenard did experience problems I recommend the export to be on the safe side.

 

[cyberjock]

Personally, I think it has to do with his hardware config or something else he did to cause the problem(human error?). I can't even begin to count the number of people that haven't had a problem.

 

[jyavenard]

I also tried it in a VM -- it did work fine. I just shutdown the VM, randomly shuffled the SATA ports without exporting the pool first and it did import/mount properly. However, as @jyavenard did experience problems I recommend the export to be on the safe side.

It could be that the device name were different between the two config I tried.
/dev/daX for LSI devices (scsi driver)
and /dev/adaX (ADA drivers) for the intel one.

The pool did get mounted; albeit in degraded state.

 

[cyberjock]

Yeah, the devices shouldn't matter. I've converted from adaX to daX without problem. Normally its an issue with hardware not being detected on bootup result in odd states or hardware not being compatible. Personally, it sounded more like a SATA cable that wasn't exactly plugged in all the way on bootup or something similar.

 

[jyavenard]

I had forgotten that you were clairvoyant and omnipresent. And that because you haven't personally experienced something it mustn't exist.
Silly me.

 

[DrKK]

really, jyavenard.

Come on. Cyberjock's post was perfectly fine (this time).

And if we're being honest, I think it's fair to say that Cyberjock *is* omnipresent in the forum. He has read 100% of the posts written in English, I am sure of that.

 

[cyberjock]

I had forgotten that you were clairvoyant and omnipresent. And that because you haven't personally experienced something it mustn't exist.
Silly me.

Nope. But you have provided no evidence of what actually went wrong. And as more than 50 people I've worked with have moved drives without problems, it's more than likely statistically you made a human error. That's all I said. If you can reproduce the problem and provide evidence supporting the actual problem(there is no doubt in my mind you are incapable of this anyway.. just look at your crap benchmark thread for a good example) then I'd be welcome to entertain that there's an error in FreeNAS.

As it stands, I've moved disks many times personally and I've done plenty of teamviewer sesions to actually know what I'm talking about.

 

[jyavenard]

really, jyavenard.
Come on. Cyberjock's post was perfectly fine (this time).

It's the implying that it must have been a user error that got me sorry... (plus all the passive aggressive he's been subjecting me to since I started posting here). So maybe I'm seeing things that aren't really there... but somehow I doubt it...

I should point that the solaris man page explicitly states that a pool should be exported first before being used in another system (be it change of OS, change of hardware or what else)
http://docs.huihoo.com/opensolaris/solaris-zfs-administration-guide/html/ch04s06.html

Storage pools should be explicitly exported to indicate that they are ready to be migrated. This operation flushes any unwritten data to disk, writes data to the disk indicating that the export was done, and removes all knowledge of the pool from the system.

but really, that is of course completely wrong, because one person never had a problem not doing it...

Edit: you see @DrKK, I didn't misread anything...

 

[DrKK]

Well if it actually says that in the solaris manual, then, I suppose we should be giving that advice officially at all times.

Thanks for that Jyavenard.

 

[cyberjock]

I should point that the solaris man page explicitly states that a pool should be exported first before being used in another system (be it change of OS, change of hardware or what else)
http://docs.huihoo.com/opensolaris/solaris-zfs-administration-guide/html/ch04s06.html

Thank the heavens this isn't Solaris, huh? If a Windows manual told us to export an NTFS partition would that have applied too? This is FreeBSD. FreeBSD =/= Solaris. And the FreeNAS manual makes it very clear that you CAN just take disks out of one machine and drop them in another machine without consequences(assuming you meet the other prerequisites such as hardware support, etc.) The manual even includes comments about how to recover your system by taking your USB stick and hard drives to another machine and continuing to operate. This was by design by the iXsystems guys to keep the "appliance-like" design in FreeNAS working properly. FreeNAS properly flushes its buffers and such on a shutdown normally. So there's no concerns with the exporting/importing being problematic at present.

Now if your PSU suddenly exploded and you move the drives to a new system then you'd potentially have a problem. But in that case I think it's totally acceptable to say that you might have weird unexpected results for that situation. And trying to go back to export your pool definitely wasn't an option in that situation anyway.

Relax, it's not a big deal that it was human error. It's not a big deal even if it wasn't human error. This isn't a big deal at all. For all we know your hardware has some weird fluke that made the first bootup not work properly or a delay in being detected. Anyone who's been here more than 2 weeks has probably seen the threads where many motherboards will bootup cold and work fine, but on a restart it won't boot properly. Those users have learned that they must do cold boots every time until the Haswell chipset support(or whatever is causing the problem) is fixed. It's not a big deal and life goes on. I'm glad you didn't lose data over it. If you had, then I'd be doing a Teamviewer session with you to figure out where whatever went wrong did go wrong and trying to help you get your data back.