Support for Truecrypt-encrypted external USB drives

[panz]

I think that support for Truecrypt would be quite useful. For example, I have a bunch of LaCie external HDs, encrypted with Truecrypt (NTFS formatted). I'd like to use them for importing all the data into FreeNAS and for emergency backup purpose.

 

[warri]

NTFS support for FreeBSD is minimal and instable, it is not recommended to use it for anything else than copying data off the volume.

 

[titan_rw]

Like warri said, ntfs support in freenas is primarily meant as a 'read only, good enough to copy data to freenas' sort of solution. It is definitely not meant to be used as a backup destination.

Plus, with the drives truecrypted, your best bet it to hook them up to a windows machine, and operate over cifs.

 

[panz]

Let me explain this more in depth. My server is running WinXP & hardware RAID. This configuration is going to be a nightmare when a hard drive fails or when XP support will be dropped next year; I don't want to enter the low reliability of Win7/Win8 NTFS+proprietary RAID so I'm considering another solution.

Actually all Win clients backup to the server with scheduled SyncBack tasks (SyncBack free is awesome for this purpose: http://www.2brightsparks.com/syncback/ ).

Server backup is handled in two ways:

- daily backups of Writer/Calc/text files & other important files (from all the workstations backups) is done on an Amazon S3 bucket with Duplicati http://www.duplicati.com/ ;

- other (but less) important files (videos of live conferences, audio files) are backed up weekly to external USB drives with SyncBack free.

All hard drives (both Server's and external USB) are encrypted with Truecrypt for security reasons.

So, here's is my question: is there a way to setup FreeNAS the same way?

For budget reasons I can't setup another NAS so I'm stuck to this machine + USB for offsite backup. And I need encryption on ALL the HDs too.

 

[joeschmuck]

Well FreeNAS does have the capability to encrypt it's hard drives but I do not know if that extends to attached USB hard drives (not sure why it wouldn't), if so then that sounds like a solution. Although I would recommend a server with hardware AES support for encryption/decryption speed.

 

[panz]

Importing encrypted volumes means decrypting them before the operation. Not what I'm looking for...

 

[cyberjock]

This whole discussion is mute...there is no version of Truecrypt for FreeBSD. Until that happens there's no point in even discussing anything else regarding truecrypt and FreeNAS in the same sentence.

 

[panz]

Sorry, I didn't explain it right. I'll be very happy doing it with any other encryption methods available for FreeNAS/FreeBSD :) (maybe encrypting those external drives with GELI?)

 

[cyberjock]

Then why no consult the manual? Geli is used for encrypted zpools beginning with 8.3.1....

 

[panz]

I've done that, but I can't find a GUI method in FreeNAS to mount an external HD (formatted whatever system the OS likes and fully AES encrypted) and commit a backup.

 

[jgreco]

I think that there's a point to be made that FreeNAS is a little weak in the "can be easily backed up to a portable drive of some sort" department.

 

[titan_rw]

Importing encrypted volumes means decrypting them before the operation. Not what I'm looking for...

How else do you plan on having encrypted drives? They have to be decrypted so they can be used. Truecrypt does this transparently in the background. GELI under freebsd does something similar.

Using external drives with freenas is generally not recommended, even if they're left connected all the time. And constantly disconnecting and reconnecting them on an 'as needed' basis is not supported. You'd have to manually mount them and start any backup each time you connected them.

This whole discussion is mute...there is no version of Truecrypt for FreeBSD. Until that happens there's no point in even discussing anything else regarding truecrypt and FreeNAS in the same sentence.

I assume you meant "moot"?

Also, Truecrypt does exist for freebsd. Check /usr/ports/security/truecrypt/. You have to download the source tarball youself, and place it in the right directory, but it does work.

Of course it's not supported under freenas. You'd have to compile your own version of freenas with truecrypt built in. Then use it entirely from the command line.

 

[joeschmuck]

I don't see this as being an impossible task however it's not like Windoze plug-n-play, there are steps you must do manually but it's not difficult or anything. If you make each USB hard drive a ZFS formatted drive, then you can automount it. It should be no different from using a USB Flash drive, which is what I do for my jail.


@panz, you need to just buckle down and create a FreeNAS in a VM and play with it. This will let you try out adding and removing drives, even USB drives if you have good VM software like VMWorkstation. It might not be "supported" but it should work without a hitch.

I feel we have beaten this horse to death.

 

[cyberjock]

Based on my experience with FreeNAS and geli encryption I'm no sure if his would be smart.

For example, if your encrypted zpool is da5 and da6 in a mirror, then after you reboot its da4 and da5 FreeNAS won't recognize the change. Furthermore if you reboot and the drives are da5 and da6 again, the zpool may not mount.

Also, I could be wrong but I believe that the drives must be detected on bootup. Unplugging(or plugging them in) with the sysem running is no going to work based on my experiments. I did just wake up though.

 

[joeschmuck]

Based on my experience with FreeNAS and geli encryption I'm no sure if his would be smart.

For example, if your encrypted zpool is da5 and da6 in a mirror, then after you reboot its da4 and da5 FreeNAS won't recognize the change. Furthermore if you reboot and the drives are da5 and da6 again, the zpool may not mount.

Also, I could be wrong but I believe that the drives must be detected on bootup. Unplugging(or plugging them in) with the sysem running is no going to work based on my experiments. I did just wake up though.

Wow, that is crappy. I wouldn't think it would care which drive location it was.

 

[jgreco]

For example, if your encrypted zpool is da5 and da6 in a mirror, then after you reboot its da4 and da5 FreeNAS won't recognize the change. Furthermore if you reboot and the drives are da5 and da6 again, the zpool may not mount.

I'm not sure that's actually true. Disks are identified by FreeNAS using several methods, with disk device name being one of the less-preferred methods, and serial number appearing to be preferred on bare metal, but uuid being used if not, and encrypted disks seem to be identified by

You can look at the drive mapping tables in the database:

# sqlite3 /data/freenas-v1.db
sqlite> select * from storage_disk;
sqlite> select * from storage_encrypteddisk;

So I have this 2011 on the bench that I've been playing with. Let's see what happens. This is an unencrypted pool that I'll destroy, just want to see how it looks "before."

Code:

[root@freenas] /# camcontrol devlist
<NECVMWar VMware IDE CDR10 1.00>   at scbus1 target 0 lun 0 (pass0,cd0)
<VMware Virtual disk 1.0>          at scbus2 target 0 lun 0 (pass1,da0)
<VMware Virtual disk 1.0>          at scbus2 target 1 lun 0 (pass2,da1)
<LSI SAS2X36 0e0b>                 at scbus3 target 24 lun 0 (ses0,pass3)
<ATA ST3400832AS 3.03>             at scbus3 target 32 lun 0 (pass4,da2)
<ATA ST4000DM000-1F21 CC52>        at scbus3 target 43 lun 0 (pass5,da3)
<ATA ST4000DM000-1F21 CC52>        at scbus3 target 44 lun 0 (pass6,da4)
<ATA ST4000DM000-1F21 CC52>        at scbus3 target 45 lun 0 (pass7,da5)
<ATA ST4000DM000-1F21 CC52>        at scbus3 target 46 lun 0 (pass8,da6)
<ATA ST4000DM000-1F21 CC51>        at scbus3 target 47 lun 0 (pass9,da7)
<ATA ST4000DM000-1F21 CC51>        at scbus3 target 48 lun 0 (pass10,da8)
<ATA ST4000DM000-1F21 CC51>        at scbus3 target 49 lun 0 (pass11,da9)
[root@freenas] /#
[root@freenas] /# sqlite3 /data/freenas-v1.db
SQLite version 3.7.13 2012-06-11 02:05:22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> select * from storage_disk;
1|Disabled|Always On|||{devicename}da1|1|Disabled|Auto||||4|da1
1|Disabled|Always On|4NF0M8EW||{serial}4NF0M8EW|1|Disabled|Auto||||5|da2
1|Disabled|Always On|Z3004TCF||{serial}Z3004TCF|1|Disabled|Auto||||6|da9
1|Disabled|Always On|Z300C8GC||{serial}Z300C8GC|1|Disabled|Auto||||7|da3
1|Disabled|Always On|Z300CVFL||{serial}Z300CVFL|1|Disabled|Auto||||8|da4
1|Disabled|Always On|Z300C2D6||{serial}Z300C2D6|1|Disabled|Auto||||9|da5
1|Disabled|Always On|Z300CVD3||{serial}Z300CVD3|1|Disabled|Auto||||10|da6
1|Disabled|Always On|Z3004S9C||{serial}Z3004S9C|1|Disabled|Auto||||11|da7
1|Disabled|Always On|Z3004T9A||{serial}Z3004T9A|1|Disabled|Auto||||12|da8
0|Disabled|Always On|||{devicename}da0|1|Disabled|Auto||||13|da0
sqlite> select * from storage_encrypteddisk;
sqlite>
[root@freenas] /#

Okay, so, destroy, create encrypted. We see encrypteddisk populate:

Code:

sqlite> select * from storage_encrypteddisk;
1|1|7|gptid/678fcb36-c89a-11e2-96a3-000c2920acf7
2|1|8|gptid/682740aa-c89a-11e2-96a3-000c2920acf7
3|1|9|gptid/68b883df-c89a-11e2-96a3-000c2920acf7
4|1|10|gptid/6952bc1a-c89a-11e2-96a3-000c2920acf7
5|1|11|gptid/69f207ca-c89a-11e2-96a3-000c2920acf7
6|1|12|gptid/6a869973-c89a-11e2-96a3-000c2920acf7
7|1|6|gptid/6b1a216f-c89a-11e2-96a3-000c2920acf7
8|1|4|gptid/6c135f8d-c89a-11e2-96a3-000c2920acf7

Great. At which point I notice a completely unrelated but interesting thing: despite having AES-NI support, pool write speeds just dumped from 300MB/sec unencrypted to 86MB/sec encrypted. Hmm.

So. Let's shuffle drives and see what happens. Shutdown. Reboot.

Code:

[root@freenas] ~# camcontrol devlist
<NECVMWar VMware IDE CDR10 1.00>   at scbus1 target 0 lun 0 (pass0,cd0)
<VMware Virtual disk 1.0>          at scbus2 target 0 lun 0 (pass1,da0)
<VMware Virtual disk 1.0>          at scbus2 target 1 lun 0 (pass2,da1)
<LSI SAS2X36 0e0b>                 at scbus3 target 24 lun 0 (ses0,pass3)
<ATA ST3400832AS 3.03>             at scbus3 target 32 lun 0 (pass4,da2)
<ATA ST4000DM000-1F21 CC52>        at scbus3 target 43 lun 0 (pass5,da3)
<ATA ST4000DM000-1F21 CC52>        at scbus3 target 44 lun 0 (pass6,da4)
<ATA ST4000DM000-1F21 CC52>        at scbus3 target 45 lun 0 (pass7,da5)
<ATA ST4000DM000-1F21 CC52>        at scbus3 target 46 lun 0 (pass8,da6)
<ATA ST4000DM000-1F21 CC51>        at scbus3 target 47 lun 0 (pass9,da7)
<ATA ST4000DM000-1F21 CC51>        at scbus3 target 48 lun 0 (pass10,da8)
<ATA ST4000DM000-1F21 CC51>        at scbus3 target 49 lun 0 (pass11,da9)
[root@freenas] ~# sqlite3 /data/freenas-v1.db
SQLite version 3.7.13 2012-06-11 02:05:22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> select * from storage_disk;
1|Disabled|Always On|||{uuid}6c135f8d-c89a-11e2-96a3-000c2920acf7|1|Disabled|Auto||||4|da1
1|Disabled|Always On|4NF0M8EW||{serial}4NF0M8EW|1|Disabled|Auto||||5|da2
1|Disabled|Always On|Z3004TCF||{serial}Z3004TCF|1|Disabled|Auto||||6|da9
1|Disabled|Always On|Z300C8GC||{serial}Z300C8GC|1|Disabled|Auto||||7|da3
1|Disabled|Always On|Z300CVFL||{serial}Z300CVFL|1|Disabled|Auto||||8|da4
1|Disabled|Always On|Z300C2D6||{serial}Z300C2D6|1|Disabled|Auto||||9|da5
1|Disabled|Always On|Z300CVD3||{serial}Z300CVD3|1|Disabled|Auto||||10|da6
1|Disabled|Always On|Z3004S9C||{serial}Z3004S9C|1|Disabled|Auto||||11|da7
1|Disabled|Always On|Z3004T9A||{serial}Z3004T9A|1|Disabled|Auto||||12|da8
0|Disabled|Always On|||{devicename}da0|1|Disabled|Auto||||13|da0
sqlite> select * from storage_encrypteddisk;
1|1|7|gptid/678fcb36-c89a-11e2-96a3-000c2920acf7
2|1|8|gptid/682740aa-c89a-11e2-96a3-000c2920acf7
3|1|9|gptid/68b883df-c89a-11e2-96a3-000c2920acf7
4|1|10|gptid/6952bc1a-c89a-11e2-96a3-000c2920acf7
5|1|11|gptid/69f207ca-c89a-11e2-96a3-000c2920acf7
6|1|12|gptid/6a869973-c89a-11e2-96a3-000c2920acf7
7|1|6|gptid/6b1a216f-c89a-11e2-96a3-000c2920acf7
8|1|4|gptid/6c135f8d-c89a-11e2-96a3-000c2920acf7
sqlite>

Okay, that's frustrating, apparently the M1015 or the SAS expander are doing something to lock the position of devices in the array. So I guess this is going nowhere. Sorry.

Also, I could be wrong but I believe that the drives must be detected on bootup. Unplugging(or plugging them in) with the sysem running is no going to work based on my experiments. I did just wake up though.

Not sure exactly which thing we're referring to here. Plug in a drive to a hotswap capable controller and FreeNAS should recognize it and list it in the volume manager as soon as FreeBSD identifies the drive. It isn't going to mount it, however.

 

[cyberjock]

Not sure exactly which thing we're referring to here. Plug in a drive to a hotswap capable controller and FreeNAS should recognize it and list it in the volume manager as soon as FreeBSD identifies the drive. It isn't going to mount it, however.

Yeah, I have a new issue with encryption. I have a 6 drive RAIDZ2 that is encrypted. I've had to replace 2 disks and using our "hack" to remove the old gptids I have a working key+passphrase and recovery key. All seems good. But something went wrong with my setup. Here's the unfiltered information I have:

1. The zpool worked just fine for more than 2 weeks(and multiple reboots) and I had no problems.
2. I did the fan testing thread data gathering (http://forums.freenas.org/threads/norco-rpc-4224-and-hard-drives-that-are-too-hot.13022/) and at one point 5 drives had "disappeared". 2 of them were in the encrypted pool. I don't know for 100% certainty that all 6 drives did in fact mount since I'm questioning everything as I can't make sense of this. Also of note is that the drives are connected by cables that have 4 drives per cable. The 6 disks were not in the same row, so this was not a SATA cable that was lose or disconnected(it would have had to have been 4 loose cables)
3. After I figured out that my zpool was degraded I rebooted(and upped the fan speed back to 50%). Despite all 6 drives definitely being detected the sqlite database now had only 4 entries. I didn't even bother trying to remount the zpool and instead opted to recover from a backup configuration that would have all 6 entries in the zpool.
4. I recovered to a backup that was from just 4 days ago and I was able to mount the encrypted zpool as "healthy".
5. I ran a scrub of the zpool which returned 25 and 26 CHKSUM errors, but no corrupted files.

I really can't explain why 2 of my encrypted gptids would suddenly be removed from the table. I didn't do anything that would have deleted them, and I'm not sure what kinds of things will trigger their "auto-deletion" from the table. I'm wondering if I mounted the zpool and it was degraded and if 2 of the disks weren't available at the time the pool was mounted they were automatically removed from the sql table since FreeNAS considered them to not be a part of the pool anymore.

 

[William Grzybowski]

There were a couple issues that could cause the cache disk table to get out sync, this is hopefully fixed now in 9.1 ALPHA.

Getting the disk table out of sync could cause disks from being removed of the encrypteddisk table as well, as there is a link between them.

 

[cyberjock]

Outstanding! Glad you're staying on top of this William! I don't know what FreeNAS would be like without you.

As for my above post I'm not submitting a ticket because I can't identify what actually went wrong, just the end result. Creating a ticket that "its broken" without a way to reproduce the issue won't help much. :P