Mini XL+ startup/encryption questions

[rwv37]

First, a little background for context:

• I recently purchased a TrueNAS Mini XL+, with TrueNAS Core on it.
• I've never used TrueNAS (or FreeNAS) before.
• I've used FreeBSD, though - a lot, and for decades.
• I've long used GELI for encryption.
• I've never used ZFS-level encryption.
• I've used ZFS, though, for years.

I was hoping to set my new box up with encryption a lot like I typically do for my FreeBSD boxes: GELI on everything with the exception of the boot stuff. When I power or reboot on a machine, before it's useable in any "real" sense, I have to physically go to the machine and enter the GELI password for the disks. I'm fine with that - in fact it's what I want. And to be clear, I'm aware that this does not protect the data while the machine is really up and running; the main threat I'm concerned with here is a physically stolen machine.

However, I see that GELI is deprecated on TrueNAS (though, incidentally, I also see that it's used for the swap space). So, before even setting up a pool with the hard drives I've installed, I've been poking around on the web interface to try to figure out how I would encrypt things instead. While doing so, I noticed that lots of stuff from the built-in "TrueNAS M.2 boot device" was already mounted -- not just boot code -- including stuff that I would greatly prefer to be encrypted (e.g. /root and Samba stuff).

So, I spent some time trying to figure out exactly what I would need to keep there, unencrypted, and how to move everything else to "real" drives which I would encrypt. I'm no low-level booting/etc. guru, and I'm pretty lazy to boot, so eventually, I gave up on this, and decided screw it, I'll just use this new machine for stuff that I either don't need encrypted or storage of files that I encrypt at the file level, plus maybe a specific encrypted pool for ad-hoc purposes.

I then proceeded to the next step of setting the machine up: Create ZFS pools for the "real" drives that I had installed. When I did so, I noticed that the first one I created had automatically been labeled as a "System Dataset Pool", and was pleasantly surprised to then discover that a bunch of the things that had been mounted from the "boot pool" were now mounted from my "real" pool instead. This gave me some hope that I might be able to configure encryption how I really want after all, or at least a lot closer to how I really want. However, I still have some questions:

1. At first, I thought everything but the boot stuff was now mounted from the "real" pool. But on closer inspection, I see that there are still other things that are apparently still coming from the boot pool, some of which I would very much like to be encrypted (e.g. /root, /usr). Can I get that stuff into the encrypted pool too, somehow? Obviously I could copy it into the encrypted pool, but I'm concerned about how the system might react if I then tried to switch, live, from the original (boot drive's) /usr or /root or whatever to the new one. Maybe there's a way to do this through some sort of TrueNAS equivalent of a FreeBSD live image, or something?

2. Similar question for /etc, which is listed as tmpfs; not sure where the actual data in there came from?

3. Ideally, I'd like the machine's root directory to be mounted from the encrypted pool, rather than needing to worry about individual items like /usr or /root. Anything on the live system that really needs to come from the built-in "boot device" would be the exception from the mounting point of view, not (as it currently seems) the default. Not sure if I'm being clear about this, so, for example: Right now, if I were to mkdir /blah, it seems to me that directory would be created on the boot device, since (unless I'm misunderstanding something here) the boot device contains what is mounted as /. But I'd like it to instead be created on the "real" hard drives, such as would happen if / were mounted from there. Can that be done?

4. In all of this, I am imagining an end goal of startup/encryption that's functionally a lot like the FreeBSD setup I'm used to: When the machine powers on, there's some very minimal system that does very little besides asking for the password to unencrypt the "real" system. Am I on the right track here?

5. In regards to #4, I'm also wondering about where/how I would enter that password. On my FreeBSD boxes, I physically go to the computer, which is hooked up to a monitor and a keyboard, and I type it in there. But I don't have any such stuff for my new TrueNAS box, so I'm wondering... would the built-in web console thing be accessible, and allow me to enter the password to start up the "real" system?

6. Just curious: What's wrong with GELI?

Thanks in advance for any help.

 

[Arwen]

Some answers.

The boot-device generally does not contain anything that needs encryption, except the stock passwords, (which are encrypted like in "/etc/shadow").

It is not possible to move or encrypt items in boot-pool, as it is intended to be a firmware type OS, that is fully replaceable. Having some items on encrypted datasets would mean they would not be available at boot. And if moved to data pool, those items / directories would not part of the boot-pool, thus, no longer part of the firmware.

With full data pool encryption, the system dataset can not be on a data pool. The issue is that some people used USB drives for their boot-pool, which have more limited life. So it was normal to move the system dataset to one of the data pools. But, with full data pool encryption, you will have to leave the system dataset on the boot-pool. This also implies that you have a more reliable boot device, like SATA drive, reliable USB, (which few USB flash drives are), or a USB to SATA adapter. Some people have used M.2 NVMe drives for boot-pools as well.

GELI encryption has its place, but works on disks that make up the data pool(s). Their have been cases where a disk was not replaced properly and data loss occurred. (User error, but never the less bad.) ZFS native encryption also allows finer tuned selection of what to encrypt, (but you probably know this already).

ZFS full data pool encryption through TrueNAS Core would need password type authentication for your use case. Using passkey does automatic authentication on boot, through the boot-pool. Thus, passkey is more useful for returning disks for service or destruction, not anti-theft. With password encryption of the whole data pool(s), you bring up the GUI and enter the password to unlock it.

TrueNAS is intended to be used like firmware. If you make modifications in certain places, they may not survive an update. And some modifications can cause support issues. For example, something broke, but can't determine what because the modifications are preventing the stock response of: Backup configuration, re-install and restore configuration.


TrueNAS Core, (or SCALE), does not accommodate every users needs.

As I said elsewhere, their is no one true NAS to rule them all.

 

[rwv37]

Thanks. That's too bad. Oh well. I guess I'll revert to my Plan B of using the machine only for nonsensitive data (unless I've specifically encrypted that data elsewhere).

Question about a possible alternative, which I don't intend to do immediately (and maybe never), as I'd still like to play around with TrueNAS while I've got it here: Is there any reason why I couldn't or shouldn't just simply install actual FreeBSD on the Mini XL+? Wiping TrueNAS off the physical boot device and replacing it with FreeBSD, booting from there and GELIifying the "real" hard drives to be unlocked by password at startup time?

 

[Arwen]

As far as I know, all the iX hardware is common x64 type, so FreeBSD should work fine.

Their are really only a few things "special" about iX hardware. For example, they all have more than a simple desktops amount of disk slots. That makes sense, because the chassis' were selected for NAS use.

Plus, in the Mini line, they tend to have SATA DOMs for the boot device. Which do seem to work perfectly fine for that purpose. May be too small storage wise for a normal, full blown server OS. But, it is easily replaceable.

Last, the iX hardware tends to use Intel or other brand of high performance network chips for Ethernet. But not RealTek or Aquanta, (spelling?), which are the lower end of performance. (And price...)

 

[Davvo]

Iirc GELI isn't supported anymore (maybe just in the WebUI?) and SEDs are the proper way to do things.

EDIT: @winnielinnie is the expert here iirc