Export/Replica VM config to secondary TrueNAS

[lucapsg]

I have twin servers running TrueNAS Scale 22.12.1, let's say a "primary" and a "secondary".
The primary runs a few VMs that are replicated regularly to the secondary.
When needed, I would like to be able to start the VMs on the secondary but to be able to do so, the VM configuration must already be present in the latter and, to avoid surprises, possibly be identical to that of the primary.
From what I understand TrueNAS doesn't store the configuration of each VM in a specific file, like it does in ESXi (*.wmx file).
The configuration of *all* the VMs can be exported by making a backup of TrueNAS BUT the relative restore procedure does not allow selective restoration of the VMs configuration.
Right?
(System Settings > General > Manage Configuration > Download/Upload File).

I've searched the forum a lot but still haven't found a solution to export/replicate the configuration of one/several VMs to another host.
Anyone have suggestions?
Thanks in advance.

 

[sretalla]

You're right that the VM config is stored in the config DB, not in the filesystem.

As you suggest, taking a config backup is probably the right first step.

WARNING: Here be dragons...
The next things you might do are entirely unsupported and in no way tested by me:

use an SQLite browser like https://sqlitebrowser.org/dl/ to open the db (which you can find called freenas-v1.db inside the tar file you downloaded)

Look for the tables vm_devices and vm_vm

With those 2 tables, you should have the config of your VMs. Save them as csv.

You could then use the same process to export the config on your target/secondary system, then edit that DB to import the csvs to the relevant tables, save and re-import that DB...

If all is good, your VMs would then be on the secondary, ready to start.

 

[WI_Hedgehog]

I have twin servers running TrueNAS Scale 22.12.1, let's say a "primary" and a "secondary".
The primary runs a few VMs that are replicated regularly to the secondary.
When needed, I would like to be able to start the VMs on the secondary but to be able to do so, the VM configuration must already be present in the latter and, to avoid surprises, possibly be identical to that of the primary.
From what I understand TrueNAS doesn't store the configuration of each VM in a specific file, like it does in ESXi (*.wmx file).
The configuration of *all* the VMs can be exported by making a backup of TrueNAS BUT the relative restore procedure does not allow selective restoration of the VMs configuration.
Right?
(System Settings > General > Manage Configuration > Download/Upload File).

I've searched the forum a lot but still haven't found a solution to export/replicate the configuration of one/several VMs to another host.
Anyone have suggestions?
Thanks in advance.

This may surprise many, but TrueNAS purpose is Network Attached Storage. Spinning up Virtual Machines on a NAS server is not a good idea, as it works counter to the purpose of NAS which is highly reliable (and can be highly available) storage.

Proxmox Virtual Environment is designed to be a Virtual Environment with highly reliable Virtual Machines (and can be highly available), which is what you're asking for. Proxmox is not good as a NAS.

If you want a dual-purpose NAS and VM server the best bet is to set up Ubuntu or Fedora or whatever works for you, specifically to be a dual-purpose server. It won't be as reliable as TrueNAS or Proxmox which are reliable because they do one thing really well, but you will have what you need.

NAS and VM servers are mutually exclusive. NAS requires everything run perfectly without error and takes great steps to ensure that so data is not lost. VM servers require speed and relative stability, if a VM crashes you restart it, and potentially have lost a few database transactions.

Computers have "a lot" of errors happening inside them. We tend to think 1+1 is always 2, but if you're pushing around huge volumes of numbers at extreme speeds sometimes $128,592 + $64,184 = $32. This is why ECC memory exists (on-chip error detection and correction), storage devices have ECC checksums on each sector, data communication paths (like RAM to CPU, CPU to drive controller) have at minimum checksums, etc.

The above ensures, with relative confidence, the data made it uncorrupted and your house mortgage + kitchen remodel loan isn't $32. How much confidence? Well, maybe one error in 100,000,000,000,000. So fetch data from drive into memory is how many thousand steps? Virtual Machines run how much more data volume than a desktop? Do you really want your memory filled with Virtual Machine tasks corrupting and an errant instruction wiping out your ZFS file system? You're better off running an ext3 or ext4 file system which has mature recovery tools so if something does go wrong you can recover most of your data. (You're thinking "that's what backups are for," but backing up corrupt data for months before you discover a problem is how ransomeware works.)

 

[lucapsg]

@sretalla
Thanks for your quick response!
I had already fiddled with SQLite browser and TrueNAS configuration backup but was hoping I could do without it.

@WI_Hedgehog
What does all this have to do with the title of the post?
It is not for me to praise or disparage one or the other platform and probably no one is in a position to do so.
Before adopting TrueNAS Scale I evaluated the pros and cons of a hypothetical solution based on Proxmox and ZFS replication seemed to me to be done better in TNS.
Right or wrong, I made my choice.
I would therefore like to solve the problems that arise from time to time, not throw everything away, only on the basis of a mirage.
After all, nobody is perfect.

But since we're at it, I just want to mention a few irrefutable aspects:
1. both are based on the same operating system, Linux/Debian;
2. both use the same hypervisor, KVM;
3. both use OpenZFS 2.x;
4. Proxmox is oriented (almost) exclusively to virtualization;
5. TrueNAS Scale is a complete NAS appliance, with various features, including virtualization, all manageable from a convenient WEB interface.

Then, we can also talk about the gender of angels and have our preferences BUT the above cannot be disputed.

Seriously, folks, what do you think of a "selective" TNS configuration restore procedure?
Any other suggestions?

 

[Johnny Fartpants]

Not very neat or seamless but just thinking off the cuff you could create a dataset and include that in your replication to your secondary system and run a cron job on the primary to make regular copies (hourly or daily) of your config file like this cp /data/freenas-v1.db /mnt/zpool/dataset and at least that way if you needed to failover you would have the config file to restore from.

Like I said not very neat or seamless but...

 

[Johnny Fartpants]

This may surprise many, but TrueNAS purpose is Network Attached Storage. Spinning up Virtual Machines on a NAS server is not a good idea, as it works counter to the purpose of NAS which is highly reliable (and can be highly available) storage.

I think if this was 100% true then SCALE wouldn't exist. You might want to google hyperconvergence.

 

[WI_Hedgehog]

I think if this was 100% true then SCALE wouldn't exist. You might want to google hyperconvergence.

I've run VMware throughout the various flavors for many years in a production environment. It certainly has its uses, as do other solutions.

However this topic is about hacking TrueNAS to work as something it was not designed for, the history of such shows that is not well-advised. One might use the cliche "Use the right tool for the job." Using a rock as a hammer works, but there are better options.

 

[sretalla]

what do you think of a "selective" TNS configuration restore procedure?

That's what I described for you. What you're asking for is specifically not intended to happen in the appliance product.

If it's not "easy enough" to use the steps I suggested, then maybe a feature request (which I doubt will get any support) is required to move you forward.

 

[lucapsg]

DON'T DO IT!

TNS saves the configuration in a real DB (SQLite), with relational constraints, indexes, default values, etc... and each table has a specific schema.
.CSV files, including those exported from SQLiteBrowser, do not contain any schema and as a result the original one is overwritten/lost during import! I got ripped off like a rookie...

It worked, or almost, until I went to change the configuration of the VMs. At that point... hope you have a valid backup!

Rule N.1: TN must be managed with its web GUI!
If something isn't planned, don't find a way to do it.

Now, if someone really wants to venture (I was forced), at their own risk, one way to export/import portions of the configuration from one host to another is this (perhaps the only way):

1. export the entire source host DB to a SQL file
2. edit the SQL file and copy the commands of interest
3. open the DB of the target host
4. run the copied commands (in SQLiteBrowser paste them in the "Run SQL" tab)
5. check that the last progressive ID of the modified tables corresponds to the one stored in the sql_sequence table and update it if necessary

Hoping no one else gets hurt.

 

[2twisty]

I use a separate VM host (xcp-ng, but used to use ESXi) and use NFS storage for my VMs. I replicate my primary TN (SAN) to my secondary (BACKUP) every hour.

I have identical dataset and share setups on both TN boxes, so that I can just shut down my VMs, change the IP address of BACKUP to match SAN, and the host is none-the-wiser.

This setup is only there for extreme disaster recovery, since once I make changes to those files on the backup server, if SAN tries to replicate to BACKUP, it will either fail or have bad consequences. I'd have to replicate them back to SAN from BACKUP to start using SAN again, so it's not something I would want to do for simple maintenance that would cause me to take SAN offline.

Since this is a homelab, it's not a big deal to tell the wife that the media files are unavailable for a while, but it would work the event of a complete hardware fail, or if I was completely rebuilding SAN.

Been thinking about how to configure a proper failover setup, but have not even explored what that would be like in TN or if it is even possible. That would be preferable since I could just take SAN out of service and it would fail over to BACKUP automatically. I guess I have some googling in my future, lol.