ZFS Datasets and Serverless Databases

[elPicione]

Hello everybody

My brother is starting his new job in the medical field, so i built him a nas using truenas. Awesome.

For radiography images AND database they use a software called vixwin.

Assuming this can be stored in a dataset:
Is it advisable to turn on write sync?
Is it advisable to turn off compression?

For the database part should these options be the safest to prevent database corruption? On the other hand, could the massive x-ray images benefit a lot from compression, since they are be uncompressed, thus saving a lot of space?

What would you suggest?

It could also be possible to make a dataset for database and a separate dataset for images, and then link them together using hard links, but that sound like complicating things further.

Thank you :)

 

[sretalla]

If you need to be assured that all writes are really on the disk before the application is told that it's the case, sync writes is a must.

Compression won't make a lot of difference to anything at a higher level, but you could opt out of it if capacity isn't a concern.

It's certainly true that TIFF images will compress well.

It's also true that the pool design that you want for a database may not be the same as you will want for image storage/archiving.

Probably one or more mirrors for the DB and RAIDZ2 for the images will work, but that means a carefully planned set of pools and VDEVs.

I doubt that hard links will work in the way that you expect, so there may be some work to do in terms of how you're presenting the storage to the application (a clue in the name of vixWIN might indicate you'll be running a Windows VM, which will make it tricky, but out of scope for this forum in terms of how to make it work).

 

[elPicione]

Hi sretalla and thank you for your reply!

If you need to be assured that all writes are really on the disk before the application is told that it's the case, sync writes is a must.

Could it also be possible to leave it to "Standard"?
The tooltip says -"Standard uses the sync settings that have been requested by the client software"- is the "client" Windows or the program itself?

If vixwin is smart enough, it should lock the database during any transaction anyway, like sqlite does (I think?). Then again, I don't know if the locking mechanism would work in a cifs/smb share since it may be filesystem dependent.

It's certainly true that TIFF images will compress well.

It's also true that the pool design that you want for a database may not be the same as you will want for image storage/archiving.

Probably one or more mirrors for the DB and RAIDZ2 for the images will work, but that means a carefully planned set of pools and VDEVs.

That would be cool. Will probably do it when the budget will be better

I doubt that hard links will work in the way that you expect, so there may be some work to do in terms of how you're presenting the storage to the application (a clue in the name of vixWIN might indicate you'll be running a Windows VM, which will make it tricky, but out of scope for this forum in terms of how to make it work).

Honestly i will probably just copy&paste the whole ass program folder in the share, map the network drive to Z: and then hope for the best LOL.

Sounds nasty but my brother said it can be done since someone already did it. Unfortunately it's hard to find discussions online for this kind of professional programs. Maybe again database and images location can be changed in the program.
If a windows machine will be necessary, we'll probably switch to Windows Server + storage spaces + RemoteApp. I prefer TrueNAS but don't like the overhead of a VM

Compression won't make a lot of difference to anything at a higher level, but you could opt out of it if capacity isn't a concern.

Following your advice, i would set compression on and sync on

 

[ChrisRJ]

I would assume that the vendor provides technical details about how the software should be installed. In addition, since we are in the health space, there will be a bunch of regulations and other legal stuff in the game, that need to be followed. And ignoring technical specifications from the vendor can probably lead to a pretty nasty legal situation, if something goes wrong. Just assume that a critical part of an image gets corrupted, resulting in a wrong diagnosis and later death of a patient.

Without knowing anything about the underlying DB (and I would hope it's something production-grade), not enforcing sync-writes would be a really huge risk. What would be the gain anyway, if you left sync writes on "standard"? It is a wrong assumption BTW that the DB, when writing to a mapped drive that comes in via SMB, has control about whether or not writes will be synchronous.

 

[elPicione]

Hi ChrisRJ thanks for replying

We want to put images and databases in TrueNAS for multiple reasons. First, being able to access radiographic images from different terminals with IPS and VA panels if we share them between PCs, to improve diagnosis.
While we are sharing data, we want to place it in the most resilient place: a ZFS filesystem in a FreeBSD OS.

Their program may be untested in a SMB/CIFS scenario, but it's always miles better than the standard Dell Windows 7 Pro machine they provide. With Admin privileges and USB storage being pulled in&out, it is begging to be nuked by some ransomware.
I will try and contact the vendors about this, but they will probably just either cover their ass saying this can't be done, or offer us some crazy expensive server-client license we can't afford now, in a small 3-4 PCs studio.

I like Windows and it can be secured. But even a Windows Server just isn't as good and robust as TrueNAS as a file server.

We can think of 3 possibilities right now:

1. Program installed on each client, just the data in the share - this can be done if the program allows reading database and images from a different path
2. The whole program in the share - backup everything and stress it for corruption
3. Program installed on each client, hard links in Windows to the share

I would assume that the vendor provides technical details about how the software should be installed. In addition, since we are in the health space, there will be a bunch of regulations and other legal stuff in the game, that need to be followed. And ignoring technical specifications from the vendor can probably lead to a pretty nasty legal situation, if something goes wrong. Just assume that a critical part of an image gets corrupted, resulting in a wrong diagnosis and later death of a patient.

Without knowing anything about the underlying DB (and I would hope it's something production-grade), not enforcing sync-writes would be a really huge risk. What would be the gain anyway, if you left sync writes on "standard"? It is a wrong assumption BTW that the DB, when writing to a mapped drive that comes in via SMB, has control about whether or not writes will be synchronous.