SOLVED Replication for SMB shares

[YujiTFD]

Greetings.

I have single pool/single storage TrueNAS Core installation with several SMB shares. Is there a way to make/schedule a SMB shares replications or TrueNAS/ZFS offers dataset replication only? I'm yet new to TrueNAS, be kind to me :)

UPDATE: No, you cannot replicate individual share unless each share situated within it's own dataset. In order to sync directory contents of
SMB shares, one would want following command: rsync --stats --progress -A -a -r -v --no-perms /mnt/src/ /mnt/dst
Mind the trailing /, it tells rsync to copy everything INSIDE your /mnt/src/, not /mnt/src itself.

 

[cat-man-du]

Hello,

Are you saying that you have multiple SMB shares, each one sharing a subdirectory of one larger dataset? If so, then ZFS replication will not be able to transfer/replicate your SMB shares individually. ZFS replication works at the dataset level, so you would only be able to configure replication with the same granularity as your datasets.

You could use something like rsync to accomplish file-level replication, but if possible, you should consider re-organizing your pool to use individual datasets. It may not be practical, depending on your setup, but it would probably make your life easier in the future. :)

Cheers

 

[YujiTFD]

Hello,

Are you saying that you have multiple SMB shares, each one sharing a subdirectory of one larger dataset? If so, then ZFS replication will not be able to transfer/replicate your SMB shares individually. ZFS replication works at the dataset level, so you would only be able to configure replication with the same granularity as your datasets.

You could use something like rsync to accomplish file-level replication, but if possible, you should consider re-organizing your pool to use individual datasets. It may not be practical, depending on your setup, but it would probably make your life easier in the future. :)

Cheers

Thank you for the prompt response :)
As for my situation, you got it right, pity I won't be able to do it because I really liked it when I was using BTRFS @ Synology NAS.
But for the rest - what exactly would I gain with individual dataset setup, why life could be easier this way? I'm asking since I've been there and I really don't like guessing future volume size. I now have 6 x 6 TB HDDs in mirrored vdevs setup, 6.87 TiB free. If aforementioned benefits really are awesome, what would it take to re-align the storage, gradually move existing data to te newly created datasets? And what if I'll fill one dataset uo, leaving others with abundant space on 'em?

 

[Patrick M. Hausen]

Commonly people create one dataset per share and replicate that. There is essentially no overhead in having as many datasets as you like unless you are going into the thousands. No limit I ever experienced for practical considerations.

Edit: reading your last reply - datasets are not fixed in size. You are not partitioning your pool in any way with datasets. You can set quota, but you don't need to. All datasets will just eat into the free space when filled with files ...

What would you gain? Snapshot and replication can only be set per dataset, not per directory. You asked how to set policies individually. So the answer is: use individual datasets. TN is really elegant with its snapshot and replication tasks. You can e.g. set a recursive policy for all datasets below a certain higher ranking one, but exclude some of the lower ones if they deserve special treatment. Then set additional policies only for those.

 

[cat-man-du]

Ah, are you putting some sort of limit on your dataset size? You can certainly do that, but it's not necessary -- you can simply create as many datasets as you want (well, technically there is a limit, but it's absurdly high), and allow each to simply take as much space as it needs. As far as re-aligning, I would just make new datasets in whatever scheme would allow you to replicate how you want (maybe that's one dataset per share, maybe you group them, it depends), and then you could use rsync.

 

[Patrick M. Hausen]

and then you could use rsync

Or zfs send|zfs receive ...

With rsync you can of course sync single directories regardless of dataset boundaries. Probably have to roll your own small scripts and activate them as cron jobs. I don't know if the UI supports anything but syncing entire datasets. But then I never use rsync, because, well ... zfs send of snapshots is just so much superior.

 

[cat-man-du]

Hey Patrick,

Or zfs send|zfs receive ...

Would send|recv work if the old data is all stored in one giant dataset, though? I'm not sure how I would do that without using some sort of file-based tool.

 

[Patrick M. Hausen]

Would send|recv work if the old data is all stored in one giant dataset, though?

Of course it would. You just send the entire dataset ... might take some time depending on bandwidth.

Second you can of course reorganise your data without (high) risk of loss. Create new child dataset as desired with a temporary name. Copy data from one share directory to that dataset when the share is not used (outside business hours etc.). Stop SMB service. Rename directory to e.g. "directory-old". Rename dataset to "directory". Start SMB service. When all looks good, delete directory-old.

Rinse and repeat as necessary.

 

[YujiTFD]

Thank you everyone, now I see I was wrong about it, different datasets really are useful.
Now, the thing I have left is to how to transfer data from oner dataset's different directories between several datasets. Since I have some free space, it's obvious I have to copy share's data one by one to their new dataset locations, emptying relevant directories also one after another. Question is: can I zfs send|recv this or I do have to copy files as usually, which will take REALLY long time? I would really like to make an internal copy since I have local access to the TrueNAS server.

 

[Patrick M. Hausen]

As I sketched out in my last post you will have to create the dataset, then copy files - one by one, probably. The zfs send | zfs receive magic works on the block level, that's why it only covers entire datasets.

 

[YujiTFD]

As I sketched out in my last post you will have to create the dataset, then copy files - one by one, probably. The zfs send | zfs receive magic works on the block level, that's why it only covers entire datasets.

Yeah, thought so. Got merely confused by https://www.truenas.com/community/threads/replication-for-smb-shares.94736/post-655373 and came to a wrong conclusion, that send/receive can operate on file level, my bad. It's just file copy over SMB is excruciating, is there really no way to correctly copy data on disk level via CLI?

 

[Patrick M. Hausen]

Nope, but you can of course copy on the file level via CLI without going through an SMB client and the network.

@anodos or anyone else in the know - could you be so kind to explain how to recursively copy a directory tree while preserving all those extended attributes and ACLs that I in my small home network intentionally avoid?

Sorry, I cannot give you the command but I do know that it is perfectly possible to do a local copy. It's just that I have all shares with guest access and no ACLs for my small family network. And at work where we use larger enterprise sized TrueNAS systems we don't use them for SMB sharing.

Someone will step in, I hope.

 

[YujiTFD]

Someone will step in, I hope.

Let me do this :)
At first I tried cp -Rp /mnt/src/ /mnt/dst, it went fine, but then I thought: how can I be sure everything actually went fine, what it something did happen among tens of thousands of files I will be copying? So I moved to rsync and stumbled upon rsync: mkstemp [...] failed: Operation not permitted (1): it seems to be that rsync is unable to sync SMB share permissions correctly, which lead me to the forum post by desertrider.
In short: for rsyncing SMB shares with ACLs, one would want following command:
rsync --stats --progress -A -a -r -v --no-perms /mnt/src/ /mnt/dst
desertstrider's example had --partial argument, but personally I'm not a fan of one, so I skipped it. Thus, not only you can see individual progress for each file, where cp leaves you hanging until whole process is over, you also end up with nice summary: