Sync, move and copy between Truenas paths and external FTP

[djb]

Hello, i have a task to execute with file handling and before i go to code development i check out whether it can be done indirectly via using the available utilities of truenas, like rsync, cloudsync, webdav etc.

Locally we have a Truenas FTP over VPN, many devices put .CSV log files in their home directory (chroot) every 5 minutes.
What i'm asking if possible are the following:

1. since those .CSV log files are thousands over a period of time, a simple monthly sorting in folder could be great.
2. once off Copy task to local truenas paths is required, some locations are SMB on same machine, some are FTP paths for other systems to have access to data.
3. once off Copy task to cloud FTP for external server to process the data.

It might be more convenient to perform those tasks in a different order. Any comments appreciated.

 

[sretalla]

1. since those .CSV log files are thousands over a period of time, a simple monthly sorting in folder could be great.

Creating script to move files based on date or filename

I have an FTP process that is constantly putting files into a directory. The date created is part of the filename in a format like this: YYYY-MM-DD-HH-MM-SS-xxxxxxxxxx.wav I would like to move the

unix.stackexchange.com

2. once off Copy task to local truenas paths is required, some locations are SMB on same machine, some are FTP paths for other systems to have access to data.

I'm not sure I understand what you mean... copying is possible to FTP and SMB destinations.

3. once off Copy task to cloud FTP for external server to process the data.

If it's once off, why wouldn't you just do the FTP transfer yourself?

If it's repetetive or you want to keep it in sync, a cloud sync task with ftp credentials could be an option.

 

[djb]

Creating script to move files based on date or filename

I have an FTP process that is constantly putting files into a directory. The date created is part of the filename in a format like this: YYYY-MM-DD-HH-MM-SS-xxxxxxxxxx.wav I would like to move the

unix.stackexchange.com

I'm not sure I understand what you mean... copying is possible to FTP and SMB destinations.


If it's once off, why wouldn't you just do the FTP transfer yourself?

If it's repetetive or you want to keep it in sync, a cloud sync task with ftp credentials could be an option.

1. Thank you , will study this.
2. Under System-> Cloud credentials, i didn't manage to find a type for SMB, only FTP. If there is another SMB accessible over VPN, \\192.168.50.50\Data , how can this be accessible for cloud sync ? Please elaborate.
3. Yes it's repetitive, since once off copy needed , sync will always copy again source file to destination.

 

[sretalla]

If there is another SMB accessible over VPN, \\192.168.50.50\Data , how can this be accessible for cloud sync ? Please elaborate.

You would use rsync for that.

 

[djb]

You would use rsync for that.

with what credentials ? Under System-> Cloud credentials, there is no option for general SMB credentials.
There are amazon S3, backblaze, box, dropbox, google etc... AND FTP that i already use.
Also on Rsync task configuration, the destination has to be SSH or rsync module type.

So i don't see any easy/obvious way rsyncing to a network SMB. can you please guide if there is a way?

 

[sretalla]

You said you need SMB on the same machine... so you don't need to use SMB, just use rsync to copy the files locally on the machine (no need for a cloud sync task. you coule use an rsync task or just a cron job to run rsync).

 

[sretalla]

Mount the SMB target if you need to using the information here:

Mounting a SMB share for plugins persistently

I have just cloned my FreeNAS 11.1-U5 to a Synology NAS to free up space for other usage. I share those files as a SMB share from the Synology NAS. I would like to mount the SMB share in FreeNAS and put them into the storages for the jails. Here is what I have done: Put the SMB credential...

www.truenas.com

Then run rsync towards it.

 

[djb]

thanks a lot, i will try the mount.