rsync? sftp? best way to auto download from seedbox

[brichard]

Hi,

I have assembled and successfully set up my first FreeNAS ;)

Have spent the last 6-7 hours trying to find out how to auto download files from my seedbox (feralhosting).
In this thread OP tries to do it via sftp
https://forums.freenas.org/index.php?threads/mount-sftp-folder-in-jail.41513/#post-265868
I think rsync maybe would work better, but can't figure out how do I connect to the remote server.

Anyone has set this up successfully?

Thank you

 

[pirateghost]

Hi,

I have assembled and successfully set up my first FreeNAS ;)

Have spent the last 6-7 hours trying to find out how to auto download files from my seedbox (feralhosting).
In this thread OP tries to do it via sftp
https://forums.freenas.org/index.php?threads/mount-sftp-folder-in-jail.41513/#post-265868
I think rsync maybe would work better, but can't figure out how do I connect to the remote server.

Anyone has set this up successfully?

Thank you

What do you mean you can't figure out how to connect to the server?

rsync -avzh --progress -e ssh user@server:/path/to/directory /path/to/destination/on/freenas

 

[brichard]

Thank you.
It seems that I am not knowledgable enough on the topic to be able to ask a question, so just keep looking around.

 

[pirateghost]

Thank you.
It seems that I am not knowledgable enough on the topic to be able to ask a question, so just keep looking around.

You don't have to be knowledgeable enough to ask a question, but your question was very open ended like you didn't actually research how to rsync. You posed it in such a way that it appeared you were looking for someone to present you with a customized walkthrough. You never stated what you tried to 'make it work'.

This configuration is quite literally, extremely simple.

In system tasks configure a cron job.

In this cron job will be either a link to a rsync script you have on the pool or directly write your rsync command in the task.

Set the times it will run and enable it.

Looking at the man page for rsync will give you options on how to output the info to logs for troubleshooting...

There are much more complicated configurations that would work but require a lot of networking understanding and time to set it up. Rsync is the quickest and easiest implementation

 

[brichard]

Hi. Yes, I was looking for someone who has experience in this specific setup.
My question was around how do I connect to the remote server.
I need to ssh using a public key authentication, but I can't figure out how to set it up on FreeNAS - as I understand it would needed to be set up in command line and then the public key copied to the seedbox to enable 'passwordless' access for the rsync chron job.
If I would be able to ask the correct question for all stage, I wouldn't need to ask, but could find the answers.
Your help is of course much appreciated.

 

[pirateghost]

All you need is your public key from freenas. Once you have that copied to clipboard, SSH from putty or whatever to your remote box. Edit ~/.ssh/authorized_keys on the remote box. Add your FreeNAS pub key at the end of the file.

You now have SSH access password less to your remote server

Test by dropping to CLI on FreeNAS and ssh user@remotehost

 

[brichard]

Do I need to set up a new user for this specific task and generate a new SSH key (saving the key in user's home directory) for it or need to generate one for root?

 

[pirateghost]

Do I need to set up a new user for this specific task and generate a new SSH key (saving the key in user's home directory) for it or need to generate one for root?

That's up to you but I think your root user already has a key

 

[brichard]

what I did so far; please correct me if something is wrong

1. ssh-keygen -t rsa
2. more id_rsa.pub
3. copied that to users > root > SSH Public Key:
4. ssh into seedbox
5. nano -w ~/.ssh/authorized_keys
6. paste key, exit, save
7. chmod 700 ~/.ssh
8. try ssh to seedbox
9. cheer
10. create dataset rsync/feralhosting
11. chron job

-user: root
-command: rsync -avzh --progress -e ssh user@hypnos.feralhosting.com:/private/rtorrent/data main/rsync/feralhosting

error I am getting now

rsync: change_dir "/private/rtorrent" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1664) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

 

[pirateghost]

/private/rtorrent doesn't exist?

Is your seedbox path actually /home/user/private/rtorrent?

 

[brichard]

missed the / before main on local

rsync -avzh --progress -e ssh user@hypnos.feralhosting.com:/private/rtorrent/data /main/rsync/feralhosting

no error emails now, but seemingly nothing happens

 

[pirateghost]

Is this in a jail? Your destination path doesn't exist in a normal freenas install. Your pool will always be in /mnt

 

[brichard]

in storage the structure looks like

>main
...>main
......>rsync
.........rsync/feralhosting

so the correct path is /mnt/main/main/rsync/feralhosting ?
Did I mess up something by running the job with wrong path?

 

[pirateghost]

in storage the structure looks like

>main
>main
>rsync
rsync/feralhosting

so the correct path is /mnt/main/main/rsync/feralhosting ?
Did I mess up something by running the job with wrong path?

--progress will output the job to the screen, and if you didn't see anything output there (like data being transferred) you didnt mess anything up.

so your pool is called 'main'? if so, the path would be /mnt/main/rsync/feralhosting

 

[brichard]

seemingly up now :)
rsync -avzh --progress -e ssh user@hypnos.feralhosting.com:~/private/rtorrent/data /mnt/main/rsync/feralhosting

Thanks a lot for your help

 

[Chris Gornall]

Hi,

I have assembled and successfully set up my first FreeNAS ;)

Have spent the last 6-7 hours trying to find out how to auto download files from my seedbox (feralhosting).
In this thread OP tries to do it via sftp
https://forums.freenas.org/index.php?threads/mount-sftp-folder-in-jail.41513/#post-265868
I think rsync maybe would work better, but can't figure out how do I connect to the remote server.

Anyone has set this up successfully?

Thank you

I just went through the same problem I found the best solution was to use lftp with sftp (rsync and other solutions were too slow) I made Instructions for how to set it up in a jail and a script to automatically set it up for the lazy.