rsync adding subdirectory

[diedrichg]

Hello, I am trying to migrate my legacy NTFS drive data to my new RAIDZ1. I don't have the proper knowledge in using rsync though.

• The legacy drive /mnt/SeagateBackup is attached locally and has been successfully imported
• The destination dataset is /mnt/family/videos

When using

Code:

rsync -arv /mnt/SeagateBackup/Videos /mnt/family/videos

my result is /mnt/family/videos/Videos. How do I keep rsync from adding the original directory? How do I get all files/subdirs in mnt/SeagateBackup/Videos directly to /mnt/family/videos without creating the extra Videos directory?

p.s. I'm sure this has been covered a thousand times but I've Google'd my a** off and I can't find the solution so in advance I'm sorry for the repetitive question

 

[cyberjock]

Oh boy. As I write this I am migrating a Windows Server to Freenas. Check out my thread: http://forums.freenas.org/showthread.php?10031-Migrating-to-FreeNAS-from-Windows-server-2008-R2

The short is that you have zero control of rsync. If rsync expects file A to be in location B and its not there, it will create it. It may also delete file A from location C because it thinks that the file shouldn't be there. From my experimenting with rsync in the last week, it tends to use date and time to determine if a file is correct. This can also cause problems. My thread explains the obstacles I've had and how I dealt with them.

The best advice I can give is to use CIFS or NFS to copy your data to your server(that's what I'm doing). I have no experience with NFS, but CIFS has packet checksums to make sure your data isn't corrupted. I assume NFS has similar technology.

Edit: Just to clarify my post. rsync seems to use data and time, but you can choose to use checksums to verify the file contents. But checksums are a terrible idea for normal use because your system will literally read every file to create a list of checksums, then the remote location will have to read every file to verify the checksums. It will then recopy any files that don't match. This is extremely I/O intensive.

 

[bollar]

Part of the problem is that Unix is case sensitive and Windows is not. If you changed the command to this:

Code:

 rsync -arv /mnt/SeagateBackup/Videos /mnt/family

Then files should be put into a folder called family/Videos -- unfortunately, that's not the same thing as family/videos.

Like noobsauce80 says, it'll be a lot easier to use CIFS to do the transfer.

 

[diedrichg]

Thank you for the replies. I was afraid of that. I guess I'll have to do it the hard way since the backup drive is mounted locally and therefore CIFS won't be an option. Thanks though!

 

[fracai]

All you should need to do is put a '/' after 'Videos'. In other words:

Code:

rsync -arv /mnt/SeagateBackup/Videos/ /mnt/family/videos

Basically, the difference between a trailing slash and no trailing slash is whether the contents of the directory, or the directory itself is transferred.
You could put a '/' on the destination as well ('/mnt/family/videos/'), but it's not necessary and won't change how rsync behaves. You could also, of course, leave off the 'videos' at the end and just rename the folder if you want it lowercase.

I also usually use a '-P' when I rsync. This turns on 'partial', so you can resume a interrupted transfer and 'progress', so you can see how much has been transferred and how much is left. Note that you shouldn't use 'partial' if you're changing the source or destination after a previous transfer; you'll end up with duplicate folders under different directories. You could also use '--delete' to remove any destination files that aren't on the source, but I wouldn't recommend that if you're just getting started with rsync.

Also, I'm fairly sure that rsync checksums transferred data so you should be safe against errors during the transfer. The size and date comparison is a quick check for determining whether a file has changed at the destination and needs to be updated. If the size or date is different, rsync performs the checksum comparison to determine which blocks of the file need to be updated.

Contrary to noobsauce80's opinion of rsync, I find it to be a very flexible tool. It may take a bit to learn how to use it, but it can be very powerful.


Oh, and '-a' is an alias for '-rlptgoD', so you don't need to specify '-r' again. I usually run '-Pva'.

 

[cyberjock]

There's no doubt in my mind that rsync can be very powerful. In fact, if I were setting 2 FreeNAS servers and wanted one as a backup server , rsync would be a very viable option. I'm not too knowledgable on snapshots(yet). But I know that snapshots are another useful tool for backups. I'm not quite privy to when one is better/worse than the other. That will be phase 2 of my project on the other thread.

 

[diedrichg]

OMG! That was it! I was missing the / at the end of the source volume! It never occurred to me. Thank you so much fracai!

 

[fracai]

diedrichg, glad I could help.

There's no doubt in my mind that rsync can be very powerful. In fact, if I were setting 2 FreeNAS servers and wanted one as a backup server , rsync would be a very viable option. I'm not too knowledgable on snapshots(yet). But I know that snapshots are another useful tool for backups. I'm not quite privy to when one is better/worse than the other. That will be phase 2 of my project on the other thread.

I think if you're transferring a full dataset, or updating a backup of a dataset, snapshot replication is probably best. If you're working with a subset of a dataset, rsync is the best option.