What method to use for mirroring HDD's across network?

[Patrick5S]

Hello everyone. I finally decided to join this forum after lurking for a few years while I tried to decide on whether FreeNAS was the way to go.

Anyways, here's the FreeNAS box for now. Running FreeNAS - 9.2.0 Release x64, running from a 4GB flash drive.

Antec 300 case
Green friendly 500w PSU
P8H77M-LE Asus motherboard
i3-2120 CPU @ 3.30GHz
16051MB of memory (2 sticks of 8GB)
Western Digital 2.0TB Green drives x 5 in a ZFS RaidZ config for 7.1TB

I created a volume and Dataset, and called it "StorageCloud" with full access permissions to anyone. I'm sharing the volume/dataset via CIFS and have it mapped to each machine in the house, calling it the "S: Drive". I can successfully create, read and delete files on this S: drive from all the machines. I'd like to keep the persmissions/users/groups as simple as possible. This is a home network solution, and I don't want my wife or anyone else having to enter user names or passwords to get into this share. If you're able to get onto my home network, you can have my files.

Anyway, I'm still in testing mode using a Win7 Ultimate desktop PC for backing up files. What I want to have happen is each night, the "StorageCloud"/FreeNAS box should look at a specific drive or folder on a computer on my network, and I want it to compare yesterday's backup with what it sees today. If there is no difference, do nothing. If files have been added/deleted/modified or anything different than what it sees, make changes to the backup on the FreeNAS box. The reason for this is because my HTPC downstairs (Win7 Ultimate machine) has a 5.45TB drive for movies and music and photos. That 5.45TB was created by setting up two 3TB drives in a Raid0 array via onboard motherboard raid controller. Since Raid0 cannot be trusted for anything besides speed, I want to crate a mirror (think of it as a Raid1) image of the drive on the FreeNAS box. Each night I'd like the FreeNAS box to look at that 5.45TB, see if any movies or music have been deleted/added/changed, and if so, make adjustments to the backup. If nothing has changed for that day, carry on. I'm looking for exact mirror images of each other, if possible, between the HTPC 5.45TB storage drive downstairs and the copy upstairs on the FreeNAS box.

I tried this with the Windows back up or restore (see pictures) but my initial tests are not what I expected. It seems to be creating compressed or zipped backup files and using a different naming convention...plus it's taking forever and for some reason the 132MB folder I told it to back up during this testing phase is now almost 100GB because it's pulling an entire image of the C: drive, which I don't remember clicking on anything except for the 132MB test file....not sure what Windows is doing.

My question then is, what bit of software or what plugin do you recommend to solve the requirements of this home network? I just started dabling into Rsync a bit for FreeNAS but got a little confused when it said I had to use groups and permissions.

 

[pirateghost]

from your screenshot of your backup/restore in Win 7, you are actually backing up all users content (note the 'CONTENTS' field, and creating a system image....that takes a long time and a lot of space

What you are asking for in your backup needs is prime candidate for RSYNC, because the windows backup does exactly what you are seeing, without too many options available for it.

I don't understand why you would need to use groups and permissions for RSYNC, but I use RSYNC with a Windows app called DeltaCopy, and it works wonderfully

 

[Patrick5S]

Thanks for the quick reply pirateghost.

I ran into a new issue this evening. Since the Windows backup procedure did not provide the outcome I expected, I tried deleting my "Test Backup" folder on the FreeNAS box via Windows, and was denied folder access. Oddly enough though, I can delete empty folders I created from Windows that are not associated with the backup. Windows seems to have applied "Archive" and "Read Only" permissions to these folders and I'm not prompted to enter a username/password for authenticating I'm the owner of the FreeNAS box. Attempting to uncheck the Read Only and Archive boxes result in error messages. Any help in this?

pirateghost, do you have a tutorial or a very beginners guide in setting up Rsync for what I'm trying to do? Or would you mind instructing me in how you have your setup running? Thank you.

 

[Patrick5S]

Checking up on this. Still an error while trying to delete those backup files.

Also, a new symptom I've been noticing each day of the week...for some reason the Win7 PC's that I mapped the network drive for lose the connectivity until I double click on the network share icon. Then it "re-syncs" without a flaw but for that first click the icon has a red X under it. Any idea why?

 

[enemy85]

I don't understand why you would need to use groups and permissions for RSYNC, but I use RSYNC with a Windows app called DeltaCopy, and it works wonderfully

Hi pirateghost,
sorry to bother you but I'm trying to set up Rsync on my FreeNAS box and DeltaCopy on windows pc, as you already did, in order to create a backup solution.
I'm just having little difficulty to set everything properly...so the question: would you be so kind to explain me how did you set Rsync on the freenas box?

Thank you very much for your help.

 

[pirateghost]

Hi pirateghost,
sorry to bother you but I'm trying to set up Rsync on my FreeNAS box and DeltaCopy on windows pc, as you already did, in order to create a backup solution.
I'm just having little difficulty to set everything properly...so the question: would you be so kind to explain me how did you set Rsync on the freenas box?

Thank you very much for your help.

read up on how rsync works (the FreeNAS manual is a good starter), and configure whichever way you like.

There are a couple of different ways you could go about it. Personally I find the DeltaCopy 'server' function better in terms of configuration. Configure your DeltaCopy as the server (it is pretty clear in the app, as it is dead simple), and then on FreeNAS you configure an RSYNC Task to connect to your 'server' and pull the files.

 

[enemy85]

so this should mean that in that case is the freenas box "pulling" out of the pc the files i want to backup, right?
I'll try as you suggest! Thanks!

 

[pirateghost]

so this should mean that in that case is the freenas box "pulling" out of the pc the files i want to backup, right?
I'll try as you suggest! Thanks!

Yes

Sent from my Nexus 5

 

[enemy85]

Yes

Sent from my Nexus 5

Sorry to bother even today...After reading how to set it up properly, I tried and I managed to make the freenas box to backup my pc stuff. Now I was doing some test in order to understand how it works exactly and for what i understood so far, it seems that if i modify (not delete) one of the files already backupped on the nas, the second time the rsync starts, FreeNas will substitute the old version with the new one. Is that correct?
how can i manage to keep "the record" of all the files copied on the nas? the only solution that comes to my mind is to set snapshots of the backup dataset to happen just at the end of the rsync task...in that case i should keep on the nas track of all the files passed by (as a sort of differential backup)...or am I thinking it wrong?
In case i'm correct another little question comes to my mind...I don't know exactly how snapshot work, but in case i need to recover just one file from a snapshot, would it be possible or i'll be obliged to have the complete snapshotted dataset back to that day?

sorry for all the answers and thank you if u would answer me.

Fra