What is the fastest method for transferring data from a NTFS drive to FreeNAS

[scottmaverick]

So I have tried using the GUI import disk option in freenas 9.10 but the transfer speeds are horrible, 10MB/s. I previously ran tests on my pool and got good speeds for read and write operations 500MB/s. Is there any other way to transfer my files from my NTFS drive to my pool that might be quicker?

 

[Mirfster]

Tried over the network?

 

[scottmaverick]

Tried over the network?

I know I'm new to this, but how would connecting over the network possibly be faster then connecting it directly via sata?

 

[nojohnny101]

just connect the drive into a computer and then connect to your freenas box however you want (CIFS share for instance) then transfer your data to the freeNAS box.

This is assuming that you already have drives in your FreeNAS box and that a pool and vdev is setup. That is what you want to do correct? just transfer the data from the drive to your existing FreeNAS box correct? Adding this drive (and subsequently all data on it) to the FreeNAS is a completely different question.

 

[scottmaverick]

So it seems like it would be useful for me to explain my setup a little bit better.

2x LSI SATA/SAS HBA connected to a backplane with 24x SATA connecotrs. I have 12 3TB hard drives in a raidz3 config connected to one of the HBA's that makes up my pool in freenas. I have connected a 1TB drive formatted NTFS to the backplane which is connected to the other HBA. I went through the webgui in freenas and used the import disk command to copy the data from the 1TB NTFS drive into my pool.

When I look at the disk IO in freenas it shows a write of only 10MB/s to each drive in the pool, and a read of 60MB/s from the NTFS 1TB drive. This seems really slow to me considering I tested the pool earlier and I was able to read and write at about 500MB/s to it.

Why is it so slow and what can I do to improve the performance, I have another 20TB of data to transfer.

 

[Mirfster]

I have 12 3TB hard drives in a raidz3 config connected to one of the HBA's that makes up my pool in freenas.

While RaidZ3 is great for redundancy, it is not the fastest configuration.

I have connected a 1TB drive formatted NTFS to the backplane which is connected to the other HBA. I went through the webgui in freenas and used the import disk command to copy the data from the 1TB NTFS drive into my pool.

This is permissible, while I have not personally done it myself. I just copy it over the network with RoboCopy.

When I look at the disk IO in freenas it shows a write of only 10MB/s to each drive in the pool, and a read of 60MB/s from the NTFS 1TB drive.

A single RaidZ3 vDev will operate at basically the speed of a single disk.

This seems really slow to me considering I tested the pool earlier and I was able to read and write at about 500MB/s to it.

Would depend on how you tested..

Other factors:
The underlying structure of FreeNAS may not be optimized for NTFS
Now sure how much RAM you have, which would help a lot
Don't know if you are copying a ton of *small* files, which for sure will impact speed

Basically, that is why I said it was probably easier to just copy it over the network. However, I am assuming that you at least have Gigabit connectivity...

 

[scottmaverick]

Only running 12GB ram right now, but the file transfer process isn't using it all. The files are all media based so smallest is >100MB. Is their a way mount multiple NTFS drives at once in freenas so I can simultaneously transfer from multiple drives?

 

[depasseg]

it shows a write of only 10MB/s to each drive in the pool, and a read of 60MB/s from the NTFS 1TB drive. This seems really slow to me considering I tested the pool earlier and I was able to read and write at about 500MB/s to it.

Clearly the issue isn't writing to the pool, it's reading from the ntfs disk. My guess is that ntfs support isn't very optimized under freebsd, and wouldn't be surprised to see one of your cpu cores heavily stressed. But that's just a guess.

And I think that's why people are suggesting the network route, since a windows client can read from an ntfs disk at a faster rate than freebsd can.

 

[scottmaverick]

All cores look to be about the same, I will give the network transfer a shot once I figure out how to do it.

 

[depasseg]

All cores

What about the mount.ntfs process (or something similarly named)?

 

[SweetAndLow]

The NTFS disk is most likely the problem. It's not going to get any faster because that disk just slow. Put more of your NTFS disks in and do multiple copies. I bet you never max out your pool.

Sent from my Nexus 5X using Tapatalk