Using TrueNAS for a DAS-like build, HELP?

[maci]

Dear TN Community, I am currently looking to turn an old PC into a storage device for video and photo editing. I have searched already existing threads but couldn't find one that applies well to my case, so here go the details in case anyone is able to help me.

My main question is how to connect the storage PC running TrueNAS (or similar free alternative if there is a more appropiate OS) to my main PC without the use of a switch board.

I do not need to access the storage on my network, I just need to have it accessible from my main PC. Is it possible to have that sort of direct connection using TrueNAS? (Be that Ethernet cable, SFP+, etc). Of course I still need the storage PC to function as a conventional NAS, not JBOD (I need expandability, RAID, cache drives, etc). I have been told that a crossover RJ45 cable would make it possible to connect directly to my PC without much hassle, but I'mnot sure how transfer speeds would be affected by this, ideally I'd need over 1GB/s transfers and as far as I've seen crossover cables only come in cat5, should I make my own crossover cable from a cat7? What are the alternatives?

I'm happy to give more details to anyone who might be able to help me or point me in the right direction. Thank you in advance.

 

[sretalla]

I have been told that a crossover RJ45 cable would make it possible to connect directly to my PC without much hassle, but I'mnot sure how transfer speeds would be affected by this, ideally I'd need over 1GB/s transfers and as far as I've seen crossover cables only come in cat5, should I make my own crossover cable from a cat7? What are the alternatives?

Crossover cables are a relic from the days of Fast Ethernet (100Mbits)... pretty much all NICs which run at Gbit or faster will automatically detect and negotiate crossover with a straight cable to another NIC.

I don't think your main concern here is the category of cable... if you have a NIC that can support more than 1Gbit/s, you'll need to consider what you have, but before that, forget the idea of getting more than 1Gbit/s out of any cable using 1Gbit NICs.

You can get reasonably priced NICs on the second hand market on Ebay and the like (although watch for "clones", which are typically dodgy), but a couple of 10Gbit NICS and a Direct Attach Cable will get you what you want.

Something like a couple of these:

Intel X520-DA2 Ethernet Converged Network Adapter for sale online | eBay

Find many great new & used options and get the best deals for Intel X520-DA2 Ethernet Converged Network Adapter at the best online prices at eBay! Free shipping for many products!

www.ebay.com

and one of these:

2 Packs For J9283B, J9283D HP HPE Aruba, 10G SFP+ to SFP+ DAC Cable 3 m | eBay

Find many great new & used options and get the best deals for 2 Packs For J9283B, J9283D HP HPE Aruba, 10G SFP+ to SFP+ DAC Cable 3 m at the best online prices at eBay! Free shipping for many products!

www.ebay.com

See this for some ideas:

10 Gig Networking Primer

This document is now available in the Resources section! Use the tabs above to navigate to the document itself. The current version also follows inside the spoiler tag: Also, if you'd like to repost this elsewhere, I'd appreciate it if you would have the courtesy to ask. And to credit the...

www.truenas.com

 

[maci]

Hi, thank you for your answer. So if I have two NICs (10Gbps) on both ends and the sfp+ cable everything should work? I will be able to see the TrueNAS drive on my PC? What do I need to do in order to set that up during installation? IPs, etc? This is the approach I originally landed on but considering I'm not that proficient in IT matters I rather ask the experts haha. Would this also work with a 10G Ethernet NIC instead of SFP? Would there be any benefit to using both ports to increase speed or is it not worth it?

 

[sretalla]

So if I have two NICs (10Gbps) on both ends and the sfp+ cable everything should work?

Yes.

I will be able to see the TrueNAS drive on my PC?

You'll be able to attach to shares (NFS/SMB) and with SSH (FileZilla) to see the whole thing.

What do I need to do in order to set that up during installation? IPs, etc?

The link between the NAS and the PC will need its own subnet and each device will need an address on that subnet...

If your home network is 192.168.1.x/24, then the "inter-connect" could be 192.168.2.x/24, with the TrueNAS taking 192.168.2.1 and the PC taking 192.168.2.2... that kind of thing.

Would this also work with a 10G Ethernet NIC instead of SFP?

Yes, but requiring more power and heat, so not preferable per se.

Would there be any benefit to using both ports to increase speed or is it not worth it?

Depends on what software you're using and how good the NAS and PC are...

2 NICs can deliver more than 1 NIC, but only to more than 1 client for a session, so you need more than 1 app to be asking for data on a discrete path for that to work... like one app using the SMB share and another app using an NFS share.

I doubt it will be worth the trouble.

 

[maci]

That makes sense, I'll look up how to set all those subnets up but it doesn't sound too complicated. I'm not too bothered about heat and power but the SFP option seems to do everything I need so might as well go with that. Some of the items on the links you provided are no longer available, what specific things am I looking for in the NICs and cables? So far I understand that I'm looking for 10Gbit NICs, if used then checking Yottamark and a SFP cable in the desired lenght (I think I need around 3-5m). Is there any other charcteristic that I shouldn't miss? Thanks.

In regards do the dual cable, I meant using the 2 ports in each NIC to connect together, so 2 cables connecting the same 2 NICs, is that at all possible? It'd just be for the purpose of incresing speeds as it'll only ever be one PC connected to the storage.
I'm planning on starting it off with x6 8TB Seagate drives, and either SATA ssds or m.2 NVMEs for a cache array, as ideally I'd like 1GB/s transfers (I usually work with video ranging from H.264 to 444 ProRes, RAW and EXRs coming from vfx.

Thanks

 

[sretalla]

SFP cable in the desired lenght (I think I need around 3-5m)

Not sure how well supported that will be in a DAC cable, so you may need to use Fiber instead for more than a couple of metres. EDIT: seems 10m is possible: https://community.fs.com/article/sfp-dac-twinax-cable-deploy-guide.html

In regards do the dual cable, I meant using the 2 ports in each NIC to connect together, so 2 cables connecting the same 2 NICs, is that at all possible?

Yes, that's what I was answering... each port on the card(s) is effectively a separate NIC, so what I said applies exactly to what you're asking... you don't get double the bandwidth for a single client/app.

 

[ChrisRJ]

Please update the thread title. We are not talking about DAS here, but a NAS that is connected without a switch.

 

[ChrisRJ]

In regards do the dual cable, I meant using the 2 ports in each NIC to connect together, so 2 cables connecting the same 2 NICs, is that at all possible?

Yes but not advisable from a complexity point of view.

It'd just be for the purpose of incresing speeds as it'll only ever be one PC connected to the storage.

If you don't get the desired speed, the first thing is to determine the bottleneck. You did not specify your hardware (please do so and also read the forum rules about the level of detail we need to be able to help you), but my guess is that the network will initially not be your problem.

I'm planning on starting it off with x6 8TB Seagate drives, and either SATA ssds or m.2 NVMEs for a cache array, as ideally I'd like 1GB/s transfers (I usually work with video ranging from H.264 to 444 ProRes, RAW and EXRs coming from vfx.

HDDs and direct video editing usually do not match/work well. For this workload latency and IOPS are the bigger challenge than simple throughput.

ZFS does not support a cache in the typical sense.

Please go over the resource linked in my signature (under "Recommended readings") to get a better understanding for the specifics of ZFS and TrueNAS.

 

[sretalla]

Please update the thread title. We are not talking about DAS here, but a NAS that is connected without a switch.

Title changed.

 

[sretalla]

ZFS does not support a cache in the typical sense.

ZFS doesn't support using fast drives as a write cache. RAM is the only write cache in ZFS.

L2ARC is a system to extend read cache (ARC, which uses RAM) to fast drives and is supported in ZFS.

 

[ChrisRJ]

@sretalla , thanks for adding those details to my admittedly rather generic statement

 

[maci]

Please update the thread title. We are not talking about DAS here, but a NAS that is connected without a switch.

Hi, What would be the difference between the two systems? I understand a traditional NAS would be hooked up to a switch and work over the network. I appreciate the input.

 

[sretalla]

Direct Attached Storage is where the disks are connected directly to (and are for the exclusive use of) the system consuming the storage.

Network Attached Storage is where the disks are made available via Network Services to consumers of the storage on other systems.

If you add storage to your computer directly, it's DAS (including external enclosures).

If you add storage to a NAS and then connect to it.... you know the rest.

 

[maci]

Yes but not advisable from a complexity point of view.


If you don't get the desired speed, the first thing is to determine the bottleneck. You did not specify your hardware (please do so and also read the forum rules about the level of detail we need to be able to help you), but my guess is that the network will initially not be your problem.


HDDs and direct video editing usually do not match/work well. For this workload latency and IOPS are the bigger challenge than simple throughput.

ZFS does not support a cache in the typical sense.

Please go over the resource linked in my signature (under "Recommended readings") to get a better understanding for the specifics of ZFS and TrueNAS.

Of course! My apologies. I'm setting up the storage with a used computer I had laying around, it has a i5 3570K, 16gb 1333MHZ ram and a Gigabyte GA-Z77-DS3H motherboard. I plan on using a Kingston 240GB SATA SSD as OS Drive and 6x Seagate Barracuda 8TB drives for storage. My initial thought was to use another M.2 drive for a fast cache, is that not supported in TrueNas? There is an ocean of info and differences in systems out there so perhaps I took that idea from somthing incompatible with my case.

 

[maci]

In regards to both of your answers here regarding cache, is there any way to increase speeds for a video-editing workflow whilst keeping HDDs as main bulk drives? I can't afford a storage made out of SSDs or NVMEs, hence why I thought (and saw somewhere) the possibility to use SSDs as cache drives in that system.

 

[maci]

One more small question which you might be able to help me with, would something like this (link below) work for me? Ignore the cable lenght for now, I still am not sure where exactly I'll place the storage in relation to my PC, but I've gone searching on amazon as shipping on ebay seems to take a few weeks and I'd ideally like to get this operational sooner.

https://www.amazon.co.uk/Dpofirs-Network-Adapter-MNPA19-Conversion/dp/B0BLGZ2C12/ref=sr_1_10?crid=2JQDMGGOGOTMH&keywords=25G+SFP+NIC+PCIe&qid=1704288948&sprefix=25g+sfp+nic+pcie%2Caps%2C50&sr=8-10

As I mentioned earlier, I'm not that well acquainted with this field and don't have enough time to dive deeply into it as much as I'd love to, is there anything I should be looking out for on that product?

Thanks to all

 

[sretalla]

Assuming the concern of "slowness" is related to IOPS, you would need to use Mirrors rather than RAIDZ for your pool... probably adding a good amount of RAM (to give a total of at least 64GB).

You could just set the M.2 drive as a separate pool for working on your videos, then transfer to the spinning disk pool (keeping it RAIDZ in that case).

 

[maci]

Assuming the concern of "slowness" is related to IOPS, you would need to use Mirrors rather than RAIDZ for your pool... probably adding a good amount of RAM (to give a total of at least 64GB).

You could just set the M.2 drive as a separate pool for working on your videos, then transfer to the spinning disk pool (keeping it RAIDZ in that case).

Could you expand on the IOPS? How can I know what will cause bottlenecks? Drives, system? or just the nature of the work I'm doing? I can't add RAM easily, as all 4 slots are full, if that proves to be the best option then I'll swap them for 64GB perhaps even at 1600mhz or higher if supported, but I'd like to keep costs down. Similarly, using mirrors I understand that would make me lose half of my storage? I'd prefer raidz as I need a fair amount of storage and can't afford a lot of drives at the moment. The idea to create a separate pool for current projects is interesting, I'll explore that option further. Thank you.

 

[sretalla]

Could you expand on the IOPS? How can I know what will cause bottlenecks? Drives, system? or just the nature of the work I'm doing?

IOPS will be most important if the transfers from the NAS to your editing PC will be large numbers of small requests (some suggest this is what happens when "scrubbing" video, I don't have personal experience with video editing directly on a disk located over the network).

If you use RAIDZ for your pool, you can expect the IOPS capacity of a single disk for the pool, something like 100-300 (depending on the drives used).

If you use 3 mirrors, you can expect something like 300-900 (additional disk worth of IOPS per mirror/VDEV).

If your workload is indeed high in IOPS requirement, there is no getting around the laws of physics on it, so you can either:

Use my suggestion of a separate pool for your editing work located on a single (or as many as you can add) NVME drive (depending on the drive, can be 10,000-100,000 IOPS).

or

Adjust your workflow in some way to eliminate the need for lots of IOPS.

using mirrors I understand that would make me lose half of my storage?

Correct.

I'd prefer raidz as I need a fair amount of storage and can't afford a lot of drives at the moment

The laws of physics don't change based on your ability to afford things... if you put together a sub-par pool, expect sub-par performance.

 

[ChrisRJ]

I would seriously consider to not use the NAS for editing the videos directly on it. The same goes for 10 Gbps networking, at least if the expectation is to constantly saturate the bandwidth. You will likely have peaks that go above 1 Gbps, perhaps up to 10 Gbps. But I would be surprised if that is for the majority of the time.

The entire system is low-end by today's standards and for a good experience will need various upgrades. In my view it would be much easier and cheaper to do the video editing locally and then store/archive the projects on the NAS. That is what I do here for videos as well as my VM hosts. I recently added and still test a pair of NVMe SSDs in the NAS only for VMS and a direct link via 10 Gbps. But the main workload is still a local SATA SSD, which is ok for me. And, to put things into perspective, I run my own company so this is business critical for me.