Adding a read and write cache to FreeNAS mini

[David Simpson]

I recently upgraded my FreeNAS mini with a Chelsio 10GB sfp networking card and it worked like a charm. With a 10gb network read/writes have started going quite a bit faster with my Mac Studios but I feel like my drives are what is killing me, WD Reds 4TB 5400rpm. I realized there is two places to install SSDs on the top of the rig.

If I added two 500GB Samsung 870 2.5" SSDs, could I add them as a read and write cache in front of the WD Reds? Would it help? Would it make no difference? In my brain TrueNAS would write data to those temporarily and offload that data back to the Red but my transfer speeds would be much faster.

Eventually I will upgrade these WD drives but I don't have any idea how to do that and don't really want to open that can of worms yet. Thanks!

 

[HoneyBadger]

Hi David,

Unfortunately it isn't quite as simple as that - the "Write Cache" in ZFS is a bit of a misnomer, as it doesn't collect the writes there and then later spool them off to the pool devices. It's more of a "Write Log" in that it provides safety and assurance that data is stored on a non-volatile device. The Samsung 870 also isn't an ideal "Write Log" device, whether EVO or Pro, as it lacks the power-loss-protection for in-flight data that makes a write log able to rapidly confirm that data is safe.

The "Read Cache" might be beneficial here, if your workflow involves repeatedly reading the same set of files multiple times or from multiple client systems. ZFS would (over time) understand that this data is more valuable based on the MFU/MRU (Most Frequently Used/Most Recently Used) algorithms for weighting, and place those blocks onto the "Read Cache" device. A Samsung 870 might be viable here, even an EVO - but don't use a QVO, as that NAND type isn't rated for a lot of writes, and cycling through multiple projects might be enough to burn them out in rather rapid order.

Could you post some more detailed information about the version of FreeNAS/TrueNAS you're running? Newer releases often default to a "safe sync write" behavior with Apple SMB clients that can slow things down a fair bit, but if you're on an older release this might not impact you.

Some details about how you and your coworkers (you did mention plural "Mac Studios" so pardon the assumption if you're a one-person shop) work with the files as well would also be beneficial.

Cheers!

 

[ChrisRJ]

I know that I will sound a bit pedantic. But being precise in terms of providing technical information is critical for being successful.

There is no Samsung 870 SSD. Which model exactly do you mean? Regardless, the idea of a write cache, which as such does not exist in ZFS, will not work with the PRO, the EVO or the QVO in descending order of potential damage. But again, neither is really suitable.

We do not know how much RAM you have, what board or CPU we are talking about, or what version of TrueNAS Core or Scale you are running.

What is the current performance and what do you expect. How did you measure things?

Please check the forum rules. Those contain a lot of good information how to get good answers. In addition, I would also suggest that you have a look at the recommended readings in my signature.

 

[David Simpson]

@ChrisRJ @HoneyBadger Does this give enough information? The drives are from 2015, and I can upgrade to 32gb of memory but I'm not sure if I should keep investing in this or if I should upgrade to something new. Transfers start fast and slow down, I'm guessing because these old drives have such a small cache it quickly runs out of steam

I primarily use it for SMB storage for Macs, Time Machine, and running a Plex Server. All of which it handles great, I just wish it had a little more in the tank for transfers now that I have dual 10Gb nics

 

[HoneyBadger]

Transfers start fast and slow down, I'm guessing because these old drives have such a small cache it quickly runs out of steam

Inbound transfers (writes) will start at nearly wire speed (10Gbps) but will rapidly throttle down to the actual speed of the backing vdevs (your four drives) - this won't be improved by a write log device unfortunately.

Reads might see improvement from an L2ARC device as discussed previously.

You're at 56% space used, so you shouldn't quite be seeing slowdown from that. One core question is what exact model of drives you have; you did mention WD Red, and 2015 should be prior to the SMR shenanigans, but that should be confirmed.

 

[ChrisRJ]

To saturate 10 Gbps with a small number of drives, you will need SSDs. For any more specific answer, we need numbers and you did the testing that lead to those. In other words: Fast means different things to different people (just like real-time, but that is a different topic).

 

[David Simpson]

Inbound transfers (writes) will start at nearly wire speed (10Gbps) but will rapidly throttle down to the actual speed of the backing vdevs (your four drives) - this won't be improved by a write log device unfortunately.

Reads might see improvement from an L2ARC device as discussed previously.

You're at 56% space used, so you shouldn't quite be seeing slowdown from that. One core question is what exact model of drives you have; you did mention WD Red, and 2015 should be prior to the SMR shenanigans, but that should be confirmed.

here is the exact model:

Western Digital 4TB WD Red Plus NAS Internal Hard Drive HDD - 5400 RPM, SATA 6 Gb/s, CMR, 64 MB Cache, 3.5" - WD40EFRX