Truenas Scale Energy efficient setup & build

[wlknsn]

So regarding power consumption settings and strategies: are there any specific settings to recommend in Truenas or Bios, ... Would it even be possible to somehow cache/stage data on ssd/nvme and then move it to the disks once in a while (scheduled) in one go so that the drives effectively or not spinning in between. (Yes I said it: spinning down).

If the latter would be possible, I could invest in a pci 4x nvme with bifurcation to handle most data. So a bit like cold (hhds)/warm/hot (ssd) storage concept, just an idea.

 

[doze5]

wlknsn​

I was going for low power consumption, because here in Poland bills for electricity are skyrocketing. My system is taking 21W-22W in idle and the same with 2 TV using Jellyfin (without transcoding). Mobo: MSI B250M Mortar, CPU: Intel G4600, RAM: 16GB DDR4, SSD: Plextor 120GB, HDD: 12TB N300 Toshiba, PSU: BE QUIET! Pure Power 11 400W 80 Plus Gold

Im not using spinning down of HDD. On Ubuntu I had ~16W from wall.

 

[wlknsn]

That's excellent (and what i'm aiming for)

 

[ChrisRJ]

Parts of this discussion seem to be a bit like folks who actually want a sports car coming to a truck dealership and complaining about missing sports car features of e.g. a Ford F-150.

ZFS, and therefore TrueNAS, is made for mission-critical data storage in a commercial context. If that is not your use-case, ZFS/TrueNAS is consequently not perfect for you in every aspect.

Over the last few months we had a bunch of people come here, who were lured by dubious YouTube content into believing that TrueNAS is suitable for their requirements. A strong aspect of the advice given in favor of enterprise gear is to do proper expectation management and correct the false claims made (or at least impressions created) by those YouTubers. We do that because we don't want people to loose their data. And we have seen that happen to many of them, often because they used unsuitable gear.

ZFS and TrueNAS are quite different from other file systems. Even someone who has 20 years professional experience with let's say Windows Server, will need to re-learn some things. One of those details is that ZFS has no tool to repair a damage on the file system; with its scale that would take too long. This has dramatic implications on the hardware requirements. Or to rephrase: Would you argue with a mainframe operator how something should be done on OS/390, based on your Linux experience?

Most of the regulars here have several decades of experience in storage technology. They offer help for free that would otherwise cost at least 150 USD per hour, probably more. If someone wants to ignore recommendations, that is of course their good right. But as the saying goes: Good judgement comes from experience and experience comes from bad judgement.

To close on a more practical note: If you feel that the hardware requirements for TrueNAS are not your cup of tea, chances are neither is TrueNAS. And that is perfectly fine! Using Linux and ext4 or XFS may indeed be a better choice and luckily there are plenty of options out there these days.

 

[WI_Hedgehog]

I have no problem when a person is happy with consumer-grade hardware. If you want real power savings look into a PicoPC. You can put a USB RAID array on it via a USB expansion hub and really sip power while having a decent server. If you roll your own Gentoo OS the results can be unbelievable.

If you want to run TrueNAS on consumer hardware or AMD CPUs where ECC is...uh, well, anyway, TrueNAS will run, until something goes really sideways and all your data is lost 9 months from now. You should have went with that PicoPC I suggested and still been in good shape, but by the time you figure this out it'll be too late--and you earned it for not listening to the "beard-stroking" members who've already learned why doing what you did will get you where you're going and tried to show you a better path instead. Learn the hard way, we did.

 

[wlknsn]

Parts of this discussion seem to be a bit like folks who actually want a sports car coming to a truck dealership and complaining about missing sports car features of e.g. a Ford F-150.

ZFS, and therefore TrueNAS, is made for mission-critical data storage in a commercial context. If that is not your use-case, ZFS/TrueNAS is consequently not perfect for you in every aspect.

Over the last few months we had a bunch of people come here, who were lured by dubious YouTube content into believing that TrueNAS is suitable for their requirements. A strong aspect of the advice given in favor of enterprise gear is to do proper expectation management and correct the false claims made (or at least impressions created) by those YouTubers. We do that because we don't want people to loose their data. And we have seen that happen to many of them, often because they used unsuitable gear.

ZFS and TrueNAS are quite different from other file systems. Even someone who has 20 years professional experience with let's say Windows Server, will need to re-learn some things. One of those details is that ZFS has no tool to repair a damage on the file system; with its scale that would take too long. This has dramatic implications on the hardware requirements. Or to rephrase: Would you argue with a mainframe operator how something should be done on OS/390, based on your Linux experience?

Most of the regulars here have several decades of experience in storage technology. They offer help for free that would otherwise cost at least 150 USD per hour, probably more. If someone wants to ignore recommendations, that is of course their good right. But as the saying goes: Good judgement comes from experience and experience comes from bad judgement.

To close on a more practical note: If you feel that the hardware requirements for TrueNAS are not your cup of tea, chances are neither is TrueNAS. And that is perfectly fine! Using Linux and ext4 or XFS may indeed be a better choice and luckily there are plenty of options out there these days.

Very clear synthesis. You seem very concerned about the whole LTT thing. For me, it only gave me the idea of the case which I didn't know existed. There is however another thing on youtube which actually got more of my interest which is this: Building a Power Efficient Home Server! The google sheet they are referring to is this: Energy efficient setups. Both combined is what brought me here. In the end, I'm trying to create a better alternative for the Synology RS822+. That's my goal, although the Synology will probably be more energy efficient so I'm here to find out if both hardware & potential strategies can be used to be as lean as possible.using Truenas.

 

[doze5]

There is a huge difference between businesses and home using for media like Jellyfin. Especially when your main goal is low power consumption, you gotta look for a different things. File security in this case? Well ... it is very simple - a copy on outside HDD. That is why I am looking for 2,5Gb/s ethernet card for TrueNAS Scale in different topic :)

Few of those Qnaps/Synologys are on j5040, j5005, j4125. There is ASRock J5040-ITX but I don't think you will get better power consumption on this mobo and CPU is weak, no comparison to Kaby Lake. Kaby Lake adds hardware acceleration for 4k HEVC 10-bit transcoding and VP9 decoding.

IMHO Linus and LTT used wrong tools and config for clustering and they paid the price, but they are doing good entertain videos. Data scrubbing is important.

 

[Etorix]

It has already been said but may be worth repeating: The whole "low power" discussion is biased if one looks at the TDP rating of the CPU (what matters is power draw at idle, not under maximal load), and further compounded if one ignores drives. It only takes a few spinners to use more power than an idling CPU.

@doze5 Don't look at J5000 for a NAS. Look at Atom C3000. A Supermicro A2SDi-HLN4F or -H-TF uses under 30W—much less than the drives if you use all 12 SATA ports! But it does not come cheap.

 

[ChrisRJ]

There is a huge difference between businesses and home using for media like Jellyfin.

Agreed if that is the only use-case and if a proper backup of the media exists. Unfortunately that is often not the case. If you are aware of this, congratulations (seriously). But too many newcomers are not, and sooner or later Murphy makes a house call.

In addition, most people store their photos on the NAS. At the same time, most folks initially think "this is for private purposes and therefore not business critical". But it is, unless you are willing to loose your photos.

Especially when your main goal is low power consumption, you gotta look for a different things.

Yes. The interesting question, though, is to ask whether or not the things that are relevant for reliability can be treated differently. It is a bit like with financial investments. There are 3 critical aspect to any investment: safety, liquidity, and return. You can optimize for any 2 of them. But never for all 3. In a similar fashion, people need to think whether or not they are willing to sacrifice reliability for power saving. If the answer is no, that has implications for the price of components.

I don't want to tell people what they should do. But I think it is important to think about those aspects in a conscious manner.

File security in this case? Well ... it is very simple - a copy on outside HDD.

Again, my impression is that too many people are not aware of the risks.

Thanks for the interesting discussion!

 

[abadr]

Why did you decide to go for TrueNAS? This will help us understand your motivation and by that provide better support. If by any chance the reason is this video from LTT (https://www.youtube.com/watch?v=boKmZKTKXHc), it was not good advice. That channel is about entertainment, not teaching technology. And they have history with TrueNAS that is not to their credit.

@ChrisRJ, Not wanting to go on a tangent here but I'm curios what was the bad advice in this video? Thanks

 

[zenon1823]

@ChrisRJ, Not wanting to go on a tangent here but I'm curios what was the bad advice in this video? Thanks

Thats a fair question i dont know if we've really done a good job calling out the real weaknesses in this video. I'll take a stab at a few...

To me the one thing more then anything else that made me shudder was the recommendation of RaidZ1. AND the fact that the OS suggested RAIDz2 and they completely dismissed it without so much as suggesting the reasons behind such decision. IMO this is irresponsible and negligent behaviour on his part given the following and reach that he has. It is without question a fact that Raid5/RAIDz1 has not been recommended for critical data for the better part of a decade. Dell advised against Raid5 starting in 2011 iirc.

Dont get me wrong there is a time and place for RAIDz1. It would have taken 30seconds of his video to outline it in simple terms like. "If your storing movies and data that important but not life altering Raidz1 is a great choice, if your storing photos and family records, you really should consider RAIDz2 or do some more research before making this decision" (or drop by the official formus to better understand how to decide)
I underscore that since the majority of his audience are NOT savvy enough to understand the ramifications of his suggestion it was irresponsible to chase the WOW factor of a few more TB of space, over VERY well established Best Practices.

ALSO i just watched it again and at time index 12:30 he said:
"If you only have to hard drives you might want to setup as mirror..." - This is perfectly correct and the only good option :)
"if you have more then two hard drives your going to move upto RAIDz..." UMM This is false, you CAN goto RAIDz, but as discussed at incredible length here there are MANY scenarios where Mirrored Vdevs are the often the best choice for a given use case.

Less so but still valid:
We can also go down the path of the suggesting a gaming MB and archtectre that doesnt support ECC memory which is also recommended for best stability. Also IMO the recommendation of a chassis without a video or serial console option for a home user is just bad... yes absolutely you can install the os on another PC, but when shit breaks and they cant find their NAS or an update fails for whatever reason (lets keep in mind this is targetted towards enthsists who arent daily admins) they got a giant paper weight with their data.

Personally his build was what $10 cheaper the the Synology? If im spending that kinda $$ and want something i can just use then id buy the Synology where i can rest knowing i have a platform that will just work.
On the subject of VALUE - I can buy a 10yr old HP Z420 workstation for $150 with 32-64GB ECC ram and xeon Quad/hex/oct processor and supports 8 drives. I ran this for years as my freenas and I still have 30 of these systems used daily in the office as workstations. You really cant compare gaming/enthousist hardware to workstation/Server hardware... and at 25% of the price.
** And for those interested in power my z420 idles at 42W iirc with the drives spun down

Lastly and im sure there is more i could highlight, but the "tutorial" they showed at the end, while i get they were going for a quick summary, and not an indepth tutorial, but there are SO MANY considerations make and things to learn. I feel it would have been approprite for him to point people who might be interested in doing such a build to drop by the official forums to learn more about Truenas...

Hope this helps :)

 

[abadr]

** And for those interested in power my z420 idles at 42W iirc with the drives spun down

Amazing timing, I'm being offered a z420/1620 and a z440/2680v3 and was looking online for idle consumption info ... thanks for that :)

I see your points and I would give them a pass as I think this video was intended for beginners and to showcase the chassis rather than a NAS setup. But even in that case using a MB/CPU without VGA introduced a complexity for beginners that would turn them away from a DIY NAS project, and they didn't explain that another choice of MB/CPU would have made things much easier. I remember when I was a newbie little things like that turned me off.

Thanks for the exhaustive reply...

 

[zenon1823]

Amazing timing, I'm being offered a z420/1620 and a z440/2680v3 and was looking online for idle consumption info ... thanks for that :)

I see your points and I would give them a pass as I think this video was intended for beginners and to showcase the chassis rather than a NAS setup. But even in that case using a MB/CPU without VGA introduced a complexity for beginners that would turn them away from a DIY NAS project, and they didn't explain that another choice of MB/CPU would have made things much easier. I remember when I was a newbie little things like that turned me off.

Thanks for the exhaustive reply...

I've been hands on the Z440 once and it was a nice little chassis. It certainly looks nicer on the outside then the z420 although if memory serves the z440 doesnt have the drive bay flexibility as the z420. The z440 chipset and v3 proc should be a little more power efficient in general, although the 12core 2680 might offset those savings over the 1620. Unless you going to do moderate to heavy VM applications the 2680 is pretty overkill for primarily NAS duties. you can pickup of a more conservitive quad or hex proc on ebay pretty cheap.

Also fun fact the spec sheets say the Z420 supports upto 64GB RAM, but in fact you can put 8 16GB dimms in for 128GB total ram. I don't know the Z440 exactly but I think it'll do 128GB as well.

 

[abadr]

I've been hands on the Z440 once and it was a nice little chassis. It certainly looks nicer on the outside then the z420 although if memory serves the z440 doesnt have the drive bay flexibility as the z420. The z440 chipset and v3 proc should be a little more power efficient in general, although the 12core 2680 might offset those savings over the 1620. Unless you going to do moderate to heavy VM applications the 2680 is pretty overkill for primarily NAS duties. you can pickup of a more conservitive quad or hex proc on ebay pretty cheap.

You're absolutely right, however since I was offered the 12 core I'm thinking consolidating the NAS with another machine and run the heavy on CPU Blue Iris IP Camera software in a VM on the NAS as well. I can go further and run Solid Works in another VM, it's currently using it's own computer as I don't run windows on my PCs and I only use it a few hours a month. Of course that means I need to add another GPU card and successfully pass it through TrueNAS to the VM, something that I'm reluctant to dive into.
I think the consolidation might offer me better power consumption in the end. I'll have to experiment.

 

[pmsan]

I think the discussion about new versus old hardware is really interesting. This discussion is maybe now a little off the original topic, but if you think not only about your energy bill, but your total environmental footprint (not just energy), then used hardware is attractive not just for the price.

I agree with the contributer suggesting to look for used hardware. A quality motherboard and cpu very rarely breaks. If the used market is good in your country that is. I have used FreeNAS, truenas core and now scale for about 7 years. Always on used consumer hardware. And for my usage all systems have been rock solid. I have however always spent money on brand new high quality hard drives made for 24/7 usage. Buy disks and PSU new.

This is for hobby or home office usage though. In a professional build I would use new hardware, and I would probably look at the Atom platform. However, I am just a developer, and I leave it to the professionals here to give professional advice!

 

[WI_Hedgehog]

Buying used drives for a 24/7 can work if running RAID Z3 and solid drives with life left in them.

Used server power supplies are fine if they haven't been flogged, I idle at 125W and full-load is only about 220W, I just can't seem to load it more than that (technically we're talking about the server load, not me idling about). Power supplies are dual server 450W, running anything larger would be wasting electricity for no benefit given the maximum load is 50% (peak efficiency is approximately 80% of rated capacity, so it's still running fat at 450W).

(note, I did have the mainboard literally go up in flames, necessitating it and some wiring be replaced, not to mention the facility vented, but that had noting to do with a PSU or HDD or my drinking issue)

 

[pmsan]

I had to look up exactly what RAIDZ3 is.

Yes, I can see your point. If you have enough bays, and you get the disks cheap enough, then it will work just fine. Also, a good backup strategy should not be forgotten whatever the raid and disk setup.

It would be a good thing if as much hardware as possible would run for as long as possible.

(note, I had an inspection in my house by the fire department last week, and he suggested I install a fire alarm in my home office. Maybe he too has seen a motherboard on fire!)