Just to make sure I'm reading and interpreting this correct---a sysctl dev.cpu.0.freq will display full frequency, but the CPU will in actuality scale down in low-use/idle situations? Any way to verify this? Just looking to understand a bit more. I'd like to be somewhat power conscious, but am also planning to go always-on (incl HDD) after reading through the nuances in a number of the posts here (including many of yours).
There's 5 things that affect CPU frequency/efficiency/power usage.
1. CPU design. If you are using an AMD, you are totally on your own with everything. I know AMD and Intel do things differently, and going AMD means you probably were cutting corners with cost, which also means you probably cut CPU efficiency features you didn't know you were cutting. If you aren't happy, buy our recommended hardware next time. If you are Intel, your CPU architecture will determine how CPU power usage changes over time.
2. Your BIOS settings. This is important to get right, and for your motherboard to support the features. If they don't support those feature, well, you are already in a bad way with getting efficiency and power usage to acceptable levels. If you bought a quality server board, you can assume this is likely correct. If you aren't choosing stuff like Supermicro quality components, well, you are on your own. And you have my sympathy because you are about to see how you saved a dollar on hardware so you could spend $10 on electricity. Good trade I'd say, eh?
3. Your system settings in FreeNAS. There's tunables and sysctls that can have a drastic impact on CPU loading and what the CPU does when its idle. You have limited control of this via tunables and sysctls. Mostly you just set the settings and the hardware will behave as you have set it. If you don't know what you are doing, your performance and reliability may suffer.
4. PowerD(yep.. here it is). Powerd used to be very important years ago when power savings were mostly in software. But now, with today's technology CPUs are clocking themselves down, shutting off cores, etc. and they are doing it often behind the back of your OS. All the time, the CPU is lying to your OS about what is actually running and what its speed and voltage is. You're pretty much on your own to figure out that some parameters will be true and correct and others are lies.
5. Your actual CPU utilization will affect how far your CPU goes to sleep. If your CPU is being pinged every second or two with load don't expect it to go idle. That's how it is. And it doesn't take more than a fraction of a core to wake it up.
These 5 characteristics, if you stop and think about them, are all related to each other in various ways. Either you accept these relationships for what they are, or you dismiss them. They will behave a certain way based on those relationships, and you are mostly on your own if you didn't go with true quality hardware.
Sorry, but that's about all I can tell you because each system will behave in a unique way based on its circumstances.
Good luck!