Auto tuning

[Darren Myers]

So i read some hit and miss stuff about auto tuning, and how some users use the auto tune and then add/remove RAM and never remove the old tune. I'm keeping my server at 48Gb of RAM for the time being, so if i auto tune and let it do its thing, is there anything else to do after words?

And if i were to go from 48Gb of ram to 64Gb or so, i would delete the old tune and then re enable the auto tune and reboot correct?

 

[Robert Trevellyan]

I believe the consensus among the FreeNAS gurus is that if autotune improves performance, then your hardware is inadequate. There's nothing obviously inadequate about your hardware, so it might make sense simply to forget the whole idea of autotune.

 

[DrKK]

I believe the consensus among the FreeNAS gurus is that if autotune improves performance, then your hardware is inadequate. There's nothing obviously inadequate about your hardware, so it might make sense simply to forget the whole idea of autotune.

Actually, auto-tune actually does something useful if you have insane amounts of RAM, if I'm recalling a conversation Cyberjock and I had a while back. In general, though, I would suggest not clicking auto-tune.

 

[rogerh]

In my case autotune produced a satisfyingly large number of items. No idea if they improved anything though! It is quite easy to reverse, and it seems sensible to remove all the settings if the hardware changes, especially RAM. According to the manual, autotune does not alter values it (or you) have already created, unless you delete them and reboot.

 

[Robert Trevellyan]

Actually, auto-tune actually does something useful if you have insane amounts of RAM, if I'm recalling a conversation Cyberjock and I had a while back.

Approximately how much RAM is an insane amount in this context?

 

[jgreco]

Approximately how much RAM is an insane amount in this context?

128? 256? ...

No harm in trying autotune, seeing what it comes up with. Take note of the settings, then undo it, see what the defaults were, and then read up.

 

[cyberjock]

Autotune artificially caps the ARC so it doesn't end up in an over-sized condition which compromises the performance (but more importantly, the reliability) of your system.

If your system isn't "right sized" ZFS may end up in this condition where, over a period of weeks (normally we're talking 1.5 months+ from what I've seen) the system will slowly use more and more RAM for the ARC. Eventually you end up in a condition where some process will need RAM. ZFS needs time to evacuate data from ARC and free RAM. If ZFS can't evacuate data fast enough and the process crashes, you may see the system crash or at the very least be very unreliable, erratic, or non functional for key services.

If you have "right sized" your RAM for your workload, ZFS will not have this desire to slowly get bigger and bigger, hence there is no problem with autotune off. Autotune really is a limiter for those that are concerned about reliability over performance.

ZFS used to have a positive feedback system at large ARC sizes (think bigger than 64GB of RAM) where the more RAM it had, the more RAM it "needed". This was obviously bad because the solution for out of memory conditions was to add more RAM. But you'd only bias ZFS into thinking it needed even more RAM, so unless you had infinite amounts of RAM, you had a major problem.

But.. let's look at people with absurdly under-resourced systems...

If you have 8GB of RAM and 30TB of disk space, most people would call you stupid around here. But ZFS will constantly be expanding because it has a genuine need for more RAM. So it will start getting larger and larger, then suddenly be forced to downsize for some process. This constant growing and shrinking hurts ZFS performance too. This can happen very frequently depending on what your server is doing. Remember, CPU resources, time, and clearing of cache all must be done to free RAM from the ARC.

So if you were silly enough to horribly undersize your system, you *might*, in theory, see a benefit from enabling autotune if you've horribly under-resourced your system (strictly from a performance standpoint). But the reality of it is that if you've under-resourced your system to that extent, you are almost certainly going to be playing russian roulette with the reliability of your server. So if you are thinking the solution is to enable autotune because you know you under-resourced your system, I'd call you silly for not going with some other operating system that is more appropriate for your hardware. Remember, with no zfs recovery tools and no fsck-type of tool for ZFS, any kind of corruption you impose on ZFS because you underresourced the system will end up being part of your zpool forever.

So be careful what you think you are doing. Arguing that ZFS is so awesome while under-resourcing it is probably a more stupid idea than going with Windows/NTFS or linux/ext4 and using hardware RAID, if you are not committed to letting ZFS do what it's supposed to do on its own.

 

[Darren Myers]

Autotune artificially caps the ARC so it doesn't end up in an over-sized condition which compromises the performance (but more importantly, the reliability) of your system.

If your system isn't "right sized" ZFS may end up in this condition where, over a period of weeks (normally we're talking 1.5 months+ from what I've seen) the system will slowly use more and more RAM for the ARC. Eventually you end up in a condition where some process will need RAM. ZFS needs time to evacuate data from ARC and free RAM. If ZFS can't evacuate data fast enough and the process crashes, you may see the system crash or at the very least be very unreliable, erratic, or non functional for key services.

If you have "right sized" your RAM for your workload, ZFS will not have this desire to slowly get bigger and bigger, hence there is no problem with autotune off. Autotune really is a limiter for those that are concerned about reliability over performance.

ZFS used to have a positive feedback system at large ARC sizes (think bigger than 64GB of RAM) where the more RAM it had, the more RAM it "needed". This was obviously bad because the solution for out of memory conditions was to add more RAM. But you'd only bias ZFS into thinking it needed even more RAM, so unless you had infinite amounts of RAM, you had a major problem.

But.. let's look at people with absurdly under-resourced systems...

If you have 8GB of RAM and 30TB of disk space, most people would call you stupid around here. But ZFS will constantly be expanding because it has a genuine need for more RAM. So it will start getting larger and larger, then suddenly be forced to downsize for some process. This constant growing and shrinking hurts ZFS performance too. This can happen very frequently depending on what your server is doing. Remember, CPU resources, time, and clearing of cache all must be done to free RAM from the ARC.

So if you were silly enough to horribly undersize your system, you *might*, in theory, see a benefit from enabling autotune if you've horribly under-resourced your system (strictly from a performance standpoint). But the reality of it is that if you've under-resourced your system to that extent, you are almost certainly going to be playing russian roulette with the reliability of your server. So if you are thinking the solution is to enable autotune because you know you under-resourced your system, I'd call you silly for not going with some other operating system that is more appropriate for your hardware. Remember, with no zfs recovery tools and no fsck-type of tool for ZFS, any kind of corruption you impose on ZFS because you underresourced the system will end up being part of your zpool forever.

So be careful what you think you are doing. Arguing that ZFS is so awesome while under-resourcing it is probably a more stupid idea than going with Windows/NTFS or linux/ext4 and using hardware RAID, if you are not committed to letting ZFS do what it's supposed to do on its own.

Oh no, my intent wasnt to change any specs to make my server fall into the "not recommended" zone, i read enough posts and enough of your posts to know what is acceptable and recommenced vs what isnt and causes people to loose there data. My FreeNAS is really just used for CFIS sharing, so i wasnt sure if i could benefit at all from autotune with my system.