Upgraded to SCALE-22.12.1 and now my ZFS arc settings are not working.

[Xero7650]

Anyone else run into an issue where they were manually setting there ARC ZFS min and max size via CLI, and it is no longer working post upgrade? I noticed it post reboot even though I set it up under System Settings --> Advanced --> Init / shutdown scripts to run the commands on boot. I manually reran the commands from the CLI and they executed without error, but the amount of memory being used is not changing.

Anyone else run into a similar issue?

 

[donnyb99]

I am facing this exact same issue. I have over 100GiB of RAM not being utilized now...

 

[Xero7650]

if you re-run the commands manually, it will eventually work. However saying that, Ive seen free ram creep back up to half un-utilized leaving me with 256GB of RAM free (no reboot).

I then re-run the commands from the shell, and it starts working again.

 

[donnyb99]

Oddly enough attempting to run this from a user or admin account with sudo did not work, nor did attempting to run from a root user chron job. However, when I manually went in to shell as root, it worked no problem. I was getting permission denied with all the other methods. Anyone know the reason for this behavior?

 

[Samuel Tai]

You're setting them in the wrong spot, if you're trying to use a cron job or a post-init script. The correct location to set sysctls that will persist is under System Settings->Advanced->Sysctls

 

[donnyb99]

Cron jobs and post-init scripts always worked for me prior to 22.12.1. Can you shed some light on what to input in the variable field for sysctls to set the max arc size?

 

[Xero7650]

so did things change, I had been using a post init script previously and it was working great. I am open to changing how I do things; but can you explain a bit more?

do I just use the variable name and set the value? Meaning the following:

Setting Name: zfs_arc_min Value: 274877906944

or do the value keypair names change when you move to sysctls?

Thanks for the info!

 

[Xero7650]

Well I guess this answers that, I'll have to look it up

 

[Xero7650]

I did some digging and i'm not seeing any tuneable for arc via sysctl - so definitely could use a pointer.

 

[Samuel Tai]

Try running arc_summary. It should provide a comprehensive list of all sysctl tunables.

 

[donnyb99]

Try running arc_summary. It should provide a comprehensive list of all sysctl tunables.

The tunables listed in arc_summary do not work as listed in sysctl on Scale. When trying to enter zfs_arc_max as a variable, Scale reports, "
Sysctl 'zfs_arc_max' does not exist in kernel."

Perhaps that only works on Core and not on Scale? As far as I can tell, there are no sysctl tunables for zfs on Scale. If there are, please give me an example that works so I can try it out.

 

[sretalla]

ls /sys/module/zfs/parameters/

 

[StevenD]

Are you doing a Init/Shutdown Command similar to below?

Code:

echo 274877906944 >> /sys/module/zfs/parameters/zfs_arc_min

For sysctl, I think it would be vfs.zfs.arc.min. Though that might be the value for Core and not Scale. I do not know if they are different. In Angelfish, the above was the work around because as far as I know sysctl did not work.

I attached an image as for reference. The image is a different variable / value but encapsulates all the other settings that are working for me on Bluefin. Hopefully, the tunables are all added properly in the future and we can use sysctl directly as we are supposed too, but for now this works.

 

[neofusion]

Core and SCALE probably work different here.
Unless something has changed ticket NAS-116472 still applies and you can't do this in the UI for tunables on SCALE.

 

[donnyb99]

Core and SCALE probably work different here.
Unless something has changed ticket NAS-116472 still applies and you can't do this in the UI for tunables on SCALE.

I think this is the correct answer to sysctl on Scale.

Are you doing a Init/Shutdown Command similar to below?

Code:

echo 274877906944 >> /sys/module/zfs/parameters/zfs_arc_min

For sysctl, I think it would be vfs.zfs.arc.min. Though that might be the value for Core and not Scale. I do not know if they are different. In Angelfish, the above was the work around because as far as I know sysctl did not work.

I attached an image as for reference. The image is a different variable / value but encapsulates all the other settings that are working for me on Bluefin. Hopefully, the tunables are all added properly in the future and we can use sysctl directly as we are supposed too, but for now this works.

This worked flawlessly before 22.12.1 however now it doesn't seem to work. It returns a permission denied error. Only by running as root through the shell was I able to set this correctly. Even a cron job or init script set to run as root doesn't seem to work.

 

[StevenD]

Interesting, I upgraded my server yesterday to 22.12.1 and the script is still working here, no permission issues.

 

[TShiny]

Same issue here. Newly installed 22.12.1, the script(sudo) was permission denied, and `Sysctls` shows `Sysctl 'zfs_arc_max' does not exist in kernel.`

 

[li_chang]

You can do some workaround like this:

Code:

sudo sh -c 'echo NEW_ARC_SIZE > /sys/module/zfs/parameters/zfs_arc_max'

Same issue here. Newly installed 22.12.1, the script(sudo) was permission denied, and `Sysctls` shows `Sysctl 'zfs_arc_max' does not exist in kernel.`

 

[TShiny]

You can do some workaround like this:

Code:

sudo sh -c 'echo NEW_ARC_SIZE > /sys/module/zfs/parameters/zfs_arc_max'

It works like a charm, thank you!