How to enable persistent L2ARC in TrueNAS Scale?

[ajgnet]

Is this the correct way to enable it?

 

[Samuel Tai]

Try vfs.zfs.l2arc.rebuild_enabled: 0.

 

[ornias]

Try vfs.zfs.l2arc.rebuild_enabled: 0.

Wait... since when is 0 on a boolean true?

 

[sretalla]

No, it needs to be 0 rather than 1 (meaning no rebuild at boot)

 

[Samuel Tai]

Wait... since when is 0 on a boolean true?

Sorry, cut and paste error. To enable, set the value at 1. However, @sretalla is correct; this should stay at 0 for now until the performance issues with L2ARC rebuild at boot get resolved.

 

[ornias]

No, it needs to be 0 rather than 1 (meaning no rebuild at boot)

This user was asking to enable it, so shouldn't it be 1 if enabled?

 

[sretalla]

This user was asking to enable it, so shouldn't it be 1 if enabled?

Maybe I am not understanding it right, but rebuilding is the opposite of persisting as I see it. Thus persistent cache is on when rebuild is off.

 

[Samuel Tai]

@sretalla, I believe you've got it backwards. When rebuild is on, then L2ARC is rebuilt from the contents of the cache drive on boot, and its contents from the previous boot then persist into the current boot. If rebuild is off, then the contents of the cache drive are discarded on boot, and L2ARC is repopulated anew on the current boot.

 

[sretalla]

@sretalla, I believe you've got it backwards. When rebuild is on, then L2ARC is rebuilt from the contents of the cache drive on boot, and its contents from the previous boot then persist into the current boot. If rebuild is off, then the contents of the cache drive are discarded on boot, and L2ARC is repopulated anew on the current boot.

OK, so when you actually do the reading, it helps.

References

The TrueNAS References section includes additional information on various topics helpful for a TrueNAS user. Copyrights and Trademarks: Provides a list of copyrights and trademarks, and related logos registered as trademarks of iXsystems, dba TrueNAS. ACL Primer: Provides general information on...

www.truenas.com

It goes against the concept of "persistent" though, so I still want to think that there will one day be the feature I thought it was and just leave the contents of L2ARC on the disk where they were rather than reading it all back (which just feels like it's wantonly wasting writes/TBW).

 

[ornias]

OK, so when you actually do the reading, it helps.

References

The TrueNAS References section includes additional information on various topics helpful for a TrueNAS user. Copyrights and Trademarks: Provides a list of copyrights and trademarks, and related logos registered as trademarks of iXsystems, dba TrueNAS. ACL Primer: Provides general information on...

www.truenas.com

It goes against the concept of "persistent" though, so I still want to think that there will one day be the feature I thought it was and just leave the contents of L2ARC on the disk where they were rather than reading it all back (which just feels like it's wantonly wasting writes/TBW).

Afaik that article is wrong (having had quite the talks during the initial PR), what is "Rebuild" is the "map" of L2ARC inside ARC afaik. Not all data read from disk. (though it's possible it also verifies hashes)

An authoritive source would be nice though, I don't think this is ever described like this on the PR nor discussions about performance on OpenZFS.

 

[Samuel Tai]

An authoritive source would be nice though, I don't think this is ever described like this on the PR nor discussions about performance on OpenZFS.

https://github.com/openzfs/zfs/blob/master/module/zfs/arc.c

Rebuild description starts at line 7922.

 

[ornias]

@Samuel Tai So:
It's about rebuilding ARC buffers, not the L2ARC.

"This allows us to rebuild the arc_buf_hdr_t structures of the * main ARC buffers"


IX manual isn't completely false though:
It does cause degraded performance at start, but not because because it would "than reading it all back" from disk, but because it needs to rebuild the main ARC buffers.

 

[ajgnet]

Is this still the best way to enable l2arc rebuild at boot on TrueNAS Scale 21.08?

Code:

vfs.zfs.l2arc.rebuild_enabled: 1