TrueNAS Apps `recordsize`

[Sawtaytoes]

For TrueNAS Apps, should the recordsize be 128K (the default) or 1M?

I've found 1M to be faster in just about every scenario so far, so I'm wondering if there's an advantage to the default mirror size of 128K?

 

[sretalla]

I'm wondering if there's an advantage to the default mirror size of 128K?

Do you mean the default recordsize?

There's a general benefit to it being large enough, but not too large if you don't think about tuning it for your workload, but it's always going to be better if you do tune it to the data you'll be putting there.

I've found 1M to be faster in just about every scenario so far,

Then it seems for most of your workloads, a larger recordsize is beneficial.

Databases, for example often want smaller recordsize and even different recordsize for the logs compared to the DB.

for example: https://openzfs.github.io/openzfs-docs/Performance and Tuning/Workload Tuning.html#mysql

 

[winnielinnie]

My personal rule of thumb: "Is this dataset going to see many files with 'in-place' modifications?" If not, I crank that sucker up to 1MiB recordsize. It's kept me very happy.

Remember, for files simply being written, read, and copied (I'm intentionally excluding in-place modifications for the sake of discussion), a 1MiB recordsize will still yield files smaller than 1MiB to its actual size rounded up to the nearest power of 2. (Single "small block" file.) As for files larger than 1MiB? They'll be rounded up to the next 1MiB.

But wait? Isn't that... wasteful? Won't many small files greatly add up? Won't files that don't land perfectly on 1MiB increments balloon my dataset with all this extraneous "padding", (i.e, "record slack")? Fear not. As long as any compression is enable on the dataset, then this "slack" gets squished into nothingness, and hence, you don't end up with "record slack". A single 1.25 MiB file on a dataset with recordsize=1MiB will be comprised of two blocks: 1MiB + 256KiB. (The second block doesn't take up a full 1MiB on your drive, because the extra 784KiB of "slack" gets compressed away.) This also applies to the "small blocks" as well.

Moral of the story: Unless you have an otherwise specific need, always leave compression enabled on the dataset. (Any compression will do.)

 

[jenksdrummer]

Currently testing my data to a 16M record size to see how it breaks down. I had good results with my previous configuration of 1M record size and 512K small blocks to a metadata vdev under core, but via UI, 1M is where it topped out (yes, could use CLI, but didn't wanna in case it borks the UI)

Anyhow, mainly aiming for balance between the pool vdev vs the meta vdev, in terms of consumption. Largest record size for me since mostly what I'm dealing with is data at rest and backups, and whatever is outside of that is mainly reads vs updates/new data.