Write throughput: 
write_throughput: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
...
fio-3.25
Starting 8 processes
write_throughput: Laying out IO file (1 file / 10240MiB)
write_throughput: Laying out IO file (1 file / 10240MiB)
write_throughput: Laying out IO file (1 file / 10240MiB)
write_throughput: Laying out IO file (1 file / 10240MiB)
write_throughput: Laying out IO file (1 file / 10240MiB)
write_throughput: Laying out IO file (1 file / 10240MiB)
write_throughput: Laying out IO file (1 file / 10240MiB)
write_throughput: Laying out IO file (1 file / 10240MiB)

write_throughput: (groupid=0, jobs=8): err= 0: pid=481961: Sun May 29 12:13:28 2022
  write: IOPS=754, BW=764MiB/s (801MB/s)(44.7GiB/60012msec); 0 zone resets
    slat (msec): min=2, max=557, avg=10.58, stdev= 4.29
    clat (usec): min=9, max=1680.1k, avg=662263.36, stdev=95766.96
     lat (msec): min=5, max=1691, avg=672.80, stdev=96.76
    clat percentiles (msec):
     |  1.00th=[  409],  5.00th=[  506], 10.00th=[  518], 20.00th=[  667],
     | 30.00th=[  676], 40.00th=[  684], 50.00th=[  684], 60.00th=[  684],
     | 70.00th=[  693], 80.00th=[  701], 90.00th=[  709], 95.00th=[  735],
     | 99.00th=[  818], 99.50th=[  852], 99.90th=[ 1687], 99.95th=[ 1687],
     | 99.99th=[ 1687]
   bw (  KiB/s): min=444886, max=1089806, per=99.00%, avg=774010.50, stdev=12406.56, samples=952
   iops        : min=  434, max= 1064, avg=755.50, stdev=12.13, samples=952
  lat (usec)   : 10=0.01%, 20=0.01%, 50=0.01%
  lat (msec)   : 10=0.01%, 20=0.02%, 50=0.06%, 100=0.09%, 250=0.24%
  lat (msec)   : 500=2.71%, 750=95.20%, 1000=2.49%, 2000=0.28%
  cpu          : usr=0.67%, sys=6.69%, ctx=384090, majf=0, minf=10054
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=0,45309,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
  WRITE: bw=764MiB/s (801MB/s), 764MiB/s-764MiB/s (801MB/s-801MB/s), io=44.7GiB (48.0GB), run=60012-60012msec
Write IOPS: 
write_iops: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.25
Starting 1 process
write_iops: Laying out IO file (1 file / 10240MiB)

write_iops: (groupid=0, jobs=1): err= 0: pid=585610: Sun May 29 12:14:31 2022
  write: IOPS=13.1k, BW=51.3MiB/s (53.8MB/s)(3081MiB/60001msec); 0 zone resets
    slat (usec): min=8, max=85998, avg=70.97, stdev=269.40
    clat (usec): min=4, max=100059, avg=4794.77, stdev=3182.76
     lat (usec): min=75, max=100290, avg=4866.37, stdev=3219.12
    clat percentiles (usec):
     |  1.00th=[ 3097],  5.00th=[ 3458], 10.00th=[ 3621], 20.00th=[ 3785],
     | 30.00th=[ 3884], 40.00th=[ 3982], 50.00th=[ 4080], 60.00th=[ 4228],
     | 70.00th=[ 4359], 80.00th=[ 4817], 90.00th=[ 6849], 95.00th=[ 8717],
     | 99.00th=[14484], 99.50th=[21103], 99.90th=[46924], 99.95th=[64226],
     | 99.99th=[93848]
   bw (  KiB/s): min=21736, max=67648, per=100.00%, avg=52628.24, stdev=5516.42, samples=119
   iops        : min= 5434, max=16912, avg=13156.97, stdev=1379.10, samples=119
  lat (usec)   : 10=0.01%, 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%
  lat (usec)   : 1000=0.01%
  lat (msec)   : 2=0.01%, 4=42.13%, 10=55.56%, 20=1.78%, 50=0.45%
  lat (msec)   : 100=0.09%, 250=0.01%
  cpu          : usr=7.41%, sys=86.90%, ctx=7816, majf=0, minf=1071
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=0,788655,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
  WRITE: bw=51.3MiB/s (53.8MB/s), 51.3MiB/s-51.3MiB/s (53.8MB/s-53.8MB/s), io=3081MiB (3231MB), run=60001-60001msec
Read throughput: 
read_throughput: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
...
fio-3.25
Starting 8 processes
read_throughput: Laying out IO file (1 file / 10240MiB)
read_throughput: Laying out IO file (1 file / 10240MiB)
read_throughput: Laying out IO file (1 file / 10240MiB)
read_throughput: Laying out IO file (1 file / 10240MiB)
read_throughput: Laying out IO file (1 file / 10240MiB)
read_throughput: Laying out IO file (1 file / 10240MiB)
read_throughput: Laying out IO file (1 file / 10240MiB)
read_throughput: Laying out IO file (1 file / 10240MiB)

read_throughput: (groupid=0, jobs=8): err= 0: pid=992057: Sun May 29 12:17:51 2022
  read: IOPS=2012, BW=2021MiB/s (2120MB/s)(118GiB/60008msec)
    slat (usec): min=468, max=49656, avg=3956.38, stdev=2885.51
    clat (usec): min=7, max=351698, avg=249809.39, stdev=62747.82
     lat (msec): min=4, max=357, avg=253.77, stdev=63.65
    clat percentiles (msec):
     |  1.00th=[   44],  5.00th=[   71], 10.00th=[  186], 20.00th=[  230],
     | 30.00th=[  249], 40.00th=[  259], 50.00th=[  271], 60.00th=[  275],
     | 70.00th=[  284], 80.00th=[  292], 90.00th=[  300], 95.00th=[  309],
     | 99.00th=[  326], 99.50th=[  330], 99.90th=[  347], 99.95th=[  347],
     | 99.99th=[  351]
   bw (  MiB/s): min= 1542, max= 4057, per=99.64%, avg=2014.23, stdev=110.18, samples=958
   iops        : min= 1542, max= 4055, avg=2013.44, stdev=110.15, samples=958
  lat (usec)   : 10=0.01%, 20=0.01%
  lat (msec)   : 10=0.01%, 20=0.02%, 50=3.98%, 100=2.95%, 250=24.74%
  lat (msec)   : 500=68.71%
  cpu          : usr=0.55%, sys=51.30%, ctx=69469, majf=0, minf=61113
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=120789,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=2021MiB/s (2120MB/s), 2021MiB/s-2021MiB/s (2120MB/s-2120MB/s), io=118GiB (127GB), run=60008-60008msec
Read IOPS: 
read_iops: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.25
Starting 1 process
read_iops: Laying out IO file (1 file / 10240MiB)

read_iops: (groupid=0, jobs=1): err= 0: pid=1216501: Sun May 29 12:19:18 2022
  read: IOPS=21.5k, BW=84.2MiB/s (88.3MB/s)(5050MiB/60001msec)
    slat (usec): min=4, max=4262, avg=43.03, stdev=13.43
    clat (usec): min=3, max=15939, avg=2925.31, stdev=259.56
     lat (usec): min=48, max=15993, avg=2968.71, stdev=262.68
    clat percentiles (usec):
     |  1.00th=[ 2573],  5.00th=[ 2671], 10.00th=[ 2704], 20.00th=[ 2769],
     | 30.00th=[ 2802], 40.00th=[ 2835], 50.00th=[ 2900], 60.00th=[ 2933],
     | 70.00th=[ 2999], 80.00th=[ 3064], 90.00th=[ 3163], 95.00th=[ 3326],
     | 99.00th=[ 3687], 99.50th=[ 3884], 99.90th=[ 5014], 99.95th=[ 5866],
     | 99.99th=[10159]
   bw (  KiB/s): min=71608, max=90976, per=100.00%, avg=86181.87, stdev=2890.69, samples=119
   iops        : min=17902, max=22744, avg=21545.37, stdev=722.64, samples=119
  lat (usec)   : 4=0.01%, 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%
  lat (usec)   : 1000=0.01%
  lat (msec)   : 2=0.01%, 4=99.62%, 10=0.37%, 20=0.01%
  cpu          : usr=7.54%, sys=92.43%, ctx=292, majf=0, minf=68
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=1292735,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=84.2MiB/s (88.3MB/s), 84.2MiB/s-84.2MiB/s (88.3MB/s-88.3MB/s), io=5050MiB (5295MB), run=60001-60001msec
