Write throughput: 
write_throughput: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
...
fio-3.25
Starting 8 processes
write_throughput: Laying out IO file (1 file / 10240MiB)
write_throughput: Laying out IO file (1 file / 10240MiB)
write_throughput: Laying out IO file (1 file / 10240MiB)
write_throughput: Laying out IO file (1 file / 10240MiB)
write_throughput: Laying out IO file (1 file / 10240MiB)
write_throughput: Laying out IO file (1 file / 10240MiB)
write_throughput: Laying out IO file (1 file / 10240MiB)
write_throughput: Laying out IO file (1 file / 10240MiB)

write_throughput: (groupid=0, jobs=8): err= 0: pid=2029018: Sun May 29 12:37:37 2022
  write: IOPS=734, BW=743MiB/s (779MB/s)(43.5GiB/60007msec); 0 zone resets
    slat (msec): min=2, max=369, avg=10.88, stdev= 5.19
    clat (usec): min=6, max=1720.5k, avg=680898.79, stdev=112010.89
     lat (msec): min=3, max=1732, avg=691.73, stdev=113.22
    clat percentiles (msec):
     |  1.00th=[  447],  5.00th=[  506], 10.00th=[  514], 20.00th=[  634],
     | 30.00th=[  684], 40.00th=[  693], 50.00th=[  701], 60.00th=[  709],
     | 70.00th=[  718], 80.00th=[  726], 90.00th=[  751], 95.00th=[  785],
     | 99.00th=[  986], 99.50th=[ 1133], 99.90th=[ 1687], 99.95th=[ 1703],
     | 99.99th=[ 1720]
   bw (  KiB/s): min=261931, max=1072830, per=98.88%, avg=752374.85, stdev=14434.76, samples=952
   iops        : min=  254, max= 1047, avg=734.54, stdev=14.10, samples=952
  lat (usec)   : 10=0.01%, 20=0.01%
  lat (msec)   : 4=0.01%, 10=0.01%, 20=0.02%, 50=0.08%, 100=0.09%
  lat (msec)   : 250=0.24%, 500=2.92%, 750=88.27%, 1000=8.56%, 2000=0.93%
  cpu          : usr=0.64%, sys=6.44%, ctx=373081, majf=0, minf=9303
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=0,44076,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
  WRITE: bw=743MiB/s (779MB/s), 743MiB/s-743MiB/s (779MB/s-779MB/s), io=43.5GiB (46.8GB), run=60007-60007msec
Write IOPS: 
write_iops: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.25
Starting 1 process
write_iops: Laying out IO file (1 file / 10240MiB)

write_iops: (groupid=0, jobs=1): err= 0: pid=2129775: Sun May 29 12:38:39 2022
  write: IOPS=13.0k, BW=50.9MiB/s (53.3MB/s)(3053MiB/60001msec); 0 zone resets
    slat (usec): min=8, max=72968, avg=71.70, stdev=194.55
    clat (usec): min=8, max=103193, avg=4842.26, stdev=2784.14
     lat (usec): min=155, max=103402, avg=4914.62, stdev=2818.45
    clat percentiles (usec):
     |  1.00th=[ 3130],  5.00th=[ 3392], 10.00th=[ 3556], 20.00th=[ 3752],
     | 30.00th=[ 3884], 40.00th=[ 4015], 50.00th=[ 4178], 60.00th=[ 4293],
     | 70.00th=[ 4490], 80.00th=[ 4948], 90.00th=[ 7373], 95.00th=[ 9110],
     | 99.00th=[13829], 99.50th=[19530], 99.90th=[36963], 99.95th=[48497],
     | 99.99th=[80217]
   bw (  KiB/s): min=22128, max=70080, per=100.00%, avg=52139.50, stdev=6664.75, samples=119
   iops        : min= 5532, max=17520, avg=13032.89, stdev=1667.93, samples=119
  lat (usec)   : 10=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
  lat (msec)   : 4=37.69%, 10=59.79%, 20=2.06%, 50=0.43%, 100=0.04%
  lat (msec)   : 250=0.01%
  cpu          : usr=7.18%, sys=87.59%, ctx=11072, majf=0, minf=413
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=0,781393,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
  WRITE: bw=50.9MiB/s (53.3MB/s), 50.9MiB/s-50.9MiB/s (53.3MB/s-53.3MB/s), io=3053MiB (3201MB), run=60001-60001msec
Read throughput: 
read_throughput: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
...
fio-3.25
Starting 8 processes
read_throughput: Laying out IO file (1 file / 10240MiB)
read_throughput: Laying out IO file (1 file / 10240MiB)
read_throughput: Laying out IO file (1 file / 10240MiB)
read_throughput: Laying out IO file (1 file / 10240MiB)
read_throughput: Laying out IO file (1 file / 10240MiB)
read_throughput: Laying out IO file (1 file / 10240MiB)
read_throughput: Laying out IO file (1 file / 10240MiB)
read_throughput: Laying out IO file (1 file / 10240MiB)

read_throughput: (groupid=0, jobs=8): err= 0: pid=2538797: Sun May 29 12:42:04 2022
  read: IOPS=1991, BW=2000MiB/s (2098MB/s)(117GiB/60004msec)
    slat (usec): min=511, max=48637, avg=3998.25, stdev=3014.24
    clat (usec): min=6, max=464061, avg=252598.15, stdev=59313.04
     lat (msec): min=2, max=471, avg=256.60, stdev=60.14
    clat percentiles (msec):
     |  1.00th=[   70],  5.00th=[  108], 10.00th=[  163], 20.00th=[  228],
     | 30.00th=[  247], 40.00th=[  259], 50.00th=[  268], 60.00th=[  275],
     | 70.00th=[  284], 80.00th=[  292], 90.00th=[  300], 95.00th=[  309],
     | 99.00th=[  347], 99.50th=[  430], 99.90th=[  451], 99.95th=[  456],
     | 99.99th=[  460]
   bw (  MiB/s): min= 1320, max= 4228, per=99.75%, avg=1995.40, stdev=73.87, samples=952
   iops        : min= 1320, max= 4228, avg=1995.00, stdev=73.89, samples=952
  lat (usec)   : 10=0.01%, 20=0.01%, 50=0.01%
  lat (msec)   : 4=0.01%, 10=0.01%, 20=0.01%, 50=0.10%, 100=4.07%
  lat (msec)   : 250=27.88%, 500=68.34%
  cpu          : usr=0.51%, sys=51.13%, ctx=68145, majf=0, minf=62760
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=119523,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=2000MiB/s (2098MB/s), 2000MiB/s-2000MiB/s (2098MB/s-2098MB/s), io=117GiB (126GB), run=60004-60004msec
Read IOPS: 
read_iops: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.25
Starting 1 process
read_iops: Laying out IO file (1 file / 10240MiB)

read_iops: (groupid=0, jobs=1): err= 0: pid=2759610: Sun May 29 12:43:32 2022
  read: IOPS=19.1k, BW=74.6MiB/s (78.2MB/s)(4474MiB/60001msec)
    slat (usec): min=4, max=10913, avg=48.63, stdev=71.12
    clat (usec): min=3, max=49033, avg=3301.70, stdev=1016.61
     lat (usec): min=49, max=49079, avg=3350.74, stdev=1029.90
    clat percentiles (usec):
     |  1.00th=[ 2704],  5.00th=[ 2802], 10.00th=[ 2868], 20.00th=[ 2933],
     | 30.00th=[ 2966], 40.00th=[ 3032], 50.00th=[ 3064], 60.00th=[ 3130],
     | 70.00th=[ 3195], 80.00th=[ 3294], 90.00th=[ 3916], 95.00th=[ 4621],
     | 99.00th=[ 6652], 99.50th=[ 7504], 99.90th=[17171], 99.95th=[24249],
     | 99.99th=[31851]
   bw (  KiB/s): min=22709, max=88320, per=100.00%, avg=76380.91, stdev=11721.48, samples=120
   iops        : min= 5677, max=22080, avg=19095.13, stdev=2930.34, samples=120
  lat (usec)   : 4=0.01%, 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%
  lat (usec)   : 1000=0.01%
  lat (msec)   : 2=0.01%, 4=91.25%, 10=8.60%, 20=0.08%, 50=0.07%
  cpu          : usr=7.52%, sys=86.14%, ctx=5503, majf=0, minf=68
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=1145334,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=74.6MiB/s (78.2MB/s), 74.6MiB/s-74.6MiB/s (78.2MB/s-78.2MB/s), io=4474MiB (4692MB), run=60001-60001msec
