17
fio存储性能压测-10931853
source link: https://blog.51cto.com/wujianwei/2498104
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
一、环境说明：
此次演示是dellr620 物理服务器，系统为centos7.6 x86_64位
硬件相关的信息如下：
[root@mysql-redis scripts]# sh hw_msg1.sh
+--------------------------------------------------------------+
| This Machine's Hyper-Threading is Enabled(recommend disable) |
+--------------------------------------------------------------+
Systembit : 64
MEM info : 20*4096 MB
Disk_totle :
Pro_SN_name : Product Name: PowerEdge R620 Serial Number: 5RV2962
System name : \S
Board_SN name: Product Name: Serial Number:
CPU model : 32 Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
CPU_phy_num : 2
CPU_core_num : 8
CPU_logic_num: 16
Netcard info : Broadcom Inc. and subsidiaries NetXtreme BCM5720 Gigabit Ethernet PCIe
+++++++++++++++++++++++++++++++++++
<<raid informations>>
Virtual Drive: 0 (Target Id: 0)
Name :
RAID Level : Primary-0, Secondary-0, RAID Level Qualifier-0
Size : 893.75 GB
State : Optimal
Strip Size : 64 KB
Number Of Drives : 1
--
Virtual Drive Information:
Virtual Drive: 1 (Target Id: 1)
Name :
RAID Level : Primary-0, Secondary-0, RAID Level Qualifier-0
Size : 223.0 GB
State : Optimal
Strip Size : 64 KB
Number Of Drives : 1
+++++++++++++++++++++++++++++++++++
<<Virtual Drive Information>>
Device Id: 3
PD Type: SATA
Raw Size: 894.252 GB [0x6fc81ab0 Sectors]
Inquiry Data: 56AB609UKCDU TOSHIBA Q300. SAFM12.3
Device Speed: 6.0Gb/s
Drive Temperature : N/A
Device Id: 0
PD Type: SATA
Raw Size: 223.570 GB [0x1bf244b0 Sectors]
Inquiry Data: PHDV723500E8240AGN INTEL SSDSC2BB240G7 N2010112
Device Speed: 6.0Gb/s
Drive Temperature :25C (77.00 F)二、安装fio
1.redhat linux系统直接yum 安装
yum install fio.x86_64
centos7.6系统上直接yum 安装的话，默认的版本是fio.3.7
2.源码编译安装
为了测试异步io，需要安装libaio
yum install libaio-devel
wget -P /data/soft http://github.com/axboe/fio/archive/fio-3.7.tar.gz
tar xf /data/soft/fio.3.7.tar.gz -C /usr/local/
cd /usr/local/fio.3.7
./configure
make && make install三、重要参数介绍
Fio各个版本的参数不尽相同，这里只列出日常较为广发使用的重要参数
filename=/dev/sdb1 或者/data/soft/test1 ： 指定测试硬盘名称或者测试文件名称（切忌选择系统设备测试，会导致系统奔溃的）
direct=1: 测试过程绕过机器自带的缓冲区，使得测试的结果更准确。
rw=randwrite: 随机写。 rw可取值：randwrite/randread/read/write/randrw/trim/randtrim/trimwrite ;分别是随机写，随机读，顺序读，顺序写，混合随机读写，
bs=16k : 单次I/O块大小为16KB,单次I/O大小对IOPS的影响比较大，一般来说要想得到最大的IOPS，bs越小越好。当然，这也和文件系统最小快的大小有关。一般linux下的文件分区最小快的大小为512B,1KB,2KB,4KB,8KB等可以根据文件系统类型进行选择。
size=5g : 测试文件大小为5GB
ioengine=libaio: I/O引擎使用libaio,libaio是异步的I/O引擎，一次提交一批I/O,然后等待这批I/O完成，这种方式减少了交互的次数，效率更好。FIO 至少支持13种不通的I/O引擎。sync,mmap,libaio,network等
iodepth=4 : I/O队列深度，主要是根据设备的并行度来调整。通常有两种I/O访问方式。也就是同步I/O，和异步I/O.同步I/O一次只能发送一个I/o请求，等待内核完成才可以返回。这对于单个线程来说iodepth总是小于1.想要iodepth大于1，可以通过多线程并发执行做到。异步I/O一次提交多个I/O请求。等I/O完成或者间隔一段时间收割一次。iodepth一般用在异步I/O模型（libaio）,用于指定一次发起多少个I/O请求。Flash存储一般有多个并行的单元。支持多个I/O并发执行。较合适异步的I/O
-numjobs=4: 使用4个线程对硬盘设备进行压测 对于Raid设备，如raid1,raid5或者并行高的设备，可以考虑适当加大测试线程数-numjobs来测试。
runtime=1000: 测试时间为1000s.和参数size共同起作用，属于双限制，即达到任何一个限制都停止测试。
rwmixread=70: 在混合读写模式下，读占70%
group_reporting: 设置显示结果，指定group_reporting 将汇总每个进程的信息。
-name iops_randwrite： 改测试命名为 iops_randwrite
--output TestResult.log: 测试结果指定输出到文件TestResult.log 中
四、测试io性能
可以使用fio模拟测试各种不同类型的io请求，入随机写，顺序写，随机读，顺序读。混合随机读写
4.1随机写：
time fio -rw=randwrite -bs=4k -runtime=60 -iodepth 1 -numjobs=4 -size=5G -filename /data/soft/test1 -ioengine libaio -direct=1 -group_reporting -name iops_randwrite --output TestResult.log 命令介绍：
采用4个线程对 /data/soft/test1文件做持续时间为60s，队列深度为1，块大小为4kb的direct y异步随机写（libaio）测试，该测试命名为iops_randwrite.输出结果不安4个jobs展示，而是按照group 组汇总 ，这样得出的在此压力下从此文件系统的随机写的IOPS。对于Raid设备，如raid1,raid5或者并行高的设备，可以考虑适当加大测试线程数-numjobs来测试。
--output TestResult.log 输出到指定的日志
下面是随机写具体执行过程：
[root@mysql-redis soft]# time fio -rw=randwrite -bs=4k -runtime=60 -iodepth 1 -numjobs=4 -size=5G -filename /data/soft/test1 -ioengine libaio -direct=1 -group_reporting -name iops_randwrite
iops_randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
...
fio-3.7
Starting 4 processes
iops_randwrite: Laying out IO file (1 file / 5120MiB)
Jobs: 4 (f=4): [w(4)][100.0%][r=0KiB/s,w=68.2MiB/s][r=0,w=17.5k IOPS][eta 00m:00s]
iops_randwrite: (groupid=0, jobs=4): err= 0: pid=13505: Thu May 21 18:50:33 2020
write: IOPS=39.2k, BW=153MiB/s (160MB/s)(9182MiB/60001msec)
slat (usec): min=6, max=27813, avg=17.42, stdev=36.21
clat (nsec): min=1177, max=123217k, avg=82183.17, stdev=323538.76
lat (usec): min=53, max=123230, avg=99.84, stdev=325.69
clat percentiles (usec):
| 1.00th=[ 47], 5.00th=[ 51], 10.00th=[ 57], 20.00th=[ 62],
| 30.00th=[ 65], 40.00th=[ 70], 50.00th=[ 75], 60.00th=[ 79],
| 70.00th=[ 84], 80.00th=[ 91], 90.00th=[ 103], 95.00th=[ 117],
| 99.00th=[ 161], 99.50th=[ 206], 99.90th=[ 330], 99.95th=[ 619],
| 99.99th=[14615]
bw ( KiB/s): min=17536, max=45304, per=25.12%, avg=39354.01, stdev=5145.44, samples=476
iops : min= 4384, max=11326, avg=9838.50, stdev=1286.36, samples=476
lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=4.22%
lat (usec) : 100=84.07%, 250=11.50%, 500=0.12%, 750=0.03%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.03%, 50=0.01%
lat (msec) : 250=0.01%
cpu : usr=4.33%, sys=19.34%, ctx=2824889, majf=0, minf=1185
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,2350467,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=153MiB/s (160MB/s), 153MiB/s-153MiB/s (160MB/s-160MB/s), io=9182MiB (9628MB), run=60001-60001msec
Disk stats (read/write):
sda: ios=0/2349456, merge=0/0, ticks=0/167181, in_queue=166516, util=96.46%
real 1m0.706s
user 0m11.167s
sys 0m47.017s4.2随机读：
time fio -readonly -rw=randread -bs=4k -runtime=60 -iodepth 1 -numjobs=2 -filename /data/soft/test2 -ioengine libaio -direct=1 -group_reporting -name iops_randread 关于命令参数说明：
命令介绍：采用2个线程对 /data/soft/test1文件做持续时间为60s，队列深度为1，块大小为4kb的direct y异步随机读（libaio）测试，该测试命名为iops_randread.输出结果不安2个jobs展示，而是按照group 组汇总 ，这样得出的在此压力下从此文件系统的随机读的IOPS
下面是随机读具体执行过程：
[root@mysql-redis39 ~]# time fio -readonly -rw=randread -bs=4k -runtime=60 -iodepth 1 -numjobs=2 -filename /data/soft/test2 -ioengine libaio -direct=1 -group_reporting -name iops_randread
iops_randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
...
fio-3.7
Starting 2 processes
Jobs: 2 (f=2): [r(2)][100.0%][r=36.7MiB/s,w=0KiB/s][r=9389,w=0 IOPS][eta 00m:00s]
iops_randread: (groupid=0, jobs=2): err= 0: pid=24566: Fri May 22 06:21:46 2020
read: IOPS=9358, BW=36.6MiB/s (38.3MB/s)(2193MiB/60001msec)
slat (usec): min=7, max=1480, avg=23.09, stdev= 8.09
clat (usec): min=8, max=123984, avg=185.97, stdev=570.65
lat (usec): min=136, max=123999, avg=209.50, stdev=570.70
clat percentiles (usec):
| 1.00th=[ 139], 5.00th=[ 141], 10.00th=[ 145], 20.00th=[ 169],
| 30.00th=[ 174], 40.00th=[ 178], 50.00th=[ 182], 60.00th=[ 188],
| 70.00th=[ 198], 80.00th=[ 204], 90.00th=[ 212], 95.00th=[ 219],
| 99.00th=[ 255], 99.50th=[ 273], 99.90th=[ 322], 99.95th=[ 537],
| 99.99th=[ 1549]
bw ( KiB/s): min=13824, max=19616, per=49.99%, avg=18713.87, stdev=1060.66, samples=238
iops : min= 3456, max= 4904, avg=4678.43, stdev=265.16, samples=238
lat (usec) : 10=0.01%, 100=0.01%, 250=98.87%, 500=1.08%, 750=0.02%
lat (usec) : 1000=0.01%
lat (msec) : 2=0.02%, 4=0.01%, 20=0.01%, 250=0.01%
cpu : usr=4.11%, sys=15.44%, ctx=561546, majf=0, minf=498
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=561503,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=36.6MiB/s (38.3MB/s), 36.6MiB/s-36.6MiB/s (38.3MB/s-38.3MB/s), io=2193MiB (2300MB), run=60001-60001msec
Disk stats (read/write):
sda: ios=560393/2, merge=0/0, ticks=102314/0, in_queue=101910, util=97.80%
real 1m0.706s
user 0m5.649s
sys 0m19.196s以上需要关注的指标：
read: IOPS=9358, BW=36.6MiB/s (38.3MB/s)(2193MiB/60001msec) io=2193MiB (2300MB) 这些参数可以了解到iops和通途量
lat (usec): min=136, max=123999, avg=209.50, stdev=570.70 这些参数也要注意：io延迟平均在209.5微妙。
4.3顺序写：
time fio -rw=write -bs=1m -runtime=60 -iodepth 1 -numjobs=5 -size=5G -filename /data/soft/test3 -ioengine libaio -direct=1 -group_reporting -name bw_write
说明：采用4个线程对/data/soft/test3文件做持续60s，队列深度为1的块大小为1M的direct异步顺序写（libaio）压力测试该测试命名为bw_write.输出结果不安4个job分别展示，而是按照group汇总展示。
[root@mysql-redis ~]# time fio -rw=write -bs=1m -runtime=60 -iodepth 1 -numjobs=5 -size=5G -filename /data/soft/test3 -ioengine libaio -direct=1 -group_reporting -name bw_write
bw_write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=1
...
fio-3.7
Starting 5 processes
bw_write: Laying out IO file (1 file / 5120MiB)
Jobs: 5 (f=5): [W(5)][100.0%][r=0KiB/s,w=184MiB/s][r=0,w=184 IOPS][eta 00m:00s]
bw_write: (groupid=0, jobs=5): err= 0: pid=26278: Fri May 22 06:44:38 2020
write: IOPS=319, BW=320MiB/s (335MB/s)(18.7GiB/60009msec)
slat (usec): min=56, max=303, avg=140.02, stdev=29.86
clat (msec): min=3, max=1092, avg=15.49, stdev=67.68
lat (msec): min=3, max=1093, avg=15.63, stdev=67.68
4.4顺序读：
time fio -readonly -rw=read -bs=1m -runtime=60 -iodepth 1 -numjobs=2 -filename /data/soft/test2 -ioengine libaio -direct=1 -group_reporting -name bw_read [root@mysql-redis ~]# time fio -readonly -rw=read -bs=1m -runtime=60 -iodepth 1 -numjobs=2 -filename /data/soft/test2 -ioengine libaio -direct=1 -group_reporting -name bw_read
bw_read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=1
...
fio-3.7
Starting 2 processes
Jobs: 2 (f=2): [R(2)][100.0%][r=259MiB/s,w=0KiB/s][r=259,w=0 IOPS][eta 00m:00s]
bw_read: (groupid=0, jobs=2): err= 0: pid=26789: Fri May 22 06:51:01 2020
read: IOPS=255, BW=256MiB/s (268MB/s)(10.0GiB/40034msec)
slat (usec): min=95, max=1149, avg=245.64, stdev=42.23
clat (msec): min=4, max=130, avg= 7.57, stdev= 2.98
lat (msec): min=4, max=130, avg= 7.81, stdev= 2.984.5混合读写
time fio -rw=randrw -rwmixread=70 -bs=16k -runtime=60 -iodepth 1 -numjobs=2 -filename /data/soft/test2 -ioengine libaio -direct=1 -group_reporting -name iops_randrw命令介绍：
2个线程对/data/soft/test2 文件做持续时间为60s，队列深度为1，块大小为16kb的direct异步混合读写的（libaio）压力测试，读写比伪7:3，测试被命名为iops_randrw
rwmixwrite=30 意思是在混合读写的模式下，写占30%。
-rwmixread=70 意思是在混合读写的模式下，读占70%。
混合读写具体输出结果：
[root@mysql-redis ~]# time fio -rw=randrw -rwmixread=70 -bs=16k -runtime=60 -iodepth 1 -numjobs=2 -filename /data/soft/test2 -ioengine libaio -direct=1 -group_reporting -name iops_randrw
iops_randrw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=libaio, iodepth=1
...
fio-3.7
Starting 2 processes
Jobs: 2 (f=2): [m(2)][100.0%][r=62.0MiB/s,w=27.2MiB/s][r=4030,w=1740 IOPS][eta 00m:00s]
iops_randrw: (groupid=0, jobs=2): err= 0: pid=27095: Fri May 22 06:55:08 2020
read: IOPS=3931, BW=61.4MiB/s (64.4MB/s)(3686MiB/60001msec)
slat (usec): min=8, max=6353, avg=36.20, stdev=19.77
clat (nsec): min=1648, max=123466k, avg=346175.56, stdev=725094.33
lat (usec): min=12, max=123516, avg=382.99, stdev=725.55
clat percentiles (usec):
| 1.00th=[ 215], 5.00th=[ 241], 10.00th=[ 255], 20.00th=[ 277],
| 30.00th=[ 293], 40.00th=[ 310], 50.00th=[ 322], 60.00th=[ 334],
| 70.00th=[ 351], 80.00th=[ 375], 90.00th=[ 412], 95.00th=[ 449],
| 99.00th=[ 1221], 99.50th=[ 1467], 99.90th=[ 1631], 99.95th=[ 1827],
| 99.99th=[16712]
bw ( KiB/s): min=23328, max=33792, per=49.97%, avg=31431.39, stdev=1649.11, samples=238
iops : min= 1458, max= 2112, avg=1964.44, stdev=103.06, samples=238
write: IOPS=1686, BW=26.4MiB/s (27.6MB/s)(1581MiB/60001msec)
slat (usec): min=12, max=2005, avg=42.06, stdev=16.09
clat (usec): min=3, max=123270, avg=225.73, stdev=432.84
lat (usec): min=86, max=123320, avg=268.44, stdev=433.42
clat percentiles (usec):
| 1.00th=[ 78], 5.00th=[ 85], 10.00th=[ 100], 20.00th=[ 129],
| 30.00th=[ 149], 40.00th=[ 174], 50.00th=[ 202], 60.00th=[ 233],
| 70.00th=[ 265], 80.00th=[ 306], 90.00th=[ 359], 95.00th=[ 408],
| 99.00th=[ 510], 99.50th=[ 1090], 99.90th=[ 1549], 99.95th=[ 1614],
| 99.99th=[ 5080]
bw ( KiB/s): min= 9152, max=15329, per=49.97%, avg=13484.37, stdev=892.91, samples=238
iops : min= 572, max= 958, avg=842.76, stdev=55.80, samples=238
lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 50=0.01%, 100=3.06%
lat (usec) : 250=22.22%, 500=72.91%, 750=0.76%, 1000=0.05%
lat (msec) : 2=0.96%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
lat (msec) : 250=0.01%
cpu : usr=4.64%, sys=15.91%, ctx=346233, majf=0, minf=266
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=235884,101198,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=61.4MiB/s (64.4MB/s), 61.4MiB/s-61.4MiB/s (64.4MB/s-64.4MB/s), io=3686MiB (3865MB), run=60001-60001msec
WRITE: bw=26.4MiB/s (27.6MB/s), 26.4MiB/s-26.4MiB/s (27.6MB/s-27.6MB/s), io=1581MiB (1658MB), run=60001-60001msec
Disk stats (read/write):
sda: ios=274477/101294, merge=0/0, ticks=88860/19119, in_queue=107576, util=95.82%
real 1m0.690s
user 0m6.300s
sys 0m19.718s4.6SSD性能测试配置文件
配置文件内容如下：
[root@mysql-redis scripts]# cat /data/soft/ssd-test.fio
##2020-05-24
[global]
bs=4k
ioengine=libaio
iodepth=4
numjobs=1
group_reporting
size=5g
direct=1
runtime=60
directory=/data/soft
filename=ssd.test.file
#inflate-log=/data/soft/TestResult.log
#output=/data/soft/TestResult.log
[seq-read]
name=iops_seqread
rw=read
stonewall
[rand-read]
name=iops_randread
rw=randread
stonewall
[seq-write]
name=iops_seqwrite
rw=write
stonewall
[rand-write]
name=iops_randwrite
rw=randwrite
stonewall
[randrw]
name=iops_randrw
rwmixread=70
rw=randrw
stonewall下面是fio测试指定配置文件输出的内容：
[root@mysql-redis soft]# fio ssd-test.fio
iops_seqread: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
iops_randread: (g=1): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
iops_seqwrite: (g=2): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
iops_randwrite: (g=3): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
iops_randrw: (g=4): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
fio-3.7
Starting 5 processes
iops_seqread: Laying out IO file (1 file / 5120MiB)
Jobs: 1 (f=1): [_(4),m(1)][90.5%][r=47.8MiB/s,w=20.7MiB/s][r=12.2k,w=5308 IOPS][eta 00m:21s]
iops_seqread: (groupid=0, jobs=1): err= 0: pid=2142: Sun May 24 20:06:20 2020
read: IOPS=57.5k, BW=225MiB/s (236MB/s)(5120MiB/22782msec)
slat (usec): min=3, max=1943, avg= 5.13, stdev= 2.21
clat (usec): min=25, max=12467, avg=63.13, stdev=202.65
lat (usec): min=40, max=12472, avg=68.41, stdev=202.72
clat percentiles (usec):
| 1.00th=[ 45], 5.00th=[ 53], 10.00th=[ 55], 20.00th=[ 56],
| 30.00th=[ 56], 40.00th=[ 56], 50.00th=[ 57], 60.00th=[ 57],
| 70.00th=[ 57], 80.00th=[ 58], 90.00th=[ 59], 95.00th=[ 62],
| 99.00th=[ 69], 99.50th=[ 79], 99.90th=[ 2073], 99.95th=[ 5080],
| 99.99th=[10814]
bw ( KiB/s): min=43579, max=193812, per=75.80%, avg=174446.82, stdev=47419.87, samples=45
iops : min=10894, max=48453, avg=43611.36, stdev=11855.06, samples=45
lat (usec) : 50=3.39%, 100=96.25%, 250=0.08%, 500=0.07%, 750=0.04%
lat (usec) : 1000=0.02%
lat (msec) : 2=0.06%, 4=0.04%, 10=0.05%, 20=0.01%
cpu : usr=19.83%, sys=39.29%, ctx=1222141, majf=0, minf=114
IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=1310720,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=4
iops_randread: (groupid=1, jobs=1): err= 0: pid=2170: Sun May 24 20:06:20 2020
read: IOPS=21.7k, BW=84.9MiB/s (89.0MB/s)(5094MiB/60001msec)
slat (usec): min=3, max=1068, avg=12.62, stdev= 6.14
clat (usec): min=42, max=38568, avg=167.76, stdev=315.76
lat (usec): min=62, max=38588, avg=180.75, stdev=316.07
clat percentiles (usec):
| 1.00th=[ 130], 5.00th=[ 135], 10.00th=[ 137], 20.00th=[ 141],
| 30.00th=[ 143], 40.00th=[ 147], 50.00th=[ 151], 60.00th=[ 155],
| 70.00th=[ 161], 80.00th=[ 172], 90.00th=[ 192], 95.00th=[ 206],
| 99.00th=[ 251], 99.50th=[ 273], 99.90th=[ 3982], 99.95th=[ 7308],
| 99.99th=[14091]
bw ( KiB/s): min= 5397, max=74196, per=79.00%, avg=68682.74, stdev=15713.88, samples=119
iops : min= 1349, max=18549, avg=17170.32, stdev=3928.48, samples=119
lat (usec) : 50=0.01%, 100=0.02%, 250=98.94%, 500=0.82%, 750=0.02%
lat (usec) : 1000=0.02%
lat (msec) : 2=0.03%, 4=0.04%, 10=0.07%, 20=0.02%, 50=0.01%
cpu : usr=11.52%, sys=37.46%, ctx=668232, majf=0, minf=319
IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=1304064,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=4
iops_seqwrite: (groupid=2, jobs=1): err= 0: pid=2231: Sun May 24 20:06:20 2020
write: IOPS=47.1k, BW=184MiB/s (193MB/s)(5120MiB/27848msec)
slat (usec): min=4, max=139, avg= 7.50, stdev= 2.96
clat (usec): min=27, max=15374, avg=76.14, stdev=278.53
lat (usec): min=49, max=15381, avg=83.78, stdev=279.26
clat percentiles (usec):
| 1.00th=[ 56], 5.00th=[ 59], 10.00th=[ 60], 20.00th=[ 61],
| 30.00th=[ 62], 40.00th=[ 64], 50.00th=[ 65], 60.00th=[ 67],
| 70.00th=[ 68], 80.00th=[ 71], 90.00th=[ 73], 95.00th=[ 74],
| 99.00th=[ 81], 99.50th=[ 89], 99.90th=[ 3326], 99.95th=[ 3392],
| 99.99th=[15139]
bw ( KiB/s): min= 3769, max=175262, per=80.83%, avg=152166.87, stdev=43009.10, samples=55
iops : min= 942, max=43815, avg=38041.35, stdev=10752.22, samples=55
lat (usec) : 50=0.24%, 100=99.43%, 250=0.07%, 500=0.01%, 750=0.01%
lat (usec) : 1000=0.01%
lat (msec) : 2=0.01%, 4=0.21%, 10=0.01%, 20=0.03%
cpu : usr=11.79%, sys=43.48%, ctx=652945, majf=0, minf=95
IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,1310720,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=4
iops_randwrite: (groupid=3, jobs=1): err= 0: pid=2262: Sun May 24 20:06:20 2020
write: IOPS=47.2k, BW=185MiB/s (194MB/s)(5120MiB/27743msec)
slat (usec): min=4, max=415, avg= 7.49, stdev= 2.82
clat (usec): min=26, max=21032, avg=75.63, stdev=276.91
lat (usec): min=48, max=21058, avg=83.27, stdev=277.60
clat percentiles (usec):
| 1.00th=[ 56], 5.00th=[ 60], 10.00th=[ 60], 20.00th=[ 62],
| 30.00th=[ 62], 40.00th=[ 64], 50.00th=[ 65], 60.00th=[ 67],
| 70.00th=[ 68], 80.00th=[ 70], 90.00th=[ 73], 95.00th=[ 74],
| 99.00th=[ 80], 99.50th=[ 86], 99.90th=[ 3326], 99.95th=[ 3392],
| 99.99th=[15139]
bw ( KiB/s): min= 3175, max=142274, per=68.21%, avg=128901.20, stdev=33671.41, samples=55
iops : min= 793, max=35568, avg=32224.95, stdev=8417.81, samples=55
lat (usec) : 50=0.21%, 100=99.50%, 250=0.04%, 500=0.01%, 750=0.01%
lat (usec) : 1000=0.01%
lat (msec) : 2=0.01%, 4=0.20%, 10=0.01%, 20=0.02%, 50=0.01%
cpu : usr=12.97%, sys=43.05%, ctx=654916, majf=0, minf=258
IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,1310720,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=4
iops_randrw: (groupid=4, jobs=1): err= 0: pid=2292: Sun May 24 20:06:20 2020
read: IOPS=11.3k, BW=44.3MiB/s (46.5MB/s)(2659MiB/60001msec)
slat (usec): min=4, max=594, avg=13.08, stdev= 6.88
clat (usec): min=61, max=45073, avg=244.45, stdev=441.39
lat (usec): min=134, max=45115, avg=257.84, stdev=441.61
clat percentiles (usec):
| 1.00th=[ 137], 5.00th=[ 143], 10.00th=[ 151], 20.00th=[ 176],
| 30.00th=[ 190], 40.00th=[ 202], 50.00th=[ 212], 60.00th=[ 221],
| 70.00th=[ 233], 80.00th=[ 258], 90.00th=[ 330], 95.00th=[ 359],
| 99.00th=[ 474], 99.50th=[ 1270], 99.90th=[ 6521], 99.95th=[10028],
| 99.99th=[17695]
bw ( KiB/s): min= 3072, max=51824, per=99.92%, avg=45336.55, stdev=11250.66, samples=119
iops : min= 768, max=12958, avg=11334.12, stdev=2812.66, samples=119
write: IOPS=4860, BW=18.0MiB/s (19.9MB/s)(1139MiB/60001msec)
slat (usec): min=6, max=874, avg=16.94, stdev= 7.42
clat (usec): min=40, max=31265, avg=192.80, stdev=413.19
lat (usec): min=56, max=31306, avg=210.09, stdev=413.32
clat percentiles (usec):
| 1.00th=[ 52], 5.00th=[ 58], 10.00th=[ 62], 20.00th=[ 78],
| 30.00th=[ 145], 40.00th=[ 169], 50.00th=[ 188], 60.00th=[ 202],
| 70.00th=[ 212], 80.00th=[ 227], 90.00th=[ 251], 95.00th=[ 277],
| 99.00th=[ 619], 99.50th=[ 1205], 99.90th=[ 5997], 99.95th=[ 9634],
| 99.99th=[16581]
bw ( KiB/s): min= 1184, max=22176, per=99.94%, avg=19428.64, stdev=4847.41, samples=119
iops : min= 296, max= 5544, avg=4857.13, stdev=1211.85, samples=119
lat (usec) : 50=0.12%, 100=7.22%, 250=74.36%, 500=17.22%, 750=0.24%
lat (usec) : 1000=0.10%
lat (msec) : 2=0.52%, 4=0.06%, 10=0.11%, 20=0.04%, 50=0.01%
cpu : usr=7.88%, sys=29.46%, ctx=340101, majf=0, minf=374
IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=680630,291634,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=4
Run status group 0 (all jobs):
READ: bw=225MiB/s (236MB/s), 225MiB/s-225MiB/s (236MB/s-236MB/s), io=5120MiB (5369MB), run=22782-22782msec
Run status group 1 (all jobs):
READ: bw=84.9MiB/s (89.0MB/s), 84.9MiB/s-84.9MiB/s (89.0MB/s-89.0MB/s), io=5094MiB (5341MB), run=60001-60001msec
Run status group 2 (all jobs):
WRITE: bw=184MiB/s (193MB/s), 184MiB/s-184MiB/s (193MB/s-193MB/s), io=5120MiB (5369MB), run=27848-27848msec
Run status group 3 (all jobs):
WRITE: bw=185MiB/s (194MB/s), 185MiB/s-185MiB/s (194MB/s-194MB/s), io=5120MiB (5369MB), run=27743-27743msec
Run status group 4 (all jobs):
READ: bw=44.3MiB/s (46.5MB/s), 44.3MiB/s-44.3MiB/s (46.5MB/s-46.5MB/s), io=2659MiB (2788MB), run=60001-60001msec
WRITE: bw=18.0MiB/s (19.9MB/s), 18.0MiB/s-18.0MiB/s (19.9MB/s-19.9MB/s), io=1139MiB (1195MB), run=60001-60001msec
Disk stats (read/write):
sda: ios=3294785/2912828, merge=0/3, ticks=430527/238028, in_queue=666582, util=99.09%
[root@mysql-redis soft]# 关于fio测试硬盘的建议：
一、建议使用顺序I/O和较大的blocksize 来测试设备的通吐量和延迟
二、建议使用随机I/O和较小的blocksize来测试设备的IOPS和延迟
三、在配置numjobs和iodepth 测试底层存储性能时，建议要深入了解应用到底采用的是同步的io还是异步的io（是多进程并发i/o请求还是一次提交一批的i/o的请求）
提示：对fio压测 磁盘性能感兴趣的话，可以参考下ezfio(直接在github.com上搜索ezfio)
重要提示：fio是 支持对裸设备直接进行I/O测试的，在测试裸设备时，一定要避开系统所在的裸设备，否则会导致系统奔溃。
例如：
[root@mysql-redis soft]# df -h / /boot
文件系统 容量 已用 可用 已用% 挂载点
/dev/sdb5 207G 7.5G 200G 4% /
/dev/sdb1 297M 120M 177M 41% /boot/dev/sdb /dev/sdb5 /dev/sdb1 这些设备是绝对禁止进行fio测试的，
同时在测试其他的设备裸设备时，一定要提前确定好此裸设备上是否有重要 的数据，一定要重要的设备提前做好备份。
以上就是我这边对fio测试硬盘的简单学习总结，仅供参考，如有网友直接复制应用造成损失，本博文不负担任何责任。
参考文档：
http://www.ssdfans.com/?p=6087
https://linux.die.net/man/1/fio
https://blog.csdn.net/don_chiang709/article/details/92628623
Recommend
-
69
作者：王清培本文原创，转载请注明作者及出处压力机及压力工具检测Linux openfiles limit 设置排查周边依赖空接口压测检测聚合报告中 throug...
-
53
本文作者来自本号的粉丝：林夕_Yume，作者微信： wxid_nvd5wwng4v2i22 欢迎大家关注他的简书：https://www.jianshu.com/u/3fa2b243f30d 大家有同样需求的同学赶紧加他好...
-
15
Dubbo 服务性能压测（with JMeter） 最近在做Dubbo服务与Prometheus的监控集成，为了测试监控组件对Dubbo RPC 调用的性能影响，就需要对添加前后做性能测试。虽然之前给组内搭建了统一的Dubbo 服务测试平台，但...
-
25
MySQL之存储过程按月创建表
-
7
pmem.ioPersistent Memory Programming Running FIO with pmem engines When we, the PMDK team, wan...
-
18
1.12. Job file parameters¶ This section describes in details each parameter associated with a job. Some parameters take an option of a given type, s...
-
7
Changelly 已集成 FIO 协议非托管即时加密货币交易所 Changelly 宣布，它已经集成了钱包间可互操作协议。这种集成将使 Changelly 用户能够交易和发送加密货币到简单易读的地址。这项新功能旨在简化用户与平台之间的交互过程。它允许用户使用诸如...
-
7
使用 fio 测试磁盘 I/O 性能fio 是测试 IOPS 的非常好的工具，用来对硬件进行压力测试和验证，支持 13 种不同的 I/O 引擎，包括: sync,mmap, libaio, posixaio, SG v3, splice, null, network, syslet, guasi, solarisaio 等等。 简单好用...
-
4
Running FIO with pmem engines Jun 25, 2018LukaszstolarczukWhen we, the PMDK team, want to check performan...
-
8
Fio Output Explained Skip to main content Previously, I blogged about setting up my
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK