readtest: (groupid=0, jobs=1): err= 0: pid=13804: Thu Dec 5 03:23:15 2013 read : io=1095.8GB, bw=319167KB/s, iops=79791, runt=3600001msec slat (usec): min=7, max=190, avg=10.26, stdev= 1.33 clat (usec): min=49, max=18699, avg=188.82, stdev=209.17 lat (usec): min=63, max=18712, avg=199.27, stdev=209.17 clat percentiles (usec): | 1.00th=[ 75], 5.00th=[ 98], 10.00th=[ 111], 20.00th=[ 129], | 30.00th=[ 145], 40.00th=[ 161], 50.00th=[ 173], 60.00th=[ 185], | 70.00th=[ 195], 80.00th=[ 205], 90.00th=[ 219], 95.00th=[ 231], | 99.00th=[ 1064], 99.50th=[ 1816], 99.90th=[ 2608], 99.95th=[ 2736], | 99.99th=[ 6624] bw (KB /s): min=25464, max=324144, per=100.00%, avg=319170.75, stdev=6391.96 lat (usec) : 50=0.01%, 100=5.78%, 250=91.67%, 500=1.04%, 750=0.28% lat (usec) : 1000=0.19% lat (msec) : 2=0.64%, 4=0.37%, 10=0.03%, 20=0.01% cpu : usr=16.36%, sys=82.60%, ctx=2393448, majf=0, minf=42 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=287250155/w=0/d=0, short=r=0/w=0/d=0 writetest: (groupid=0, jobs=1): err= 0: pid=13805: Thu Dec 5 03:23:15 2013 write: io=264691MB, bw=75290KB/s, iops=18822, runt=3600001msec slat (usec): min=8, max=646, avg=44.86, stdev=31.68 clat (usec): min=32, max=3832, avg=799.43, stdev=82.64 lat (usec): min=55, max=3841, avg=845.00, stdev=85.81 clat percentiles (usec): | 1.00th=[ 430], 5.00th=[ 684], 10.00th=[ 724], 20.00th=[ 756], | 30.00th=[ 780], 40.00th=[ 796], 50.00th=[ 812], 60.00th=[ 820], | 70.00th=[ 836], 80.00th=[ 860], 90.00th=[ 884], 95.00th=[ 900], | 99.00th=[ 940], 99.50th=[ 956], 99.90th=[ 1004], 99.95th=[ 1208], | 99.99th=[ 1304] bw (KB /s): min=73208, max=197440, per=100.00%, avg=75286.59, stdev=2249.77 lat (usec) : 50=0.01%, 100=0.01%, 250=0.01%, 500=1.55%, 750=15.56% lat (usec) : 1000=82.78% lat (msec) : 2=0.11%, 4=0.01% cpu : usr=5.03%, sys=94.18%, ctx=1230144, majf=0, minf=27 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=0/w=67760982/d=0, short=r=0/w=0/d=0 Run status group 0 (all jobs): READ: io=1095.8GB, aggrb=319166KB/s, minb=319166KB/s, maxb=319166KB/s, mint=3600001msec, maxt=3600001msec WRITE: io=264691MB, aggrb=75289KB/s, minb=75289KB/s, maxb=75289KB/s, mint=3600001msec, maxt=3600001msec Disk stats (read/write): dm-0: ios=287250155/67748969, merge=0/0, ticks=43504117/5435253, in_queue=48649946, util=100.00%, aggrios=143625077/33880491, aggrmerge=0/0, aggrticks=20322288/2404898, aggrin_queue=22710318, aggrutil=100.00% sdc: ios=143808391/33697177, merge=0/0, ticks=20337376/2394340, in_queue=22716266, util=100.00% sdd: ios=143441764/34063805, merge=0/0, ticks=20307201/2415457, in_queue=22704371, util=100.00%