readtest: (groupid=0, jobs=1): err= 0: pid=14420: Thu Dec 5 05:25:28 2013 read : io=1069.8GB, bw=311593KB/s, iops=77898, runt=3600001msec slat (usec): min=4, max=4885, avg=10.63, stdev= 4.36 clat (usec): min=56, max=111122, avg=809.45, stdev=353.40 lat (usec): min=65, max=111128, avg=820.28, stdev=356.21 clat percentiles (usec): | 1.00th=[ 644], 5.00th=[ 684], 10.00th=[ 700], 20.00th=[ 724], | 30.00th=[ 740], 40.00th=[ 748], 50.00th=[ 764], 60.00th=[ 780], | 70.00th=[ 788], 80.00th=[ 804], 90.00th=[ 812], 95.00th=[ 828], | 99.00th=[ 3152], 99.50th=[ 3248], 99.90th=[ 3888], 99.95th=[ 5024], | 99.99th=[ 7648] bw (KB /s): min=75512, max=325320, per=100.00%, avg=311596.35, stdev=52587.77 lat (usec) : 100=0.01%, 250=0.01%, 500=0.07%, 750=38.78%, 1000=58.38% lat (msec) : 2=0.86%, 4=1.81%, 10=0.09%, 20=0.01%, 50=0.01% lat (msec) : 100=0.01%, 250=0.01% cpu : usr=16.05%, sys=83.75%, ctx=26752, majf=0, minf=90 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0% issued : total=r=280433911/w=0/d=0, short=r=0/w=0/d=0 writetest: (groupid=0, jobs=1): err= 0: pid=14421: Thu Dec 5 05:25:28 2013 write: io=298899MB, bw=85020KB/s, iops=21255, runt=3600001msec slat (usec): min=5, max=1441, avg=39.69, stdev=32.40 clat (usec): min=54, max=101683, avg=2966.24, stdev=1018.17 lat (usec): min=63, max=101729, avg=3006.59, stdev=1031.26 clat percentiles (usec): | 1.00th=[ 804], 5.00th=[ 812], 10.00th=[ 812], 20.00th=[ 3248], | 30.00th=[ 3344], 40.00th=[ 3376], 50.00th=[ 3408], 60.00th=[ 3408], | 70.00th=[ 3440], 80.00th=[ 3472], 90.00th=[ 3504], 95.00th=[ 3536], | 99.00th=[ 3632], 99.50th=[ 3696], 99.90th=[ 3888], 99.95th=[ 3920], | 99.99th=[ 4016] bw (KB /s): min=18968, max=319432, per=100.00%, avg=85018.71, stdev=50247.05 lat (usec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=17.29% lat (msec) : 2=0.03%, 4=82.65%, 10=0.01%, 20=0.01%, 50=0.01% lat (msec) : 100=0.01%, 250=0.01% cpu : usr=5.45%, sys=93.70%, ctx=1249907, majf=0, minf=27 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0% issued : total=r=0/w=76518249/d=0, short=r=0/w=0/d=0 Run status group 0 (all jobs): READ: io=1069.8GB, aggrb=311593KB/s, minb=311593KB/s, maxb=311593KB/s, mint=3600001msec, maxt=3600001msec WRITE: io=298899MB, aggrb=85020KB/s, minb=85020KB/s, maxb=85020KB/s, mint=3600001msec, maxt=3600001msec Disk stats (read/write): dm-0: ios=280433911/76505532, merge=0/0, ticks=42961357/6023271, in_queue=48695766, util=100.00%, aggrios=140216955/38259124, aggrmerge=0/0, aggrticks=20053298/2637382, aggrin_queue=22673908, aggrutil=100.00% sdc: ios=140317994/38158086, merge=0/0, ticks=20050174/2631521, in_queue=22666470, util=100.00% sdd: ios=140115917/38360163, merge=0/0, ticks=20056423/2643244, in_queue=22681346, util=100.00%