徒然メモ

技術的なことを書いたり書かなかったり

General Purpose (SSD)なEBSでベンチマーク

General Purpose (SSD)なEBSでベンチマーク

リージョン:ap-northeast-1 インスタンスタイプ:c3large ボリュームサイズ:8GB OS:AmazonLinux64bit AMIID:amzn-ami-pv-2014.03.1.x86_64-ebs (ami-c9562fc8)

fioコマンド2.1.7を使って計測 ベンチマークのパターンはhttp://www.winkey.jp/downloads/index.php/fio-crystaldiskmarkで公開されている ジョブファイルを利用。

連続して10回実施した結果は以下の通りでした。

取り急ぎまるっと結果だけはってます。

1回目

[root@ip-10-125-0-53 tmp]# fio crystaldiskmark.fio

Seq-Read: (g=0): rw=read, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Seq-Write: (g=1): rw=write, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Rand-Read-512K: (g=2): rw=randread, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Write-512K: (g=3): rw=randwrite, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Read-4K: (g=4): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Write-4K: (g=5): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Read-4K-QD32: (g=6): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
Rand-Write-4K-QD32: (g=7): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1.7
Starting 8 processes
Seq-Read: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [_______w] [92.4% done] [0KB/12252KB/0KB /s] [0/3063/0 iops] [eta 00m:25s]
Seq-Read: (groupid=0, jobs=1): err= 0: pid=15430: Wed Jun 18 00:29:36 2014
  read : io=1024.0MB, bw=68737KB/s, iops=67, runt= 15255msec
    slat (usec): min=58, max=576, avg=64.20, stdev=19.68
    clat (msec): min=1, max=138, avg=14.83, stdev=24.51
     lat (msec): min=1, max=138, avg=14.89, stdev=24.51
    clat percentiles (usec):
     |  1.00th=[ 1592],  5.00th=[ 1720], 10.00th=[ 1880], 20.00th=[ 2288],
     | 30.00th=[ 2768], 40.00th=[ 3504], 50.00th=[ 4640], 60.00th=[ 6944],
     | 70.00th=[ 7904], 80.00th=[ 9024], 90.00th=[57600], 95.00th=[66048],
     | 99.00th=[103936], 99.50th=[104960], 99.90th=[107008], 99.95th=[138240],
     | 99.99th=[138240]
    bw (KB  /s): min=50452, max=158476, per=100.00%, avg=69473.50, stdev=27190.21
    lat (msec) : 2=12.60%, 4=32.52%, 10=35.45%, 20=2.73%, 50=5.57%
    lat (msec) : 100=9.28%, 250=1.86%
  cpu          : usr=0.00%, sys=0.34%, ctx=1025, majf=0, minf=283
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1024/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Seq-Write: (groupid=1, jobs=1): err= 0: pid=15431: Wed Jun 18 00:29:36 2014
  write: io=1024.0MB, bw=63087KB/s, iops=61, runt= 16621msec
    slat (usec): min=251, max=448, avg=319.05, stdev=38.56
    clat (msec): min=8, max=67, avg=15.90, stdev= 5.19
     lat (msec): min=8, max=68, avg=16.22, stdev= 5.20
    clat percentiles (usec):
     |  1.00th=[ 8768],  5.00th=[ 8768], 10.00th=[ 9024], 20.00th=[11072],
     | 30.00th=[15680], 40.00th=[17024], 50.00th=[17280], 60.00th=[17280],
     | 70.00th=[17280], 80.00th=[17280], 90.00th=[17792], 95.00th=[21376],
     | 99.00th=[32128], 99.50th=[41728], 99.90th=[66048], 99.95th=[68096],
     | 99.99th=[68096]
    bw (KB  /s): min=44609, max=102400, per=100.00%, avg=63271.16, stdev=13080.46
    lat (msec) : 10=16.80%, 20=77.73%, 50=5.08%, 100=0.39%
  cpu          : usr=0.07%, sys=1.97%, ctx=1035, majf=0, minf=27
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=1024/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-512K: (groupid=2, jobs=1): err= 0: pid=15432: Wed Jun 18 00:29:36 2014
  read : io=1024.0MB, bw=68449KB/s, iops=133, runt= 15319msec
    slat (usec): min=34, max=333, avg=38.85, stdev=11.21
    clat (usec): min=850, max=135718, avg=7435.01, stdev=18348.99
     lat (usec): min=887, max=135759, avg=7475.03, stdev=18348.52
    clat percentiles (usec):
     |  1.00th=[  876],  5.00th=[  916], 10.00th=[  948], 20.00th=[ 1032],
     | 30.00th=[ 1128], 40.00th=[ 1304], 50.00th=[ 1800], 60.00th=[ 2672],
     | 70.00th=[ 3888], 80.00th=[ 3984], 90.00th=[ 9024], 95.00th=[45312],
     | 99.00th=[95744], 99.50th=[97792], 99.90th=[132096], 99.95th=[134144],
     | 99.99th=[136192]
    bw (KB  /s): min=51480, max=159107, per=100.00%, avg=68926.59, stdev=26690.57
    lat (usec) : 1000=16.99%
    lat (msec) : 2=36.38%, 4=27.10%, 10=10.11%, 20=1.27%, 50=3.81%
    lat (msec) : 100=4.05%, 250=0.29%
  cpu          : usr=0.00%, sys=0.37%, ctx=2049, majf=0, minf=153
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2048/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-512K: (groupid=3, jobs=1): err= 0: pid=15433: Wed Jun 18 00:29:36 2014
  write: io=1024.0MB, bw=63141KB/s, iops=123, runt= 16607msec
    slat (usec): min=134, max=270, avg=152.12, stdev=11.11
    clat (msec): min=4, max=44, avg= 7.95, stdev= 2.11
     lat (msec): min=5, max=44, avg= 8.10, stdev= 2.11
    clat percentiles (usec):
     |  1.00th=[ 4960],  5.00th=[ 5024], 10.00th=[ 5088], 20.00th=[ 5792],
     | 30.00th=[ 8256], 40.00th=[ 8512], 50.00th=[ 8512], 60.00th=[ 8640],
     | 70.00th=[ 8640], 80.00th=[ 8768], 90.00th=[ 8768], 95.00th=[ 9152],
     | 99.00th=[12864], 99.50th=[13376], 99.90th=[37120], 99.95th=[41728],
     | 99.99th=[44288]
    bw (KB  /s): min=58135, max=91136, per=100.00%, avg=63216.55, stdev=10648.52
    lat (msec) : 10=97.12%, 20=2.69%, 50=0.20%
  cpu          : usr=0.07%, sys=1.93%, ctx=2061, majf=0, minf=24
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=2048/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K: (groupid=4, jobs=1): err= 0: pid=15434: Wed Jun 18 00:29:36 2014
  read : io=735588KB, bw=12260KB/s, iops=3064, runt= 60001msec
    slat (usec): min=8, max=44, avg= 8.96, stdev= 0.58
    clat (usec): min=129, max=26382, avg=312.83, stdev=372.24
     lat (usec): min=147, max=26392, avg=322.91, stdev=372.23
    clat percentiles (usec):
     |  1.00th=[  143],  5.00th=[  147], 10.00th=[  151], 20.00th=[  159],
     | 30.00th=[  171], 40.00th=[  187], 50.00th=[  203], 60.00th=[  221],
     | 70.00th=[  239], 80.00th=[  278], 90.00th=[  596], 95.00th=[ 1176],
     | 99.00th=[ 1768], 99.50th=[ 1928], 99.90th=[ 2224], 99.95th=[ 3120],
     | 99.99th=[10176]
    bw (KB  /s): min=11344, max=14573, per=100.00%, avg=12265.44, stdev=266.52
    lat (usec) : 250=74.23%, 500=14.71%, 750=2.48%, 1000=2.06%
    lat (msec) : 2=6.18%, 4=0.29%, 10=0.03%, 20=0.01%, 50=0.01%
  cpu          : usr=0.51%, sys=4.55%, ctx=183904, majf=0, minf=26
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=183897/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-4K: (groupid=5, jobs=1): err= 0: pid=15435: Wed Jun 18 00:29:36 2014
  write: io=414696KB, bw=6911.5KB/s, iops=1727, runt= 60001msec
    slat (usec): min=10, max=124, avg=11.87, stdev= 1.31
    clat (usec): min=340, max=15056, avg=562.30, stdev=207.90
     lat (usec): min=393, max=15070, avg=575.29, stdev=207.93
    clat percentiles (usec):
     |  1.00th=[  406],  5.00th=[  430], 10.00th=[  446], 20.00th=[  478],
     | 30.00th=[  506], 40.00th=[  524], 50.00th=[  540], 60.00th=[  556],
     | 70.00th=[  572], 80.00th=[  604], 90.00th=[  652], 95.00th=[  740],
     | 99.00th=[ 1160], 99.50th=[ 1464], 99.90th=[ 2800], 99.95th=[ 4128],
     | 99.99th=[ 8640]
    bw (KB  /s): min= 4616, max= 7528, per=100.00%, avg=6917.13, stdev=428.66
    lat (usec) : 500=28.21%, 750=66.99%, 1000=3.23%
    lat (msec) : 2=1.37%, 4=0.15%, 10=0.05%, 20=0.01%
  cpu          : usr=0.23%, sys=3.21%, ctx=103679, majf=0, minf=25
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=103674/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K-QD32: (groupid=6, jobs=1): err= 0: pid=15436: Wed Jun 18 00:29:36 2014
  read : io=735708KB, bw=12260KB/s, iops=3064, runt= 60010msec
    slat (usec): min=6, max=48, avg= 9.02, stdev= 2.73
    clat (usec): min=594, max=27848, avg=10426.62, stdev=876.68
     lat (usec): min=646, max=27856, avg=10436.82, stdev=876.08
    clat percentiles (usec):
     |  1.00th=[ 8896],  5.00th=[ 9792], 10.00th=[ 9920], 20.00th=[10048],
     | 30.00th=[10048], 40.00th=[10048], 50.00th=[10176], 60.00th=[10688],
     | 70.00th=[10944], 80.00th=[10944], 90.00th=[11072], 95.00th=[11712],
     | 99.00th=[11968], 99.50th=[12352], 99.90th=[17280], 99.95th=[18560],
     | 99.99th=[22656]
    bw (KB  /s): min=12047, max=14616, per=100.00%, avg=12267.29, stdev=219.16
    lat (usec) : 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.22%, 4=0.09%, 10=21.38%, 20=78.29%, 50=0.03%
  cpu          : usr=1.29%, sys=3.43%, ctx=108093, majf=0, minf=58
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=183927/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32
Rand-Write-4K-QD32: (groupid=7, jobs=1): err= 0: pid=15437: Wed Jun 18 00:29:36 2014
  write: io=735436KB, bw=12255KB/s, iops=3063, runt= 60011msec
    slat (usec): min=7, max=7638, avg=11.92, stdev=51.40
    clat (usec): min=772, max=39037, avg=10427.89, stdev=980.24
     lat (usec): min=808, max=39050, avg=10440.96, stdev=978.75
    clat percentiles (usec):
     |  1.00th=[ 8096],  5.00th=[ 9664], 10.00th=[ 9920], 20.00th=[10048],
     | 30.00th=[10048], 40.00th=[10176], 50.00th=[10176], 60.00th=[10688],
     | 70.00th=[10816], 80.00th=[10944], 90.00th=[11072], 95.00th=[11456],
     | 99.00th=[12352], 99.50th=[13760], 99.90th=[18816], 99.95th=[21888],
     | 99.99th=[34048]
    bw (KB  /s): min=11992, max=14608, per=100.00%, avg=12261.61, stdev=219.91
    lat (usec) : 1000=0.01%
    lat (msec) : 2=0.04%, 4=0.34%, 10=15.46%, 20=84.09%, 50=0.07%
  cpu          : usr=1.16%, sys=4.61%, ctx=143595, majf=0, minf=24
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=183859/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=1024.0MB, aggrb=68736KB/s, minb=68736KB/s, maxb=68736KB/s, mint=15255msec, maxt=15255msec

Run status group 1 (all jobs):
  WRITE: io=1024.0MB, aggrb=63087KB/s, minb=63087KB/s, maxb=63087KB/s, mint=16621msec, maxt=16621msec

Run status group 2 (all jobs):
   READ: io=1024.0MB, aggrb=68449KB/s, minb=68449KB/s, maxb=68449KB/s, mint=15319msec, maxt=15319msec

Run status group 3 (all jobs):
  WRITE: io=1024.0MB, aggrb=63140KB/s, minb=63140KB/s, maxb=63140KB/s, mint=16607msec, maxt=16607msec

Run status group 4 (all jobs):
   READ: io=735588KB, aggrb=12259KB/s, minb=12259KB/s, maxb=12259KB/s, mint=60001msec, maxt=60001msec

Run status group 5 (all jobs):
  WRITE: io=414696KB, aggrb=6911KB/s, minb=6911KB/s, maxb=6911KB/s, mint=60001msec, maxt=60001msec

Run status group 6 (all jobs):
   READ: io=735708KB, aggrb=12259KB/s, minb=12259KB/s, maxb=12259KB/s, mint=60010msec, maxt=60010msec

Run status group 7 (all jobs):
  WRITE: io=735436KB, aggrb=12255KB/s, minb=12255KB/s, maxb=12255KB/s, mint=60011msec, maxt=60011msec

Disk stats (read/write):
  xvda1: ios=384208/303843, merge=0/31, ticks=2083220/2102420, in_queue=4185736, util=98.78%

2回目

# fio crystaldiskmark.fio

Seq-Read: (g=0): rw=read, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Seq-Write: (g=1): rw=write, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Rand-Read-512K: (g=2): rw=randread, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Write-512K: (g=3): rw=randwrite, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Read-4K: (g=4): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Write-4K: (g=5): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Read-4K-QD32: (g=6): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
Rand-Write-4K-QD32: (g=7): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1.7
Starting 8 processes
Jobs: 1 (f=1): [_______w] [92.2% done] [0KB/12252KB/0KB /s] [0/3063/0 iops] [eta 00m:26s]
Seq-Read: (groupid=0, jobs=1): err= 0: pid=15441: Wed Jun 18 00:41:26 2014
  read : io=1024.0MB, bw=68624KB/s, iops=67, runt= 15280msec
    slat (usec): min=57, max=606, avg=67.93, stdev=22.09
    clat (msec): min=1, max=137, avg=14.85, stdev=24.75
     lat (msec): min=1, max=137, avg=14.92, stdev=24.75
    clat percentiles (usec):
     |  1.00th=[ 1592],  5.00th=[ 1688], 10.00th=[ 1800], 20.00th=[ 2256],
     | 30.00th=[ 2736], 40.00th=[ 3664], 50.00th=[ 5472], 60.00th=[ 6944],
     | 70.00th=[ 7904], 80.00th=[ 9280], 90.00th=[56576], 95.00th=[68096],
     | 99.00th=[103936], 99.50th=[105984], 99.90th=[130560], 99.95th=[136192],
     | 99.99th=[136192]
    bw (KB  /s): min=53056, max=159107, per=100.00%, avg=69583.00, stdev=27367.41
    lat (msec) : 2=15.33%, 4=27.25%, 10=38.28%, 20=3.03%, 50=5.57%
    lat (msec) : 100=8.69%, 250=1.86%
  cpu          : usr=0.00%, sys=0.37%, ctx=1025, majf=0, minf=284
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1024/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Seq-Write: (groupid=1, jobs=1): err= 0: pid=15442: Wed Jun 18 00:41:26 2014
  write: io=1024.0MB, bw=63122KB/s, iops=61, runt= 16612msec
    slat (usec): min=251, max=416, avg=318.40, stdev=38.29
    clat (msec): min=8, max=46, avg=15.90, stdev= 3.21
     lat (msec): min=8, max=46, avg=16.22, stdev= 3.22
    clat percentiles (usec):
     |  1.00th=[ 8768],  5.00th=[ 9152], 10.00th=[ 9920], 20.00th=[13504],
     | 30.00th=[17024], 40.00th=[17024], 50.00th=[17280], 60.00th=[17280],
     | 70.00th=[17280], 80.00th=[17280], 90.00th=[17536], 95.00th=[18304],
     | 99.00th=[21888], 99.50th=[22912], 99.90th=[30080], 99.95th=[46336],
     | 99.99th=[46336]
    bw (KB  /s): min=58113, max=99952, per=100.00%, avg=63344.56, stdev=12284.54
    lat (msec) : 10=10.84%, 20=86.91%, 50=2.25%
  cpu          : usr=0.00%, sys=2.05%, ctx=1033, majf=0, minf=28
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=1024/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-512K: (groupid=2, jobs=1): err= 0: pid=15443: Wed Jun 18 00:41:26 2014
  read : io=1024.0MB, bw=68818KB/s, iops=134, runt= 15237msec
    slat (usec): min=34, max=332, avg=39.19, stdev=11.78
    clat (usec): min=856, max=136378, avg=7394.45, stdev=17870.99
     lat (usec): min=893, max=136418, avg=7434.83, stdev=17870.60
    clat percentiles (usec):
     |  1.00th=[  892],  5.00th=[  940], 10.00th=[  996], 20.00th=[ 1144],
     | 30.00th=[ 1320], 40.00th=[ 1624], 50.00th=[ 2096], 60.00th=[ 2928],
     | 70.00th=[ 3856], 80.00th=[ 3984], 90.00th=[ 8096], 95.00th=[46336],
     | 99.00th=[95744], 99.50th=[97792], 99.90th=[130560], 99.95th=[134144],
     | 99.99th=[136192]
    bw (KB  /s): min=49192, max=159107, per=100.00%, avg=69459.11, stdev=27577.85
    lat (usec) : 1000=10.64%
    lat (msec) : 2=37.65%, 4=35.40%, 10=6.88%, 20=1.22%, 50=3.76%
    lat (msec) : 100=4.15%, 250=0.29%
  cpu          : usr=0.00%, sys=0.39%, ctx=2049, majf=0, minf=154
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2048/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-512K: (groupid=3, jobs=1): err= 0: pid=15444: Wed Jun 18 00:41:26 2014
  write: io=1024.0MB, bw=63110KB/s, iops=123, runt= 16615msec
    slat (usec): min=135, max=316, avg=163.01, stdev=22.30
    clat (msec): min=4, max=45, avg= 7.94, stdev= 2.86
     lat (msec): min=5, max=45, avg= 8.11, stdev= 2.86
    clat percentiles (usec):
     |  1.00th=[ 4960],  5.00th=[ 5024], 10.00th=[ 5152], 20.00th=[ 5408],
     | 30.00th=[ 6880], 40.00th=[ 8384], 50.00th=[ 8512], 60.00th=[ 8640],
     | 70.00th=[ 8640], 80.00th=[ 8768], 90.00th=[ 8896], 95.00th=[10176],
     | 99.00th=[16064], 99.50th=[20096], 99.90th=[42752], 99.95th=[43264],
     | 99.99th=[45312]
    bw (KB  /s): min=53948, max=90592, per=100.00%, avg=63170.97, stdev=10308.69
    lat (msec) : 10=94.68%, 20=4.79%, 50=0.54%
  cpu          : usr=0.14%, sys=2.00%, ctx=2059, majf=0, minf=25
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=2048/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K: (groupid=4, jobs=1): err= 0: pid=15445: Wed Jun 18 00:41:26 2014
  read : io=735572KB, bw=12259KB/s, iops=3064, runt= 60001msec
    slat (usec): min=8, max=117, avg= 9.12, stdev= 0.66
    clat (usec): min=56, max=23133, avg=312.70, stdev=370.10
     lat (usec): min=147, max=23143, avg=322.94, stdev=370.08
    clat percentiles (usec):
     |  1.00th=[  145],  5.00th=[  147], 10.00th=[  151], 20.00th=[  155],
     | 30.00th=[  163], 40.00th=[  175], 50.00th=[  191], 60.00th=[  209],
     | 70.00th=[  229], 80.00th=[  266], 90.00th=[  652], 95.00th=[ 1240],
     | 99.00th=[ 1816], 99.50th=[ 1976], 99.90th=[ 2192], 99.95th=[ 2544],
     | 99.99th=[ 7136]
    bw (KB  /s): min=11456, max=14616, per=100.00%, avg=12265.87, stdev=245.67
    lat (usec) : 100=0.01%, 250=77.24%, 500=11.19%, 750=2.31%, 1000=2.11%
    lat (msec) : 2=6.75%, 4=0.38%, 10=0.03%, 20=0.01%, 50=0.01%
  cpu          : usr=0.37%, sys=4.74%, ctx=183901, majf=0, minf=27
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=183893/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-4K: (groupid=5, jobs=1): err= 0: pid=15446: Wed Jun 18 00:41:26 2014
  write: io=421456KB, bw=7024.2KB/s, iops=1756, runt= 60001msec
    slat (usec): min=10, max=58, avg=11.80, stdev= 1.19
    clat (usec): min=378, max=20110, avg=553.04, stdev=242.98
     lat (usec): min=391, max=20122, avg=565.97, stdev=243.01
    clat percentiles (usec):
     |  1.00th=[  406],  5.00th=[  422], 10.00th=[  438], 20.00th=[  466],
     | 30.00th=[  490], 40.00th=[  516], 50.00th=[  532], 60.00th=[  548],
     | 70.00th=[  564], 80.00th=[  588], 90.00th=[  636], 95.00th=[  708],
     | 99.00th=[ 1160], 99.50th=[ 1544], 99.90th=[ 3216], 99.95th=[ 4896],
     | 99.99th=[ 9920]
    bw (KB  /s): min= 6376, max= 7768, per=100.00%, avg=7033.66, stdev=272.52
    lat (usec) : 500=32.92%, 750=63.22%, 1000=2.31%
    lat (msec) : 2=1.30%, 4=0.17%, 10=0.06%, 20=0.01%, 50=0.01%
  cpu          : usr=0.59%, sys=2.90%, ctx=105369, majf=0, minf=26
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=105364/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K-QD32: (groupid=6, jobs=1): err= 0: pid=15447: Wed Jun 18 00:41:26 2014
  read : io=735688KB, bw=12259KB/s, iops=3064, runt= 60010msec
    slat (usec): min=6, max=119, avg= 9.34, stdev= 2.73
    clat (usec): min=673, max=45777, avg=10426.62, stdev=1246.84
     lat (usec): min=723, max=45786, avg=10437.16, stdev=1246.46
    clat percentiles (usec):
     |  1.00th=[ 8896],  5.00th=[ 9792], 10.00th=[ 9920], 20.00th=[10048],
     | 30.00th=[10048], 40.00th=[10048], 50.00th=[10176], 60.00th=[10816],
     | 70.00th=[10944], 80.00th=[10944], 90.00th=[11072], 95.00th=[11328],
     | 99.00th=[11968], 99.50th=[12224], 99.90th=[24448], 99.95th=[35584],
     | 99.99th=[44288]
    bw (KB  /s): min=12191, max=14624, per=100.00%, avg=12265.04, stdev=218.64
    lat (usec) : 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.43%, 4=0.09%, 10=18.98%, 20=80.35%, 50=0.14%
  cpu          : usr=1.53%, sys=3.31%, ctx=113904, majf=0, minf=59
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=183922/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32
Rand-Write-4K-QD32: (groupid=7, jobs=1): err= 0: pid=15448: Wed Jun 18 00:41:26 2014
  write: io=735620KB, bw=12258KB/s, iops=3064, runt= 60011msec
    slat (usec): min=7, max=7941, avg=12.18, stdev=52.03
    clat (usec): min=713, max=29926, avg=10424.98, stdev=846.15
     lat (usec): min=770, max=29938, avg=10438.31, stdev=844.56
    clat percentiles (usec):
     |  1.00th=[ 8640],  5.00th=[ 9792], 10.00th=[ 9920], 20.00th=[10048],
     | 30.00th=[10048], 40.00th=[10176], 50.00th=[10176], 60.00th=[10688],
     | 70.00th=[10816], 80.00th=[10944], 90.00th=[11072], 95.00th=[11584],
     | 99.00th=[11968], 99.50th=[12992], 99.90th=[16512], 99.95th=[18304],
     | 99.99th=[24704]
    bw (KB  /s): min=12102, max=14608, per=100.00%, avg=12265.32, stdev=218.18
    lat (usec) : 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.03%, 4=0.28%, 10=14.08%, 20=85.58%, 50=0.03%
  cpu          : usr=0.33%, sys=5.52%, ctx=149457, majf=0, minf=25
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=183905/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=1024.0MB, aggrb=68624KB/s, minb=68624KB/s, maxb=68624KB/s, mint=15280msec, maxt=15280msec

Run status group 1 (all jobs):
  WRITE: io=1024.0MB, aggrb=63121KB/s, minb=63121KB/s, maxb=63121KB/s, mint=16612msec, maxt=16612msec

Run status group 2 (all jobs):
   READ: io=1024.0MB, aggrb=68817KB/s, minb=68817KB/s, maxb=68817KB/s, mint=15237msec, maxt=15237msec

Run status group 3 (all jobs):
  WRITE: io=1024.0MB, aggrb=63110KB/s, minb=63110KB/s, maxb=63110KB/s, mint=16615msec, maxt=16615msec

Run status group 4 (all jobs):
   READ: io=735572KB, aggrb=12259KB/s, minb=12259KB/s, maxb=12259KB/s, mint=60001msec, maxt=60001msec

Run status group 5 (all jobs):
  WRITE: io=421456KB, aggrb=7024KB/s, minb=7024KB/s, maxb=7024KB/s, mint=60001msec, maxt=60001msec

Run status group 6 (all jobs):
   READ: io=735688KB, aggrb=12259KB/s, minb=12259KB/s, maxb=12259KB/s, mint=60010msec, maxt=60010msec

Run status group 7 (all jobs):
  WRITE: io=735620KB, aggrb=12258KB/s, minb=12258KB/s, maxb=12258KB/s, mint=60011msec, maxt=60011msec

Disk stats (read/write):
  xvda1: ios=384199/305340, merge=0/31, ticks=2084948/2097044, in_queue=4182204, util=98.79%

3回目

date;fio crystaldiskmark.fio

Wed Jun 18 00:43:07 UTC 2014
Seq-Read: (g=0): rw=read, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Seq-Write: (g=1): rw=write, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Rand-Read-512K: (g=2): rw=randread, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Write-512K: (g=3): rw=randwrite, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Read-4K: (g=4): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Write-4K: (g=5): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Read-4K-QD32: (g=6): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
Rand-Write-4K-QD32: (g=7): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1.7
Starting 8 processes
Jobs: 1 (f=1): [_______w] [92.2% done] [0KB/12252KB/0KB /s] [0/3063/0 iops] [eta 00m:26s]
Seq-Read: (groupid=0, jobs=1): err= 0: pid=15453: Wed Jun 18 00:48:13 2014
  read : io=1024.0MB, bw=68521KB/s, iops=66, runt= 15303msec
    slat (usec): min=58, max=603, avg=66.17, stdev=21.14
    clat (msec): min=1, max=136, avg=14.87, stdev=25.04
     lat (msec): min=1, max=136, avg=14.94, stdev=25.04
    clat percentiles (usec):
     |  1.00th=[ 1576],  5.00th=[ 1672], 10.00th=[ 1784], 20.00th=[ 2160],
     | 30.00th=[ 2704], 40.00th=[ 3600], 50.00th=[ 4960], 60.00th=[ 7008],
     | 70.00th=[ 7904], 80.00th=[ 9792], 90.00th=[58624], 95.00th=[69120],
     | 99.00th=[102912], 99.50th=[107008], 99.90th=[136192], 99.95th=[136192],
     | 99.99th=[136192]
    bw (KB  /s): min=50478, max=159425, per=100.00%, avg=69447.29, stdev=26928.36
    lat (msec) : 2=16.02%, 4=28.03%, 10=36.23%, 20=3.91%, 50=4.59%
    lat (msec) : 100=9.28%, 250=1.95%
  cpu          : usr=0.18%, sys=0.18%, ctx=1025, majf=0, minf=285
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1024/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Seq-Write: (groupid=1, jobs=1): err= 0: pid=15454: Wed Jun 18 00:48:13 2014
  write: io=1024.0MB, bw=63137KB/s, iops=61, runt= 16608msec
    slat (usec): min=257, max=459, avg=314.61, stdev=40.80
    clat (msec): min=8, max=64, avg=15.90, stdev= 3.52
     lat (msec): min=8, max=64, avg=16.21, stdev= 3.53
    clat percentiles (usec):
     |  1.00th=[ 8640],  5.00th=[ 8896], 10.00th=[ 9408], 20.00th=[12992],
     | 30.00th=[17024], 40.00th=[17024], 50.00th=[17280], 60.00th=[17280],
     | 70.00th=[17280], 80.00th=[17280], 90.00th=[17536], 95.00th=[17792],
     | 99.00th=[23168], 99.50th=[25216], 99.90th=[26752], 99.95th=[64256],
     | 99.99th=[64256]
    bw (KB  /s): min=56776, max=102601, per=100.00%, avg=63315.50, stdev=12483.04
    lat (msec) : 10=11.33%, 20=86.33%, 50=2.25%, 100=0.10%
  cpu          : usr=0.05%, sys=1.98%, ctx=1033, majf=0, minf=29
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=1024/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-512K: (groupid=2, jobs=1): err= 0: pid=15455: Wed Jun 18 00:48:13 2014
  read : io=1024.0MB, bw=68687KB/s, iops=134, runt= 15266msec
    slat (usec): min=35, max=334, avg=39.48, stdev=11.73
    clat (usec): min=859, max=136644, avg=7408.55, stdev=18822.23
     lat (usec): min=896, max=136682, avg=7449.22, stdev=18821.75
    clat percentiles (usec):
     |  1.00th=[  884],  5.00th=[  924], 10.00th=[  972], 20.00th=[ 1080],
     | 30.00th=[ 1224], 40.00th=[ 1448], 50.00th=[ 1848], 60.00th=[ 2672],
     | 70.00th=[ 3824], 80.00th=[ 3984], 90.00th=[ 8256], 95.00th=[45312],
     | 99.00th=[95744], 99.50th=[99840], 99.90th=[132096], 99.95th=[136192],
     | 99.99th=[136192]
    bw (KB  /s): min=49637, max=159107, per=100.00%, avg=69287.79, stdev=26756.97
    lat (usec) : 1000=12.89%
    lat (msec) : 2=39.65%, 4=28.37%, 10=10.16%, 20=1.37%, 50=3.08%
    lat (msec) : 100=4.15%, 250=0.34%
  cpu          : usr=0.00%, sys=0.39%, ctx=2049, majf=0, minf=155
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2048/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-512K: (groupid=3, jobs=1): err= 0: pid=15456: Wed Jun 18 00:48:13 2014
  write: io=1024.0MB, bw=63099KB/s, iops=123, runt= 16618msec
    slat (usec): min=133, max=277, avg=165.33, stdev=25.47
    clat (msec): min=4, max=56, avg= 7.94, stdev= 3.30
     lat (msec): min=5, max=56, avg= 8.11, stdev= 3.30
    clat percentiles (usec):
     |  1.00th=[ 5024],  5.00th=[ 5088], 10.00th=[ 5152], 20.00th=[ 5408],
     | 30.00th=[ 6560], 40.00th=[ 8384], 50.00th=[ 8512], 60.00th=[ 8640],
     | 70.00th=[ 8640], 80.00th=[ 8768], 90.00th=[ 8896], 95.00th=[ 9920],
     | 99.00th=[16768], 99.50th=[36096], 99.90th=[45824], 99.95th=[46848],
     | 99.99th=[56576]
    bw (KB  /s): min=54272, max=94664, per=100.00%, avg=63128.70, stdev=9632.46
    lat (msec) : 10=95.31%, 20=4.00%, 50=0.63%, 100=0.05%
  cpu          : usr=0.24%, sys=1.95%, ctx=2060, majf=0, minf=26
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=2048/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K: (groupid=4, jobs=1): err= 0: pid=15457: Wed Jun 18 00:48:13 2014
  read : io=735572KB, bw=12259KB/s, iops=3064, runt= 60001msec
    slat (usec): min=8, max=44, avg= 9.09, stdev= 0.56
    clat (usec): min=136, max=17854, avg=312.65, stdev=373.00
     lat (usec): min=146, max=17864, avg=322.86, stdev=372.98
    clat percentiles (usec):
     |  1.00th=[  143],  5.00th=[  147], 10.00th=[  149], 20.00th=[  155],
     | 30.00th=[  161], 40.00th=[  173], 50.00th=[  189], 60.00th=[  207],
     | 70.00th=[  225], 80.00th=[  249], 90.00th=[  684], 95.00th=[ 1272],
     | 99.00th=[ 1832], 99.50th=[ 1976], 99.90th=[ 2160], 99.95th=[ 2384],
     | 99.99th=[ 6368]
    bw (KB  /s): min=12184, max=14608, per=100.00%, avg=12266.35, stdev=217.32
    lat (usec) : 250=80.01%, 500=8.30%, 750=2.02%, 1000=2.07%
    lat (msec) : 2=7.20%, 4=0.37%, 10=0.02%, 20=0.01%
  cpu          : usr=0.85%, sys=4.23%, ctx=183897, majf=0, minf=28
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=183893/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-4K: (groupid=5, jobs=1): err= 0: pid=15458: Wed Jun 18 00:48:13 2014
  write: io=425160KB, bw=7085.1KB/s, iops=1771, runt= 60001msec
    slat (usec): min=10, max=121, avg=11.91, stdev= 1.26
    clat (usec): min=373, max=21976, avg=547.95, stdev=206.48
     lat (usec): min=385, max=21989, avg=560.99, stdev=206.51
    clat percentiles (usec):
     |  1.00th=[  406],  5.00th=[  426], 10.00th=[  442], 20.00th=[  474],
     | 30.00th=[  510], 40.00th=[  524], 50.00th=[  532], 60.00th=[  548],
     | 70.00th=[  556], 80.00th=[  580], 90.00th=[  620], 95.00th=[  668],
     | 99.00th=[  988], 99.50th=[ 1304], 99.90th=[ 2608], 99.95th=[ 4128],
     | 99.99th=[ 7840]
    bw (KB  /s): min= 6328, max= 7592, per=100.00%, avg=7089.56, stdev=260.41
    lat (usec) : 500=27.30%, 750=70.08%, 1000=1.64%
    lat (msec) : 2=0.80%, 4=0.12%, 10=0.05%, 20=0.01%, 50=0.01%
  cpu          : usr=0.88%, sys=2.63%, ctx=106297, majf=0, minf=28
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=106290/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K-QD32: (groupid=6, jobs=1): err= 0: pid=15459: Wed Jun 18 00:48:13 2014
  read : io=735704KB, bw=12260KB/s, iops=3064, runt= 60010msec
    slat (usec): min=6, max=121, avg= 9.40, stdev= 3.03
    clat (usec): min=548, max=49814, avg=10426.12, stdev=1760.22
     lat (usec): min=601, max=49825, avg=10436.72, stdev=1759.91
    clat percentiles (usec):
     |  1.00th=[ 5536],  5.00th=[ 9792], 10.00th=[ 9920], 20.00th=[10048],
     | 30.00th=[10048], 40.00th=[10048], 50.00th=[10176], 60.00th=[10816],
     | 70.00th=[10944], 80.00th=[10944], 90.00th=[11072], 95.00th=[11456],
     | 99.00th=[11968], 99.50th=[12864], 99.90th=[41216], 99.95th=[44288],
     | 99.99th=[47872]
    bw (KB  /s): min=12136, max=14616, per=100.00%, avg=12266.10, stdev=218.15
    lat (usec) : 750=0.01%, 1000=0.04%
    lat (msec) : 2=0.75%, 4=0.11%, 10=19.80%, 20=79.08%, 50=0.23%
  cpu          : usr=0.95%, sys=3.77%, ctx=107956, majf=0, minf=60
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=183926/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32
Rand-Write-4K-QD32: (groupid=7, jobs=1): err= 0: pid=15460: Wed Jun 18 00:48:13 2014
  write: io=735620KB, bw=12258KB/s, iops=3064, runt= 60011msec
    slat (usec): min=7, max=7071, avg=11.96, stdev=49.19
    clat (msec): min=1, max=48, avg=10.43, stdev= 1.05
     lat (msec): min=1, max=48, avg=10.44, stdev= 1.05
    clat percentiles (usec):
     |  1.00th=[ 8096],  5.00th=[ 9792], 10.00th=[ 9920], 20.00th=[10048],
     | 30.00th=[10048], 40.00th=[10176], 50.00th=[10176], 60.00th=[10688],
     | 70.00th=[10816], 80.00th=[10944], 90.00th=[11072], 95.00th=[11584],
     | 99.00th=[12096], 99.50th=[13632], 99.90th=[18304], 99.95th=[21376],
     | 99.99th=[44288]
    bw (KB  /s): min=12103, max=14584, per=100.00%, avg=12264.90, stdev=216.03
    lat (msec) : 2=0.03%, 4=0.42%, 10=14.72%, 20=84.76%, 50=0.07%
  cpu          : usr=0.76%, sys=5.07%, ctx=148555, majf=0, minf=26
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=183905/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=1024.0MB, aggrb=68520KB/s, minb=68520KB/s, maxb=68520KB/s, mint=15303msec, maxt=15303msec

Run status group 1 (all jobs):
  WRITE: io=1024.0MB, aggrb=63136KB/s, minb=63136KB/s, maxb=63136KB/s, mint=16608msec, maxt=16608msec

Run status group 2 (all jobs):
   READ: io=1024.0MB, aggrb=68687KB/s, minb=68687KB/s, maxb=68687KB/s, mint=15266msec, maxt=15266msec

Run status group 3 (all jobs):
  WRITE: io=1024.0MB, aggrb=63098KB/s, minb=63098KB/s, maxb=63098KB/s, mint=16618msec, maxt=16618msec

Run status group 4 (all jobs):
   READ: io=735572KB, aggrb=12259KB/s, minb=12259KB/s, maxb=12259KB/s, mint=60001msec, maxt=60001msec

Run status group 5 (all jobs):
  WRITE: io=425160KB, aggrb=7085KB/s, minb=7085KB/s, maxb=7085KB/s, mint=60001msec, maxt=60001msec

Run status group 6 (all jobs):
   READ: io=735704KB, aggrb=12259KB/s, minb=12259KB/s, maxb=12259KB/s, mint=60010msec, maxt=60010msec

Run status group 7 (all jobs):
  WRITE: io=735620KB, aggrb=12258KB/s, minb=12258KB/s, maxb=12258KB/s, mint=60011msec, maxt=60011msec

Disk stats (read/write):
  xvda1: ios=384203/306268, merge=0/31, ticks=2088504/2098004, in_queue=4186708, util=98.83%  

4回目

date;fio crystaldiskmark.fio

Wed Jun 18 00:50:27 UTC 2014
Seq-Read: (g=0): rw=read, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Seq-Write: (g=1): rw=write, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Rand-Read-512K: (g=2): rw=randread, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Write-512K: (g=3): rw=randwrite, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Read-4K: (g=4): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Write-4K: (g=5): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Read-4K-QD32: (g=6): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
Rand-Write-4K-QD32: (g=7): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1.7
Starting 8 processes
Jobs: 1 (f=1): [_______w] [92.2% done] [0KB/12235KB/0KB /s] [0/3058/0 iops] [eta 00m:26s]
Seq-Read: (groupid=0, jobs=1): err= 0: pid=15465: Wed Jun 18 00:55:33 2014
  read : io=1024.0MB, bw=68633KB/s, iops=67, runt= 15278msec
    slat (usec): min=56, max=594, avg=65.11, stdev=22.55
    clat (msec): min=1, max=140, avg=14.85, stdev=24.88
     lat (msec): min=1, max=140, avg=14.91, stdev=24.88
    clat percentiles (usec):
     |  1.00th=[ 1976],  5.00th=[ 2128], 10.00th=[ 2224], 20.00th=[ 2416],
     | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3984], 60.00th=[ 6944],
     | 70.00th=[ 7904], 80.00th=[ 9280], 90.00th=[47360], 95.00th=[69120],
     | 99.00th=[103936], 99.50th=[104960], 99.90th=[134144], 99.95th=[140288],
     | 99.99th=[140288]
    bw (KB  /s): min=49053, max=159107, per=100.00%, avg=69279.96, stdev=27382.78
    lat (msec) : 2=1.46%, 4=48.54%, 10=30.66%, 20=2.25%, 50=7.91%
    lat (msec) : 100=6.54%, 250=2.64%
  cpu          : usr=0.00%, sys=0.39%, ctx=1025, majf=0, minf=284
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1024/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Seq-Write: (groupid=1, jobs=1): err= 0: pid=15466: Wed Jun 18 00:55:33 2014
  write: io=1024.0MB, bw=63144KB/s, iops=61, runt= 16606msec
    slat (usec): min=146, max=430, avg=221.44, stdev=67.46
    clat (msec): min=8, max=24, avg=15.99, stdev= 3.08
     lat (msec): min=8, max=24, avg=16.21, stdev= 3.10
    clat percentiles (usec):
     |  1.00th=[ 8768],  5.00th=[ 8896], 10.00th=[ 9280], 20.00th=[15808],
     | 30.00th=[17280], 40.00th=[17280], 50.00th=[17280], 60.00th=[17280],
     | 70.00th=[17280], 80.00th=[17536], 90.00th=[17536], 95.00th=[17792],
     | 99.00th=[20096], 99.50th=[21888], 99.90th=[23424], 99.95th=[24704],
     | 99.99th=[24704]
    bw (KB  /s): min=58227, max=104818, per=100.00%, avg=63316.81, stdev=13873.16
    lat (msec) : 10=13.57%, 20=85.35%, 50=1.07%
  cpu          : usr=0.00%, sys=1.45%, ctx=1031, majf=0, minf=28
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=1024/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-512K: (groupid=2, jobs=1): err= 0: pid=15467: Wed Jun 18 00:55:33 2014
  read : io=1024.0MB, bw=68624KB/s, iops=134, runt= 15280msec
    slat (usec): min=34, max=333, avg=39.01, stdev=12.61
    clat (msec): min=1, max=130, avg= 7.42, stdev=18.36
     lat (msec): min=1, max=130, avg= 7.46, stdev=18.36
    clat percentiles (usec):
     |  1.00th=[ 1096],  5.00th=[ 1160], 10.00th=[ 1208], 20.00th=[ 1304],
     | 30.00th=[ 1416], 40.00th=[ 1576], 50.00th=[ 1864], 60.00th=[ 2544],
     | 70.00th=[ 3792], 80.00th=[ 3984], 90.00th=[ 7456], 95.00th=[46336],
     | 99.00th=[96768], 99.50th=[98816], 99.90th=[99840], 99.95th=[100864],
     | 99.99th=[130560]
    bw (KB  /s): min=50390, max=159107, per=100.00%, avg=68858.07, stdev=26657.81
    lat (msec) : 2=52.39%, 4=29.59%, 10=9.77%, 20=0.49%, 50=2.88%
    lat (msec) : 100=4.69%, 250=0.20%
  cpu          : usr=0.00%, sys=0.47%, ctx=2049, majf=0, minf=154
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2048/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-512K: (groupid=3, jobs=1): err= 0: pid=15468: Wed Jun 18 00:55:33 2014
  write: io=1024.0MB, bw=63156KB/s, iops=123, runt= 16603msec
    slat (usec): min=80, max=233, avg=93.48, stdev=22.99
    clat (msec): min=4, max=45, avg= 8.01, stdev= 2.09
     lat (msec): min=4, max=45, avg= 8.10, stdev= 2.10
    clat percentiles (usec):
     |  1.00th=[ 4960],  5.00th=[ 5024], 10.00th=[ 5088], 20.00th=[ 5600],
     | 30.00th=[ 8384], 40.00th=[ 8640], 50.00th=[ 8640], 60.00th=[ 8640],
     | 70.00th=[ 8768], 80.00th=[ 8768], 90.00th=[ 8896], 95.00th=[ 9152],
     | 99.00th=[12480], 99.50th=[14784], 99.90th=[36096], 99.95th=[40192],
     | 99.99th=[45312]
    bw (KB  /s): min=53841, max=97280, per=100.00%, avg=63234.76, stdev=12031.01
    lat (msec) : 10=97.22%, 20=2.54%, 50=0.24%
  cpu          : usr=0.34%, sys=0.94%, ctx=2051, majf=0, minf=25
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=2048/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K: (groupid=4, jobs=1): err= 0: pid=15469: Wed Jun 18 00:55:33 2014
  read : io=735576KB, bw=12259KB/s, iops=3064, runt= 60001msec
    slat (usec): min=7, max=124, avg= 8.91, stdev= 0.66
    clat (usec): min=44, max=21192, avg=312.89, stdev=371.87
     lat (usec): min=134, max=21202, avg=322.90, stdev=371.85
    clat percentiles (usec):
     |  1.00th=[  131],  5.00th=[  135], 10.00th=[  139], 20.00th=[  147],
     | 30.00th=[  159], 40.00th=[  173], 50.00th=[  193], 60.00th=[  213],
     | 70.00th=[  231], 80.00th=[  266], 90.00th=[  700], 95.00th=[ 1224],
     | 99.00th=[ 1800], 99.50th=[ 1944], 99.90th=[ 2192], 99.95th=[ 2448],
     | 99.99th=[ 7840]
    bw (KB  /s): min=11456, max=14592, per=100.00%, avg=12267.15, stdev=286.30
    lat (usec) : 50=0.01%, 250=77.52%, 500=9.88%, 750=2.92%, 1000=2.48%
    lat (msec) : 2=6.84%, 4=0.33%, 10=0.02%, 20=0.01%, 50=0.01%
  cpu          : usr=0.73%, sys=4.39%, ctx=183899, majf=0, minf=27
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=183894/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-4K: (groupid=5, jobs=1): err= 0: pid=15470: Wed Jun 18 00:55:33 2014
  write: io=442880KB, bw=7381.3KB/s, iops=1845, runt= 60001msec
    slat (usec): min=9, max=125, avg=11.23, stdev= 1.30
    clat (usec): min=358, max=16890, avg=525.98, stdev=218.37
     lat (usec): min=370, max=16902, avg=538.32, stdev=218.39
    clat percentiles (usec):
     |  1.00th=[  390],  5.00th=[  410], 10.00th=[  426], 20.00th=[  458],
     | 30.00th=[  486], 40.00th=[  506], 50.00th=[  516], 60.00th=[  524],
     | 70.00th=[  532], 80.00th=[  556], 90.00th=[  604], 95.00th=[  644],
     | 99.00th=[  900], 99.50th=[ 1144], 99.90th=[ 2736], 99.95th=[ 5152],
     | 99.99th=[10048]
    bw (KB  /s): min= 6720, max= 7912, per=100.00%, avg=7384.22, stdev=214.49
    lat (usec) : 500=35.20%, 750=62.79%, 1000=1.31%
    lat (msec) : 2=0.55%, 4=0.08%, 10=0.06%, 20=0.01%
  cpu          : usr=1.03%, sys=2.57%, ctx=110725, majf=0, minf=26
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=110720/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K-QD32: (groupid=6, jobs=1): err= 0: pid=15471: Wed Jun 18 00:55:33 2014
  read : io=735692KB, bw=12259KB/s, iops=3064, runt= 60011msec
    slat (usec): min=6, max=55, avg= 8.47, stdev= 1.97
    clat (usec): min=489, max=49563, avg=10427.82, stdev=1315.39
     lat (usec): min=502, max=49574, avg=10437.41, stdev=1315.10
    clat percentiles (usec):
     |  1.00th=[ 8896],  5.00th=[ 9792], 10.00th=[ 9920], 20.00th=[10048],
     | 30.00th=[10048], 40.00th=[10048], 50.00th=[10176], 60.00th=[10816],
     | 70.00th=[10944], 80.00th=[10944], 90.00th=[11072], 95.00th=[11712],
     | 99.00th=[11968], 99.50th=[12096], 99.90th=[19328], 99.95th=[40704],
     | 99.99th=[46848]
    bw (KB  /s): min=12223, max=14616, per=100.00%, avg=12266.43, stdev=217.50
    lat (usec) : 500=0.01%, 750=0.01%, 1000=0.11%
    lat (msec) : 2=0.32%, 4=0.06%, 10=21.05%, 20=78.36%, 50=0.09%
  cpu          : usr=0.00%, sys=4.69%, ctx=95582, majf=0, minf=59
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=183923/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32
Rand-Write-4K-QD32: (groupid=7, jobs=1): err= 0: pid=15472: Wed Jun 18 00:55:33 2014
  write: io=735628KB, bw=12258KB/s, iops=3064, runt= 60011msec
    slat (usec): min=7, max=8882, avg=11.60, stdev=51.11
    clat (usec): min=789, max=26396, avg=10425.31, stdev=899.60
     lat (usec): min=826, max=26407, avg=10438.05, stdev=898.12
    clat percentiles (usec):
     |  1.00th=[ 8256],  5.00th=[ 9792], 10.00th=[ 9920], 20.00th=[10048],
     | 30.00th=[10048], 40.00th=[10176], 50.00th=[10176], 60.00th=[10688],
     | 70.00th=[10816], 80.00th=[10944], 90.00th=[11072], 95.00th=[11584],
     | 99.00th=[12224], 99.50th=[13632], 99.90th=[17536], 99.95th=[19328],
     | 99.99th=[22656]
    bw (KB  /s): min=12048, max=14594, per=100.00%, avg=12264.08, stdev=219.84
    lat (usec) : 1000=0.01%
    lat (msec) : 2=0.04%, 4=0.30%, 10=14.26%, 20=85.36%, 50=0.04%
  cpu          : usr=1.92%, sys=3.83%, ctx=145811, majf=0, minf=25
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=183907/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=1024.0MB, aggrb=68633KB/s, minb=68633KB/s, maxb=68633KB/s, mint=15278msec, maxt=15278msec

Run status group 1 (all jobs):
  WRITE: io=1024.0MB, aggrb=63144KB/s, minb=63144KB/s, maxb=63144KB/s, mint=16606msec, maxt=16606msec

Run status group 2 (all jobs):
   READ: io=1024.0MB, aggrb=68624KB/s, minb=68624KB/s, maxb=68624KB/s, mint=15280msec, maxt=15280msec

Run status group 3 (all jobs):
  WRITE: io=1024.0MB, aggrb=63155KB/s, minb=63155KB/s, maxb=63155KB/s, mint=16603msec, maxt=16603msec

Run status group 4 (all jobs):
   READ: io=735576KB, aggrb=12259KB/s, minb=12259KB/s, maxb=12259KB/s, mint=60001msec, maxt=60001msec

Run status group 5 (all jobs):
  WRITE: io=442880KB, aggrb=7381KB/s, minb=7381KB/s, maxb=7381KB/s, mint=60001msec, maxt=60001msec

Run status group 6 (all jobs):
   READ: io=735692KB, aggrb=12259KB/s, minb=12259KB/s, maxb=12259KB/s, mint=60011msec, maxt=60011msec

Run status group 7 (all jobs):
  WRITE: io=735628KB, aggrb=12258KB/s, minb=12258KB/s, maxb=12258KB/s, mint=60011msec, maxt=60011msec

Disk stats (read/write):
  xvda1: ios=384201/310704, merge=0/31, ticks=2086540/2095084, in_queue=4181976, util=98.76%

5回目

Wed Jun 18 01:01:14 UTC 2014
Seq-Read: (g=0): rw=read, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Seq-Write: (g=1): rw=write, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Rand-Read-512K: (g=2): rw=randread, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Write-512K: (g=3): rw=randwrite, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Read-4K: (g=4): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Write-4K: (g=5): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Read-4K-QD32: (g=6): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
Rand-Write-4K-QD32: (g=7): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1.7
Starting 8 processes
Jobs: 1 (f=1): [_______w] [92.2% done] [0KB/12239KB/0KB /s] [0/3059/0 iops] [eta 00m:26s]
Seq-Read: (groupid=0, jobs=1): err= 0: pid=15490: Wed Jun 18 01:06:20 2014
  read : io=1024.0MB, bw=68877KB/s, iops=67, runt= 15224msec
    slat (usec): min=56, max=595, avg=74.87, stdev=25.06
    clat (msec): min=1, max=141, avg=14.78, stdev=25.02
     lat (msec): min=1, max=141, avg=14.86, stdev=25.02
    clat percentiles (msec):
     |  1.00th=[    3],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    4], 50.00th=[    5], 60.00th=[    7],
     | 70.00th=[    8], 80.00th=[    9], 90.00th=[   47], 95.00th=[   94],
     | 99.00th=[  104], 99.50th=[  106], 99.90th=[  116], 99.95th=[  141],
     | 99.99th=[  141]
    bw (KB  /s): min=49395, max=159107, per=100.00%, avg=69800.00, stdev=26878.07
    lat (msec) : 2=0.78%, 4=45.70%, 10=34.47%, 20=2.73%, 50=7.32%
    lat (msec) : 100=6.93%, 250=2.05%
  cpu          : usr=0.00%, sys=0.47%, ctx=1025, majf=0, minf=284
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1024/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Seq-Write: (groupid=1, jobs=1): err= 0: pid=15491: Wed Jun 18 01:06:20 2014
  write: io=1024.0MB, bw=63148KB/s, iops=61, runt= 16605msec
    slat (usec): min=146, max=429, avg=271.91, stdev=72.60
    clat (msec): min=8, max=40, avg=15.93, stdev= 3.32
     lat (msec): min=8, max=40, avg=16.21, stdev= 3.34
    clat percentiles (usec):
     |  1.00th=[ 8768],  5.00th=[ 8768], 10.00th=[ 8896], 20.00th=[15936],
     | 30.00th=[17024], 40.00th=[17280], 50.00th=[17280], 60.00th=[17280],
     | 70.00th=[17280], 80.00th=[17536], 90.00th=[17536], 95.00th=[17792],
     | 99.00th=[19840], 99.50th=[21632], 99.90th=[38144], 99.95th=[40192],
     | 99.99th=[40192]
    bw (KB  /s): min=58227, max=108111, per=100.00%, avg=63335.81, stdev=13748.25
    lat (msec) : 10=15.14%, 20=83.98%, 50=0.88%
  cpu          : usr=0.00%, sys=1.78%, ctx=1035, majf=0, minf=28
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=1024/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-512K: (groupid=2, jobs=1): err= 0: pid=15492: Wed Jun 18 01:06:20 2014
  read : io=1024.0MB, bw=68369KB/s, iops=133, runt= 15337msec
    slat (usec): min=32, max=314, avg=37.72, stdev=11.55
    clat (msec): min=1, max=136, avg= 7.44, stdev=18.50
     lat (msec): min=1, max=136, avg= 7.48, stdev=18.50
    clat percentiles (usec):
     |  1.00th=[ 1080],  5.00th=[ 1128], 10.00th=[ 1176], 20.00th=[ 1288],
     | 30.00th=[ 1368], 40.00th=[ 1512], 50.00th=[ 1752], 60.00th=[ 2352],
     | 70.00th=[ 3760], 80.00th=[ 3984], 90.00th=[ 8256], 95.00th=[44800],
     | 99.00th=[94720], 99.50th=[97792], 99.90th=[134144], 99.95th=[134144],
     | 99.99th=[136192]
    bw (KB  /s): min=51804, max=158720, per=100.00%, avg=69654.18, stdev=27451.45
    lat (msec) : 2=55.37%, 4=26.61%, 10=9.08%, 20=0.68%, 50=3.91%
    lat (msec) : 100=4.00%, 250=0.34%
  cpu          : usr=0.00%, sys=0.47%, ctx=2049, majf=0, minf=154
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2048/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-512K: (groupid=3, jobs=1): err= 0: pid=15493: Wed Jun 18 01:06:20 2014
  write: io=1024.0MB, bw=62883KB/s, iops=122, runt= 16675msec
    slat (usec): min=80, max=232, avg=118.37, stdev=36.04
    clat (msec): min=4, max=46, avg= 8.02, stdev= 3.18
     lat (msec): min=5, max=47, avg= 8.14, stdev= 3.19
    clat percentiles (usec):
     |  1.00th=[ 5024],  5.00th=[ 5152], 10.00th=[ 5344], 20.00th=[ 6112],
     | 30.00th=[ 6880], 40.00th=[ 8256], 50.00th=[ 8512], 60.00th=[ 8640],
     | 70.00th=[ 8640], 80.00th=[ 8768], 90.00th=[ 8896], 95.00th=[ 9280],
     | 99.00th=[12992], 99.50th=[39680], 99.90th=[43776], 99.95th=[45312],
     | 99.99th=[46848]
    bw (KB  /s): min=57229, max=94476, per=100.00%, avg=62933.79, stdev=9021.41
    lat (msec) : 10=96.09%, 20=3.22%, 50=0.68%
  cpu          : usr=0.00%, sys=1.63%, ctx=2052, majf=0, minf=25
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=2048/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K: (groupid=4, jobs=1): err= 0: pid=15494: Wed Jun 18 01:06:20 2014
  read : io=735580KB, bw=12259KB/s, iops=3064, runt= 60001msec
    slat (usec): min=7, max=117, avg= 8.99, stdev= 0.65
    clat (usec): min=123, max=20865, avg=312.78, stdev=387.62
     lat (usec): min=133, max=20875, avg=322.88, stdev=387.60
    clat percentiles (usec):
     |  1.00th=[  131],  5.00th=[  135], 10.00th=[  137], 20.00th=[  143],
     | 30.00th=[  153], 40.00th=[  169], 50.00th=[  191], 60.00th=[  211],
     | 70.00th=[  225], 80.00th=[  245], 90.00th=[  804], 95.00th=[ 1256],
     | 99.00th=[ 1752], 99.50th=[ 1912], 99.90th=[ 2192], 99.95th=[ 2960],
     | 99.99th=[ 8896]
    bw (KB  /s): min=12071, max=14608, per=100.00%, avg=12267.55, stdev=217.99
    lat (usec) : 250=80.86%, 500=6.42%, 750=2.35%, 1000=2.40%
    lat (msec) : 2=7.67%, 4=0.26%, 10=0.03%, 20=0.01%, 50=0.01%
  cpu          : usr=1.41%, sys=3.75%, ctx=183906, majf=0, minf=27
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=183895/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-4K: (groupid=5, jobs=1): err= 0: pid=15495: Wed Jun 18 01:06:20 2014
  write: io=446992KB, bw=7449.8KB/s, iops=1862, runt= 60001msec
    slat (usec): min=9, max=57, avg=11.10, stdev= 1.12
    clat (usec): min=359, max=13417, avg=521.22, stdev=193.52
     lat (usec): min=371, max=13428, avg=533.43, stdev=193.54
    clat percentiles (usec):
     |  1.00th=[  390],  5.00th=[  406], 10.00th=[  422], 20.00th=[  450],
     | 30.00th=[  478], 40.00th=[  502], 50.00th=[  510], 60.00th=[  516],
     | 70.00th=[  532], 80.00th=[  556], 90.00th=[  604], 95.00th=[  644],
     | 99.00th=[  924], 99.50th=[ 1240], 99.90th=[ 2608], 99.95th=[ 4512],
     | 99.99th=[ 7392]
    bw (KB  /s): min= 6640, max= 7840, per=100.00%, avg=7452.26, stdev=262.47
    lat (usec) : 500=39.87%, 750=58.13%, 1000=1.21%
    lat (msec) : 2=0.62%, 4=0.11%, 10=0.05%, 20=0.01%
  cpu          : usr=0.00%, sys=3.59%, ctx=111753, majf=0, minf=26
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=111748/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K-QD32: (groupid=6, jobs=1): err= 0: pid=15496: Wed Jun 18 01:06:20 2014
  read : io=735704KB, bw=12259KB/s, iops=3064, runt= 60011msec
    slat (usec): min=6, max=117, avg= 8.49, stdev= 2.25
    clat (usec): min=448, max=46349, avg=10427.54, stdev=1122.83
     lat (usec): min=498, max=46358, avg=10437.13, stdev=1122.38
    clat percentiles (usec):
     |  1.00th=[ 8896],  5.00th=[ 9920], 10.00th=[ 9920], 20.00th=[10048],
     | 30.00th=[10048], 40.00th=[10048], 50.00th=[10048], 60.00th=[10816],
     | 70.00th=[10944], 80.00th=[10944], 90.00th=[11072], 95.00th=[11584],
     | 99.00th=[11968], 99.50th=[12864], 99.90th=[19584], 99.95th=[23168],
     | 99.99th=[42240]
    bw (KB  /s): min=12143, max=14608, per=100.00%, avg=12266.78, stdev=217.27
    lat (usec) : 500=0.01%, 750=0.01%, 1000=0.05%
    lat (msec) : 2=0.30%, 4=0.10%, 10=22.99%, 20=76.47%, 50=0.09%
  cpu          : usr=0.00%, sys=4.62%, ctx=81835, majf=0, minf=59
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=183926/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32
Rand-Write-4K-QD32: (groupid=7, jobs=1): err= 0: pid=15497: Wed Jun 18 01:06:20 2014
  write: io=735628KB, bw=12258KB/s, iops=3064, runt= 60011msec
    slat (usec): min=7, max=7265, avg=11.58, stdev=49.64
    clat (usec): min=1013, max=23251, avg=10425.46, stdev=863.02
     lat (usec): min=1072, max=23262, avg=10438.19, stdev=861.45
    clat percentiles (usec):
     |  1.00th=[ 8096],  5.00th=[ 9792], 10.00th=[ 9920], 20.00th=[10048],
     | 30.00th=[10048], 40.00th=[10176], 50.00th=[10176], 60.00th=[10688],
     | 70.00th=[10816], 80.00th=[10944], 90.00th=[11072], 95.00th=[11456],
     | 99.00th=[12224], 99.50th=[13760], 99.90th=[16512], 99.95th=[17792],
     | 99.99th=[20352]
    bw (KB  /s): min=12079, max=14616, per=100.00%, avg=12264.29, stdev=219.59
    lat (msec) : 2=0.02%, 4=0.27%, 10=15.01%, 20=84.69%, 50=0.01%
  cpu          : usr=1.09%, sys=4.59%, ctx=142985, majf=0, minf=25
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=183907/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=1024.0MB, aggrb=68876KB/s, minb=68876KB/s, maxb=68876KB/s, mint=15224msec, maxt=15224msec

Run status group 1 (all jobs):
  WRITE: io=1024.0MB, aggrb=63148KB/s, minb=63148KB/s, maxb=63148KB/s, mint=16605msec, maxt=16605msec

Run status group 2 (all jobs):
   READ: io=1024.0MB, aggrb=68369KB/s, minb=68369KB/s, maxb=68369KB/s, mint=15337msec, maxt=15337msec

Run status group 3 (all jobs):
  WRITE: io=1024.0MB, aggrb=62883KB/s, minb=62883KB/s, maxb=62883KB/s, mint=16675msec, maxt=16675msec

Run status group 4 (all jobs):
   READ: io=735580KB, aggrb=12259KB/s, minb=12259KB/s, maxb=12259KB/s, mint=60001msec, maxt=60001msec

Run status group 5 (all jobs):
  WRITE: io=446992KB, aggrb=7449KB/s, minb=7449KB/s, maxb=7449KB/s, mint=60001msec, maxt=60001msec

Run status group 6 (all jobs):
   READ: io=735704KB, aggrb=12259KB/s, minb=12259KB/s, maxb=12259KB/s, mint=60011msec, maxt=60011msec

Run status group 7 (all jobs):
  WRITE: io=735628KB, aggrb=12258KB/s, minb=12258KB/s, maxb=12258KB/s, mint=60011msec, maxt=60011msec

Disk stats (read/write):
  xvda1: ios=384205/311883, merge=0/31, ticks=2087108/2095768, in_queue=4183016, util=98.68%

6回目

Wed Jun 18 01:06:21 UTC 2014
Seq-Read: (g=0): rw=read, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Seq-Write: (g=1): rw=write, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Rand-Read-512K: (g=2): rw=randread, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Write-512K: (g=3): rw=randwrite, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Read-4K: (g=4): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Write-4K: (g=5): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Read-4K-QD32: (g=6): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
Rand-Write-4K-QD32: (g=7): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1.7
Starting 8 processes
Jobs: 1 (f=1): [_______w] [92.2% done] [0KB/12252KB/0KB /s] [0/3063/0 iops] [eta 00m:26s]
Seq-Read: (groupid=0, jobs=1): err= 0: pid=15501: Wed Jun 18 01:11:26 2014
  read : io=1024.0MB, bw=68759KB/s, iops=67, runt= 15250msec
    slat (usec): min=56, max=606, avg=66.37, stdev=22.44
    clat (msec): min=1, max=114, avg=14.82, stdev=24.52
     lat (msec): min=2, max=114, avg=14.89, stdev=24.52
    clat percentiles (msec):
     |  1.00th=[    3],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    4], 40.00th=[    4], 50.00th=[    5], 60.00th=[    7],
     | 70.00th=[    8], 80.00th=[    9], 90.00th=[   59], 95.00th=[   68],
     | 99.00th=[  103], 99.50th=[  105], 99.90th=[  110], 99.95th=[  115],
     | 99.99th=[  115]
    bw (KB  /s): min=52264, max=158162, per=100.00%, avg=69543.82, stdev=26689.84
    lat (msec) : 2=0.10%, 4=41.21%, 10=40.62%, 20=2.05%, 50=5.27%
    lat (msec) : 100=8.59%, 250=2.15%
  cpu          : usr=0.00%, sys=0.42%, ctx=1026, majf=0, minf=284
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1024/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Seq-Write: (groupid=1, jobs=1): err= 0: pid=15502: Wed Jun 18 01:11:26 2014
  write: io=1024.0MB, bw=63114KB/s, iops=61, runt= 16614msec
    slat (usec): min=158, max=400, avg=293.74, stdev=52.12
    clat (msec): min=8, max=57, avg=15.92, stdev= 3.56
     lat (msec): min=8, max=58, avg=16.22, stdev= 3.58
    clat percentiles (usec):
     |  1.00th=[ 8896],  5.00th=[ 9152], 10.00th=[ 9664], 20.00th=[12992],
     | 30.00th=[17024], 40.00th=[17024], 50.00th=[17280], 60.00th=[17280],
     | 70.00th=[17280], 80.00th=[17280], 90.00th=[17536], 95.00th=[18304],
     | 99.00th=[21888], 99.50th=[23168], 99.90th=[50432], 99.95th=[58112],
     | 99.99th=[58112]
    bw (KB  /s): min=57229, max=100195, per=100.00%, avg=63318.69, stdev=12525.16
    lat (msec) : 10=11.91%, 20=86.04%, 50=1.86%, 100=0.20%
  cpu          : usr=0.05%, sys=1.88%, ctx=1035, majf=0, minf=28
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=1024/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-512K: (groupid=2, jobs=1): err= 0: pid=15503: Wed Jun 18 01:11:26 2014
  read : io=1024.0MB, bw=69013KB/s, iops=134, runt= 15194msec
    slat (usec): min=33, max=308, avg=39.45, stdev=12.17
    clat (msec): min=1, max=134, avg= 7.37, stdev=17.37
     lat (msec): min=1, max=134, avg= 7.41, stdev=17.37
    clat percentiles (usec):
     |  1.00th=[ 1176],  5.00th=[ 1304], 10.00th=[ 1400], 20.00th=[ 1560],
     | 30.00th=[ 1800], 40.00th=[ 2096], 50.00th=[ 2384], 60.00th=[ 2864],
     | 70.00th=[ 3376], 80.00th=[ 4128], 90.00th=[ 6240], 95.00th=[45824],
     | 99.00th=[95744], 99.50th=[97792], 99.90th=[103936], 99.95th=[103936],
     | 99.99th=[134144]
    bw (KB  /s): min=50196, max=158720, per=100.00%, avg=69858.18, stdev=27577.74
    lat (msec) : 2=37.01%, 4=40.77%, 10=13.87%, 20=0.44%, 50=3.47%
    lat (msec) : 100=4.25%, 250=0.20%
  cpu          : usr=0.00%, sys=0.47%, ctx=2049, majf=0, minf=154
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2048/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-512K: (groupid=3, jobs=1): err= 0: pid=15504: Wed Jun 18 01:11:26 2014
  write: io=1024.0MB, bw=63065KB/s, iops=123, runt= 16627msec
    slat (usec): min=80, max=264, avg=114.95, stdev=35.39
    clat (msec): min=4, max=59, avg= 8.00, stdev= 3.44
     lat (msec): min=5, max=59, avg= 8.11, stdev= 3.45
    clat percentiles (usec):
     |  1.00th=[ 5024],  5.00th=[ 5216], 10.00th=[ 5408], 20.00th=[ 6048],
     | 30.00th=[ 6624], 40.00th=[ 7456], 50.00th=[ 8512], 60.00th=[ 8640],
     | 70.00th=[ 8640], 80.00th=[ 8768], 90.00th=[ 8896], 95.00th=[ 9792],
     | 99.00th=[15936], 99.50th=[39680], 99.90th=[44288], 99.95th=[47872],
     | 99.99th=[59648]
    bw (KB  /s): min=43856, max=82944, per=100.00%, avg=63112.21, stdev=7919.84
    lat (msec) : 10=95.51%, 20=3.61%, 50=0.83%, 100=0.05%
  cpu          : usr=0.10%, sys=1.47%, ctx=2054, majf=0, minf=25
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=2048/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K: (groupid=4, jobs=1): err= 0: pid=15505: Wed Jun 18 01:11:26 2014
  read : io=735580KB, bw=12260KB/s, iops=3064, runt= 60000msec
    slat (usec): min=7, max=118, avg= 9.06, stdev= 0.76
    clat (usec): min=114, max=15663, avg=312.69, stdev=346.97
     lat (usec): min=137, max=15675, avg=322.87, stdev=346.94
    clat percentiles (usec):
     |  1.00th=[  137],  5.00th=[  145], 10.00th=[  153], 20.00th=[  165],
     | 30.00th=[  177], 40.00th=[  193], 50.00th=[  209], 60.00th=[  225],
     | 70.00th=[  239], 80.00th=[  278], 90.00th=[  572], 95.00th=[ 1160],
     | 99.00th=[ 1736], 99.50th=[ 1912], 99.90th=[ 2320], 99.95th=[ 2992],
     | 99.99th=[ 7136]
    bw (KB  /s): min=11536, max=14602, per=100.00%, avg=12265.04, stdev=249.39
    lat (usec) : 250=74.39%, 500=14.71%, 750=2.56%, 1000=1.99%
    lat (msec) : 2=6.02%, 4=0.30%, 10=0.02%, 20=0.01%
  cpu          : usr=2.07%, sys=3.09%, ctx=183901, majf=0, minf=27
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=183895/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-4K: (groupid=5, jobs=1): err= 0: pid=15506: Wed Jun 18 01:11:26 2014
  write: io=435320KB, bw=7255.3KB/s, iops=1813, runt= 60001msec
    slat (usec): min=9, max=58, avg=11.19, stdev= 1.28
    clat (usec): min=367, max=23791, avg=535.48, stdev=213.14
     lat (usec): min=379, max=23803, avg=547.82, stdev=213.15
    clat percentiles (usec):
     |  1.00th=[  394],  5.00th=[  410], 10.00th=[  422], 20.00th=[  442],
     | 30.00th=[  466], 40.00th=[  490], 50.00th=[  516], 60.00th=[  532],
     | 70.00th=[  556], 80.00th=[  580], 90.00th=[  636], 95.00th=[  724],
     | 99.00th=[ 1048], 99.50th=[ 1336], 99.90th=[ 2768], 99.95th=[ 3888],
     | 99.99th=[ 7520]
    bw (KB  /s): min= 6568, max= 7688, per=100.00%, avg=7259.88, stdev=234.11
    lat (usec) : 500=44.24%, 750=51.39%, 1000=3.20%
    lat (msec) : 2=0.97%, 4=0.15%, 10=0.04%, 20=0.01%, 50=0.01%
  cpu          : usr=0.00%, sys=3.54%, ctx=108832, majf=0, minf=27
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=108830/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K-QD32: (groupid=6, jobs=1): err= 0: pid=15507: Wed Jun 18 01:11:26 2014
  read : io=735704KB, bw=12259KB/s, iops=3064, runt= 60011msec
    slat (usec): min=6, max=127, avg= 8.70, stdev= 2.38
    clat (usec): min=455, max=50852, avg=10427.14, stdev=1179.43
     lat (usec): min=508, max=50861, avg=10436.98, stdev=1179.03
    clat percentiles (usec):
     |  1.00th=[ 8896],  5.00th=[ 9792], 10.00th=[ 9920], 20.00th=[10048],
     | 30.00th=[10048], 40.00th=[10048], 50.00th=[10176], 60.00th=[10816],
     | 70.00th=[10944], 80.00th=[10944], 90.00th=[11072], 95.00th=[11712],
     | 99.00th=[11968], 99.50th=[12224], 99.90th=[18816], 99.95th=[35584],
     | 99.99th=[45824]
    bw (KB  /s): min=12087, max=14586, per=100.00%, avg=12267.39, stdev=215.50
    lat (usec) : 500=0.01%, 750=0.01%, 1000=0.03%
    lat (msec) : 2=0.33%, 4=0.09%, 10=21.25%, 20=78.20%, 50=0.08%
    lat (msec) : 100=0.01%
  cpu          : usr=1.17%, sys=3.51%, ctx=96555, majf=0, minf=59
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=183926/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32
Rand-Write-4K-QD32: (groupid=7, jobs=1): err= 0: pid=15508: Wed Jun 18 01:11:26 2014
  write: io=735620KB, bw=12258KB/s, iops=3064, runt= 60012msec
    slat (usec): min=7, max=7264, avg=11.88, stdev=50.97
    clat (usec): min=588, max=30569, avg=10425.27, stdev=975.59
     lat (usec): min=659, max=30578, avg=10438.30, stdev=974.10
    clat percentiles (usec):
     |  1.00th=[ 7712],  5.00th=[ 9792], 10.00th=[ 9920], 20.00th=[10048],
     | 30.00th=[10048], 40.00th=[10176], 50.00th=[10176], 60.00th=[10688],
     | 70.00th=[10816], 80.00th=[10944], 90.00th=[11072], 95.00th=[11584],
     | 99.00th=[12480], 99.50th=[14400], 99.90th=[18560], 99.95th=[20608],
     | 99.99th=[26496]
    bw (KB  /s): min=11913, max=14608, per=100.00%, avg=12265.45, stdev=223.61
    lat (usec) : 750=0.01%
    lat (msec) : 2=0.04%, 4=0.35%, 10=14.97%, 20=84.57%, 50=0.06%
  cpu          : usr=1.52%, sys=4.21%, ctx=142411, majf=0, minf=25
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=183905/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=1024.0MB, aggrb=68759KB/s, minb=68759KB/s, maxb=68759KB/s, mint=15250msec, maxt=15250msec

Run status group 1 (all jobs):
  WRITE: io=1024.0MB, aggrb=63114KB/s, minb=63114KB/s, maxb=63114KB/s, mint=16614msec, maxt=16614msec

Run status group 2 (all jobs):
   READ: io=1024.0MB, aggrb=69012KB/s, minb=69012KB/s, maxb=69012KB/s, mint=15194msec, maxt=15194msec

Run status group 3 (all jobs):
  WRITE: io=1024.0MB, aggrb=63064KB/s, minb=63064KB/s, maxb=63064KB/s, mint=16627msec, maxt=16627msec

Run status group 4 (all jobs):
   READ: io=735580KB, aggrb=12259KB/s, minb=12259KB/s, maxb=12259KB/s, mint=60000msec, maxt=60000msec

Run status group 5 (all jobs):
  WRITE: io=435320KB, aggrb=7255KB/s, minb=7255KB/s, maxb=7255KB/s, mint=60001msec, maxt=60001msec

Run status group 6 (all jobs):
   READ: io=735704KB, aggrb=12259KB/s, minb=12259KB/s, maxb=12259KB/s, mint=60011msec, maxt=60011msec

Run status group 7 (all jobs):
  WRITE: io=735620KB, aggrb=12257KB/s, minb=12257KB/s, maxb=12257KB/s, mint=60012msec, maxt=60012msec

Disk stats (read/write):
  xvda1: ios=384205/308799, merge=0/31, ticks=2084404/2098636, in_queue=4183248, util=98.89%

7回目

Wed Jun 18 01:11:27 UTC 2014
Seq-Read: (g=0): rw=read, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Seq-Write: (g=1): rw=write, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Rand-Read-512K: (g=2): rw=randread, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Write-512K: (g=3): rw=randwrite, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Read-4K: (g=4): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Write-4K: (g=5): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Read-4K-QD32: (g=6): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
Rand-Write-4K-QD32: (g=7): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1.7
Starting 8 processes
Jobs: 1 (f=1): [_______w] [92.4% done] [0KB/12252KB/0KB /s] [0/3063/0 iops] [eta 00m:25s]
Seq-Read: (groupid=0, jobs=1): err= 0: pid=15512: Wed Jun 18 01:16:32 2014
  read : io=1024.0MB, bw=69022KB/s, iops=67, runt= 15192msec
    slat (usec): min=56, max=572, avg=68.39, stdev=22.46
    clat (msec): min=2, max=118, avg=14.76, stdev=24.59
     lat (msec): min=2, max=118, avg=14.83, stdev=24.59
    clat percentiles (msec):
     |  1.00th=[    3],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    4], 40.00th=[    4], 50.00th=[    5], 60.00th=[    7],
     | 70.00th=[    8], 80.00th=[    9], 90.00th=[   56], 95.00th=[   70],
     | 99.00th=[  103], 99.50th=[  105], 99.90th=[  110], 99.95th=[  119],
     | 99.99th=[  119]
    bw (KB  /s): min=53410, max=158476, per=100.00%, avg=69558.96, stdev=27469.54
    lat (msec) : 4=45.41%, 10=36.72%, 20=1.46%, 50=6.15%, 100=8.20%
    lat (msec) : 250=2.05%
  cpu          : usr=0.00%, sys=0.42%, ctx=1026, majf=0, minf=284
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1024/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Seq-Write: (groupid=1, jobs=1): err= 0: pid=15515: Wed Jun 18 01:16:32 2014
  write: io=1024.0MB, bw=63129KB/s, iops=61, runt= 16610msec
    slat (usec): min=152, max=392, avg=278.49, stdev=56.45
    clat (msec): min=8, max=60, avg=15.93, stdev= 3.54
     lat (msec): min=8, max=60, avg=16.21, stdev= 3.55
    clat percentiles (usec):
     |  1.00th=[ 8896],  5.00th=[ 9152], 10.00th=[ 9792], 20.00th=[13504],
     | 30.00th=[17024], 40.00th=[17280], 50.00th=[17280], 60.00th=[17280],
     | 70.00th=[17280], 80.00th=[17280], 90.00th=[17536], 95.00th=[17792],
     | 99.00th=[20352], 99.50th=[22400], 99.90th=[57088], 99.95th=[60160],
     | 99.99th=[60160]
    bw (KB  /s): min=58000, max=93090, per=100.00%, avg=63344.59, stdev=11898.35
    lat (msec) : 10=10.94%, 20=87.70%, 50=1.17%, 100=0.20%
  cpu          : usr=0.05%, sys=1.76%, ctx=1029, majf=0, minf=28
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=1024/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-512K: (groupid=2, jobs=1): err= 0: pid=15516: Wed Jun 18 01:16:32 2014
  read : io=1024.0MB, bw=68985KB/s, iops=134, runt= 15200msec
    slat (usec): min=33, max=318, avg=39.95, stdev=12.96
    clat (msec): min=1, max=129, avg= 7.38, stdev=17.61
     lat (msec): min=1, max=129, avg= 7.42, stdev=17.61
    clat percentiles (usec):
     |  1.00th=[ 1112],  5.00th=[ 1192], 10.00th=[ 1272], 20.00th=[ 1448],
     | 30.00th=[ 1592], 40.00th=[ 1816], 50.00th=[ 2128], 60.00th=[ 2640],
     | 70.00th=[ 3408], 80.00th=[ 4016], 90.00th=[ 6752], 95.00th=[46336],
     | 99.00th=[95744], 99.50th=[97792], 99.90th=[100864], 99.95th=[101888],
     | 99.99th=[129536]
    bw (KB  /s): min=49840, max=158791, per=100.00%, avg=69825.00, stdev=27702.41
    lat (msec) : 2=46.48%, 4=32.86%, 10=12.01%, 20=0.49%, 50=3.66%
    lat (msec) : 100=4.35%, 250=0.15%
  cpu          : usr=0.00%, sys=0.47%, ctx=2049, majf=0, minf=154
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=2048/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-512K: (groupid=3, jobs=1): err= 0: pid=15517: Wed Jun 18 01:16:32 2014
  write: io=1024.0MB, bw=63167KB/s, iops=123, runt= 16600msec
    slat (usec): min=80, max=264, avg=105.99, stdev=33.11
    clat (msec): min=4, max=27, avg= 7.99, stdev= 1.55
     lat (msec): min=4, max=27, avg= 8.10, stdev= 1.56
    clat percentiles (usec):
     |  1.00th=[ 5024],  5.00th=[ 5088], 10.00th=[ 5088], 20.00th=[ 6112],
     | 30.00th=[ 8512], 40.00th=[ 8640], 50.00th=[ 8640], 60.00th=[ 8640],
     | 70.00th=[ 8768], 80.00th=[ 8768], 90.00th=[ 8768], 95.00th=[ 8896],
     | 99.00th=[10304], 99.50th=[11584], 99.90th=[15040], 99.95th=[16768],
     | 99.99th=[27520]
    bw (KB  /s): min=58251, max=95041, per=100.00%, avg=63252.12, stdev=12144.06
    lat (msec) : 10=98.58%, 20=1.37%, 50=0.05%
  cpu          : usr=0.12%, sys=1.33%, ctx=2055, majf=0, minf=25
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=2048/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K: (groupid=4, jobs=1): err= 0: pid=15518: Wed Jun 18 01:16:32 2014
  read : io=735568KB, bw=12259KB/s, iops=3064, runt= 60001msec
    slat (usec): min=7, max=115, avg= 8.97, stdev= 0.64
    clat (usec): min=124, max=28631, avg=312.78, stdev=375.85
     lat (usec): min=134, max=28641, avg=322.86, stdev=375.83
    clat percentiles (usec):
     |  1.00th=[  131],  5.00th=[  135], 10.00th=[  137], 20.00th=[  141],
     | 30.00th=[  149], 40.00th=[  165], 50.00th=[  187], 60.00th=[  209],
     | 70.00th=[  225], 80.00th=[  251], 90.00th=[  820], 95.00th=[ 1240],
     | 99.00th=[ 1752], 99.50th=[ 1912], 99.90th=[ 2128], 99.95th=[ 2256],
     | 99.99th=[ 6048]
    bw (KB  /s): min=12191, max=14632, per=100.00%, avg=12266.00, stdev=219.39
    lat (usec) : 250=79.97%, 500=6.65%, 750=2.90%, 1000=2.59%
    lat (msec) : 2=7.62%, 4=0.24%, 10=0.02%, 20=0.01%, 50=0.01%
  cpu          : usr=2.61%, sys=2.60%, ctx=183897, majf=0, minf=27
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=183892/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-4K: (groupid=5, jobs=1): err= 0: pid=15519: Wed Jun 18 01:16:32 2014
  write: io=443272KB, bw=7387.8KB/s, iops=1846, runt= 60001msec
    slat (usec): min=9, max=122, avg=11.09, stdev= 1.38
    clat (usec): min=334, max=21207, avg=525.68, stdev=210.77
     lat (usec): min=372, max=21219, avg=537.88, stdev=210.79
    clat percentiles (usec):
     |  1.00th=[  390],  5.00th=[  410], 10.00th=[  426], 20.00th=[  454],
     | 30.00th=[  482], 40.00th=[  502], 50.00th=[  510], 60.00th=[  516],
     | 70.00th=[  532], 80.00th=[  556], 90.00th=[  612], 95.00th=[  652],
     | 99.00th=[  948], 99.50th=[ 1240], 99.90th=[ 2704], 99.95th=[ 4640],
     | 99.99th=[ 8640]
    bw (KB  /s): min= 6736, max= 7856, per=100.00%, avg=7391.13, stdev=240.55
    lat (usec) : 500=37.78%, 750=59.92%, 1000=1.46%
    lat (msec) : 2=0.68%, 4=0.09%, 10=0.06%, 20=0.01%, 50=0.01%
  cpu          : usr=0.00%, sys=3.59%, ctx=110824, majf=0, minf=27
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=110818/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K-QD32: (groupid=6, jobs=1): err= 0: pid=15520: Wed Jun 18 01:16:32 2014
  read : io=735704KB, bw=12259KB/s, iops=3064, runt= 60011msec
    slat (usec): min=5, max=127, avg= 8.44, stdev= 1.99
    clat (usec): min=475, max=50615, avg=10427.74, stdev=1127.14
     lat (usec): min=498, max=50624, avg=10437.30, stdev=1126.81
    clat percentiles (usec):
     |  1.00th=[ 9024],  5.00th=[ 9920], 10.00th=[ 9920], 20.00th=[10048],
     | 30.00th=[10048], 40.00th=[10048], 50.00th=[10176], 60.00th=[10816],
     | 70.00th=[10944], 80.00th=[10944], 90.00th=[11072], 95.00th=[11712],
     | 99.00th=[11968], 99.50th=[12096], 99.90th=[19072], 99.95th=[28288],
     | 99.99th=[45312]
    bw (KB  /s): min=12079, max=14576, per=100.00%, avg=12267.32, stdev=214.82
    lat (usec) : 500=0.01%, 750=0.01%, 1000=0.03%
    lat (msec) : 2=0.32%, 4=0.05%, 10=20.90%, 20=78.61%, 50=0.09%
    lat (msec) : 100=0.01%
  cpu          : usr=0.74%, sys=3.94%, ctx=94329, majf=0, minf=59
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=183926/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32
Rand-Write-4K-QD32: (groupid=7, jobs=1): err= 0: pid=15521: Wed Jun 18 01:16:32 2014
  write: io=735624KB, bw=12258KB/s, iops=3064, runt= 60011msec
    slat (usec): min=7, max=8837, avg=11.69, stdev=51.84
    clat (usec): min=982, max=27197, avg=10425.30, stdev=894.78
     lat (usec): min=1009, max=27208, avg=10438.12, stdev=893.16
    clat percentiles (usec):
     |  1.00th=[ 8160],  5.00th=[ 9792], 10.00th=[ 9920], 20.00th=[10048],
     | 30.00th=[10048], 40.00th=[10176], 50.00th=[10176], 60.00th=[10688],
     | 70.00th=[10816], 80.00th=[10944], 90.00th=[10944], 95.00th=[11456],
     | 99.00th=[12224], 99.50th=[13760], 99.90th=[17536], 99.95th=[19840],
     | 99.99th=[22656]
    bw (KB  /s): min=12095, max=14608, per=100.00%, avg=12265.33, stdev=218.35
    lat (usec) : 1000=0.01%
    lat (msec) : 2=0.04%, 4=0.30%, 10=13.87%, 20=85.75%, 50=0.04%
  cpu          : usr=0.64%, sys=5.09%, ctx=144637, majf=0, minf=25
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=183906/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=1024.0MB, aggrb=69021KB/s, minb=69021KB/s, maxb=69021KB/s, mint=15192msec, maxt=15192msec

Run status group 1 (all jobs):
  WRITE: io=1024.0MB, aggrb=63129KB/s, minb=63129KB/s, maxb=63129KB/s, mint=16610msec, maxt=16610msec

Run status group 2 (all jobs):
   READ: io=1024.0MB, aggrb=68985KB/s, minb=68985KB/s, maxb=68985KB/s, mint=15200msec, maxt=15200msec

Run status group 3 (all jobs):
  WRITE: io=1024.0MB, aggrb=63167KB/s, minb=63167KB/s, maxb=63167KB/s, mint=16600msec, maxt=16600msec

Run status group 4 (all jobs):
   READ: io=735568KB, aggrb=12259KB/s, minb=12259KB/s, maxb=12259KB/s, mint=60001msec, maxt=60001msec

Run status group 5 (all jobs):
  WRITE: io=443272KB, aggrb=7387KB/s, minb=7387KB/s, maxb=7387KB/s, mint=60001msec, maxt=60001msec

Run status group 6 (all jobs):
   READ: io=735704KB, aggrb=12259KB/s, minb=12259KB/s, maxb=12259KB/s, mint=60011msec, maxt=60011msec

Run status group 7 (all jobs):
  WRITE: io=735624KB, aggrb=12258KB/s, minb=12258KB/s, maxb=12258KB/s, mint=60011msec, maxt=60011msec

Disk stats (read/write):
  xvda1: ios=384202/310649, merge=0/40, ticks=2087952/2094092, in_queue=4182236, util=98.99%

8回目

Wed Jun 18 01:16:32 UTC 2014
Seq-Read: (g=0): rw=read, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Seq-Write: (g=1): rw=write, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Rand-Read-512K: (g=2): rw=randread, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Write-512K: (g=3): rw=randwrite, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Read-4K: (g=4): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Write-4K: (g=5): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Read-4K-QD32: (g=6): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
Rand-Write-4K-QD32: (g=7): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1.7
Starting 8 processes
Jobs: 1 (f=1): [_______w] [54.0% done] [0KB/48KB/0KB /s] [0/12/0 iops] [eta 06m:15s]
Seq-Read: (groupid=0, jobs=1): err= 0: pid=15525: Wed Jun 18 01:23:54 2014
  read : io=1024.0MB, bw=68786KB/s, iops=67, runt= 15244msec
    slat (usec): min=55, max=580, avg=65.94, stdev=21.93
    clat (msec): min=1, max=150, avg=14.81, stdev=25.70
     lat (msec): min=1, max=150, avg=14.88, stdev=25.70
    clat percentiles (usec):
     |  1.00th=[ 1992],  5.00th=[ 2128], 10.00th=[ 2256], 20.00th=[ 2448],
     | 30.00th=[ 2672], 40.00th=[ 3152], 50.00th=[ 4256], 60.00th=[ 6944],
     | 70.00th=[ 7904], 80.00th=[ 8640], 90.00th=[46848], 95.00th=[93696],
     | 99.00th=[104960], 99.50th=[105984], 99.90th=[136192], 99.95th=[150528],
     | 99.99th=[150528]
    bw (KB  /s): min=51543, max=159425, per=100.00%, avg=69068.83, stdev=27223.50
    lat (msec) : 2=1.07%, 4=47.36%, 10=32.71%, 20=3.12%, 50=6.54%
    lat (msec) : 100=6.25%, 250=2.93%
  cpu          : usr=0.00%, sys=0.39%, ctx=1025, majf=0, minf=284
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1024/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Seq-Write: (groupid=1, jobs=1): err= 0: pid=15526: Wed Jun 18 01:23:54 2014
  write: io=1015.0MB, bw=17092KB/s, iops=16, runt= 60810msec
    slat (usec): min=151, max=42543, avg=486.54, stdev=2935.64
    clat (msec): min=8, max=1018, avg=59.41, stdev=200.38
     lat (msec): min=8, max=1060, avg=59.90, stdev=201.40
    clat percentiles (msec):
     |  1.00th=[    9],  5.00th=[    9], 10.00th=[   10], 20.00th=[   15],
     | 30.00th=[   18], 40.00th=[   18], 50.00th=[   18], 60.00th=[   18],
     | 70.00th=[   18], 80.00th=[   18], 90.00th=[   18], 95.00th=[   23],
     | 99.00th=[  979], 99.50th=[ 1020], 99.90th=[ 1020], 99.95th=[ 1020],
     | 99.99th=[ 1020]
    bw (KB  /s): min=  966, max=112191, per=100.00%, avg=26053.12, stdev=31921.90
    lat (msec) : 10=12.61%, 20=81.08%, 50=1.67%, 100=0.10%, 750=0.10%
    lat (msec) : 1000=3.55%, 2000=0.89%
  cpu          : usr=0.00%, sys=0.50%, ctx=1025, majf=0, minf=28
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=1015/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-512K: (groupid=2, jobs=1): err= 0: pid=15527: Wed Jun 18 01:23:54 2014
  read : io=63488KB, bw=1051.4KB/s, iops=2, runt= 60386msec
    slat (usec): min=71, max=310, avg=77.28, stdev=21.16
    clat (msec): min=202, max=529, avg=486.89, stdev=26.57
     lat (msec): min=202, max=530, avg=486.97, stdev=26.55
    clat percentiles (msec):
     |  1.00th=[  486],  5.00th=[  490], 10.00th=[  490], 20.00th=[  490],
     | 30.00th=[  490], 40.00th=[  490], 50.00th=[  490], 60.00th=[  490],
     | 70.00th=[  490], 80.00th=[  490], 90.00th=[  490], 95.00th=[  490],
     | 99.00th=[  529], 99.50th=[  529], 99.90th=[  529], 99.95th=[  529],
     | 99.99th=[  529]
    bw (KB  /s): min=  967, max= 1484, per=100.00%, avg=1052.51, stdev=57.34
    lat (msec) : 250=0.81%, 500=96.77%, 750=2.42%
  cpu          : usr=0.00%, sys=0.01%, ctx=125, majf=0, minf=154
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=124/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-512K: (groupid=3, jobs=1): err= 0: pid=15528: Wed Jun 18 01:23:54 2014
  write: io=62464KB, bw=1037.8KB/s, iops=2, runt= 60190msec
    slat (usec): min=192, max=42345, avg=1592.39, stdev=7525.66
    clat (msec): min=275, max=529, avg=491.75, stdev=24.30
     lat (msec): min=275, max=572, avg=493.34, stdev=27.44
    clat percentiles (msec):
     |  1.00th=[  486],  5.00th=[  490], 10.00th=[  490], 20.00th=[  490],
     | 30.00th=[  490], 40.00th=[  490], 50.00th=[  490], 60.00th=[  490],
     | 70.00th=[  490], 80.00th=[  490], 90.00th=[  529], 95.00th=[  529],
     | 99.00th=[  529], 99.50th=[  529], 99.90th=[  529], 99.95th=[  529],
     | 99.99th=[  529]
    bw (KB  /s): min=  896, max= 1342, per=99.63%, avg=1033.18, stdev=53.60
    lat (msec) : 500=86.89%, 750=13.11%
  cpu          : usr=0.01%, sys=0.04%, ctx=128, majf=0, minf=25
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=122/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K: (groupid=4, jobs=1): err= 0: pid=15529: Wed Jun 18 01:23:54 2014
  read : io=5772.0KB, bw=98444B/s, iops=24, runt= 60039msec
    slat (usec): min=9, max=41, avg=24.08, stdev= 6.13
    clat (usec): min=149, max=82982, avg=41571.93, stdev=2234.48
     lat (usec): min=159, max=83007, avg=41597.30, stdev=2234.58
    clat percentiles (usec):
     |  1.00th=[40704],  5.00th=[40704], 10.00th=[40704], 20.00th=[41216],
     | 30.00th=[41216], 40.00th=[41728], 50.00th=[41728], 60.00th=[41728],
     | 70.00th=[41728], 80.00th=[41728], 90.00th=[42240], 95.00th=[42240],
     | 99.00th=[42240], 99.50th=[42240], 99.90th=[42752], 99.95th=[83456],
     | 99.99th=[83456]
    bw (KB  /s): min=   88, max=  118, per=100.00%, avg=96.05, stdev= 2.21
    lat (usec) : 250=0.14%, 500=0.07%
    lat (msec) : 50=99.72%, 100=0.07%
  cpu          : usr=0.03%, sys=0.07%, ctx=1444, majf=0, minf=27
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1443/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-4K: (groupid=5, jobs=1): err= 0: pid=15530: Wed Jun 18 01:23:54 2014
  write: io=5700.0KB, bw=97218B/s, iops=23, runt= 60038msec
    slat (usec): min=10, max=42163, avg=356.71, stdev=3650.24
    clat (usec): min=419, max=84354, avg=41764.00, stdev=4572.03
     lat (usec): min=431, max=125413, avg=42122.05, stdev=7810.92
    clat percentiles (usec):
     |  1.00th=[40704],  5.00th=[40704], 10.00th=[40704], 20.00th=[41216],
     | 30.00th=[41216], 40.00th=[41728], 50.00th=[41728], 60.00th=[41728],
     | 70.00th=[41728], 80.00th=[42240], 90.00th=[42240], 95.00th=[42240],
     | 99.00th=[42752], 99.50th=[82432], 99.90th=[84480], 99.95th=[84480],
     | 99.99th=[84480]
    bw (KB  /s): min=   80, max=  141, per=100.00%, avg=94.85, stdev= 6.25
    lat (usec) : 500=0.21%, 750=0.21%
    lat (msec) : 50=98.81%, 100=0.77%
  cpu          : usr=0.00%, sys=0.12%, ctx=1437, majf=0, minf=27
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=1425/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K-QD32: (groupid=6, jobs=1): err= 0: pid=15531: Wed Jun 18 01:23:54 2014
  read : io=5900.0KB, bw=98516B/s, iops=24, runt= 61326msec
    slat (usec): min=6, max=47, avg=25.88, stdev= 5.87
    clat (usec): min=449, max=2583.7K, avg=1329539.10, stdev=175916.15
     lat (usec): min=501, max=2583.8K, avg=1329566.29, stdev=175917.78
    clat percentiles (msec):
     |  1.00th=[  367],  5.00th=[ 1336], 10.00th=[ 1336], 20.00th=[ 1336],
     | 30.00th=[ 1336], 40.00th=[ 1336], 50.00th=[ 1336], 60.00th=[ 1336],
     | 70.00th=[ 1336], 80.00th=[ 1336], 90.00th=[ 1336], 95.00th=[ 1369],
     | 99.00th=[ 2008], 99.50th=[ 2278], 99.90th=[ 2540], 99.95th=[ 2573],
     | 99.99th=[ 2573]
    bw (KB  /s): min=    6, max=  142, per=99.34%, avg=95.36, stdev= 9.49
    lat (usec) : 500=0.07%, 750=0.34%
    lat (msec) : 50=0.07%, 100=0.07%, 250=0.27%, 500=0.41%, 750=0.41%
    lat (msec) : 1000=0.41%, 2000=96.95%, >=2000=1.02%
  cpu          : usr=0.00%, sys=0.10%, ctx=1471, majf=0, minf=59
  IO depths    : 1=0.1%, 2=0.1%, 4=0.3%, 8=0.5%, 16=1.1%, 32=97.9%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=1475/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32
Rand-Write-4K-QD32: (groupid=7, jobs=1): err= 0: pid=15532: Wed Jun 18 01:23:54 2014
  write: io=5824.0KB, bw=97267B/s, iops=23, runt= 61313msec
    slat (usec): min=8, max=1334.7K, avg=10103.57, stdev=115495.15
    clat (usec): min=801, max=2625.2K, avg=1336526.40, stdev=540903.56
     lat (usec): min=862, max=2625.2K, avg=1346631.30, stdev=529772.00
    clat percentiles (msec):
     |  1.00th=[   84],  5.00th=[  293], 10.00th=[  545], 20.00th=[ 1045],
     | 30.00th=[ 1336], 40.00th=[ 1336], 50.00th=[ 1336], 60.00th=[ 1336],
     | 70.00th=[ 1336], 80.00th=[ 1631], 90.00th=[ 2114], 95.00th=[ 2376],
     | 99.00th=[ 2573], 99.50th=[ 2638], 99.90th=[ 2638], 99.95th=[ 2638],
     | 99.99th=[ 2638]
    bw (KB  /s): min=    2, max=  328, per=100.00%, avg=109.99, stdev=79.13
    lat (usec) : 1000=0.27%
    lat (msec) : 2=0.14%, 50=0.07%, 100=0.82%, 250=3.16%, 500=5.01%
    lat (msec) : 750=4.95%, 1000=4.95%, 2000=67.58%, >=2000=13.05%
  cpu          : usr=0.10%, sys=0.00%, ctx=1126, majf=0, minf=25
  IO depths    : 1=0.1%, 2=0.1%, 4=0.3%, 8=0.5%, 16=1.1%, 32=97.9%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=1456/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=1024.0MB, aggrb=68786KB/s, minb=68786KB/s, maxb=68786KB/s, mint=15244msec, maxt=15244msec

Run status group 1 (all jobs):
  WRITE: io=1015.0MB, aggrb=17091KB/s, minb=17091KB/s, maxb=17091KB/s, mint=60810msec, maxt=60810msec

Run status group 2 (all jobs):
   READ: io=63488KB, aggrb=1051KB/s, minb=1051KB/s, maxb=1051KB/s, mint=60386msec, maxt=60386msec

Run status group 3 (all jobs):
  WRITE: io=62464KB, aggrb=1037KB/s, minb=1037KB/s, maxb=1037KB/s, mint=60190msec, maxt=60190msec

Run status group 4 (all jobs):
   READ: io=5772KB, aggrb=96KB/s, minb=96KB/s, maxb=96KB/s, mint=60039msec, maxt=60039msec

Run status group 5 (all jobs):
  WRITE: io=5700KB, aggrb=94KB/s, minb=94KB/s, maxb=94KB/s, mint=60038msec, maxt=60038msec

Run status group 6 (all jobs):
   READ: io=5900KB, aggrb=96KB/s, minb=96KB/s, maxb=96KB/s, mint=61326msec, maxt=61326msec

Run status group 7 (all jobs):
  WRITE: io=5824KB, aggrb=94KB/s, minb=94KB/s, maxb=94KB/s, mint=61313msec, maxt=61313msec

Disk stats (read/write):
  xvda1: ios=11606/11582, merge=0/46, ticks=2227220/2216848, in_queue=4445944, util=99.63%

9回目

Wed Jun 18 01:23:54 UTC 2014
Seq-Read: (g=0): rw=read, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Seq-Write: (g=1): rw=write, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Rand-Read-512K: (g=2): rw=randread, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Write-512K: (g=3): rw=randwrite, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Read-4K: (g=4): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Write-4K: (g=5): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Read-4K-QD32: (g=6): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
Rand-Write-4K-QD32: (g=7): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1.7
Starting 8 processes
Jobs: 1 (f=1): [_______w] [53.6% done] [0KB/55KB/0KB /s] [0/13/0 iops] [eta 07m:00s]
Seq-Read: (groupid=0, jobs=1): err= 0: pid=15536: Wed Jun 18 01:31:59 2014
  read : io=63488KB, bw=1055.4KB/s, iops=1, runt= 60176msec
    slat (usec): min=106, max=606, avg=119.03, stdev=62.89
    clat (msec): min=562, max=1017, avg=970.43, stdev=52.87
     lat (msec): min=563, max=1018, avg=970.55, stdev=52.80
    clat percentiles (msec):
     |  1.00th=[  562],  5.00th=[  979], 10.00th=[  979], 20.00th=[  979],
     | 30.00th=[  979], 40.00th=[  979], 50.00th=[  979], 60.00th=[  979],
     | 70.00th=[  979], 80.00th=[  979], 90.00th=[  979], 95.00th=[  979],
     | 99.00th=[ 1020], 99.50th=[ 1020], 99.90th=[ 1020], 99.95th=[ 1020],
     | 99.99th=[ 1020]
    bw (KB  /s): min= 1005, max= 1818, per=100.00%, avg=1060.61, stdev=97.93
    lat (msec) : 750=1.61%, 1000=96.77%, 2000=1.61%
  cpu          : usr=0.00%, sys=0.01%, ctx=64, majf=0, minf=286
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=62/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Seq-Write: (groupid=1, jobs=1): err= 0: pid=15537: Wed Jun 18 01:31:59 2014
  write: io=62464KB, bw=1039.5KB/s, iops=1, runt= 60096msec
    slat (usec): min=336, max=42389, avg=5153.08, stdev=13392.21
    clat (msec): min=750, max=1018, avg=980.00, stdev=33.89
     lat (msec): min=751, max=1060, avg=985.16, stdev=41.18
    clat percentiles (msec):
     |  1.00th=[  750],  5.00th=[  979], 10.00th=[  979], 20.00th=[  979],
     | 30.00th=[  979], 40.00th=[  979], 50.00th=[  979], 60.00th=[  979],
     | 70.00th=[  979], 80.00th=[  979], 90.00th=[ 1020], 95.00th=[ 1020],
     | 99.00th=[ 1020], 99.50th=[ 1020], 99.90th=[ 1020], 99.95th=[ 1020],
     | 99.99th=[ 1020]
    bw (KB  /s): min=  966, max= 1363, per=100.00%, avg=1041.72, stdev=50.18
    lat (msec) : 1000=81.97%, 2000=18.03%
  cpu          : usr=0.00%, sys=0.04%, ctx=71, majf=0, minf=29
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=61/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-512K: (groupid=2, jobs=1): err= 0: pid=15538: Wed Jun 18 01:31:59 2014
  read : io=63488KB, bw=1051.7KB/s, iops=2, runt= 60371msec
    slat (usec): min=66, max=307, avg=72.72, stdev=21.28
    clat (msec): min=269, max=529, avg=486.77, stdev=20.04
     lat (msec): min=269, max=529, avg=486.84, stdev=20.02
    clat percentiles (msec):
     |  1.00th=[  486],  5.00th=[  490], 10.00th=[  490], 20.00th=[  490],
     | 30.00th=[  490], 40.00th=[  490], 50.00th=[  490], 60.00th=[  490],
     | 70.00th=[  490], 80.00th=[  490], 90.00th=[  490], 95.00th=[  490],
     | 99.00th=[  490], 99.50th=[  529], 99.90th=[  529], 99.95th=[  529],
     | 99.99th=[  529]
    bw (KB  /s): min= 1005, max= 1352, per=100.00%, avg=1053.08, stdev=38.99
    lat (msec) : 500=99.19%, 750=0.81%
  cpu          : usr=0.00%, sys=0.01%, ctx=126, majf=0, minf=155
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=124/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-512K: (groupid=3, jobs=1): err= 0: pid=15539: Wed Jun 18 01:31:59 2014
  write: io=62464KB, bw=1036.7KB/s, iops=2, runt= 60256msec
    slat (usec): min=197, max=42298, avg=1926.07, stdev=8331.12
    clat (msec): min=256, max=529, avg=491.96, stdev=25.90
     lat (msec): min=257, max=572, avg=493.88, stdev=29.51
    clat percentiles (msec):
     |  1.00th=[  486],  5.00th=[  490], 10.00th=[  490], 20.00th=[  490],
     | 30.00th=[  490], 40.00th=[  490], 50.00th=[  490], 60.00th=[  490],
     | 70.00th=[  490], 80.00th=[  490], 90.00th=[  529], 95.00th=[  529],
     | 99.00th=[  529], 99.50th=[  529], 99.90th=[  529], 99.95th=[  529],
     | 99.99th=[  529]
    bw (KB  /s): min=  966, max= 1374, per=99.79%, avg=1033.79, stdev=55.03
    lat (msec) : 500=86.07%, 750=13.93%
  cpu          : usr=0.00%, sys=0.05%, ctx=129, majf=0, minf=26
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=122/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K: (groupid=4, jobs=1): err= 0: pid=15540: Wed Jun 18 01:31:59 2014
  read : io=5772.0KB, bw=98494B/s, iops=24, runt= 60009msec
    slat (usec): min=9, max=42, avg=25.71, stdev= 5.43
    clat (usec): min=135, max=83980, avg=41548.74, stdev=3259.71
     lat (usec): min=146, max=84010, avg=41575.76, stdev=3260.19
    clat percentiles (usec):
     |  1.00th=[40704],  5.00th=[40704], 10.00th=[40704], 20.00th=[41216],
     | 30.00th=[41216], 40.00th=[41728], 50.00th=[41728], 60.00th=[41728],
     | 70.00th=[41728], 80.00th=[41728], 90.00th=[42240], 95.00th=[42240],
     | 99.00th=[42240], 99.50th=[42240], 99.90th=[83456], 99.95th=[84480],
     | 99.99th=[84480]
    bw (KB  /s): min=   88, max=  141, per=100.00%, avg=96.10, stdev= 4.42
    lat (usec) : 250=0.28%, 500=0.07%
    lat (msec) : 10=0.07%, 50=99.38%, 100=0.21%
  cpu          : usr=0.00%, sys=0.11%, ctx=1445, majf=0, minf=28
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1443/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-4K: (groupid=5, jobs=1): err= 0: pid=15541: Wed Jun 18 01:31:59 2014
  write: io=5700.0KB, bw=97271B/s, iops=23, runt= 60005msec
    slat (usec): min=11, max=42268, avg=358.61, stdev=3643.00
    clat (usec): min=397, max=83938, avg=41738.56, stdev=4684.52
     lat (usec): min=409, max=125099, avg=42098.55, stdev=7872.09
    clat percentiles (usec):
     |  1.00th=[40192],  5.00th=[40704], 10.00th=[40704], 20.00th=[40704],
     | 30.00th=[41216], 40.00th=[41728], 50.00th=[41728], 60.00th=[41728],
     | 70.00th=[41728], 80.00th=[41728], 90.00th=[42240], 95.00th=[42240],
     | 99.00th=[43776], 99.50th=[83456], 99.90th=[83456], 99.95th=[83456],
     | 99.99th=[83456]
    bw (KB  /s): min=   80, max=  150, per=100.00%, avg=94.89, stdev= 6.89
    lat (usec) : 500=0.14%, 750=0.28%
    lat (msec) : 2=0.07%, 50=98.74%, 100=0.77%
  cpu          : usr=0.00%, sys=0.13%, ctx=1437, majf=0, minf=28
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=1425/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K-QD32: (groupid=6, jobs=1): err= 0: pid=15542: Wed Jun 18 01:31:59 2014
  read : io=5900.0KB, bw=98567B/s, iops=24, runt= 61294msec
    slat (usec): min=7, max=45, avg=26.88, stdev= 5.23
    clat (usec): min=456, max=2582.1K, avg=1328827.61, stdev=178575.40
     lat (usec): min=506, max=2583.7K, avg=1328855.82, stdev=178577.01
    clat percentiles (msec):
     |  1.00th=[  334],  5.00th=[ 1336], 10.00th=[ 1336], 20.00th=[ 1336],
     | 30.00th=[ 1336], 40.00th=[ 1336], 50.00th=[ 1336], 60.00th=[ 1336],
     | 70.00th=[ 1336], 80.00th=[ 1336], 90.00th=[ 1336], 95.00th=[ 1369],
     | 99.00th=[ 2008], 99.50th=[ 2278], 99.90th=[ 2540], 99.95th=[ 2573],
     | 99.99th=[ 2573]
    bw (KB  /s): min=   22, max=  151, per=99.49%, avg=95.51, stdev= 8.73
    lat (usec) : 500=0.07%, 750=0.34%
    lat (msec) : 2=0.07%, 50=0.07%, 100=0.07%, 250=0.20%, 500=0.41%
    lat (msec) : 750=0.41%, 1000=0.41%, 2000=97.02%, >=2000=0.95%
  cpu          : usr=0.00%, sys=0.10%, ctx=1473, majf=0, minf=60
  IO depths    : 1=0.1%, 2=0.1%, 4=0.3%, 8=0.5%, 16=1.1%, 32=97.9%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=1475/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32
Rand-Write-4K-QD32: (groupid=7, jobs=1): err= 0: pid=15543: Wed Jun 18 01:31:59 2014
  write: io=5816.0KB, bw=97157B/s, iops=23, runt= 61298msec
    slat (usec): min=8, max=1334.2K, avg=10116.27, stdev=115581.32
    clat (usec): min=644, max=2626.1K, avg=1338037.70, stdev=539530.32
     lat (usec): min=704, max=2626.2K, avg=1348155.26, stdev=528322.62
    clat percentiles (msec):
     |  1.00th=[   84],  5.00th=[  293], 10.00th=[  545], 20.00th=[ 1045],
     | 30.00th=[ 1336], 40.00th=[ 1336], 50.00th=[ 1336], 60.00th=[ 1336],
     | 70.00th=[ 1336], 80.00th=[ 1631], 90.00th=[ 2114], 95.00th=[ 2376],
     | 99.00th=[ 2573], 99.50th=[ 2638], 99.90th=[ 2638], 99.95th=[ 2638],
     | 99.99th=[ 2638]
    bw (KB  /s): min=    2, max=  311, per=100.00%, avg=109.93, stdev=78.18
    lat (usec) : 750=0.07%, 1000=0.21%
    lat (msec) : 10=0.07%, 50=0.07%, 100=0.83%, 250=3.16%, 500=5.02%
    lat (msec) : 750=4.95%, 1000=4.95%, 2000=67.54%, >=2000=13.14%
  cpu          : usr=0.00%, sys=0.10%, ctx=1125, majf=0, minf=26
  IO depths    : 1=0.1%, 2=0.1%, 4=0.3%, 8=0.6%, 16=1.1%, 32=97.9%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=1454/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=63488KB, aggrb=1055KB/s, minb=1055KB/s, maxb=1055KB/s, mint=60176msec, maxt=60176msec

Run status group 1 (all jobs):
  WRITE: io=62464KB, aggrb=1039KB/s, minb=1039KB/s, maxb=1039KB/s, mint=60096msec, maxt=60096msec

Run status group 2 (all jobs):
   READ: io=63488KB, aggrb=1051KB/s, minb=1051KB/s, maxb=1051KB/s, mint=60371msec, maxt=60371msec

Run status group 3 (all jobs):
  WRITE: io=62464KB, aggrb=1036KB/s, minb=1036KB/s, maxb=1036KB/s, mint=60256msec, maxt=60256msec

Run status group 4 (all jobs):
   READ: io=5772KB, aggrb=96KB/s, minb=96KB/s, maxb=96KB/s, mint=60009msec, maxt=60009msec

Run status group 5 (all jobs):
  WRITE: io=5700KB, aggrb=94KB/s, minb=94KB/s, maxb=94KB/s, mint=60005msec, maxt=60005msec

Run status group 6 (all jobs):
   READ: io=5900KB, aggrb=96KB/s, minb=96KB/s, maxb=96KB/s, mint=61294msec, maxt=61294msec

Run status group 7 (all jobs):
  WRITE: io=5816KB, aggrb=94KB/s, minb=94KB/s, maxb=94KB/s, mint=61298msec, maxt=61298msec

Disk stats (read/write):
  xvda1: ios=3910/3946, merge=0/46, ticks=2421280/2209320, in_queue=4634616, util=99.70%

10回目

Wed Jun 18 01:32:00 UTC 2014
Seq-Read: (g=0): rw=read, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Seq-Write: (g=1): rw=write, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=1
Rand-Read-512K: (g=2): rw=randread, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Write-512K: (g=3): rw=randwrite, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=1
Rand-Read-4K: (g=4): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Write-4K: (g=5): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
Rand-Read-4K-QD32: (g=6): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
Rand-Write-4K-QD32: (g=7): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1.7
Starting 8 processes
Jobs: 1 (f=1): [_______w] [53.7% done] [0KB/0KB/0KB /s] [0/0/0 iops] [eta 06m:59s]
Seq-Read: (groupid=0, jobs=1): err= 0: pid=15547: Wed Jun 18 01:40:05 2014
  read : io=63488KB, bw=1053.3KB/s, iops=1, runt= 60281msec
    slat (usec): min=105, max=592, avg=118.79, stdev=61.13
    clat (msec): min=667, max=1018, avg=972.13, stdev=39.70
     lat (msec): min=668, max=1018, avg=972.25, stdev=39.64
    clat percentiles (msec):
     |  1.00th=[  668],  5.00th=[  979], 10.00th=[  979], 20.00th=[  979],
     | 30.00th=[  979], 40.00th=[  979], 50.00th=[  979], 60.00th=[  979],
     | 70.00th=[  979], 80.00th=[  979], 90.00th=[  979], 95.00th=[  979],
     | 99.00th=[ 1020], 99.50th=[ 1020], 99.90th=[ 1020], 99.95th=[ 1020],
     | 99.99th=[ 1020]
    bw (KB  /s): min= 1004, max= 1532, per=100.00%, avg=1055.95, stdev=61.72
    lat (msec) : 750=1.61%, 1000=96.77%, 2000=1.61%
  cpu          : usr=0.00%, sys=0.01%, ctx=63, majf=0, minf=285
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=62/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Seq-Write: (groupid=1, jobs=1): err= 0: pid=15548: Wed Jun 18 01:40:05 2014
  write: io=62464KB, bw=1037.9KB/s, iops=1, runt= 60185msec
    slat (usec): min=334, max=42552, avg=6523.03, stdev=14915.10
    clat (msec): min=756, max=1018, avg=980.09, stdev=33.29
     lat (msec): min=756, max=1060, avg=986.61, stdev=42.48
    clat percentiles (msec):
     |  1.00th=[  758],  5.00th=[  979], 10.00th=[  979], 20.00th=[  979],
     | 30.00th=[  979], 40.00th=[  979], 50.00th=[  979], 60.00th=[  979],
     | 70.00th=[  979], 80.00th=[  979], 90.00th=[ 1020], 95.00th=[ 1020],
     | 99.00th=[ 1020], 99.50th=[ 1020], 99.90th=[ 1020], 99.95th=[ 1020],
     | 99.99th=[ 1020]
    bw (KB  /s): min=  966, max= 1352, per=100.00%, avg=1040.28, stdev=50.51
    lat (msec) : 1000=81.97%, 2000=18.03%
  cpu          : usr=0.00%, sys=0.04%, ctx=71, majf=0, minf=28
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=61/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-512K: (groupid=2, jobs=1): err= 0: pid=15549: Wed Jun 18 01:40:05 2014
  read : io=63488KB, bw=1050.2KB/s, iops=2, runt= 60459msec
    slat (usec): min=73, max=323, avg=79.95, stdev=22.19
    clat (msec): min=275, max=529, avg=487.47, stdev=20.27
     lat (msec): min=275, max=530, avg=487.55, stdev=20.25
    clat percentiles (msec):
     |  1.00th=[  490],  5.00th=[  490], 10.00th=[  490], 20.00th=[  490],
     | 30.00th=[  490], 40.00th=[  490], 50.00th=[  490], 60.00th=[  490],
     | 70.00th=[  490], 80.00th=[  490], 90.00th=[  490], 95.00th=[  490],
     | 99.00th=[  529], 99.50th=[  529], 99.90th=[  529], 99.95th=[  529],
     | 99.99th=[  529]
    bw (KB  /s): min=  966, max= 1342, per=100.00%, avg=1050.35, stdev=40.41
    lat (msec) : 500=97.58%, 750=2.42%
  cpu          : usr=0.00%, sys=0.01%, ctx=125, majf=0, minf=154
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=124/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-512K: (groupid=3, jobs=1): err= 0: pid=15550: Wed Jun 18 01:40:05 2014
  write: io=62464KB, bw=1037.6KB/s, iops=2, runt= 60204msec
    slat (usec): min=190, max=42273, avg=1580.52, stdev=7471.73
    clat (msec): min=245, max=530, avg=491.88, stdev=26.73
     lat (msec): min=246, max=572, avg=493.46, stdev=29.59
    clat percentiles (msec):
     |  1.00th=[  486],  5.00th=[  490], 10.00th=[  490], 20.00th=[  490],
     | 30.00th=[  490], 40.00th=[  490], 50.00th=[  490], 60.00th=[  490],
     | 70.00th=[  490], 80.00th=[  490], 90.00th=[  529], 95.00th=[  529],
     | 99.00th=[  529], 99.50th=[  529], 99.90th=[  529], 99.95th=[  529],
     | 99.99th=[  529]
    bw (KB  /s): min=  966, max= 1393, per=99.78%, avg=1034.70, stdev=56.30
    lat (msec) : 250=0.82%, 500=85.25%, 750=13.93%
  cpu          : usr=0.05%, sys=0.00%, ctx=127, majf=0, minf=25
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=122/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K: (groupid=4, jobs=1): err= 0: pid=15551: Wed Jun 18 01:40:05 2014
  read : io=5780.0KB, bw=98577B/s, iops=24, runt= 60041msec
    slat (usec): min=8, max=41, avg=26.02, stdev= 5.02
    clat (usec): min=146, max=82938, avg=41513.07, stdev=3112.89
     lat (usec): min=156, max=82967, avg=41540.38, stdev=3113.49
    clat percentiles (usec):
     |  1.00th=[40704],  5.00th=[40704], 10.00th=[40704], 20.00th=[41216],
     | 30.00th=[41216], 40.00th=[41728], 50.00th=[41728], 60.00th=[41728],
     | 70.00th=[41728], 80.00th=[41728], 90.00th=[42240], 95.00th=[42240],
     | 99.00th=[42240], 99.50th=[42240], 99.90th=[82432], 99.95th=[82432],
     | 99.99th=[82432]
    bw (KB  /s): min=   88, max=  140, per=100.00%, avg=96.18, stdev= 4.28
    lat (usec) : 250=0.28%, 500=0.14%
    lat (msec) : 50=99.45%, 100=0.14%
  cpu          : usr=0.00%, sys=0.11%, ctx=1446, majf=0, minf=27
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=1445/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Write-4K: (groupid=5, jobs=1): err= 0: pid=15552: Wed Jun 18 01:40:05 2014
  write: io=5700.0KB, bw=97220B/s, iops=23, runt= 60037msec
    slat (usec): min=10, max=42200, avg=359.73, stdev=3654.66
    clat (usec): min=492, max=83953, avg=41759.71, stdev=4562.30
     lat (usec): min=507, max=125051, avg=42120.83, stdev=7802.72
    clat percentiles (usec):
     |  1.00th=[40192],  5.00th=[40704], 10.00th=[40704], 20.00th=[41216],
     | 30.00th=[41216], 40.00th=[41728], 50.00th=[41728], 60.00th=[41728],
     | 70.00th=[41728], 80.00th=[41728], 90.00th=[42240], 95.00th=[42240],
     | 99.00th=[43264], 99.50th=[82432], 99.90th=[83456], 99.95th=[83456],
     | 99.99th=[83456]
    bw (KB  /s): min=   80, max=  141, per=100.00%, avg=94.81, stdev= 6.36
    lat (usec) : 500=0.07%, 750=0.35%
    lat (msec) : 50=98.81%, 100=0.77%
  cpu          : usr=0.00%, sys=0.13%, ctx=1438, majf=0, minf=27
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=1425/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Rand-Read-4K-QD32: (groupid=6, jobs=1): err= 0: pid=15553: Wed Jun 18 01:40:05 2014
  read : io=5900.0KB, bw=98517B/s, iops=24, runt= 61325msec
    slat (usec): min=6, max=46, avg=24.97, stdev= 6.27
    clat (usec): min=797, max=2582.1K, avg=1329507.32, stdev=176068.45
     lat (usec): min=846, max=2582.1K, avg=1329533.60, stdev=176070.23
    clat percentiles (msec):
     |  1.00th=[  367],  5.00th=[ 1336], 10.00th=[ 1336], 20.00th=[ 1336],
     | 30.00th=[ 1336], 40.00th=[ 1336], 50.00th=[ 1336], 60.00th=[ 1336],
     | 70.00th=[ 1336], 80.00th=[ 1336], 90.00th=[ 1336], 95.00th=[ 1369],
     | 99.00th=[ 2008], 99.50th=[ 2278], 99.90th=[ 2540], 99.95th=[ 2573],
     | 99.99th=[ 2573]
    bw (KB  /s): min=   29, max=  142, per=99.49%, avg=95.51, stdev= 7.71
    lat (usec) : 1000=0.41%
    lat (msec) : 50=0.07%, 100=0.07%, 250=0.27%, 500=0.41%, 750=0.41%
    lat (msec) : 1000=0.41%, 2000=97.02%, >=2000=0.95%
  cpu          : usr=0.10%, sys=0.00%, ctx=1471, majf=0, minf=59
  IO depths    : 1=0.1%, 2=0.1%, 4=0.3%, 8=0.5%, 16=1.1%, 32=97.9%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=1475/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32
Rand-Write-4K-QD32: (groupid=7, jobs=1): err= 0: pid=15554: Wed Jun 18 01:40:05 2014
  write: io=5824.0KB, bw=97266B/s, iops=23, runt= 61314msec
    slat (usec): min=7, max=1334.2K, avg=10103.17, stdev=115500.31
    clat (usec): min=798, max=2625.2K, avg=1336535.50, stdev=540925.54
     lat (usec): min=830, max=2625.2K, avg=1346639.98, stdev=529793.23
    clat percentiles (msec):
     |  1.00th=[   84],  5.00th=[  293], 10.00th=[  545], 20.00th=[ 1045],
     | 30.00th=[ 1336], 40.00th=[ 1336], 50.00th=[ 1336], 60.00th=[ 1336],
     | 70.00th=[ 1336], 80.00th=[ 1631], 90.00th=[ 2114], 95.00th=[ 2376],
     | 99.00th=[ 2573], 99.50th=[ 2638], 99.90th=[ 2638], 99.95th=[ 2638],
     | 99.99th=[ 2638]
    bw (KB  /s): min=    2, max=  311, per=100.00%, avg=110.44, stdev=78.32
    lat (usec) : 1000=0.41%
    lat (msec) : 50=0.07%, 100=0.82%, 250=3.30%, 500=4.95%, 750=4.95%
    lat (msec) : 1000=4.88%, 2000=67.65%, >=2000=12.98%
  cpu          : usr=0.00%, sys=0.10%, ctx=1124, majf=0, minf=25
  IO depths    : 1=0.1%, 2=0.1%, 4=0.3%, 8=0.5%, 16=1.1%, 32=97.9%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=1456/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=63488KB, aggrb=1053KB/s, minb=1053KB/s, maxb=1053KB/s, mint=60281msec, maxt=60281msec

Run status group 1 (all jobs):
  WRITE: io=62464KB, aggrb=1037KB/s, minb=1037KB/s, maxb=1037KB/s, mint=60185msec, maxt=60185msec

Run status group 2 (all jobs):
   READ: io=63488KB, aggrb=1050KB/s, minb=1050KB/s, maxb=1050KB/s, mint=60459msec, maxt=60459msec

Run status group 3 (all jobs):
  WRITE: io=62464KB, aggrb=1037KB/s, minb=1037KB/s, maxb=1037KB/s, mint=60204msec, maxt=60204msec

Run status group 4 (all jobs):
   READ: io=5780KB, aggrb=96KB/s, minb=96KB/s, maxb=96KB/s, mint=60041msec, maxt=60041msec

Run status group 5 (all jobs):
  WRITE: io=5700KB, aggrb=94KB/s, minb=94KB/s, maxb=94KB/s, mint=60037msec, maxt=60037msec

Run status group 6 (all jobs):
   READ: io=5900KB, aggrb=96KB/s, minb=96KB/s, maxb=96KB/s, mint=61325msec, maxt=61325msec

Run status group 7 (all jobs):
  WRITE: io=5824KB, aggrb=94KB/s, minb=94KB/s, maxb=94KB/s, mint=61314msec, maxt=61314msec

Disk stats (read/write):
  xvda1: ios=3912/3947, merge=0/46, ticks=2423364/2207904, in_queue=4637596, util=99.72%

バーストタイム中は3000IOPSほど出ていますが後半バーストタイムが終了すると最低値の24IOPSしか出なくなっています。