 |
|
You are here: Foswiki>ATLAS Web>BCacheBenchmarking (28 Mar 2014, CarstenAulbert)Edit Attach
Benchmarking bcache
Evaluating if bcache can/should be used on our compute nodes. Benchmarking was performed with iozone (revision 3.397) with the command lineiozone -r 32 -s 1048576 -i 0 -i 1 -i 2 -i 8 -O -I -t 4 as 32kB record size looks like a representative one. As the machines are quad core machines, we use 4 parallel threads and perform tests with O_DIRECT in async fashion. To eliminate RAM caching we use mem=512M as a kernel boot option.
Summary (average number of IOPS per thread and workload)
| type | HDD | SSD | bcache | bcache writeback |
|---|---|---|---|---|
| initial writer | 648.77 | 1166.57 | 616.25 | 545.60 |
| rewriter | 619.52 | 1139.51 | 586.93 | 479.31 |
| reader | 86.72 | 2546.75 | 34.65 | 1672.49 |
| re-reader | 88.95 | 2549.07 | 37.62 | 1667.63 |
| random reader | 38.59 | 2481.68 | 38.57 | 1841.83 |
| mixed workload | 51.11 | 981.93 | 90.52 | 610.79 |
| random writer | 69.31 | 1103.15 | 60.05 | 357.18 |
Raw results
HDD only
Iozone: Performance Test of File I/O
Version $Revision: 3.397 $
Compiled for 64 bit mode.
Build: linux-AMD64
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer.
Ben England.
Run began: Thu Mar 27 16:33:55 2014
Record Size 32 KB
File size set to 1048576 KB
OPS Mode. Output is in operations per second.
O_DIRECT feature enabled
Command line used: iozone -r 32 -s 1048576 -i 0 -i 1 -i 2 -i 8 -O -I -t 4
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 4 processes
Each process writes a 1048576 Kbyte file in 32 Kbyte records
Children see throughput for 4 initial writers = 2595.10 ops/sec
Parent sees throughput for 4 initial writers = 2565.82 ops/sec
Min throughput per process = 648.77 ops/sec
Max throughput per process = 648.79 ops/sec
Avg throughput per process = 648.77 ops/sec
Min xfer = 32767.00 ops
Children see throughput for 4 rewriters = 2478.06 ops/sec
Parent sees throughput for 4 rewriters = 2454.33 ops/sec
Min throughput per process = 619.50 ops/sec
Max throughput per process = 619.52 ops/sec
Avg throughput per process = 619.52 ops/sec
Min xfer = 32767.00 ops
Children see throughput for 4 readers = 346.86 ops/sec
Parent sees throughput for 4 readers = 346.68 ops/sec
Min throughput per process = 65.19 ops/sec
Max throughput per process = 123.47 ops/sec
Avg throughput per process = 86.72 ops/sec
Min xfer = 17304.00 ops
Children see throughput for 4 re-readers = 355.80 ops/sec
Parent sees throughput for 4 re-readers = 355.67 ops/sec
Min throughput per process = 71.76 ops/sec
Max throughput per process = 125.71 ops/sec
Avg throughput per process = 88.95 ops/sec
Min xfer = 18708.00 ops
Children see throughput for 4 random readers = 154.37 ops/sec
Parent sees throughput for 4 random readers = 154.33 ops/sec
Min throughput per process = 7.58 ops/sec
Max throughput per process = 98.02 ops/sec
Avg throughput per process = 38.59 ops/sec
Min xfer = 2535.00 ops
Children see throughput for 4 mixed workload = 204.43 ops/sec
Parent sees throughput for 4 mixed workload = 204.02 ops/sec
Min throughput per process = 11.66 ops/sec
Max throughput per process = 89.93 ops/sec
Avg throughput per process = 51.11 ops/sec
Min xfer = 4249.00 ops
Children see throughput for 4 random writers = 277.22 ops/sec
Parent sees throughput for 4 random writers = 276.77 ops/sec
Min throughput per process = 69.30 ops/sec
Max throughput per process = 69.31 ops/sec
Avg throughput per process = 69.31 ops/sec
Min xfer = 32761.00 ops
SSD
Iozone: Performance Test of File I/O
Version $Revision: 3.397 $
Compiled for 64 bit mode.
Build: linux-AMD64
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer.
Ben England.
Run began: Thu Mar 27 16:29:44 2014
Record Size 32 KB
File size set to 1048576 KB
OPS Mode. Output is in operations per second.
O_DIRECT feature enabled
Command line used: iozone -r 32 -s 1048576 -i 0 -i 1 -i 2 -i 8 -O -I -t 4
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 4 processes
Each process writes a 1048576 Kbyte file in 32 Kbyte records
Children see throughput for 4 initial writers = 4666.29 ops/sec
Parent sees throughput for 4 initial writers = 4299.21 ops/sec
Min throughput per process = 1073.92 ops/sec
Max throughput per process = 1267.17 ops/sec
Avg throughput per process = 1166.57 ops/sec
Min xfer = 27771.00 ops
Children see throughput for 4 rewriters = 4558.05 ops/sec
Parent sees throughput for 4 rewriters = 4555.59 ops/sec
Min throughput per process = 1135.01 ops/sec
Max throughput per process = 1143.81 ops/sec
Avg throughput per process = 1139.51 ops/sec
Min xfer = 32516.00 ops
Children see throughput for 4 readers = 10187.00 ops/sec
Parent sees throughput for 4 readers = 10185.98 ops/sec
Min throughput per process = 2530.82 ops/sec
Max throughput per process = 2557.96 ops/sec
Avg throughput per process = 2546.75 ops/sec
Min xfer = 32421.00 ops
Children see throughput for 4 re-readers = 10196.26 ops/sec
Parent sees throughput for 4 re-readers = 10195.77 ops/sec
Min throughput per process = 2537.48 ops/sec
Max throughput per process = 2570.97 ops/sec
Avg throughput per process = 2549.07 ops/sec
Min xfer = 32342.00 ops
Children see throughput for 4 random readers = 9926.73 ops/sec
Parent sees throughput for 4 random readers = 9926.29 ops/sec
Min throughput per process = 2459.02 ops/sec
Max throughput per process = 2511.85 ops/sec
Avg throughput per process = 2481.68 ops/sec
Min xfer = 32079.00 ops
Children see throughput for 4 mixed workload = 3927.72 ops/sec
Parent sees throughput for 4 mixed workload = 3913.55 ops/sec
Min throughput per process = 208.71 ops/sec
Max throughput per process = 1757.69 ops/sec
Avg throughput per process = 981.93 ops/sec
Min xfer = 3891.00 ops
Children see throughput for 4 random writers = 4412.60 ops/sec
Parent sees throughput for 4 random writers = 4390.54 ops/sec
Min throughput per process = 1097.77 ops/sec
Max throughput per process = 1107.07 ops/sec
Avg throughput per process = 1103.15 ops/sec
Min xfer = 32493.00 ops
iozone test complete.
bcache (default)
Iozone: Performance Test of File I/O
Version $Revision: 3.397 $
Compiled for 64 bit mode.
Build: linux-AMD64
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer.
Ben England.
Run began: Thu Mar 27 17:07:45 2014
Record Size 32 KB
File size set to 1048576 KB
OPS Mode. Output is in operations per second.
O_DIRECT feature enabled
Command line used: iozone -r 32 -s 1048576 -i 0 -i 1 -i 2 -i 8 -O -I -t 4
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 4 processes
Each process writes a 1048576 Kbyte file in 32 Kbyte records
Children see throughput for 4 initial writers = 2465.01 ops/sec
Parent sees throughput for 4 initial writers = 2427.73 ops/sec
Min throughput per process = 615.18 ops/sec
Max throughput per process = 617.12 ops/sec
Avg throughput per process = 616.25 ops/sec
Min xfer = 32665.00 ops
Children see throughput for 4 rewriters = 2347.74 ops/sec
Parent sees throughput for 4 rewriters = 2320.40 ops/sec
Min throughput per process = 574.66 ops/sec
Max throughput per process = 601.04 ops/sec
Avg throughput per process = 586.93 ops/sec
Min xfer = 31330.00 ops
Children see throughput for 4 readers = 138.62 ops/sec
Parent sees throughput for 4 readers = 138.59 ops/sec
Min throughput per process = 25.01 ops/sec
Max throughput per process = 47.44 ops/sec
Avg throughput per process = 34.65 ops/sec
Min xfer = 17277.00 ops
Children see throughput for 4 re-readers = 150.46 ops/sec
Parent sees throughput for 4 re-readers = 150.44 ops/sec
Min throughput per process = 26.45 ops/sec
Max throughput per process = 47.85 ops/sec
Avg throughput per process = 37.62 ops/sec
Min xfer = 18117.00 ops
Children see throughput for 4 random readers = 154.27 ops/sec
Parent sees throughput for 4 random readers = 154.25 ops/sec
Min throughput per process = 22.60 ops/sec
Max throughput per process = 49.57 ops/sec
Avg throughput per process = 38.57 ops/sec
Min xfer = 14939.00 ops
Children see throughput for 4 mixed workload = 362.08 ops/sec
Parent sees throughput for 4 mixed workload = 361.10 ops/sec
Min throughput per process = 36.68 ops/sec
Max throughput per process = 138.96 ops/sec
Avg throughput per process = 90.52 ops/sec
Min xfer = 8670.00 ops
Children see throughput for 4 random writers = 240.20 ops/sec
Parent sees throughput for 4 random writers = 239.12 ops/sec
Min throughput per process = 59.98 ops/sec
Max throughput per process = 60.24 ops/sec
Avg throughput per process = 60.05 ops/sec
Min xfer = 32624.00 ops
iozone test complete.
bcache writeback cache
Iozone: Performance Test of File I/O
Version $Revision: 3.397 $
Compiled for 64 bit mode.
Build: linux-AMD64
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer.
Ben England.
Run began: Thu Mar 27 18:05:01 2014
Record Size 32 KB
File size set to 1048576 KB
OPS Mode. Output is in operations per second.
O_DIRECT feature enabled
Command line used: iozone -r 32 -s 1048576 -i 0 -i 1 -i 2 -i 8 -O -I -t 4
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 4 processes
Each process writes a 1048576 Kbyte file in 32 Kbyte records
Children see throughput for 4 initial writers = 2182.38 ops/sec
Parent sees throughput for 4 initial writers = 2169.55 ops/sec
Min throughput per process = 541.76 ops/sec
Max throughput per process = 548.65 ops/sec
Avg throughput per process = 545.60 ops/sec
Min xfer = 32357.00 ops
Children see throughput for 4 rewriters = 1917.25 ops/sec
Parent sees throughput for 4 rewriters = 1916.55 ops/sec
Min throughput per process = 477.48 ops/sec
Max throughput per process = 480.88 ops/sec
Avg throughput per process = 479.31 ops/sec
Min xfer = 32538.00 ops
Children see throughput for 4 readers = 6689.95 ops/sec
Parent sees throughput for 4 readers = 6650.93 ops/sec
Min throughput per process = 1635.10 ops/sec
Max throughput per process = 1778.46 ops/sec
Avg throughput per process = 1672.49 ops/sec
Min xfer = 30127.00 ops
Children see throughput for 4 re-readers = 6670.52 ops/sec
Parent sees throughput for 4 re-readers = 6631.52 ops/sec
Min throughput per process = 1632.89 ops/sec
Max throughput per process = 1768.31 ops/sec
Avg throughput per process = 1667.63 ops/sec
Min xfer = 30259.00 ops
Children see throughput for 4 random readers = 7367.33 ops/sec
Parent sees throughput for 4 random readers = 7318.86 ops/sec
Min throughput per process = 1779.25 ops/sec
Max throughput per process = 2021.97 ops/sec
Avg throughput per process = 1841.83 ops/sec
Min xfer = 28835.00 ops
Children see throughput for 4 mixed workload = 2443.17 ops/sec
Parent sees throughput for 4 mixed workload = 2251.08 ops/sec
Min throughput per process = 552.74 ops/sec
Max throughput per process = 637.42 ops/sec
Avg throughput per process = 610.79 ops/sec
Min xfer = 28415.00 ops
Children see throughput for 4 random writers = 1428.71 ops/sec
Parent sees throughput for 4 random writers = 1415.27 ops/sec
Min throughput per process = 355.79 ops/sec
Max throughput per process = 359.59 ops/sec
Avg throughput per process = 357.18 ops/sec
Min xfer = 32422.00 ops
iozone test complete.
-- CarstenAulbert - 28 Mar 2014 Edit | Attach | Print version | History: r2 < r1 | Backlinks | View wiki text | Edit wiki text | More topic actions
Topic revision: r2 - 28 Mar 2014, CarstenAulbert
ATLAS
Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors. Ideas, requests, problems regarding Foswiki? Send feedback