Benchmarking distributed file systems
stupid fast tests first, all using small compute nodes and use
iozone -r 32 -s $((2**24+2**25)) -i 0 -i 1 -i 2 -i 8 -O -I
Single iozone thread
local SSD test
a9004 with
/local/ssd
iozone -r 32 -s $((2**24+2**25)) -i 0 -i 1 -i 2 -i 8 -O -I
[...]
random random bkwd record stride
KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
50331648 32 1466 4644 12709 12561 4286 564
Quobyte
Use a9001/2/3 as storage servers and a9004 as a networked client
Iozone: Performance Test of File I/O
[...]
random random bkwd record stride
KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
50331648 32 201 248 736 753 788 339
BeeGFS
Using a9005 as config, a9006 as meta data, a9007...a9009 as storage and a9010 as client:
[...]
random random bkwd record stride
KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
50331648 32 1025 1047 1031 1033 935 1022
Comparison
This table summarizes the single thread performance, all values in ops/s according to above definition.
Test |
local SSD |
Quobyte |
!BeeGFS |
write |
1466 |
201 |
1025 |
rewrite |
4644 |
248 |
1047 |
read |
12709 |
736 |
1031 |
reread |
12561 |
753 |
1033 |
random read |
4286 |
788 |
935 |
random write |
564 |
339 |
1022 |
Massively parallel test
Here, we run with 15 parallel threads using
iozone -r 32 -s 4194304 -i 0 -i 1 -i 2 -i 8 -O -I -t 15
. For comparison, we will only concentrate on the throughput the parent sees ignoring potential problems with min/avg/max for now. On the local system, the tests are run twice, once with 15 threads and once with 5 threads but thrice the size.
All results are again in ops/s.
Note for BeeGFS single client test: Most of the test was limited by the 1 Gb/s NIC of a9010, hence the constant results in most categories!
--
CarstenAulbert - 21 Aug 2014