Benchmarking distributed file systems

stupid fast tests first, all using small compute nodes and use iozone -r 32 -s $((2**24+2**25)) -i 0 -i 1 -i 2 -i 8 -O -I

Single iozone thread

local SSD test

a9004 with /local/ssd

iozone -r 32 -s $((2**24+2**25)) -i 0 -i 1 -i 2 -i 8 -O -I
 [...]
                                                      random  random    bkwd   record   stride
              KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread
        50331648      32    1466    4644    12709    12561    4286     564

Quobyte

Use a9001/2/3 as storage servers and a9004 as a networked client
Iozone: Performance Test of File I/O
[...]
                                                            random  random    bkwd   record   stride
              KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread
        50331648      32     201     248      736      753     788     339

BeeGFS

Using a9005 as config, a9006 as meta data, a9007...a9009 as storage and a9010 as client:

[...]
     
                                                            random  random    bkwd   record   stride
              KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread
        50331648      32    1025    1047     1031     1033     935    1022

Comparison

This table summarizes the single thread performance, all values in ops/s according to above definition.

Test local SSD Quobyte !BeeGFS
write 1466 201 1025
rewrite 4644 248 1047
read 12709 736 1031
reread 12561 753 1033
random read 4286 788 935
random write 564 339 1022

Massively parallel test

Here, we run with 15 parallel threads using iozone -r 32 -s 4194304 -i 0 -i 1 -i 2 -i 8 -O -I -t 15. For comparison, we will only concentrate on the throughput the parent sees ignoring potential problems with min/avg/max for now. On the local system, the tests are run twice, once with 15 threads and once with 5 threads but thrice the size.

All results are again in ops/s.

Test local SSD 5 threads local SSD 15 threads Quobyte BeeGFS single 1Gb/s client BeeGFS 3 clients (summed) BeeGFS single 10Gb/s client
Initial write 3861 3703 796 3718 7648 6901
rewrite 4556 4469 1182 3745 6076 7905
reader 10331 10523 1841 3771 11289 10871
re-read 10296 10519 1860 3771 11296 10918
random read 10128 10479 3682 3771 10888 9895
mixed workload 3917 3981 1584 6346 8260 7914
random write 4286 4196 1538 3721 6074 7097

Note for BeeGFS single client test: Most of the test was limited by the 1 Gb/s NIC of a9010, hence the constant results in most categories!

-- CarstenAulbert - 21 Aug 2014
Topic revision: r8 - 03 Sep 2014, CarstenAulbert
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback