Einstein@home RAID setup testing

Over time the einstein-abp1 download server (mostly BRP project) went through various updates to increase download throughput (apache2-mpm-worker, lighttpd, nginx on the web server side, move from 8bit to 4bit sampled data, deadline scheduler optimiztion) - however we seem to hit a limit around 900k downloads a day with pre-procesing writing new files to the machine at the same speed as it's deleted, up to 1.6TB/day (2050kB files + housekeeping files). Next logical step before going to a clustered FS or similar solution is to go with more download servers, each running its own work unit generator. Going that route we also looked briefly into an alternative to the current 16 disk Areca 1261ML RAID6 setup.

Web server tests

The machine was re-installed and a test data set created (1024 download directories each with 2048 2050kB files plus an md5 file for each data file). Webserver was nginx with varying number of workers, each worker accepting up to 512 connections. Underneath was either a RAID6 or RADI10 setup with 16 750 GB Hitachi Ultradrives. The test server was connected by a single 1 GbE connection while the up to 9 clients were connected via 10 GbE to the network. Clients would randomly download any of the created files along with the md5 file for comparison (done on the client), in total 2502 data and 2502 md5 files were downloaded each run (~5GB volume). web server host was limited to 1000MB memory.

raidlevel: 6 10
deadline: default optimized default optimized
nginx worker: 1 1 2 4 8 1 1 4 8
num clients (run time in s, 5009 MB downloaded each time)
1 434 225 237 225 226 382 246 252 244
2 380 180 177 166 153 346 204 174 159
3 388 183 164 136 122 334 207 141 132
4 375 174 133 115 106 323 198 119 106
5 372 178 117 106 97 154 190 108 97
6 375 160 110 99 89 395 189 105 89
7 373 158 105 92 91 310 177 93 85
8 378 149 102 102 82 315 171 87 79
9 375 158 99 84 82 274 89 83 76
labels r6-def-1 r6-opt-1 r6-opt-2 r6-opt-4 r6-opt-8 r10-def-1 r10-opt-1 r10-opt-4 r10-opt-8

Write tests

Here we created a data set with 32 directories and 256 files each. Each file again 2050kB large along with a small md5 file.

number of parallel writes raid6 raid10
1 420 406
2 122 111
4 55 50
8 24 15

conclusion

RAID10 seems to be marginally better in these synthetic tests, as we have extensive experience with such a server under RAID6, we will opt for RAID10 for next one. -- CarstenAulbert - 11 Sep 2012
Topic revision: r1 - 11 Sep 2012, CarstenAulbert
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback