Einstein@home RAID setup testing

Over time the einstein-abp1 download server (mostly BRP project) went through various updates to increase download throughput (apache2-mpm-worker, lighttpd, nginx on the web server side, move from 8bit to 4bit sampled data, deadline scheduler optimiztion) - however we seem to hit a limit around 900k downloads a day with pre-procesing writing new files to the machine at the same speed as it's deleted, up to 1.6TB/day (2050kB files + housekeeping files). Next logical step before going to a clustered FS or similar solution is to go with more download servers, each running its own work unit generator. Going that route we also looked briefly into an alternative to the current 16 disk Areca 1261ML RAID6 setup.

Web server tests

The machine was re-installed and a test data set created (1024 download directories each with 2048 2050kB files plus an md5 file for each data file). Webserver was nginx with varying number of workers, each worker accepting up to 512 connections. Underneath was either a RAID6 or RADI10 setup with 16 750 GB Hitachi Ultradrives. The test server was connected by a single 1 GbE connection while the up to 9 clients were connected via 10 GbE to the network. Clients would randomly download any of the created files along with the md5 file for comparison (done on the client), in total 2502 data and 2502 md5 files were downloaded each run (~5GB volume). web server host was limited to 1000MB memory.

raidlevel:	6					10
deadline:	default	optimized				default	optimized
nginx worker:	1	1	2	4	8	1	1	4	8
num clients	(run time in s, 5009 MB downloaded each time)
1	434	225	237	225	226	382	246	252	244
2	380	180	177	166	153	346	204	174	159
3	388	183	164	136	122	334	207	141	132
4	375	174	133	115	106	323	198	119	106
5	372	178	117	106	97	154	190	108	97
6	375	160	110	99	89	395	189	105	89
7	373	158	105	92	91	310	177	93	85
8	378	149	102	102	82	315	171	87	79
9	375	158	99	84	82	274	89	83	76
labels	r6-def-1	r6-opt-1	r6-opt-2	r6-opt-4	r6-opt-8	r10-def-1	r10-opt-1	r10-opt-4	r10-opt-8

Write tests

Here we created a data set with 32 directories and 256 files each. Each file again 2050kB large along with a small md5 file.

number of parallel writes	raid6	raid10
1	420	406
2	122	111
4	55	50
8	24	15

conclusion

RAID10 seems to be marginally better in these synthetic tests, as we have extensive experience with such a server under RAID6, we will opt for RAID10 for next one. -- CarstenAulbert - 11 Sep 2012