Compute cluster racks

In each compute cluster rack will be stuffed with 42 compute nodes. Each compute nodes needs:

• Height units needed: 1
• Ethernet connections needed (high speed): 1
• Ethernet connections needed (management): 1
• Power cables: 1
• Networking cables: 2
• Total Wattage: 180W

Total per rack:

• Height units needed: 42
• Ethernet connections needed (high speed): 42
• Ethernet connections needed (management): 42
• Power cables: 42
• Networking cables: 84
• Total Wattage: 7560W

Woven

• 1 TRX needed
• 42 ports will be used in 4 groups of 12 ports
• thus slight overbooking for 2 groups (11:10), line speed for the other 2 (10:10)
• 4 CX4 cables to EFX1000
• 4 ports needed on EFX1000
• might need to use only 3 uplinks for 4 racks (see below)

Foundry

• 7 fan-out cables needed
• 42 ports on RX32 needed

Rounding error

There are 4 free height units available over all the 32 racks available. Suggestion: racks 31 and 32 will only host 40 compute nodes each. No problem for the Woven layout, slight problem for the Foundry system because 4 ports will be wasted.

Storage racks

Each storage rack will house 8 file servers, 1 head node and 3 Sun X4500.

Storage & Head

Each file server:

• Height units needed: 3
• Ethernet connections needed (cluster): 2
• Ethernet connections needed (external): 2
• Ethernet connections needed (management): 1
• Power cables: 2
• Total Wattage: 475W

Each X4500:

• Height units needed: 4
• Ethernet connections needed (cluster): 1 (10GE)
• Ethernet connections needed (external): 4
• Ethernet connections needed (management): 1
• Power cables: 2
• Total Wattage: 1500W

Each head node: Each file server:

• Height units needed: 3
• Ethernet connections needed (cluster): 2
• Ethernet connections needed (external): 2
• Ethernet connections needed (management): 1
• Power cables: 2
• Total Wattage: 400W

Rack total:

• Height units needed: 39
• Ethernet connections needed (cluster): 18 + 3 (10GE)
• Ethernet connections needed (external): 30
• Ethernet connections needed (management): 11
• Power cables: 24
• Total Wattage: 8700W

Suggestion: Should put only one storage rack per rack group to guard against damage in case of smoke/fire.

Woven

• 1 TRX for rack
• only 18 connections used in 2 groups
• ideally neighboring rack should accommodate extra hardware needing GE ports
• only 2 CX4 uplinks needed (will save ports on EFX1000)
• 2 ports on EFX1000
• 3 optical fibers/3 optical ports on EFX1000

Foundry

• 3 fan-out cables needed
• 18 ports on RX32 needed
• 3 optical fibers/3 optical ports on RX32

Cisco

• 30 GE ports needed

Rounding error

Only 31 file servers are present initially, but together with file server bought previously we can fill this up to 32. Thus no rounding errors.

Network port summary

Woven

• CX4: 32 * 4 + 4 * 2 = 136 --- 4 too many(!) (11 lines cards only have 132 ports)
• Optical: 12 --- 1 line card

Foundry

• Copper: 236 fan-out cables, i.e. 1416 GE ports (30 line cards)
• Optical: 12 --- 3 line cards (problem: only 32 line card slots)