TerraWulf II - Technical Specifications

The innards of TII: 96 blade nodes.

• Introduction

Terrawulf II is the new High Performance Computing cluster in the Earth Physics area. Its role is to solve large complex computational problems in the Earth Sciences using parallel processing techniques.

• Hardware

TerraWulf II is a cluster of 96 IBM x3455 compute nodes and one IBM x3655 head node.

• Computational Elements

  • Head Node

    • The server node is an IBM System x3655 with :
    • 2 AMD Opteron Dual-core 2.8 GHz Processors
    • 9GB ECC PC2-5300 DDR2 Memory
    • 1.5 TB SAS disks, 5x300Gb in RAID5 with hot-swap
    • Integrated dual Gigabit Ethernet with TCP/IP Offload Engine (TOE)
    • Voltaire Infiniband card on PCI-E slot
    • 7TB IBM DS3200 storage system - 12x 750GB SATAII in Raid5 with dual SAS controllers and dual path to server.

  • Compute Nodes:

    • Each node is an IBM System x3455 built-up with :
    • 2 AMD Opteron Dual-core 2.8 GHz processors
    • 160 GB SATA Hard Disk - 9GB (or 17GB for 24 nodes) ECC PC2-5300 DDR2 Memory
    • Integrated dual Gigabit Ethernet
    • Voltaire Infiniband card on PCI-E slot (for 48 nodes) 48 nodes also have an Infiniband card and half of those nodes have 17 GB of RAM

Network

All the nodes are interconnected through three SMC8848 Gigabit ethernet switches. In addition, half of the cluster (48 nodes) are also inter-connected via three 24port Voltaire ISR9024S Infiniband switches providing 10Gbit inter-process communication.

• Software

The head node runs SUSE Linux Enterprise Server 10 and the compute nodes are configured with Open SUSE 10.3. The resource manager used is TORQUE and
the parallel shell is PDSH. Two kinds of MPI environment have been installed, MPICH2 and VLTMPI.  The current FORTRAN compiler is Intel Fortran 10.1The cluster is monitored with GANGLIA


Help & support Online resources:

MPICH2 : http://www.mcs.anl.gov/research/projects/mpich2/
VLTMPI: http://mvapich.cse.ohio-state.edu/
TORQUE: http://www.clusterresources.com/products/torque/docs/torqueadmin.shtml
PDSH: http://www.llnl.gov/linux/pdsh.html
Ganglia: http://ganglia.sourceforge.net/

 

Hardware Diagram: Click to Enlarge



---

¹Beowulf is a design for high-performance parallel computing clusters on inexpensive personal computer hardware