18 March 2013
ClusterVision Completes Top500 Class HPC Cluster at the University of Paderborn
ClusterVision has worked as prime contractor to the University of Paderborn to design and build its latest generation high performance computing (HPC) cluster. The 200 Tflops cluster system, which is located at the Paderborn Center for Parallel Computing (PC²), contains 614 compute nodes with 10,000 cores, and incorporates the latest GPU and many-core acceleration technologies available. The system will run a wide range of commercial and open source HPC applications in technology and science. It will also provide a foundation for the University’s fundamental research and development projects in advanced computational design and performance.
Paderborn Germany, 18 March, 2013— ClusterVision, Europe’s dedicated specialist in high performance computing solutions, has announced the successful completion of a new high performance computing (HPC) cluster system at the University of Paderborn. The 10,000 core, 200 Tflops system, which is part of a 4 million Euro HPC systems and infrastructure investment by the University of Paderborn, and its partners Bielefeld University, Hamm-Lippstadt University of Applied Sciences East Westphalia, Lippe University of Applied Sciences, and Bielefeld University of Applied Sciences, is the most recent HPC cluster at Paderborn and is anticipated to claim a high ranking position in the next publication of the Top500 World supercomputer listings.
Originally founded in 1614, the University of Paderborn is one of the oldest academic institutions in Germany. It was re-established in 1972 as Universität-Gesamthochschule (Institute for Higher Education), and subsequently transformed into the Universität Paderborn in its own right in 2002. Today the University is home to 18,800 students offering over 60 under-graduate and post-graduate degrees in the faculties of Arts, Humanities, Business and Sciences.
The University of Paderborn's HPC Cluster System
The new HPC cluster is located at the Paderborn Center for Parallel Computing (known as PC²). PC² is an integral interdisciplinary institute of the University of Paderborn, specialising in distributed and parallel high performance computing. Staff and students at PC² work alongside collaboration partners to investigate a range of high performance computing topics in research, development and emerging practical applications.
“This system is a powerful compute resource for all researchers in the region of East Westphalia and Lippe, and our partners in Germany and Europe,” Prof.Dr.Holger Karl, head of the PC² board.
Since its foundation in 1991 the PC² facility has housed several generations of the University’s HPC technology, including several Top500 ranked systems. PC² currently has 5 HPC systems many of which are used to develop methods and principles for the future construction and efficient use of distributed and parallel computer systems. The new HPC cluster system will offer approximately 10,000 Intel Xeon processor cores arranged in 614 compute nodes, giving an anticipated performance in the range of 200 Tflops. A combination of 32 NVIDIA K20 GPUs and 8 Intel Xeon Phi co-processors provide 40 Tflops, giving it a high top-half ranking by the standards of the current Top500 listing.
The Paderborn HPC cluster will run a range of commercial and open source scientific applications. Manufacturing and engineering applications include the popular multi-physics Finite Element Analysis (FEA) codes ANSYS, Abaqus, and COMSOL, and the Computational Fluid Dynamics (CFD) applications Fluent, StarCD and OpenFOAM. Technical applications also include the commercial code MATLAB and its open source equivalent Octave, both used for performing and displaying numerical computations in engineering and mathematics. The application tools from Xilinx and Altera are used in integrated circuit design to program the hardware description language (HDL) for field programmable gate arrays (FPGA). The Paderborn cluster will also host a variety of popular applications for computational Physics and Chemistry. These include the molecular modelling codes Gromacs, Gaussian, MOE, ORCA, TurboMole, VASP, VMD and Siesta.
In addition to running HPC applications PC² works with the European Commission and other partners to conduct fundamental research and development in a number of HPC related areas. These include performance acceleration, fault tolerant middleware and system software, and network virtualisation.
Performance acceleration using custom machines and many-cores architecture is one of the forefront research projects at PC². Custom computing machines use massively parallel, programmable hardware, for example field-programmable gate arrays (FPGA), to build processing units that are highly tailored to specific applications. FPGA based systems have been shown to be able to accelerate computationally intensive applications by orders of magnitude over traditional architectures. Alongside FPGA’s, researchers at PC² are also exploring the use of a number of other acceleration technologies. These include many-core architectures, GP-GPU coprocessors and floating point arrays. In designing the new Paderborn system, ClusterVision therefore worked closely with staff at PC² to incorporate some of the very latest acceleration technologies available.
ClusterVision's Engineering Production team on-site at the University of Paderborn
As primary contractor for the design and build of the Paderborn cluster, ClusterVision brought together the technology and expertise of a large number of HPC partners, including ASUS, Intel, Dell, Supermicro, NVIDIA, Knürr Emerson, Bright Computing, and the Fraunhofer Institute.
The Paderborn system configuration comprises 614 compute nodes. Of these 552 are ASUS E7 rack-mount compute servers, each with 2 Intel Xeon E5-2670 (16 cores) 2.6GHz processors. The smaller compute nodes have 64 GByte of main memory per node, whilst the larger nodes are enhanced with 256 GByte of main memory. The remaining nodes are 40 x ES4000 (32 GPU + 8 Phi) and 20 Intel Servers (256G).
Compute power is significantly enhanced with 40 GPU nodes arranged in 2 different configurations. Type-1 enhances the Intel Xeon E5-2670 processors with 32 NVIDIA Tesla K20 GPUs. Type-2 takes the same arrangement but uses 8 of Intel’s own Xeon Phi co-processors to achieve the required acceleration. Running alongside other PC² systems this novel arrangement allows researchers to optimally run a range of applications, and specifically to investigate the relative performance of the leading HPC acceleration technologies which are available today.
The system also has 2 SMP nodes, using Dell PowerEdge R820 servers with four 8-Core Xeon E5-4650 processors, and 6 front-end and management and administration nodes in arranged in Supermicro Superserver 825TQ-R740WB and 1027GR-TRFchassis. The interconnect system is a fast QDR InfiniBand, 40 Gbit/s system from Mellanox Technologies.
All of the systems components are housed in 14 42U Emerson/Knürr server racks, 12 of which incorporate Knürr’sbackdoor chilled water cooling technology.
The software components of the cluster include Bright Cluster Manager from Bright Computing, which is used to provide the provisioning and administrative management of the system, and FraunhoferFS (FhGFS) the parallel file-system from the Fraunhofer Institutefor Technological and Industrial Mathematics (ITWM).
The size and complexity of the new Paderborn system required a high level of expertise in engineering assembly and on-site integration. ClusterVision began off-site engineering the assembly components at their headquarters in Amsterdam in November 2012, followed by an intensive 6-man on-site build at the end of the year. The build and software installation process culminated in a program of system provisioning, burn-in testing, and benchmark tuning, with acceptance and handover being successfully completed in early February 2013. Post-delivery services include user support and maintenance for the Bright Cluster Manager and Fraunhofer FhGFS software installations and multi-year warranty and repair provisions for the critical hardware components.
“It is always exciting for our company to work on Top500 class systems like the new HPC cluster at Paderborn. Large scale, complex systems like this understandably represent a showcase of possibility to the HPC community, both in academia and commercial enterprise, and enable our team to draw upon and demonstrate all of their experience in system design and their expertise in build and configuration,” Christopher Huggins, Commercial Director at ClusterVision.
ClusterVision were proud guests at the official opening of the new cluster which was held after the meeting of the ZKI Supercomputing Working Group at the University of Paderborn on 14/15 March 2013. The inauguration ceremony was attended by staff and students from Paderborn University and members of the PC² Management and Advisory boards. The proceedings included an opening address by Prof. Dr. HolgerKarl, head of the board of PC², and greetings from Prof. Dr. Nikolaus Risch, president of the University of Paderborn, Dr. Oliver Hermann, president of the East Westphalia-Lippe University of Applied Sciences, Prof. Dr. Friedrich Biegler-King, Vice President for Planning and Infrastructure of the Bielefeld University of Applied Sciences and Prof. Dr. Gudrun Oevel, CIO of the University of Paderborn. The opening ceremony also showcased some of the scientific applications in system design and nano-engineering simulation through presentations from Prof. Dr. Volker Lindenstruth of the Goethe University Frankfurt and Prof. Dr.-Ing. Jadran Vrabec of the University of Paderborn.
“The engineering sciences must take full advantage of exponentially increasing computing power. The new HPC cluster in Paderborn offers excellent opportunities for research and development,” Prof.Dr.-Ing. Jadran Vrabec, member of the PC² board.
ClusterVision and The University of Paderborn will also collaborate at this year’s International Supercomputing Conference (ISC’13) which will take place in Leipzig, Germany in June. Visitors to the ClusterVision booth and Exhibitor Forum at ISC’13 will be able to see a presentation of the new Paderborn cluster and hear more on some of its innovative research applications.
About The University of Paderborn
The University of Paderborn has developed rapidly since its beginnings. The University counts about 18,800 students today with an upward trend. In its capacity as “The University of the Information Society”, Paderborn aims to promote developments in society by advancing the natural sciences and technology, whilst always maintaining a critical eye. There are five faculties: I Arts and Humanities, II Business Administration, III Science IV Mechanical Engineering V Computer Science, Electrical Engineering and Mathematics.
With more than 2,100 staff, the University of Paderborn is one of the region’s major employers. Some 200 professors work there and women currently account for 31% of the professorships held here, meaning that Paderborn has headed the rankings in the state of North Rhine Westphalia since 2003.
ClusterVision specialises in the design, deployment and operation of High Performance Compute (HPC) clusters. Compute clusters are used by pioneering organisations in academia and industry to run high-intensity computing in fields such as scientific research and development, manufacturing, healthcare and finance. By combining cutting-edge hardware and software components with a range of customised professional services, ClusterVision helps its customers create top-quality, efficient and reliable HPC solutions. In addition to systems technologies from leading manufacturers, ClusterVision's solutions typically include a range of HPC software components, such as easy to use cluster provisioning, management and monitoring. ClusterVision offers a full end-to-end portfolio of professional services - from system design, assembly and certification, to operational management, support, and training. With a background in applied scientific research, and practical experience in a wide range of HPC technologies, the ClusterVision team has designed and built some of the largest and most complex computational, storage and database clusters in Europe.
For more information
Dr. Tony Kent
Marketing & PR Manager
Tel: +44(0)7795 44 1655
pr [at] clustervision [dot] com
Images courtesy of AWP, Andy Wallis Photography