SGI Announces Next-Generation Altix ICE Scale-Out Supercomputer

SGI, a global leader in HPC and datacenter solutions, today announced the immediate availability of Altix ICE 8400, the next generation of the award-winning Altix ICE scale-out high performance computing (HPC) blade platform. SGI also announced today it has submitted world record breaking SPEC benchmark performance results with Altix ICE 8400, demonstrating its readiness to implement multi-petascale supercomputers.

Altix ICE 8400 contains significant enhancements over prior Altix ICE generations, including through and through Quad Data Rate InfiniBand interconnect. SGI's elegantly-designed integrated backplane sports up to three times the link-to-node bandwidth relative to competitive QDR InfiniBand clusters to maximize performance where most job traffic occurs. Altix ICE 8400 supports up to 128 processors (1,536 cores) per cabinet with support for 130W CPU sockets. Three on-board Mellanox ConnectX-2 InfiniBand HCA compute blade configurations are also supported, and include single-port, dual-port or two single-port chipsets.

Altix ICE 8400, with its innovative blade design, easily and affordably scales to up to 65,536 compute nodes with integrated single or dual plane InfiniBand backplane interconnect. Open x86 architecture makes it equally simple to deploy commercial, open source or custom applications on completely unmodified Novell SUSE or Red Hat Linux operating systems.

Altix ICE 8400 easily meets the needs of the world's largest supercomputing deployments. Recognized for its design win at the NASA Advanced Supercomputing (NAS) facility at Ames Research Center, SGI helps Pleiades Supercomputer, the world's largest InfiniBand cluster, scale with an additional 32 cabinets of Altix ICE 8400 to nearly a petaflop. The system fully leverages SGI's hypercube topology to enable seamless cabinet-level upgrades without any production downtime, saving millions of core hours in the process.

(Full version of this article can be obtained from HPCwire's web pages)

Ceph: The Distributed File System Creature from the Object Lagoon

The last two years have seen a large number of file systems added to the kernel with many of them maturing to the point where they are useful, reliable, and in production in some cases. In the run up to the 2.6.34 kernel, Linus recently added the Ceph client. What is unique about Ceph is that it is a distributed parallel file system promising scalability and performance, something that NFS lacks.

High-level view of Ceph

One might ask about the origin of Ceph since it is somewhat unusual. Ceph is really short for Cephalopod which is the class of moulluscs to which the octopus belongs. So it’s really short for octopus, sort of. If you want more detail, talk a look at the Wikipedia article about Ceph. Now that name has been partially explained, let’s look at the file system.

Ceph was started by Sage Weil for his PhD dissertation at the University of California, Santa Cruz in the Storage Systems Research Center in the Jack Baskin School of Engineering. The lab is funded by the DOE/NNSA involving LLNL (Lawrence Livermore National Labs), LANL (Los Alamos National Labs), and Sandia National Laboratories. He graduated in the fall of 2007 and has kept developing Ceph. As mentioned previously, his efforts have been rewarded with the integration of the Ceph client into the upcoming 2.6.34 kernel.

The design goals of Ceph are to create a POSIX file system (or close to POSIX) that is scalable, reliable, and has very good performance. To reach these goals Ceph has the following major features:

• It is object-based
• It decouples metadata and data (many parallel file systems do this as well)
• It uses a dynamic distributed metadata approach

These three features and how they are implemented are at the core of Ceph (more on that in the next section).

However, probably the most fundamental core assumption in the design of Ceph is that large-scale storage systems are dynamic and there are guaranteed to be failures. The first part of the assumption, assuming storage systems are dynamic, means that storage hardware is added and removed and the workloads on the system are changing. Included in this assumption is that it is presumed there will be hardware failures and the file system needs to adaptable and resilient.

(Full version of this article can be obtained from Linux Magazine's web pages)

New Cray OS Brings ISVs in for a Soft Landing

Cray has never made a big deal about the custom Linux operating system it packages with its XT supercomputing line. In general, companies don't like to tout proprietary OS environments since they tend to lock custom codes in and third-party ISV applications out. But the third generation Cray Linux Environment (CLE3) that the company announced  is designed to make elite supercomputing an ISV-friendly experience.

Besides adding compatibility to off-the-shelf ISV codes, which we'll get to in a moment, the newly-minted Cray OS contains a number of other enhancements. In the performance realm, CLE3 increases overall scalability to greater than 500,000 cores (up from 200,000 in CLE2), adds Lustre 1.8 support, and includes some advanced scheduler features. Cray also added a feature called "core specialization," which allows the user to pin a single core on the node to the OS and devote the remainder to application code. According to Cray, on some types of codes, this can bump performance 10 to 20 percent. CLE3 also brings with it some additional reliability features, including NodeKARE, a diagnostic capability that makes sure jobs are running on healthy nodes.

But the biggest new feature added to CLE3 is compatibility with standard HPC codes from independent software vendors (ISVs). This new capability has the potential to open up a much broader market for Cray's flagship XT product line, and further blur the line between proprietary supercomputers and traditional HPC clusters.

Cray has had an on-again off-again relationship with HPC software vendors. Many of the established ISVs in this space grew up alongside Cray Research, and software from companies like CEI, LSTC, SIMULIA, and CD-adapco actually ran on the original Cray Research machines. Over time, these vendors migrated to standard x86 Linux and Windows systems, which became their prime platforms, and dropped products that required customized solutions for supercomputers. Cray left most of the commercial ISVs behind as it focused on high-end HPC and custom applications.


Programming Environment of CLE
The CLE programming environment includes tools designed to complement and enhance each other, resulting in a rich, easy-to-use programming environment that facilitates the development of scalable applications.

• Parallel programming models: MPI, SHMEM, UPC, OpenMP, and Co-Array Fortran within the node
• MPI 2.0 standard, optimized to take advantage of the scalable interconnect in the Cray XT system
• Various MPI libraries supported under Cluster Compatibility Mode
• Optimized C, C++, UPC, Fortran90, and Fortran 2003 compilers
• High-performance optimized math libraries of BLAS, FFTs, LAPACK, ScaLAPACK, SuperLU, and Cray Scientiific Libraries
• Cray Apprentice2 performance analysis tools 

(Full version of this article can be obtained from HPCwire's web pages)