Hardware

Blue Waters will be based on IBM's multicore POWER7 processor. Each POWER7 processor will:

• Include eight high-performance cores, with each core providing 12 execution units.
• Feature simultaneous multithreading that delivers four virtual threads per core.
• Have three levels of cache—private L1 (32KB) instruction and data caches, private L2 (256KB) cache and L3 (32 MB) cache that can be used either as shared cache or separated into dedicated caches for each core—reducing latency.
• Combine the dense, low-power attributes of eDRAM with the speed and bandwidth advantages of SRAM for optimized performance and power usage.
• Have two dual-channel DDR3 memory controllers, delivering 100GB/sec of sustained bandwidth.
• Employ new IBM interconnect technology, providing a high-bandwidth, low-latency interconnect that scales to hundreds of thousands of cores.
• Have six fabric bus interfaces to connect to other cores and groups of cores, providing improved reliability.
• Provide an eight-channel memory subsystem, to enable the solution of memory-intensive problems.
• Provide 32 or more gigabytes of memory per SMP and 2 or more gigabytes of memory per core.
• Support 10 or more data streams.
• Offer vector multimedia extensions on each core with four or more floating-point operations per cycle.

The POWER7 processors will be packaged in clusters of four, forming standalone 32-core SMP nodes called multichip modules (MCMs).

Blue Waters will employ 200,000+ processor cores and will provide more than 800 terabytes of memory.

Blue Waters is projected to have as much as 500 petabytes of archival disk storage and much more than 10 petabytes of usable disk space. IBM's General Parallel File System (GPFS) and HPSS will be combined into managed file systems using the GPFS-HPSS Interface (GHI) software. The apparent disk space to which users will have direct access will be an order of magnitude greater than what is available today, with corresponding bandwidth increases. The file system and archive will be substantially larger, faster, more reliable, and easier to use than similar systems on today's platforms.

The file system will automate many storage and data transfer tasks. Researchers will have a simplified and easily searched view of their data. They will be able to set lifetime information-management policies that establish where their data are stored, how long data are kept in the faster-access file system instead of the tape-based archive, and how the data are backed up and retrieved. This is markedly more efficient than current systems where researchers must log into multiple systems, manually transferring their data, keeping track of where those data are stored, and confirming that transfers have been completed successfully. Blue Waters is also enhancing HPSS by adding Redundant Array of Inexpensive Tape functions to increase resiliency while still fully automating all data storage.

NCSA is testing the GPFS-GHI-HPSS managed file system on interim computing systems and other testbeds.

Blue Waters will be housed at the new Illinois Petascale Computing Facility; the facility's network will be capable of transferring 100 to 400 gigabits of data per second.