|
About NCSAbench
The Performance Engineering and Computational Methods
(PECM) group tested a combination of applications and
kernels on NCSA’s
production machines. These
applications and kernels are part of NCSA’s benchmark
suite known as the NCSAbench. The NCSAbench has a dual
role: to serve as a component in the acceptance criteria
and to serve as a reference point when assessing the
relative performance of new systems.
All NCSAbench runs were executed on dedicated systems.
We
used the single processor per node option to maximize
the memory usage for an application, per node. When
feasible, an application can utilize more than 80%
of a node’s memory. We ran scalability tests
by fixing the problem size and increasing the number
of processors. In some cases, we also ran scalability
tests in which we increased the problem size as we
increased the number of processors.
The following performance metrics were derived from
these benchmarks: GFLOPS (billions of floating points
operations per second), % of peak (percent of peak
GFLOPS possible on a machine), parallel efficiency
(the ratio of the speed-up to the number of processors),
memory rate (the rate of accessing memory during a
run measured in GB/s or billions of bytes per second),
and wall clock time measured in hours. The performance
metrics are indexed by either a machine name or by
an application/kernel name.
We used NCSA’s PerfSuite, an open-source software
performance analysis tool, to collect performance data
on Intel-based systems. On IBM systems, we used HPM
Toolkit to measure performance
and collect data. Both tools are automatic, easy to
use, and require no code instrumentation.
The benchmarking suite will evolve to include applications
that are typical of NCSA users' codes that consume large
amounts of computer resources, as kernels that isolate
key components for general perfromance understanding.
|
