Highlights of the 43rd TOP500 List

Tianhe-2 (Milky Way-2), a system developed by China’s National University of Defense Technology (NUDT) and deployed at the National Supercomputer Center in Guangzho, China remains the No. 1 system with 33.86 petaflop/s (Pflop/s) on the Linpack benchmark. The system currently has 16,000 nodes, each with two Intel Xeon Ivy Bridge processors and three Xeon Phi processors for a combined total of 3,120,000 computing cores. It features a number of Chinese-developed components, including the TH Express-2 interconnect network, front-end processors, operating system and software tools. The Tianhe-2 uses the Kylin Linux operating system. The power consumption of Tianhe-2 while running Linpack was 17.8 MW.

Other highlights from the Top 10:

  • Titan, a Cray XK7 system installed at the Department of Energy’s (DOE) Oak Ridge National Laboratory remains the No. 2 system. It achieved 17.59 Pflop/s on the Linpack benchmark using 261,632 of its NVIDIA K20x accelerator cores. Titan is one of the most energy efficient systems on the list consuming a total of 8.21 MW and delivering 2.143 Gflops/W.
  • Sequoia, an IBM BlueGene/Q system installed at DOE’s Lawrence Livermore National Laboratory, is again the No. 3 system. It was first delivered in 2011 and has achieved 17.17 Pflop/s on the Linpack benchmark using 1,572,864 cores.
  • Fujitsu’s K computer installed at the RIKEN Advanced Institute for Computational Science (AICS) in Kobe, Japan, is the No. 4 system with 10.51 Pflop/s on the Linpack benchmark using 705,024 SPARC64 processing cores.
  • Mira, a BlueGene/Q system installed at DOE’s Argonne National Laboratory, is No. 5 with 8.59 Pflop/s on the Linpack benchmark using 786,432 cores.
  • At No. 6 is Piz Daint, a Cray XC30 system installed at the Swiss National Supercomputing Centre (CSCS) in Lugano, Switzerland and the most powerful system in Europe. Piz Daint achieved 6.27 Pflop/s on the Linpack benchmark using 73,808 NVIDIA K20x accelerator cores.  Piz Daint is also the most energy efficient systems in the TOP10 consuming a total of 2.33 MW and delivering 2.7 Gflops/W.
  • Stampede, a Dell PowerEdge C8220 system installed at the Texas Advanced Computing Center of the University of Texas, Austin, is at No. 7. It also uses Intel Xeon Phi processors (previously known as MIC) to achieve its 5.17 Pflop/s.
  • The second system in Europe is at No. 8. It is also a BlueGene/Q system called JUQEEN installed at the Forschungszentrum Juelich in Germany and is listed with 5.01 Pflop/s.
  • No. 9 is taken by Vulcan, another IBM BlueGene/Q system at Lawrence Livermore National Laboratory. It was temporarily combined with the No. 3 system but is now operated independently. It achieved 4.29 Pflop/s.
  • At No. 10 is the only new system in the Top10, a Cray XC30 installed at a Government location in the USA with 3.14 Pflop/s.

Highlights from the Overall List

  • The overall list-by-list growth rates of performance are for the second time in a row at historical low values. 
  • The performance of the last system on the list (#500) has systematically lagged behind historical trends for the last 5 years and now appears to be on a different growth trajectory then before.  From 1994 to 2008 it grew by 90% per year. Since 2008 it only grows by 55% per year
  • The growth of the average performance of all systems in the list lagged only for the last two lists behind historical averages.  This average is noticeably influenced by the very large systems on the top of the list.  Recent installations of very large systems until June 2013 have counteracted the reduced growth rate at the bottom of the list.  This offers an indication that the market for the very largest systems might currently behave differently from the market of mid-sized and smaller supercomputers.
  • There are 37 systems with performance greater than a Pflop/s on the list, up from 31 six months ago.
  • The No. 1 system, Tianhe-2, and the No. 7 system, Stampede, use Intel Xeon Phi processors to speed up their computational rate. The No. 2 system Titan and the No. 6 system Piz Daint are using NVIDIA GPUs to accelerate computation.
  • A total of 62 systems on the list are using accelerator/co-processor technology, up from 53 from November 2013. Forty-four (44) of these use NVIDIA chips, two use ATI Radeon, and there are now 17 systems with Intel MIC technology (Xeon Phi).
  • The average number of accelerator cores for these 62 systems is 78,127 cores/system.
  • Intel continues to provide the processors for the largest share (85.4 percent) of TOP500 systems.
  • Ninety-six percent of the systems use processors with six or more cores and eighty-three percent use eight or more cores.
  • IBM’s BlueGene/Q is still the most popular system in the TOP10 with four entries including the No. 3, 5, 8 and 9 systems.
  • The number of systems installed in the USA has fallen to 233 down from 264 six month ago.  This is near its lowest share ever seen before (226 in the early 2000s).
  • The number of systems installed in China has grown to 76, compared to 63 on the last list.  China occupies the No. 2 position as a user of HPC, ahead of Japan, UK, France, and Germany.  China has almost as many systems installed as UK, France and Germany combined.  Due to Tianhe-2, China is also holding the No. 2 position in the performance share, ahead of Japan.

General highlights from the TOP500 since the November 2013 edition:

  • The entry level to the list moved up to the 133.7 Tflop/s mark on the Linpack benchmark, compared to 117.8 Tflop/s six months ago.
  • The last system on the newest list was listed at position 384 in the previous TOP500.  This represents the lowest turnover rate in the list in two decades.
  • Total combined performance of all 500 systems has grown to 274 Pflop/s, compared to 250 Pflop/s six months ago and 223 Pflop/s one year ago. This increase in installed performance also exhibits a noticeable slowdown in growth compared to the previous long-term trend.
  • The entry point for the TOP100 increased in six months to 390 Tflop/s from 327 Tflop/s.
  • The average concurrency level in the TOP500 is 43,301 cores per system, up from 41,434 six months ago and 38,700 one year ago.

Vendor Trends

  • A total of 427 systems (85 percent) are now using Intel processors, slightly up from 82 percent six months ago.
  • The share of IBM Power processors is stable at 38 systems (8 percent).
  • The AMD Opteron family is used in 31 systems (6 percent), down from 9 percent on the previous list.
  • InfiniBand technology is now found on 221 systems, up from 207 systems, and is the most-used internal system interconnect technology. Gigabit Ethernet stayed at 202 systems slightly down from 212 systems, in large part thanks to 75 systems now using 10G interfaces.
  • IBM and Hewlett-Packard continue to sell the bulk of the systems at all performance levels of the TOP500.
  • HP has the lead in systems and now has 182 systems (36 percent) compared to IBM with 176 systems (35 percent). HP had 196 systems (39 percent) six months ago, and IBM had 164 systems (33 percent) six months ago. In the system category, Cray remains third with 10 percent (50 systems).

Performance Trends

  • IBM remains the clear leader in the TOP500 list in performance and has a considerable lead with a 32 percent share of installed total performance (unchanged from 32 percent).
  • Thanks to Tianhe-2 and Tianhe-1A, NUDT contributes 13.7 percent of the total performance of the list, down from 15 percent.
  • Cray’s share in performance is now at 18.2 percent, up from 16.7 percent.
  • HP is again fourth, even though it increased its share to 15.6 percent from 15.3 percent. 

Geographical observations

  • The U.S. is clearly the leading consumer of HPC systems with 233 of the 500 systems (265 last time) although its share had dropped close to its all time low. The European share (116 systems compared to 102 last time) is still lower than the Asian share (132 systems, up from 116 last time).
  • Dominant countries in Asia are China with 76 systems (up from 63) and Japan with 30 systems (up from 28).  
  • In Europe, UK, France, and Germany, are almost equal with 30, 27, and 23 respectively.  China has almost as many systems installed as UK, France and Germany combined. 

About the TOP500 List

The first version of what became today’s TOP500 list started as an exercise for a small conference in Germany in June 1993. Out of curiosity, the authors decided to revisit the list in November 1993 to see how things had changed. About that time they realized they might be on to something and decided to continue compiling the list, which is now a much-anticipated, much-watched and much-debated twice-yearly event.

The TOP500 list is compiled by Erich Strohmaier and Horst Simon of Lawrence Berkeley National Laboratory; Jack Dongarra of the University of Tennessee, Knoxville; and Martin Meuer of Prometeus, Germany.