DOI: https://doi.org/10.26089/NumMet.v22r102

Developing a model for holistic workload analysis of large supercomputer systems

Authors

  • P.A. Shvets
  • V.V. Voevodin
  • S.A. Zhumatiy

Keywords:

high-performance computing
supercomputer
workload analysis
application performance
model development

Abstract

Any modern supercomputer has an extremely complex architecture, and efficient usage of its resources is often a very difficult task, even for experienced users. At the same time, the field of high-performance computing is becoming more and more in demand, so the issue of efficient utilization of supercomputers is very urgent. Therefore, users should know everything important about performance of their jobs running on a supercomputer in order to be able to optimize them, and administrators should be able to monitor and analyze all the nuances of the efficient functioning of such systems. However, there is currently no complete understanding of what data are best to be studied (and how it should be analyzed) in order to have a whole picture of the state of the supercomputer and the processes taking place there. In this paper, we make our first attempt to answer this question. To do this, we are developing a model that describes all the potential factors that may be important when analyzing the performance of supercomputer applications and the HPC system as a whole. The paper provides both a detailed description of this model for users and administrators and some interesting real-life examples discovered on the Lomonosov-2 supercomputer using a software implementation based on the proposed model.

Downloads

Published

2021-03-03

Issue

Vol. 22 (2021): Issue 1.

Section

Parallel software tools and technologies

Author Biographies

P.A. Shvets

Lomonosov Moscow State University,
Research Computing Center
• Programmer

V.V. Voevodin

Lomonosov Moscow State University,
Research Computing Center
• Senior Researcher

S.A. Zhumatiy

Lomonosov Moscow State University,
Research Computing Center
• Leading Researcher

References

  1. E. Joseph and S. Conway, “Major Trends in the Worldwide HPC Market,”
    https://hpcuserforum.com/ presentations/stuttgart2017/IDC-update-HLRS.pdf . Cited January 29, 2021.
  2. V. V. Voevodin, A. S. Antonov, D. A. Nikitenko, et al., “Supercomputer Lomonosov-2: Large Scale, Deep Monitoring and Fine Analytics for the User Community,” Supercomput. Front. Innov. 6 (2), 4-11 (2019).
    doi 10.14529/jsfi190201
  3. D. A. Nikitenko, P. A. Shvets, and V. V. Voevodin, “Why do Users Need to Take Care of Their HPC Applications Efficiency?,” Lobachevskii J. Math. 41 (8), 1521-1532 (2020).
    doi 10.1134/S1995080220080132
  4. Intel VTune Amplifier Documentation.
    https://software.intel.com/en-us/vtune . Cited January 29, 2021.
  5. N. Nethercote and J. Seward, “Valgrind: A Framework for Heavyweight Dynamic Binary Instrumentation,” SIGPLAN Not. 42 (6), 89-100 (2007).
    doi 10.1145/1273442.1250746
  6. M. Geimer, F. Wolf, B. J. N. Wylie, et al., “The Scalasca Performance Toolset Architecture,” Concurr. Comput. Pract. Exp. 22 (6), 702-719 (2010).
    doi 10.1002/cpe.1556
  7. Vampir Framework Home Page.
    https://vampir.eu . Cited January 29, 2021.
  8. Arm Forge | Cross Platform Parallel Debugger for C++ and Cuda.
    https://www.arm.com/products/ development-tools/server-and-hpc/forge . Cited January 29, 2021.
  9. TotalView Debugger for HPC Computing.
    https://totalview.io . Cited January 29, 2021.
  10. M. D. Jones, J. P. White, M. Innus, et al., Workload Analysis of Blue Waters , arXiv preprint: 1703.00924v1 [cs.DC] (Cornell Univ. Library, Ithaca, 2017).
    https://arxiv.org/abs/1703.00924 . Cited January 29, 2021.
  11. A. Brian et al., 2014 NERSC Workload Analysis.
    https://www.yumpu.com/en/document/read/55341970/2014-nersc-workload-analysis . Cited January 29, 2021.
  12. N. A. Simakov, J. P. White, R. L. DeLeon, et al., A Workload Analysis of NSF’s Innovative HPC Resources Using XDMoD , arXiv preprint: 1801.04306v1 [cs.DC] (Cornell Univ. Library, Ithaca, 2018).
    https://arxiv.org/abs/1801.04306 . Cited January 29, 2021.
  13. M. J. Abraham, T. Murtola, R. Schulz, et al., “GROMACS: High Performance Molecular Simulations through Multi-Level Parallelism from Laptops to Supercomputers,” SoftwareX 1-2}, 19-25 (2015).
    doi 10.1016/j.softx.2015.06.001
  14. J. C. Phillips, R. Braun, W. Wang, et al., “Scalable Molecular Dynamics with NAMD,” J. Comput. Chem. 26 (16), 1781-1802 (2005).
    doi 10.1002/jcc.20289
  15. J. Hafner, “Ab-initio Simulations of Materials Using VASP: Density-Functional Theory and Beyond,” J. Comput. Chem. 29 (13), 2044-2078 (2008).
    doi 10.1002/jcc.21057
  16. P. A. Shvets and V. V. Voevodin, “’Endless’ Workload Analysis of Large-Scale Supercomputers,” Lobachevskii J. Math. 42 (1) [in press].
  17. Redash Homepage.
    https://redash.io . Cited January 29, 2021.
  18. P. Shvets, V. Voevodin, and D. Nikitenko, “Approach to Workload Analysis of Large HPC Сenters,” in Communications in Computer and Information Science (Springer, Cham, 2020), Vol. 1263, pp. 16-30.
  19. J. Hutter, M. Iannuzzi, F. Schiffmann, and J. VandeVondele, “cp2k: Atomistic Simulations of Condensed Matter Systems,” Wiley Interdiscip. Rev. Comput. Mol. Sci. 4 (1), 15-25 (2014).
    doi 10.1002/wcms.1159