HOPLANG: advancement of the monitoring data stream processing language

Authors

Keywords:

parallel processing, monitoring, supercomputing

Abstract

The efficiency analysis of supercomputer applications is a very actual problem. The main aim of HOPSA project is the efficiency analysis using monitoring data. Because supercomputer applications are large, the amount of monitoring data is also very large. One of the most widespread tool for large data amounts is the pair formed Hadoop and PIG. Hadoop is a very famous toolkit for the distributed data processing and PIG is a language for creating requests to Hadoop. However, it was found that this pair is not suitable for HOPSA project aims. Any request is processed for a very long time because of long compiling, the data streams are not supported, very few databases are supported, and there is no support for database indices. Thus, a special language for huge monitoring data streams is required. We develop such a language named HopLang. Just now, HopLang can process real monitoring data. This paper describes the last half-year language advance and some future plans.

Author Biographies

A.V. Adinets

Lomonosov Moscow State University,
Research Computing Center,
Ленинские горы, 119991, Москва
• Junior Researcher

P.A. Bryzgalov

Lomonosov Moscow State University,
Research Computing Center,
Ленинские горы, 119991, Москва
• Researcher

Vad.V. Voevodin

Lomonosov Moscow State University,
Research Computing Center,
Ленинские горы, 119991, Москва
• Junior Researcher

S.A. Zhumatii

Lomonosov Moscow State University,
Research Computing Center,
Ленинские горы, 119991, Москва
• Senior Researcher

D.A. Nikitenko

Lomonosov Moscow State University,
Research Computing Center,
Ленинские горы, 119991, Москва
• Junior Researcher

K.S. Stefanov

Lomonosov Moscow State University,
Research Computing Center,
Ленинские горы, 119991, Москва
• Researcher

References

  1. Адинец А.В., Брызгалов П.А., Воеводин Вад.В., Жуматий С.А., Никитенко Д.А. Об одном подходе к мониторингу, анализу и визуализации потока заданий на кластерной системе // Вычислительные методы и программирование. 2011. 12. 90-93.
  2. Hadoop homepage (http://hadoop.apache.org/).
  3. Адинец А.В., Брызгалов П.А., Жуматий С.А., Никитенко Д.А. Система визуализации параметров работы больших вычислительных систем // Тр. Междунар. конф. «Параллельные вычислительные технологии-2012 (ПаВТ-2012)’’. Новосибирск, 26-30 марта 2012 г. Челябинск: Издательский Центр ЮУрГУ, 2012. 714.
  4. Banker K. MongoDB in action. Shelter Island: Manning Publications, 2011.
  5. HopLang Project Homepage (http://github.com/zhum/hoplang).

Published

2012-10-17

How to Cite

Адинец А.В., Брызгалов П.А., Воеводин Вад.В., Жуматий С.А., Никитенко Д.А., Стефанов К.С. HOPLANG: Advancement of the Monitoring Data Stream Processing Language // Numerical methods and programming. 2012. 13. 126-131

Issue

Section

Section 2. Programming

Most read articles by the same author(s)

1 2 > >>