DOI: https://doi.org/10.26089/NumMet.v25r211

Performance analysis methodology of deep neural networks inference on the example of an image classification problem

Authors

  • Murad R. Alibekov
  • Natalia E. Berezina
  • Evgenii P. Vasiliev
  • Ivan B. Vikhrev
  • Yulia D. Kamelina
  • Valentina D. Kustikova
  • Zoya A. Maslova
  • Ivan S. Mukhin
  • Alexandra K. Sidorova
  • Vladislav N. Suchkov

Keywords:

deep learning
neural networks
inference
performance
MobileNetV2
Deep Learning Inference Benchmark

Abstract

Deploying of deep neural networks requires inference performance analysis on the target hardware. Performance results are aimed to be used as motivation to evaluate a decision for deployment, find the best performing hardware and software configurations, decide is there’s a need for optimization of DL model and DL inference software. The paper describes a technique for analyzing and comparing inference performance using an example of image classification problem: converting a trained model to the formats of different frameworks, quality analysis, determining optimal inference execution parameters, model optimization and quality reanalysis, analyzing and comparing inference performance for the considered frameworks. Deep Learning Inference Benchmark Tool is aimed to support the performance analysis cycle. The technique is implemented on the example of the MobileNetV2 model.

Downloads

Published

2024-04-10

Issue

Vol. 25 (2024): Issue 2.

Section

Methods and algorithms of computational mathematics and their applications

Author Biographies

Murad R. Alibekov

Lobachevsky State University of Nizhni Novgorod
• Student

Natalia E. Berezina

LLС "Yadro Reserch and Development Center"
• AI Benchmarking Team Leader

Evgenii P. Vasiliev

Lobachevsky State University of Nizhni Novgorod
• Junior Researcher

Ivan B. Vikhrev

LLС "Yadro Reserch and Development Center"
• AI software engineer

Yulia D. Kamelina

LLС "Yadro Reserch and Development Center"
• AI software engineer

Valentina D. Kustikova

Lobachevsky State University of Nizhni Novgorod
• Senior Researcher

Zoya A. Maslova

LLС "Yadro Reserch and Development Center"
• AI framework Senior Engineer

Ivan S. Mukhin

Lobachevsky State University of Nizhni Novgorod
• Student

Alexandra K. Sidorova

Lobachevsky State University of Nizhni Novgorod
• Student

Vladislav N. Suchkov

Lobachevsky State University of Nizhni Novgorod
• Student

References

  1. Open Model Zoo for OpenVINO Toolkit.
    https://docs.openvino.ai/latest/model_zoo.html . Cited March 22, 2024.
  2. GluonCV Model Zoo.
    https://cv.gluon.ai/model_zoo/index.html . Cited March 22, 2024.
  3. Deep Learning Inference Benchmark.
    https://github.com/itlab-vision/dl-benchmark . Cited March 22, 2024.
  4. V. Kustikova, E. Vasiliev, A. Khvatov, et al., “DLI: Deep Learning Inference Benchmark,” in Communications in Computer and Information Science (Springer, Cham, 2019), Vol. 1129, pp. 542-553.
    doi 10.1007/978-3-030-36592-9_44
  5. A. K. Sidorova, M. R. Alibekov, A. A. Makarov, et al., “Automating the Collection of Deep Neural Network Inference Performance Metrics in the Deep Learning Inference Benchmark System,” in Proc. XXI Int. Conf. on Mathematical Modeling and Supercomputer Technologies, Nizhny Novgorod, Russia, November 22-26, 2021 (Nizhny Novgorod University Press, Nizhny Novgorod, 2021), pp. 318-325 [in Russian].
  6. DLI: Deep Learning Inference Benchmark.
    https://hpc-education.unn.ru/dli-ru . Cited March 22, 2024.
  7. Deep Learning Inference Benchmark Wiki.
    https://github.com/itlab-vision/dl-benchmark/wiki . Cited March 22, 2024.
  8. A. Demidovskij, Yu. Gorbachev, M. Fedorov, et al., “OpenVINO Deep Learning Workbench: Comprehensive Analysis and Tuning of Neural Networks Inference,”
    https://openaccess.thecvf.com/content_ICCVW_2019/papers/SDL-CV/Gorbachev_OpenVINO_Deep_Learning_Workbench_Comprehensive_Analysis_and_Tuning_of_Neural_ICCVW_2019_paper.pdf . Cited March 22, 2024.
  9. OpenVINO Deep Learning Workbench Overview.
    https://docs.openvino.ai/latest/workbench_docs_Workbench_DG_Introduction.html . Cited March 22, 2024.
  10. Intel Distribution of OpenVINO Toolkit.
    https://docs.openvino.ai/latest/home.html . Cited March 22, 2024.
  11. O. Fagbohungbe and L. Qian, “Benchmarking Inference Performance of Deep Learning Models on Analog Devices,” in Proc. 2021 Int. Joint Conf. on Neural Networks, Shenzhen, China, July 18-22, 2021 (IEEE Press, Piscataway, 2021), pp. 1-9.
    doi 10.1109/IJCNN52387.2021.9534143
  12. P. Torelli and M. Bangale, “Measuring Inference Performance of Machine-Learning Frameworks on Edge-class Devices with the MLMark Benchmark,”
    https://www.eembc.org/techlit/articles/MLMARK-WHITEPAPER-FINAL-1.pdf . Cited March 22, 2024.
  13. EEMBC’s Machine-Learning Inference Benchmark targeted at edge devices.
    https://github.com/eembc/mlmark . Cited March 22, 2024.
  14. V. J. Reddi, C. Cheng, D. Kanter, et al., “MLPerf Inference Benchmark,”
    https://arxiv.org/abs/1911.02549 . Cited March 22, 2024.
  15. MLPerf Inference Benchmarks for Image Classification and Object Detection Tasks.
    https://github.com/mlcommons/inference . Cited March 22, 2024.
  16. E. P. Vasiliev, V. D. Kustikova, V. D. Volokitin, et al., “Performance Analysis of Deep Learning Inference in Convolutional Neural Networks on Intel Cascade Lake CPUs,” in Communications in Computer and Information Science (Springer, Cham, 2021), Vol. 1413, pp. 346-360.
    doi 10.1007/978-3-030-78759-2_29
  17. Intel Optimization for Caffe.
    https://github.com/intel/caffe . Cited March 22, 2024.
  18. TensorFlow.
    https://pypi.org/project/intel-tensorflow . Cited March 22, 2024.
  19. TensorFlow Lite.
    https://www.tensorflow.org/lite . Cited March 22, 2024.
  20. MXNet.
    https://mxnet.apache.org . Cited March 22, 2024.
  21. OpenCV.
    https://opencv.org . Cited March 22, 2024.
  22. ONNX Runtime.
    https://onnxruntime.ai . Cited March 22, 2024.
  23. M. Sandler, A. Howard, M. Zhu, et al., “MobileNetV2: Inverted Residuals and Linear Bottlenecks,”
    https://arxiv.org/abs/1801.04381 . Cited March 22, 2024.
  24. ImageNet.
    https://www.image-net.org . Cited March 22, 2024.
  25. OpenVINO Toolkit - Open Model Zoo repository.
    https://github.com/openvinotoolkit/open_model_zoo . Cited March 22, 2024.
  26. TensorFlow Hub. Mobilenet V2 trained on Imagenet.
    https://tfhub.dev/iree/lite-model/mobilenet_v2_100_224/uint8/1 . Cited March 22, 2024.
  27. Intel Launches World’s Best Processor for Thin-and-Light Laptops: 11th Gen Intel Core.
    https://www.intc.com/news-events/press-releases/detail/1411/intel-launches-worlds-best-processor-for-thin-and-light . Cited March 22, 2024.

License

Copyright (c) 2024 М. Р. Алибеков, Н. Е. Березина, Е. П. Васильев, И. Б. Вихрев, Ю. Д. Камелина, В. Д. Кустикова, З. А. Маслова, И. С. Мухин, А. К. Сидорова, В. Н. Сучков

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.