DOI: https://doi.org/10.26089/NumMet.v18r319

Application of block low-rank matrices in Gaussian processes for regression

Authors

  • D.А. Sushnikova

Keywords:

Gaussian processes
H² matrix
sparse matrix
Cholesky factorization

Abstract

The Gaussian processes for regression are considered. During simulation of correlated noises using the Gaussian processes, the main difficulty is the computation of the posterior mean and dispersion of the prediction. This computation requires the inversion of the dense covariance matrix of order n, where n is the sample size. In addition, for the likelihood evaluation we need to compute the logarithm of the determinant of the dense covariance matrix, which is also a time-consuming problem. A new method for the fast computation of the covariance matrix logarithm is proposed. This method is based on the approximation of this matrix by a sparse matrix. The proposed method appears to be time efficient compared to the HODLR (Hierarchically Off-Diagonal Low-Rank) method and the traditional dense method.

New computational technologies

Downloads

Published

2017-06-12

Issue

Vol. 18 (2017): Issue 3.

Section

Section 1. Numerical methods and applications

Author Biography

D.А. Sushnikova

Skolkovo Institute of Science and Technology
• Research Assistant

References

  1. S. Ambikasaran and E. Darve, “An 𝒪(𝒩 log 𝒩) Fast Direct Solver for Partial Hierarchically Semi-Separable Matrices,” J. Sci. Comput. 57 (3), 477-501 (2013).
  2. S. Ambikasaran, D. Foreman-Mackey, L. Greengard, et al., Fast Direct Methods for Gaussian Processes arXiv preprint: 1403.6015 [math.NA] (Cornell Univ. Library, Ithaca, 2014),
    available at
    https://arxiv.org/abs/1403.6015/.
  3. C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning (MIT Press, Cambridge, 2006).
  4. D. Sushnikova and I. Oseledets, Simple Non-Extensive Sparsification of the Hierarchical Matrices arXiv preprint: 1705.04601v1 [math.NA] (Cornell Univ. Library, Ithaca, 2017),
    available at
    https://arxiv.org/abs/1705.04601/.
  5. T. A. Davis and W. W. Hager, “Dynamic Supernodes in Sparse Cholesky Update/Downdate and Triangular Solves,” ACM Trans. Math. Software 35 (4), 1-23 (2009).
  6. H. G. Lalgudi, A. Bilgin, M. W. Marcellin, et al., “Four-Dimensional Compression of fMRI Using JPEG2000,” SPIE Proc. 5747, 1028-1037 (2005).
  7. S. Ambikasaran, D. Foreman-Mackey, L. Greengard, et al., Fast Direct Methods for Gaussian Processes and the Analysis of NASA Kepler Mission Data arXiv preprint: 1403.6015v2 [math.NA] (Cornell Univ. Library, Ithaca, 2015), available at
    https://arxiv.org/abs/1403.6015/.
  8. NumPy Package.
    http://
    http://www.numpy.org . Cited June 7, 2017.
  9. M. Belyaev, E. Burnaev, and E. Kapushev, “Computationally Efficient Algorithm for Gaussian Process Regression in Case of Structured Samples,” Zh. Vychisl. Mat. Mat. Fiz. 56 (4), 507-522 (2016) [Comput. Math. Math. Phys. 56 (4), 499-513 (2016)].
  10. E. Snelson and Z. Ghahramani, “Sparse Gaussian Processes Using Pseudo-Inputs,” in Proc. 18th Int. Conf. on Advances in Neural Information Processing Systems, Vancouver, Canada, December 05-08, 2005 (MIT Press, Cambridge, 2006), Vol. 18, pp. 1257-1264.
  11. Y. Zhang, W. E. Leithead, D. J. Leith, and L. Walshe, “Log-Det Approximation Based on Uniformly Distributed Seeds and its Application to Gaussian Process Regression,” J. Comput. Appl. Math. 220 (1-2), 198-214 (2008).
  12. W. E. Leithead, Y. Zhang, and D. J. Leith, “Efficient Gaussian Process Based on BFGS Updating and Logdet Approximation,” IFAC Proc. Volumes 38 (1), 1305-1310 (2005).

License

Copyright (c) 2017 Вычислительные методы и программирование

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.