Application of block low-rank matrices in Gaussian processes for regression

Authors

DOI:

https://doi.org/10.26089/NumMet.v18r319

Keywords:

Gaussian processes, H² matrix, sparse matrix, Cholesky factorization

Abstract

The Gaussian processes for regression are considered. During simulation of correlated noises using the Gaussian processes, the main difficulty is the computation of the posterior mean and dispersion of the prediction. This computation requires the inversion of the dense covariance matrix of order n, where n is the sample size. In addition, for the likelihood evaluation we need to compute the logarithm of the determinant of the dense covariance matrix, which is also a time-consuming problem. A new method for the fast computation of the covariance matrix logarithm is proposed. This method is based on the approximation of this matrix by a sparse matrix. The proposed method appears to be time efficient compared to the HODLR (Hierarchically Off-Diagonal Low-Rank) method and the traditional dense method.

Author Biography

D.А. Sushnikova

References

  1. S. Ambikasaran and E. Darve, “An 𝒪(𝒩 log 𝒩) Fast Direct Solver for Partial Hierarchically Semi-Separable Matrices,” J. Sci. Comput. 57 (3), 477-501 (2013).
  2. S. Ambikasaran, D. Foreman-Mackey, L. Greengard, et al., Fast Direct Methods for Gaussian Processes arXiv preprint: 1403.6015 [math.NA] (Cornell Univ. Library, Ithaca, 2014),
    available at
    https://arxiv.org/abs/1403.6015/.
  3. C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning (MIT Press, Cambridge, 2006).
  4. D. Sushnikova and I. Oseledets, Simple Non-Extensive Sparsification of the Hierarchical Matrices arXiv preprint: 1705.04601v1 [math.NA] (Cornell Univ. Library, Ithaca, 2017),
    available at
    https://arxiv.org/abs/1705.04601/.
  5. T. A. Davis and W. W. Hager, “Dynamic Supernodes in Sparse Cholesky Update/Downdate and Triangular Solves,” ACM Trans. Math. Software 35 (4), 1-23 (2009).
  6. H. G. Lalgudi, A. Bilgin, M. W. Marcellin, et al., “Four-Dimensional Compression of fMRI Using JPEG2000,” SPIE Proc. 5747, 1028-1037 (2005).
  7. S. Ambikasaran, D. Foreman-Mackey, L. Greengard, et al., Fast Direct Methods for Gaussian Processes and the Analysis of NASA Kepler Mission Data arXiv preprint: 1403.6015v2 [math.NA] (Cornell Univ. Library, Ithaca, 2015), available at
    https://arxiv.org/abs/1403.6015/.
  8. NumPy Package.
    http://
    http://www.numpy.org . Cited June 7, 2017.
  9. M. Belyaev, E. Burnaev, and E. Kapushev, “Computationally Efficient Algorithm for Gaussian Process Regression in Case of Structured Samples,” Zh. Vychisl. Mat. Mat. Fiz. 56 (4), 507-522 (2016) [Comput. Math. Math. Phys. 56 (4), 499-513 (2016)].
  10. E. Snelson and Z. Ghahramani, “Sparse Gaussian Processes Using Pseudo-Inputs,” in Proc. 18th Int. Conf. on Advances in Neural Information Processing Systems, Vancouver, Canada, December 05-08, 2005 (MIT Press, Cambridge, 2006), Vol. 18, pp. 1257-1264.
  11. Y. Zhang, W. E. Leithead, D. J. Leith, and L. Walshe, “Log-Det Approximation Based on Uniformly Distributed Seeds and its Application to Gaussian Process Regression,” J. Comput. Appl. Math. 220 (1-2), 198-214 (2008).
  12. W. E. Leithead, Y. Zhang, and D. J. Leith, “Efficient Gaussian Process Based on BFGS Updating and Logdet Approximation,” IFAC Proc. Volumes 38 (1), 1305-1310 (2005).

Published

12-06-2017

How to Cite

Сушникова Д.А. Application of Block Low-Rank Matrices in Gaussian Processes for Regression // Numerical Methods and Programming (Vychislitel’nye Metody i Programmirovanie). 2017. 18. 214-220. doi 10.26089/NumMet.v18r319

Issue

Section

Section 1. Numerical methods and applications