Application of block low-rank matrices in Gaussian processes for regression
Keywords:Gaussian processes, H² matrix, sparse matrix, Cholesky factorization
The Gaussian processes for regression are considered. During simulation of correlated noises using the Gaussian processes, the main difficulty is the computation of the posterior mean and dispersion of the prediction. This computation requires the inversion of the dense covariance matrix of order n, where n is the sample size. In addition, for the likelihood evaluation we need to compute the logarithm of the determinant of the dense covariance matrix, which is also a time-consuming problem. A new method for the fast computation of the covariance matrix logarithm is proposed. This method is based on the approximation of this matrix by a sparse matrix. The proposed method appears to be time efficient compared to the HODLR (Hierarchically Off-Diagonal Low-Rank) method and the traditional dense method.
- S. Ambikasaran and E. Darve, “An 𝒪(𝒩 log 𝒩) Fast Direct Solver for Partial Hierarchically Semi-Separable Matrices,” J. Sci. Comput. 57 (3), 477-501 (2013).
- S. Ambikasaran, D. Foreman-Mackey, L. Greengard, et al., Fast Direct Methods for Gaussian Processes arXiv preprint: 1403.6015 [math.NA] (Cornell Univ. Library, Ithaca, 2014),
- C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning (MIT Press, Cambridge, 2006).
- D. Sushnikova and I. Oseledets, Simple Non-Extensive Sparsification of the Hierarchical Matrices arXiv preprint: 1705.04601v1 [math.NA] (Cornell Univ. Library, Ithaca, 2017),
- T. A. Davis and W. W. Hager, “Dynamic Supernodes in Sparse Cholesky Update/Downdate and Triangular Solves,” ACM Trans. Math. Software 35 (4), 1-23 (2009).
- H. G. Lalgudi, A. Bilgin, M. W. Marcellin, et al., “Four-Dimensional Compression of fMRI Using JPEG2000,” SPIE Proc. 5747, 1028-1037 (2005).
- S. Ambikasaran, D. Foreman-Mackey, L. Greengard, et al., Fast Direct Methods for Gaussian Processes and the Analysis of NASA Kepler Mission Data arXiv preprint: 1403.6015v2 [math.NA] (Cornell Univ. Library, Ithaca, 2015), available at
- NumPy Package.
http://www.numpy.org . Cited June 7, 2017.
- M. Belyaev, E. Burnaev, and E. Kapushev, “Computationally Efficient Algorithm for Gaussian Process Regression in Case of Structured Samples,” Zh. Vychisl. Mat. Mat. Fiz. 56 (4), 507-522 (2016) [Comput. Math. Math. Phys. 56 (4), 499-513 (2016)].
- E. Snelson and Z. Ghahramani, “Sparse Gaussian Processes Using Pseudo-Inputs,” in Proc. 18th Int. Conf. on Advances in Neural Information Processing Systems, Vancouver, Canada, December 05-08, 2005 (MIT Press, Cambridge, 2006), Vol. 18, pp. 1257-1264.
- Y. Zhang, W. E. Leithead, D. J. Leith, and L. Walshe, “Log-Det Approximation Based on Uniformly Distributed Seeds and its Application to Gaussian Process Regression,” J. Comput. Appl. Math. 220 (1-2), 198-214 (2008).
- W. E. Leithead, Y. Zhang, and D. J. Leith, “Efficient Gaussian Process Based on BFGS Updating and Logdet Approximation,” IFAC Proc. Volumes 38 (1), 1305-1310 (2005).
How to Cite
Copyright (c) 2017 Вычислительные методы и программирование
This work is licensed under a Creative Commons Attribution 4.0 International License.