Sparse Gaussian Processes: inference, subspace identification and model selection
PDF GZIPGaussian Process (GP) inference is a probabilistic kernel method where the GP is treated as a latent function. The inference is carried out using the Bayesian online learning and its extension to the more general iterative approach which we call TAP/EP learning. Sparsity is introduced in this context to make the TAP/EP method applicable to large datasets. We address the prohibitive scaling of the number of parameters by defining a subset of the training data that is used as the support the GP, thus the number of required parameters is independent of the training set, similar to the case of ``Support--‘‘ or ``Relevance--Vectors‘‘. An advantage of the full probabilistic treatment is that allows the computation of the marginal data likelihood or evidence, leading to hyper-parameter estimation within the GP inference. An EM algorithm to choose the hyper-parameters is proposed. The TAP/EP learning is the E-step and the M-step then updates the hyper-parameters. Due to the sparse E-step the resulting algorithm does not involve manipulation of large matrices. The presented algorithm is applicable to a wide variety of likelihood functions. We present results of applying the algorithm on classification and nonstandard regression problems for artificial and real datasets.
| Author(s): | Csato, L. and Opper, M. |
| Links: | |
| Journal: | Proceedings |
| Pages: | 1-6 |
| Year: | 2003 |
| Month: | August |
| Day: | 0 |
| Editors: | Van der Hof, , Wahlberg |
| BibTeX Type: | Conference Paper (inproceedings) |
| Address: | The Netherlands |
| Event Name: | 13th IFAC Symposium on System Identifiaction |
| Event Place: | Rotterdam |
| Digital: | 0 |
| Electronic Archiving: | grant_archive |
| Institution: | MPI for Biological Cybernetics, Tuebingen |
| Note: | electronical version; Index ThA02-2 |
| Organization: | Max-Planck-Gesellschaft |
| School: | Biologische Kybernetik |
BibTeX
@inproceedings{2610,
title = {Sparse Gaussian Processes: inference, subspace identification and model selection},
journal = {Proceedings},
abstract = {Gaussian Process (GP) inference is a probabilistic kernel method where the GP is treated as a latent function. The inference is carried out using the Bayesian online learning and its extension to the more general iterative approach which we call TAP/EP learning.
Sparsity is introduced in this context to make the TAP/EP method applicable to large datasets. We address the prohibitive scaling of the number of parameters by defining a subset of the training data that is used as the support the GP, thus the number of required parameters is independent of the training set, similar to the case of ``Support--‘‘ or ``Relevance--Vectors‘‘.
An advantage of the full probabilistic treatment is that allows the computation of the marginal data likelihood or evidence, leading to hyper-parameter estimation within the GP inference.
An EM algorithm to choose the hyper-parameters is proposed. The TAP/EP learning is the E-step and the M-step then updates the hyper-parameters. Due to the sparse E-step the resulting algorithm does not involve manipulation of large matrices. The presented algorithm is applicable to a wide variety of likelihood functions. We present results of applying the algorithm on classification and nonstandard regression problems for artificial and real datasets.},
pages = {1-6},
editors = {Van der Hof, , Wahlberg},
organization = {Max-Planck-Gesellschaft},
institution = {MPI for Biological Cybernetics, Tuebingen},
school = {Biologische Kybernetik},
address = {The Netherlands},
month = aug,
year = {2003},
note = {electronical version; Index ThA02-2},
author = {Csato, L. and Opper, M.},
month_numeric = {8}
}