Measuring Statistical Dependence with Hilbert-Schmidt Norms
PDF
We propose an independence criterion based on the eigenspectrum of covariance operators in reproducing kernel Hilbert spaces (RKHSs), consisting of an empirical estimate of the Hilbert-Schmidt norm of the cross-covariance operator (we term this a Hilbert-Schmidt Independence Criterion, or HSIC). This approach has several advantages, compared with previous kernel-based independence criteria. First, the empirical estimate is simpler than any other kernel dependence test, and requires no user-defined regularisation. Second, there is a clearly defined population quantity which the empirical estimate approaches in the large sample limit, with exponential convergence guaranteed between the two: this ensures that independence tests based on {methodname} do not suffer from slow learning rates. Finally, we show in the context of independent component analysis (ICA) that the performance of HSIC is competitive with that of previously published kernel-based criteria, and of other recently published ICA methods.
| Author(s): | Gretton, A. and Bousquet, O. and Smola, A. and Schoelkopf, B. |
| Links: | |
| Book Title: | Algorithmic Learning Theory, Lecture Notes in Computer Science, Vol. 3734 |
| Journal: | Algorithmic Learning Theory: 16th International Conference, ALT 2005 |
| Pages: | 63-78 |
| Year: | 2005 |
| Month: | October |
| Day: | 8 |
| Editors: | S Jain and H-U Simon and E Tomita |
| Publisher: | Springer |
| BibTeX Type: | Conference Paper (inproceedings) |
| Address: | Berlin, Germany |
| DOI: | 10.1007/11564089_7 |
| Event Name: | 16th International Conference ALT 2005 |
| Event Place: | Singapore |
| Digital: | 0 |
| Electronic Archiving: | grant_archive |
| Language: | en |
| Organization: | Max-Planck-Gesellschaft |
| School: | Biologische Kybernetik |
BibTeX
@inproceedings{3774,
title = {Measuring Statistical Dependence with Hilbert-Schmidt Norms},
journal = {Algorithmic Learning Theory: 16th International Conference, ALT 2005},
booktitle = {Algorithmic Learning Theory, Lecture Notes in Computer Science, Vol. 3734},
abstract = {We propose an independence criterion based on the eigenspectrum of covariance operators in reproducing kernel Hilbert spaces (RKHSs), consisting of an empirical estimate of the Hilbert-Schmidt norm of the cross-covariance operator (we term this a Hilbert-Schmidt Independence Criterion, or HSIC). This approach has several advantages, compared with previous kernel-based independence criteria. First, the empirical estimate is simpler than any other kernel dependence test, and requires no user-defined regularisation. Second, there is a clearly defined population quantity which the empirical estimate approaches in the large sample limit, with exponential convergence guaranteed between the two: this ensures that independence tests based on {methodname} do not suffer from slow learning rates.
Finally, we show in the context of independent component analysis (ICA) that the performance of HSIC is competitive with that of previously published kernel-based criteria, and of other recently published ICA methods.},
pages = {63-78},
editors = {S Jain and H-U Simon and E Tomita},
publisher = {Springer},
organization = {Max-Planck-Gesellschaft},
school = {Biologische Kybernetik},
address = {Berlin, Germany},
month = oct,
year = {2005},
author = {Gretton, A. and Bousquet, O. and Smola, A. and Schoelkopf, B.},
doi = {10.1007/11564089_7},
month_numeric = {10}
}