Empirical Inference Conference Paper 2005

Measuring Statistical Dependence with Hilbert-Schmidt Norms

PDF
no image
Empirical Inference
no image
Empirical Inference

We propose an independence criterion based on the eigenspectrum of covariance operators in reproducing kernel Hilbert spaces (RKHSs), consisting of an empirical estimate of the Hilbert-Schmidt norm of the cross-covariance operator (we term this a Hilbert-Schmidt Independence Criterion, or HSIC). This approach has several advantages, compared with previous kernel-based independence criteria. First, the empirical estimate is simpler than any other kernel dependence test, and requires no user-defined regularisation. Second, there is a clearly defined population quantity which the empirical estimate approaches in the large sample limit, with exponential convergence guaranteed between the two: this ensures that independence tests based on {methodname} do not suffer from slow learning rates. Finally, we show in the context of independent component analysis (ICA) that the performance of HSIC is competitive with that of previously published kernel-based criteria, and of other recently published ICA methods.

Author(s): Gretton, A. and Bousquet, O. and Smola, A. and Schoelkopf, B.
Links:
Book Title: Algorithmic Learning Theory, Lecture Notes in Computer Science, Vol. 3734
Journal: Algorithmic Learning Theory: 16th International Conference, ALT 2005
Pages: 63-78
Year: 2005
Month: October
Day: 8
Editors: S Jain and H-U Simon and E Tomita
Publisher: Springer
Bibtex Type: Conference Paper (inproceedings)
Address: Berlin, Germany
DOI: 10.1007/11564089_7
Event Name: 16th International Conference ALT 2005
Event Place: Singapore
Digital: 0
Electronic Archiving: grant_archive
Language: en
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik

BibTex

@inproceedings{3774,
  title = {Measuring Statistical Dependence with Hilbert-Schmidt Norms},
  journal = {Algorithmic Learning Theory: 16th International Conference, ALT 2005},
  booktitle = {Algorithmic Learning Theory, Lecture Notes in Computer Science, Vol. 3734},
  abstract = {We propose an independence criterion based on the eigenspectrum of covariance operators in reproducing kernel Hilbert spaces (RKHSs), consisting of an empirical estimate of the Hilbert-Schmidt norm of the cross-covariance operator  (we term this a Hilbert-Schmidt Independence Criterion, or HSIC).  This approach has several advantages, compared with previous kernel-based independence criteria.  First, the empirical estimate is simpler than any other kernel dependence test, and requires no user-defined regularisation. Second, there is a clearly defined population quantity which the empirical estimate approaches in the large sample limit, with exponential convergence guaranteed between the two: this ensures that independence tests based on {methodname} do not suffer from slow learning rates.
  Finally, we show in the context of independent component analysis (ICA) that the performance of HSIC is competitive with that of previously published kernel-based criteria, and of other recently published ICA methods.},
  pages = {63-78},
  editors = {S Jain and H-U Simon and E Tomita},
  publisher = {Springer},
  organization = {Max-Planck-Gesellschaft},
  school = {Biologische Kybernetik},
  address = {Berlin, Germany},
  month = oct,
  year = {2005},
  slug = {3774},
  author = {Gretton, A. and Bousquet, O. and Smola, A. and Schoelkopf, B.},
  month_numeric = {10}
}