An Information Theory Perspective on Variance-Invariance-Covariance Regularization.
Ravid Shwartz-ZivRandall BalestrieroKenji KawaguchiTim G. J. RudnerYann LeCunPublished in: NeurIPS (2023)
Keyphrases
- information theory
- information theoretic
- shannon entropy
- relative entropy
- information geometry
- covariance matrix
- jensen shannon divergence
- statistical learning
- statistical mechanics
- conditional entropy
- kullback leibler divergence
- regularization parameter
- kl divergence
- rate distortion theory
- statistical physics
- bregman divergences
- prediction error