To Compress or Not to Compress - Self-Supervised Learning and Information Theory: A Review.

Ravid Shwartz-Ziv Yann LeCun

Published in: CoRR (2023)

Keyphrases

information theory
statistical learning
information theoretic
learning algorithm
learning process
supervised learning
computer vision
reinforcement learning
mutual information
kullback leibler divergence
learning curve
relative entropy
jensen shannon divergence
shannon entropy