Information-theoretic Characterizations of Generalization Error for the Gibbs Algorithm.
Gholamali AminianYuheng BuLaura ToniMiguel R. D. RodriguesGregory W. WornellPublished in: CoRR (2022)
Keyphrases
- information theoretic
- learning algorithm
- log likelihood
- generalization error
- mutual information
- information theory
- computational complexity
- objective function
- information bottleneck
- np hard
- relative entropy
- worst case
- information theoretic measures
- jensen shannon divergence
- sample size
- multi class
- upper bound
- support vector machine
- probabilistic model
- learning process
- optimal solution
- training data