Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach.
Ryo KarakidaShotaro AkahoShun-ichi AmariPublished in: AISTATS (2019)
Keyphrases
- fisher information
- neural network
- free energy
- competitive learning
- information geometry
- euclidean space
- positive definite
- noise level
- probability density function
- exponential family
- posterior distribution
- estimation error
- belief networks
- markov random field
- bayesian inference
- belief propagation
- geodesic distance
- variational methods
- closed form
- self organizing maps
- bayesian networks
- approximate inference
- statistical inference
- kullback leibler divergence
- multidimensional scaling
- kl divergence
- em algorithm
- upper bound