Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks.
Minmin ChenJeffrey PenningtonSamuel S. SchoenholzPublished in: ICML (2018)
Keyphrases
- recurrent neural networks
- long short term memory
- echo state networks
- neural network
- signal processing
- feedforward neural networks
- hidden layer
- complex valued
- feed forward
- cascade correlation
- vector field
- neuro fuzzy
- reservoir computing
- recurrent networks
- artificial neural networks
- belief networks
- frequency domain
- neural model
- non stationary
- hebbian learning
- high frequency
- statistical mechanics
- nonlinear dynamic systems
- decision making