When Can Self-Attention Be Replaced by Feed Forward Layers?
Shucong ZhangErfan LoweimiPeter BellSteve RenalsPublished in: CoRR (2020)
Keyphrases
- feed forward
- back propagation
- artificial neural networks
- neural network
- neural nets
- recurrent neural networks
- hidden layer
- biologically plausible
- artificial neural
- multi layer
- activation function
- visual cortex
- single layer
- error back propagation
- feed forward neural networks
- neural architecture
- multiple layers
- spiking neural networks
- primate visual cortex
- visual attention
- artificial intelligence
- learning tasks
- spiking neurons
- input output
- machine learning