Building on Efficient Foundations: Effectively Training LLMs with Structured Feedforward Layers.
Xiuying WeiSkander MoallaRazvan PascanuCaglar GulcehrePublished in: CoRR (2024)
Keyphrases
- feed forward
- error back propagation
- recurrent networks
- back propagation
- feed forward neural networks
- biologically plausible
- neural nets
- artificial neural networks
- artificial neural
- recurrent neural networks
- multi layer
- training algorithm
- training phase
- activation function
- visual cortex
- closed loop
- structured data
- higher order
- adaptive neural
- neuron model
- knowledge base