Hardware Accelerator for Multi-Head Attention and Position-Wise Feed-Forward in the Transformer.
Siyuan LuMeiqi WangShuang LiangJun LinZhongfeng WangPublished in: SoCC (2020)
Keyphrases
- feed forward
- back propagation
- neural network
- real time
- artificial neural networks
- neural nets
- biologically plausible
- hidden layer
- recurrent neural networks
- low cost
- activation function
- hardware and software
- fuzzy logic
- feed forward neural networks
- visual cortex
- visual attention
- field programmable gate array
- neural architecture
- gaze direction
- error back propagation
- fault diagnosis
- hardware implementation
- focus of attention
- computing systems
- feedforward neural networks
- embedded systems
- computer systems
- spiking neural networks
- pairwise
- artificial neural
- neuron model
- image sequences