Hardware Accelerator for Multi-Head Attention and Position-Wise Feed-Forward in the Transformer.
Siyuan LuMeiqi WangShuang LiangJun LinZhongfeng WangPublished in: CoRR (2020)
Keyphrases
- feed forward
- back propagation
- neural nets
- artificial neural networks
- neural network
- real time
- fuzzy logic
- recurrent neural networks
- hidden layer
- hardware and software
- biologically plausible
- field programmable gate array
- low cost
- visual cortex
- artificial neural
- activation function
- power system
- feed forward neural networks
- computer systems
- error back propagation
- hardware implementation
- neural architecture
- gaze direction
- focus of attention
- fault diagnosis
- learning algorithm
- expert systems
- parallel implementation
- learning rate
- support vector
- genetic algorithm
- support vector machine svm
- recurrent networks
- artificial intelligence
- pairwise
- neuron model