Best Fit Activation Functions for Attention Mechanism: Comparison and Enhancement.
Maan AlhazmiAbdulrahman AltahhanPublished in: IJCNN (2023)
Keyphrases
- activation function
- attention mechanism
- neural network
- feed forward
- neural architecture
- visual attention
- back propagation
- hidden layer
- image processing
- radial basis function
- saliency map
- network architecture
- neural nets
- input output
- basis functions
- higher order
- multilayer perceptron
- training phase
- artificial neural networks
- learning algorithm