Escaping the Gradient Vanishing: Periodic Alternatives of Softmax in Attention Mechanism.

Shulun Wang Feng Liu Bin Liu

Published in: IEEE Access (2021)

Keyphrases

attention mechanism
visual attention
saliency map
decision makers
visual attention model
vision system
machine learning
high quality
pairwise
visual information
video retrieval
activation function