A Study on ReLU and Softmax in Transformer.

Kai Shen Junliang Guo Xu Tan Siliang Tang Rui Wang Jiang Bian

Published in: CoRR (2023)

Keyphrases

empirical studies
fuzzy logic
databases
neural network
machine learning
information retrieval
learning algorithm
information systems
video sequences
quantitative and qualitative