Replacing softmax with ReLU in Vision Transformers.

Mitchell Wortsman Jaehoon Lee Justin Gilmer Simon Kornblith

Published in: CoRR (2023)

Keyphrases

vision system
real time
computer vision
activation function
data mining
data sets
information retrieval
image processing
wide range
fuzzy logic
self organizing maps
multilayer perceptron
human vision
active vision
hidden nodes
computational vision