Addressing Token Uniformity in Transformers via Singular Value Transformation.

Hanqi Yan Lin Gui Wenjie Li Yulan He

Published in: CoRR (2022)

Keyphrases

singular values
singular value decomposition
low rank
fundamental matrix
neural network
three dimensional
small number
principal component analysis