Attention is Naturally Sparse with Gaussian Distributed Input.

Yichuan Deng Zhao Song Chiwun Yang

Published in: CoRR (2024)

Keyphrases

distributed environment
cooperative
distributed systems
multi agent
lightweight
maximum likelihood
communication overhead
sparse data
fault tolerant
input data
multiscale
database
visual attention
computer networks
high dimensional
multi agent systems
focus of attention