Jump Self-attention: Capturing High-order Statistics in Transformers.

Haoyi Zhou Siyang Xiao Shanghang Zhang Jieqi Peng Shuai Zhang Jianxin Li

Published in: NeurIPS (2022)

Keyphrases

high order statistics
markov chain
higher order
information content