How to Capture Higher-order Correlations? Generalizing Matrix Softmax Attention to Kronecker Computation.
Josh AlmanZhao SongPublished in: ICLR (2024)
Keyphrases
- higher order
- lower order
- linear algebra
- pairwise
- natural images
- high order
- conditional random fields
- kronecker product
- square root
- hidden markov models
- efficient computation
- singular value decomposition
- adjacency matrix
- matrix representation
- data sets
- highly correlated
- focus of attention
- similarity matrix
- markov random field
- matrix inversion