How to Capture Higher-order Correlations? Generalizing Matrix Softmax Attention to Kronecker Computation.
Josh AlmanZhao SongPublished in: CoRR (2023)
Keyphrases
- higher order
- lower order
- linear algebra
- high order
- matrix inversion
- pairwise
- natural images
- correlation analysis
- activation function
- kronecker product
- correlation matrix
- matrix representation
- strongly correlated
- square root
- neural network
- low rank
- conditional random fields
- linear combination
- reinforcement learning
- multiscale