Loki: Low-Rank Keys for Efficient Sparse Attention.
Prajwal SinghaniaSiddharth SinghShwai HeSoheil FeiziAbhinav BhatelePublished in: CoRR (2024)
Keyphrases
- low rank
- rank minimization
- low rank matrix
- low rank matrices
- matrix factorization
- low rank subspace
- missing data
- linear combination
- robust principal component analysis
- matrix decomposition
- sparsity constraints
- singular value decomposition
- nuclear norm
- matrix completion
- low rank representation
- sparse linear
- convex optimization
- group sparsity
- tensor decomposition
- kernel matrix
- high order
- collaborative filtering
- feature selection
- low rank approximation
- binary matrices
- high dimensional data
- higher order
- semi supervised