A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time.
Yeqi GaoZhao SongWeixin WangJunze YinPublished in: CoRR (2023)
Keyphrases
- single layer
- matrix multiplication
- extreme learning machine
- quadratic programming
- support vector
- neural nets
- multi layer
- support vector machine svm
- support vector machine
- higher order
- neural network
- image processing
- feed forward neural networks
- knn
- message passing
- hopfield neural network
- linear programming
- activation function
- fuzzy logic
- computational complexity
- feature extraction
- genetic algorithm
- machine learning