Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length.
Xuezhe MaXiaomeng YangWenhan XiongBeidi ChenLili YuHao ZhangJonathan MayLuke ZettlemoyerOmer LevyChunting ZhouPublished in: CoRR (2024)
Keyphrases
- contextual information
- cost effective
- efficient learning
- evolutionary algorithm
- context aware
- lightweight
- inference process
- bayesian model
- belief networks
- context sensitive
- conceptual model
- computationally efficient
- neural network
- hidden markov models
- recommender systems
- video sequences
- bayesian networks
- computer vision
- search engine
- learning algorithm