LongHeads: Multi-Head Attention is Secretly a Long Context Processor.

Yi Lu Xin Zhou Wei He Jun Zhao Tao Ji Tao Gui Qi Zhang Xuanjing Huang

Published in: CoRR (2024)

Keyphrases

contextual information
real time
context sensitive
high speed
knowledge retrieval
neural network
artificial intelligence
knowledge base
face recognition
parallel processing