A Text is Worth Several Tokens: Text Embedding from LLMs Secretly Aligns Well with The Key Tokens.

Zhijie Nie Richong Zhang Zhanyu Wu

Published in: CoRR (2024)

Keyphrases

text retrieval
text processing
text mining
text analysis
natural language generation
text information
data mining
information retrieval
text documents
automatically extracted
string matching
database
machine learning
keywords
probabilistic model
web documents