Login / Signup
Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance.
Omer Goldman
Avi Caciularu
Matan Eyal
Kris Cao
Idan Szpektor
Reut Tsarfaty
Published in:
ACL (Findings) (2024)
Keyphrases
</>
cost function
probabilistic model
genetic algorithm
data sets
similarity measure
keywords
management system
co occurrence
theoretical analysis
conceptual model
correlation coefficient
formal model