Impact of Tokenization on Language Models: An Analysis for Turkish.

Cagri Toraman Eyup Halit Yilmaz Furkan Sahinuç Oguzhan Ozcelik

Published in: CoRR (2022)

Keyphrases

language model
language modeling
n gram
probabilistic model
statistical language models
query expansion
speech recognition
document retrieval