Login / Signup
How do different tokenizers perform on downstream tasks in scriptio continua languages?: A case study in Japanese.
Takuro Fujii
Koki Shibata
Atsuki Yamaguchi
Terufumi Morishita
Yasuhiro Sogawa
Published in:
CoRR (2023)
Keyphrases
</>
case study
databases
supply chain
test bed
real world
expressive power
machine learning
artificial intelligence
computer vision
feature selection
decision trees
bayesian networks
grammatical inference