Joint Optimization of Tokenization and Downstream Model.

Tatsuya Hiraoka Sho Takase Kei Uchiumi Atsushi Keyaki Naoaki Okazaki

Published in: CoRR (2021)

Keyphrases

joint optimization
statistical model
theoretical framework
mathematical model
prior knowledge
computational model
high level
artificial neural networks
cost function
em algorithm
experimental data
formal model