Downstream Task-Oriented Neural Tokenizer Optimization with Vocabulary Restriction as Post Processing.

Tatsuya Hiraoka Tomoya Iwakura

Published in: CoRR (2023)

Keyphrases

post processing
preprocessing
neural network
shape error concealment
filtering method
pattern extraction
post processed
evolutionary algorithm
network architecture
projections onto convex sets
supply chain
data mining techniques
human experts
data mining process