Tokenization Consistency Matters for Generative Models on Extractive NLP Tasks.
Kaiser SunPeng QiYuhao ZhangLan LiuWilliam Yang WangZhiheng HuangPublished in: EMNLP (Findings) (2023)
Keyphrases
- generative model
- probabilistic model
- discriminative learning
- mixture model
- hierarchical hidden markov models
- natural language processing
- text summarization
- prior knowledge
- maximum entropy principle
- em algorithm
- information extraction
- named entities
- discriminative models
- question answering
- conditional random fields
- representational power
- hidden variables
- topic models
- generative and discriminative models
- maximum entropy
- expectation maximization
- semi supervised
- global constraints
- object categories
- decision trees
- text mining
- natural language