Improving Self Consistency in LLMs through Probabilistic Tokenization.

Ashutosh Sathe Divyanshu Aggarwal Sunayana Sitaram

Published in: CoRR (2024)

Keyphrases

probabilistic model
bayesian networks
data driven
information theoretic
hidden markov models
data sets
real world
data mining
clustering algorithm
posterior probability
uncertain data
global constraints
consistency checking
biomedical information retrieval