How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse.
Mohamed El Amine SeddikSuei-Wen ChenSoufiane HayouPierre YoussefMérouane DebbahPublished in: CoRR (2024)
Keyphrases
- synthetic data
- language model
- statistical analysis
- language modeling
- n gram
- document retrieval
- information retrieval
- data sets
- probabilistic model
- test collection
- retrieval model
- speech recognition
- real world
- query expansion
- mixture model
- real image data
- training set
- statistical language models
- context sensitive
- ad hoc information retrieval
- query terms
- smoothing methods
- query specific
- translation model
- language model for information retrieval
- pseudo relevance feedback
- statistical machine translation
- document ranking
- synthetic datasets
- vector space model
- word error rate
- retrieval effectiveness
- supervised learning
- language models for information retrieval
- relevance model
- document length
- co occurrence
- statistical language modeling
- machine learning