Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs.
Angelica ChenRavid Shwartz-ZivKyunghyun ChoMatthew L. LeavittNaomi SaphraPublished in: ICLR (2024)
Keyphrases
- phase transition
- random constraint satisfaction problems
- constraint satisfaction
- satisfiability problem
- combinatorial problems
- np complete
- randomly generated
- random instances
- graph coloring
- hard problems
- np complete problems
- deep structure
- frequent itemsets
- sat problem
- cellular automata
- constraint satisfaction problems
- itemsets
- evolutionary algorithm