Pre-training Small Base LMs with Fewer Tokens.

Sunny Sanyal Sujay Sanghavi Alexandros G. Dimakis

Published in: CoRR (2024)

Keyphrases

learning management systems
labeled data for training
training set
online learning
e learning
training algorithm
small number
training process
databases
training phase
expert systems
training examples
learning objects
test set
back propagation
supervised learning
color images
knowledge base
information retrieval