Learning to Grow Pretrained Models for Efficient Transformer Training.
Peihao WangRameswar PandaLucas Torroba HennigenPhilip GreengardLeonid KarlinskyRogério FerisDavid Daniel CoxZhangyang WangYoon KimPublished in: ICLR (2023)
Keyphrases
- structured prediction
- online learning
- learning models
- learning algorithm
- knowledge acquisition
- prior knowledge
- learning process
- online training
- fuzzy logic
- supervised learning
- decision trees
- learning problems
- neural nets
- learning rules
- learning stage
- genetic algorithm
- accurate models
- learned models
- deep architectures
- efficient learning
- hidden variables
- fault diagnosis
- learning systems
- training examples
- active learning
- learning environment
- training data