Salience-Based Adaptive Masking: Revisiting Token Dynamics for Enhanced Pre-training.

Hyesong Choi Hyejin Park Kwang Moo Yi Sungmin Cha Dongbo Min

Published in: CoRR (2024)

Keyphrases

training set
dynamic model
training phase
supervised learning
data driven
training examples
recurrent networks
online learning
human visual system
training process
database
real time
knowledge base
bit rate
statistically significant
training algorithm