Enhancing Fine-Tuning in Low Data Regime by Increasing Representation Entropy During Pre-Training Phase.

Jaeill Kim Jungwook Shin Wonjong Rhee

Published in: ICTC (2023)

Keyphrases

training phase
data sets
fine tuning
data analysis
database
data processing
data collection
raw data
data quality
data structure
data points
nearest neighbor
training examples
synthetic data
information theory
training data
image data
small number
input data
highly correlated