Information-Theory Interpretation of the Skip-Gram Negative-Sampling Objective Function.

Oren Melamud Jacob Goldberger

Published in: ACL (2) (2017)

Keyphrases

information theory
objective function
information theoretic
jensen shannon divergence
statistical learning
conditional entropy
relative entropy
statistical mechanics
optimal solution
random sampling
kullback leibler divergence
mdl principle
pattern recognition
mutual information
loss function
shannon entropy