Login / Signup

Improve Transformer Pre-Training with Decoupled Directional Relative Position Encoding and Representation Differentiations.

Haojie ZhangMingfei LiangRuobing XieZhenlong SunBo ZhangLeyu Lin
Published in: CoRR (2022)
Keyphrases
  • relative position
  • geometric properties
  • encoding scheme
  • fuzzy logic
  • machine learning
  • computer vision
  • database systems
  • pairwise
  • training set