Saturated Transformers are Constant-Depth Threshold Circuits.

William Merrill Ashish Sabharwal Noah A. Smith

Published in: Trans. Assoc. Comput. Linguistics (2022)

Keyphrases

high speed
depth information
depth map
circuit design
real time
depth cues
logic circuits
threshold selection
tunnel diode
data sets
neural network
case study
multiscale
high resolution
delay insensitive