Which transformer architecture fits my data? A vocabulary bottleneck in self-attention.
Noam WiesYoav LevineDaniel JannaiAmnon ShashuaPublished in: ICML (2021)
Keyphrases
- data collection
- data sets
- data structure
- data analysis
- original data
- data processing
- raw data
- data objects
- probability distribution
- prior knowledge
- data sources
- spatial data
- attribute values
- synthetic data
- computer systems
- neural network
- multimedia data
- small number
- fuzzy logic
- management system
- knowledge discovery
- end users
- data quality
- database
- historical data
- data acquisition
- sensor data
- fault diagnosis
- data mining techniques
- dimensionality reduction
- image data
- data points
- training data
- data mining