Tim Dettmers

Publication Activity (10 Years)

Years Active: 2016-2024
Publications (10 Years): 34

Top Topics

Matrix Multiplication

Language Models For Information Retrieval

Top Venues

Publications

Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, Dan Alistarh
SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression. ICLR (2024)
Rulin Shao, Jacqueline He, Akari Asai, Weijia Shi, Tim Dettmers, Sewon Min, Luke Zettlemoyer, Pang Wei Koh
Scaling Retrieval-Based Language Models with a Trillion-Token Datastore. CoRR (2024)
Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, Dan Alistarh
SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression. CoRR (2023)
Tim Dettmers, Luke Zettlemoyer
The case for 4-bit precision: k-bit Inference Scaling Laws. ICML (2023)
Leo Z. Liu, Tim Dettmers, Xi Victoria Lin, Veselin Stoyanov, Xian Li
Towards A Unified View of Sparse Feed-Forward Network in Pretraining Large Language Model. CoRR (2023)
Alexander Borzunov, Dmitry Baranchuk, Tim Dettmers, Maksim Riabinin, Younes Belkada, Artem Chumachenko, Pavel Samygin, Colin Raffel
Petals: Collaborative Inference and Fine-tuning of Large Models. ACL (demo) (2023)
Mitchell Wortsman, Tim Dettmers, Luke Zettlemoyer, Ari Morcos, Ali Farhadi, Ludwig Schmidt
Stable and low-precision training for large-scale vision-language models. CoRR (2023)
Mitchell Wortsman, Tim Dettmers, Luke Zettlemoyer, Ari Morcos, Ali Farhadi, Ludwig Schmidt
Stable and low-precision training for large-scale vision-language models. NeurIPS (2023)
Max Ryabinin, Tim Dettmers, Michael Diskin, Alexander Borzunov
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient. ICML (2023)
Max Ryabinin, Tim Dettmers, Michael Diskin, Alexander Borzunov
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient. CoRR (2023)
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer
QLoRA: Efficient Finetuning of Quantized LLMs. NeurIPS (2023)
Alexander Borzunov, Max Ryabinin, Artem Chumachenko, Dmitry Baranchuk, Tim Dettmers, Younes Belkada, Pavel Samygin, Colin A. Raffel
Distributed Inference and Fine-tuning of Large Language Models Over The Internet. NeurIPS (2023)
Zeyu Liu, Tim Dettmers, Xi Lin, Veselin Stoyanov, Xian Li
Towards A Unified View of Sparse Feed-Forward Network in Pretraining Large Language Model. EMNLP (2023)
Alexander Borzunov, Max Ryabinin, Artem Chumachenko, Dmitry Baranchuk, Tim Dettmers, Younes Belkada, Pavel Samygin, Colin Raffel
Distributed Inference and Fine-tuning of Large Language Models Over The Internet. CoRR (2023)
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer
QLoRA: Efficient Finetuning of Quantized LLMs. CoRR (2023)
Devvrit, Sneha Kudugunta, Aditya Kusupati, Tim Dettmers, Kaifeng Chen, Inderjit S. Dhillon, Yulia Tsvetkov, Hannaneh Hajishirzi, Sham M. Kakade, Ali Farhadi, Prateek Jain
MatFormer: Nested Transformer for Elastic Inference. CoRR (2023)
Tim Dettmers, Mike Lewis, Sam Shleifer, Luke Zettlemoyer
8-bit Optimizers via Block-wise Quantization. ICLR (2022)
Margaret Li, Suchin Gururangan, Tim Dettmers, Mike Lewis, Tim Althoff, Noah A. Smith, Luke Zettlemoyer
Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models. CoRR (2022)
Tim Dettmers, Luke Zettlemoyer
The case for 4-bit precision: k-bit Inference Scaling Laws. CoRR (2022)
Tim Dettmers, Mike Lewis, Younes Belkada, Luke Zettlemoyer
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale. CoRR (2022)
Alexander Borzunov, Dmitry Baranchuk, Tim Dettmers, Max Ryabinin, Younes Belkada, Artem Chumachenko, Pavel Samygin, Colin Raffel
Petals: Collaborative Inference and Fine-tuning of Large Models. CoRR (2022)
Tim Dettmers, Mike Lewis, Younes Belkada, Luke Zettlemoyer
GPT3.int8(): 8-bit Matrix Multiplication for Transformers at Scale. NeurIPS (2022)
Alexander Borzunov, Max Ryabinin, Tim Dettmers, Quentin Lhoest, Lucile Saulnier, Michael Diskin, Yacine Jernite, Thomas Wolf
Training Transformers Together. CoRR (2022)
Tim Dettmers, Mike Lewis, Sam Shleifer, Luke Zettlemoyer
8-bit Optimizers via Block-wise Quantization. CoRR (2021)
Mike Lewis, Shruti Bhosale, Tim Dettmers, Naman Goyal, Luke Zettlemoyer
BASE Layers: Simplifying Training of Large, Sparse Models. ICML (2021)
Alexander Borzunov, Max Ryabinin, Tim Dettmers, Quentin Lhoest, Lucile Saulnier, Michael Diskin, Yacine Jernite, Thomas Wolf
Training Transformers Together. NeurIPS (Competition and Demos) (2021)
Mike Lewis, Shruti Bhosale, Tim Dettmers, Naman Goyal, Luke Zettlemoyer
BASE Layers: Simplifying Training of Large, Sparse Models. CoRR (2021)
Gabriel Ilharco, Cesar Ilharco, Iulia Turc, Tim Dettmers, Felipe Ferreira, Kenton Lee
High Performance Natural Language Processing. EMNLP (Tutorial Abstracts) (2020)
Tim Dettmers, Luke Zettlemoyer
Sparse Networks from Scratch: Faster Training without Losing Performance. CoRR (2019)
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel
Convolutional 2D Knowledge Graph Embeddings. AAAI (2018)
Dirk Weissenborn, Pasquale Minervini, Tim Dettmers, Isabelle Augenstein, Johannes Welbl, Tim Rocktäschel, Matko Bosnjak, Jeff Mitchell, Thomas Demeester, Pontus Stenetorp, Sebastian Riedel
Jack the Reader - A Machine Reading Framework. CoRR (2018)
Dirk Weissenborn, Pasquale Minervini, Isabelle Augenstein, Johannes Welbl, Tim Rocktäschel, Matko Bosnjak, Jeff Mitchell, Thomas Demeester, Tim Dettmers, Pontus Stenetorp, Sebastian Riedel
Jack the Reader - A Machine Reading Framework. ACL (4) (2018)
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel
Convolutional 2D Knowledge Graph Embeddings. CoRR (2017)
Tim Dettmers
8-Bit Approximations for Parallelism in Deep Learning. ICLR (Poster) (2016)