Dan Alistarh

ORCID

Publication Activity (10 Years)

Years Active: 2008-2024
Publications (10 Years): 224

Top Venues

Publications

Abhinav Agarwalla, Abhay Gupta, Alexandre Marques, Shubhra Pandit, Michael Goin, Eldar Kurtic, Kevin Leong, Tuan Nguyen, Mahmoud Salem, Dan Alistarh, Sean Lie, Mark Kurtz
Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment. CoRR (2024)
Mahdi Nikdan, Soroush Tabesh, Dan Alistarh
RoSA: Accurate Parameter-Efficient Fine-Tuning via Robust Adaptation. CoRR (2024)
Elias Frantar, Dan Alistarh
QMoE: Sub-1-Bit Compression of Trillion Parameter Models. MLSys (2024)
Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Martin Jaggi, Dan Alistarh, Torsten Hoefler, James Hensman
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs. CoRR (2024)
Rustem Islamov, Mher Safaryan, Dan Alistarh
AsGrad: A Sharp Unified Analysis of Asynchronous-SGD Algorithms. AISTATS (2024)
Ilia Markov, Kaveh Alim, Elias Frantar, Dan Alistarh
L-GreCo: Layerwise-adaptive Gradient Compression For Efficient Data-parallel Deep Learning. MLSys (2024)
Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, Dan Alistarh
SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression. ICLR (2024)
Hossein Zakerinia, Shayan Talaei, Giorgi Nadiradze, Dan Alistarh
Communication-Efficient Federated Learning With Data and Client Heterogeneity. AISTATS (2024)
Vage Egiazarian, Andrei Panferov, Denis Kuznedelev, Elias Frantar, Artem Babenko, Dan Alistarh
Extreme Compression of Large Language Models via Additive Quantization. CoRR (2024)
Vladimir Malinovskii, Denis Mazur, Ivan Ilin, Denis Kuznedelev, Konstantin Burlachenko, Kai Yi, Dan Alistarh, Peter Richtárik
PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression. CoRR (2024)
Dan Alistarh, Krishnendu Chatterjee, Mehrdad Karrabi, John Lazarsfeld
Game Dynamics and Equilibrium Computation in the Population Protocol Model. PODC (2024)
Ionut-Vlad Modoranu, Mher Safaryan, Grigory Malinovsky, Eldar Kurtic, Thomas Robert, Peter Richtárik, Dan Alistarh
MicroAdam: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence. CoRR (2024)
Elias Frantar, Carlos Riquelme Ruiz, Neil Houlsby, Dan Alistarh, Utku Evci
Scaling Laws for Sparsely-Connected Foundation Models. ICLR (2024)
Eldar Kurtic, Amir Moeini, Dan Alistarh
Mathador-LM: A Dynamic Benchmark for Mathematical Reasoning on Large Language Models. CoRR (2024)
Shashata Sawmya, Linghao Kong, Ilia Markov, Dan Alistarh, Nir Shavit
Sparse Expansion and Neuronal Disentanglement. CoRR (2024)
Ilya Kokorin, Victor Yudov, Vitaly Aksenov, Dan Alistarh
Wait-free Trees with Asymptotically-Efficient Range Queries. IPDPS (2024)
Aniruddha Nrusimha, Mayank Mishra, Naigang Wang, Dan Alistarh, Rameswar Panda, Yoon Kim
Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization. CoRR (2024)
Dan Alistarh
. SIGACT News 55 (2) (2024)
Armand Nicolicioiu, Eugenia Iofinova, Eldar Kurtic, Mahdi Nikdan, Andrei Panferov, Ilia Markov, Nir Shavit, Dan Alistarh
Panza: A Personalized Text Writing Assistant via Data Playback and Local Fine-Tuning. CoRR (2024)
Denis Kuznedelev, Eldar Kurtic, Eugenia Iofinova, Elias Frantar, Alexandra Peste, Dan Alistarh
Accurate Neural Network Pruning Requires Rethinking Sparse Optimization. Trans. Mach. Learn. Res. 2024 (2024)
Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, Dan Alistarh
SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression. CoRR (2023)
Dan Alistarh
Distributed Computing Column 87 Recent Advances in Multi-Pass Graph Streaming Lower Bounds. SIGACT News 54 (3) (2023)
Saleh Ashkboos, Ilia Markov, Elias Frantar, Tingxuan Zhong, Xincheng Wang, Jie Ren, Torsten Hoefler, Dan Alistarh
Towards End-to-end 4-Bit Inference on Generative Large Language Models. CoRR (2023)
Eldar Kurtic, Elias Frantar, Dan Alistarh
ZipLM: Hardware-Aware Structured Pruning of Language Models. CoRR (2023)
Nikita Koval, Dmitry Khalanskiy, Dan Alistarh
CQS: A Formally-Verified Framework for Fair and Abortable Synchronization. Proc. ACM Program. Lang. 7 (PLDI) (2023)
Arshia Soltani Moakhar, Eugenia Iofinova, Dan Alistarh
SPADE: Sparsity-Guided Debugging for Deep Neural Networks. CoRR (2023)
Ionut-Vlad Modoranu, Aleksei Kalinov, Eldar Kurtic, Dan Alistarh
Error Feedback Can Accurately Compress Preconditioners. CoRR (2023)
Alexander Fedorov, Diba Hashemi, Giorgi Nadiradze, Dan Alistarh
Provably-Efficient and Internally-Deterministic Parallel Union-Find. CoRR (2023)
Alexander Slastin, Dan Alistarh, Vitaly Aksenov
Efficient Self-Adjusting Search Trees via Lazy Updates. CoRR (2023)
Nikita Koval, Dan Alistarh, Roman Elizarov
Fast and Scalable Channels in Kotlin Coroutines. PPoPP (2023)
Ilia Markov, Adrian Vladu, Qi Guo, Dan Alistarh
Quantized Distributed Training of Large Models with Convergence Guarantees. CoRR (2023)
Eugenia Iofinova, Alexandra Peste, Dan Alistarh
Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures. CVPR (2023)
Mher Safaryan, Alexandra Peste, Dan Alistarh
Knowledge Distillation Performs Partial Variance Reduction. NeurIPS (2023)
Mahdi Nikdan, Tommaso Pegolotti, Eugenia Iofinova, Eldar Kurtic, Dan Alistarh
SparseProp: Efficient Sparse Backpropagation for Faster Training of Neural Networks. CoRR (2023)
Nikita Koval, Alexander Fedorov, Maria Sokolova, Dmitry Tsitelov, Dan Alistarh
Lincheck: A Practical Framework for Testing Concurrent Data Structures on JVM. CAV (1) (2023)
Mahdi Nikdan, Tommaso Pegolotti, Eugenia Iofinova, Eldar Kurtic, Dan Alistarh
SparseProp: Efficient Sparse Backpropagation for Faster Training of Neural Networks at the Edge. ICML (2023)
Denis Kuznedelev, Soroush Tabesh, Kimia Noorbakhsh, Elias Frantar, Sara Beery, Eldar Kurtic, Dan Alistarh
Vision Models Can Be Efficiently Specialized via Few-Shot Task-Aware Compression. CoRR (2023)
Denis Kuznedelev, Eldar Kurtic, Elias Frantar, Dan Alistarh
CAP: Correlation-Aware Pruning for Highly-Accurate Sparse Vision Models. NeurIPS (2023)
Dan Alistarh
. SIGACT News 54 (4) (2023)
Ilya Kokorin, Dan Alistarh, Vitaly Aksenov
Wait-free Trees with Asymptotically-Efficient Range Queries. CoRR (2023)
Eldar Kurtic, Elias Frantar, Dan Alistarh
ZipLM: Inference-Aware Structured Pruning of Language Models. NeurIPS (2023)
Eldar Kurtic, Torsten Hoefler, Dan Alistarh
How to Prune Your Language Model: Recovering Accuracy on the "Sparsity May Cry" Benchmark. CoRR (2023)
Eugenia Iofinova, Alexandra Peste, Dan Alistarh
Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures. CoRR (2023)
Elias Frantar, Dan Alistarh
QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models. CoRR (2023)
Dan Alistarh, Faith Ellen, Joel Rybicki
Wait-free approximate agreement on graphs. Theor. Comput. Sci. 948 (2023)
Denis Kuznedelev, Eldar Kurtic, Eugenia Iofinova, Elias Frantar, Alexandra Peste, Dan Alistarh
Accurate Neural Network Pruning Requires Rethinking Sparse Optimization. CoRR (2023)
Alexandra Peste, Adrian Vladu, Eldar Kurtic, Christoph H. Lampert, Dan Alistarh
CrAM: A Compression-Aware Minimizer. ICLR (2023)
Dan Alistarh, Krishnendu Chatterjee, Mehrdad Karrabi, John Lazarsfeld
Repeated Game Dynamics in Population Protocols. CoRR (2023)
Paniz Halvachi, Alexandra Peste, Dan Alistarh, Christoph H. Lampert
ELSA: Partial Weight Freezing for Overhead-Free Sparse Network Deployment. CoRR (2023)
Ilia Markov, Adrian Vladu, Qi Guo, Dan Alistarh
Quantized Distributed Training of Large Models with Convergence Guarantees. ICML (2023)
Eldar Kurtic, Denis Kuznedelev, Elias Frantar, Michael Goin, Dan Alistarh
Sparse Fine-tuning for Inference Acceleration of Large Language Models. CoRR (2023)
Elias Frantar, Carlos Riquelme, Neil Houlsby, Dan Alistarh, Utku Evci
Scaling Laws for Sparsely-Connected Foundation Models. CoRR (2023)
Mher Safaryan, Alexandra Peste, Dan Alistarh
Knowledge Distillation Performs Partial Variance Reduction. CoRR (2023)
Vitaly Aksenov, Dan Alistarh, Alexandra Drozdova, Amirkeivan Mohtashami
The splay-list: a distribution-adaptive concurrent skip-list. Distributed Comput. 36 (3) (2023)
Dan Alistarh, James Aspnes, Faith Ellen, Rati Gelashvili, Leqi Zhu
Why Extension-Based Proofs Fail. SIAM J. Comput. 52 (4) (2023)
Elias Frantar, Saleh Ashkboos, Torsten Hoefler, Dan Alistarh
OPTQ: Accurate Quantization for Generative Pre-trained Transformers. ICLR (2023)
John Lazarsfeld, Dan Alistarh
Decentralized Learning Dynamics in the Gossip Model. CoRR (2023)
Dan Alistarh
Distributed Computing Column 86: A Summary of PODC 2022. SIGACT News 54 (1) (2023)
Elias Frantar, Dan Alistarh
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot. CoRR (2023)
Tommaso Pegolotti, Elias Frantar, Dan Alistarh, Markus Püschel
QIGen: Generating Efficient Kernels for Quantized Inference on Large Language Models. CoRR (2023)
Dan Alistarh, Alkida Balliu, Dimitrios Los, Sean Ovens
A Brief Summary of PODC 2022. SIGACT News 54 (1) (2023)
Alexander Fedorov, Diba Hashemi, Giorgi Nadiradze, Dan Alistarh
Provably-Efficient and Internally-Deterministic Parallel Union-Find. SPAA (2023)
Elias Frantar, Dan Alistarh
SparseGPT: Massive Language Models Can be Accurately Pruned in One-Shot. ICML (2023)
Rustem Islamov, Mher Safaryan, Dan Alistarh
AsGrad: A Sharp Unified Analysis of Asynchronous-SGD Algorithms. CoRR (2023)
Dan Alistarh, Giorgi Nadiradze, Amirmojtaba Sabour
Dynamic Averaging Load Balancing on Cycles. Algorithmica 84 (4) (2022)
Eldar Kurtic, Dan Alistarh
GMP*: Well-Tuned Global Magnitude Pruning Can Outperform Most BERT-Pruning Methods. CoRR (2022)
Dan Alistarh, Ilia Markov, Giorgi Nadiradze
Elastic Consistency: A Consistency Criterion for Distributed Optimization. SIGACT News 53 (2) (2022)
Elias Frantar, Dan Alistarh
SPDY: Accurate Pruning with Speedup Guarantees. ICML (2022)
Nikita Koval, Dan Alistarh, Roman Elizarov
Fast and Scalable Channels in Kotlin Coroutines. CoRR (2022)
Elias Frantar, Dan Alistarh
SPDY: Accurate Pruning with Speedup Guarantees. CoRR (2022)
Eldar Kurtic, Daniel Campos, Tuan Nguyen, Elias Frantar, Mark Kurtz, Benjamin Fineran, Michael Goin, Dan Alistarh
The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models. CoRR (2022)
Trevor Brown, William Sigouin, Dan Alistarh
PathCAS: an efficient middle ground for concurrent search data structures. PPoPP (2022)
Ilia Markov, Hamidreza Ramezani-Kebrya, Dan Alistarh
CGX: adaptive system support for communication-efficient deep learning. Middleware (2022)
Elias Frantar, Saleh Ashkboos, Torsten Hoefler, Dan Alistarh
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. CoRR (2022)
Dan Alistarh, Joel Rybicki, Sasha Voitovych
Near-Optimal Leader Election in Population Protocols on Graphs. CoRR (2022)
Anastasiia Postnikova, Nikita Koval, Giorgi Nadiradze, Dan Alistarh
Multi-queues can be state-of-the-art priority schedulers. PPoPP (2022)
Alexandra Peste, Adrian Vladu, Dan Alistarh, Christoph H. Lampert
CrAM: A Compression-Aware Minimizer. CoRR (2022)
Elias Frantar, Dan Alistarh
Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning. CoRR (2022)
Denis Kuznedelev, Eldar Kurtic, Elias Frantar, Dan Alistarh
oViT: An Accurate Second-Order Pruning Framework for Vision Transformers. CoRR (2022)
Dan Alistarh, Joel Rybicki, Sasha Voitovych
Near-Optimal Leader Election in Population Protocols on Graphs. PODC (2022)
Hossein Zakerinia, Shayan Talaei, Giorgi Nadiradze, Dan Alistarh
QuAFL: Federated Averaging Can Be Both Asynchronous and Communication-Efficient. CoRR (2022)
Bapi Chatterjee, Vyacheslav Kungurtsev, Dan Alistarh
Scaling the Wild: Decentralizing Hogwild!-style Shared-memory SGD. CoRR (2022)
Mohammadreza Alimohammadi, Ilia Markov, Elias Frantar, Dan Alistarh
L-GreCo: An Efficient and General Framework for Layerwise-Adaptive Gradient Compression. CoRR (2022)
Trevor Brown, William Sigouin, Dan Alistarh
PathCAS: An Efficient Middle Ground for Concurrent Search Data Structures. CoRR (2022)
Shayan Talaei, Giorgi Nadiradze, Dan Alistarh
Hybrid Decentralized Optimization: First- and Zeroth-Order Optimizers Can Be Jointly Leveraged For Faster Convergence. CoRR (2022)
Elias Frantar, Dan Alistarh
Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning. NeurIPS (2022)
Eugenia Iofinova, Alexandra Peste, Mark Kurtz, Dan Alistarh
How Well Do Sparse ImageNet Models Transfer? CVPR (2022)
Dan Alistarh
Distributed Computing Column 85 Elastic Consistency: A Consistency Criterion for Distributed Optimization. SIGACT News 53 (2) (2022)
Eldar Kurtic, Daniel Campos, Tuan Nguyen, Elias Frantar, Mark Kurtz, Benjamin Fineran, Michael Goin, Dan Alistarh
The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models. EMNLP (2022)
Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, Alexandra Peste
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks. J. Mach. Learn. Res. 22 (2021)
Ali Ramezani-Kebrya, Fartash Faghri, Ilya Markov, Vitalii Aksenov, Dan Alistarh, Daniel M. Roy
NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization. J. Mach. Learn. Res. 22 (2021)
Alexander Fedorov, Nikita Koval, Dan Alistarh
A Scalable Concurrent Algorithm for Dynamic Connectivity. SPAA (2021)
Dan Alistarh, Rati Gelashvili, Giorgi Nadiradze
Lower Bounds for Shared-Memory Leader Election under Bounded Write Contention. CoRR (2021)
Dan Alistarh, Faith Ellen, Joel Rybicki
Wait-free approximate agreement on graphs. CoRR (2021)
Dan Alistarh
Distributed Computing Column 84: Perspectives on the Paper "CCS Expressions, Finite State Processes, and Three Problems of Equivalence". SIGACT News 52 (4) (2021)
Dan Alistarh
Distributed Computing Column 81: Byzantine Agreement with Less Communication: Recent Advances. SIGACT News 52 (1) (2021)
Peter Davies, Vijaykrishna Gurunanthan, Niusha Moshrefi, Saleh Ashkboos, Dan Alistarh
New Bounds For Distributed Mean Estimation and Variance Reduction. ICLR (2021)
Shigang Li, Tal Ben-Nun, Giorgi Nadiradze, Salvatore Di Girolamo, Nikoli Dryden, Dan Alistarh, Torsten Hoefler
Breaking (Global) Barriers in Parallel Stochastic Optimization With Wait-Avoiding Group Averaging. IEEE Trans. Parallel Distributed Syst. 32 (7) (2021)
Dan Alistarh, Rati Gelashvili, Joel Rybicki
Brief Announcement: Fast Graphical Population Protocols. DISC (2021)
Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, Alexandra Peste
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks. CoRR (2021)

Dan Alistarh.css-eqlu8i{font-size:small;padding-left:4px;}

Publication Activity (10 Years)

Top Topics

Top Venues

Dan Alistarh