Konstantin Mishchenko

Publication Activity (10 Years)

Years Active: 2018-2024
Publications (10 Years): 51

Top Topics

Distributed Learning

Variance Reduction

Adaptive Learning Rate

Top Venues

Publications

Hao Mark Chen, Wayne Luk, Ka Fai Cedric Yiu, Rui Li, Konstantin Mishchenko, Stylianos I. Venieris, Hongxiang Fan
Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inference. CoRR (2024)
Aaron Defazio, Xingyu Yang, Harsh Mehta, Konstantin Mishchenko, Ahmed Khaled, Ashok Cutkosky
The Road Less Scheduled. CoRR (2024)
Nikita Doikov, Konstantin Mishchenko, Yurii E. Nesterov
Super-Universal Regularized Newton Method. SIAM J. Optim. 34 (1) (2024)
Konstantin Mishchenko, Rustem Islamov, Eduard Gorbunov, Samuel Horváth
Partially Personalized Federated Learning: Breaking the Curse of Data Heterogeneity. CoRR (2023)
Aaron Defazio, Konstantin Mishchenko
Learning-Rate-Free Learning by D-Adaptation. CoRR (2023)
Yura Malitsky, Konstantin Mishchenko
Adaptive Proximal Gradient Method for Convex Optimization. CoRR (2023)
Ahmed Khaled, Konstantin Mishchenko, Chi Jin
DoWG Unleashed: An Efficient Universal Parameter-Free Gradient Descent Method. NeurIPS (2023)
Blake E. Woodworth, Konstantin Mishchenko, Francis R. Bach
Two Losses Are Better Than One: Faster Optimization Using a Cheaper Proxy. CoRR (2023)
Blake E. Woodworth, Konstantin Mishchenko, Francis R. Bach
Two Losses Are Better Than One: Faster Optimization Using a Cheaper Proxy. ICML (2023)
Ahmed Khaled, Konstantin Mishchenko, Chi Jin
DoWG Unleashed: An Efficient Universal Parameter-Free Gradient Descent Method. CoRR (2023)
Konstantin Mishchenko
Regularized Newton Method with Global \({\boldsymbol{\mathcal{O}(1/{k}^2)}}\) Convergence. SIAM J. Optim. 33 (3) (2023)
Grigory Malinovsky, Konstantin Mishchenko, Peter Richtárik
Server-Side Stepsizes and Sampling Without Replacement Provably Help in Federated Optimization. DistributedML@CoNEXT (2023)
Konstantin Mishchenko, Slavomír Hanzely, Peter Richtárik
Convergence of First-Order Algorithms for Meta-Learning with Moreau Envelopes. CoRR (2023)
Konstantin Mishchenko, Aaron Defazio
Prodigy: An Expeditiously Adaptive Parameter-Free Learner. CoRR (2023)
Aaron Defazio, Ashok Cutkosky, Harsh Mehta, Konstantin Mishchenko
When, Why and How Much? Adaptive Learning Rate Scheduling by Refinement. CoRR (2023)
Aaron Defazio, Konstantin Mishchenko
Learning-Rate-Free Learning by D-Adaptation. ICML (2023)
Samuel Horváth, Dmitry Kovalev, Konstantin Mishchenko, Peter Richtárik, Sebastian U. Stich
Stochastic distributed learning with gradient quantization and double-variance reduction. Optim. Methods Softw. 38 (1) (2023)
Adil Salim, Laurent Condat, Konstantin Mishchenko, Peter Richtárik
Dualize, Split, Randomize: Toward Fast Nonsmooth Optimization Algorithms. J. Optim. Theory Appl. 195 (1) (2022)
Konstantin Mishchenko, Ahmed Khaled, Peter Richtárik
Proximal and Federated Random Reshuffling. ICML (2022)
Konstantin Mishchenko, Francis R. Bach, Mathieu Even, Blake E. Woodworth
Asynchronous SGD Beats Minibatch SGD Under Arbitrary Delays. CoRR (2022)
Konstantin Mishchenko, Grigory Malinovsky, Sebastian Stich, Peter Richtárik
ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally! CoRR (2022)
Nikita Doikov, Konstantin Mishchenko, Yurii E. Nesterov
Super-Universal Regularized Newton Method. CoRR (2022)
Konstantin Mishchenko, Bokun Wang, Dmitry Kovalev, Peter Richtárik
IntSGD: Adaptive Floatless Compression of Stochastic Gradients. ICLR (2022)
Konstantin Mishchenko, Francis R. Bach, Mathieu Even, Blake E. Woodworth
Asynchronous SGD Beats Minibatch SGD Under Arbitrary Delays. NeurIPS (2022)
Samuel Horváth, Konstantin Mishchenko, Peter Richtárik
Adaptive Learning Rates for Faster Stochastic Gradient Methods. CoRR (2022)
Konstantin Mishchenko, Grigory Malinovsky, Sebastian Stich, Peter Richtárik
ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally! ICML (2022)
Grigory Malinovsky, Konstantin Mishchenko, Peter Richtárik
Server-Side Stepsizes and Sampling Without Replacement Provably Help in Federated Optimization. CoRR (2022)
Konstantin Mishchenko, Bokun Wang, Dmitry Kovalev, Peter Richtárik
IntSGD: Floatless Compression of Stochastic Gradients. CoRR (2021)
Konstantin Mishchenko
) Convergence. CoRR (2021)
Konstantin Mishchenko, Ahmed Khaled, Peter Richtárik
Proximal and Federated Random Reshuffling. CoRR (2021)
Yura Malitsky, Konstantin Mishchenko
Adaptive Gradient Descent without Descent. ICML (2020)
Ahmed Khaled, Konstantin Mishchenko, Peter Richtárik
Tighter Theory for Local SGD on Identical and Heterogeneous Data. AISTATS (2020)
Saeed Soori, Konstantin Mishchenko, Aryan Mokhtari, Maryam Mehri Dehnavi, Mert Gürbüzbalaban
DAve-QN: A Distributed Averaged Quasi-Newton Method with Local Superlinear Convergence Rate. AISTATS (2020)
Konstantin Mishchenko, Filip Hanzely, Peter Richtárik
99% of Worker-Master Communication in Distributed Optimization Is Not Needed. UAI (2020)
Konstantin Mishchenko, Franck Iutzeler, Jérôme Malick
A Distributed Flexible Delay-Tolerant Proximal Gradient Algorithm. SIAM J. Optim. 30 (1) (2020)
Adil Salim, Laurent Condat, Konstantin Mishchenko, Peter Richtárik
Dualize, Split, Randomize: Fast Nonsmooth Optimization Algorithms. CoRR (2020)
Konstantin Mishchenko, Ahmed Khaled, Peter Richtárik
Random Reshuffling: Simple Analysis with Vast Improvements. NeurIPS (2020)
Konstantin Mishchenko, Dmitry Kovalev, Egor Shulgin, Peter Richtárik, Yura Malitsky
Revisiting Stochastic Extragradient. AISTATS (2020)
Konstantin Mishchenko, Ahmed Khaled, Peter Richtárik
Random Reshuffling: Simple Analysis with Vast Improvements. CoRR (2020)
Konstantin Mishchenko, Eduard A. Gorbunov, Martin Takác, Peter Richtárik
Distributed Learning with Compressed Gradient Differences. CoRR (2019)
Konstantin Mishchenko, Dmitry Kovalev, Egor Shulgin, Peter Richtárik, Yura Malitsky
Revisiting Stochastic Extragradient. CoRR (2019)
Ahmed Khaled, Konstantin Mishchenko, Peter Richtárik
First Analysis of Local GD on Heterogeneous Data. CoRR (2019)
Konstantin Mishchenko, Filip Hanzely, Peter Richtárik
99% of Parallel Optimization is Inevitably a Waste of Time. CoRR (2019)
Yura Malitsky, Konstantin Mishchenko
Adaptive gradient descent without descent. CoRR (2019)
Dmitry Kovalev, Konstantin Mishchenko, Peter Richtárik
Stochastic Newton and Cubic Newton Methods with Simple Local Linear-Quadratic Rates. CoRR (2019)
Ahmed Khaled, Konstantin Mishchenko, Peter Richtárik
Better Communication Complexity for Local SGD. CoRR (2019)
Konstantin Mishchenko
Sinkhorn Algorithm as a Special Case of Stochastic Mirror Descent. CoRR (2019)
Konstantin Mishchenko, Mallory Montgomery, Federico Vaggi
A Self-supervised Approach to Hierarchical Forecasting with Applications to Groupwise Synthetic Controls. CoRR (2019)
Filip Hanzely, Konstantin Mishchenko, Peter Richtárik
SEGA: Variance Reduction via Gradient Sketching. NeurIPS (2018)
Filip Hanzely, Konstantin Mishchenko, Peter Richtárik
SEGA: Variance Reduction via Gradient Sketching. CoRR (2018)
Konstantin Mishchenko, Franck Iutzeler, Jérôme Malick, Massih-Reza Amini
A Delay-tolerant Proximal-Gradient Algorithm for Distributed Learning. ICML (2018)