Amirkeivan Mohtashami

Publication Activity (10 Years)

Years Active: 2020-2024
Publications (10 Years): 18

Top Topics

Convergence Theorem

Special Properties

Top Venues

Distributed Comput.

Publications

Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Martin Jaggi, Dan Alistarh, Torsten Hoefler, James Hensman
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs. CoRR (2024)
Matteo Pagliardini, Amirkeivan Mohtashami, François Fleuret, Martin Jaggi
DenseFormer: Enhancing Information Flow in Transformers via Depth Weighted Averaging. CoRR (2024)
Amirkeivan Mohtashami, Florian Hartmann, Sian Gooding, Lukas Zilka, Matt Sharifi, Blaise Agüera y Arcas
Social Learning: Towards Collaborative Learning with Large Language Models. CoRR (2023)
Amirkeivan Mohtashami, Matteo Pagliardini, Martin Jaggi
CoTFormer: More Tokens With Attention Make Up For Less Depth. CoRR (2023)
Amirkeivan Mohtashami, Mauro Verzetti, Paul K. Rubenstein
Learning Translation Quality Evaluation on Low Resource Languages from Large Language Models. CoRR (2023)
Amirkeivan Mohtashami, Martin Jaggi
Landmark Attention: Random-Access Infinite Context Length for Transformers. CoRR (2023)
Vitaly Aksenov, Dan Alistarh, Alexandra Drozdova, Amirkeivan Mohtashami
The splay-list: a distribution-adaptive concurrent skip-list. Distributed Comput. 36 (3) (2023)
Zeming Chen, Alejandro Hernández-Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi, Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, Alexandre Sallinen, Alireza Sakhaeirad, Vinitra Swamy, Igor Krawczuk, Deniz Bayazit, Axel Marmet, Syrielle Montariol, Mary-Anne Hartley, Martin Jaggi, Antoine Bosselut
MEDITRON-70B: Scaling Medical Pretraining for Large Language Models. CoRR (2023)
Amirkeivan Mohtashami, Martin Jaggi, Sebastian U. Stich
Special Properties of Gradient Descent with Large Learning Rates. ICML (2023)
Amirkeivan Mohtashami, Martin Jaggi
Random-Access Infinite Context Length for Transformers. NeurIPS (2023)
Amirkeivan Mohtashami, Martin Jaggi, Sebastian U. Stich
On Avoiding Local Minima Using Gradient Descent With Large Learning Rates. CoRR (2022)
Amirkeivan Mohtashami, Martin Jaggi, Sebastian U. Stich
Masked Training of Neural Networks with Partial Gradients. AISTATS (2022)
Amirkeivan Mohtashami, Sebastian U. Stich, Martin Jaggi
Characterizing & Finding Good Data Orderings for Fast Convergence of Sequential Gradient Methods. CoRR (2022)
Sebastian U. Stich, Amirkeivan Mohtashami, Martin Jaggi
Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates. CoRR (2021)
Amirkeivan Mohtashami, Martin Jaggi, Sebastian U. Stich
Simultaneous Training of Partially Masked Neural Networks. CoRR (2021)
Sebastian U. Stich, Amirkeivan Mohtashami, Martin Jaggi
Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates. AISTATS (2021)
Vitaly Aksenov, Dan Alistarh, Alexandra Drozdova, Amirkeivan Mohtashami
The Splay-List: A Distribution-Adaptive Concurrent Skip-List. DISC (2020)
Vitaly Aksenov, Dan Alistarh, Alexandra Drozdova, Amirkeivan Mohtashami
The Splay-List: A Distribution-Adaptive Concurrent Skip-List. CoRR (2020)