Ryo Karakida

Publication Activity (10 Years)

Years Active: 2016-2024
Publications (10 Years): 36

Top Topics

Fisher Information

Learning Curves

Boltzmann Machines

Natural Gradient

Top Venues

Publications

Satoki Ishikawa, Ryo Karakida
On the Parameterization of Second-Order Optimization Effective towards the Infinite Width. ICLR (2024)
Ryo Karakida, Toshihiro Ota, Masato Taki
Hierarchical Associative Memory, Parallelized MLP-Mixer, and Symmetry Breaking. CoRR (2024)
Han Bao, Ryuichiro Hataya, Ryo Karakida
Self-attention Networks Localize When QK-eigenspectrum Concentrates. CoRR (2024)
Satoki Ishikawa, Ryo Karakida
On the Parameterization of Second-Order Optimization Effective Towards the Infinite Width. CoRR (2023)
Kaito Watanabe, Kotaro Sakamoto, Ryo Karakida, Sho Sonoda, Shun-ichi Amari
Deep learning in random neural fields: Numerical experiments via neural tangent kernel. Neural Networks 160 (2023)
Tomohiro Hayase, Ryo Karakida
MLP-Mixer as a Wide and Sparse MLP. CoRR (2023)
Ryo Karakida, Tomoumi Takase, Tomohiro Hayase, Kazuki Osawa
Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias. ICML (2023)
Toshihiro Ota, Ryo Karakida
Attention in a Family of Boltzmann Machines Emerging From Modern Hopfield Networks. Neural Comput. 35 (8) (2023)
Kaito Watanabe, Kotaro Sakamoto, Ryo Karakida, Sho Sonoda, Shun-ichi Amari
Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent Kernel. CoRR (2022)
Ryo Karakida, Tomoumi Takase, Tomohiro Hayase, Kazuki Osawa
Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias. CoRR (2022)
Toshihiro Ota, Ryo Karakida
Attention in a family of Boltzmann machines emerging from modern Hopfield networks. CoRR (2022)
Ryo Karakida, Shotaro Akaho
Learning curves for continual learning in neural networks: Self-knowledge transfer and forgetting. ICLR (2022)
Tomohiro Hayase, Ryo Karakida
The Spectrum of Fisher Information of Deep Networks Achieving Dynamical Isometry. AISTATS (2021)
Tomoumi Takase, Ryo Karakida, Hideki Asoh
Self-paced data augmentation for training neural networks. Neurocomputing 442 (2021)
Ryo Karakida, Shotaro Akaho
Learning Curves for Sequential Training of Neural Networks: Self-Knowledge Transfer and Forgetting. CoRR (2021)
Ryo Karakida, Shotaro Akaho, Shun-ichi Amari
Pathological Spectra of the Fisher Information Metric and Its Variants in Deep Neural Networks. Neural Comput. 33 (8) (2021)
Tomoumi Takase, Ryo Karakida, Hideki Asoh
Self-paced Data Augmentation for Training Neural Networks. CoRR (2020)
Ryo Karakida, Kazuki Osawa
Understanding Approximate Fisher Information for Fast Convergence of Natural Gradient Descent in Wide Neural Networks. NeurIPS (2020)
Tomohiro Hayase, Ryo Karakida
The Spectrum of Fisher Information of Deep Networks Achieving Dynamical Isometry. CoRR (2020)
Ryo Karakida, Kazuki Osawa
Understanding Approximate Fisher Information for Fast Convergence of Natural Gradient Descent in Wide Neural Networks. CoRR (2020)
Ryo Karakida, Shotaro Akaho, Shun-ichi Amari
Pathological spectra of the Fisher information metric and its variants in deep neural networks. CoRR (2019)
Ryo Karakida, Shotaro Akaho, Shun-ichi Amari
The Normalization Method for Alleviating Pathological Sharpness in Wide Neural Networks. NeurIPS (2019)
Ryo Karakida, Shotaro Akaho, Shun-ichi Amari
Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach. AISTATS (2019)
Ryo Karakida, Shotaro Akaho, Shun-ichi Amari
The Normalization Method for Alleviating Pathological Sharpness in Wide Neural Networks. CoRR (2019)
Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi, Marco Cuturi
Information Geometry for Regularized Optimal Transport and Barycenters of Patterns. Neural Comput. 31 (5) (2019)
Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi
Fisher Information and Natural Gradient Learning in Random Deep Networks. AISTATS (2019)
Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi
Fisher Information and Natural Gradient Learning of Random Deep Networks. CoRR (2018)
Ryo Karakida, Shotaro Akaho, Shun-ichi Amari
Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach. CoRR (2018)
Shun-ichi Amari, Tomoko Ozeki, Ryo Karakida, Yuki Yoshida, Masato Okada
Dynamics of Learning in MLP: Natural Gradient and Singularity Revisited. Neural Comput. 30 (1) (2018)
Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi
Statistical Neurodynamics of Deep Networks: Geometry of Signal Spaces. CoRR (2018)
Ryo Karakida, Shun-ichi Amari
Information Geometry of Wasserstein Divergence. GSI (2017)
Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi
Information Geometry Connecting Wasserstein Distance and Kullback-Leibler Divergence via the Entropy-Relaxed Transportation Problem. CoRR (2017)
Yoshihiro Nagano, Ryo Karakida, Masato Okada
Concept Formation and Dynamics of Repeated Inference in Deep Generative Models. CoRR (2017)
Ryo Karakida, Masato Okada, Shun-ichi Amari
Maximum likelihood learning of RBMs with Gaussian visible units on the Stiefel manifold. ESANN (2016)
Ryo Karakida, Masato Okada, Shun-ichi Amari
Adaptive Natural Gradient Learning Algorithms for Unnormalized Statistical Models. ICANN (1) (2016)
Ryo Karakida, Masato Okada, Shun-ichi Amari
Dynamical analysis of contrastive divergence learning: Restricted Boltzmann machines with Gaussian visible units. Neural Networks 79 (2016)