Nicholas Carlini

Publication Activity (10 Years)

Years Active: 2012-2024
Publications (10 Years): 139

Top Venues

CoRR

NeurIPS

USENIX Security Symposium

ICLR

University of California, Berkeley, USA (PHD)

Education History

University of California, Berkeley, USA

PHD

Publications

Yiming Zhang, Avi Schwarzschild, Nicholas Carlini, Zico Kolter, Daphne Ippolito
Forcing Diffuse Distributions out of Language Models. CoRR (2024)
Jonathan Hayase, Ema Borevkovic, Nicholas Carlini, Florian Tramèr, Milad Nasr
Query-Based Adversarial Prompt Generation. CoRR (2024)
Sanghyun Hong, Nicholas Carlini, Alexey Kurakin
Diffusion Denoising as a Certified Defense against Clean-label Poisoning. CoRR (2024)
Robert Hönig, Javier Rando, Nicholas Carlini, Florian Tramèr
Adversarial Perturbations Cannot Reliably Protect Artists From Generative AI. CoRR (2024)
Nicholas Carlini
Cutting through buggy adversarial example defenses: fixing 1 line of code breaks Sabre. CoRR (2024)
Nicholas Carlini, Daniel Paleka, Krishnamurthy (Dj) Dvijotham, Thomas Steinke, Jonathan Hayase, A. Feder Cooper, Katherine Lee, Matthew Jagielski, Milad Nasr, Arthur Conmy, Eric Wallace, David Rolnick, Florian Tramèr
Stealing Part of a Production Language Model. CoRR (2024)
Yuxin Wen, Leo Marchyok, Sanghyun Hong, Jonas Geiping, Tom Goldstein, Nicholas Carlini
Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models. CoRR (2024)
Edoardo Debenedetti, Giorgio Severi, Milad Nasr, Christopher A. Choquette-Choo, Matthew Jagielski, Eric Wallace, Nicholas Carlini, Florian Tramèr
Privacy Side Channels in Machine Learning Systems. USENIX Security Symposium (2024)
Edoardo Debenedetti, Nicholas Carlini, Florian Tramèr
Evading Black-box Classifiers Without Breaking Eggs. SaTML (2024)
Nikhil Kandpal, Matthew Jagielski, Florian Tramèr, Nicholas Carlini
Backdoor Attacks for In-Context Learning with Language Models. CoRR (2023)
Daphne Ippolito, Nicholas Carlini, Katherine Lee, Milad Nasr, Yun William Yu
Reverse-Engineering Decoding Strategies Given Blackbox Access to a Language Generation System. CoRR (2023)
Matthew Jagielski, Milad Nasr, Christopher A. Choquette-Choo, Katherine Lee, Nicholas Carlini
Students Parrot Their Teachers: Membership Inference on Model Distillation. CoRR (2023)
Nicholas Carlini, Matthew Jagielski, Christopher A. Choquette-Choo, Daniel Paleka, Will Pearce, Hyrum Anderson, Andreas Terzis, Kurt Thomas, Florian Tramèr
Poisoning Web-Scale Training Datasets is Practical. CoRR (2023)
Clark W. Barrett, Brad Boyd, Ellie Burzstein, Nicholas Carlini, Brad Chen, Jihye Choi, Amrita Roy Chowdhury, Mihai Christodorescu, Anupam Datta, Soheil Feizi, Kathleen Fisher, Tatsunori Hashimoto, Dan Hendrycks, Somesh Jha, Daniel Kang, Florian Kerschbaum, Eric Mitchell, John C. Mitchell, Zulfikar Ramzan, Khawaja Shams, Dawn Song, Ankur Taly, Diyi Yang
Identifying and Mitigating the Security Risks of Generative AI. CoRR (2023)
Matthew Jagielski, Milad Nasr, Katherine Lee, Christopher A. Choquette-Choo, Nicholas Carlini, Florian Tramèr
Students Parrot Their Teachers: Membership Inference on Model Distillation. NeurIPS (2023)
Andong Hua, Jindong Gu, Zhiyu Xue, Nicholas Carlini, Eric Wong, Yao Qin
Initialization Matters for Adversarial Transfer Learning. CoRR (2023)
Nicholas Carlini, Florian Tramèr, Krishnamurthy (Dj) Dvijotham, Leslie Rice, Mingjie Sun, J. Zico Kolter
(Certified!!) Adversarial Robustness for Free! ICLR (2023)
Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramèr, Borja Balle, Daphne Ippolito, Eric Wallace
Extracting Training Data from Diffusion Models. USENIX Security Symposium (2023)
Chiyuan Zhang, Daphne Ippolito, Katherine Lee, Matthew Jagielski, Florian Tramèr, Nicholas Carlini
Counterfactual Memorization in Neural Language Models. NeurIPS (2023)
Edoardo Debenedetti, Nicholas Carlini, Florian Tramèr
Evading Black-box Classifiers Without Breaking Eggs. CoRR (2023)
Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A. Feder Cooper, Daphne Ippolito, Christopher A. Choquette-Choo, Eric Wallace, Florian Tramèr, Katherine Lee
Scalable Extraction of Training Data from (Production) Language Models. CoRR (2023)
Milad Nasr, Jamie Hayes, Thomas Steinke, Borja Balle, Florian Tramèr, Matthew Jagielski, Nicholas Carlini, Andreas Terzis
Tight Auditing of Differentially Private Machine Learning. CoRR (2023)
Keane Lucas, Matthew Jagielski, Florian Tramèr, Lujo Bauer, Nicholas Carlini
Randomness in ML Defenses Helps Persistent Attackers and Hinders Evaluators. CoRR (2023)
Clark W. Barrett, Brad Boyd, Elie Bursztein, Nicholas Carlini, Brad Chen, Jihye Choi, Amrita Roy Chowdhury, Mihai Christodorescu, Anupam Datta, Soheil Feizi, Kathleen Fisher, Tatsunori Hashimoto, Dan Hendrycks, Somesh Jha, Daniel Kang, Florian Kerschbaum, Eric Mitchell, John C. Mitchell, Zulfikar Ramzan, Khawaja Shams, Dawn Song, Ankur Taly, Diyi Yang
Identifying and Mitigating the Security Risks of Generative AI. Found. Trends Priv. Secur. 6 (1) (2023)
Daphne Ippolito, Florian Tramèr, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher A. Choquette-Choo, Nicholas Carlini
Preventing Generation of Verbatim Memorization in Language Models Gives a False Sense of Privacy. INLG (2023)
Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramèr, Borja Balle, Daphne Ippolito, Eric Wallace
Extracting Training Data from Diffusion Models. CoRR (2023)
Nicholas Carlini, Milad Nasr, Christopher A. Choquette-Choo, Matthew Jagielski, Irena Gao, Anas Awadalla, Pang Wei Koh, Daphne Ippolito, Katherine Lee, Florian Tramèr, Ludwig Schmidt
Are aligned neural networks adversarially aligned? CoRR (2023)
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramèr, Chiyuan Zhang
Quantifying Memorization Across Neural Language Models. ICLR (2023)
Nicholas Carlini
A LLM Assisted Exploitation of AI-Guardian. CoRR (2023)
Zhouxing Shi, Nicholas Carlini, Ananth Balashankar, Ludwig Schmidt, Cho-Jui Hsieh, Alex Beutel, Yao Qin
Effective Robustness against Natural Distribution Shifts for Models with Different Training Data. CoRR (2023)
Milad Nasr, Jamie Hayes, Thomas Steinke, Borja Balle, Florian Tramèr, Matthew Jagielski, Nicholas Carlini, Andreas Terzis
Tight Auditing of Differentially Private Machine Learning. USENIX Security Symposium (2023)
Zhouxing Shi, Nicholas Carlini, Ananth Balashankar, Ludwig Schmidt, Cho-Jui Hsieh, Alex Beutel, Yao Qin
Effective Robustness against Natural Distribution Shifts for Models with Different Training Data. NeurIPS (2023)
Chawin Sitawarin, Florian Tramèr, Nicholas Carlini
Preprocessors Matter! Realistic Decision-Based Attacks on Machine Learning Systems. ICML (2023)
Nicholas Carlini, Milad Nasr, Christopher A. Choquette-Choo, Matthew Jagielski, Irena Gao, Pang Wei Koh, Daphne Ippolito, Florian Tramèr, Ludwig Schmidt
Are aligned neural networks adversarially aligned? NeurIPS (2023)
Sanghyun Hong, Nicholas Carlini, Alexey Kurakin
Publishing Efficient On-device Models Increases Adversarial Vulnerability. SaTML (2023)
A. Feder Cooper, Katherine Lee, James Grimmelmann, Daphne Ippolito, Christopher Callison-Burch, Christopher A. Choquette-Choo, Niloofar Mireshghallah, Miles Brundage, David Mimno, Madiha Zahrah Choksi, Jack M. Balkin, Nicholas Carlini, Christopher De Sa, Jonathan Frankle, Deep Ganguli, Bryant Gipson, Andres Guadamuz, Swee Leng Harris, Abigail Z. Jacobs, Elizabeth Joh, Gautam Kamath, Mark Lemley, Cass Matthews, Christine McLeavey, Corynne McSherry, Milad Nasr, Paul Ohm, Adam Roberts, Tom Rubin, Pamela Samuelson, Ludwig Schubert, Kristen Vaccaro, Luis Villa, Felix Wu, Elana Zeide
Report of the 1st Workshop on Generative AI and Law. CoRR (2023)
Edoardo Debenedetti, Giorgio Severi, Nicholas Carlini, Christopher A. Choquette-Choo, Matthew Jagielski, Milad Nasr, Eric Wallace, Florian Tramèr
Privacy Side Channels in Machine Learning Systems. CoRR (2023)
Matthew Jagielski, Om Thakkar, Florian Tramèr, Daphne Ippolito, Katherine Lee, Nicholas Carlini, Eric Wallace, Shuang Song, Abhradeep Guha Thakurta, Nicolas Papernot, Chiyuan Zhang
Measuring Forgetting of Memorized Training Examples. ICLR (2023)
Daphne Ippolito, Nicholas Carlini, Katherine Lee, Milad Nasr, Yun William Yu
Reverse-Engineering Decoding Strategies Given Blackbox Access to a Language Generation System. INLG (2023)
Chawin Sitawarin, Kornrapat Pongmala, Yizheng Chen, Nicholas Carlini, David A. Wagner
Part-Based Models Improve Adversarial Robustness. ICLR (2023)
Evani Radiya-Dixit, Sanghyun Hong, Nicholas Carlini, Florian Tramèr
Data Poisoning Won't Save You From Facial Recognition. ICLR (2022)
Chawin Sitawarin, Kornrapat Pongmala, Yizheng Chen, Nicholas Carlini, David A. Wagner
Part-Based Models Improve Adversarial Robustness. CoRR (2022)
Roland S. Zimmermann, Wieland Brendel, Florian Tramèr, Nicholas Carlini
Increasing Confidence in Adversarial Robustness Evaluations. CoRR (2022)
Nicholas Carlini, Matthew Jagielski, Chiyuan Zhang, Nicolas Papernot, Andreas Terzis, Florian Tramèr
The Privacy Onion Effect: Memorization is Relative. NeurIPS (2022)
Matthew Jagielski, Om Thakkar, Florian Tramèr, Daphne Ippolito, Katherine Lee, Nicholas Carlini, Eric Wallace, Shuang Song, Abhradeep Thakurta, Nicolas Papernot, Chiyuan Zhang
Measuring Forgetting of Memorized Training Examples. CoRR (2022)
Battista Biggio, Nicholas Carlini, Pavel Laskov, Konrad Rieck, Antonio Emanuele Cinà
Security of Machine Learning (Dagstuhl Seminar 22281). Dagstuhl Reports 12 (7) (2022)
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, Florian Tramèr
Membership Inference Attacks From First Principles. IEEE Symposium on Security and Privacy (2022)
Florian Tramèr, Reza Shokri, Ayrton San Joaquin, Hoang Le, Matthew Jagielski, Sanghyun Hong, Nicholas Carlini
Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets. CCS (2022)
Maura Pintor, Luca Demetrio, Angelo Sotgiu, Ambra Demontis, Nicholas Carlini, Battista Biggio, Fabio Roli
Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examples. NeurIPS (2022)
Nicholas Carlini, Florian Tramèr, Krishnamurthy Dvijotham, J. Zico Kolter
(Certified!!) Adversarial Robustness for Free! CoRR (2022)
Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, Nicholas Carlini
Deduplicating Training Data Makes Language Models Better. ACL (1) (2022)
Florian Tramèr, Reza Shokri, Ayrton San Joaquin, Hoang Le, Matthew Jagielski, Sanghyun Hong, Nicholas Carlini
Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets. CoRR (2022)
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramèr, Chiyuan Zhang
Quantifying Memorization Across Neural Language Models. CoRR (2022)
Chawin Sitawarin, Florian Tramèr, Nicholas Carlini
Preprocessors Matter! Realistic Decision-Based Attacks on Machine Learning Systems. CoRR (2022)
Nicholas Carlini, Andreas Terzis
Poisoning and Backdooring Contrastive Learning. ICLR (2022)
Sanghyun Hong, Nicholas Carlini, Alexey Kurakin
Handcrafted Backdoors in Deep Neural Networks. NeurIPS (2022)
Nicholas Carlini, Vitaly Feldman, Milad Nasr
No Free Lunch in "Privacy for Free: How does Dataset Condensation Help Privacy". CoRR (2022)
Nicholas Carlini, Matthew Jagielski, Chiyuan Zhang, Nicolas Papernot, Andreas Terzis, Florian Tramèr
The Privacy Onion Effect: Memorization is Relative. CoRR (2022)
Daphne Ippolito, Florian Tramèr, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher A. Choquette-Choo, Nicholas Carlini
Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy. CoRR (2022)
Florian Tramèr, Andreas Terzis, Thomas Steinke, Shuang Song, Matthew Jagielski, Nicholas Carlini
Debugging Differential Privacy: A Case Study for Privacy Auditing. CoRR (2022)
Florian Tramèr, Gautam Kamath, Nicholas Carlini
Considerations for Differentially Private Learning with Large-Scale Public Pretraining. CoRR (2022)
Sanghyun Hong, Nicholas Carlini, Alexey Kurakin
Publishing Efficient On-device Models Increases Adversarial Vulnerability. CoRR (2022)
Roland S. Zimmermann, Wieland Brendel, Florian Tramèr, Nicholas Carlini
Increasing Confidence in Adversarial Robustness Evaluations. NeurIPS (2022)
David Berthelot, Rebecca Roelofs, Kihyuk Sohn, Nicholas Carlini, Alexey Kurakin
AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation. ICLR (2022)
Oliver Bryniarski, Nabeel Hingun, Pedro Pachuca, Vincent Wang, Nicholas Carlini
Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descent. ICLR (2022)
Milad Nasr, Shuang Song, Abhradeep Thakurta, Nicolas Papernot, Nicholas Carlini
Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning. IEEE Symposium on Security and Privacy (2021)
Chiyuan Zhang, Daphne Ippolito, Katherine Lee, Matthew Jagielski, Florian Tramèr, Nicholas Carlini
Counterfactual Memorization in Neural Language Models. CoRR (2021)
David Berthelot, Rebecca Roelofs, Kihyuk Sohn, Nicholas Carlini, Alex Kurakin
AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation. CoRR (2021)
Christopher A. Choquette-Choo, Florian Tramèr, Nicholas Carlini, Nicolas Papernot
Label-Only Membership Inference Attacks. ICML (2021)
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, Florian Tramèr
Membership Inference Attacks From First Principles. CoRR (2021)
Nicholas Carlini
How Private is Machine Learning? IH&MMSec (2021)
Nicholas Carlini
Poisoning the Unlabeled Dataset of Semi-Supervised Learning. USENIX Security Symposium (2021)
Nicholas Carlini, Sanjam Garg, Somesh Jha, Saeed Mahloujifar, Mohammad Mahmoody, Florian Tramèr
NeuraCrypt is not private. CoRR (2021)
Oliver Bryniarski, Nabeel Hingun, Pedro Pachuca, Vincent Wang, Nicholas Carlini
Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descent. CoRR (2021)
Nicholas Carlini, Andreas Terzis
Poisoning and Backdooring Contrastive Learning. CoRR (2021)
Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom B. Brown, Dawn Song, Úlfar Erlingsson, Alina Oprea, Colin Raffel
Extracting Training Data from Large Language Models. USENIX Security Symposium (2021)
Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, Nicholas Carlini
Deduplicating Training Data Makes Language Models Better. CoRR (2021)
Dan Hendrycks, Nicholas Carlini, John Schulman, Jacob Steinhardt
Unsolved Problems in ML Safety. CoRR (2021)
Milad Nasr, Shuang Song, Abhradeep Thakurta, Nicolas Papernot, Nicholas Carlini
Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning. CoRR (2021)
Nicholas Carlini
Session details: Session 2A: Machine Learning for Cybersecurity. AISec@CCS (2021)
Maura Pintor, Luca Demetrio, Angelo Sotgiu, Giovanni Manca, Ambra Demontis, Nicholas Carlini, Battista Biggio, Fabio Roli
Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examples. CoRR (2021)
Nicholas Carlini, Samuel Deng, Sanjam Garg, Somesh Jha, Saeed Mahloujifar, Mohammad Mahmoody, Abhradeep Thakurta, Florian Tramèr
Is Private Learning Possible with Instance Encoding? IEEE Symposium on Security and Privacy (2021)
Sanghyun Hong, Nicholas Carlini, Alexey Kurakin
Handcrafted Backdoors in Deep Neural Networks. CoRR (2021)
Nicholas Carlini
Session details: Session 1: Adversarial Machine Learning. AISec@CCS (2021)
Nicholas Carlini
Poisoning the Unlabeled Dataset of Semi-Supervised Learning. CoRR (2021)
Nicholas Carlini, Hany Farid
Evading Deepfake-Image Detectors with White- and Black-Box Attacks. CoRR (2020)
Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, Ludwig Schmidt
Measuring Robustness to Natural Distribution Shifts in Image Classification. NeurIPS (2020)
Florian Tramèr, Jens Behrmann, Nicholas Carlini, Nicolas Papernot, Jörn-Henrik Jacobsen
Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations. ICML (2020)
Nicholas Carlini
A Partial Break of the Honeypots Defense to Catch Adversarial Attacks. CoRR (2020)
Christopher A. Choquette-Choo, Florian Tramèr, Nicholas Carlini, Nicolas Papernot
Label-Only Membership Inference Attacks. CoRR (2020)
Florian Tramèr, Nicholas Carlini, Wieland Brendel, Aleksander Madry
On Adaptive Attacks to Adversarial Example Defenses. NeurIPS (2020)
Nicholas Carlini, Hany Farid
Evading Deepfake-Image Detectors with White- and Black-Box Attacks. CVPR Workshops (2020)
Florian Tramèr, Nicholas Carlini, Wieland Brendel, Aleksander Madry
On Adaptive Attacks to Adversarial Example Defenses. CoRR (2020)
Nicholas Carlini, Matthew Jagielski, Ilya Mironov
Cryptanalytic Extraction of Neural Network Models. CoRR (2020)
Guneet S. Dhillon, Nicholas Carlini
Erratum Concerning the Obfuscated Gradients Attack on Stochastic Activation Pruning. CoRR (2020)
Sadia Afroz, Nicholas Carlini, Ambra Demontis
AISec'20: 13th Workshop on Artificial Intelligence and Security. CCS (2020)
Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Han Zhang, Colin Raffel
FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence. CoRR (2020)
Florian Tramèr, Jens Behrmann, Nicholas Carlini, Nicolas Papernot, Jörn-Henrik Jacobsen
Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations. CoRR (2020)
Nicholas Carlini, Samuel Deng, Sanjam Garg, Somesh Jha, Saeed Mahloujifar, Mohammad Mahmoody, Shuang Song, Abhradeep Thakurta, Florian Tramèr
An Attack on InstaHide: Is Private Learning Possible with Instance Encoding? CoRR (2020)
David Berthelot, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, Colin Raffel
ReMixMatch: Semi-Supervised Learning with Distribution Matching and Augmentation Anchoring. ICLR (2020)

Nicholas Carlini.css-eqlu8i{font-size:small;padding-left:4px;}

Publication Activity (10 Years)

Top Topics

Top Venues

Nicholas Carlini