Kenneth Holstein

Publication Activity (10 Years)

Years Active: 2014-2024
Publications (10 Years): 88

Top Topics

Event Sequences

Causal Structure

Intelligent Tutoring

Top Venues

Proc. ACM Hum. Comput. Interact.

Publications

Alicia DeVrio, Motahhare Eslami, Kenneth Holstein
Building, Shifting, & Employing Power: A Taxonomy of Responses From Below to Algorithmic Harm. FAccT (2024)
Anna Kawakami, Amanda Coston, Hoda Heidari, Kenneth Holstein, Haiyi Zhu
Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and Use. CoRR (2024)
Anna Kawakami, Amanda Coston, Haiyi Zhu, Hoda Heidari, Kenneth Holstein
The Situate AI Guidebook: Co-Designing a Toolkit to Support Multi-Stakeholder Early-stage Deliberations Around Public Sector AI Proposals. CoRR (2024)
Anna Kawakami, Amanda Coston, Haiyi Zhu, Hoda Heidari, Kenneth Holstein
The Situate AI Guidebook: Co-Designing a Toolkit to Support Multi-Stakeholder, Early-stage Deliberations Around Public Sector AI Proposals. CHI (2024)
Frederic Gmeiner, Jamie Lynn Conlin, Eric Handa Tang, Nikolas Martelaro, Kenneth Holstein
An Evidence-based Workflow for Studying and Designing Learning Supports for Human-AI Co-creation. CHI Extended Abstracts (2024)
Tzu-Sheng Kuo, Aaron Halfaker, Zirui Cheng, Jiwoo Kim, Meng-Hsin Wu, Tongshuang Wu, Kenneth Holstein, Haiyi Zhu
Wikibench: Community-Driven Data Curation for AI Evaluation on Wikipedia. CoRR (2024)
Tzu-Sheng Kuo, Aaron Lee Halfaker, Zirui Cheng, Jiwoo Kim, Meng-Hsin Wu, Tongshuang Wu, Kenneth Holstein, Haiyi Zhu
Wikibench: Community-Driven Data Curation for AI Evaluation on Wikipedia. CHI (2024)
Zahra Ashktorab, Gagan Bansal, Zana Buçinca, Kenneth Holstein, Jessica Hullman, Alison Marie Smith-Renner, Tongshuang Wu, Wenjuan Zhang
Trust and Reliance in Evolving Human-AI Workflows (TREW). CHI Extended Abstracts (2024)
Ningjing Tang, Jiayin Zhi, Tzu-Sheng Kuo, Calla Kainaroi, Jeremy J. Northup, Kenneth Holstein, Haiyi Zhu, Hoda Heidari, Hong Shen
AI Failure Cards: Understanding and Supporting Grassroots Efforts to Mitigate AI Failures in Homeless Services. FAccT (2024)
Luke Guerdan, Amanda Coston, Kenneth Holstein, Zhiwei Steven Wu
Predictive Performance Comparison of Decision Policies Under Confounding. CoRR (2024)
Tzu-Sheng Kuo, Hong Shen, Jisoo Geum, Nev Jones, Jason I. Hong, Haiyi Zhu, Kenneth Holstein
Understanding Frontline Workers' and Unhoused Individuals' Perspectives on AI Used in Homeless Services. CoRR (2023)
Kexin Bella Yang, Vanessa Echeverría, Zijing Lu, Hongyu Mao, Kenneth Holstein, Nikol Rummel, Vincent Aleven
Pair-Up: Prototyping Human-AI Co-orchestration of Dynamic Transitions between Individual and Collaborative Learning in the Classroom. CHI (2023)
Wesley Hanwen Deng, Michelle S. Lam, Ángel Alexander Cabrera, Danaë Metaxa, Motahhare Eslami, Kenneth Holstein
Supporting User Engagement in Testing, Auditing, and Contesting AI. CSCW Companion (2023)
Anna Kawakami, Luke Guerdan, Yanghuidi Cheng, Kate Glazko, Matthew L. Lee, Scott A. Carter, Nikos Aréchiga, Haiyi Zhu, Kenneth Holstein
Training Towards Critical Use: Learning to Situate AI Predictions Relative to Human Knowledge. CI (2023)
Wesley Hanwen Deng, Bill Boyuan Guo, Alicia DeVrio, Hong Shen, Motahhare Eslami, Kenneth Holstein
Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice. CHI (2023)
Luke Guerdan, Amanda Coston, Zhiwei Steven Wu, Kenneth Holstein
Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making. CoRR (2023)
Katelyn Morrison, Donghoon Shin, Kenneth Holstein, Adam Perer
Evaluating the Impact of Human Explanation Strategies on Human-AI Visual Decision-Making. Proc. ACM Hum. Comput. Interact. 7 (CSCW1) (2023)
Anna Kawakami, Amanda Coston, Haiyi Zhu, Hoda Heidari, Kenneth Holstein
Recentering Validity Considerations through Early-Stage Deliberations Around AI and Policy Design. CoRR (2023)
Luke Guerdan, Amanda Coston, Kenneth Holstein, Zhiwei Steven Wu
Counterfactual Prediction Under Outcome Measurement Error. FAccT (2023)
Gagan Bansal, Zana Buçinca, Kenneth Holstein, Jessica Hullman, Alison Marie Smith-Renner, Simone Stumpf, Sherry Wu
Workshop on Trust and Reliance in AI-Human Teams (TRAIT). CHI Extended Abstracts (2023)
Wesley Hanwen Deng, Motahhare Eslami, Kenneth Holstein
Towards "Anytime, Anywhere" Community Learning and Engagement around the Design of Public Sector AI. CoRR (2023)
Tzu-Sheng Kuo, Hong Shen, Jisoo Geum, Nev Jones, Jason I. Hong, Haiyi Zhu, Kenneth Holstein
Understanding Frontline Workers' and Unhoused Individuals' Perspectives on AI Used in Homeless Services. CHI (2023)
Luke Guerdan, Amanda Coston, Zhiwei Steven Wu, Kenneth Holstein
Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making. FAccT (2023)
Ruyuan Wan, Adriana Alvarado Garcia, Devansh Saxena, Catalina Vajiac, Anna Kawakami, Logan Stapleton, Haiyi Zhu, Kenneth Holstein, Heloisa Candello, Karla A. Badillo-Urquiola
Community-driven AI: Empowering people through responsible data-driven decision-making. CSCW Companion (2023)
Wesley Hanwen Deng, Nur Yildirim, Monica Chang, Motahhare Eslami, Kenneth Holstein, Michael Madaio
Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practice. FAccT (2023)
Duri Long, Jessica Roberts, Brian Magerko, Kenneth Holstein, Daniella DiPaola, Fred Martin
AI Literacy: Finding Common Threads between Education, Design, Policy, and Explainability. CHI Extended Abstracts (2023)
Chinmay Kulkarni, Tongshuang Wu, Kenneth Holstein, Q. Vera Liao, Min Kyung Lee, Mina Lee, Hariharan Subramonyam
LLMs and the Infrastructure of CSCW. CSCW Companion (2023)
Frederic Gmeiner, Humphrey Yang, Lining Yao, Kenneth Holstein, Nikolas Martelaro
Exploring Challenges and Opportunities to Support Designers in Learning to Co-create with AI-based Manufacturing Design Tools. CHI (2023)
Frederic Gmeiner, Humphrey Yang, Lining Yao, Kenneth Holstein, Nikolas Martelaro
Exploring Challenges and Opportunities to Support Designers in Learning to Co-create with AI-based Manufacturing Design Tools. CoRR (2023)
Kenneth Holstein, Maria De-Arteaga, Lakshmi Tumati, Yanghuidi Cheng
Toward Supporting Perceptual Complementarity in Human-AI Collaboration via Reflection on Unobservables. Proc. ACM Hum. Comput. Interact. 7 (CSCW1) (2023)
Ángel Alexander Cabrera, Erica Fu, Donald Bertucci, Kenneth Holstein, Ameet Talwalkar, Jason I. Hong, Adam Perer
Zeno: An Interactive Framework for Behavioral Evaluation of Machine Learning. CoRR (2023)
Anna Kawakami, Luke Guerdan, Yanghuidi Cheng, Matthew L. Lee, Scott A. Carter, Nikos Aréchiga, Kate Glazko, Haiyi Zhu, Kenneth Holstein
Training Towards Critical Use: Learning to Situate AI Predictions Relative to Human Knowledge. CoRR (2023)
Luke Guerdan, Amanda Coston, Kenneth Holstein, Zhiwei Steven Wu
Counterfactual Prediction Under Outcome Measurement Error. CoRR (2023)
Ángel Alexander Cabrera, Erica Fu, Donald Bertucci, Kenneth Holstein, Ameet Talwalkar, Jason I. Hong, Adam Perer
Zeno: An Interactive Framework for Behavioral Evaluation of Machine Learning. CHI (2023)
Logan Stapleton, Devansh Saxena, Anna Kawakami, Tonya Nguyen, Asbjørn Ammitzbøll Flügge, Motahhare Eslami, Naja L. Holten Møller, Min Kyung Lee, Shion Guha, Kenneth Holstein, Haiyi Zhu
Who Has an Interest in "Public Interest Technology"?: Critical Questions for Working with Local Governments & Impacted Communities. CSCW Companion (2022)
Alicia DeVos, Aditi Dhabalia, Hong Shen, Kenneth Holstein, Motahhare Eslami
Toward User-Driven Algorithm Auditing: Investigating users' strategies for uncovering harmful algorithmic behavior. CHI (2022)
Logan Stapleton, Hao Fei Cheng, Anna Kawakami, Venkatesh Sivaraman, Yanghuidi Cheng, Diana Qing, Adam Perer, Kenneth Holstein, Zhiwei Steven Wu, Haiyi Zhu
Extended Analysis of "How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions". CoRR (2022)
Anna Kawakami, Venkatesh Sivaraman, Hao Fei Cheng, Logan Stapleton, Yanghuidi Cheng, Diana Qing, Adam Perer, Zhiwei Steven Wu, Haiyi Zhu, Kenneth Holstein
Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support. CoRR (2022)
Connie W. Chau, Kenneth Holstein, Michael Madaio
Part of the Conversation: Workforce Professionals' Perspectives on the Roles and Impacts of Workforce Technologies. Proc. ACM Hum. Comput. Interact. 6 (CSCW2) (2022)
Kenneth Holstein, Vincent Aleven
Designing for Human-AI Complementarity in K-12 Education. AI Mag. 43 (2) (2022)
Charvi Rastogi, Liu Leqi, Kenneth Holstein, Hoda Heidari
A Unifying Framework for Combining Complementary Strengths of Humans and ML toward Better Predictive Decision-Making. CoRR (2022)
Wesley Hanwen Deng, Manish Nagireddy, Michelle Seng Ah Lee, Jatinder Singh, Zhiwei Steven Wu, Kenneth Holstein, Haiyi Zhu
Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits. FAccT (2022)
Gagan Bansal, Alison Marie Smith-Renner, Zana Buçinca, Tongshuang Wu, Kenneth Holstein, Jessica Hullman, Simone Stumpf
Workshop on Trust and Reliance in AI-Human Teams (TRAIT). CHI Extended Abstracts (2022)
Vince Bartle, Janice Lyu, Freesoul El Shabazz-Thompson, Yunmin Oh, Angela Anqi Chen, Yu-Jan Chang, Kenneth Holstein, Nicola Dell
"A Second Voice": Investigating Opportunities and Challenges for Interactive Voice Assistants to Support Home Health Aides. CHI (2022)
Hao Fei Cheng, Logan Stapleton, Anna Kawakami, Venkatesh Sivaraman, Yanghuidi Cheng, Diana Qing, Adam Perer, Kenneth Holstein, Zhiwei Steven Wu, Haiyi Zhu
How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions. CHI (2022)
Anna Kawakami, Venkatesh Sivaraman, Logan Stapleton, Hao Fei Cheng, Adam Perer, Zhiwei Steven Wu, Haiyi Zhu, Kenneth Holstein
"Why Do I Care What's Similar?" Probing Challenges in AI-Assisted Child Welfare Decision-Making through Worker-AI Interface Design Concepts. Conference on Designing Interactive Systems (2022)
Anna Kawakami, Luke Guerdan, Yang Cheng, Anita Sun, Alison Hu, Kate Glazko, Nikos Aréchiga, Matthew L. Lee, Scott A. Carter, Haiyi Zhu, Kenneth Holstein
Towards a Learner-Centered Explainable AI: Lessons from the learning sciences. CoRR (2022)
Wesley Hanwen Deng, Manish Nagireddy, Michelle Seng Ah Lee, Jatinder Singh, Zhiwei Steven Wu, Kenneth Holstein, Haiyi Zhu
Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits. CoRR (2022)
Kenneth Holstein, Maria De-Arteaga, Lakshmi Tumati, Yanghuidi Cheng
Toward Supporting Perceptual Complementarity in Human-AI Collaboration via Reflection on Unobservables. CoRR (2022)
Wesley Hanwen Deng, Bill Boyuan Guo, Alicia DeVrio, Hong Shen, Motahhare Eslami, Kenneth Holstein
Understanding Practices, Challenges, and Opportunities for User-Driven Algorithm Auditing in Industry Practice. CoRR (2022)
Anna Kawakami, Venkatesh Sivaraman, Hao Fei Cheng, Logan Stapleton, Yanghuidi Cheng, Diana Qing, Adam Perer, Zhiwei Steven Wu, Haiyi Zhu, Kenneth Holstein
Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support. CHI (2022)
Connie W. Chau, Kenneth Holstein, Michael Madaio
Part of the Conversation: Workforce Professionals' Perspectives on the Roles and Impacts of Workforce Technologies. Proc. ACM Hum. Comput. Interact. 6 (CSCW2) (2022)
Frederic Gmeiner, Kenneth Holstein, Nikolas Martelaro
Team Learning as a Lens for Designing Human-AI Co-Creative Systems. CoRR (2022)
Logan Stapleton, Min Hun Lee, Diana Qing, Marya Wright, Alexandra Chouldechova, Kenneth Holstein, Zhiwei Steven Wu, Haiyi Zhu
Imagining new futures beyond predictive systems in child welfare: A qualitative study with impacted stakeholders. CoRR (2022)
Donghoon Shin, Sachin Grover, Kenneth Holstein, Adam Perer
Characterizing Human Explanation Strategies to Inform the Design of Explainable AI for Building Damage Assessment. CoRR (2021)
Hong Shen, Alicia DeVos, Motahhare Eslami, Kenneth Holstein
Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. CoRR (2021)
Kexin Bella Yang, Xuejian Wang, Vanessa Echeverría, LuEttaMae Lawrence, Kenneth Holstein, Nikol Rummel, Vincent Aleven
SimPairing - Exploring Dynamic Pairing Policies through Historical Data Simulation and User-centered Research. EDM (2021)
Kenneth Holstein, Vincent Aleven
Designing for human-AI complementarity in K-12 education. CoRR (2021)
LuEttaMae Lawrence, Kenneth Holstein, Susan R. Berman, Stephen Fancsali, Bruce M. McLaren, Steven Ritter, Vincent Aleven
Teachers' Orchestration Needs During the Shift to Remote Learning. EC-TEL (2021)
Hong Shen, Alicia DeVos, Motahhare Eslami, Kenneth Holstein
Everyday Algorithm Auditing: Understanding the Power of Everyday Users in Surfacing Harmful Algorithmic Behaviors. Proc. ACM Hum. Comput. Interact. 5 (CSCW2) (2021)
Kexin Bella Yang, Tomohiro Nagashima, Junhui Yao, Joseph Jay Williams, Kenneth Holstein, Vincent Aleven
Can Crowds Customize Instructional Materials with Minimal Expert Guidance?: Exploring Teacher-guided Crowdsourcing for Improving Hints in an AI-based Tutor. Proc. ACM Hum. Comput. Interact. 5 (CSCW1) (2021)
Kenneth Holstein, Shayan Doroudi
Equity and Artificial Intelligence in Education: Will "AIEd" Amplify or Alleviate Inequities in Education? CoRR (2021)
Shayan Doroudi, Kenneth Holstein, Petr Johanes
Probing Learning Scientists' Beliefs About Learning and Science. ICLS (2020)
Vanessa Echeverría, Kenneth Holstein, Jennifer Huang, Jonathan Sewall, Nikol Rummel, Vincent Aleven
Exploring Human-AI Control Over Dynamic Transitions Between Individual and Collaborative Learning. EC-TEL (2020)
Kenneth Holstein, Vincent Aleven, Nikol Rummel
A Conceptual Framework for Human-AI Hybrid Adaptivity in Education. AIED (1) (2020)
Pengcheng An, Kenneth Holstein, Bernice d'Anjou, Berry Eggen, Saskia Bakker
The TA Framework: Designing Real-time Teaching Augmentation for K-12 Classrooms. CHI (2020)
Kenneth Holstein, Erik Harpstead, Rebecca Gulotta, Jodi Forlizzi
Replay Enactments: Exploring Possible Futures through Historical Data. Conference on Designing Interactive Systems (2020)
Pengcheng An, Kenneth Holstein, Bernice d'Anjou, Berry Eggen, Saskia Bakker
The TA Framework: Designing Real-time Teaching Augmentation for K-12 Classrooms. CoRR (2020)
Stephen E. Fancsali, Kenneth Holstein, Michael Sandbothe, Steven Ritter, Bruce M. McLaren, Vincent Aleven
Towards Practical Detection of Unproductive Struggle. AIED (2) (2020)
Kenneth Holstein, Bruce M. McLaren, Vincent Aleven
Designing for Complementarity: Teacher and Student Needs for Orchestration Support in AI-Enhanced Classrooms. AIED (1) (2019)
Chuankai Zhang, Yanzun Huang, Jingyu Wang, Dongyang Lu, Weiqi Fang, John C. Stamper, Stephen Fancsali, Kenneth Holstein, Vincent Aleven
Early Detection of Wheel Spinning: Comparison across Tutors, Models, Features, and Operationalizations. EDM (2019)
Kenneth Holstein, Bruce M. McLaren, Vincent Aleven
Co-Designing a Real-Time Classroom Orchestration Tool to Support Teacher-AI Complementarity. J. Learn. Anal. 6 (2) (2019)
Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé III, Miroslav Dudík, Hanna M. Wallach
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need? CHI (2019)
Kenneth Holstein, Gena Hong, Mera Tegene, Bruce M. McLaren, Vincent Aleven
The classroom as a dashboard: co-designing wearable cognitive augmentation for K-12 teachers. LAK (2018)
Kenneth Holstein, Bruce M. McLaren, Vincent Aleven
Informing the Design of Teacher Awareness Tools through Causal Alignment Analysis. ICLS (2018)
Kenneth Holstein, Zac Yu, Jonathan Sewall, Octav Popescu, Bruce M. McLaren, Vincent Aleven
Opening Up an Intelligent Tutoring System Development Environment for Extensible Student Modeling. AIED (1) (2018)
Kenneth Holstein, Bruce M. McLaren, Vincent Aleven
Student Learning Benefits of a Mixed-Reality Teacher Awareness Tool in AI-Enhanced Classrooms. AIED (1) (2018)
Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé III, Miroslav Dudík, Hanna M. Wallach
Improving fairness in machine learning systems: What do industry practitioners need? CoRR (2018)
Yanjin Long, Kenneth Holstein, Vincent Aleven
What exactly do students learn when they practice equation solving?: refining knowledge components with the additive factors model. LAK (2018)
Anouschka van Leeuwen, Nikol Rummel, Kenneth Holstein, Bruce M. McLaren, Vincent Aleven, Inge Molenaar, Carolien Knoop van Campen, Baruch B. Schwarz, Naomi Prusak, Osama Swidan, Avi Segal, Kobi Gal
Orchestration tools for teachers in the context of individual and collaborative learning: what information do teachers need and what do they do with it? ICLS (2018)
Kenneth Holstein, Bruce M. McLaren, Vincent Aleven
SPACLE: investigating learning across virtual and physical spaces using spatial replays. LAK (2017)
Kenneth Holstein, Bruce M. McLaren, Vincent Aleven
Intelligent tutors as teachers' aides: exploring teacher needs for real-time analytics in blended classrooms. LAK (2017)
Shayan Doroudi, Kenneth Holstein, Vincent Aleven, Emma Brunskill
Sequence Matters, But How Do I Discover How? Towards a Workflow for Evaluating Activity Sequences from Data. EDM (Workshops) (2016)
Vincent Aleven, Franceska Xhakaj, Kenneth Holstein, Bruce M. McLaren
Developing a Teacher Dashboard For Use with Intelligent Tutoring Systems. IWTA@EC-TEL (2016)
Shayan Doroudi, Kenneth Holstein, Vincent Aleven, Emma Brunskill
Sequence Matters, But How Exactly? A Method for Evaluating Activity Sequences from Data. EDM (2016)
Kenneth Holstein, Franceska Xhakaj, Vincent Aleven, Bruce M. McLaren
Luna: A Dashboard for Teachers Using Intelligent Tutoring Systems. IWTA@EC-TEL (2016)
Christopher G. Lucas, Kenneth Holstein, Michael Pacer
Inferring causal structure and hidden causes from event sequences. CogSci (2015)
Shayan Doroudi, Kenneth Holstein, Vincent Aleven, Emma Brunskill
Towards Understanding How to Leverage Sense-making, Induction/Refinement and Fluency to Improve Robust Learning. EDM (2015)
Christopher G. Lucas, Kenneth Holstein, Charles Kemp
Discovering hidden causes using statistical evidence. CogSci (2014)