publications

2025

1. UIST

   

   Thing2Reality: Enabling Spontaneous Creation of 3D Objects from 2D Content using Generative AI in XR Meetings

   Erzhen Hu, Mingyi Li, Jungtaek Hong, Xun Qian, Alex Olwal, David Kim, Seongkook Heo, and Ruofei Du

   In Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology, 2025

   Abs DOI

   During remote communication, participants often share both digital and physical content, such as product designs, digital assets, and environments, to enhance mutual understanding. Recent advances in augmented communication have facilitated users to swiftly create and share digital 2D copies of physical objects from video feeds into a shared space. However, conventional 2D representations of digital objects limits spatial referencing in immersive environments. To address this, we propose Thing2Reality, an Extended Reality (XR) meeting platform that facilitates spontaneous discussions of both digital and physical items during remote sessions. With Thing2Reality, users can quickly materialize ideas or objects in immersive environments and share them as conditioned multiview renderings or 3D Gaussians. Thing2Reality enables users to interact with remote objects or discuss concepts in a collaborative manner. Our user studies revealed that the ability to interact with and manipulate 3D representations of objects significantly enhances the efficiency of discussions, with the potential to augment discussion of 2D artifacts.

2. UIST

   

   DialogLab: Authoring, Simulating, and Testing Dynamic Human-AI Group Conversations

   Erzhen Hu, Yanhe Chen, Mingyi Li, Vrushank Phadnis, Pingmei Xu, Xun Qian, Alex Olwal, David Kim, Seongkook Heo, and Ruofei Du

   In Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology, 2025

   Abs DOI

   Designing compelling multi-party conversations involving both humans and AI agents presents significant challenges, particularly in balancing scripted structure with emergent, human-like interactions. We introduce DialogLab, a prototyping toolkit for authoring, simulating, and testing hybrid human-AI dialogues. DialogLab provides a unified interface to configure conversational scenes, define agent personas, manage group structures, specify turn-taking rules, and orchestrate transitions between scripted narratives and improvisation. Crucially, DialogLab allows designers to introduce controlled deviations from the script—through configurable agents that emulate human unpredictability—to systematically probe how conversations adapt and recover. DialogLab facilitates rapid iteration and evaluation of complex, dynamic multi-party human-AI dialogues. An evaluation with both end users and domain experts demonstrates that DialogLab supports efficient iteration and structured verification, with applications in training, rehearsal, and research on social dynamics. Our findings show the value of integrating real-time, human-in-the-loop improvisation with structured scripting to support more realistic and adaptable multi-party conversation design.

2024

1. CHI

   

   CareJournal: A Voice-Based Conversational Agent for Supporting Care Communications

   John Rudnik, Sharadhi Raghuraj, Mingyi Li, and Robin N. Brewer

   In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 2024

   Abs DOI

   Effective communication between older adult care recipients and unpaid caregivers is essential to both care partners’ well-being. To understand communication in care relationships, we conducted a two-part study with older adult care recipients and caregivers. First, we conducted a two-week diary study to gain insight into care-related communication challenges. While caregivers discussed the benefits of emotional attachment, care recipients expressed concerns about emotional fluctuation and losing autonomy. These findings, along with literature on self-disclosure and conversational scaffolding informed our design of CareJournal—a voice-based conversational agent that supports care-related disclosure between care partners. We evaluated CareJournal with 40 care partners to inform future design considerations and learn more about their communication practices. Our findings highlight the impact of distance and tensions between care and independence, providing insight into how care partners imagine computer-mediated care communication impacting their relationships.

2. CSCW

   

   ProcessGallery: Contrasting Early and Late Iterations for Design Principle Learning

   Yu-Chun Grace Yen, Jane L. E, Hyoungwook Jin, Mingyi Li, Grace Lin, Isabelle Yan Pan, and Steven P. Dow

   Proc. ACM Hum.-Comput. Interact., Apr 2024

   Abs DOI

   Traditional design galleries enable users to search for examples based on surface attributes (e.g., color or style), and largely obscure underlying principles (e.g., hierarchy or readability). We conducted three studies to explore how galleries could be constructed to help novices learn key design principles. Study 1 revealed that novices gain perspective by observing how designs evolve throughout a process. Study 2 found that novices are better at identifying design issues when viewing iterations that show improvements for just one principle at a time, rather than multiple. Building on these insights, we created ProcessGallery, a tool that enables users to browse contrasting pairs of early-and-late iterations of designs that highlight key improvements organized by design principles. In Study 3, a within-subjects experiment, sixteen participants iterated on a seed design after viewing examples in ProcessGallery versus a traditional gallery. Using ProcessGallery, participants found more appropriate examples, assessed designs better, and preferred ProcessGallery for learning compared to a traditional gallery.

3. C&C

   

   When to Give Feedback: Exploring Tradeoffs in the Timing of Design Feedback

   Jane L. E, Yu-Chun Grace Yen, Isabelle Yan Pan, Grace Lin, Mingyi Li, Hyoungwook Jin, Mengyi Chen, Haijun Xia, and Steven P. Dow

   In Proceedings of the 16th Conference on Creativity & Cognition, Chicago, IL, USA, 2024

   Abs DOI

   Advances in AI have opened up the potential for creativity tools to computationally generate design feedback. In a future when designers can request feedback anytime on demand, how would the timing of these requests impact novices’ creative learning processes? What are the tradeoffs of providing access to feedback throughout a design task (in-action) versus only providing feedback after (on-action)? We explored these questions through a Wizard-of-Oz study (N=20) using an interactive design probe, where participants could request feedback either throughout the design process or only after they complete a full draft. We found that in-action participants frequently request feedback, resulting in better improvements as indicated by a greater decrease in issues in their final design. However, we saw that in-action feedback can also risk users overly relying on feedback instead of engaging in more holistic self-evaluation. We discuss the implications of our insights on designing tools for creative feedback.