Background & Research Approach

I hold a Ph.D. in Computer Science from Columbia University, advised by Shree K. Nayar. My research connects computational imaging, audio, and multimodal generation with a broader question: how can intelligent systems use the structure of physical signals and human behavior to perceive, generate, and interact in the real world?

Two principles guide my work: ground learning in domain knowledge, and build around human behavior, participation, and intent. I combine classical methods, physical models, and explicit constraints with deep learning, using each where it contributes most. Acoustic priors, geometric relationships, and motion structure help shape both the inference problem and the architecture of a solution.

I pursue these ideas through end-to-end research and system development, from problem formulation and data preparation to modeling, evaluation, and deployment. My work with Snap Research includes live interactive experiences such as Billie Eilish events, while my speech-denoising research contributed to a granted US patent. Real-time performance, personalization, and usability are part of the research problem: they determine whether a method can become a useful interactive system.

Selected Research Areas

In computational imaging, my work on Trapping Light uses mirror geometry and time-of-flight measurements to reconstruct occluded surfaces. It connects optical design with geometric inversion and hardware validation, establishing a foundation in recovering information through the physics of sensing.

In speech enhancement, I combine signal structure with learned inference. Listening to Sounds of Silence draws on natural pauses in human speech to estimate changing background noise, outperforming tested audio-only and audio-visual baselines using audio alone and generalizing to languages unseen during training. Personalized Dereverberation combines a measured acoustic prior, feature-based Wiener deconvolution, and GAN-based refinement; a one-time recording procedure lets users participate in adapting the model to their voice and environment. These projects connect human behavior and participation with knowledge-guided modeling.

In multimodal generation and interaction, I led the development of DanceCraft, which turns live music into continuous 3D dance motion. Human motion-capture data, graph-based motion selection, and neural transitions work together to balance coherence and diversity, while live input and personalized avatars connect generation to human performance and creative intent. Across my projects, I evaluate methods through objective measurements, perceptual studies, and working systems.

Research Vision

My long-term goal is to connect multimodal perception, knowledge-guided generation, and human-guided interaction. I want to build systems that recover meaningful structure from physical signals, use that understanding to generate coherent behavior, and respond to people in real time. This direction grows from my work in imaging, speech, and motion generation toward a closer relationship between sensing the world and interacting within it.

One part of this vision is wave-based intelligence: combining visual perception with complementary sensing channels to understand aspects of the physical world that images alone may not reveal. My current experience is grounded in computational imaging and audio; I see opportunities to extend these methods to vibration, ultrasound, and RF sensing, where different signal-material interactions may provide information about hidden structure, motion, and physical state.

Alongside richer perception, I am interested in controllable multimodal generation that incorporates domain constraints and human guidance. Interactive creative tools provide a concrete setting for this work; embodied systems offer a longer-term direction for connecting environmental understanding, action generation, and human collaboration. Across these settings, my aim is to make knowledge and human participation integral to how intelligent systems learn and operate.

My Research Interests