Research Statement

I'm a computer science Ph.D. candidate at Worcester Polytechnic Institute and a member of TheCakeLab. My research interest lies in ubiquitous computing, augmented reality, and computer vision. My recent research projects have a strong focus on improving photorealism in mobile AR by developing novel environment lighting sensing systems. The insights of my research are mainly derived from the investigation of mobile device dynamics, physical modeling of spatial transformations, and data-driven studies. In the past, I have designed machine learning models and real-time systems for mobile AR environment lighting estimation. I’m also interested in security and privacy issues associated with photorealism in AR. I’m fortunate to have conducted my research works in these areas under the guidance of Prof. Tian Guo alongside many wonderful collaborators, and I’m passionate to continue studying these problems in my Ph.D.

Designing visual sensing systems has been a long-standing research topic for ubiquitous computing, mobile computing and computer vision. In recent years, emerging AR applications raise new standards for environment sensing on mobile devices to satisfy the crave of increasing demand of photorealism and interactivity. Lighting estimation, a task that aims to estimate omnidirectional lighting from limited environment observations, is at the key position of pursuing photorealistic and visually coherent rendering in AR. Traditionally, high-quality environment lighting is acquired using physical devices like light probes or 360° cameras. Such physical approach has satisfied the needs of professional many users from filming industries and computer graphics researchers. However, the data acquisition process could be expensive and impractical for mobile AR applications.

On a different vein, learning-based lighting estimation models have been adopted by the majority of AR frameworks in favor of their abilities to estimate environment lighting from single or multiple camera images. However, in complex real-world scenarios, such models often fail to adapt, as estimating 360° environment lighting from a low FoV camera image is a highly unconstrained task. The deployment time difficulties challenges the traditional definition of lighting estimation, which aims to use image color and structure clues to estimate the environment lighting. My past research has focused on leveraging user, device and environment dynamics to opportunistically generate lighting without interrupting user's application interaction.

Check out my research project details here.

I'm generally interested in collaborating with ubiquitous computing and machine learning projects related to AR/VR. If you share the same interests, please shoot me an email!