Neal Wadhwa


Email / Google Scholar / LinkedIn

About Me

I work in Google Research in Mountain View, CA on Computational Photography problems. My work at Google is the basis for Portrait Mode on the Google Pixel 2 and Pixel 3. Previously, I completed a Ph. D under the supervision of Bill Freeman at the Computer Science and Artificial Intelligence Laboratory (CSAIL) at MIT. I worked on ways to visualize tiny changes in images and videos by amplifying them.


News


June 2018 New paper at SIGGRAPH 2018 about the Google Pixel 2's Portrait Mode.
October 2017 New Paper in Proceedings of the National Academy of Sciences: Motion microscopy for visualizing and quantifying small motions.
October 2016 I am starting a job at Google Research in Marc Levoy's group.
January 2016 I graduated and submitted my Ph. D dissertation: Revealing and Analyzing Imperceptible Deviations in Images and Videos.
December 2015 I gave an invited talk at the Computer Vision for Affordable and Accessible Healthcare Workshop at the 2015 International Conference in Computer Vision (ICCV).
October 2015 I have a new paper at SIGGRAPH Asia titled Deviation Magnification: Revealing Departures from Ideal Geometries.
May 2015 I gave an invited talk at the National Board for Medical Examiners in Philadelphia, Pennsylvania.
June 2014 Awarded Best Demo at CVPR 2014 for Real-time Video Magnification.

Software

Phase Based Video Motion Processing code
Matlab code for implementing the new and improved phase-based motion magnification pipeline.

Motion Microscopy for Visualizing and Quantifying Small Motions code
Matlab code to quantify small motions in videos.

Mobile Photography Publications

Synthetic Depth-of-Field with a Single-Camera Mobile Phone

Neal Wadhwa, Rahul Garg, David E. Jacobs, Bryan E. Feldman, Nori Kanazawa, Robert Carroll, Yair Movshovitz-Attias, Jonathan T. Barron, Yael Pritch, Marc Levoy

ACM Transactions on Graphics, Volume 37, Number 4 (Proc. SIGGRAPH), 2018.
[Paper] [Blog Post]
Depth from dual-pixels and semantic segmentation can be used to produce shallow depth-of-field images. Basis for Portrait Mode on Google Pixel 2.
Aperture Supervision for Monocular Depth Estimation

Pratul P. Srinivasan, Rahul Garg, Neal Wadhwa, Ren Ng, Jonathan T. Barron

Proc. of Computer Vision and Pattern Recognition (CVPR), 2018.
[Paper]
Varying a camera's aperture provides a supervisory signal that can teach a neural network to do monocular depth estimation.
Bilateral Guided Upsampling

Jiawen Chen, Andrew Adams, Neal Wadhwa, Samuel W. Hasinoff

ACM Transactions on Graphics, Volume 35, Number 6 (Proc. SIGGRAPH Asia), 2016.
[Paper] [Code] [Video]
Black-box image processing operators can be accelerated by fitting a fast bilateral-space model.

Motion Magnification and Analysis Publications

Motion Microscopy for Visualizing and Quantifying Small Motions

Neal Wadhwa, Justin G. Chen, Jonathan B. Sellon, Donglai Wei, Michael Rubinstein, Roozbeh Ghaffari, Dennis M. Freeman, Oral Buyukozturk, Pai Wang, Sijie Sun, Sung Hoon Kang, Katia Bertoldi, Frédo Durand, and William T. Freeman

Proceedings of the National Academy of Sciences, October 2017.
[Paper] [Webpage] [Code]
Motion magnification and analysis reveal phenomena in three different scientific domains.
Eulerian Video Magnification and Analysis

Neal Wadhwa, Hao-Yu Wu, Abe Davis, Michael Rubinstein, Eugene Shih, Gautham J. Mysore, Justin G. Chen, Oral Buyukozturk, John V. Guttag, William T. Freeman, Frédo Durand

Communications of the ACM, Volume 60, Number 1, January 2017.
[Paper]
An expository paper summarizing the various motion magnification and subtle motion analysis techniques we have developed.
Modal Identification of Simple Structures with High-Speed Video using Motion Magnification

Justin G. Chen, Neal Wadhwa, Young-Jin Cha, Frédo Durand, William T. Freeman, Oral Buyukozturk

Journal of Sound and Vibration Volume 345, 2015.
[Paper]
Motion magnification can be used to identify the modal shapes of simple structures such as pipes and beams.
Deviation Magnification: Revealing Departures from Ideal Geometries

Neal Wadhwa, Tali Dekel, Donglai Wei, Frédo Durand, William T. Freeman

ACM Transactions on Graphics, Volume 34, Number 6 (Proc. SIGGRAPH Asia), 2015.
[Paper] [Webpage]
A method to reveal subtle deviations of objects from ideal geometries, like lines or cirlces, in a single image by magnifying them.
Refraction Wiggles for Measuring Fluid Depth and Velocity from Video

Tianfan Xue, Michael Rubinstein, Neal Wadhwa, Anat Levin, Frédo Durand, William T. Freeman

Proc. of European Conference on Computer Vision (ECCV), 2014.
[Paper] [Webpage]
A method to recover the velocity and depth of hot air from video.
The Visual Microphone: Passive Recovery of Sound from Video

Abe Davis, Michael Rubinstein, Neal Wadhwa, Gautham Mysore, Frédo Durand, William T. Freeman

ACM Transactions on Graphics, Volume 33, Number 4 (Proc. SIGGRAPH), 2014.
[Paper] [Webpage]
A technique to recover sound from videos of objects subtly vibrating in response to sound.
Structural Modal Identification through High Speed Camera Video: Motion Magnification

Justin G. Chen, Neal Wadhwa, Young-Jin Cha, Frédo Durand, William T. Freeman, Oral Buyukozturk

Proceedings of the 32nd International Modal Analysis Conference (2014)
[Paper]
A validation that the motion magnified motions are indeed real and a way to compute the mode shapes of a cantilevered beam from video.
Riesz Pyramids for Fast Phase-Based Video Magnification

Neal Wadhwa, Michael Rubinstein, Frédo Durand, William T. Freeman

Computational Photography (ICCP), 2014 IEEE International Conference on
[Paper] [Webpage] [Pseudocode]
Provides the quality of the previous phase-based technique with the real-time speed of the original linear technique.
Phase-based Video Motion Processing

Neal Wadhwa, Michael Rubinstein, Frédo Durand, William T. Freeman

ACM Transactions on Graphics, Volume 32, Number 4 (Proc. SIGGRAPH), 2013.
[Paper] [Webpage] [BibTeX]
A new technique to amplify small motions that solves the noise amplification and intensity clipping artifacts of the previous linear method by manipulating the phase in sub-bands of videos.
Revealing Invisible Changes In The World

Michael Rubinstein, Neal Wadhwa, Frédo Durand, William T. Freeman

Science Vol. 339 No. 6119, Feb 1 2013
[Article in Science] [Video] [NSF SciVis 2012] [BibTeX]
An expository video showcasing our results and explaining our technique.