James Tompkin
Associate Professor
Visual Computing
BlueSky @brownvc.bsky.socialGithub @brownvc
Brown student researcher?
Group Onboarding Process
Contact
BlueSky @jamestompkin.bsky.socialGoogle Scholar
Office hours: Weds 1300 EST
Book appointment
Brown folks: Save an email,
use GCal 'Find a Time'
and include an agenda.
Instructions
Center for Information Technology
Room 547
115 Waterman Street
Providence, RI, 02912
Acknowledgements
My intrepid collaborators and co-authors.
Funding:
- US NSF, DARPA, NASA
- UK EPSRC, BBC
- Industry Activision, Adobe, Amazon, Cognex, Google, Intel, Meta, Snap, AI Foundation
The open source Web community: HTML5 Boilerplate, Ryan Johnston, Joshua N. Hibbert, PracticalTypography.com, EB Garamond.
Hosted on GitHub Pages using Jekyll — basic theme by orderedlist.
James TompkinAssociate ProfessorVisual Computing BlueSky @brownvc.bsky.socialGithub @brownvc
Brown student researcher? |
Contact BlueSky @jamestompkin.bsky.socialGoogle Scholar
Office hours: Weds 1300 EST
Brown folks: Save an email,
Center for Information Technology
|
Acknowledgements
My intrepid collaborators and co-authors.
Funding:
- US NSF, DARPA, NASA
- UK EPSRC, BBC
- Industry Activision, Adobe, Amazon, Cognex, Google, Intel, Meta, Snap, AI Foundation
The open source Web community: HTML5 Boilerplate, Ryan Johnston, Joshua N. Hibbert, PracticalTypography.com, EB Garamond.
Hosted on GitHub Pages using Jekyll — basic theme by orderedlist.
I am a visual computing researcher—computer vision, computer graphics, and human-computer interaction. My lab develops techniques for image and video creation, editing, analysis, and interaction. This requires image and scene reconstruction techniques, especially from multi-camera systems and for complex dynamic scenes, and with applications on 2D, multi-view, and VR/AR displays.
Active Illumination for Dynamic 3D Reconstruction
Physical modelling of active illumination from raw sensor measurements can improve scene estimation and avoid errors from derived depth.
Time-of-flight and structured-light cameras are typically used as depth sensors: their raw measurements are processed into a per-pixel depth map, and downstream reconstruction methods treat that depth as input. But, depth processing often simplifies scene assumptions, creating noise in low-reflectance regions, flying pixels with multi-path interference, and motion artifacts in fast-moving scenes from requiring multiple illumination readings for depth estimates. Further, it is difficult to integrate these raw measurements with other sensor modalities, like colour cameras.
This line of work rethinks reconstruction for heterogeneous multi-shot imaging processes. Built upon a differentiable forward model of how the active illumination produces the raw sensor output for a given scene, these methods optimise a 4D volumetric scene representation (like NeRF or 3DGS) so that rendered measurements match what the sensor captured. This lets us principally integrate sensor measurements over spacetime, including across modalities, to reduce noise, resolve ambiguities in multi-shot sensing, and improve robustness to multi-path interference. And, as we model motion over time, then we can estimate and resample fast motion like swinging baseball bats to slow motion.
Authors
Benjamin Attal · Eliot Laidlaw · Aaron Gokaslan · Changil Kim · Christian Richardt · Matthew O'Toole · Mikhail Okunev · Marc Mapeke · Runfeng Li · Zixuan Guo · Anh Duong · Aarrushi Shandilya · Andreas Meuleman · Hakyeong Kim · Min H. Kim
Papers in this thread
Related papers
|
|
3D Vision,
2026
|
|
MDPI Remote Sensing,
2026
|
|
|
SIGGRAPH Asia,
2025
|
|
2025
|
|
|
Computer Vision and Pattern Recognition (CVPR),
2025
|
|
|
Computer Vision and Pattern Recognition (CVPR),
2025
|
|
|
SIGGRAPH Asia,
2024
|
|
Neural Information Processing Systems (NeurIPS),
2024
|
|
Transactions on Machine Learning Research,
2025
|
|
|
European Conference on Computer Vision (ECCV),
2024
|
|
Transactions on Visualization and Computer Graphics (IEEE Visualization short paper),
2024
|
|
Computer Vision and Pattern Recognition (CVPR),
2024
|
|
2024
Gives insight into why disentangling with CLIP is difficult—it's the prompt noise!
|
|
Winter Conference on Applications of Computer Vision (WACV) and AI for Content Creation (AI4CC) @ CVPR 2023,
2024
|
|
arXiv (Dec.~2023) + WACV,
2025
|
|
|
ACM Transactions on Graphics (SIGGRAPH Asia),
2023
|
|
|
International Conference on Computer Vision (ICCV),
2023
|
|
|
International Conference on Computer Vision (ICCV),
2023
|
|
International Journal of Computer Vision (IJCV),
2024
|
|
On Human-like Biases in CNNs for the Perception of Slant from Texture
ACM Transactions on Applied Perception,
2023
|
|
SIGCHI,
2023
|
|
Learning Vector Quantized Shape Codes for Amodal Blastomere Instance Segmentation
IEEE International Symposium on Biomedical Imaging (ISBI),
2023
|
|
|
ACM Transactions on Graphics (SIGGRAPH),
2022
|
|
Eurographics State of the Art Report + CVPR Tutorial + SIGGRAPH Course,
2022
|
|
|
European Conference on Computer Vision (ECCV),
2022
|
|
International Conference on Computational Photography (ICCP),
2022
|
|
SIGCHI,
2022
|
|
|
Computer Vision and Pattern Recognition (CVPR),
2022
|
|
Computers and Graphics,
2022
For recovering depth, this follows up Blind Video Spatio-Temporal Consistency and Blind Video Temporal Consistency.
|
|
Learning Physically-based Face Material and Lighting Decomposition
International Conference on Computational Visual Media,
2022
Also appeared at CVPR 2021 Workshop on AI for Content Creation
|
|
Transactions on Visualization and Computer Graphics,
2022
|
|
|
Advances in Neural Information Processing Systems (NeurIPS),
2021
|
|
International Conference on Computer Vision (ICCV),
2021
|
|
|
Computer Vision and Pattern Recognition (CVPR),
2021
|
|
BMVC 2021 and CVPR Workshop on AI for Content Creation,
2021
|
|
Human-Robot Interaction (Late Breaking Report),
2021
|
|
Transactions on Visualization and Computer Graphics (TVCG),
2021
|
|
European Conference on Computer Vision (ECCV),
2020
|
|
European Conference on Computer Vision (ECCV),
2020
|
|
BMVC,
2021
Fast 4D depth with accurate occlusion edges across two papers:
Edge-aware Bi-directional Diffusion for Dense Depth Estimation from Light Fields and View-consistent 4D Light Field Depth Estimation |
|
International Conference on 3D Vision (3DV),
2020
|
|
Real VR—Immersive Digital Reality,
2020
|
|
CVPR Workshop on AI for Content Creation,
2020
|
|
CVPR Workshop on Media Forensics,
2020
|
|
|
Transactions on Visualization and Computer Graphics (IEEE Visualization),
2020
|
|
MICCAI,
2020
|
|
International Conference on Computer Vision (ICCV),
2019
This work also produces an occlusion-aware piecewise planar scene reconstruction as a byproduct!
|
|
User Interface Software and Technology (UIST),
2019
|
|
VRCAI,
2019
|
|
|
SIGCHI,
2019
|
|
|
Transactions on Visualization and Computer Graphics (IEEE Visualization short paper),
2019
One-line SVG pan/zoom, plus a pan/zoom injecting bookmark for any SVG!
|
|
International Journal of Robotics Research,
2019
|
|
Neural Information Processing Systems (NeurIPS),
2018
|
|
European Conference on Computer Vision (ECCV),
2018
|
|
Transactions on Visualization and Computer Graphics (IEEE Visualization),
2018
|
|
Computer Vision and Pattern Recognition (CVPR),
2018
|
|
Computer Vision and Pattern Recognition (CVPR),
2018
|
|
ACM Symposium on Eye Tracking Research and Applications (ETRA),
2018
|
|
British Machine Vision Conference,
2017
|
|
Computer Graphics Forum (Eurographics),
2017
We could have called it Blind Video Spatio-Temporal Consistency as it follows up Blind Video Temporal Consistency.
|
|
International Conference on Computer Vision (ICCV),
2017
|
|
Conference on Human-Robot Interaction (HRI),
2017
|
|
|
International Symposium on Robotics Research,
2017
|
|
IEEE Visualization Workshop on Visual Analytics for Deep Learning,
2017
|
|
MDPI Informatics—Special Issue on Scalable Interactive Visualization,
2017
|
|
Transactions on Visualization and Computer Graphics (IEEE Visualization),
2016
|
|
Pacific Graphics 2016 (Short Paper),
2016
|
|
User Interface Software and Technology (UIST),
2015
Also at SIGGRAPH Emerging Technologies 2012: Interactive Light Field Painting
|
|
ACM Transactions on Graphics (SIGGRAPH Asia),
2015
Builds upon project: Direct Motion Mapping
|
|
ACM Transactions on Graphics (SIGGRAPH Asia),
2015
Related project: Blind Video Spatio-Temporal Consistency
|
|
ACM Transactions on Graphics (SIGGRAPH Asia),
2015
|
|
ACM Symposium on Computer Animation (SCA),
2015
|
|
Computer Vision and Pattern Recognition (CVPR),
2015
|
|
International Conference on Computer Vision (ICCV),
2015
|
|
Computer Vision and Pattern Recognition (CVPR),
2015
|
|
ACM Transactions on Graphics (SIGGRAPH Asia),
2014
|
|
Transactions on Pattern Analysis and Machine Intelligence (TPAMI),
2014
|
|
European Conference on Visual Media Production (CVMP),
2014
Related project: Vidicontexts
|
|
Computer Graphics Forum (Eurographics),
2014
Related project: Generalized Wave Gestures
|
|
ACM Transactions on Graphics (SIGGRAPH Asia),
2013
|
|
International Conference on Computer Vision (ICCV),
2013
|
|
User Interface Software and Technology (UIST),
2013
Related study into display device effect: Device Effect on Panoramic Video+Context Tasks
|
|
ACM Transactions on Applied Perception (TAP),
2013
|
|
ACM Transactions on Graphics (SIGGRAPH),
2013
Printing light field displays with varying spatio-angular resolution.
|
|
EngD Thesis @ University College London,
2013
|
|
European Conference on Visual Media Production (CVMP),
2012
Alt title: Light Field Video Textures
|
|
European Conference on Computer Vision (ECCV),
2012
|
|
European Conference on Computer Vision (ECCV),
2012
Useful for building correspondence graphs for image matching, e.g., in search or large-scale reconstruction.
|
|
ACM Transactions on Graphics (SIGGRAPH),
2012
|
|
SIGGRAPH Emerging Technologies,
2012
Early demo of our later UIST 2015 publication Joint 5D Pen Input for Light Field Displays.
|
|
Computer Graphics Forum (Eurographics),
2012
|
|
Computer Graphics Forum (Eurographics),
2012
|
|
ACM Transactions on Graphics (SIGGRAPH),
2011
|
|
European Conference on Visual Media Production (CVMP),
2011
|
|
International Brain-Computer Interface Conference (BCI),
2011
|
|
ACM Transactions on Computer-Human Interaction (SIGCHI),
2010
|
|
British HCI Group Annual Conference on People and Computers (BCS-HCI),
2009
Webpage contains many projects and events! Schematics and WebGL model viewer!
|
|
MSci Dissertation @ King's College, London,
2006
|
HuggingFaceWorkshops and Courses
AI for Content Creation
CVPR 2019–2025 Workshop
Physics-inspired 3D Vision and Imaging
CVPR 2025 Workshop
Neural Fields Beyond Conventional Cameras
ECCV 2024 Workshop
Neural Fields in Visual Computing
CVPR 2022 Tutorial + SIGGRAPH 2023 Course
New England Compter Vision Symposium
Brown 2019
Video for Virtual Reality
SIGGRAPH 2017 Course
User-centric Computational Videography
SIGGRAPH 2015 Course
University Courses
CSCI 1430—Introduction to Computer Vision
Brown University
2016–now.
CSCI 1290—Computational Photography
Brown University
2018–now.
CSCI 2951-I—Computer Vision for Graphics and Interaction
Brown University
2016–now.
CSCI 2000—Computer Science Research Methods or How to be a CS PhD Student
Brown University
2021 Fall.
CSCI 1950-N—2D Game Engines
Brown University
2017–now. Mentoring student-led course.
GISP 0002—NFTs, Blockchain, and Art, led by Ally Zhu and Nikolas Lazar
Brown University
2022 Spring.
CS171—Visualization
Harvard University
2016 Spring, 2015 Spring.
Computer Vision for Computer Graphics
Max-Planck-Institute for Informatics
2013 Summer.
Doctoral Students
|
2025–
|
|
Yiwen (Nick) Huang
2024–
Publications:
|
|
2022–
Publications:
|
|
2021–
Publications:
|
|
2021–2025
Onto: Luma AI Research Scientist
Publications:
|
|
2018–
Publications:
|
|
2016–2021
Onto: Meta Reality Labs Research Scientist
Publications:
|
Masters Students
|
2023–2025
Onto:
Publications:
|
|
2021–2024
Onto: Meta Reality Labs
Publications:
|
|
Yiwen (Nick) Huang
2021–2023
Onto: Brown PhD
Publications:
|
|
2020–2022
Onto: UMass Amherst PhD
Publications:
|
|
2019–2021
Onto: UToronto PhD
Publications:
|
|
2016–2020
Onto: US Congressional Innovation Scholar
Publications:
|
|
2018–2020
Onto: Stanford PhD
Publications:
|
|
2018–2020
Onto: Facebook AI on VR/AR
Publications:
|
|
2018–2020
Onto: Google
Thesis: "Fauxtoshop: Modeling Image Editing Operations with Kernel Prediction Networks and Parameter Blocks"
|
|
2019–2020
Onto: Allen Institute for AI Residency; MIT PhD
Publications:
|
|
2017–2019
Onto: CMU PhD
Publications:
|
|
2017–2019
Onto: Facebook AI Residency, Cornell PhD
Publications:
|
Undergraduate Students
|
Anika Bahl
2023–2024
Onto:
|
|
Troy Conklin
2022–2024
Onto: General Dynamics
|
|
2021–2023
Onto: CMU Research Masters | UW PhD
Publications:
|
|
2022–2023
Onto: Harvard Data Science Masters
Publications:
|
|
2021–2023
Onto: Harvard Computational Science and Engineering Masters
Publications:
|
|
2020–2022
Onto: Common Sense Machines
Publications:
|
|
2017–2020
Onto: UC Berkeley PhD
Publications:
|
|
2019–2021
Onto: Common Sense Machines
Publications:
|
|
Henry Stone
2018–2020
Publications:
|
|
Lucas Kasser
2018–2019
Publications:
|
|
2017–2018
Onto: Allen Institute for AI Residency; UWashington PhD
Publications:
|
Extended Family PhDs
|
2022–2025
Onto:
Publications:
|
|
2021–2025
Onto: Alibaba
Publications:
|
|
2018–2021
Onto: Synthesia
Publications:
|
|
2014–2020
Onto: Google
Publications:
|
|
2014–2019
Onto: UMass Boston Faculty
Publications:
|
|
2014–2017
Onto: RealityDefender
Publications:
|
Biography (2024)
James Tompkin is an Associate Professor of Computer Science at Brown University. His research at the intersection of computer vision, computer graphics, and human-computer interaction helps develop new visual computing tools and experiences from cameras. For this, his lab creates techniques for 3D scene reconstruction from multi-camera systems and for dynamics. His doctoral work at University College London on large-scale video processing and exploration techniques led to creative exhibition work in the Museum of the Moving Image in New York City. Postdoctoral work at Max-Planck-Institute for Informatics and Harvard University helped create new methods to edit content within images and videos. Recent research has developed new techniques for low-level reconstruction of dynamic scenes, view synthesis for VR, and AI content editing and generation.
Academic lineage
- Post-doc with Prof. Hanspeter Pfister at the Harvard Paulson School of Engineering and Applied Sciences.
- Post-doc with Prof. Christian Theobalt at the Max-Planck-Institute for Informatics and the Intel VCI.
- Research intern with Prof. Wojciech Matusik at Disney Research Cambridge.
- EngD VEIV student with Prof. Jan Kautz at University College London, sponsored by the BBC.
- MSci at King's College London with Ian Mackie.
Please find my research summary video from 2015—our newer lab work is on the 'Research' tab.
SIGGRAPH 50th—2023
I supported SIGGRAPH's 50th conference in 2023 as the chair of the Posters program, which was coincidentally running its 20th iteration too. Here's a meta-poster about the program's history and its outstanding contributors (low-res PNG).
Faculty and Tenure Application Materials
To share my experience, here is the material I sent to Brown to apply for a tenure-track assistant professor position in Dec. 2015.CV (Sept. 2016)
Research Statement
Teaching Statement
Here is the material I used for my tenure case at Brown in Dec. 2023.
CV
Research Statement
Teaching Statement
Exhibitions
I supported the Discover program and club at Brown/RISD to pair arts and science students and put on an exhibition (2017--2021). I have also tried to contribute myself.
Bad Art @ Brown, 2018
with Aaron Gokaslan and Vivek Ramanujan
Rear Window Augmented
with Jeff Desom
Museum of the Moving Image
New York City
7th Nov. 2015 to 10th April 2016
ISCP
New York City
7–9th November 2014
Festival Imaginales
Epinal, France
26–29th May 2014
Luxembourg Film Festival
28th February to 9th March 2014