cv

Research Scientist Intern

FieldAI

On-site internship.

• Leading research to push frontiers of embodied AI agents and human-robot interaction through vision-language action models and spatial AI.

Graduate Research Student: Robotics

Carnegie Mellon University, Robotics Institute

Member of AirLab (UAS intelligent autonomy) and AART lab (multi-agent coordination).

• Wildfire Tracking: developed informative path planning algorithms (RRT, MCTS, and learning-based planners) for wildfire monitoring from UAS; deployed an NVIDIA Jetson Xavier NX and FLIR thermal camera on a DJI M600 drone tested over real prescribed wildland fires in Pennsylvania; led a team of 8 students; led development of the world's first high-fidelity wildfire simulator in AirSim simulating thermal signatures in a 3D fire environment; led writing of a grant awarded $1.2M over 3 years by the NSF National Robotics Initiative.
• UAS Reconnaissance: searched and tracked vessels at sea via UAS; developed a particle filter to track and update belief of targets; trained a YOLOv5 vision network on simulated ship data; used a Kalman Filter to estimate vessel odometry; developed an informed RRT-based algorithm.
• Multi-drone Subject-Tracking in the Wild: developed UAS algorithms to track and film dynamic outdoor subjects and perform 3D reconstruction of subject motion; implemented drone-to-drone communication via DDS and ROS (second-author paper published at IROS 2021).

Research Intern: Computational Photography

Sensebrain, Inc.

Applied deep learning to demosaicing in image signal processing pipelines.

• Trained neural networks in PyTorch and designed loss functions to reduce artifacts in image reconstruction.
• Optimized low-level neural networks with compression and acceleration languages such as Halide and OpenCL.
• Deployed networks on mobile hardware: DSP, NPU, GPU, and CPU.

Research Assistant: Generative Adversarial Networks for Virtual Try-on

San Jose State University, Department of Computer Science

Supervised by Dr. Teng-Sheng Moh.

• Independently developed and submitted a project proposal for virtual try-on.
• Reimplemented the leading virtual try-on paper, SwapNet, from scratch in PyTorch.
• Discovered that video input achieves comparable results with less training data; results published at HPCS 2019.

Research Engineer: Exercise Tracking with Computer Vision

FitLens, Inc.

Lead developer on a working prototype that tracks exercise and human form.

• Built a webcam pipeline that tracks user pose and calculates correctness relative to an exercise coach.
• Implemented rep counting and real-time feedback to improve exercise form.

Research Assistant: NLP Network Architectures

Stanford University, Center for the Study of Language and Information (CSLI)

Supervised by Dr. Christopher Potts and Ignacio Cases (PhD student).

• Evaluated novel NLP architectures to inherently model hierarchical compositionality in language.
• Used reinforcement learning to route compositionality of multi-task problems within the architecture.
• Designed, ran, and analyzed deep learning experiments in PyTorch.

Research Associate: Computer Vision for Biology

NASA Ames Research Center

Supervised by Dr. Joshua Alwood.

• Trained convolutional neural networks to analyze reproductive tissue health.
• Used Bayesian optimization for hyperparameter search and GANs for data augmentation.
• Achieved a 200x speedup over manual methods; presented at the national Society for the Study of Reproduction conference.

Software Engineering Intern: Web Development

NASA Ames Research Center

Full-stack web development.

• Designed, coded, and deployed an application to track lab diagnostics.
• Built a React UX/UI, a NodeJS server, and MySQL databases.

Research Assistant: Natural Language Processing

Stanford University, Center for the Study of Language and Information (CSLI)

Supervised by Dr. Christopher Potts and Will Monroe (former PhD student).

• Explored natural language models to understand the nuances of language pragmatics.
• Collected, cleaned, and annotated new corpora; ran experiments on Amazon Mechanical Turk.
• Published and peer-reviewed at NAACL 2018.

Founder and President

ML@SJSU Machine Learning Club

Recognized Student Organization at San Jose State University.

• Founded a machine learning community of 27 high-impact members and 200+ online participants.
• Led weekly PyTorch workshops and reading groups; created a committee on Generative Adversarial Networks.
• Hosted guest speakers from Stanford and Intel.

PhD

Carnegie Mellon University

Robotics

• Member of AirLab (UAS intelligent autonomy) and AART lab (multi-agent coordination)
• Coursework: Planning, SLAM, Deep Computer Vision, Machine Learning, Kinematics, Dynamics, and Control
• Degree expected May 2027

Master of Science

Carnegie Mellon University

Robotics

• Member of AirLab (UAS intelligent autonomy) and AART lab (multi-agent coordination)
• Coursework: Planning, SLAM, Deep Computer Vision, Machine Learning, Kinematics, Dynamics, and Control

Bachelor of Science

San Jose State University

Computer Science

• Artificial intelligence specialization; mathematics minor
• Graduated with Honors; Summa cum Laude

Outstanding Graduating Senior, Department of Computer Science

San Jose State University

Highest and most prestigious annual award of the CS department; nominated and selected by faculty; 1 recipient annually.

Department of Computer Science Scholarship ($1,000)

San Jose State University

Awarded in recognition of outstanding academic achievement; 5 recipients annually.

SmokeSeer: 3D Gaussian Splatting for Smoke Removal and Scene Reconstruction

International Conference on 3D Vision (3DV)

Jain, Neham, Andrew Jong, Sebastian A. Scherer, Ioannis Gkioulekas.

IA-TIGRIS: An Incremental and Adaptive Sampling-Based Planner for Online Informative Path Planning

IEEE Transactions on Robotics (T-RO)

Moon, Brady G., Nayana Suvarna, Andrew Jong, Satrajit Chatterjee, Junbin Yuan, Muqing Cao, Sebastian A. Scherer.

AnyThermal: Towards Learning Universal Representations for Thermal Perception

IEEE International Conference on Robotics and Automation (ICRA)

Maheshwari, Parv, Jay Karhade, Yogesh Chawla, Isaiah Adu, Florian Heisen, Andrew Porco, Andrew Jong, Yifei Liu, Santosh Pitla, Sebastian Scherer, Wenshan Wang.

Don't Fool Me Twice: Adapting to Adversity in the Wild with Experience-Driven Reasoning

arXiv preprint (cs.RO)

Ravie, Navin Sriram, Andrew Jong, Krrish Jain, John Liu, Omar Alama, Bijo Sebastian, Sebastian Scherer.

FIReStereo: Forest InfraRed Stereo Dataset for UAS Depth Perception in Visually Degraded Environments

IEEE Robotics and Automation Letters (RA-L)

Dhrafani, Devansh, Yifei Liu, Andrew Jong, Ukcheol Shin, Yao He, Tyler Harp, Yaoyu Hu, Jean Oh, Sebastian A. Scherer.

WIT-UAS: A Wildland-Fire Infrared Thermal Dataset to Detect Crew Assets From Aerial Views

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Jong, Andrew, Mukai Yu, Devansh Dhrafani, Siva Kailas, Brady Moon, Katia Sycara, Sebastian Scherer.

Robot Information Gathering for Wildland Fire Monitoring (Master's Thesis)

Carnegie Mellon University

Jong, Andrew. Master's Thesis Archives, Carnegie Mellon University.

3D Human Reconstruction in the Wild with Collaborative Aerial Cameras

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Ho, Cherie, Andrew Jong, Harry Freeman, Rohan Rao, Rogerio Bonatti, Sebastian Scherer.

ShineOn: Illuminating Design Choices for Practical Video-Based Virtual Clothing Try-On

IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops

Kuppa, Gaurav, Andrew Jong, Xin Liu, Ziwei Liu, Teng-Sheng Moh. Kuppa and Jong are co-first authors.

Virtual Try-On With Generative Adversarial Networks: A Taxonomical Survey

Advancements in Computer Vision Applications in Intelligent Systems and Multimedia Technologies (IGI Global)

Jong, Andrew, Melody Moh, Teng-Sheng Moh. Edited by Muhammad Sarfraz, pp. 76-100.

Short Video Datasets Show Potential for Outfits in Augmented Reality

International Conference on High Performance Computing & Simulation (HPCS)

Jong, Andrew, and Teng-Sheng Moh.

A High-Altitude Balloon Platform for Space Life Sciences Education

American Society for Gravitational and Space Research (ASGSR)

Mckaig, Jordan, et al. (incl. Andrew Jong).

Generating Bilingual Pragmatic Color References

North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)

Monroe, Will, Jennifer Hu, Andrew Jong, Christopher Potts.