Analyzing synthetic datasets through the training and inference domain gap

Vickram Rajendran; Chuck Tang; Frits van Paasschen

doi:10.1117/12.2664569

13 June 2023 Analyzing synthetic datasets through the training and inference domain gap

Vickram Rajendran, Chuck Tang, Frits van Paasschen

Proceedings Volume 12529, Synthetic Data for Artificial Intelligence and Machine Learning: Tools, Techniques, and Applications; 125290R (2023) https://doi.org/10.1117/12.2664569
Event: SPIE Defense + Commercial Sensing, 2023, Orlando, Florida, United States

Abstract

Synthetic data has been shown to be incredibly useful for improving the performance of perception algorithms; however, it is still unclear how to identify the right techniques to generate and to train perception models with those synthetic datasets. In this work, we show how creating a digital twin of a real-world dataset allows us to have a more principled evaluation of the synthetic-to-real domain gaps for both training and inference, and use this information to test and evaluate synthetic datasets themselves, rather than the models trained on them. Furthermore, we show how this framework allows for a measure of the inference domain gap—a measure that tells us whether testing a perception model on synthetic data is representative of testing in the real world. We use these measures to generate a synthetic dataset from the nuScenes autonomous driving dataset, targeted to maximally improve performance on specific rare classes. We optimize the synthetic data generation parameters for this dataset in order to reduce the inference and training domain gaps. We show performance improvements of over 18% on the bicycle class. Our training results provide a way of measuring the training domain gap to analyze synthetic datasets.

Conference Presentation

Citation Download Citation

Vickram Rajendran, Chuck Tang, and Frits van Paasschen "Analyzing synthetic datasets through the training and inference domain gap", Proc. SPIE 12529, Synthetic Data for Artificial Intelligence and Machine Learning: Tools, Techniques, and Applications, 125290R (13 June 2023); https://doi.org/10.1117/12.2664569

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available