Deep learning with limited data: A synthetic approach

This report focuses on how synthetic data, created using simulation or generative models, can be used to address the deep learning data challenge. These techniques offer many advantages: 1) data can be created for rare cases that are difficult to observe in the real world; 2) data can be automatically labeled without errors; and 3) data can be created with little or no infringement on privacy and integrity. Synthetic data can be integrated into the deep learning process using techniques such as data augmentation or by mixing synthetic data with real-world data prior to training. This report, however, focuses mainly on the use of transfer learning techniques where knowledge gained while solving one problem is transferred to more efficiently solve another related problem. Besides introducing synthetic data generation and transfer learning techniques, this report presents experimental results that provide valuable insights into the potential of the synthetic data approach in the context of pilot behavior cloning, vehicle detection and face verification tasks. Preliminary results from the experiments show that military simulators and generative models can be used to support deep learning applications. However, the performance is often limited by the fidelity gap between synthetic and real-world data.