Model-based robust deep learning algorithms allow for more accurate and robust predictions in varying environmental conditions.

In applications that rely on safe and reliable predictions such as autonomous vehicles, medical imaging, and robotics, it is essential that the prediction systems be made robust against nuisance variability such as changes in lighting or weather conditions. It is well known that various deep learning algorithms are fragile to seemingly innocuous changes to the environment such as background, lighting, blurring, or contrast. Currently there are no general-purpose algorithms that provide robustness against these variations in data.

The novel model-based training paradigm comprises three training algorithms which provide robustness against many different forms of nuisance-based variation. The three training algorithms, which are broadly applicable to deep learning, produce training data that leads to more reliable predictive systems.

A model of natural variation—a function which captures the variation in data—is first obtained either by learning the variation directly from data or by exploiting the physical laws governing the problem.  Depending on the setting, such models can capture nuisances in the data such as lighting, background color, haze, snow, or contrast.  Once a model has been obtained, it is used in conjunction with a robust optimization problem to train deep neural networks which are robust to the source of variation captured by the model.  The main insight is that the optimization problem seeks worst-case nuisances captured by the model, and then trains the neural networks to classify correctly on these worst-case data points.  In this work, three algorithmic variants are proposed which collectively facilitate a trade-off between robustness and efficiency.  This pipeline is, in principle, more flexible than standard classification, as it can integrate human priors regarding relevant nuisances by selecting or composing models of natural variation.

• Provides safety to systems that rely on accurate prediction such as autonomous vehicles
• Reusable in any number of different machine-learning systems, allowing for a wide range of applications
• 30 percentage point improvement over state-of-the-art image classifiers when tested on subsets of ImageNet-c
• Can be used in non-computer vision applications, providing more versatility than current image classifiers