This repository contains the datasets and some code for the paper Benchmarking Neural Network Robustness to Common Corruptions and Perturbations (ICLR 2019) by Dan Hendrycks and Thomas Dietterich.
Requires Python 3+ and PyTorch 0.3+.
ImageNet-C Robustness with a ResNet-50 Backbone
| Method | Reference | mCE |
|---|---|---|
| Stylized ImageNet Data Augmentation | Geirhos et al. (ICLR 2019) | 69.3% |
| ResNet-50 Baseline | 76.7% |
Other backbones can obtain better results. For example, a vanilla ResNeXt-101 has an mCE of 62.2%.
Submit a pull request if you beat the state-of-the-art on ImageNet-C.
Download Tiny ImageNet-C here. (Mirror.)
Download ImageNet-C here. (Mirror.)
Tiny ImageNet-C has 200 classes with images of size 64x64, while ImageNet-C has all 1000 classes where each image is the standard size. For even quicker experimentation, there is CIFAR-10-C, but improvements on CIFAR-10-C may be much less indicative of ImageNet-C improvements.
ImageNet-P sequences are MP4s not GIFs. The spatter perturbation sequence is a validation sequence.
Download Tiny ImageNet-P here. (Mirror.)
If you find this useful in your research, please consider citing:
@article{hendrycks2019robustness,
title={Benchmarking Neural Network Robustness to Common Corruptions and Perturbations},
author={Dan Hendrycks and Thomas Dietterich},
journal={Proceedings of the International Conference on Learning Representations},
year={2019}
}
Part of the code was contributed by Tom Brown.
