Rethinking Generalization

This repo replicates a subset of experiments performed in Zhang et al.'s "Understanding Deep Learning Requires Rethinking Generalization" (2017). I only replicate experiments conducted on CIFAR10, and use AlexNet and two MLP variants (1x512, and 3x512). As a small extension, I also track dead neuron prevalence in the MLPs. This is mostly an exercise in writing quick and clean PyTorch for me! Run experiments via python runner.py (e.g., runner.py --lr 0.001 --model-name mlp-3x512 --use-adam --random-labels). See python runner.py -h for params.

Results

Train Accuracy	Test Accuracy

Citation

@article{Zhang2017UnderstandingDL,
  title={Understanding deep learning requires rethinking generalization},
  author={Chiyuan Zhang and Samy Bengio and Moritz Hardt and Benjamin Recht and Oriol Vinyals},
  journal={ArXiv},
  year={2017},
  volume={abs/1611.03530}
}