Code for the experiments in Regularized linear convolutional networks inherit frequency sensitivity from image statistics.

Core functionality computing model gradients with respect to the Fourier basis as well as various statistics thereof is in freq_sens.py. Code for training the models used in our experiments is in zoo.py and its imports. The remaining scripts are either dependencies of the previous two or used for analysis (e.g. the aptly named generate_all_plots.py).

We gratefully acknowledge dependence on two submodules, namely learning_with_noise and pytorch-vgg-cifar10.

A conda environment is specified in freq_sens.yaml. In theory it can be created using the following command.

conda env create --file=freq_sens.yaml 

Tables of training hyperparameters and model accuracies can be found in ./model_details.

This repository was extracted from a larger research codebase to which Eleanor Byler and Elise Bishoff made many contributions. In particular, Eleanor Byler wrote the first version of training.py and both Charles Godfrey and Elise Bishoff made further modifications, and datasets.py was a collaborative effort of Charles Godfrey and Eleanor Byler. The procedural generation (using the wavelet marginal model) and unsupervised training of AlexNets using learning_with_noise was implemented by Davis Brown. The remainder of the code was written by Charles Godfrey, although it should be noted that all authors listed in the citation below contributed substantially in the form of experiment ideas, feedback, suggestions and debugging advice.

If you find this repository useful, please cite:

@article{frequency_sensitivity,
  doi = {10.48550/ARXIV.2210.01257},
  url = {https://arxiv.org/abs/2210.01257},
  author = {Godfrey, Charles and Bishoff, Elise and Mckay, Myles and Brown, Davis and Jorgenson, Grayson and Kvinge, Henry and Byler, Eleanor},
  keywords = {Machine Learning (cs.LG), Machine Learning (stat.ML), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {Regularized linear convolutional networks inherit frequency sensitivity from image statistics},
  publisher = {arXiv},
  year = {2022},
  copyright = {arXiv.org perpetual, non-exclusive license}
}