Awesome material(papers, tools, etc.) about testing machine learning system, including deep learning system.
This repo will be updated continuously, don't hesitate to add new Pull Request or Issues if you find anything is missing! Please use the format here.
Web page for paper list in ONLINE Thanks to @Troublor.
This repo use a specific format. When you open a new issues, you will find the template.
For tools:
- Project Name: <URL_to_project>
- A short description (please add the related paper if this tool is from academic paper)"
For paper:
- Paper Name: <URL_to_paper>
- Author(s) Name(s)
- Conference/Journal name
- A short description
-
EvalDNN: https://github.com/yqtianust/EvalDNN
- EvalDNN: A Toolbox for Evaluating Deep Neural Network Models.
- Benchmark: https://yqtianust.github.io/EvalDNN-benchmark/index.html
-
MuDNN: https://github.com/microsoft/MMdnn
- MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.
-
Netron: https://github.com/lutzroeder/netron
- a viewer for neural network, deep learning and machine learning models.
-
AIF360: https://github.com/IBM/AIF360, http://aif360.mybluemix.net/
- A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.
-
sotabench: https://sotabench.com
- A free benchmarking service for all open source ML repositories. You can submit your model and this web will run it against some benchmakrs.
-
Paper with Code: https://paperswithcode.com/sota
- A website contains evalution results reported in DL/ML papers, as well as their code (if any). Note the code could be implemented by 3rd party.
- APRICOT: A Dataset of Physical Adversarial Attacks on Object Detection: https://arxiv.org/abs/1912.08166
See Paper List. Thanks to @Troublor
- CS 590 Program Analysis For Deep Learning at Purdue University by Xiangyu Zhang: http://xyz-wiki.cs.purdue.edu/cs590-19f/doku.php?id=home
- Google Colab
- Paperspace gradient: https://gradient.paperspace.com/