Security evaluation module with onnx, pytorch, and SecML.
License: MIT License
I am Maura Pintor from 🇮🇹!
For more info, visit my webpage @ https://maurapintor.github.io/
You can find me on...
I am one of the organizers of the ML Security Seminar Series on YouTube. For more information about the series, follow this link.
Standard image preprocessing (mean/std standardizer) should be made available to the users.
The adversarial_examples API is already defined in the routes module, however it hasn't been tested with the latest changes of the library.
Attack choices not shown.
On the GUI, if the perturbation type is picked, the selector for the attack should visualize the attack choices for the specified perturbation model.
The attack choices are not updated.
Possible conflict with the jquery call that updates the perturbation values.
The sec-eval curve is now presenting results in a "linspace" way. The possibility of adding scatter values should be added, so that the list of eps values can be dynamically adjusted to arbitrary ranges.
The user should have the possibility to check the accuracy of an ML model at specified values of perturbation.
Docker compose file format is incompatible with old versions.
The command:
docker build . -t pandavision && docker-compose build && docker-compose upshould build the container and run smoothly.
The command produces, with some Docker-compose versions, the following output:
Successfully tagged pandavision:latest
ERROR: The Compose file './docker-compose.yml' is invalid because:
Unsupported config option for services: 'worker'
The problem seems related to the docker-compose versions that have incompatible specifications for the expected yaml: https://docs.docker.com/compose/compose-file/compose-versioning/#versioning
A suggested solution, from this StackOverflow question, is to upgrade the docker-compose version, and specify the version number in the top of the yaml file.
version: "3" in the header.GPU can be currently used by running the server and worker locally. Using a container that also works with GPU might be beneficial for speedups and ease installation.
Models from RobustBench should be available through the interface.
The choice should be available next to the upload model button, where a dropdown menu should be displayed.
The worker container shoud include the ML requirements, while the server container should include only web-server-related ones.
A configuration file should be used to set up paths and shared folders with docker or with the local app.
Evaluation fails with some particular configuration of parameters.
The reason seems to be related to cached adversarial examples.
The attack should not make the ram memory explode.
The ram memory fills, then the swap memory, then everything freezes.
Possibly free unused data, such as the attack paths.
The evaluation fails with the following set of parameters:
Right now, the only included attack is PGD-Linf. It would be great to have other choices, with configurable hyperparameters.
[x] feature request
The interface should allow for selecting subsamples of commonly-used datasets without uploading them to the server.
At least a sample from the following datasets should be included:
A button for downloading results as csv file should be added to the GUI, with implemented functionalities.
A multi-attack interface should be used. The interface should allow to specify a sequence of attacks that is used for testing the robustness of a model. The sequence will run the first attack on the whole dataset, then run the next attack in the sequence only on the points that fail for the given perturbation model.
Add visual interface for testing APIs. It should display at least the model and data selection, plus the results of the security evaluation when completed.
More use cases could be supported, as in https://gitlab.com/aloha.eu/security_evaluation.
Possible use cases are:
The security evaluation plot should be downloadable as pdf or png format.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.