trevillie / magnet Goto Github PK
View Code? Open in Web Editor NEWMagNet: a Two-Pronged Defense against Adversarial Examples
License: BSD 2-Clause "Simplified" License
MagNet: a Two-Pronged Defense against Adversarial Examples
License: BSD 2-Clause "Simplified" License
Hi! I have implemented your results of defense in MNIST dataset, but the results is not as good as yours.
I have trained the defense model and the classify model, and download the data which you upload to the Baiduyun , and test the defense performance . My classify model accuracy is 99%+, and the mean_squared_error of two autoencoder model is less than 0.003. Finally,I got the following graph.
I think there may be a problem with the detector,but I dont known the reason!
Hi I've been trying to reproduce your CIFAR results for a couple of weeks now, but after following the architecture in your paper, I've only gotten ~40% accuracy with detector and reformer. Would it be possible to upload your implementation of the CIFAR MagNet architecture? Thank you
In the original paper section 5.2.1, the paper said
Detector II and detector I(see Table 3) used the
$L^2$ and$L^1$ norm to measure reconstruction error, respectively.
However, in the following implementation, you use L1 for detector II and L2 for detector I.
Line 14 in b115cf8
Line 15 in b115cf8
I am asking is this a just typo? Can you confirm which one should be the correct one.
It seems the code in the repo gives the better result.
Hi! I'm trying to reproduce your results of defense performance with different confidence of Carlini’s L2 attack on MNIST & CIFAR10 (Figure 4&5). But I met some issues.
For MNIST:
I generated about 10000 adversarial samples for each confidence(0.0, 10.0, 20.0, 30.0, 40.0). For total about 50000 samples. I got the following graph.
My no_defense curve has a downward trend with a high accuracy at confidence of 0.0. But your no_defense accuracy keeps 0% at any confidence. Since higher confidence yields higher attack success rate, I wonder why you got such a curve.
For CIFAR10:
I generated about 10000 adversarial samples for each confidence(0.0, 20.0, 40.0, 60.0, 80.0, 100.0). For total about 60000 samples. I got the following graph.
Besides the no_defense issue in MNIST, my with_detector curve is also inconsistent with yours. Your curve in the paper shows a upward trend. And my test data shows that nearly 99% of the adversarial samples pass the detector. The detector seems not to work.
I've seen your talk in ISSUE 1. So I also used your new autoencoder architecture and a better classifier with an accuracy of 86%. I got the following graph.
The trends of curves seem not to change. Only a higher beginning accuracy was acquired.
To sum up, there are 2 issues:
Hi,I know the demo is for C&W adversarial attack,now I need to test other adversarial samples,Could you tell me how to make it in a easy way.Thank you!
Hi, we didn't try our old model but we can get a even better result with the following structure:
input -> 3x3x32 conv -> BN -> Relu -> 3x3x32 conv -> Relu ->BN -> 3x3x3 conv -> outputBTW, since we don't have our old classifier, we used a DenseNet classifier with acc ~92%.
Hope this helps.
Originally posted by @Trevillie in #2 (comment)
In your response to issue #2, you suggest using an even better structure: input -> 3x3x32 conv -> BN -> Relu -> 3x3x32 conv -> Relu ->BN -> 3x3x3 conv -> output.
But when I implement this structure for CIFAR10 Detector or Reformer, I found it happens to errors because it cannot make the output in the range of [0, 1].
Is that structure really work in your experiments? how do you do to make it work in your experiments?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.