ejnnr / cupbearer Goto Github PK
View Code? Open in Web Editor NEWA library for mechanistic anomaly detection
License: MIT License
A library for mechanistic anomaly detection
License: MIT License
An example of this issue as @ejnnr observed, is for the WaNet Backdoor. A trained model with this type of backdoor responds to a specific randomly generated warping field. Later you would typically like to evaluate a detector detector based on absence or presence of this warping field in question. As code currently looks this isn't possible.
I'd like to take this issue to consider some possible design choices.
This benchmark is about evaluating a Detector on how well anomalous behaviour can be detected in a Model for some input.
To make this possible there needs to be a known Task for which a dataset that is known to produce anomalous and normal behaviour in a model has to be known.
A Detector is then evaluated on a Task. Some Tasks need to initialized which in the case of the Backdoor task consists of:
When a Task has been initialized it should be possible to load this Task, at least in cases when initialization is costly (e.g. training a model). However, it is also of interest to load a modified Task. For example a different transform than the one used in training the model to see that these are not detected as anomalous.
Main script is
eval_detector --task TASK --detector DETECTOR
so far this is the same set-up as before. Before this step there will be a
train_detector ...
init_task ... # e.g. train_classifier
I don't see any need to change anything with the detector set-up.
The changes will have to be with how the task is handled both in initialization and the command line arguments to eval detector
.
Add a --task from_run
similar as for the detector. This should preferably take as much as possible from the initialization of the task and then any additional settings.
Most of the information already exist in the config.yml and model/ under dir
. All that is missing is either a:
For saving the backdoor I would probably suggest to pass the path to where to save the backdoor in a call to train_loader.transforms.save_state(cfg.dir.path)
in train_classifier.py
and add stuff in Transform and its children where needed to save what states are necessary for reproduction.
Add an option --task.my_task.run_path
, task.my_task.[...].run_path
or similar only when needed. In that case I would strongly prefer to have a reasonable default that uses an equivalent instance of a Task as was used when the Task was initialized.
I can't think of a good way to set the default in e.g. WanetBackdoor such that this is the same path as was specified in train_classifier.py
if this was called in that function. And then this something equivalent should also be the done when loading in eval_detector.py
.
I don't think any of these alternatives is obviously good. The easiest one and the one I would probably pick in the short term is probably to just add something like
class WanetBackdoor:
...
init_seed: int = np.random.randint(2**64 - 1)
def __post_init__(self):
...
tmp_rng = np.random.default_rng(seed=self.seed)
self.control_grid = 2 * tmp_rng.random((control_grid_shape)) - 1
I'm not sure how this is/should be picked in eval_detector.py
but the information should exist in config.yml
so shouldn't be too hard to retrieve.
test_wanet
in test_pipeline.py
currently fails if we set use multiple workers: the cfg.train_data.backdoor
instance doesn't have its warping field initialized. My assumption is that this is because each worker ends up using its own copy of that, and the warping field is only initialized after making those copies (on the first __call__
). More importantly than failing this test case, I think that means the warping fields won't be shared across workers.
I've added a check in train_classifier.py
that raises an error in this setting, so not super high priority. But we should fix this, probably by initializing the warping field earlier.
Scripts are supposed to print detailed usage when given the --help
flag, but they typically just error out with a very short usage line complaining about missing arguments. I don't know if this is an issue with simple_parsing
or my code, need to investigate. Not very high priority, but I've been annoyed by it a few times already, and it's going to be even more annoying for new users I'd imagine.
docs/
Currently, all tasks are image classification, it would be good to also support language input and using pretrained/finetuned language models. Rough guess at the todos for that:
Model
or change the entire system of how we extract activations from networks. Probably easiest to get this to work for one specific language model first, we'll probably have to add some custom code for every new model anyway.Model
, or maybe we'd have to coordinate data processing and the model somehow.Model
implementation.Running pytest
gives quite a few deprecation and other warnings (and some of the scripts also give a a few). None of them are actually concerning, but would be nice to either fix or just suppress them where appropriate.
We currently have 3 detectors. In this issue I will investigate some possible new additions.
Top candidates:
This issue is for tracking various features that could make running large-scale experiments more convenient. Unclear which ones are actually worth implementing, will probably add them over time whenever one seems like it would actually be helpful right now.
cupbearer ...
instead of python -m cupbearer.scripts...
Keeping track of things we probably want to fix at some point but that aren't priorities. I think we should fix these whenever we'd otherwise have to make the problem worse or write a bunch of code that a refactor would change.
Debug
config classes are pretty cumbersome. I'm leaning towards just getting rid of them entirely and letting automated tests manually configure things for fast runs.Config
in their name? It could be nice in order to avoid name clashes with e.g. the detector class itself, but from a user perspective, the Config classes are actually the more important ones so should maybe get a short namesimple_parsing
(by using something else to serialize/deserialize configs). We could also reconsider the whole hierarchical dataclass config systems (i.e. just instantiate detectors etc. directly, pass them to a script, and make detectors responsible for serializing their config)TaskConfigBase
. It's not used anywhere and we now actually have some code that relies on the specifics of TaskConfig
. So I think that should be the only ABC for tasks, and we should just generalize it if actually necessary in the future.A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.