braindecode / braindecode Goto Github PK
View Code? Open in Web Editor NEWDeep learning software to decode EEG, ECG or MEG signals
Home Page: https://braindecode.org/
License: BSD 3-Clause "New" or "Revised" License
Deep learning software to decode EEG, ECG or MEG signals
Home Page: https://braindecode.org/
License: BSD 3-Clause "New" or "Revised" License
Skorch does not do that, see skorch-dev/skorch#419 , we should do it by default, much more intuitive.
Our training loop should behave like setting drop_last to True here in dataloader https://pytorch.org/docs/stable/data.html
For unclear reasons, CircleCI sometimes downloads the physionet data multiple times (https://circleci.com/gh/braindecode/braindecode/153 2 times, https://circleci.com/gh/braindecode/braindecode/108 3 times).
And it puts them in exactly the same locations, e.g.:
First download returns (among other names):
'/home/circleci/mne_data/MNE-eegbci-data/physiobank/database/eegmmidb/S001/S001R04.edf'
Second download returns (among other names):
'/home/circleci/mne_data/MNE-eegbci-data/physiobank/database/eegmmidb/S001/S001R04.edf'
Other times it seems to have it cached/not download it, https://circleci.com/gh/braindecode/braindecode/147
I don't understand the behavior and how to make it work properly. @agramfort
I've started looking at the performance of various ways of getting windows:
1- MNE: epochs.get_data(ind)[0]
with lazy loading (preload=False
)
2- MNE: epochs.get_data(ind)[0]
with eager loading (preload=True
)
3- MNE: direct access to the internal numpy array with epochs._data[index]
(requires eager loading)
4- HDF5: using h5py (lazy loading)
The script that I used to run the comparison is here:
https://github.com/hubertjb/braindecode/blob/profiling-mne-epochs/test/others/profiling_mne_epochs.py
Also, I ran the comparison on a single CPU using:
>>> taskset -c 0 python profiling_mne_epochs.py
Here's the resulting figure, where the x-axis is the number of time samples in the continuous recording:
For the moment, it looks like:
1- ._data[index]
is unsurprisingly the fastest, however it requires to load the entire data into memory.
2- hdf5 is very close, with around 0.5 ms per loop, which is great knowing it's able to only load one window at a time.
3- get_data(index)
is much slower, but this is expected as we know it creates a new mne.Epochs
object every time it's called. Also, the gap between preload=True
and preload=False
is about 1.5 ms, which might be OK. The main issue though seems to be the linear increase of execution time as the continuous data gets bigger and bigger.
Considering the benefits of using MNE for handling the EEG data inside the Dataset classes, I think it would be important to dive deeper into the inner workings of get_data()
to see whether simple changes could make this more efficient. I can do some actual profiling on that. What do you think @agramfort @robintibor @gemeinl ?
Note: I haven't included the extraction of labels in this test.
The resampy library is used to resample continuous signals, see https://github.com/braindecode/braindecode/blob/master/braindecode/mne_ext/signalproc.py#L34-L72. The dependency should be removed, and resampling should be implemented through transforms (https://github.com/braindecode/braindecode/blob/master/braindecode/datautil/transforms.py#L18-L49) using mne.
While rewriting and refactoring for new tutorials, I thought about one possible renaming: Transforms->Preprocessors.
The reason is that our current transforms are applied directy to the data, not on-the-fly as torchvision transforms. This renaming would also allow to distinguish to-be-added transforms that are applied on-the-fly like in torchvision. However, don't feel 100% sure about it. What do you think @agramfort @sliwy @gemeinl
Create new package(or not, see below) with name such as engine, trainer(s) or training for skorchbased training/experiment loop api, check bcic_iv_2a_cropped.py (bcic_iv_2a.py already checked) and replace then in other examples as well (also reply amir email)
create_network
in all of braindecode and remove)For name for new package, some ways of others:
Right now, one only used by EEGNet is in nn_init.py another one is inside eegresnet... should both be in nn_init? Or both be in models and private?
Some applications might benefit of preprocessing / scaling of targets, e.g. age regression. How could we include this in braindecode? Do we introduce another function next to transform_concat_ds
to apply transforms to targets or would you argue for a combination of both?
Some examples now don't have much formatting or explanation. Should be much nicer, easy to understand for new user...
I was trying to translate my codes to use braindecode
. Previously I had a function train
that took pytorch neural net and trained model. Now I wanted to replace this function with skorch classifier. I created an example that should use the same network, the same parameters and the same dataset and it gives different results (with my train
function it gives 1.0 train accuracy after around 10-15 epoch instead of not even similar results after 50 epochs). On my dataset it does not work at all, so that is why I got interested in the skorch performance.
I'm stuck and I don't have any idea what causes this behavior. If anyone has some time and would like to help me here is a link to the python file with the example, maybe fresh look can help https://github.com/sliwy/braindecode/blob/strange_example/examples/skorch_slow_learning.py
Clear functionality how/where to enter X/y dataset or mne epochs(?) datasets with tutorial documentation
Search your mail for WG: example code
and translate once new API is integrated
@agramfort you wanted to set something up, right? Similar to this https://mne.tools/dev/whats_new.html
Make https://braindecode.org/auto_examples/plot_bcic_iv_2a_moabb_trial.html and https://braindecode.org/auto_examples/plot_bcic_iv_2a_moabb_cropped.htm properly into two proper tutorials, like similar to https://tntlfreiburg.github.io/braindecode/notebooks/Cropped_Decoding.html
Use different, quite successful architecture!
1 Re-fine definition of a BaseDataset:
2 Have a BaseConcatDataset
3 Datasets should not know the windower
4 Windowers should be applied to datasets
5 Avoid variable name ambiguities
Ensure that all epochscoring computed metrics are computed on all sets, maybe in braindecodeClassifier constructor.
Create a more structure in the Readme for https://braindecode.org/.
Should allow beginners directly to understand how to get started using braindecode, "Quickstart"-like, similar to before https://tntlfreiburg.github.io/braindecode/ .
Hi! First of all, thank you for your interesting work and for publishing and documenting everything here! :-)
For my master's thesis, I'm trying to classify the attentive state of patients with disorders of consciousness. The major challenge is that there is no ground truth available for these patients. To tackle this, I have two approaches. First, I'm using a Keras implementation of your deep network in combination with learning mechanisms that are robust to label noise. After training, gaining insight into what the network learned is the second aspect I'd like to investigate. So the input-feature unit-output correlation maps seem to be a promising method. I followed your tutorial on Amplitude Perturbation Visualization, which worked using the learned model's prediction function.
But regarding the input-feature unit-output correlation maps (without the perturbation), I don't know how I can calculate them. I read through the classes in the visualization package, but I'm not sure where to start. My goal would be an evaluation similar to figures 6/7 in your paper.
Can you point me in the right direction?
Kind regards, Constantin
test_cropped_decoding.py
, test_trialwise_decoding.py
refactor to use new dataset classes
Idea was:
input_time_length -> n_in_times
n_preds_per_input -> n_out_times
For more consistency and easier understanding. Can be in many places inside braindecode.
Easy to understand interface
The MOABBDataset object has attributes and methods that should probably be moved to a base dataset class.
Fetch the BCIC IV 2a data for the realistic examples (examples/bcic_iv_2a_*
) without relying on downloaded files in a specific folder.
Run Large Experiments, save resources
mne_apply
and common_average_reference_cnt
should be removed and replaced with mne functions through the transforms API (https://github.com/braindecode/braindecode/blob/master/braindecode/datautil/transforms.py#L18)concatenate_raws_with_events
could potentially be replaced by mne.concatenate_raws
. However, this modifies the first raw in-place which is probably not intended. Furthermore, there might be a bug, where events of raws are not properly combined. To be tested. If there actually is a bug, report to mne.Put new braindecode on pypi
Make EEGClassifier
docstring display all the parameteres (including base class). Probably we should use a way similar to skorch which uses __doc__
attribute of base class and modifies it (see here https://github.com/skorch-dev/skorch/blob/560e72149914b2d99ce6477226dd91835db8c37f/skorch/classifier.py#L61).
The most recent implementation of transforms asks for an OrderedDict of callables or mne.Raw/Epochs methods that are called successively on the internal Raw/Epochs objects in a ConcatDataset (https://github.com/braindecode/braindecode/blob/master/braindecode/datautil/transforms.py#L18). I wanted to discuss possible improvements to this design:
.resample
and .filter,
most mne methods do not seem to act in-place. As for callables, they can be made to modify the objects in-place, but this would mean all the transforms would have to be mne object-aware. I don't have a specific solution in mind, but maybe we could discuss this here.__getitem__
. This is how torch-vision does it for instance.exponential_running_standardize
and exponential_running_demean
should be adapted to accept data
input as n_channels x n_times
.highpass_cnt
, low_pass_cnt
, bandpass_cnt
, and filter_is_stable
should be removed and usages be replaced with mne functions and follwoing transform API (https://github.com/braindecode/braindecode/blob/master/braindecode/datautil/transforms.py#L18).Currently, defining tmin
will change the actual size of the windows. This should not be the case.
Description:
When we have the first integrated version skorch-api+dataset, we can make a pass over the code to enforce some more consistency and see what style we prefer (beyond just pep8).
Let's collect open questions for now:
i
(=i_trial
) or with idx
=idx_trial
As mentioned in the last PR, we thought about moving to the annotations objects to get the events.
So far, the events are read from the stimulus channel for the BCIC IV 2a dataset. Other datasets from MOABB might not have a stimulus channel, but come with an annotation object right away.
I have implemented a method to set the annotations object for the MOABB datasets, but Lukas and I came up with a few questions.
The event description in an annotations object is the description string, e.g. 'left_feet', instead of the integer values. For the event windower, we would need to call events_from_annotations(raw)
to get the indices of the events. Here, a new mapping is needed/generated as the events now become integers again. The default option here would result in events starting from 1 on. For classification we would need events from 0 on. Anyway it would be handy to have the resulting mapping saved in the dataset.
The main questions here are:
events_desc
attribute containing the mapping for the dataset?Beside that, the onset of the annotations would start at the actual experiment onset, e.g. shifted by 2s for bcic, and the duration is contained as well. These information can be accessed via MOABB.
Ensure we meet people's needs/cover use cases
Put very good defaults so calling without anything will usually give you a good result.
Put only link to new braindecode in readme, move old readme, remove parts about pip install. make sure github pages as well as github readme is updated.
Need unit tests and possibly acceptance tests for the dataset classes and functions.
See old discussion under
Originally posted by @robintibor in https://github.com/braindecode/braindecode/pull/83/files
(I'm new to machine-learning, so forgive me if this bug-report is misguided.)
I downloaded the "plot_bcic_iv_2a_moabb_trial.ipynb" example from here.
I noticed this code:
seed = 20200220 # random seed to make results reproducible
# Set random seed to be able to reproduce results
set_random_seeds(seed=seed, cuda=cuda)
Shouldn't this code make it so that every time I run the notebook, the results are the same in the learning-progress logging, and in the displayed plot?
However, it gives different results each time it runs, despite no code modifications: Screen Capture
Questions:
Allow many people to use it
Rename Deep4Net -> Deep4Model, ShallowFBCSPNet -> ShallowFBCSPModel
And/But: EEGNet ->EEGNetModel
Easier to understand interface, less code
Using the scikit-learn API, BrainDecode could be used as a regular scikit-learn pipeline and benchmarked against others classical BCI pipelines.
Right now, skorch seems to take the predictions obtained during the training loop iteration to compute metrics for the training set (EpochScoring with on_train=True).
This is not what we want.
Is there a reason for the force_update=True
introduced in 01aa0ce ?
These lines
braindecode/.circleci/config.yml
Lines 93 to 97 in 01aa0ce
seem to lead to quite long downloads in case there is a example different from master branch as far as I understand.
Wouldn't update_path=False, force_update=False work? This is what we do inside code usually (force_update=False per default from mne), e.g.
braindecode/examples/plot_skorch_crop_decoding.py
Lines 43 to 45 in 01aa0ce
Or will this break CircleCI completely again? Annoying to wait for long times in new pull requests always redownloading same stuff, even multiple times :/ @agramfort
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.