xindongol / benn-pytorch Goto Github PK
View Code? Open in Web Editor NEWCodes for Binary Ensemble Neural Network: More Bits per Network or More Networks per Bit?
Codes for Binary Ensemble Neural Network: More Bits per Network or More Networks per Bit?
Hi, thank you for your impressing work. I have some questions about this code.
What is the role of the object 'bin_op'? It seems that it can not change the parameters 'weight' of the network 'model'? In the training process, it seems that all the parameters in network are not binary. All the calculated binary parameters are stored in 'bin_op.target_modules', but they are not used in the training process of the network 'model'. It looks like the training process only uses binary input but real weight. Do I misunderstand something? Thanks for your answer.
Hi authors,
I am trying to train the CIFAR10 dataset using boostA-sequential-AB BNN model from scratch:
Command used
python main_boostA_AB_seq.py --epochs 10 --data ../../SB/boosting/data/
Error message
Traceback (most recent call last):
File "main_boostA_AB_seq.py", line 353, in <module>
train(epoch)
File "main_boostA_AB_seq.py", line 106, in train
output = model(data)
File "/home/wadhwae/miniconda3/envs/benn/lib/python3.7/site-packages/torch/nn/modules
/module.py", line 541, in __call__
result = self.forward(*input, **kwargs)
File "/home/wadhwae/miniconda3/envs/benn/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward
return self.module(*inputs[0], **kwargs[0])
File "/home/wadhwae/miniconda3/envs/benn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
result = self.forward(*input, **kwargs)
File "/home/wadhwae/workspace/Training/BENN-PyTorch/cifar/AB/boosting/../../models/nin.py", line
225, in forward
x = self.xnor(x)
File "/home/wadhwae/miniconda3/envs/benn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
result = self.forward(*input, **kwargs)
File "/home/wadhwae/miniconda3/envs/benn/lib/python3.7/site-packages/torch/nn/modules
/container.py", line 92, in forward
input = module(input)
File "/home/wadhwae/miniconda3/envs/benn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
result = self.forward(*input, **kwargs)
File "/home/wadhwae/workspace/Training/BENN-PyTorch/cifar/AB/boosting/../../models/nin.py", line 44, in forward
x = self.bn(x)
File "/home/wadhwae/miniconda3/envs/benn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
result = self.forward(*input, **kwargs)
File "/home/wadhwae/miniconda3/envs/benn/lib/python3.7/site-packages/torch/nn/modules/batchnorm.py", line 59, in forward
self._check_input_dim(input)
File "/home/wadhwae/miniconda3/envs/benn/lib/python3.7/site-packages/torch/nn/modules/batchnorm.py", line 244, in _check_input_dim
.format(input.dim()))
ValueError: expected 4D input (got 2D input)
Even if I retrain the model using python main_boostA_AB_seq.py --epochs 0 --retrain_epochs 0 --data ../../SB/boosting/data/
, (with all the 31 epochs tar files in models_boostA_AB_seq/ ) I am still getting the above error.
Error Message
==> Load pretrained model from ...
boosting 0
2020-01-04 15:51:51.598063 Start boosting iter: 0
===> Start retraining ...
Traceback (most recent call last):
File "main_boostA_AB_seq.py", line 368, in <module>
sample_weights_new, alpha_m = sample_models(boosting_iters=i, sample_weights=sample_weights_new, retrain_epoch=args.retrain_epochs)
File "main_boostA_AB_seq.py", line 294, in sample_models
batch_softmax_output = get_error_output(data, target, batch_sample_weights)
File "main_boostA_AB_seq.py", line 263, in get_error_output
output = model(data)
File "/home/wadhwae/miniconda3/envs/benn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
result = self.forward(*input, **kwargs)
File "/home/wadhwae/miniconda3/envs/benn/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward
return self.module(*inputs[0], **kwargs[0])
File "/home/wadhwae/miniconda3/envs/benn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
result = self.forward(*input, **kwargs)
File "/home/wadhwae/workspace/Training/BENN-PyTorch/cifar/AB/boosting/../../models/nin.py", line 225, in forward
x = self.xnor(x)
File "/home/wadhwae/miniconda3/envs/benn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
result = self.forward(*input, **kwargs)
File "/home/wadhwae/miniconda3/envs/benn/lib/python3.7/site-packages/torch/nn/modules/container.py", line 92, in forward
input = module(input)
File "/home/wadhwae/miniconda3/envs/benn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
result = self.forward(*input, **kwargs)
File "/home/wadhwae/workspace/Training/BENN-PyTorch/cifar/AB/boosting/../../models/nin.py", line 44, in forward
x = self.bn(x)
File "/home/wadhwae/miniconda3/envs/benn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
result = self.forward(*input, **kwargs)
File "/home/wadhwae/miniconda3/envs/benn/lib/python3.7/site-packages/torch/nn/modules/batchnorm.py", line 59, in forward
self._check_input_dim(input)
File "/home/wadhwae/miniconda3/envs/benn/lib/python3.7/site-packages/torch/nn/modules/batchnorm.py", line 244, in _check_input_dim
.format(input.dim()))
ValueError: expected 4D input (got 2D input)
I only can suspect here that I have wrongly saved / pre-processed the CIFAR10 dataset (I have saved the dataset as train_data, train_labels, test_data and test_labels in numpy files and read from it using the data.py file provided, no pre-processing done).
Hi
Thanks for the nice project.
But could you please help to provide the pretrained models to facilitate more future studies?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.