1adrianb / binary-networks-pytorch Goto Github PK
View Code? Open in Web Editor NEWBinarize convolutional neural networks using pytorch :fire:
Home Page: https://www.adrianbulat.com
License: BSD 3-Clause "New" or "Revised" License
Binarize convolutional neural networks using pytorch :fire:
Home Page: https://www.adrianbulat.com
License: BSD 3-Clause "New" or "Revised" License
when i run cifar10.py there's an error
ModuleNotFoundError: No module named 'main.utils'; 'main' is not a package
even when i removed the . ; it also error with ModuleNotFoundError: No module named 'requests'
any suggestions? thank you
In the example code, you forgot to put {} around the "string:Bconfig()" pair.
Right now:
bmodel = prepare_binary_model(model, bconfig, custom_config_layers_name=['conv1' : BConfig()])
Should be:
bmodel = prepare_binary_model(model, bconfig, custom_config_layers_name=[{'conv1' : BConfig()}])
binary-networks-pytorch/bnn/models/layers/res_block.py l 34-35
self.act1 = activation(inpace=True) if activation == nn.ReLU else activation(num_parameters=planes)
self.act2 = activation(inpace=True) if activation == nn.ReLU else activation(num_parameters=planes)
shouldn't it be :
self.act1 = activation(inplace=True) if activation == nn.ReLU else activation(num_parameters=planes)
self.act2 = activation(inplace=True) if activation == nn.ReLU else activation(num_parameters=planes)
the error starts when Train Epoch: 0, the error says
RuntimeError:
An attempt has been made to start a new process before
the current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
ForkingPickler(file, protocol).dump(obj)
BrokenPipeError: [Errno 32] Broken pipe
in https://stackoverflow.com/questions/18204782/runtimeerror-on-windows-trying-python-multiprocessing, I can put if name == 'main' in the main code but I'm a bit confused where should I put it
Hi,
When running the cifar.py example, I got an error in the forward function (torch version '1.9.0+cu111.)
If I understand correctly, in the forward function of the conv.py module, there is a call
self.activation_post_process(
self._conv_forward(input_proc, self.weight_pre_process(self.weight), bias=self.bias),
input
)
where self.activation_post_process
is the forward function of the nn.Identity module.
This call raises an exception as this function expect only two arguments (i.e. 1 + the self object) and a third one is provided here.
Could you please confirm that the code is working on your side?
Below is the full stack trace and the error I obtained
Train Epoch: 0
/home/pgay/miniconda3/lib/python3.7/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at /pytorch/c10/core/TensorImpl.h:1156.)
return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
Traceback (most recent call last):
File "examples/cifar10.py", line 171, in <module>
train(epoch)
File "examples/cifar10.py", line 109, in train
outputs = net(inputs)
File "/home/pgay/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/pgay/miniconda3/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 166, in forward
return self.module(*inputs[0], **kwargs[0])
File "/home/pgay/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/pgay/miniconda3/lib/python3.7/site-packages/bnn/models/resnet.py", line 167, in forward
return self._forward_impl(x)
File "/home/pgay/miniconda3/lib/python3.7/site-packages/bnn/models/resnet.py", line 155, in _forward_impl
x = self.layer1(x)
File "/home/pgay/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/pgay/miniconda3/lib/python3.7/site-packages/torch/nn/modules/container.py", line 139, in forward
input = module(input)
File "/home/pgay/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/pgay/miniconda3/lib/python3.7/site-packages/bnn/models/layers/res_block.py", line 43, in forward
out = self.conv1(x)
File "/home/pgay/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/pgay/miniconda3/lib/python3.7/site-packages/bnn/layers/conv.py", line 92, in forward
input
File "/home/pgay/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
TypeError: forward() takes 2 positional arguments but 3 were given
Hello, thanks for your great work! In theory, binary network can achieve 32x reduction of the network parameters. I try to save the binarized weights and the corresponding alpha list. The best way I can think of is to convert the binarized weights to bool values ββand save them. However, each weight still requires one byte. This resulted in a model size reduction of only 4x. I would like to ask if there is any way you can save the binarized network, or any suggestions?
can the model use it for classification? do you have the code to run it? thank you
Hi, I know it has been a year since is has been done but I am not sure if you can help me. When using implicit calls, I get the following issue during training after calling the loss.backward() function.
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [1000, 1]], which is output 0 of NormBackward1, is at version 1; expected version 0 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!
I basically just grabbed the VGG19 model off pytorch and convert it. ResNet-18 have the same issue.
import torch
import torchvision
import torchvision.models as models
import torchvision.transforms as transforms
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
model = models.vgg19()
from bnn import BConfig, prepare_binary_model
# Import a few examples of quantizers
from bnn.ops import *
# Define the binarization configuration and assign it to the model
bconfig = BConfig(
activation_pre_process = BasicInputBinarizer,
activation_post_process = BasicScaleBinarizer,
# optionally, one can pass certain custom variables
weight_pre_process = XNORWeightBinarizer.with_args(center_weights=True)
)
# Convert the model appropiately, propagating the changes from parent node to leafs
# The custom_config_layers_name syntax will perform a match based on the layer name, setting a custom quantization function.
bmodel = prepare_binary_model(model, bconfig, custom_config_layers_name=[{'conv1' : BConfig()}])
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(bmodel.parameters(), lr=0.001, momentum=0.9)
print("Training begin!")
# Select GPU 4 as execution device
device = torch.device("cuda:4" if torch.cuda.is_available() else "cpu")
print("The model will be running on", device, "device")
# Convert model parameters and buffers to CPU or Cuda
bmodel.to(device)
save_path = './models/vgg19.pth'
bestaccuracy = 0.0
#break_epoch = 0
t_begin = time()
for epoch in range(50): # loop over the dataset multiple times
running_loss = 0.0
break_epoch = epoch + 1
correct = 0
total = 0
for i, data in enumerate(trainloader, 0):
# get the inputs; data is a list of [inputs, labels]
inputs, labels = data
inputs, labels = inputs.cuda(), labels.cuda()
# zero the parameter gradients
optimizer.zero_grad()
#print(inputs.size(1))
# forward + backward + optimize
outputs = bmodel(inputs)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
# check for correct answer
_, predictions = torch.max(outputs, 1)
total += labels.size(0)
correct += (predictions == labels).sum().item()
# print statistics
running_loss += loss.item()
if i % 50 == 49: # print every 50 mini-batches
print(f'[{epoch + 1}, {i + 1:5d}] loss: {running_loss / 50:.3f}')
running_loss = 0.0
#calculate accurary of epoch
accuracy = 100 * correct / total
print(f'Epoch {epoch + 1} accuracy: {accuracy:.3f}')
#If accuracy is better than the last, save the model
if accuracy > bestaccuracy:
torch.save(bmodel.state_dict(), save_path)
bestaccuracy = accuracy
time_taken = int(time()-t_begin)
time_min = int(time_taken/60)
time_sec = time_taken - (time_min*60)
print(f'Finished Training! Best accuracy: {bestaccuracy:.3f} - Training time (mm:ss): {time_min}:{time_sec}')
I am really intesressted in your new work EXPERT BINARY CONVOLUTION presented in the new paper HIGH-CAPACITY EXPERT BINARY NETWORKS ICLR 2021.
Maybe add it for the choices as one of the weight_pre_process
?
Hi! How can I install warmup_scheduler?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.