choyingw / cfcnet Goto Github PK
View Code? Open in Web Editor NEWNeurIPS 2019: Deep RGB-D Canonical Correlation Analysis For Sparse Depth Completion
License: MIT License
NeurIPS 2019: Deep RGB-D Canonical Correlation Analysis For Sparse Depth Completion
License: MIT License
I downloaded the pretrained nyu weights (500.pth), and edited line 138 of base_model.py to
state_dict = torch.load('.../500.pth', map_location=str(self.device))
When I run python evaluate.py --name nyu --checkpoints_dir ... --train_path ~/nyudepthv2/nyudepthv2/val/ --test_path ~/nyudepthv2/nyudepthv2/val/
I get:
batch_size: 16
checkpoints_dir: /home/haozhen/Downloads/500.pth [default: ./checkpoints]
continue_train: False
epoch: latest
epoch_count: 1
gpu_ids: 0
init_gain: 0.02
init_type: xavier
isTrain: False [default: None]
lr: 0.001
lr_decay_epochs: 100
lr_decay_iters: 5000000
lr_gamma: 0.9
lr_policy: lambda
max_dataset_size: inf
model: DCCA_sparse
momentum: 0.9
nP: 500
name: nyu [default: experiment_name]
niter: 400
no_flip: True
norm: instance
num_threads: 8
phase: train
print_freq: 1
resize_or_crop: none
save_epoch_freq: 1
seed: 0
serial_batches: False
suffix:
test_path: /home/haozhen/Documents/nyudepthv2/val [default: None]
train_path: /home/haozhen/Documents/nyudepthv2/val [default: None]
verbose: False
weight_decay: 0.0005
torch.Size([1, 6, 224, 224]) torch.Size([1, 1, 224, 224])
Traceback (most recent call last):
File "/home/haozhen/codes/CFCNet/evaluate.py", line 99, in
model.set_new_input(data,target)
File "/home/haozhen/codes/CFCNet/models/DCCA_sparse_model.py", line 68, in set_new_input
self.mask = input[:,7,:,:].to(self.device).unsqueeze(1)
IndexError: index 7 is out of bounds for dimension 1 with size 6
I downloaded the pretrained nyu weights (500.pth), and placed them at nyu/latest_net_DCCASparseNet.pth
.
When I run python evaluate.py --name nyu --checkpoints_dir . --train_path ~/nyudepthv2/nyudepthv2/train/ --test_path ~/nyudepthv2/nyudepthv2/val/
I get:
Found 654 images in val folder.
#test images = 654
initialize network with xavier
initialize network with xavier
model [DCCASparseNetModel] was created
loading the model from ./nyu/latest_net_DCCASparseNet.pth
---------- Networks initialized -------------
[Network DCCASparseNet] Total number of parameters : 40.417 M
-----------------------------------------------
Traceback (most recent call last):
File "evaluate.py", line 95, in <module>
model.test_depth_evaluation()
File "/home/pcheng/CFCNet/models/DCCA_sparse_model.py", line 99, in test_depth_evaluation
self.test_result.evaluate(self.depth_est.data, self.depth_image.data)
File "/home/pcheng/CFCNet/models/DCCA_sparse_model.py", line 161, in evaluate
output = output[valid_mask]
IndexError: The shape of the mask [1, 1, 228, 141] at index 3 does not match the shape of the indexed tensor [1, 1, 228, 912] at index 3
Thanks for your work!
As shown in paper,there are three strategys of sampling the dense/semi-dense depth maps. I read your code and understand two of them(Uniform & Stereo), but the ORB Sparsifier is not shown in the code. Could you tell me how to do it?
Hello,I tried to run your train_depth_complete.py but an error occurred. I run the code on Windows system,and I set num_threads to 0 and I set path in options.py.
As a beginner, I don't know whether it is caused by not running on Linux system or not setting the running parameters correctly. I hope you can help me find out the problem. Thank you very much.Here are the results of my run
Traceback (most recent call last):
File "D:/科研资料/CCA/CFCNet-master/train_depth_complete.py", line 72, in
nn = next(iterator)
File "D:\Anaconda\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 435, in next
data = self._next_data()
File "D:\Anaconda\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 475, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "D:\Anaconda\envs\pytorch\lib\site-packages\torch\utils\data_utils\fetch.py", line 47, in fetch
return self.collate_fn(data)
File "D:\Anaconda\envs\pytorch\lib\site-packages\torch\utils\data_utils\collate.py", line 83, in default_collate
return [default_collate(samples) for samples in transposed]
File "D:\Anaconda\envs\pytorch\lib\site-packages\torch\utils\data_utils\collate.py", line 83, in
return [default_collate(samples) for samples in transposed]
File "D:\Anaconda\envs\pytorch\lib\site-packages\torch\utils\data_utils\collate.py", line 55, in default_collate
return torch.stack(batch, 0, out=out)
RuntimeError: stack expects each tensor to be equal size, but got [6, 224, 224] at entry 0 and [8, 224, 224] at entry 3
进程已结束,退出代码为 1
I want to konw your total epoch when you train the model,thank you very much
Thanks fot your great work.
In SAConv:
class SAConv(nn.Module):
# Convolution layer for sparse data
def __init__(self, in_channels, out_channels, kernel_size=3, stride=1, padding=0, dilation=1, bias=True):
super(SAConv, self).__init__()
self.conv = nn.Conv2d(in_channels, out_channels, kernel_size, stride=stride, padding=padding, dilation=dilation, bias=False)
self.if_bias = bias
if self.if_bias:
self.bias = nn.Parameter(torch.zeros(out_channels).float(), requires_grad=True)
self.pool = nn.MaxPool2d(kernel_size, stride=stride, padding=padding, dilation=dilation)
nn.init.kaiming_normal_(self.conv.weight, mode='fan_out', nonlinearity='relu')
self.pool.require_grad = False
def forward(self, input):
x, m = input
x = x * m
x = self.conv(x)
weights = torch.ones(torch.Size([1, 1, 3, 3])).cuda()
mc = F.conv2d(m, weights, bias=None, stride=self.conv.stride, padding=self.conv.padding, dilation=self.conv.dilation)
mc = torch.clamp(mc, min=1e-5)
mc = 1. / mc * 9
if self.if_bias:
x = x + self.bias.view(1, self.bias.size(0), 1, 1).expand_as(x)
m = self.pool(m)
return x, m
It seems that the weights
and mc
are not used.
What does that mean?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.