tobypde / frrn Goto Github PK
View Code? Open in Web Editor NEWFull Resolution Residual Networks for Semantic Image Segmentation
License: MIT License
Full Resolution Residual Networks for Semantic Image Segmentation
License: MIT License
What batch size did you use for training? I use a GeForce GTX 1080 Ti and can use a maximum batch size of 3. Isn't it better to use larger batch sizes for training on convolutional networks?
Is it possible to do multi gpu training and can I increase the batch size if I use multi gpu?
So many questions :) hope you can help me
Thank you so much for your open source code.
I run into problems when install theano and lasagne on python3.5.1.
First, I install python 3.5.1 from source code.
Then, I use pip3.5 install theano and lasagne.
Here is the error message
Problem occurred during compilation with the command line below:
/usr/bin/g++ -shared -g -march=core-avx2 -mcx16 -msahf -mmovbe -maes -mpclmul -mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mbmi2 -mno-tbm -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mrtm -mhle -mrdrnd -mf16c -mfsgsbase -mrdseed -mprfchw -madx -mfxsr -mxsave -mxsaveopt --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=51200 -mtune=generic -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC -I/usr/local/lib/python3.5/site-packages/numpy/core/include -I/usr/local/include/python3.5m -I/usr/local/lib/python3.5/site-packages/theano/gof -fvisibility=hidden -o /root/.theano/compiledir_Linux-4.2--generic-x86_64-with-debian-jessie-sid-x86_64-3.5.1-64/lazylinker_ext/lazylinker_ext.so /root/.theano/compiledir_Linux-4.2--generic-x86_64-with-debian-jessie-sid-x86_64-3.5.1-64/lazylinker_ext/mod.cpp -L/usr/local/lib -lpython3.5m
/usr/bin/ld: /usr/local/lib/libpython3.5m.a(abstract.o): relocation R_X86_64_32S against `_Py_NotImplementedStruct' can not be used when making a shared object; recompile with -fPIC
/usr/local/lib/libpython3.5m.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
Operating System :
Ubuntu 14
Hello! Recently I encountered a problem when I ran your Networks.
When I ran the predict.py, it came out as follow:
Traceback (most recent call last):
File "/home/dingbaojin/anaconda2/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
self.run()
File "/home/dingbaojin/anaconda2/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/home/dingbaojin/deep_learning/Full_Resolution_Residual_Networks_test/FRRN-master/dltools/data.py", line 70, in init
self.start_loading()
File "/home/dingbaojin/deep_learning/Full_Resolution_Residual_Networks_test/FRRN-master/dltools/data.py", line 78, in start_loading
images, targets = self.load_batch()
File "/home/dingbaojin/deep_learning/Full_Resolution_Residual_Networks_test/FRRN-master/dltools/data.py", line 91, in load_batch
targets = [self._convert_target(d[1]) for d in data]
File "/home/dingbaojin/deep_learning/Full_Resolution_Residual_Networks_test/FRRN-master/dltools/data.py", line 125, in _convert_target
selects = class_matrix[flat_targets]
IndexError: index 70 is out of bounds for axis 0 with size 20
I hope your help, Thanks a lot!
Hello Tobias!
Thanks for a great paper!
I have a question about actual last layers dimensions in your work: in the paper you state that after last FRRU96
and streams concatenation RU48
goes โ that means for me that we need such dimensionality reductor as 48 convolutions 1x1x(96+32)
but in the FRRNABuilder::build
code I see that after concatenation you add RU with self.base_channels + self.lanes
to self.base_channels
reductor:
network = self.add_ru(
network, self.base_channels + self.lanes, self.base_channels)
In the add_ru()
method if in_channels != out_channels
the additional auto-reductor is added. Does it mean that lasagne in fact adds self.base_channels * self.multiplier + self.lanes
to self.base_channels
reductor before meaningful RU convolutions and that line in FRRNABuilder::build
is just a false track?
You might, if you have time, consider adding in associative memory to the structure of your neural networks on the presumption that it will give them extra "powers".
Associative memory was a serious topic in the 1960's (see archive.org) through the 1970's and up to the time of Hopfield.
Unfortunately all those researchers missed a very simple simple algorithm for doing associative memory:
https://devtalk.nvidia.com/default/topic/1023786/cuda-programming-and-performance/walsh-hadamard-transform-based-ai/post/5208577/#5208577
The idea is to provide the deep network with "soft" memory that it can actually learn to use.
Hi, I was wondering how effective the data augmentation is. Particularly, I was wondering if you observed significant performance difference on the validation and testing set, if without using the data augmentation. Thanks.
Hello!
I know your code is still somewhat in change, but would it be possible for you to list the version numbers of Theano and Lasagna you're using in the readme? I had to bounce around a bit. It appears that the code requires Lasagna 0.2, and Theano 0.9, but you're using their 'old' gpu backend?
Also, is there any way to run the code without cuDNN? If not, it's fine; I should really break down and buy a better GPU, but I thought I'd ask :)
Hey, thanks for sharing the code. This is a fantastic idea to do semantic segmentation.
May I ask: is there a way to utilize your code on other data, like my own data ? Is there an instruction for this purpose ?
Best
He Zhao
How do I do prediction in my own image?
Define network...
[16.67s]
Load model... [0.33s]
Define loss... [1.14s]
Compile update functions... [82.92s]
Compile validation function... [14.34s]
Optimize... [3.68s]
Traceback (most recent call last):
File "/raid/code/FRRN/FRRN-master/train_frrn_a.py", line 213, in
optimizer.optimize()
File "/raid/code/FRRN/FRRN-master/dltools/optimizer.py", line 58, in optimize
losses = self.train_fn(data.imgs, data.targets, update_counter)
File "/raid/code/FRRN/FRRN-master/train_frrn_a.py", line 141, in compute_update
loss, grads = dltools.hybrid_training.compute_grads(grad_fns, param_blocks, imgs, targets)
File "/raid/code/FRRN/FRRN-master/dltools/hybrid_training.py", line 150, in compute_grads
result = grad_fnsi
File "/usr/local/lib/python3.4/dist-packages/theano/compile/function_module.py", line 867, in call
self.inv_finder[c]))
TypeError: Missing required input: <TensorType(float32, 4D)>
Process finished with exit code 1
Hi, @TobyPDE
After switching to install python3.5, I pass the command "python train.py". However, after inputing "path to ityScapes folder", "model filename", "log filename", it runs into the following troubles:
(root) root@milton-All-Series:/data/code/FRRN# python train.py
Enter path to CityScapes folder [/data/CityScrapes_dataset/FRRN_data/leftImg8bit_trainvaltest/train]: /data/CityScrapes_dataset/FRRN_data/leftImg8bit_trainvaltest/train
Enter model filename [models/frrn_b.npz]:
Enter log filename [logs/frrn_b.log]:
Define network... [0.46s]
Load model... [0.90s]
Define loss... [0.88s]
Traceback (most recent call last):
File "train.py", line 78, in <module>
all_predictions, split_outputs = dltools.hybrid_training.get_split_outputs(network, deterministic=False)
File "/data/code/FRRN/dltools/hybrid_training.py", line 18, in get_split_outputs
all_outputs = lasagne.layers.get_output(network.output_layers + sum(network.splits, []), **kwargs)
File "/root/anaconda3/lib/python3.5/site-packages/lasagne/layers/helper.py", line 191, in get_output
all_outputs[layer] = layer.get_output_for(layer_inputs, **kwargs)
File "/root/anaconda3/lib/python3.5/site-packages/lasagne/layers/special.py", line 52, in get_output_for
return self.nonlinearity(input)
File "/data/code/FRRN/dltools/architectures.py", line 88, in log_softmax_4d
x = theano.sandbox.cuda.basic_ops.gpu_contiguous(x)
File "/root/anaconda3/lib/python3.5/site-packages/theano/gof/op.py", line 604, in __call__
node = self.make_node(*inputs, **kwargs)
File "/root/anaconda3/lib/python3.5/site-packages/theano/sandbox/cuda/basic_ops.py", line 3910, in make_node
input = as_cuda_ndarray_variable(input)
File "/root/anaconda3/lib/python3.5/site-packages/theano/sandbox/cuda/basic_ops.py", line 46, in as_cuda_ndarray_variable
return gpu_from_host(tensor_x)
File "/root/anaconda3/lib/python3.5/site-packages/theano/gof/op.py", line 604, in __call__
node = self.make_node(*inputs, **kwargs)
File "/root/anaconda3/lib/python3.5/site-packages/theano/sandbox/cuda/basic_ops.py", line 132, in make_node
dtype=x.dtype)()])
File "/root/anaconda3/lib/python3.5/site-packages/theano/sandbox/cuda/type.py", line 95, in __init__
(self.__class__.__name__, dtype, name))
TypeError: CudaNdarrayType only supports dtype float32 for now. Tried using dtype float64 for variable None
Any suggestion to fix it?
Here are my packages installed:
python 3.5.2
opencv 3.1.0
Theano 0.9.0.dev4
Lasagne 0.2.dev1
numpy 1.11.2
scikit-learn 0.17.1
Thanks~
I have problem when running the train.py, it shows me each time an error 'cannot import name 'downsample' and when i change the version of theano and Lasagne i have a problem with gpuarray.
Can you please cite the exact version of Theano and Lasagne.
Thank you
Getting this error, has anyone had this error and worked it out. I did open this issue on theano thread but no one seems to know what is going on.
>>> import theano
WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be removed in the next release (v0.10). Please switch to the gpuarray backend. You can get more information about how to switch at this URL:
https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29
Using gpu device 0: GeForce GTX 1060 (CNMeM is enabled with initial size: 100 MB, cuDNN Mixed dnn version. The header is from one version, but we link with a different version (6021, 5005))
I tried tuning off cudnn with the line below..but FRRN seems to require cudnn and throws an error.
export THEANO_FLAGS='dnn.enabled=False'
Trying to evaluation with my own image....not sure how to set the inputs correctly. Can you offer some correction.
im = cv2.imread('./eagle.jpg')
im = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)
imgray = cv2.cvtColor(im, cv2.COLOR_RGB2GRAY)
imgray = cv2.normalize(imgray, dst=imgray, alpha=0, beta=1, norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_32F)[0:-1]
imgray = imgray[np.newaxis,...]
batch1 = [im]
batch2 = [imgray]
predictions, loss = val_fn(batch1, batch2)
ERROR:
Traceback (most recent call last):
File "cam_predict.py", line 90, in <module>
main()
File "cam_predict.py", line 74, in main
predictions, loss = val_fn([batch1], [batch2])
File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 917, in __call__
storage_map=getattr(self.fn, 'storage_map', None))
File "/usr/local/lib/python2.7/dist-packages/theano/gof/link.py", line 325, in raise_with_op
reraise(exc_type, exc_value, exc_trace)
File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 903, in __call__
self.fn() if output_subset is None else\
ValueError: images and kernel must have the same stack size
Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
File "cam_predict.py", line 90, in <module>
main()
File "cam_predict.py", line 62, in main
val_fn = train.compile_validation_function(network, BATCH_SIZE)
File "/home/rjn/opencv3-p3-code/classification_and_boxing/newstuff/FRRN/train.py", line 153, in compile_validation_function
network.output_layers, deterministic=True)[0]
File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/helper.py", line 197, in get_output
all_outputs[layer] = layer.get_output_for(layer_inputs, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/conv.py", line 333, in get_output_for
conved = self.convolve(input, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/conv.py", line 611, in convolve
filter_flip=self.flip_filters)
First, thanks for the new update.
I do not have access to Cityscape, how can I use the pre trained models on other images?
Not sure how to transform pics and how to input them to the net properly.
thanks
Tets
2 warning and 1 error
Using gpu device 0: GeForce GTX 1080 Ti (CNMeM is disabled, cuDNN 6021)
/usr/local/lib/python2.7/dist-packages/theano/sandbox/cuda/init.py:600: UserWarning: Your cuDNN version is more recent than the one Theano officially supports. If you see any problems, try updating Theano or downgrading cuDNN to version 5.
warnings.warn(warn)
/usr/local/lib/python2.7/dist-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module.
"downsample module has been moved to the theano.tensor.signal.pool module.")
2017-09-08 19:34:11,781 ERROR Cannot create cuDNN spatial log softmax. Install cuDNN and make sure that theano uses the GPU.
need your help
Thanks for your greate work, have you ever test your algorithm on PASCAL VOC or COCO datasets?
Hi,
I am trying to run "python predict.py" and when I have to enter the model name I enter ""/home/user/Documents/FRRN-master/models/frrn_p.npz" and I get always the error "TypeError: super() takes at least 1 argument (0 given)"
How should I enter this path? I am using ubuntu 14.04 distribution
Thanks in advance and best regards
Jugaris
Is there any number of iteration of training?
or Just put Ctrl+C to stop training?
Hi, @TobyPDE ,
Could you list the version for each prerequisite package you mentioned in https://github.com/TobyPDE/FRRN#prerequisite?
Thanks!
pip3.5 install theano==0.9.0
Collecting theano==0.9.0
Could not find a version that satisfies the requirement theano==0.9.0 (from versions: 0.3.0rc4, 0.3.0, 0.3.1rc1, 0.3.1rc2, 0.3.1, 0.4.0rc1, 0.4.0rc2, 0.4.0rc3, 0.4.0rc4, 0.4.0, 0.4.1rc1, 0.4.1rc2, 0.4.1, 0.5.0rc1, 0.5.0rc2, 0.5.0, 0.6.0rc1, 0.6.0rc2, 0.6.0rc3, 0.6.0rc5, 0.6.0, 0.7.0rc1, 0.7.0rc2, 0.7.0, 0.8.0rc1, 0.8.0, 0.8.1, 0.8.2)
No matching distribution found for theano==0.9.0
It seems your old github didn't use chinati library? Why you write a separate C++ library?
Hi !@TobyPDE
I installed all the prerequisites as mentioned in README file. But i still get an error when i try to run predict_frrn_a.py
Please help..
(frrn) mrl@mrl:~/FRRN$ python predict_frrn_a.py
Enter path to CityScapes folder [/home/mrl/FRRN/leftImg8bit_trainvaltestt/leftImg8bit]: /home/mrl/FRRN/cityscapes
Enter model filename [/home/mrl/FRRN/models/frrn_a.npz]: /home/mrl/FRRN/models/frrn_a.npz
Define network... [0.34s]
Load model... [0.37s]
Compile validation function... [0.27s]
Traceback (most recent call last): File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/type.py", line 267, in dtype_specs }[self.dtype] KeyError: 'object'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/basic.py", line 408, in constant_or_value TensorType(dtype=x_.dtype, broadcastable=bcastable), File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/type.py", line 50, in init self.dtype_specs() # error checking is done there File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/type.py", line 270, in dtype_specs % (self.class.name, self.dtype)) TypeError: Unsupported dtype for TensorType: object
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/basic.py", line 202, in as_tensor_variable return constant(x, name=name, ndim=ndim) File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/basic.py", line 422, in constant dtype=dtype) File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/basic.py", line 417, in constant_or_value raise TypeError("Could not convert %s to TensorType" % x, type(x)) TypeError: not all arguments converted during string formatting
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "predict_frrn_a.py", line 66, in test_predictions = lasagne.layers.get_output(network.output_layers, deterministic=True)[0] File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/lasagne/layers/helper.py", line 190, in get_output all_outputs[layer] = layer.get_output_for(layer_inputs, **kwargs) File "/home/mrl/FRRN/dltools/layers.py", line 36, in get_output_for return T.nnet.abstract_conv.bilinear_upsampling(input, self.factor) File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/nnet/abstract_conv.py", line 569, in bilinear_upsampling row * ratio, col * ratio)) File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/var.py", line 327, in reshape return theano.tensor.basic.reshape(self, shape, ndim=ndim)
File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/basic.py", line 4526, in reshape newshape = as_tensor_variable(newshape) File "/home/mrl/anaconda3/envs/frrn/lib/python3.5/sitepackages/theano/tensor/basic.py", line 208, in as_tensor_variable raise AsTensorError("Cannot convert %s to TensorType" % str_x, type(x)) theano.tensor.var.AsTensorError: ('Cannot convert (None, None, Elemwise{mul,no_inplace}.0, Elemwise{mul,no_inplace}.0) to TensorType', <class 'tuple'>)
Is this a version error incompatibility error? This is using the mypredict code for single image processing.
Using cuDNN version 5110 on context None
Mapped name None to device cuda0: GeForce GTX 1060 (0000:01:00.0)
2017-09-23 17:11:07,721 INFO Only predict function
Images 1
Traceback (most recent call last):
File "predict.py", line 167, in <module>
main()
File "predict.py", line 142, in main
predictions = pred_fn(image)
File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 917, in __call__
storage_map=getattr(self.fn, 'storage_map', None))
File "/usr/local/lib/python2.7/dist-packages/theano/gof/link.py", line 325, in raise_with_op
reraise(exc_type, exc_value, exc_trace)
File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 903, in __call__
self.fn() if output_subset is None else\
RuntimeError: error getting worksize: CUDNN_STATUS_BAD_PARAM
Apply node that caused the error: GpuDnnConv{algo='small', inplace=True, num_groups=1}(GpuContiguous.0, GpuContiguous.0, GpuAllocEmpty{dtype='float32', context_name=None}.0, GpuDnnConvDesc{border_mode='half', subsample=(1, 1), dilation=(1, 1), conv_mode='conv', precision='float64', num_groups=1}.0, Constant{1.0}, Constant{0.0})
Toposort index: 757
Inputs types: [GpuArrayType<None>(float32, 4D), GpuArrayType<None>(float64, 4D), GpuArrayType<None>(float32, 4D), <theano.gof.type.CDataType object at 0x7f173b6f5b50>, Scalar(float32), Scalar(float32)]
Inputs shapes: [(1, 3, 256, 512), (48, 3, 5, 5), (1, 48, 256, 512), 'No shapes', (), ()]
Inputs strides: [(1572864, 524288, 2048, 4), (600, 200, 40, 8), (25165824, 524288, 2048, 4), 'No strides', (), ()]
Inputs values: ['not shown', 'not shown', 'not shown', <capsule object NULL at 0x7f1731d45a80>, 1.0, 0.0]
Outputs clients: [[GpuElemwise{Composite{(i0 * (Composite{(((i0 - i1) * i2 * i3) + i4)}(i1, i2, i3, i4, i5) + Abs(Composite{(((i0 - i1) * i2 * i3) + i4)}(i1, i2, i3, i4, i5))))}}[]<gpuarray>(GpuArrayConstant{[[[[ 0.5]]]]}, GpuDnnConv{algo='small', inplace=True, num_groups=1}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0)]]
Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
File "predict.py", line 167, in <module>
main()
File "predict.py", line 81, in main
pred_fn = train.only_predict_function(network)
File "/home/rjn/opencv3-p3-code/classification_and_boxing/newstuff/onhold/FRRN/FRRN/train.py", line 181, in only_predict_function
network.output_layers, deterministic=True)[0]
File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/helper.py", line 197, in get_output
all_outputs[layer] = layer.get_output_for(layer_inputs, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/conv.py", line 333, in get_output_for
conved = self.convolve(input, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/conv.py", line 611, in convolve
filter_flip=self.flip_filters)
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
Hi there,
I wanted to test your network, but somehow I can't get ride of this error:
File "/media/---/Volume/---/FRRN/dltools/data.py", line 245, in
self._map_func = np.vectorize(lambda px: self._intensity_map[px])
TypeError: list indices must be integers, not NoneType
Do I miss something, or is there a problem with the code?
I use Anaconda environment with python 2.7 on Ubuntu 16.04 with Cuda 8.
The dependencies are installed:
python check_dependencies.py --cs_folder=cityscapes_lk
2017-08-05 10:23:13,954 INFO Found supported Python version 2.7.
2017-08-05 10:23:13,979 INFO Successfully imported numpy.
2017-08-05 10:23:13,998 INFO Successfully imported cv2.
2017-08-05 10:23:14,096 INFO Successfully imported sklearn.
2017-08-05 10:23:14,098 INFO Successfully imported sklearn.metrics.
2017-08-05 10:23:14,098 INFO Successfully imported scipy.
WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be removed in the next release (v0.10). Please switch to the gpuarray backend. You can get more information about how to switch at this URL:
https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29
2017-08-05 10:23:14,223 WARNING The cuda backend is deprecated and will be removed in the next release (v0.10). Please switch to the gpuarray backend. You can get more information about how to switch at this URL:
https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29
Using gpu device 1: GeForce GTX 1080 (CNMeM is enabled with initial size: 95.0% of memory, cuDNN 5110)
2017-08-05 10:23:15,236 INFO Successfully imported theano.
2017-08-05 10:23:15,239 INFO Successfully imported lasagne.
2017-08-05 10:23:15,239 INFO Theano float is float32.
2017-08-05 10:23:15,240 INFO cuDNN spatial softmax found.
2017-08-05 10:23:15,241 INFO Chianti C++ library is not available. Will default to slower Python implementation.
2017-08-05 10:23:15,249 INFO Found CityScapes training set.
2017-08-05 10:23:15,250 INFO Found CityScapes validation set.
Full Traceback until ctrl-c:
python predict.py --architecture frrn_a --model_file models/frrn_a.npz --cs_folder cityscapes_lk
WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be removed in the next release (v0.10). Please switch to the gpuarray backend. You can get more information about how to switch at this URL:
https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29
Using gpu device 1: GeForce GTX 1080 (CNMeM is enabled with initial size: 95.0% of memory, cuDNN 5110)
2017-08-05 10:23:23,614 INFO Compile validation function
Process Process-2:
Traceback (most recent call last):
File "/home/---/anaconda3/envs/frnn/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/---/anaconda3/envs/frnn/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/media/---/Volume/---/FRRN/dltools/data.py", line 70, in init
self.start_loading()
File "/media/---/Volume/---/FRRN/dltools/data.py", line 78, in start_loading
images, targets = self.load_batch()
File "/media/---/Volume/---/FRRN/dltools/data.py", line 87, in load_batch
data = [self._data_loader.get_pair() for _ in range(self._batch_size)]
File "/media/---/Volume/---/FRRN/dltools/data.py", line 47, in get_pair
target = self._target_loader.load(target_file)
File "/media/---/Volume/---/FRRN/dltools/data.py", line 257, in load
img = self._map_func(img)
File "/home/---/anaconda3/envs/frnn/lib/python2.7/site-packages/numpy/lib/function_base.py", line 2734, in call
return self._vectorize_call(func=func, args=vargs)
File "/home/---/anaconda3/envs/frnn/lib/python2.7/site-packages/numpy/lib/function_base.py", line 2804, in _vectorize_call
ufunc, otypes = self._get_ufunc_and_otypes(func=func, args=args)
File "/home/---/anaconda3/envs/frnn/lib/python2.7/site-packages/numpy/lib/function_base.py", line 2764, in _get_ufunc_and_otypes
outputs = func(*inputs)
File "/media/---/Volume/---/FRRN/dltools/data.py", line 245, in
self._map_func = np.vectorize(lambda px: self._intensity_map[px])
TypeError: list indices must be integers, not NoneType
Thanks in advance!
Hi there,
I'm really interested in using your network and performance does matter, that's why I installed chianti.
However I get this error:
python predict.py --architecture frrn_a --model_file models/frrn_a.npz --cs_folder ../Datasets/cityscapes_lk
Using cuDNN version 6021 on context None
Mapped name None to device cuda0: GeForce GTX 960 (0000:01:00.0)
2017-08-11 11:30:03,314 INFO Compile validation function
Traceback (most recent call last):
File "predict.py", line 84, in
main()
File "predict.py", line 63, in main
BATCH_SIZE)
File "/---/FRRN/train.py", line 264, in get_validation_provider
pychianti.Iterator.Sequential(validation_images),
TypeError: No registered converter was able to produce a C++ rvalue of type std::__cxx11::basic_string<char, std::char_traits, std::allocator > from this Python object of type str
I'm using python 2.7 in Anaconda. I have all dependencies. Boost version is 1.64. I tried different version of boost / boost-python and still got the same error. Any clue?
Hi, @TobyPDE ,
I installed the prerequisite packaged as listed in readme.md, then I cloned FRRN and then issue command:
python predict.py, I got the following error :
(cv_python3) root@milton-OptiPlex-9010:/data/code2/FRRN# python predict.py /root/.virtualenvs/cv_python3/lib/python3.4/site-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module.
"downsample module has been moved to the theano.tensor.signal.pool module.")
Traceback (most recent call last):
File "predict.py", line 3, in <module>
import dltools
File "/data/code2/FRRN/dltools/__init__.py", line 12, in <module>
from . import hybrid_training
File "/data/code2/FRRN/dltools/hybrid_training.py", line 112
result = grad_fns[i](*args, *prev)
^
SyntaxError: invalid syntax
(cv_python3) root@milton-OptiPlex-9010:/data/code2/FRRN#
Any suggestion to fix it? Thx~
Hi @TobyPDE
I can inference this network and result is good, but there are some issue in train.py
This is my log.
Using gpu device 0: GeForce GTX 970 (CNMeM is disabled, cuDNN 5005)
Enter path to CityScapes folder [DataSet/]:
Enter model filename [models/frrn_b.npz]:
Enter log filename [logs/frrn_b.log]:
Define network... [0.32s]
Load model... [0.26s]
Define loss... [1.54s]
Compile update functions... [158.36s]
Compile validation function... [22.07s]
Optimize...Evaluate block 5/5
[0.70s]
Traceback (most recent call last):
File "train.py", line 197, in <module>
optimizer.optimize()
File "/home/FRRN/dltools/optimizer.py", line 55, in optimize
losses = self.train_fn(*train_fn_args)
File "train.py", line 140, in compute_update
loss, grads = dltools.hybrid_training.compute_grads(grad_fns, param_blocks, *args)
File "/home/unicornx/FRRN/dltools/hybrid_training.py", line 112, in compute_grads
result = grad_fns[i](*args, *prev)
File "/home/Theano/theano/compile/function_module.py", line 788, in __call__
allow_downcast=s.allow_downcast)
File "/home/Theano/theano/tensor/type.py", line 178, in filter
data.shape))
TypeError: Bad input argument to theano function with name "/home/FRRN/dltools/hybrid_training.py:84" at index 1 (0-based).
Backtrace when that variable is created:
File "train.py", line 74, in <module>
target_var = T.itensor3()
Wrong number of dimensions: expected 3, got 4 with shape (3, 3, 512, 1024).
Any idea?
Thanks!!
could you provide the init.model to train?
@TobyPDE
The batch_size in test_classification_loss is same to the batch_size in train_loss, which is default by 3 different to the value 1 in validation_provider.
# Validation classification loss (supervised)
test_classification_loss = dltools.utility.bootstrapped_categorical_cross_entropy4d_loss(
test_predictions,
target_var,
batch_size = config["batch_size"],
multiplier = 64)
validation_provider = dltools.data.CityscapesHDDDataProvider(
config["cityscapes_folder"],
file_folder="val",
batch_size=1,
augmentor=None,
sampling_factor=config["sample_factor"],
random=False
)
Hey, I tested your framework with the cityscape dataset and the trained models (predict.py) but the output is very bad (losses between 100 - 500). I checked the output images(image, gt_img and pred_img) and saw that the RGB is just black. The gt is the same like from the dataset and the pred is a very bad segmented image.
The paths to the cityscape seems to be right and the dependency check runs with no error.
Thank you for help.
When I trained the model with new semantic DB, I used empty folders("logs" and "models") for train.py.
For finish training process, I put Ctrl+C. Logs folder has the files for log. But there are no files in models folder.
Should I use the exising models file and folder for training(models/frrn_b.npz)?
Does the train.py makes new model file?
Do I have to get any initial stuff before train?
thanks
I want to adapt the the resolution to 1280x800. Therefore I changed the IMAGE_ROWS and IMAGE_COLS in the train.py script and resized the train and val images. The training works but for prediction I get an "Segmentation fault (core dumped)" error. The images for prediction are also adapted to 1280x800. I tried also different sample sizes.
any suggestions?
Thanks
Hi there,
I'm really interested in using your network.
However I get this error:
2017-11-01 17:15:45,539 ERROR Cannot create cuDNN spatial log softmax. Install cuDNN and make sure that uses the GPU.
2017-11-01 17:15:45,540 INFO Chianti C++ library is not available.Will default to slower Python implementation.
I have updated theano and added " [lib]\n cnmem=1"to a .theanorc file in my home location.
I really need your help,so I beg your reply.
I'd like to train on 400x600 images, but I am encountering a problem.
ValueError: Mismatch: not all input shapes are the same
is raised in https://github.com/TobyPDE/FRRN/blob/master/dltools/architectures.py#L315 because autobahn.input_shapes = [(None, 32, 600, 400), (None, 32, 592, 400)]
Are there any constraints on the input dimensions?
Hi @TobyPDE
I have use train.py to train a new model, but if I would like to fine tune the pre trained model, how should I do?
Thanks!!
I tried to run FRRN using other pictures for prediction. But I had to change the names of pictures and the names had to be just like the pictures of Cityscapes. Can you offer a version that FRRN can predict all pictures without change the names of pictures?
I tried to change the code to realize this idea but the results were so bad that I doubted that there was something wrong with my code. Once I removed the "batch[1]" from the "val_fn" in prediction.py and, of course, correspondingly changed the code in train.py, the results changed and the results got bad.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.