I'm trying to run the training on two GTX 1080Ti but it stuck on the first epoch. Is this due to some post-processing operations? How can I solve it?

Hello
How are you?
Thanks for contributing this project.
I am going to train a new model with my custom dataset(supervisely-person-dataset).
How should I prepare the training data from my custom dataset?
Thanks

Hello, I tried the model provided, and I tried to re train the network with citys and nothing helps. When I use the validation images I have good results and when I use images somehow it never works.

Original :

Result :

@Tramac .When I run eval.py to predict my own images.I met a problem.The code can inference the first image in the folder of validation ,but when I inference the second image and then an error occurred,as fellow:
Traceback (most recent call last): File "eval.py", line 96, in <module> evaluator.eval() File "eval.py", line 66, in eval outputs = self.model(image) File "/home/liyanzhou/anaconda3/envs/py36torch10_fastscnn/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__ result = self.forward(*input, **kwargs) File "/home/liyanzhou/pycharm_project/demo/Fast-SCNN-pytorch-master/models/fast_scnn.py", line 35, in forward higher_res_features = self.learning_to_downsample(x) File "/home/liyanzhou/anaconda3/envs/py36torch10_fastscnn/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__ result = self.forward(*input, **kwargs) File "/home/liyanzhou/pycharm_project/demo/Fast-SCNN-pytorch-master/models/fast_scnn.py", line 159, in forward x = self.dsconv1(x) File "/home/liyanzhou/anaconda3/envs/py36torch10_fastscnn/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__ result = self.forward(*input, **kwargs) File "/home/liyanzhou/pycharm_project/demo/Fast-SCNN-pytorch-master/models/fast_scnn.py", line 79, in forward return self.conv(x) File "/home/liyanzhou/anaconda3/envs/py36torch10_fastscnn/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__ result = self.forward(*input, **kwargs) File "/home/liyanzhou/anaconda3/envs/py36torch10_fastscnn/lib/python3.6/site-packages/torch/nn/modules/container.py", line 92, in forward input = module(input) File "/home/liyanzhou/anaconda3/envs/py36torch10_fastscnn/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__ result = self.forward(*input, **kwargs) File "/home/liyanzhou/anaconda3/envs/py36torch10_fastscnn/lib/python3.6/site-packages/torch/nn/modules/batchnorm.py", line 83, in forward exponential_average_factor, self.eps) File "/home/liyanzhou/anaconda3/envs/py36torch10_fastscnn/lib/python3.6/site-packages/torch/nn/functional.py", line 1697, in batch_norm training, momentum, eps, torch.backends.cudnn.enabled RuntimeError: CUDA out of memory. Tried to allocate 152.00 MiB (GPU 0; 10.76 GiB total capacity; 9.88 GiB already allocated; 36.19 MiB free; 68.67 MiB cached)
The size of the image I entered was w*h=5440*3648,the crop-size is 2816.How to solve it? Can you give me some advice? Thank you!

I use this code to train my own dataset of cityscrapes format. The loss becomes 0.001 after 500 epochs, but the test effect is very poor, and the trained class is not displayed. Can the author upload the examples of the training cityscraps dataset to the datasets folder（ in the ./datasets/citys dir.）?

Hi,

I have trained Fast-SCNN with coco dataset for ~20 epochs, the inference output has nothing(entire image is blank). Please can throw so tips to debug this issue. Do i need to do any preprocessing on coco dataset.
Highly appreciate your help on this.

training logs:
Epoch: [18/160] Iter [1694/1927] || Time: 143237.6596 sec || lr: 0.00893152 || Loss: 0.6481
Epoch: [18/160] Iter [1704/1927] || Time: 143277.4028 sec || lr: 0.00893123 || Loss: 0.6621
Epoch: [18/160] Iter [1714/1927] || Time: 143315.9710 sec || lr: 0.00893093 || Loss: 0.5438
Epoch: [18/160] Iter [1724/1927] || Time: 143354.9870 sec || lr: 0.00893064 || Loss: 0.7725
Epoch: [18/160] Iter [1734/1927] || Time: 143394.4705 sec || lr: 0.00893034 || Loss: 0.5357
Epoch: [18/160] Iter [1744/1927] || Time: 143434.3466 sec || lr: 0.00893005 || Loss: 0.5382
Epoch: [18/160] Iter [1754/1927] || Time: 143475.0332 sec || lr: 0.00892975 || Loss: 0.4904
Epoch: [18/160] Iter [1764/1927] || Time: 143515.1107 sec || lr: 0.00892945 || Loss: 0.5637
Epoch: [18/160] Iter [1774/1927] || Time: 143554.9711 sec || lr: 0.00892916 || Loss: 0.6457
Epoch: [18/160] Iter [1784/1927] || Time: 143594.5968 sec || lr: 0.00892886 || Loss: 0.4945
Epoch: [18/160] Iter [1794/1927] || Time: 143633.9663 sec || lr: 0.00892857 || Loss: 0.3928
Epoch: [18/160] Iter [1804/1927] || Time: 143674.1125 sec || lr: 0.00892827 || Loss: 0.4256
Epoch: [18/160] Iter [1814/1927] || Time: 143712.8005 sec || lr: 0.00892798 || Loss: 0.7373
Epoch: [18/160] Iter [1824/1927] || Time: 143752.1207 sec || lr: 0.00892768 || Loss: 0.4910
Epoch: [18/160] Iter [1834/1927] || Time: 143791.0121 sec || lr: 0.00892738 || Loss: 0.6223
Epoch: [18/160] Iter [1844/1927] || Time: 143831.8926 sec || lr: 0.00892709 || Loss: 0.6388
Epoch: [18/160] Iter [1854/1927] || Time: 143870.4438 sec || lr: 0.00892679 || Loss: 0.6908
Epoch: [18/160] Iter [1864/1927] || Time: 143909.4174 sec || lr: 0.00892650 || Loss: 0.4131
Epoch: [18/160] Iter [1874/1927] || Time: 143948.1374 sec || lr: 0.00892620 || Loss: 0.4725
Epoch: [18/160] Iter [1884/1927] || Time: 143987.2357 sec || lr: 0.00892591 || Loss: 0.6805
Epoch: [18/160] Iter [1894/1927] || Time: 144026.5409 sec || lr: 0.00892561 || Loss: 0.7085
Epoch: [18/160] Iter [1904/1927] || Time: 144066.6354 sec || lr: 0.00892532 || Loss: 0.6547
Epoch: [18/160] Iter [1914/1927] || Time: 144105.0635 sec || lr: 0.00892502 || Loss: 0.4116
Epoch: [18/160] Iter [1924/1927] || Time: 144145.9772 sec || lr: 0.00892472 || Loss: 0.5329
Epoch: [19/160] Iter [ 7/1927] || Time: 144185.6879 sec || lr: 0.00892443 || Loss: 0.5201
Epoch: [19/160] Iter [ 17/1927] || Time: 144224.2033 sec || lr: 0.00892413 || Loss: 0.6291
Epoch: [19/160] Iter [ 27/1927] || Time: 144263.8119 sec || lr: 0.00892384 || Loss: 0.8196
Epoch: [19/160] Iter [ 37/1927] || Time: 144304.1935 sec || lr: 0.00892354 || Loss: 0.6305
Epoch: [19/160] Iter [ 47/1927] || Time: 144342.7663 sec || lr: 0.00892325 || Loss: 0.4930
Epoch: [19/160] Iter [ 57/1927] || Time: 144382.1391 sec || lr: 0.00892295 || Loss: 0.4747
Epoch: [19/160] Iter [ 67/1927] || Time: 144420.8839 sec || lr: 0.00892265 || Loss: 0.5396
Epoch: [19/160] Iter [ 77/1927] || Time: 144460.3386 sec || lr: 0.00892236 || Loss: 0.5535
Epoch: [19/160] Iter [ 87/1927] || Time: 144499.7273 sec || lr: 0.00892206 || Loss: 0.5598
Epoch: [19/160] Iter [ 97/1927] || Time: 144539.2534 sec || lr: 0.00892177 || Loss: 0.4794

@Tramac @BorisTestov @prianchoI try to use the coco dataset and only include two classes person and background , when I train ,the errors are follows:

_File "train.py", line 203, in
trainer.train()
File "train.py", line 139, in train
loss = self.criterion(outputs, targets)
File "/home/data/anaconda3/envs/caffe2_py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/data/code/Fast-SCNN-pytorch/utils/loss.py", line 120, in forward
return super(MixSoftmaxCrossEntropyOHEMLoss, self).forward(*inputs)
File "/home/data/code/Fast-SCNN-pytorch/utils/loss.py", line 96, in forward
return self.criterion(predict, target)
File "/home/data/anaconda3/envs/caffe2_py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/data/anaconda3/envs/caffe2_py36/lib/python3.6/site-packages/torch/nn/modules/loss.py", line 942, in forward
ignore_index=self.ignore_index, reduction=self.reduction)
File "/home/data/anaconda3/envs/caffe2_py36/lib/python3.6/site-packages/torch/nn/functional.py", line 2056, in cross_entropy
return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
File "/home/data/anaconda3/envs/caffe2_py36/lib/python3.6/site-packages/torch/nn/functional.py", line 1873, in nll_loss
ret = torch._C._nn.nll_loss2d(input, target, weight, Reduction.get_enum(reduction), ignore_index)
RuntimeError: weight tensor should be defined either for all or no classes at /pytorch/aten/src/THCUNN/generic/SpatialClassNLLCriterion.cu:27

Do you have any suggestion or solution? I don't know how to slove it.

if the input picture's dim is not divisible by 32，in forward step higher_res_feature and low_res_feature can't be concatenated together.

Hi, Thanks your great repo. I successed to run demo.py according to README, but when I try to other images ,it maked error as follows:
File "demo.py", line 160, in
demo()
File "demo.py", line 52, in demo
outputs = model(image)
File "/home/data/anaconda3/envs/caffe2_py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/data/code/Fast-SCNN-pytorch/models/fast_scnn.py", line 37, in forward
x = self.feature_fusion(higher_res_features, x)
File "/home/data/anaconda3/envs/caffe2_py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/data/code/Fast-SCNN-pytorch/models/fast_scnn.py", line 215, in forward
out = higher_res_feature + lower_res_feature
RuntimeError: The size of tensor a (135) must match the size of tensor b (136) at non-singleton dimension 2
the image size is 1080*1920, should I resize the fixed size?

I got a error in loss.py line 54. I print my target size is (2,768,768,3), but the code shows it should be dim 3. my predict size is (2,3,768,768).
I use my own dataset, picture annotation like cityscapes. then I get this error. I don't know what happen.

RuntimeError: The size of tensor a (129) must match the size of tensor b (132) at non-singleton dimension 2.

Thanks for your contributions. But I have no idea what's wrong. Please help.

Hi, @Tramac , I use your code with default setting, and find the speed of training is too slow. The usage of GPU is less than 10%, resulting in about 130 seconds to train a mini-batch. How to solve this problem?

There was no such problem during the first training. The same data set for the second training is reported as follows：

Namespace(aux=False, aux_weight=0.4, base_size=1, batch_size=2, crop_size=768, dataset='citys', device=device(type='cuda', index=0), epochs=160, eval=False, lr=0.01, model='fast_scnn', momentum=0.9, no_val=True, resume=None, save_folder='./weights', start_epoch=0, train_split='train', weight_decay=0.0001) Found 5 images in the folder ./datasets/citys/leftImg8bit/train Found 3 images in the folder ./datasets/citys/leftImg8bit/val w/ class balance Starting Epoch: 0, Total Epochs: 160 Traceback (most recent call last): File "/home/lyl/000_Code/Fast-SCNN/train.py", line 200, in <module> trainer.train() File "/home/lyl/000_Code/Fast-SCNN/train.py", line 127, in train for i, (images, targets) in enumerate(self.train_loader): File "/home/lyl/anaconda3/envs/fastscnn1.0/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 615, in __next__ batch = self.collate_fn([self.dataset[i] for i in indices]) File "/home/lyl/anaconda3/envs/fastscnn1.0/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 615, in <listcomp> batch = self.collate_fn([self.dataset[i] for i in indices]) File "/home/lyl/000_Code/Fast-SCNN/data_loader/cityscapes.py", line 93, in __getitem__ img, mask = self._sync_transform(img, mask) File "/home/lyl/000_Code/Fast-SCNN/data_loader/cityscapes.py", line 141, in _sync_transform img = img.resize((ow, oh), Image.BILINEAR) File "/home/lyl/anaconda3/envs/fastscnn1.0/lib/python3.6/site-packages/PIL/Image.py", line 1923, in resize return self._new(self.im.resize(size, resample, box)) ValueError: height and width must be > 0

Training for COCO dataset using mscoco.py,

File "train.py", line 201, in
trainer.train()
File "train.py", line 137, in train
loss = self.criterion(outputs, targets)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/Documents/Fast-SCNN-pytorch/utils/loss.py", line 114, in forward
return super(MixSoftmaxCrossEntropyOHEMLoss, self).forward(*inputs)
File "/home/Documents/Fast-SCNN-pytorch/utils/loss.py", line 90, in forward
return self.criterion(predict, target)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/loss.py", line 862, in forward
ignore_index=self.ignore_index, reduction=self.reduction)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py", line 1550, in cross_entropy
return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py", line 1409, in nll_loss
return torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
RuntimeError: weight tensor should be defined either for all or no classes at /pytorch/aten/src/THCUNN/generic/SpatialClassNLLCriterion.cu:27

Hello, impressive repo. Do you have the label of the cityscapes? Also do you have a timeline for ADE20k pretrained model release? Thanks

Hello,

Are the weights automatically saved in eachh epoch, or how frequently they are saved during training ? How can I restore them if I need to stop the training at some point and continue later?

Thank you for your support.

Hello
How are you?
Thanks for contributing this project.
It seems that the script demo.py can not process any size of image.
Could u revise the script so that it can process any size of image?
Thanks.

You report 54% mIoU, I got around 49% after training with the train script (160 epochs, batch size 12). Any idea why this is much lower as the reported 69% in the paper?

@Tramac, Thanks for the repo. Was able to train on cityscapes and coco dataset and prediction is good on the objects (both cityscapes and coco) but there is noise in most of the predictions. I see there is a noise even in the image you have posted ((c) predicted result - noise on the road red color at the bottom). Is it possible to reduce this noise by tuning any hyperparameter while training? Any thoughts ?

Sorry for asking same question as it is asked before but not answered.
i am interested about the speed 123 fps mentioned by the author for high resolution images. i have trained this network for 1000 epochs and got 94% pixel accuracy and 63.5% mIoU. But i got only 10 fps on GeForce RTX 2080 with resolution of (1024x2048). can you please share your speed.

Hi,

do you support voc or coco dataset ? if not can you guide me through the training process (changes need). Or if you support please can you share model files or script to train.

thanks for your help

Hi, I use this project to train Cityscapes, use adam optim，lr is 3e-3,but the loss decline is too slow ,how can i solve this problem'
Thanks

How does it solve？

hi @Tramac , thanks for your job. i have reproduced the miou(54.3%) and pa(92.1%) at cropsize=768 . but i want to know whether you reproduce the paper's result(68.0%) or not.

Hello,

I'm trying to run the train.py file however the training does not start. I increased the batch size to 4 and decreased it to 2 again but it does not start either way. You can check the attached file to see where it gets stuck.

How can I solve it?
Thank you for your time.

Can I train with my own data set?if i can,What should I do

I've tested it on TITAN X shows it can only run on 43.85 iter/s using (1024, 2048) resolution. So I wonder that How the FastScnn can run on 123.5 iter/s using (1024, 2048) resolution? Or, can you report your speed on inference
Thank you.

你好，我想训练一下aeroscapes数据集，他和pascal voc很像，所以我用了您另外一个project中的pascal_voc.py，我在运行之后，出现了如下问题：

Traceback (most recent call last):
File "train.py", line 211, in
trainer.train()
File "train.py", line 138, in train
for i, (images, targets) in enumerate(self.train_loader):
ValueError: too many values to unpack (expected 2)

我感觉他应该是和图像的大小有关系，因为我用的数据集的图像是1280x720，我想问一下我该如何转换图片的大小使其于网络匹配？

hello,What is the different of _DWconv and _DSconv function?

Hello,
Is the ADE20K dataset already supported? If that's not the case, would you have directions on how to adapt it?

Best Regards.

Can you please upload the trained model on cityscapes?

i can use split test and val, but for train it getting error, i have already try to change the variable mode for testval and trainval to test but it still error because the y label is not work

I can't save images from annotation so the dataset can't be train and predict

The valid_classes, key, and mapping in the code seem confusing.

Dear Tramac,
Thanks for your work at first!
The way you calculate the MeanIOU is incorrect. From my understanding you at first have to calculate the Intersection of Union for each class that occures in the target image. Then you have to take the mean. What you do is summing up all pixels that fall inside the union or the intersection and simply take the average. This leads to an incorrect metric from my understand. Please correct me if I am wrong. It is pretty clearly stated here averaged over all classes: \url{https://www.jeremyjordan.me/evaluating-image-segmentation-models/}

Hi,Thanks for your work.I have adapted the data_loder file of VOC from your another repo https://github.com/Tramac/awesome-semantic-segmentation-pytorch to train the VOC data.But when I went to the validation step after my training, I met the runtime error as following:

Namespace(aux=False, aux_weight=0.4, base_size=520, batch_size=2, crop_size=480, dataset='pascal_voc', device=device(type='cuda', index=0), epochs=160, eval=False, lr=0.01, model='fast_scnn', momentum=0.9, no_val=True, resume=None, save_folder='./weights', start_epoch=0, train_split='train', weight_decay=0.0001) Found 1449 images in the folder ./datasets/voc/VOC2012 Finished loading model! Testing model: fast_scnn Traceback (most recent call last): File "eval.py", line 62, in <module> evaluator.eval() File "eval.py", line 43, in eval outputs = self.model(image) File "/home/eli/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__ result = self.forward(*input, **kwargs) File "/home/eli/Fast-scnn_voc/models/fast_scnn.py", line 37, in forward x = self.feature_fusion(higher_res_features, x) File "/home/eli/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__ result = self.forward(*input, **kwargs) File "/home/eli/Fast-scnn_voc/models/fast_scnn.py", line 213, in forward out = higher_res_feature + lower_res_feature RuntimeError: The size of tensor a (63) must match the size of tensor b (64) at non-singleton dimension 3
Is there any suggestion to solve this problem? Thanks.

I am not sure how to resolve this error.
Please let me know @Tramac @BorisTestov

Hi,

I am wondering why data_loader.py creates zero padded crop images when short_size is smaller than crop_size.

• https://github.com/Tramac/Fast-SCNN-pytorch/blob/master/data_loader/cityscapes.py#L133

I read the original paper, but they only mentions that they used crop.
Is there any reason not to discard such an image?

Thanks for a great s/w.

Best regards,
Han-Cheol

Description

In train.py, MixSoftmaxCrossEntropyOHEMLoss() method (https://github.com/Tramac/Fast-SCNN-pytorch/blob/master/train.py#L103) doesn't allow to control use_weight option, which is True by default.

I think that it should be False by default because this class balancing weights should be applied only to cityscapes dataset.
It might be better to provide class balancing weights from cmd-args or from data_loader class (e.g., data_loader/cityscapes.py).

1. I ran into an issue with the eval.py and demo.py scripts that is missing keys in the state_dict:
   ##########
   Traceback (most recent call last): File "demo.py", line 55, in demo() File "demo.py", line 43, in demo model = get_fast_scnn(args.dataset, pretrained=True, root=args.weights_folder, map_cpu=args.cpu).to(device) File "/mnt/git/Fast-SCNN-pytorch/models/fast_scnn.py", line 251, in get_fast_scnn model.load_state_dict(torch.load(os.path.join(root, 'fast_scnn_%s.pth' % acronyms[dataset]))) File "/home/user/miniconda/envs/py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 769, in load_state_dict self.class.name, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for FastSCNN: Missing key(s) in state_dict: "learning_to_downsample.conv.conv.0.weight",
   ...
   "module.classifier.conv.1.bias".
   ##########

2. I could do the training via:
   python train.py --model fast_scnn --dataset citys --batch-size 60 --epochs 80

3. I am using the follwing nvidia-docker with
   docker pull anibali/pytorch:cuda-10.0
   --> CUDA10.0
   --> PyTorch 1.0.0
   --> Python 3.6.5 :: Anaconda, Inc.
   --> Torchvision 0.2.1
   --> Using 2 Titan RTX GPUs
   --> nvidia-smi: NVIDIA-SMI 410.104 Driver Version: 410.104 CUDA Version: 10.0

Thank you in advance!

Hi！Excuse me！
I use this network to detect defects.Use MixSoftmaxCrossEntropyLoss loss function, training to 400 eopch, but loss function has been drop down between 0.5 to 0.7.May I ask what the problem is?

Hi,

I'm trying to train your framework on Mac OS and got this assertion error, as expected because my device has not got an Nvidia GPU. Can I run the framework on my CPU, if yes how can I do it, which lines should I change or add?

Thank you for your time.

I tried to recreate the training of the model. I trained the model with the newest version of cityscapes. Before training the following error occurs:
Found 2970 images in the folder ./datasets/citys/leftImg8bit/train Found 500 images in the folder ./datasets/citys/leftImg8bit/val w/ class balance Starting Epoch: 0, Total Epochs: 160 Traceback (most recent call last): File "/home/hannes/.vscode/extensions/ms-python.python-2020.2.64397/pythonFiles/ptvsd_launcher.py", line 48, in <module> main(ptvsdArgs) File "/home/hannes/.vscode/extensions/ms-python.python-2020.2.64397/pythonFiles/lib/python/old_ptvsd/ptvsd/__main__.py", line 432, in main run() File "/home/hannes/.vscode/extensions/ms-python.python-2020.2.64397/pythonFiles/lib/python/old_ptvsd/ptvsd/__main__.py", line 316, in run_file runpy.run_path(target, run_name='__main__') File "/usr/lib/python3.7/runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "/usr/lib/python3.7/runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "/usr/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/media/hannes/SanDisk/Fast-SCNN-pytorch/train.py", line 201, in <module> trainer.train() File "/media/hannes/SanDisk/Fast-SCNN-pytorch/train.py", line 128, in train for i, (images, targets) in enumerate(self.train_loader): File "/home/hannes/.local/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 345, in __next__ data = self._next_data() File "/home/hannes/.local/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 385, in _next_data data = self._dataset_fetcher.fetch(index) # may raise StopIteration File "/home/hannes/.local/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/hannes/.local/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp> data = [self.dataset[idx] for idx in possibly_batched_index] File "/media/hannes/SanDisk/Fast-SCNN-pytorch/data_loader/cityscapes.py", line 79, in __getitem__ mask = Image.open(self.mask_paths[index]) File "/usr/lib/python3/dist-packages/PIL/Image.py", line 2822, in open raise IOError("cannot identify image file %r" % (filename if filename else fp)) OSError: cannot identify image file './datasets/citys/gtFine/train/darmstadt/darmstadt_000046_000019_gtFine_labelIds.png'

Please help.

I'm running the training file using the Cityscapes datasets mentioned, however the loss does not decrease accordingly. In the beginningg it was around 2, in 90th epoch it's struggling around 1. All my hyper parameters are at default; the learning rate is 1e-2, momentum is 0.9 and weight decay is 1e-4.

I tried running eval.py on the cityscapes test set and I get 0 mIoU and 0 Pixel accuracy on all images. The results look fine though. Any idea why this might be happening?

Is there a possible implementation for running the segmentation network on a video through open cv? Any help would be much appreciated.

When I try to load a video through open cv and use my webcam as the source, I get the following error:

File "/Users/user/Downloads/Fast-SCNN-pytorch/models/fast_scnn.py", line 213, in forward out = higher_res_feature + lower_res_feature RuntimeError: The size of tensor a (90) must match the size of tensor b (92) at non-singleton dimension 2

The edited code in demo.py is:

def demo():
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    # output folder
    if not os.path.exists(args.outdir):
        os.makedirs(args.outdir)

    # image transform
    transform = transforms.Compose([
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
    ])

    model = get_fast_scnn(args.dataset, pretrained=True,
                          root=args.weights_folder, map_cpu=args.cpu).to(device)
    print('Finished loading model!')
    model.eval()

    capture = cv2.VideoCapture(0)
    while capture.isOpened():
        ret, frame = capture.read()
        if ret:
            frame = Image.fromarray(frame.astype('uint8')).convert('RGB')
            image = transform(frame).unsqueeze(0).to(device)

            with torch.no_grad():
                result = model(image)
            pred = torch.argmax(result[0], 1).squeeze(0).cpu().data.numpy()
            pred = pred.astype('float64')
            cv2.imshow("frame", pred)

            key = cv2.waitKey(1)
            if key & 0xFF == ord('q'):
                break
        else:
            break

There is no problem in training and testing process ,but for each image, the ac is 100, miou is 50, and the predicted mask is whole black, what shoud i do?

Hello author, may I ask what is the learning rate and batch_size you are using? I ran the code directly, and its loss fluctuated between 1.1 and 0.9. I also used other learning rates, but the effect was not very good. Can you share me with your learning rate and batch_size? Thank you very much!

when i change the default crop-size, it give the follow error:

Traceback (most recent call last):
File "/home/zyx/.vscode-server/extensions/ms-python.python-2019.11.50794/pythonFiles/ptvsd_launcher.py", line 43, in
main(ptvsdArgs)
File "/home/zyx/.vscode-server/extensions/ms-python.python-2019.11.50794/pythonFiles/lib/python/old_ptvsd/ptvsd/main.py", line 432, in main
run()
File "/home/zyx/.vscode-server/extensions/ms-python.python-2019.11.50794/pythonFiles/lib/python/old_ptvsd/ptvsd/main.py", line 316, in run_file
runpy.run_path(target, run_name='main')
File "/home/zyx/anaconda3/envs/py36/lib/python3.6/runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "/home/zyx/anaconda3/envs/py36/lib/python3.6/runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "/home/zyx/anaconda3/envs/py36/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/zyx/Fast-SCNN-pytorch/train.py", line 229, in
trainer.train()
File "/home/zyx/Fast-SCNN-pytorch/train.py", line 153, in train
outputs = self.model(images)
File "/home/zyx/anaconda3/envs/py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/zyx/Fast-SCNN-pytorch/models/fast_scnn.py", line 37, in forward
x = self.feature_fusion(higher_res_features, x)
File "/home/zyx/anaconda3/envs/py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/zyx/Fast-SCNN-pytorch/models/fast_scnn.py", line 213, in forward
out = higher_res_feature + lower_res_feature
RuntimeError: The size of tensor a (90) must match the size of tensor b (92) at non-singleton dimension 3

and i really do not konw why? can you give me any suggestion? thanks

Description

When a machine is equipped with more than one GPU, the current code sets device_ids parameter of torch.nn.DataParallel() method to [0, 1, 2].

        # current code:
        #  https://github.com/Tramac/Fast-SCNN-pytorch/blob/master/train.py#L90
        if torch.cuda.device_count() > 1:
            self.model = torch.nn.DataParallel(self.model, device_ids=[0, 1, 2])
        self.model.to(args.device)

Possible solution

The following code change will make it work for machines with 2, 4, .. GPUs.

        if torch.cuda.device_count() > 1:
            device_ids = list(range(torch.cuda.device_count()))
            self.model = torch.nn.DataParallel(self.model, device_ids=device_ids)
        self.model.to(args.device)