potterhsu / easy-faster-rcnn.pytorch Goto Github PK

Hi, I noticed that the smooth L1 regression loss for the fast r cnn goes to 0 when there are no proposals that have an iou of > 0.5. Is this deliberately done? The fg_indices are empty when there are no matches

For the WarmUpMultiStepLR, if we maintain the default factor and warm up steps, the learning rate will tend to be 0 after several steps since the lr is mulplicated by factor for number of steps times.

I have some problems when compiling under windows,I intercepted the wrong part，Does compiling under windows need to modify the system files?My configuration is cuda10.1+pytorch1.2+VS2015
E:/anaconda/envs/pytorch/lib/site-packages/torch/include\THC/generic/THCTensor.h(128): warning C4190: “THCudaBFloat16Tensor_get4d”有指定的 C 链接，但返回了与 C 不兼容的 UDT“c10::BFloat16”
E:/anaconda/envs/pytorch/lib/site-packages/torch/include\c10/util/BFloat16.h(48): note: 参见“c10::BFloat16”的声明
E:/anaconda/envs/pytorch/lib/site-packages/torch/include\THC/generic/THCTensorMathReduce.h(16): warning C4190: “THCudaHalfTensor_minall”有指定的 C 链接，但返回了与 C 不兼容的 UDT“c10::Half”
E:/anaconda/envs/pytorch/lib/site-packages/torch/include\c10/util/Half.h(324): note: 参见“c10::Half”的声明
E:/anaconda/envs/pytorch/lib/site-packages/torch/include\THC/generic/THCTensorMathReduce.h(17): warning C4190: “THCudaHalfTensor_maxall”有指定的 C 链接，但返回了与 C 不兼容的 UDT“c10::Half”
E:/anaconda/envs/pytorch/lib/site-packages/torch/include\c10/util/Half.h(324): note: 参见“c10::Half”的声明
E:/anaconda/envs/pytorch/lib/site-packages/torch/include\THC/generic/THCTensorMathReduce.h(38): warning C4190: “THCudaHalfTensor_medianall”有指定的 C 链接，但返回了与 C 不兼容的 UDT“c10::Half”
E:/anaconda/envs/pytorch/lib/site-packages/torch/include\c10/util/Half.h(324): note: 参见“c10::Half”的声明
error: command 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin\nvcc.exe' failed with exit status 2

@potterhsu When i train with PASCAL VOC 2007 ,learning rate is always 0 like this.

i run code 'python support/setup.py develop' and error. how i can solve it?

1 error detected in the compilation of "C:/Users/jizef/AppData/Local/Temp/tmpxft_0000a6d8_00000000-10_nms.cpp1.ii".
error: command 'D:\solfware\CUDA-install\CUDA Samples\v8.0\bin\nvcc.exe' failed with exit status 1

hi @potterhsu
iam trying to train using faster R-CNN model on my custom dataset. Basically my custom data just include 2 classes, and the data had labelled too, based on VOC format. On the part of the installation guidance, specially at Main folder that contain .txt, iam not sure to proceed. first this the format of csv convert from XML files.

is it right just convert the file into .txt and proceed it?

I get a lot of error while compiling...
can anyone help?

I have torch = 1.0
python 3.6
gcc = 9.3

still not working

Hi, potterhsu,

Thank you very much for your code, I run it well but I have a question here about the detach of proposal boxes, why it should be detached, does it mean in total loss items there are always some items invalid for the two parts of network split by proposal_boxes when updating parameters each step?

Hi, I wanted to use light-weight base models from torchvision like mobilenet, squeezenet and densenet. Any tips on how to edit the code?

Thanks for your clean and useful code implementation.I would like to consult you why 'class_bboxes' on the NMS are not arranged in descending order of 'class_probs' in 'model.py' when validating or testing. The code is as follows:
kept_indices = nms(class_bboxes, class_probs, threshold)

Hi @potterhsu

I am trying to train a Faster R-CNN(Resnet101) model on my custom dataset. The dataset is in the same format as MS COCO, with 16 categories. However, I am not sure how to proceed. I am new to pytorch and don't have the slightest of ideas about where I can start from. I would really appreciate your help regarding this.

Thanks and Regards
Chaitanya Agrawal

Both training and also inference needs CUDA, the Nvidia framework for their GPU hardware. This could be mentioned in the README documentation upfront.

easy-faster-rcnn.pytorch/support/_C.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _Z21ROIAlign_forward_cudaRKN2at6TensorES2_fiii

I have this error when running test_nms.py
Any ideas why this is happening? I've tried searching for relevant issues online but they do not have this kind of error

Hi,

You have 2 ResNet101 .pth checkpoints (the last 2 linked under COCO2017 in the README). While I can run infer.py with my own images and get a result on the first of the two .pth files, simply changing the path to the second .pth (while keeping everything else the same), produces the below error.

Do you know what's going on here?

The default configuration seems to work just fine for the COCO2017 VGG-16 backbone as well. It's just the second of the two COCO2017 ResNet101 backbones that seems to be a problem.

Thanks.

Config:
ANCHOR_RATIOS = [(1, 2), (1, 1), (2, 1)]
ANCHOR_SIZES = [64, 128, 258, 512]
IMAGE_MAX_SIDE = 1333
IMAGE_MIN_SIDE = 800
POOLING_MODE = Mode.ALIGN
RPN_POST_NMS_TOP_N = 1000
RPN_PRE_NMS_TOP_N = 6000

Traceback (most recent call last):
File "infer.py", line 115, in
main()
File "infer.py", line 113, in main
_infer(path_to_input_image, path_to_output_image, path_to_checkpoint, dataset_name, backbone_name, prob_thresh)
File "infer.py", line 24, in _infer
model.load(path_to_checkpoint)
File "/home/dev/easy-faster-rcnn.pytorch/model.py", line 145, in load
self.load_state_dict(checkpoint['state_dict'])
File "/home/theaitherapist/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 719, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for Model:
size mismatch for rpn._objectness.weight: copying a param of torch.Size([24, 512, 1, 1]) from checkpoint, where the shape is torch.Size([30, 512, 1, 1]) in current model.
size mismatch for rpn._objectness.bias: copying a param of torch.Size([24]) from checkpoint, where the shape is torch.Size([30]) in current model.
size mismatch for rpn._transformer.weight: copying a param of torch.Size([48, 512, 1, 1]) from checkpoint, where the shape is torch.Size([60, 512, 1, 1]) in current model.
size mismatch for rpn._transformer.bias: copying a param of torch.Size([48]) from checkpoint, where the shape is torch.Size([60]) in current model.

Do you have plans to support PyTorch 1.0?

Hi guys,
Has any of you faced the issue of batch size mismatch when the input images have different sizes, or different number of bboxes/labels when the batch size is set to more than 1 ?

Hi,

Is this repository allow considering negative images as well for training? For example, while training a cat detector, one may have some images containing cats (+ve images) and some images do not containing cats (-ve images). I noticed that the training script only considers the positive images for training and discard others.

What would be the good way to add support for Negative Samples in training in this repository?

Best Regards,
Maaz

Hi,

I'm using the pretrained model on coco2017 with resnet-101 backbone for inference, but sometimes I come across a strange CUDA error:

Traceback (most recent call last): File "C:\Users\Simone\anaconda3\envs\phd\lib\threading.py", line 932, in _bootstrap_inner self.run() File "C:\Users\Simone\PycharmProjects\Smart_Mapping\online_operations\main_thread.py", line 29, in run out_obj_det_img, bboxes, obj_classes, obj_prob = obj_det(img_tensor=img, steps=self.tot_path_steps, File "c:\users\simone\pycharmprojects\smart_mapping\faster-rcnn\infer.py", line 65, in obj_det model.eval().forward(image_tensor.unsqueeze(dim=0).cuda()) File "c:\users\simone\pycharmprojects\smart_mapping\faster-rcnn\model.py", line 62, in forward proposal_classes, proposal_transformers = self.detection.forward(features, proposal_bboxes) File "c:\users\simone\pycharmprojects\smart_mapping\faster-rcnn\model.py", line 111, in forward proposal_classes = self._proposal_class(hidden) File "C:\Users\Simone\anaconda3\envs\phd\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "C:\Users\Simone\anaconda3\envs\phd\lib\site-packages\torch\nn\modules\linear.py", line 93, in forward return F.linear(input, self.weight, self.bias) File "C:\Users\Simone\anaconda3\envs\phd\lib\site-packages\torch\nn\functional.py", line 1690, in linear ret = torch.addmm(bias, input, weight.t()) RuntimeError: CUDA error: CUBLAS_STATUS_ALLOC_FAILED when calling cublasCreate(handle)

It's very strange because sometimes this happens and then casually disappears. It seems to be a problem with the number of classes (as I saw here), but I don't know how to solve it.

Have you ever come across that? Do you know if it can be a problem with my CUDA or a problem with the model?

Thanks.

number_class is wrong! coco2017 is actually not 92.

Is there any way to use NMS for inference in a system that has no GPU?

C:\WinPython\WPy64-3741\easy-faster-rcnn.pytorch-master\easy-faster-rcnn.pytorch-master>python support/setup.py develop
running develop
running egg_info
writing support.egg-info\PKG-INFO
writing dependency_links to support.egg-info\dependency_links.txt
writing top-level names to support.egg-info\top_level.txt
reading manifest file 'support.egg-info\SOURCES.txt'
writing manifest file 'support.egg-info\SOURCES.txt'
running build_ext
C:\WinPython\WPy64-3741\python-3.7.4.amd64\lib\site-packages\torch\utils\cpp_extension.py:209: UserWarning: Error checking compiler version for cl: [WinError 2] The system cannot find the file specified
warnings.warn('Error checking compiler version for {}: {}'.format(compiler, error))
copying build\lib.win-amd64-3.7\support_C.cp37-win_amd64.pyd -> support
Creating c:\winpython\wpy64-3741\python-3.7.4.amd64\lib\site-packages\support.egg-link (link to .)
Adding support 0.1 to easy-install.pth file

Installed c:\winpython\wpy64-3741\easy-faster-rcnn.pytorch-master\easy-faster-rcnn.pytorch-master
Processing dependencies for support==0.1
Finished processing dependencies for support==0.1

Hi potterhsu.
I'm studying deep learning detection with your code. It helps a lot. really thank you

I'm trying to train with custom data and to insert some 'spatial transform networks' things.
Then I'm curious what exactly methods in class BBox is working for. Especially 'calc_transformer' and 'apply_transformer'
Are they related to affine transform? or others?. Can i get some explains what they are?

I'm not sure this question is okay in 'Issues', but i don't know elsewhere to ask to you. Sorry for if i make some mistake.
I will wait your answer. Thank U

@staticmethod
def from_center_base(center_based_bboxes: Tensor) -> Tensor:
return torch.stack([
center_based_bboxes[..., 0] - center_based_bboxes[..., 2] / 2,
center_based_bboxes[..., 1] - center_based_bboxes[..., 3] / 2,
center_based_bboxes[..., 0] + center_based_bboxes[..., 2] / 2,
center_based_bboxes[..., 1] + center_based_bboxes[..., 3] / 2
], dim=-1)

@staticmethod
def calc_transformer(src_bboxes: Tensor, dst_bboxes: Tensor) -> Tensor:
    center_based_src_bboxes = BBox.to_center_base(src_bboxes)
    center_based_dst_bboxes = BBox.to_center_base(dst_bboxes)
    transformers = torch.stack([
        (center_based_dst_bboxes[..., 0] - center_based_src_bboxes[..., 0]) / center_based_src_bboxes[..., 2],
        (center_based_dst_bboxes[..., 1] - center_based_src_bboxes[..., 1]) / center_based_src_bboxes[..., 3],
        torch.log(center_based_dst_bboxes[..., 2] / center_based_src_bboxes[..., 2]),
        torch.log(center_based_dst_bboxes[..., 3] / center_based_src_bboxes[..., 3])
    ], dim=-1)
    return transformers

Hello! Excuse me, what's the problem with this situation? How to solve it! thank you

size mismatch for detection._proposal_class.weight: copying a param with shape torch.Size([21, 2048]) from checkpoint, the shape in current model is torch.Size([6, 2048]).
size mismatch for detection._proposal_class.bias: copying a param with shape torch.Size([21]) from checkpoint, the shape in current model is torch.Size([6]).
size mismatch for detection._proposal_transformer.weight: copying a param with shape torch.Size([84, 2048]) from checkpoint, the shape in current model is torch.Size([24, 2048]).
size mismatch for detection._proposal_transformer.bias: copying a param with shape torch.Size([84]) from checkpoint, the shape in current model is torch.Size([24]).

Hi,author
When I compile the program with code:
python test/nms/test_nms.py
I met a error: ImportError: /home/liushuai/STUDY/MyWorkSpace/object_detection_test/easy-faster-rcnn.pytorch-master/support/_C.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN3c105ErrorC1ENS_14SourceLocationERKSs

How can I solve it?
I hope you can give me some advice
Thank you

if i write python support/setup.py develop in ubunto app for window10
i have nvrtcGetProgramLogSize error
how can i solve it?

Thanks for your good implementation. I have run your code on my own dataset. And I want to know, do your code provide multiple images inferring?

g++: error: /home/hit/shux_workspace/codes/easy-faster-rcnn.pytorch/build/temp.linux-x86_64-3.6/home/hit/shux_workspace/codes/easy-faster-rcnn.pytorch/support/src/cuda/nms.o: No such file or directory
g++: error: /home/hit/shux_workspace/codes/easy-faster-rcnn.pytorch/build/temp.linux-x86_64-3.6/home/hit/shux_workspace/codes/easy-faster-rcnn.pytorch/support/src/cuda/ROIAlign_cuda.o: No such file or directory

What problem of this?

https://www.mediafire.com/file/ehhg3r2hxe8yb8j/faster_rcnn.tar.xz

1. fixed warning in pytorch
2. install_data.sh will prepare all package install and cpp build
3. python3 ./train.py --dataset voc2007 --backbone resnet101 --cuda True
4. python3 ./train.py --dataset voc2007 --backbone resnet101
   step 4 is for cpu - test ok

by Jesse Stone

By any chance, do you have the checkpoints for resnet50 or resnet18?
I'm using the inference part of the script because my GPU isn't strong enough for training it. It would be really helpful if you have it.

Thanks!

As VOC bounding boxes are 1 indexed, shouldn't it be substracted by 1 after loading them here?

Thank you for your clean and effective implementation.

In the source code of calc_transformer in bbox.py, I found that the implementation for computing the center offset of the bounding boxes is slightly different from the formula given in the original paper of the Fast R-CNN. In the paper, t_x = (x - x_a)/w_a, where x, x_a, w_a are the x coordinate of the center of the predicted box, anchor box and the width of the anchor box, respectively. But in the code, it is divided by the center_based_dst_bboxes[..., 2], which seems to be the width of the predicted boxes. This also contradicts the implementation in the apply_transformer function.

Could you please explain it a little bit?

hello
I have saved 'inputdata' to '/home/harin/harin/easy-faster-rcnn.pytorch-master/data/VOCdevkit/VOC2007/JPEGImages'. There are no errors. are you okay? Help me please,,

/home/harin/anaconda3/envs/pytorch_easy_faster_R_CNN/bin/python /home/harin/harin/easy-faster-rcnn.pytorch-master/infer.py
Arguments:
dataset = voc2007
backbone = resnet101
checkpoint = /home/harin/harin/easy-faster-rcnn.pytorch-master/outputs/checkpoints-20211103214119-voc2007-resnet101-5eb3026a/model-90000.pth
probability_threshold = 0.6
image_min_side = None
image_max_side = None
anchor_ratios = None
anchor_sizes = None
pooler_mode = None
rpn_pre_nms_top_n = None
rpn_post_nms_top_n = 1000
input = /home/harin/harin/easy-faster-rcnn.pytorch-master/data/VOCdevkit/VOC2007/JPEGImages
output = /home/harin/harin/easy-faster-rcnn.pytorch-master/outputs

Config:
ANCHOR_RATIOS = [(1, 2), (1, 1), (2, 1)]
ANCHOR_SIZES = [128, 256, 512]
IMAGE_MAX_SIDE = 1000.0
IMAGE_MIN_SIDE = 600.0
POOLER_MODE = Mode.ALIGN
RPN_POST_NMS_TOP_N = 1000
RPN_PRE_NMS_TOP_N = 6000

Traceback (most recent call last):
File "/home/harin/harin/easy-faster-rcnn.pytorch-master/infer.py", line 97, in
main()
File "/home/harin/harin/easy-faster-rcnn.pytorch-master/infer.py", line 95, in main
_infer(path_to_input_image, path_to_output_image, path_to_checkpoint, dataset_name, backbone_name, prob_thresh)
File "/home/harin/harin/easy-faster-rcnn.pytorch-master/infer.py", line 25, in _infer
image = transforms.Image.open(path_to_input_image)
File "/home/harin/anaconda3/envs/pytorch_easy_faster_R_CNN/lib/python3.6/site-packages/PIL/Image.py", line 2912, in open
fp = builtins.open(filename, "rb")
IsADirectoryError: [Errno 21] Is a directory: '/home/harin/harin/easy-faster-rcnn.pytorch-master/data/VOCdevkit/VOC2007/JPEGImages'

Process finished with exit code 1

add type-cast to fix the error as facebookresearch repo has done with this issue