/home/dl/qiqi/github/vision/torchvision/csrc/ROIAlign.h:80:24: error: ‘torch::autograd::AutogradContext’ has not been declared
using torch::autograd::AutogradContext;
^
/home/dl/qiqi/github/vision/torchvision/csrc/ROIAlign.h:84:58: error: expected template-name before ‘<’ token
class ROIAlignFunction : public torch::autograd::Function {
^
/home/dl/qiqi/github/vision/torchvision/csrc/ROIAlign.h:84:58: error: expected ‘{’ before ‘<’ token
/home/dl/qiqi/github/vision/torchvision/csrc/ROIAlign.h:84:58: error: expected unqualified-id before ‘<’ token
error: command 'gcc' failed with exit status 1
How can i slove it?

Sorry to bother you,i have a question about the code in default.py.

I don't understand the this sentence in default.py: the string must match a function that is...,
and how does the network use the registry to import backbone model and RPNhead model also makes me confused.

Could you please help me ,thanks a lot!

hi:
When I ran train_net.py, I got the error(ImportError: cannot import name '_C') . But using the terminal command, it works well. What should I do to fix the problem. Thanks a lot!

Traceback (most recent call last):
File "/disk6/spz/codes/RRPN_pytorch-master/tools/train_net.py", line 18, in
from maskrcnn_benchmark.engine.inference import inference
File "/disk6/spz/codes/RRPN_pytorch-master/maskrcnn_benchmark/engine/inference.py", line 10, in
from maskrcnn_benchmark.data.datasets.evaluation import evaluate
File "/disk6/spz/codes/RRPN_pytorch-master/maskrcnn_benchmark/data/datasets/evaluation/init.py", line 3, in
from .coco import coco_evaluation
File "/disk6/spz/codes/RRPN_pytorch-master/maskrcnn_benchmark/data/datasets/evaluation/coco/init.py", line 1, in
from .coco_eval import do_coco_evaluation
File "/disk6/spz/codes/RRPN_pytorch-master/maskrcnn_benchmark/data/datasets/evaluation/coco/coco_eval.py", line 10, in
from maskrcnn_benchmark.structures.boxlist_ops import boxlist_iou
File "/disk6/spz/codes/RRPN_pytorch-master/maskrcnn_benchmark/structures/boxlist_ops.py", line 6, in
from maskrcnn_benchmark.layers import nms as _box_nms
File "/disk6/spz/codes/RRPN_pytorch-master/maskrcnn_benchmark/layers/init.py", line 8, in
from .nms import nms
File "/disk6/spz/codes/RRPN_pytorch-master/maskrcnn_benchmark/layers/nms.py", line 3, in
from maskrcnn_benchmark import _C
ImportError: cannot import name '_C'

when i run the training command in terminal of ubuntu, it starts training successfuly. But when i run the same file through pycharm it gives me this error. i AM ALMOST GETTING MAD TRYING TO SOLVE THIS WITHOUT ANY SUCCESS. what could be the problem ? do you have any idea ? i have to use pycharm because i wish to debug the code.

run RRPN_demo.py occurs

from demo.predictor import ICDARDemo, RRPNDemo
: cannot import name 'ICDARDemo'

I find three classes in demo.predictor.I can import COCODemo,but can't import ICDARDemo, RRPNDemo.How to solve it?

many thanks

FPN默认的anchor size不是32512么，如果是232会不会太小了，难以检测比较大的物体？向大佬请教。

Hello,

In your paper the Algorithm 2 states that you do max pooling over a single region fro an inclined proposal to a point in the RROI. In the original caffe repo is implemented as max pooling.
https://github.com/mjq11302010044/RRPN/blob/8df1627e74f79db8be0467e08e3bdded63570fb5/caffe-fast-rcnn/src/caffe/layers/rotate_roi_pooling_layer.cu#L82

However when I took a look at the code in this repo, it seems like you do bilinear interpolation of 4 vertices for each subregions. Is there any other reason? My one guess is to make it comparable with the maskrcnn benchmark. Or did I miss some details?

https://github.com/mjq11302010044/RRPN_pytorch/blob/master/maskrcnn_benchmark/csrc/cuda/RROIAlign_cuda.cu#L133

Best,

Ahyun

I just want to test the algorithm in my own data set, so could you share the weight of the model.

Can you share the pretrained model, it will be useful to help me debug the code. thanks!

i have this problem, when i try the train demo. could you give me some advice.thanks.

请问rotations中 overlaps中的代码有说明吗?

I found you didn't write some description about the pre-training weight. If we train it on self-build dataset, don't need the pre-training weight?

when i use :
CUDA_VISIBLE_DEVICES="3,4" python -m torch.distributed.launch --nproc_per_node 2 train_net.py
i meet some trouble, like this:
RuntimeError: CUDA error: an illegal memory access was encountered
terminate called without an active exception
How can I solve it，thanks a lot!

感谢您分享mask rcnn的rrpn模型，接触深度学习时间不太长，所以有些地方没有看明白
1.我理解的是我需要把我的数据组织成VOC数据集的格式。我手里有自己用labelme制作的json格式的数据集，我把他转化为xml格式然后组织成
VOC
|_ JPEGImages
| |_ .jpg
| |_ ...
| |_ .jpg
|_ Annotations
| |_ .json
| |_ ...
| |_ .xml
| |_ ...
| |_ .xml
|_ VOCdevkit
这样可以吗（用labelme制作的时候是按物体的形状进行的标注，并不是画的矩形）
如果不可以的话我可以用rolabeiimg直接生成的xml文件作为输入吗
2.VOCdevkit这个文件夹里什么都不用放吗

具体情况如下：
loss: nan (nan) loss_classifier: 0.2950 (1.0475) loss_box_reg: 0.0002 (0.0087) loss_objectness: nan (nan) loss_rpn_box_reg: nan (nan)

求各位大佬指点迷津，感激不尽！
@mjq11302010044

下面是我使用的bbox格式：

boxes.append([x_ctr, y_ctr, width, height, angle, words])
分别为x中心坐标，y中心坐标，宽、高，角度，类别，不知是否是这种坐标形式

im_info格式：

        im_info = {
            'gt_classes': gt_classes,
            'max_classes': max_classes,
            'image': im_path,
            'boxes': gt_boxes,
            'flipped': False,
            'gt_overlaps': overlaps,
            'seg_areas': seg_areas,
            'height': im.shape[0],
            'width': im.shape[1],
            'max_overlaps': max_overlaps,
            'rotated': True
        }

还有DOTA的类别：

    cls_list = \
        {
            'background': 0,
            'roundabout': 1,
            'tennis-court': 2,
            'swimming-pool': 3,
            'storage-tank': 4,
            'soccer-ball-field': 5,
            'small-vehicle': 6,
            'ship': 7,
            'plane': 8,
            'large-vehicle': 9,
            'helicopter': 10,
            'harbor': 11,
            'ground-track-field': 12,
            'bridge': 13,
            'basketball-court': 14,
            'baseball-diamond': 15,
            'helipad': 16,
            'airport': 17,
            'container-crane': 18
        }

DATASET = {
    'IC13':get_ICDAR2013,
    'IC15':get_ICDAR2015_RRC_PICK_TRAIN,
    'IC17mlt':get_ICDAR2017_mlt,
    'LSVT':get_ICDAR_LSVT_full,
    'ArT':get_ICDAR_ArT,
    'ReCTs':get_ICDAR_ReCTs_full,
    'DOTA':get_DOTA,   # clw modify
}


_DEBUG = False
class RotationDataset(torch.utils.data.Dataset):
    CLASSES = (
        "__background__ ",      #"background",
        "roundabout",
        "tennis-court",
        "swimming-pool",
        "storage-tank",
        "soccer-ball-field",
        "small-vehicle",
        "ship",
        "plane",
        "large-vehicle",
        "helicopter",
        "harbor",
        "ground-track-field",
        "bridge",
        "basketball-court",
        "baseball-diamond",
        "helipad",
        "airport",
        "container-crane"
    )

请问一下如何使用多GPU训练，我在使用maskrcnn-benchmark中多GPU训练的方法时，遇到了一些问题，请问如何解决？谢谢！

can you share your model config yaml to us so that we can train our stable model ?
i use your code to train on ic15 and mlt17 , but the result is not good ,so may i ask you to
share your training yaml or your model to us :) , and another question is how long have you train your model ? my email address is [email protected], hope you can help us ,thank you !

Ｈello,The IC3 label for the competition is a horizontal box with only the upper left and lower right coordinates.The code I see in get_ICDAR2013(mode, dataset_dir) is to calculate the center point, length and width, and rotation angle from the four vertices of the rotation box.Did you relabel your data with rotation bounding box.
Thank you very much.

when i test the demo.py, it put out this problem. could you give some advice.

It seems RROI Alignment didn't use choose dynamic sampling points according to the size of bin but just one center point within each bin. Am I correct or did I misunderstand the code?

This is really a good work, but i still have some question?
First, can you give me some suggestions about use this model with FPN, i just replace the config with:
MODEL: META_ARCHITECTURE: "RRPN" WEIGHT: "catalog://ImageNetPretrained/MSRA/R-50" BACKBONE: CONV_BODY: "R-50-FPN" OUT_CHANNELS: 256 RPN: USE_FPN: True PRE_NMS_TOP_N_TRAIN: 2000 PRE_NMS_TOP_N_TEST: 1000 POST_NMS_TOP_N_TEST: 1000 FPN_POST_NMS_TOP_N_TEST: 2000 RPN_HEAD: "SingleConvRRPNHead" RRPN: ANCHOR_STRIDE: (4, 8, 16, 32, 64) ANCHOR_SIZES: (32, 64, 128, 256, 512) ROI_HEADS: USE_FPN: True ROI_BOX_HEAD: NUM_CLASSES: 11 POOLER_RESOLUTION: 7 POOLER_SCALES: (0.25, 0.125, 0.0625, 0.03125) POOLER_SAMPLING_RATIO: 2 FEATURE_EXTRACTOR: "FPN2MLPFeatureExtractor" PREDICTOR: "FPNPredictor" DATASETS: TRAIN: ("DATA_train", ) SOLVER: BASE_LR: 0.002 WEIGHT_DECAY: 0.0001 STEPS: (40000, 50000) MAX_ITER: 60000 IMS_PER_BATCH: 1 OUTPUT_DIR: 'models/FPN/'
But the result seems to be bad compared with model without FPN

The second question is the assign about positive R-anchor,
In the paper, you say:
positive R-anchors feature the following: (i) the highest IoU overlap or an IoU larger than 0.7 with respect to the ground truth, and (ii) an intersection angle with respect to the ground truth of less than π/12.

While in this code, it seems that just follow the first role about IOU and I cannot find the role about angle, may there is something I missed

Thanks

Hello, what can I do if I want to use the mask branch. Can you give me some advice? If I want to use the mask branch, how should I change the format of the input dataset?

How can I solve this problem? I completely follow

install.md.python RRPN_Demo.py
Traceback (most recent call last):
File "RRPN_Demo.py", line 14, in
cfg.merge_from_file(config_file)
File "/home/ltr/anaconda3/envs/rrpn_pytorch/lib/python3.7/site-packages/yacs/config.py", line 211, in merge_from_file
with open(cfg_filename, "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: './configs/rrpn/e2e_rrpn_R_50_C4_1x_ICDAR13_15_17_trial_again_test.yaml'
`

I followed the option1 of install.md file step by step. But now this problem has arisen, I don't know how to solve it, can you help me?

The following is Traceback

Traceback (most recent call last):
  File "tools/train_net.py", line 175, in <module>
    main()
  File "tools/train_net.py", line 168, in main
    model = train(cfg, args.local_rank, args.distributed)
  File "tools/train_net.py", line 30, in train
    model = build_detection_model(cfg)
  File "/home/y182235010/github/maskrcnn-benchmark/maskrcnn_benchmark/modeling/detector/detectors.py", line 9, in build_detection_model
    meta_arch = _DETECTION_META_ARCHITECTURES[cfg.MODEL.META_ARCHITECTURE]
KeyError: 'RRPN'
(rrpn_pytorch) y182235010@node01:~/deeplearning/RRPN_ROOT$ 

I can not find the file "irrationally_interface.py".

i trained the model on my own dataset, but when i use the demo to infer the testing data, it occurs out of memory error, after i replace the testing data with training data, the error remains. my gpu is 2080Ti with 11GB memory.

my config as blow:

INPUT:
    MIN_SIZE_TEST: 800 
    MAX_SIZE_TEST: 1333
MODEL:
  META_ARCHITECTURE: "RRPN"
  WEIGHT: "./models/SHIP/model_0150000.pth"
  RPN:
    PRE_NMS_TOP_N_TEST: 3000
    POST_NMS_TOP_N_TEST: 1000
    RPN_HEAD: "SingleConvRRPNHead"

  ROI_BOX_HEAD:
    NUM_CLASSES: 2
  ROI_HEADS:
    BATCH_SIZE_PER_IMAGE: 256 
    NMS: 0.3 
    DETECTIONS_PER_IMG: 300 
  DEVICE: "cuda:2"
TEST:
  IMS_PER_BATCH: 1
DATALOADER:
  SIZE_DIVISIBILITY: 1

i have tried tuning the INPUT SIZE, but it seems not working.

the error message as blow:

***** META INFO *******
config_file: ./configs/rrpn/e2e_rrpn_R_50_C4_1x_SHIP_test.yaml
result_dir: results/e2e_rrpn_R_50_C4_1x_SHIP_test/model_0150000
image_dir: /home/xxx/data/ship/test/img
weights: ./models/SHIP/model_0150000.pth

---

image: /home/xxx/data/ship/test/img/P0440.png
torch.Size([3, 1000, 1133])
<maskrcnn_benchmark.structures.image_list.ImageList object at 0x7fab5ea6e0f0>
cuda:2
THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1544176307774/work/aten/src/THC/THCGeneral.cpp line=405 error=11 : invalid argument
out of memory
out of memory
out of memory
out of memory
out of memory
out of memory
Traceback (most recent call last):
  File "demo/RRPN_Demo.py", line 81, in <module>
    predictions, bounding_boxes = coco_demo.run_on_opencv_image(img)
  File "/home/xxx/project/github/RRPN_pytorch/demo/predictor.py", line 710, in run_on_opencv_image
    predictions = self.compute_prediction(image)
  File "/home/xxx/project/github/RRPN_pytorch/demo/predictor.py", line 744, in compute_prediction
    predictions = self.model(image_list)
  File "/home/xxx/.conda/envs/rrpn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
    result = self.forward(input, **kwargs)
  File "/home/xxx/project/github/maskrcnn-benchmark/maskrcnn_benchmark/modeling/detector/generalized_rrpn_rcnn.py", line 61, in forward
    x, result, detector_losses = self.roi_heads(features, proposals, targets)
  File "/home/xxx/.conda/envs/rrpn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
    result = self.forward(input, kwargs)
  File "/home/xxx/project/github/maskrcnn-benchmark/maskrcnn_benchmark/modeling/roi_heads/rroi_heads.py", line 27, in forward
    x, detections, loss_box = self.box(features, proposals, targets)
  File "/home/xxx/.conda/envs/rrpn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
    result = self.forward(*input, kwargs)
  File "/home/xxx/project/github/maskrcnn-benchmark/maskrcnn_benchmark/modeling/roi_heads/rbox_head/box_head.py", line 55, in forward
    x = self.feature_extractor(features, recur_proposals)
  File "/home/xxx/.conda/envs/rrpn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
    result = self.forward(input, **kwargs)
  File "/home/xxx/project/github/maskrcnn-benchmark/maskrcnn_benchmark/modeling/roi_heads/rbox_head/roi_box_feature_extractors.py", line 42, in forward
    x = self.pooler(x, proposals)
  File "/home/xxx/.conda/envs/rrpn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
    result = self.forward(input, kwargs)
  File "/home/xxx/project/github/maskrcnn-benchmark/maskrcnn_benchmark/modeling/poolers.py", line 103, in forward
    return self.poolers0
  File "/home/xxx/.conda/envs/rrpn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
    result = self.forward(*input, kwargs)
  File "/home/xxx/project/github/maskrcnn-benchmark/maskrcnn_benchmark/layers/rroi_align.py", line 71, in forward
    input, rois_reverse, self.output_size, self.spatial_scale
  File "/home/xxx/project/github/maskrcnn-benchmark/maskrcnn_benchmark/layers/rroi_align.py", line 19, in forward
    input, roi, spatial_scale, output_size[0], output_size[1]
RuntimeError: cuda runtime error (2) : out of memory at /home/xxx/project/github/maskrcnn-benchmark/maskrcnn_benchmark/csrc/cuda/RROIAlign_cuda.cu:236

If I want to train it on my dataset,the dataset has the same form like voc dataset,what I should modify the code?can you give me the accurate position?

i get this error when i run command build/lib/rotation/*.so ./rotation in the INSTALL.md. KiINDLY TELL ME HOW TO SOLVE IT

Can you give some suggestions about how to use your code with multi-gpu in train phase?
Thanks a lot!

is it must test in cuda evironment?

THCudaCheck FAIL file=/build/python-pytorch/src/pytorch-1.1.0-opt-cuda/aten/src/THC/generic/THCTensorMath.cu line=16 error=700 : an illegal memory access was encountered
Traceback (most recent call last):
File "", line 89, in train
loss.backward()
File "/usr/lib/python3.7/site-packages/torch/tensor.py", line 107, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/usr/lib/python3.7/site-packages/torch/autograd/init.py", line 93, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 205, in
main()
File "", line 188, in main
train(model, train_loader, criterion, scheduler, optimizer, epoch, logger)
File "", line 93, in train
loss.backward()
File "/usr/lib/python3.7/site-packages/torch/tensor.py", line 107, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/usr/lib/python3.7/site-packages/torch/autograd/init.py", line 87, in backward
grad_tensors = _make_grads(tensors, grad_tensors)
File "/usr/lib/python3.7/site-packages/torch/autograd/init.py", line 29, in _make_grads
new_grads.append(torch.ones_like(out))
RuntimeError: cuda runtime error (700) : an illegal memory access was encountered at /build/python-pytorch/src/pytorch-1.1.0-opt-cuda/aten/src/THC/generic/THCTensorMath.cu:16

Hi~ This is really a good work.
I notice that there are five kinds of files under rotation folder, i.e., rbbox overlap, rotate circle nms, rotate cpu nms, rotate gpu nms, and rotate polygon nms. So what's the difference between these modules?

执行测试demo/RRPN_Demo.py
错误提示：
No module named rotate_polygon_nms

my pytorch vision is pytorch-nightly==1.0.0 and cuda version is 9.0.
The torchvision code is cloned from https://github.com/pytorch/vision.git, and the branch is master.
I think the reason is that the pytorch vision is incompatible with the pytorch vision.
I have tried different torchvision installed from conda and the code have some problems to run.
So please tell me the correct the version of pytorch and torchvision. Thank you!

请问您提到的在maskrcnn-benchmark已经有了对应的实现，但是我没有在maskrcnn-benchmark找到，还请麻烦给个具体的链接，谢谢啦。

When I compile the vision module, I meet the following issue, which seems to be not related to the RRoIAlign module.

Is it the issue of the torchvision version？
Can I install the torchvision by command “conda install torchvision=0.2.2", like mask-rcnn-benchmark？ I worry about some APIs in the source code that invoke the RRoIAlign module will be missed out if I use the above command.

Many thanks.

I want to train the RRPN in my dataset and I changed the input size to 800×800 pixels in my GTX 1080 TI GPU with 11G memory, but it occured the erreor: out of memory. I want to know what caused this. If I want to train the network with the input of 1024×1024 pixels, how much memory would be required? Can you evaluate it or did you have tried it? @mjq11302010044

Sorry to bother you ,i have met a question when i train the rrpn model.

as shown in the figure,it seems that there occurs bug in box_regression.So i print the value of intermediate process variables. I'm surprised to find that the ouput of resnet_layer2 is nan .I try to lower the base lr,but it is useless.

Could you please give me some advice?Thanks a lot. @mjq11302010044

• [configure ]
  pytorch 1.0.1
  cuda 8.0
  linux_x86_64

I use this code to train my own dataset, I used to use the facebook maskrcnn-benchmark and the speed is about 20 ours when traning 180000 iteration, but in this code , i train the rrpn with

`MODEL:
META_ARCHITECTURE: "RRPN"
WEIGHT: "catalog://ImageNetPretrained/MSRA/R-50"
RPN:
PRE_NMS_TOP_N_TEST: 6000
POST_NMS_TOP_N_TEST: 1000
RPN_HEAD: "SingleConvRRPNHead"

ROI_BOX_HEAD:
NUM_CLASSES: 11
ROI_HEADS:
BATCH_SIZE_PER_IMAGE: 256

DATASETS:
TRAIN: ("RRPN_train", )
SOLVER:
BASE_LR: 0.005
WEIGHT_DECAY: 0.0001
STEPS: (50000, 100000)
MAX_ITER: 150000
IMS_PER_BATCH: 2

OUTPUT_DIR: './models/XXX`

It shows that it use smore than 2 days to finish the training phase

Is there something i ignored in the code or can U give me your training speed

Thanks a lot

Can you tell me what is the fps of this model? and do you think your implementation of nms fast?

could you update the pretrained weight. thanks