Aux_loss is contradict to the core insight of the paper. Concretely, the insight of the paper is to design an one-to-one label asssignment method, however, aux_loss is an one-to-many assignment method. So the proposed method can also be realized by using ATSS with an proper one-to-one label assignment (e.g., POTO in this paper). No correponding results varify which loss is more import to the performance.

I have done Get Started successfully, but it happened -bash: pods_train: command not found

Hi, when i try to run the command 'pods_test' or 'pods_train', i got the error as follow:

Traceback (most recent call last):
File "xxx/cvpods/tools/test_net.py", line 192, in
from config import config # isort:skip # noqa: E402

Any suggestion to fix this?
many thanks

I am very interested in your research direction. Could you please send me a copy of your training code? I promise it will never be used for commercial purposes.

If the gt_bbox is to small, no shift_point in each fpn_level is in the range of spatial prior, so no correponding regression target for this gt_bbox. How to set the regression target in this situation?
Looking forward to your reply, thank you!

how to train deFCN with 2 gpu？

Hi! Thanks for sharing your great work here.
I have a small question though. The 3DMF in the paper (fig 2) is calculated from the regression branch features, and used on the classification branch results. May I ask why this design? What about calculating it directly from the classification branch, or from a feature fusion between the branches?

您好！
由于cvpods封装程度较高，有些地方没有理解。
请问DeFCN训练时使用的是CrowdHuman中的Visible box还是Full box？
配置文件中num_classes=1，那么类别中的mask是被去除还是转化为person呢？
box的格式是表示xmin,ymin,xmax,ymax两个点的坐标还是中心点坐标与长宽呢？

请问代码里包含3DMF实现的部分吗？

ImportError: cannot import name 'PSRoIAlign'

我的环境： pytorch1.3.2，torchvision0.4.2
请问是否是环境造成的错误?
请问源代码使用的具体环境是什么呢？

完整错误:Traceback (most recent call last):
File "/home/cjj/cvpods/tools/train_net.py", line 28, in
from cvpods.engine import DefaultTrainer, default_argument_parser, default_setup, hooks, launch
File "/home/cjj/cvpods/cvpods/engine/init.py", line 3, in
from .hooks import *
File "/home/cjj/cvpods/cvpods/engine/hooks.py", line 16, in
from cvpods.modeling.nn_utils.precise_bn import get_bn_modules, update_bn_stats
File "/home/cjj/cvpods/cvpods/modeling/init.py", line 8, in
from .meta_arch import GeneralizedRCNN, PanopticFPN, ProposalNetwork, RetinaNet, SemanticSegmentor
File "/home/cjj/cvpods/cvpods/modeling/meta_arch/init.py", line 15, in
from .pointrend import CoarseMaskHead, PointRendROIHeads, PointRendSemSegHead, StandardPointHead
File "/home/cjj/cvpods/cvpods/modeling/meta_arch/pointrend.py", line 15, in
from cvpods.modeling.roi_heads import StandardROIHeads, select_foreground_proposals
File "/home/cjj/cvpods/cvpods/modeling/roi_heads/init.py", line 2, in
from .roi_heads import Res5ROIHeads, ROIHeads, StandardROIHeads, select_foreground_proposals
File "/home/cjj/cvpods/cvpods/modeling/roi_heads/roi_heads.py", line 17, in
from ..poolers import ROIPooler
File "/home/cjj/cvpods/cvpods/modeling/poolers.py", line 8, in
from torchvision.ops import PSRoIAlign, PSRoIPool, RoIPool
ImportError: cannot import name 'PSRoIAlign'

Can you tell me about the FPS of your detetor ?

@zengarden hi thanks for opensourcing the code base , i had one query
can we modify the current architecture to perform single bounding box detection wtih mulit label classification ? if so what is the modifications which has to be done
eg:

Thanks in advance

NMS post-processing is not used in POTO. The code:

It quite easy appears "nan" in the box_delta when not using center sampling.
Is it an normal state? or something what I miss? Thank for your great work

The filtered feature after 3DMF is added by the original one to form the final output:

It is a little contradictory to the purpose of the 3DMF which is used to filter the "unnecessary" point in the feature. If the original feature is used to form the final output, the output will not be a sparse feature map, and it may bring some adverse effects to the results without NMS processing.

I wonder how to use this to demo the inference results of owned video, thanks.

In the function apply_deltas(/cvpods/modeling/box_regression.py), there is a assertion at the first of this function. I think there may be some considerations for this assertion, and maybe sometimes, the deltas will be infinite. So could you share some experiences when debuging the code here?

There are only 2 gpus free in my machine now, so I tried to limit the gpu index and numbers as follow, and it raised such error:
(open-mmlab) liangtian@node001:~/project/DeFCN/playground/detection/coco/poto.res50.fpn.coco.800size.3x_ms.3dmf$ CUDA_VISIBLE_DEVICES=4,5 pods_train --num-gpus 2
Traceback (most recent call last):
File "/mnt/xfs1/home/liangtian/project/cvpods/tools/train_net.py", line 27, in
from cvpods.checkpoint import DetectionCheckpointer
File "/mnt/xfs1/home/liangtian/project/cvpods/cvpods/init.py", line 3, in
from .utils import setup_environment
File "/mnt/xfs1/home/liangtian/project/cvpods/cvpods/utils/init.py", line 27, in
from .visualizer import ColorMode, VideoVisualizer, VisImage, Visualizer, colormap, random_color
File "/mnt/xfs1/home/liangtian/project/cvpods/cvpods/utils/visualizer/init.py", line 5, in
from .video_visualizer import *
File "/mnt/xfs1/home/liangtian/project/cvpods/cvpods/utils/visualizer/video_visualizer.py", line 6, in
from .visualizer import ColorMode, Visualizer, _create_text_labels, _PanopticPrediction
File "/mnt/xfs1/home/liangtian/project/cvpods/cvpods/utils/visualizer/visualizer.py", line 17, in
from cvpods.structures import BitMasks, Boxes, BoxMode, Keypoints, PolygonMasks, RotatedBoxes
File "/mnt/xfs1/home/liangtian/project/cvpods/cvpods/structures/init.py", line 2, in
from .boxes import Boxes, BoxMode, pairwise_ioa, pairwise_iou
File "/mnt/xfs1/home/liangtian/project/cvpods/cvpods/structures/boxes.py", line 11, in
from cvpods.layers import cat
File "/mnt/xfs1/home/liangtian/project/cvpods/cvpods/layers/init.py", line 4, in
from .deform_conv import DeformConv, ModulatedDeformConv
File "/mnt/xfs1/home/liangtian/project/cvpods/cvpods/layers/deform_conv.py", line 11, in
from cvpods import _C
ImportError: /mnt/xfs1/home/liangtian/project/cvpods/cvpods/_C.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN6cvpods26psroi_pooling_forward_cudaERN2at6TensorES2_S2_iifii

Hello,
Thanks for your excellent work.
I just install the cvpods with (pytorch=1.4.0, cudatoolkit=10.1, torchvision=0.5.0), and can run pods_train with the POTO's config.py.
But when I download the weight file "poto.res50.fpn.coco.800size.3x_ms.3dmf.aux.cf438670.pth" and run pods_test, it raise the error like the following:

RuntimeError: version_ <= kMaxSupportedFileFormatVersion INTERNAL ASSERT FAILED at /opt/conda/conda-bld/pytorch_1579022034529/work/caffe2/serialize/inline_container.cc:132, please report a bug to PyTorch. Attempted to read a PyTorch file with version 3, but the maximum supported version for reading is 2. Your PyTorch installation may be too old. (init at /opt/conda/conda-bld/pytorch_1579022034529/work/caffe2/seri
alize/inline_container.cc:132)

I google this and find that maybe the pytorch version should be larger than 1.6.0, like this issue

Hi there,

I am facing a strange situation. I did an inference at step no. 1.000.000 out of 2.000.000 on trained with the same config as poto.res50.fpn.coco.800size.3x_ms.3dmf_wo_gn.aux. The problem is that I cannot see any difference in inference results when I do it again at step 1.500.000. The results of the inference are the same.

Do you know why this might happen?

Thank you.

I want to export the model of DeFCN to onnx to better see the details of this model,so i try to export with add some code ,but get error.
DeFCN/playground/detection/coco/poto.res50.fpn.coco.800size.3x_ms.3dmf_wo_gn.aux/net.py
i modify poto.res50.fpn.coco.800size.3x_ms.3dmf_wo_gn.aux as poto_res50_fpn_coco_800size_3x_ms_3dmf_wo_gn_aux for import,
add some code and run:

from cvpods.engine import default_argument_parser, default_setup
from DeFCN.playground.detection.coco.poto_res50_fpn_coco_800size_3x_ms_3dmf_wo_gn_aux.config import config
args = default_argument_parser().parse_args()
print("Command Line Args:", args)
config.link_log()
print("soft link to {}".format(config.OUTPUT_DIR))
cfg,logger=default_setup(config,args)
model=build_model(cfg)

model.eval()
input=torch.randn((1,3,48,48))
input=torch.rand(1)
output=model(input)
print(output.shape)
input_names=['input']
output_names=['output']
torch.onnx.export(model,input,'1.onnx',input_names=input_names,output_names=output_names,verbose=True,opset_version=11)

but get error as follows:

[09/22 11:36:18 c2.utils.env.env]: Using a generated random seed 18403068
Traceback (most recent call last):
File "/media/liang/linux_data/github_download/cvpods-master/DeFCN/playground/detection/coco/poto_res50_fpn_coco_800size_3x_ms_3dmf_wo_gn_aux/net.py", line 66, in
output=model(input)
File "/home/liang/miniconda3/envs/mmdet/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/media/liang/linux_data/github_download/cvpods-master/DeFCN/playground/detection/coco/poto_res50_fpn_coco_800size_3x_ms_3dmf_wo_gn_aux/fcos.py", line 117, in forward
images = self.preprocess_image(batched_inputs)
File "/media/liang/linux_data/github_download/cvpods-master/DeFCN/playground/detection/coco/poto_res50_fpn_coco_800size_3x_ms_3dmf_wo_gn_aux/fcos.py", line 546, in preprocess_image
images = [x["image"].to(self.device) for x in batched_inputs]
File "/media/liang/linux_data/github_download/cvpods-master/DeFCN/playground/detection/coco/poto_res50_fpn_coco_800size_3x_ms_3dmf_wo_gn_aux/fcos.py", line 546, in
images = [x["image"].to(self.device) for x in batched_inputs]
IndexError: too many indices for tensor of dimension 0

Process finished with exit code 1