unsky / fpn Goto Github PK

Feature Pyramid Networks for Object Detection

CMake 1.42% Makefile 0.31% Shell 0.46% HTML 0.10% CSS 0.12% Jupyter Notebook 47.33% C++ 38.36% Python 8.13% Cuda 3.19% MATLAB 0.46% C 0.13%

fpn feature-pyramid-network object-detection

fpn's Introduction

Feature Pyramid Network on caffe

This is the unoffical version Feature Pyramid Network for Feature Pyramid Networks for Object Detection https://arxiv.org/abs/1612.03144

results

FPN(resnet50)-end2end result is implemented without OHEM and train with pascal voc 2007 + 2012 test on 2007

merged rcnn

[email protected]	aeroplane	bicycle	bird	boat	bottle	bus	car	cat	chair	cow
0.788	0.8079	0.8036	0.8010	0.7293	0.6743	0.8680	0.8766	0.8967	0.6122	0.8646

diningtable	dog	horse	motorbike	person	pottedplant	sheep	sofa	train	tv
0.7330	0.8855	0.8760	0.8063	0.7999	0.5138	0.7905	0.7755	0.8637	0.7736

shared rcnn

[email protected]	aeroplane	bicycle	bird	boat	bottle	bus	car	cat	chair	cow
0.7833	0.8585	0.8001	0.7970	0.7174	0.6522	0.8668	0.8768	0.8929	0.5842	0.8658

diningtable	dog	horse	motorbike	person	pottedplant	sheep	sofa	train	tv
0.7022	0.8891	0.8680	0.7991	0.7944	0.5065	0.7896	0.7707	0.8697	0.7653

framework

megred rcnn framework

Network overview: link

shared rcnn

Network overview: link

the red and yellow are shared params

about the anchor size setting

In the paper the anchor setting is Ratios： [0.5,1,2],scales :[8,]

With the setting and P2~P6, all anchor sizes are [32,64,128,512,1024],but this setting is suit for COCO dataset which has so many small targets.

But the voc dataset targets are range [128,256,512].

So, we desgin the anchor setting:Ratios： [0.5,1,2],scales :[8,16], this is very import for voc dataset.

usage

download voc07,12 dataset ResNet50.caffemodel and rename to ResNet50.v2.caffemodel

cp ResNet50.v2.caffemodel data/pretrained_model/

OneDrive download: link

In my expriments, the codes require ~10G GPU memory in training and ~6G in testing. your can design the suit image size, mimbatch size and rcnn batch size for your GPUS.

compile caffe & lib

cd caffe-fpn
mkdir build
cd build
cmake ..
make -j16 all
cd lib
make

train & test

shared rcnn

./experiments/scripts/FP_Net_end2end.sh 1 FPN pascal_voc
./test.sh 1 FPN pascal_voc

megred rcnn

 ./experiments/scripts/FP_Net_end2end_merge_rcnn.sh 0 FPN pascal_voc
 ./test_mergercnn.sh 0 FPN pascal_voc

0 1 is GPU id.

TODO List

all tests passed
evaluate object detection performance on voc
evaluate merged rcnn version performance on voc

feature pyramid networks for object detection

Lin, T. Y., Dollár, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2016). Feature pyramid networks for object detection. arXiv preprint arXiv:1612.03144.

fpn's People

Contributors

Stargazers

Watchers

Forkers

rongchangzhao zhengyu0921 boosting insmod-he 10183308 abner2015 holygen xxradon wxbxj missdores chenchuanch yougoforward foralliance guanerdan gavinzhang1995 dodgaga last-one cv9527 suzhenghang hulaifeng wangzhe0623 justdolearning sfzhoucode bigtangmm jeromexin dreadlord1984 darknesszlx tongyoungg yuzcccc sunting78 da-he blankworld keyky opencvfun tigercouple zhjpqq giserh superhero1991 fangwudi ceciliapyy fanglw huipengzhang theone4ever humengdoudou amazefan ichem dimplesl xtanitfy dqruid zgsxwsdxg yhy1993824 lbj66 lzd0825 gitwithmch mqchen1993 soledad89 fendou201398 fendaq zhang1992wen1023 jackhenry1992 wanglaotou grseb9s azamattokhtaev liuguoyou bigcowpeking hhgxx123 arasharchor xyt2008 mati1994 wishinger-li pustar liyuanyaun lfdeep yowhatever 1093842024 youngdav 14151203 guangshengshi feitiandemiaomi tqdavid sinexue abeliuxl jzyztzn husteryxpeng firefox1031 scalaboy aiaini66 646677064 wendywlx maxime2046 luan-g zhulei2016 pandazha hanimiao mythke asagamiyou huliang2016 hhmaizi luolongqiang bikong2

fpn's Issues

Why adding group for deconvolution?

Hi,

Is there any reason for adding a group parameter for deconvolution?

anchor_size

FPN是一个多层预测框架，每一个分支都要设置对应的anchors．那么在不同层设置anchors，设计原则是什么？
还有您README中的about the anchor size setting一部分，能不能解释一下．
麻烦了！！

when i training my data,i get a error(ValueError: cannot convert float NaN to integer).i dont konw why?

I1114 21:43:12.214639 28872 sgd_solver.cpp:106] Iteration 0, lr = 0.0002
/home/gavin/icpr/FPN-caffe/tools/../lib/rpn/proposal_target_layer.py:225: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
fg_inds = npr.choice(fg_inds, size=fg_rois_per_this_image, replace=False)
/home/gavin/icpr/FPN-caffe/tools/../lib/rpn/proposal_target_layer.py:236: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
bg_inds = npr.choice(bg_inds, size=bg_rois_per_this_image, replace=False)
/home/gavin/icpr/FPN-caffe/tools/../lib/rpn/proposal_target_layer.py:243: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
labels[fg_rois_per_this_image:] = 0
/home/gavin/icpr/FPN-caffe/tools/../lib/rpn/anchor_target_layer.py:104: RuntimeWarning: invalid value encountered in sqrt
k = np.floor(k0 + np.log2(np.sqrt(areas)*1.0 / 64))
/home/gavin/icpr/FPN-caffe/tools/../lib/rpn/anchor_target_layer.py:106: RuntimeWarning: invalid value encountered in less
inds = np.where(k < min_k)[0]
/home/gavin/icpr/FPN-caffe/tools/../lib/rpn/anchor_target_layer.py:108: RuntimeWarning: invalid value encountered in greater
inds = np.where(k > max_k)[0]
Traceback (most recent call last):
File "./tools/train_net.py", line 112, in
max_iters=args.max_iters)
File "/home/gavin/icpr/FPN-caffe/tools/../lib/fast_rcnn/train.py", line 160, in train_net
model_paths = sw.train_model(max_iters)
File "/home/gavin/icpr/FPN-caffe/tools/../lib/fast_rcnn/train.py", line 101, in train_model
self.solver.step(1)
File "/home/gavin/icpr/FPN-caffe/tools/../lib/rpn/anchor_target_layer.py", line 116, in forward
if int(item)==p:
ValueError: cannot convert float NaN to integer
Command exited with non-zero status 1

FPN训练小目标数据

你好，unsky。我在做小目标检测的实验，之前用你的旧代码（VGG16模型）召回率可以达到90%左右，用新的代码（RESNET50 sharedrcnn）召回率只有70%左右。尝试改了scales为[4,8]和FPN/lib/fast_rcnn/config.py里的一些参数，但是效果都不理想，能不能给一些建议，谢谢！

代码细节询问

在源码的RPN中proposal_target_layer.py里
if sample_type == 'fpn':
#print 0
w = (rois[:,3]-rois[:,1])
h = (rois[:,4]-rois[:,2])
s = w * h
s[s<=0]=1e-6
layer_index = np.floor(k0+np.log2(np.sqrt(s)/224))

    layer_index[layer_index<2]=2
    layer_index[layer_index>5]=5

这部分中的layer_index是怎样计算的呀？有些许不懂，希望有人能够解答，非常感谢

Check failed: error == cudaSuccess (2 vs. 0) out of memory

I have met the error as follow when using FPN to train my own dataset:
Loading pretrained model weights from data/pretrained_model/ResNet50.v2.caffemodel
I0126 16:08:01.051177 10326 net.cpp:816] Ignoring source layer pool5
I0126 16:08:01.051213 10326 net.cpp:816] Ignoring source layer fc1000
I0126 16:08:01.051214 10326 net.cpp:816] Ignoring source layer prob
Solving...
F0126 16:08:02.152665 10326 syncedmem.cpp:56] Check failed: error == cudaSuccess (2 vs. 0) out of memory
*** Check failure stack trace: ***
./experiments/scripts/FP_Net_end2end.sh: 行 57: 10326 已放弃 (核心已转储) ./tools/train_net.py --gpu ${GPU_ID} --solver models/${PT_DIR}/${NET}/FP_Net_end2end/solver.prototxt --weights data/pretrained_model/ResNet50.v2.caffemodel --imdb ${TRAIN_IMDB} --iters ${ITERS} --cfg experiments/cfgs/FP_Net_end2end.yml ${EXTRA_ARGS}

I tried to reduce __C.TRAIN.IMS_PER_BATCH and __C.TRAIN.BATCH_SIZE in lib/fast_rcnn/config.py, but it doesn't work. How to reduce the batch_size to reduce the cost of memory?

gpu内存不足导致aborted（core dumped）

@unsky
你好，代码写的很好，不过我在运行时出现了一个问题，可能是由于我的gpu内存（1050ti，4g）不足导致Initializing net parameters时出现错误，下面是出错的内容：
zq@zq-System-Product-Name:~/FPN-master$
./experiments/scripts/FP_Net_end2end_merge_rcnn.sh 0 FPN pascal_voc
......
I0308 20:29:01.180505 2233 net.cpp:49] Initializing net from parameters:

.....
layer {
name: "res4b6"
type: "Eltwise"
bottom: "res4b5"
bottom: "res4b6_branch2c"
top: "res4b6"
}
layer {
name: "res4b6_relu"
type: "ReLU"
bottom: "res4b6"
top: "res4b6"
}
layer {
name: "res4b7_branch2a"
type: "Convolution"
bottom: "res4b6"
top: "res4b7_branch2a"
param {
lr_mult: 1
}
convolution_param {
num_output: 256
bias_term: false
pad: 0
kernel_size: 1
stride: 1
}
}
layer {
name: "scale4b7_branch2a"
type: "Scale"
bottom: "res4b7_branch2a"
top: "res4b7_branch2a"
param {
lr_mult: 0
decay_mult: 0
}
param {
lr_mult: 0
decay_mult: 0
}
scale_param {
bias_term: true
}
}
layer {
name: "res4b7_branch2a_relu"
type: "ReLU"
bottom: "res4b7_branch2a"
top: "res4b7_branch2a"
}
layer {
name: "res4b7_branch2b"
type: "Convolution"
bottom: "res4b7_branch2a"
top: "res4b7_branch2b"
param {
lr_mult: 1
}
convolution_param {
num_output: 256
bias_term: false
pad: 1
kernel_size: 3
stride: 1
}
}

layer {
name: "scale4b7_branch2b"
type: "Scale"
bottom: "res4b7_branch2b"
top: "res4b7_branch2b"
param {
lr_mult: 0
decay_mult: 0
}
param {
lr_mult: 0
decay_mult: 0
}
scale_param {
bias_term: true
}
}
layer {
name: "res4b7_branch2b_relu"
type: "ReLU"
bottom: "res4b7_branch2b"
top: "res4b7_branch2b"
}
layer {
name: "res4b7_branch2c"
type: "Convolution"
bottom: "res4b7_branch2b"
top: "res4b7_branch2c"
param {
lr_mult: 1
}
convolution_param {
num_output: 1024
bias_term: false
pad: 0
kernel_size: 1
stride: 1
}
}
layer {
name: "scale4b7
I0308 20:29:01.184592 2233 layer_factory.hpp:77] Creating layer input-data
F0308 20:29:01.184634 2233 layer_factory.hpp:81] Check failed: registry.count(type) == 1 (0 vs. 1) Unknown layer type: Python (known types: AbsVal, Accuracy, ArgMax, BNLL, BatchNorm, BatchReindex, Bias, Concat, ContrastiveLoss, Convolution, Data, Deconvolution, Dropout, DummyData, ELU, Eltwise, Embed, EuclideanLoss, Exp, Filter, Flatten, HDF5Data, HDF5Output, HingeLoss, Im2col, ImageData, InfogainLoss, InnerProduct, LRN, Log, MVN, MemoryData, MultinomialLogisticLoss, PReLU, Pooling, Power, ROIPooling, ReLU, Reduction, Reshape, SPP, Scale, Sigmoid, SigmoidCrossEntropyLoss, Silence, Slice, SmoothL1Loss, Softmax, SoftmaxWithLoss, Split, TanH, Threshold, Tile, WindowData)
*** Check failure stack trace: ***
Aborted (core dumped)

你之前在readme 里写过：
“In my expriments, the codes require ~10G GPU memory in training and ~6G in testing. your can design the suit image size, mimbatch size and rcnn batch size for your GPUS.”
但是我的gpu只有4G，而且我不知道要修改的这些batch size在哪些地方，你能告诉我应该怎样修改吗？
非常感谢！！！

A problem occured when I run ./experiments/scripts/FP_Net_end2end.sh

Hi unsky:
when I run ./experiments/scripts/FP_Net_end2end.sh, I got a problem as follow:
Traceback (most recent call last):
File "./tools/train_net.py", line 112, in
max_iters=args.max_iters)
File "/home/qi/Desktop/FPN-caffe-master/tools/../lib/fast_rcnn/train.py", line 157, in train_net
pretrained_model=pretrained_model)
File "/home/qi/Desktop/FPN-caffe-master/tools/../lib/fast_rcnn/train.py", line 43, in init
self.solver = caffe.SGDSolver(solver_prototxt)
File "/home/qi/Desktop/FPN-caffe-master/tools/../lib/roi_data_layer/layer.py", line 125, in setup
top[idx].reshape(1, self._num_classes * 4)
IndexError: Index out of range
And the pretrained_model I used is VGG16.v2.caffemodel, hope you can give me some help, Thanks

各个rpn中的anchor问题

unsky，你好!我在查看rpn网络的时候，发现这五个rpn模块用的都是相同的achor大小，而不是原论文中的那种说法。你是否尝试过每个level只对应一个固定且不同的anchor大小？

请问使用resnet101、resnet152caffemodel的时候，train.prototxt和test.prototxt应该如何修改呢

loss for loss_box = 0?

I1220 22:57:08.836243 26285 solver.cpp:229] Iteration 13940, loss = 6.60338
I1220 22:57:08.836273 26285 solver.cpp:245] Train net output #0: loss_bbox = 0 (* 1 = 0 loss)
I1220 22:57:08.836279 26285 solver.cpp:245] Train net output #1: loss_cls = 0.0678312 (* 1 = 0.0678312 loss)
I1220 22:57:08.836283 26285 solver.cpp:245] Train net output #2: rpn_cls_loss/p2 = 0.366679 (* 1 = 0.366679 loss)
I1220 22:57:08.836287 26285 solver.cpp:245] Train net output #3: rpn_cls_loss/p3 = 0.195475 (* 1 = 0.195475 loss)
I1220 22:57:08.836292 26285 solver.cpp:245] Train net output #4: rpn_cls_loss/p4 = 0.513616 (* 1 = 0.513616 loss)
I1220 22:57:08.836297 26285 solver.cpp:245] Train net output #5: rpn_cls_loss/p5 = 0.559214 (* 1 = 0.559214 loss)
I1220 22:57:08.836309 26285 solver.cpp:245] Train net output #6: rpn_cls_loss/p6 = 0.303023 (* 1 = 0.303023 loss)
I1220 22:57:08.836313 26285 solver.cpp:245] Train net output #7: rpn_loss_bbox/p2 = 3.02821 (* 1 = 3.02821 loss)
I1220 22:57:08.836316 26285 solver.cpp:245] Train net output #8: rpn_loss_bbox/p3 = 1.88581 (* 1 = 1.88581 loss)
I1220 22:57:08.836330 26285 solver.cpp:245] Train net output #9: rpn_loss_bbox/p4 = 0.740322 (* 1 = 0.740322 loss)
I1220 22:57:08.836334 26285 solver.cpp:245] Train net output #10: rpn_loss_bbox/p5 = 0.256244 (* 1 = 0.256244 loss)
I1220 22:57:08.836349 26285 solver.cpp:245] Train net output #11: rpn_loss_bbox/p6 = 0.278417 (* 1 = 0.278417 loss)

一个训练的问题

训练的时候遇到了这个问题：
/home/AISSD/FPN-caffe/tools/../lib/rpn/anchor_target_layer.py:105: RuntimeWarning: invalid value encountered in sqrt
k = np.floor(k0 + np.log2(np.sqrt(areas)*1.0 / 64))
Traceback (most recent call last):
File "./tools/train_net.py", line 112, in
max_iters=args.max_iters)
File "/home/AISSD/FPN-caffe/tools/../lib/fast_rcnn/train.py", line 162, in train_net
model_paths = sw.train_model(max_iters)
File "/home/AISSD/FPN-caffe/tools/../lib/fast_rcnn/train.py", line 101, in train_model
self.solver.step(1)
File "/home/AISSD/FPN-caffe/tools/../lib/rpn/anchor_target_layer.py", line 117, in forward
if int(item)==p:
ValueError: cannot convert float NaN to integer
输出了一下这个anchor_target_layer.py中的
ws = gt_boxes[:, 2] - gt_boxes[:, 0]
hs = gt_boxes[:, 3] - gt_boxes[:, 1]
areas = ws * hs
#print "debug::",ws,hs
k = np.floor(k0 + np.log2(np.sqrt(areas)*1.0 / 64))
发现hs是负的，然后输出了一下gt_boxes的值：
DEBUG:: [[ 5.97701132e-01 3.91703438e+04 5.20000000e+02 2.70160919e+02 1.00000000e+00]]
但是同样的数据在faster-rcnn中运行没有这个错，请问有谁知道这个问题怎么回事吗？

config.py parameters error

@unsky Hi, unsky. I tried to use your code to train FPN network for my dataset and I found an error after changing two parameters in fast_rcnn/config.py to change longest and shortest image side that:
__C.TRAIN.SCALES = (1280,) (the origin is 768)
__C.TRAIN.MAX_SIZE = 2000 (the origin is 1280)

the error part happens here:
I0125 09:46:02.151782 18521 net.cpp:106] Creating Layer p4
I0125 09:46:02.151787 18521 net.cpp:454] p4 <- c4
I0125 09:46:02.151793 18521 net.cpp:454] p4 <- upP5
I0125 09:46:02.151796 18521 net.cpp:411] p4 -> p4
F0125 09:46:02.151804 18521 eltwise_layer.cpp:34] Check failed: bottom[i]->shape() == bottom[0]->shape()
*** Check failure stack trace: ***
./experiments/scripts/FP_Net_end2end.sh: 行 57: 18521 已放弃 (核心已转储) ./tools/train_net.py --gpu ${GPU_ID} --solver models/${PT_DIR}/${NET}/FP_Net_end2end/solver.prototxt --weights data/pretrained_model/ResNet50.v2.caffemodel --imdb ${TRAIN_IMDB} --iters ${ITERS} --cfg experiments/cfgs/FP_Net_end2end.yml ${EXTRA_ARGS}

And if I change back the code could run well. could you please help me deal with it?
Thank you very much!

make -j16 and get error: ‘DeformableConvolutionParameter’ was not declared in this scope

`FPN/caffe-fpn/src/caffe/layer_factory.cpp:80:3: error: ‘DeformableConvolutionParameter’ was not declared in this scope
DeformableConvolutionParameter defor_conv_param = param.deformable_convolution_param();
^

FPN/caffe-fpn/src/caffe/layer_factory.cpp:81:3: error: ‘DeformableConvolutionParameter_Engine’ was not declared in this scope
DeformableConvolutionParameter_Engine engine = defor_conv_param.engine();
^

FPN/caffe-fpn/src/caffe/layer_factory.cpp:83:7: error: ‘engine’ was not declared in this scope
if (engine == DeformableConvolutionParameter_Engine_DEFAULT) {
^

FPN/caffe-fpn/src/caffe/layer_factory.cpp:83:17: error: ‘DeformableConvolutionParameter_Engine_DEFAULT’ was not declared in this scope
if (engine == DeformableConvolutionParameter_Engine_DEFAULT) {
^

FPN/caffe-fpn/src/caffe/layer_factory.cpp:84:14: error: ‘DeformableConvolutionParameter_Engine_CAFFE’ was not declared in this scope
engine = DeformableConvolutionParameter_Engine_CAFFE;
^

FPN/caffe-fpn/src/caffe/layer_factory.cpp:86:7: error: ‘engine’ was not declared in this scope
if (engine == DeformableConvolutionParameter_Engine_CAFFE) {
^

FPN/caffe-fpn/src/caffe/layer_factory.cpp:86:17: error: ‘DeformableConvolutionParameter_Engine_CAFFE’ was not declared in this scope
if (engine == DeformableConvolutionParameter_Engine_CAFFE) {`

About how to using python api to generate your faster_rcnn_resnet50_fpn

Hello, unsky;
would you please share your python scripts of generating your faster_rcnn_resnet50_fpn or give me some advice of your lib/rpn? I want to reimplement faster_rcnn_resnet100_fpn based on your work.
thanks very much.

Does VGG work for FPN?

I want to use VGG to implement FPN， but I have got a bad mAP( around 60 ). Is it a bad choice to use VGG as the backbone?

VOC2007 data set is trained when loss_bbox = 0

@unsky
Hello, thank you for providing the source code. I revised your network with reference to the code you wrote, but when training occurs, loss_bbox = 0. What adjustments do you have in mind? No other methods have been tried. If you make an opinion, thank you very much.
I0314 11:00:26.190989 18580 solver.cpp:229] Iteration 0, loss = 2.42246
I0314 11:00:26.191026 18580 solver.cpp:245] Train net output #0: loss_bbox = 0 (* 0.5 = 0 loss)
I0314 11:00:26.191032 18580 solver.cpp:245] Train net output #1: loss_cls = 3.05018 (* 0.5 = 1.52509 loss)
I0314 11:00:26.191038 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.879062 (* 0.5 = 0.439531 loss)
I0314 11:00:26.191045 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.771223 (* 0.5 = 0.385611 loss)
I0314 11:00:26.191049 18580 sgd_solver.cpp:107] Iteration 0, lr = 0.001
I0314 11:00:34.803771 18580 solver.cpp:229] Iteration 20, loss = 1.20221
I0314 11:00:34.803809 18580 solver.cpp:245] Train net output #0: loss_bbox = 0 (* 0.5 = 0 loss)
I0314 11:00:34.803817 18580 solver.cpp:245] Train net output #1: loss_cls = 1.56594 (* 0.5 = 0.782969 loss)
I0314 11:00:34.803822 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.681606 (* 0.5 = 0.340803 loss)
I0314 11:00:34.803829 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.0196237 (* 0.5 = 0.00981186 loss)
I0314 11:00:34.803836 18580 sgd_solver.cpp:107] Iteration 20, lr = 0.001
I0314 11:00:43.548290 18580 solver.cpp:229] Iteration 40, loss = 1.34465
I0314 11:00:43.548328 18580 solver.cpp:245] Train net output #0: loss_bbox = 4.46253e-06 (* 0.5 = 2.23126e-06 loss)
I0314 11:00:43.548336 18580 solver.cpp:245] Train net output #1: loss_cls = 1.51926 (* 0.5 = 0.75963 loss)
I0314 11:00:43.548341 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.678311 (* 0.5 = 0.339155 loss)
I0314 11:00:43.548346 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.0992929 (* 0.5 = 0.0496464 loss)
I0314 11:00:43.548352 18580 sgd_solver.cpp:107] Iteration 40, lr = 0.001
I0314 11:00:52.411149 18580 solver.cpp:229] Iteration 60, loss = 1.19288
I0314 11:00:52.411186 18580 solver.cpp:245] Train net output #0: loss_bbox = 3.54063e-06 (* 0.5 = 1.77032e-06 loss)
I0314 11:00:52.411193 18580 solver.cpp:245] Train net output #1: loss_cls = 1.48073 (* 0.5 = 0.740365 loss)
I0314 11:00:52.411198 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.694064 (* 0.5 = 0.347032 loss)
I0314 11:00:52.411204 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.450236 (* 0.5 = 0.225118 loss)
I0314 11:00:52.411209 18580 sgd_solver.cpp:107] Iteration 60, lr = 0.001
I0314 11:01:01.210149 18580 solver.cpp:229] Iteration 80, loss = 1.16331
I0314 11:01:01.210186 18580 solver.cpp:245] Train net output #0: loss_bbox = 2.98784e-06 (* 0.5 = 1.49392e-06 loss)
I0314 11:01:01.210193 18580 solver.cpp:245] Train net output #1: loss_cls = 1.58441 (* 0.5 = 0.792203 loss)
I0314 11:01:01.210199 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.758525 (* 0.5 = 0.379262 loss)
I0314 11:01:01.210206 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.193615 (* 0.5 = 0.0968074 loss)
I0314 11:01:01.210211 18580 sgd_solver.cpp:107] Iteration 80, lr = 0.001
I0314 11:01:09.967835 18580 solver.cpp:229] Iteration 100, loss = 1.2145
I0314 11:01:09.967872 18580 solver.cpp:245] Train net output #0: loss_bbox = 1.69471e-06 (* 0.5 = 8.47353e-07 loss)
I0314 11:01:09.967878 18580 solver.cpp:245] Train net output #1: loss_cls = 1.47326 (* 0.5 = 0.736631 loss)
I0314 11:01:09.967885 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.752394 (* 0.5 = 0.376197 loss)
I0314 11:01:09.967890 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.281891 (* 0.5 = 0.140946 loss)
I0314 11:01:09.967895 18580 sgd_solver.cpp:107] Iteration 100, lr = 0.001
I0314 11:01:18.736949 18580 solver.cpp:229] Iteration 120, loss = 0.995279
I0314 11:01:18.736986 18580 solver.cpp:245] Train net output #0: loss_bbox = 0 (* 0.5 = 0 loss)
I0314 11:01:18.736992 18580 solver.cpp:245] Train net output #1: loss_cls = 1.38187 (* 0.5 = 0.690937 loss)
I0314 11:01:18.736999 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.556746 (* 0.5 = 0.278373 loss)
I0314 11:01:18.737004 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.0684626 (* 0.5 = 0.0342313 loss)
I0314 11:01:18.737010 18580 sgd_solver.cpp:107] Iteration 120, lr = 0.001
I0314 11:01:27.601275 18580 solver.cpp:229] Iteration 140, loss = 1.00772
I0314 11:01:27.601315 18580 solver.cpp:245] Train net output #0: loss_bbox = 0 (* 0.5 = 0 loss)
I0314 11:01:27.601321 18580 solver.cpp:245] Train net output #1: loss_cls = 1.34108 (* 0.5 = 0.670541 loss)
I0314 11:01:27.601327 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.575482 (* 0.5 = 0.287741 loss)
I0314 11:01:27.601333 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.134016 (* 0.5 = 0.067008 loss)
I0314 11:01:27.601338 18580 sgd_solver.cpp:107] Iteration 140, lr = 0.001
I0314 11:01:36.287109 18580 solver.cpp:229] Iteration 160, loss = 1.02634
I0314 11:01:36.287149 18580 solver.cpp:245] Train net output #0: loss_bbox = 0 (* 0.5 = 0 loss)
I0314 11:01:36.287156 18580 solver.cpp:245] Train net output #1: loss_cls = 1.29888 (* 0.5 = 0.649439 loss)
I0314 11:01:36.287163 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.508832 (* 0.5 = 0.254416 loss)
I0314 11:01:36.287168 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.225303 (* 0.5 = 0.112652 loss)
I0314 11:01:36.287173 18580 sgd_solver.cpp:107] Iteration 160, lr = 0.001
I0314 11:01:45.057557 18580 solver.cpp:229] Iteration 180, loss = 1.31192
I0314 11:01:45.057595 18580 solver.cpp:245] Train net output #0: loss_bbox = 2.033e-05 (* 0.5 = 1.0165e-05 loss)
I0314 11:01:45.057602 18580 solver.cpp:245] Train net output #1: loss_cls = 1.31927 (* 0.5 = 0.659635 loss)
I0314 11:01:45.057608 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.728466 (* 0.5 = 0.364233 loss)
I0314 11:01:45.057615 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.872437 (* 0.5 = 0.436219 loss)
I0314 11:01:45.057620 18580 sgd_solver.cpp:107] Iteration 180, lr = 0.001
speed: 0.439s / iter
I0314 11:01:53.781239 18580 solver.cpp:229] Iteration 200, loss = 0.924464
I0314 11:01:53.781275 18580 solver.cpp:245] Train net output #0: loss_bbox = 0 (* 0.5 = 0 loss)
I0314 11:01:53.781281 18580 solver.cpp:245] Train net output #1: loss_cls = 1.2256 (* 0.5 = 0.612798 loss)
I0314 11:01:53.781287 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.470908 (* 0.5 = 0.235454 loss)
I0314 11:01:53.781293 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.0965142 (* 0.5 = 0.0482571 loss)
I0314 11:01:53.781298 18580 sgd_solver.cpp:107] Iteration 200, lr = 0.001
I0314 11:02:02.579058 18580 solver.cpp:229] Iteration 220, loss = 0.921759
I0314 11:02:02.579099 18580 solver.cpp:245] Train net output #0: loss_bbox = 0 (* 0.5 = 0 loss)
I0314 11:02:02.579108 18580 solver.cpp:245] Train net output #1: loss_cls = 1.21418 (* 0.5 = 0.607091 loss)
I0314 11:02:02.579118 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.503071 (* 0.5 = 0.251536 loss)
I0314 11:02:02.579125 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.0527053 (* 0.5 = 0.0263527 loss)

And with the increase of the number of training, a mistake was terminated in the back.
/lib/rpn/proposal_layer.py:248: RuntimeWarning: invalid value encountered in greater_equal
keep = np.where((ws >= min_size) & (hs >= min_size))[0]
Traceback (most recent call last):
File "./tools/train_net.py", line 112, in
max_iters=args.max_iters)
File "/home/sins/py-faster-rcnn/tools/../lib/fast_rcnn/train.py", line 161, in train_net
model_paths = sw.train_model(max_iters)
File "/home/sins/py-faster-rcnn/tools/../lib/fast_rcnn/train.py", line 102, in train_model
self.solver.step(1)
File "/home/sins/py-faster-rcnn/tools/../lib/rpn/proposal_layer.py", line 203, in forward
pad = npr.choice(keep, size=post_nms_topN - len(keep))
File "mtrand.pyx", line 1126, in mtrand.RandomState.choice
Thanks very much！！！！！

have you successfully trained your network?

Dear unsky, have you successfully trained your network? can you provide a copy of the output caffemodel?

rpn_cls_score/p2 output参数不一致

rpn_cls_score/p2这个layer在trian.prototxt和test.prototxt，output参数不一致

can you share you pretrained model?

when I train the net, I meet an error. I'm new

I0103 17:59:46.463167 29140 layer_factory.hpp:77] Creating layer fpn_loss_cls
I0103 17:59:46.463174 29140 net.cpp:106] Creating Layer fpn_loss_cls
I0103 17:59:46.463178 29140 net.cpp:454] fpn_loss_cls <- rpn_cls_score_reshape
I0103 17:59:46.463183 29140 net.cpp:454] fpn_loss_cls <- rpn_labels
I0103 17:59:46.463191 29140 net.cpp:411] fpn_loss_cls -> FPNClsLoss
I0103 17:59:46.463199 29140 layer_factory.hpp:77] Creating layer fpn_loss_cls
F0103 17:59:46.465251 29140 softmax_loss_layer.cpp:47] Check failed: outer_num_ * inner_num_ == bottom[1]->count() (245520 vs. 552960) Number of labels must match number of predictions; e.g., if softmax axis == 1 and prediction shape is (N, C, H, W), label count (number of labels) must be NHW, with integer values in {0, 1, ..., C-1}.
*** Check failure stack trace: ***

Thank you

Some questions about your FPN

1.According to the paper，each rpn should have 3 different anchors（1 erea * 3 aspect ratios) because each levle has their own erea. But you still set 9 anchors （3 erea * 3 aspect ratios）at each level of rpn

2.Have you tried to share rpn weights and bias? It will save time and memory, and might improve the result

run ./experiments/scripts/FP_Net_end2end.sh 1 VGG16 pascal_voc,got an error

hi unsky,when i run ./experiments/scripts/FP_Net_end2end.sh 1 VGG16 pascal_voc,i have got the error:
++ set -e
++ export PYTHONUNBUFFERED=True
++ PYTHONUNBUFFERED=True
++ GPU_ID=0
++ NET=VGG16
++ NET_lc=vgg16
++ DATASET=pascal_voc
++ array=($@)
++ len=3
++ EXTRA_ARGS=
++ EXTRA_ARGS_SLUG=
++ case $DATASET in
++ TRAIN_IMDB=voc_2007_trainval
++ TEST_IMDB=voc_2007_test
++ PT_DIR=pascal_voc
++ ITERS=150000
+++ date +%Y-%m-%d_%H-%M-%S
++ LOG=experiments/logs/faster_rcnn_end2end_VGG16_.txt.2017-11-04_19-35-27
++ exec
+++ tee -a experiments/logs/faster_rcnn_end2end_VGG16_.txt.2017-11-04_19-35-27
tee: experiments/logs/faster_rcnn_end2end_VGG16_.txt.2017-11-04_19-35-27: No such file or directory
++ echo Logging output to experiments/logs/faster_rcnn_end2end_VGG16_.txt.2017-11-04_19-35-27
Logging output to experiments/logs/faster_rcnn_end2end_VGG16_.txt.2017-11-04_19-35-27
++ ./tools/train_net.py --gpu 0 --solver models/pascal_voc/VGG16/FP_Net_end2end/solver.prototxt
Called with args:
Namespace(cfg_file=None, gpu_id=0, imdb_name='voc_2007_trainval', max_iters=40000, pretrained_model=None, randomize=False, set_cfgs=None, solver='models/pascal_voc/VGG16/FP_Net_end2end/solver.prototxt')
Using config:
{'DATA_DIR': '/home/tanglin/FPN-master/data',
'DEDUP_BOXES': -1,
'EPS': 1e-14,
'EXP_DIR': 'default',
'GPU_ID': 0,
'MATLAB': 'matlab',
'MODELS_DIR': '/home/tanglin/FPN-master/models/pascal_voc',
'PIXEL_MEANS': array([[[ 102.9801, 115.9465, 122.7717]]]),
'RNG_SEED': 3,
'ROOT_DIR': '/home/tanglin/FPN-master',
'TEST': {'BBOX_REG': True,
'HAS_RPN': False,
'MAX_SIZE': 520,
'NMS': 0.3,
'PROPOSAL_METHOD': 'rpn',
'RPN_MIN_SIZE': 16,
'RPN_NMS_THRESH': 0.7,
'RPN_POST_NMS_TOP_N': 2000,
'RPN_PRE_NMS_TOP_N': 6000,
'SCALES': [375],
'SVM': False},
'TRAIN': {'ASPECT_GROUPING': True,
'BATCH_SIZE': 128,
'BBOX_INSIDE_WEIGHTS': [1.0, 1.0, 1.0, 1.0],
'BBOX_NORMALIZE_MEANS': [0.0, 0.0, 0.0, 0.0],
'BBOX_NORMALIZE_STDS': [0.1, 0.1, 0.2, 0.2],
'BBOX_NORMALIZE_TARGETS': True,
'BBOX_NORMALIZE_TARGETS_PRECOMPUTED': False,
'BBOX_REG': True,
'BBOX_THRESH': 0.7,
'BG_THRESH_HI': 0.5,
'BG_THRESH_LO': 0.0,
'FG_FRACTION': 0.25,
'FG_THRESH': 0.5,
'HAS_RPN': False,
'IMS_PER_BATCH': 2,
'MAX_SIZE': 520,
'PROPOSAL_METHOD': 'rpn',
'RPN_BATCHSIZE': 256,
'RPN_BBOX_INSIDE_WEIGHTS': [1.0, 1.0, 1.0, 1.0],
'RPN_CLOBBER_POSITIVES': False,
'RPN_FG_FRACTION': 0.5,
'RPN_MIN_SIZE': 0,
'RPN_NEGATIVE_OVERLAP': 0.3,
'RPN_NMS_THRESH': 0.7,
'RPN_POSITIVE_OVERLAP': 0.7,
'RPN_POSITIVE_WEIGHT': -1.0,
'RPN_POST_NMS_TOP_N': 2000,
'RPN_PRE_NMS_TOP_N': 12000,
'SCALES': [375],
'SNAPSHOT_INFIX': '',
'SNAPSHOT_ITERS': 5000,
'USE_FLIPPED': True,
'USE_PREFETCH': False},
'USE_GPU_NMS': True}
Loaded dataset voc_2007_trainval for training
Set proposal method: rpn
Appending horizontally-flipped training examples...
voc_2007_trainval gt roidb loaded from /home/tanglin/FPN-master/data/cache/voc_2007_trainval_gt_roidb.pkl
loading None
Traceback (most recent call last):
File "./tools/train_net.py", line 104, in
imdb, roidb = combined_roidb(args.imdb_name)
File "./tools/train_net.py", line 69, in combined_roidb
roidbs = [get_roidb(s) for s in imdb_names.split('+')]
File "./tools/train_net.py", line 66, in get_roidb
roidb = get_training_roidb(imdb)
File "/home/tanglin/FPN-master/tools/../lib/fast_rcnn/train.py", line 118, in get_training_roidb
imdb.append_flipped_images()
File "/home/tanglin/FPN-master/tools/../lib/datasets/imdb.py", line 106, in append_flipped_images
boxes = self.roidb[i]['boxes'].copy()
File "/home/tanglin/FPN-master/tools/../lib/datasets/imdb.py", line 67, in roidb
self._roidb = self.roidb_handler()
File "/home/tanglin/FPN-master/tools/../lib/datasets/pascal_voc.py", line 141, in rpn_roidb
rpn_roidb = self._load_rpn_roidb(gt_roidb)
File "/home/tanglin/FPN-master/tools/../lib/datasets/pascal_voc.py", line 151, in _load_rpn_roidb
assert os.path.exists(filename),
File "/usr/lib/python2.7/genericpath.py", line 18, in exists
os.stat(path)
TypeError: coercing to Unicode: need string or buffer, NoneType found

but when i run faster_rcnn_alt_opt.sh,it is ok?
how can i solve it?

This model can not support LMDB train data?

Traceback (most recent call last):
File "./tools/train_net.py", line 104, in
imdb, roidb = combined_roidb(args.imdb_name)
File "./tools/train_net.py", line 69, in combined_roidb
roidbs = [get_roidb(s) for s in imdb_names.split('+')]
File "./tools/train_net.py", line 62, in get_roidb
imdb = get_imdb(imdb_name)
File "/home/t91/pengyulong/workspace/FPN/tools/../lib/datasets/factory.py", line 37, in get_imdb
raise KeyError('Unknown dataset: {}'.format(name))
KeyError: 'Unknown dataset: /home/t91/data/lmdb/train_lmdb'

training on coco dataset

hi, unsky
Did you have ever trained FPN on coco dataset? when I did this work, there are some errors. Like,
../lib/rpn/proposal_layer.py", line 209, in forward
pad = npr.choice(keep, size=int(post_nms_topN) - len(keep))
File "mtrand.pyx", line 1121, in mtrand.RandomState.choice
ValueError: a must be non-empty

and,
../lib/fast_rcnn/bbox_transform.py:50: RuntimeWarning: overflow encountered in exp
pred_h = np.exp(dh) * heights[:, np.newaxis]

thanks.

training on cityscape dataset

您好，我最近用ResNet50-fpn在cityscape数据集上训练的时候总是出现梯度爆炸，然后loss出现nan，但是用ResNet50单分支就不会。想请教一下您对这个问题怎么看，这个是哪里出了问题呢。目前，把学习率设置为0.0001不会崩掉，但是训练结果不是很好。我看您的这个网络中，不同的层初始化权重还不一样，这个weight的初始化权重对网络是不是有很大影响呢，我尝试着去调了weight的初始化权重，但是没有出现很好的效果，请问有没有什么调节的技巧吗。

as_rois.py & simple questions..

First of all, Thank you for sharing FPN-caffe version
I have some simple questions.
1)
You apply "as_rois layer" in model/test.prototxt. However, It seems that you did not apply "as_rois_layer" in model/train.prototxt
why this?
2)
fc layers are weight sharing for this version. however, this is time consuming because when fc/h2 is done and then fc/h3 starts....(fc/h2 , fc/h3, fc/h4, fc/h5 each is done and merge for socre & bounding box)
I think it is better gathering roi_pooling results (roi_pool/h2, h3, h4, h5) and forward fc layers. (i.e. merge ->fc // your version : fc2,3,4,5 -> merge)
this is better when training & testing...
Thank you!

training problem

when i training ,i have a problem :

Traceback (most recent call last):
File "/home/hncs/liuwei/FPN1/tools/../lib/rpn/proposal_layer.py", line 14, in
from fast_rcnn.nms_wrapper import nms
File "/home/hncs/liuwei/FPN1/tools/../lib/fast_rcnn/nms_wrapper.py", line 9, in
from nms.gpu_nms import gpu_nms
ImportError: No module named gpu_nms
Traceback (most recent call last):
File "./tools/train_net.py", line 113, in
max_iters=args.max_iters)
File "/home/hncs/liuwei/FPN1/tools/../lib/fast_rcnn/train.py", line 143, in train_net
pretrained_model=pretrained_model)
File "/home/hncs/liuwei/FPN1/tools/../lib/fast_rcnn/train.py", line 46, in init
self.solver = caffe.SGDSolver(solver_prototxt)
SystemError: NULL result without error in PyObject_Call

But i running on the GPU,and the id is right, i don't know what's wrong.
thank you very much!
@unsky

The FPN-caffe has a poor result

Hi @unsky

I train the FPN-caffe in voc dataset, and the AP is 59%. I test the model in our video and the result is poor. Do you test the AP in voc dataset? Do you test the model in other video?

Why adding stride?

@unsky

Why you add stride to pad the input image?

https://github.com/unsky/FPN/blob/master/lib/utils/blob.py#L47

train result about mAP

when i run ./experiments/scripts/FP_Net_end2end.sh 1 FPN pascal_voc
mean mAP = 0.719 ，not 0.788 i don't know why. Please help me.

label and rois->num() not equal.

I1103 18:47:46.824345 300 layer_factory.hpp:77] Creating layer loss_cls
F1103 18:47:46.824635 300 loss_layer.cpp:19] Check failed: bottom[0]->num() == bottom[1]->num() (4 vs. 1) The data and label should have the same number.

In your train.prototxt, after roi-data layer, you concat(axis=0) 4 roi_pool, then the num is 4, but the label num is 1, how to solve it?
I run it on my caffe, with your whole rpn files.
THANKS!!

细节询问

请问您这里的caffe-fpn与Ross的py-faster-rcnn使用的caffe什么不同之处？
是只多出了deformable conv的支持吗？

process

hi ,i'm a green hand, i can't understand i run the
sudo./experiments/scripts/FP_Net_end2end.sh 1 FPN pascal_voc,
but:
bash: sudo./experiments/scripts/FP_Net_end2end.sh: No such file or directory.
do you have a detailed process thank you very much!

网络的训练测试以及数据集的准备

您好，想请教一下FPN的训练测试以及准备数据集是不是和faster rcnn是一样的？我准备好准备工作demo.py出现bash 权限不足的问题是怎么回事呢？希望很快收到您的回复，谢谢！

process

i'm a green hand, do you have a detailed process？what can can i should do use my dataset? thank you very much.

Can you share the trained model?

Could you share the trained model?
Thank you very much

make: *** No targets specified and no makefile found. Stop

请问在build/lib里make 提示这个错误怎么办？

I want to konw the Memory Of GPU?

i set the batch size si 320 ,but the memory of gpu is 4417MiB .

I want to kown ,is it normal?

small objects decetion

Has anyone done experiments.how about the effect of small objects decetion?

resnet

where is resnet?

error occured in the test phase

Hello, sir. I met an error in the test phase like this:
"
Traceback (most recent call last):
File "./tools/test_net.py", line 90, in
test_net(net, imdb, max_per_image=args.max_per_image, vis=args.vis)
File "/home/gaokun/FPN-caffe/tools/../lib/fast_rcnn/test.py", line 268, in test_net
scores, boxes = im_detect(net, im, box_proposals)
File "/home/gaokun/FPN-caffe/tools/../lib/fast_rcnn/test.py", line 183, in im_detect
pred_boxes = bbox_transform_inv(boxes, box_deltas)
File "/home/gaokun/FPN-caffe/tools/../lib/fast_rcnn/bbox_transform.py", line 47, in bbox_transform_inv
pred_ctr_x = dx * widths[:, np.newaxis] + ctr_x[:, np.newaxis]
ValueError: operands could not be broadcast together with shapes (225,2) (300,1)
"
I don't know how to solve it. can you give me some advice ?

Thank you!

单张1080ti上跑训练会报out of memory错误

单张1080ti上跑训练会报错误：
syncedmem.cpp:56] Check failed: error == cudaSuccess (2 vs. 0) out of memory

我在solver文件把iter_size改为1，还是报一样的错。

是显存的不够的原因吗？

Floating point exception (core dumped)

@unsky
hi,when I train FPN on my own dataset,I met error:

I0312 16:25:25.883342 2983 sgd_solver.cpp:106] Iteration 0, lr = 0.0005
/home/zq/FPN/tools/../lib/rpn/proposal_layer.py:175: RuntimeWarning: invalid value encountered in greater_equal
keep = np.where((ws >= min_size) & (hs >= min_size))[0]
Floating point exception (core dumped)

I try to change lr from 0.001 to 0.0001,but it didn't work.I also change RNG_SEED,and it also didn't work.
I don't know how to solve it.please help me,thanks so much!

model sharing

Is that OK to sharing your final model? it may cannot achieve this mAP because of the equipment.

loss concussion

Hi, I add the fpn to ZF and i train the pasalvoc datasets. The loss is high, can you give me some advice?
ps: I just add two feature pyramid layer p4，p3 to ZF

when I train the model ,I meet the error.

Do you consider "avoid aliasing" layer?

According to the paper,

Finally, we append a 3×3 convolution on each merged map togenerate the final feature map, which is to reduce the alias-ing effect of upsampling.

But in the network you defined, there is no such layer.

Why?

loss=nan

I add your rpn operator to my caffe to train fpn, I set batch_size as 2, base_lr to 0.0001 to 0.001, the loss reduce from larger than 3 to smaller than 1 after hundreds of iterations, then the loss became to Nan. Can you give me some advises? And serveral warnings occured during training progress, shown as below.

training on my own dataset

@unsky Hi, unsky. I really appreciate the code for FPN. But when I used it I got very bad mAP for my single object detection task on my own dataset. I have changed the parameters in train.prototxt and train_mergercnn.prototxt with changing 21 to 2 (object + background) and 84 to 8. The code could run well and loss also looks fine. So could you please tell me are there other places I need to change parameters to for training on my dataset?

unsky / fpn Goto Github PK

fpn's Introduction

results

framework

about the anchor size setting

usage

compile caffe & lib

train & test

TODO List

feature pyramid networks for object detection

fpn's People

Contributors

Stargazers

Watchers

Forkers

fpn's Issues

Recommend Projects

Recommend Topics

Recommend Org