xuannianz / keras-centernet Goto Github PK

Thank you for great work, I would want to know where I can download the pretrained model "ResNet-50-model.keras.h5"

It's possible that there is no object in a background image, is It reasonable "assert bboxes.shape[0] != 0 " in generator/common.py （line 398）

Nice work ! This project support Multi-GPUs ???

like this,why it has val_loss?
Traceback (most recent call last):
File "train.py", line 356, in
main()
File "train.py", line 351, in main
validation_data=validation_generator
File "/usr/local/lib/python3.6/dist-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/keras/engine/training.py", line 1732, in fit_generator
initial_epoch=initial_epoch)
File "/usr/local/lib/python3.6/dist-packages/keras/engine/training_generator.py", line 260, in fit_generator
callbacks.on_epoch_end(epoch, epoch_logs)
File "/usr/local/lib/python3.6/dist-packages/keras/callbacks/callbacks.py", line 152, in on_epoch_end
callback.on_epoch_end(epoch, logs)
File "/usr/local/lib/python3.6/dist-packages/keras/callbacks/callbacks.py", line 702, in on_epoch_end
filepath = self.filepath.format(epoch=epoch + 1, **logs)
KeyError: 'val_loss'

您好！感谢您的巨大工作，mAP=0.729 是用了nms的吗？那没有用nms时mAP是多少？

centernet 不是free anchor的吗？为什么还要用nms？

which tf version did you use?

The link you shared for the model didn't work with me, can you please share it again with another website. Thank you.

What exactly is this input? It isn't like the heatmap because the input shape is (max,) and not an image...

Hi.
I use this repo with my own dataset but I just see Epoch 1/200 and Killed.

Allocation of 1426063360 exceeds 10% of system memory.

What happen?
And is this OK that each image in my dataset has many objects?

This error how to deal with ? According to the Tensorflow 3.6 tf.gather() API the param 'batch_dims' change to 'axis'? Then axis=1 ?

When I reload a saved model with keras.models.load_model(model_path)
I got a the following error:

TypeError: ('Keyword argument not understood:', 'freeze')

I meet the same problem as broadinstitute/keras-resnet#29,
model = load_model(model_path, custom_objects={'BatchNormalization': keras_resnet.layers.BatchNormalization}) not works for me, I don't know why.

def preprocess_image(self, image, c, s, tgt_w, tgt_h):
trans_input = get_affine_transform(c, s, (tgt_w, tgt_h))
image = cv2.warpAffine(image, trans_input, (tgt_w, tgt_h), flags=cv2.INTER_LINEAR)
I don't understand：
Why does preprocessing need affine transformation? What is the role?

I tried

python train.py --freeze-backbone --gpu 0 --random-transform --compute-val-loss --batch-size 32 --steps 1000 coco datasets/coco

but got AssertError

assert annotations['bboxes'].shape[0] != 0
AssertionError

Looks like not found some annotation

I have train on own dataset.
In an image, I have more than 100 objects. The code will raise an error :
/keras-CenterNet/generators/common.py", line 419, in compute_inputs batch_indices[b, i] = ct_int[1] * self.output_size + ct_int[0] IndexError: index 100 is out of bounds for axis 1 with size 100
If I change "max_object=1000" at line 28 in common.py, will get an error:
Traceback (most recent call last): File "train.py", line 356, in <module> main() File "train.py", line 351, in main validation_data=validation_generator File "/home/po/anaconda3/envs/keras_centernet/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper return func(*args, **kwargs) File "/home/po/anaconda3/envs/keras_centernet/lib/python3.6/site-packages/keras/engine/training.py", line 1658, in fit_generator initial_epoch=initial_epoch) File "/home/po/anaconda3/envs/keras_centernet/lib/python3.6/site-packages/keras/engine/training_generator.py", line 215, in fit_generator class_weight=class_weight) File "/home/po/anaconda3/envs/keras_centernet/lib/python3.6/site-packages/keras/engine/training.py", line 1443, in train_on_batch class_weight=class_weight) File "/home/po/anaconda3/envs/keras_centernet/lib/python3.6/site-packages/keras/engine/training.py", line 757, in _standardize_user_data exception_prefix='input') File "/home/po/anaconda3/envs/keras_centernet/lib/python3.6/site-packages/keras/engine/training_utils.py", line 141, in standardize_input_data str(data_shape)) ValueError: Error when checking input: expected input_3 to have shape (100, 2) but got array with shape (1000, 2)

I faced this issue when running inference.py

Hi, There's no code in "train.py" in function create_generators, that will make train generator and val generator for COCO dataset, though file coco.py with COCOgenerator class exists in your repository and it might be corrected easily, perhaps.

Received this error when I was training on my custom dataset. I used the CSV generator for my custom dataset and inferencing worked fine but running the training gave me this error. Any help?

Here's the command I used to run the training:

python train.py --snapshot checkpoints/pascal_81_1.5415_3.0741_0.6860_0.7057_0.7209_0.7290.h5 --gpu 0 --random-transform --compute-val-loss --batch-size 32 --steps 1000 csv datasets/oil_palm/images/annotations.csv datasets/oil_palm/images/classes.csv

I tray to train my dataset and 1 epoch takes 2 hours is that normal ? I use GPU so I expect it to be faster!! How can I speed up training time?

Maybe this is not a literate "issue" about this repo.

Just noticed that you are using TF 1.15. The first image fed in to model during inference is always slow on my TF 1.14, while pytorch performs normally. Do you have a similiar condition？

BTW, https://github.com/fizyr/keras-retinanet/issues/747#issue-372118785 this issue said setTF_CUDNN_USE_AUTOTUNE=0 on TF1.11 works, but not works on my 1.14.

Any experience on you about this?

hi, I clone your project. but it can not run. the error message is "ModuleNotFoundError: No module named 'compute_overlap'". What should I do ?

thanks for your sharing .

I got an error when i train model with coco data.

The command is:
python3 train.py --snapshot ~/data/deepfashiondata/model/ResNet-50-model.keras.h5 --freeze-backbone --gpu 0 --random-transform --compute-val-loss --batch-size 1 --steps 1000 coco ~/data/deepfashiondata/smalldata

The error message is:

Epoch 1/200
Traceback (most recent call last):
File "train.py", line 357, in
main()
File "train.py", line 352, in main
validation_data=validation_generator
File "/home/zhaozilong/.local/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/home/zhaozilong/.local/lib/python3.6/site-packages/keras/engine/training.py", line 1658, in fit_generator
initial_epoch=initial_epoch)
File "/home/zhaozilong/.local/lib/python3.6/site-packages/keras/engine/training_generator.py", line 181, in fit_generator
generator_output = next(output_generator)
File "/home/zhaozilong/.local/lib/python3.6/site-packages/keras/utils/data_utils.py", line 616, in get
six.reraise(*sys.exc_info())
File "/home/zhaozilong/.local/lib/python3.6/site-packages/six.py", line 703, in reraise
raise value
File "/home/zhaozilong/.local/lib/python3.6/site-packages/keras/utils/data_utils.py", line 603, in get
inputs = future.get(timeout=30)
File "/usr/lib/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/home/zhaozilong/.local/lib/python3.6/site-packages/keras/utils/data_utils.py", line 406, in get_index
return _SHARED_SEQUENCES[uid][i]
File "/home/zhaozilong/thirdparty/keras-CenterNet/generators/common.py", line 526, in getitem
inputs, targets = self.compute_inputs_targets(group)
File "/home/zhaozilong/thirdparty/keras-CenterNet/generators/common.py", line 503, in compute_inputs_targets
inputs = self.compute_inputs(image_group, annotations_group)
File "/home/zhaozilong/thirdparty/keras-CenterNet/generators/common.py", line 413, in compute_inputs
heatmap_temp = batch_hms[b, :, :, cls_id]
IndexError: only integers, slices (:), ellipsis (...), numpy.newaxis (None) and integer or boolean arrays are valid indices

The target in this case are the heatmaps, height-widths and their offsets, which are being used as input to the model at here https://github.com/xuannianz/keras-CenterNet/blob/master/models/resnet.py#L120 .
We can create those tensors from the bounding box input information, but I can't find the corresponding code where we transform the inputs(bounding box) to heatmaps.

Please can you point me that part of the code ?
Thanks.

i do not found subsampling with 1/4 and maxpooling layers

Thanks for your code. Is there any idea to load the pre-trained model of Pytorch CentetNet to this Keras? So, we can use the original pre-trained model for fine-tuning. Thanks. @xuannianz

will using flip_test promote map?

Thanks for your nice work! Could you upload a license file?

Hi, everyone.
I wonder if prediction model is being trained while training model is being trained ?
I think there's an error in file eval/coco.py line 111 :
def init(self, generator, tensorboard=None, threshold=0.05):
There should be extra argument - "prediction_model" since in train.py line 88 looks like this
evaluation = CocoEval(validation_generator, prediction_model, tensorboard=tensorboard_callback)
and this CocoEval gives argument "prediction_model". Thus line 111 of eval/coco.py should be like this:
def init(self, generator, model, tensorboard=None, threshold=0.05):

And there's an error occurs in line 47 of eval/coco.py because it tries to use training model instead of prediction model. Though we send argument prediction_model to CocoEval, never the less this variable is modified somehow (I didn't find where exactly) in other packages of library. I think, variable self.model should be changes to another unique, for example, self.prediction_model. And it will become like this:

class CocoEval(keras.callbacks.Callback):
""" Performs COCO evaluation on each epoch.
"""

def __init__(self, generator, model, tensorboard=None, threshold=0.05):
    """ CocoEval callback intializer.

    Args
        generator   : The generator used for creating validation data.
        tensorboard : If given, the results will be written to tensorboard.
        threshold   : The score threshold to use.
    """
    self.generator = generator
    self.threshold = threshold
    self.tensorboard = tensorboard
    self.prediction_model = model
    super(CocoEval, self).__init__()

And even if you'll correct all the above mistakes I'm not sure that training model for COCO dataset will be correct, because right now my trained model on my custom COCO dataset has zero detections on all images, though val loss fell from about 200 to 6.

各位好，我自制的数据集，用于模型训练。结果每轮训练完做evaluation，mAP始终为0。
大家碰过这种情况吗，现在不知道该怎么办？

hi，thank you for your hard work,
Have you ever tried to run it in a CPU environment?

你好，代码运行到 detections = tf.concat([topk_x1, topk_y1, topk_x2, topk_y2, scores, class_ids], axis=-1)。出现了下列的错误，ValueError: Shape must be rank 4 but is rank 3 for 'lambda_1/concat' (op: 'ConcatV2') with input shapes: [?,?,100,?], [?,?,100,?], [?,?,100,?], [?,?,100,?], [?,100,1], [?,100,1], [].

I have measured the AP@IoU 0.5 for the pretrained model you shared for PascalVOC.

Measured on VOC17TEST, the results are mAP: 0.7293 when nms=True and it reduces to mAP: 0.60 when nms=False.

Could you observe similar results? In the paper, they did experiments on CoCo and the reduction in mAP was quite small when NMS was not used.

I wonder if this performance is due to implementation or a limitation of the model itself.

Can you please provide a script to convert the H5 model to pb model, also an inference sample for pb model would be helpful.

大神能提供一个可以把H5转成PB的代码吗？谢谢了

what's the mean of loss here?

Hello,

I've been trying to train a model following the example available in the Readme but when executing the train.py file I'm getting this error:

Traceback (most recent call last):
  File "train.py", line 356, in <module>
    main()
  File "train.py", line 334, in main
    args,
  File "train.py", line 90, in create_callbacks
    from eval.pascal import Evaluate
  File "/home/keras-CenterNet/eval/pascal.py", line 18, in <module>
    from .common import evaluate
  File "/home/keras-CenterNet/eval/common.py", line 21, in <module>
    from utils.compute_overlap import compute_overlap
ModuleNotFoundError: No module named 'utils.compute_overlap'

I've used keras-retinanet before and I was able to easily run the inference code in this repository.

hello author,

I am generating custom dataset for centernet model. can you please help me understand how to create height width labels for multi class. I know how to create Gaussian heatmap. For example if I have 20 classes and if output size is 128x128, I will create empty numpy array of size 128x128x20 and append every class heatmap into 3rd channel.

but for height width according to your paper, it is 128,128,2. I know 2 channel for height and width. Does it mean maximum object limit is 128x128=16284. Can you please explain how to append the data and which axis. I am confused how it can work for multi classes.

Thank you

他的代码有问题，还很大。
训练时候跑不通。
测试也不行，conv2d_2的参数完全带不进去。
C:\Program Files\Python36\lib\site-packages\keras\engine\saving.py:1319: UserWarning: Skipping loading of weights for layer conv2d_2 due to mismatch in shape ((1, 1, 64, 80) vs (20, 64, 1, 1)). weight_values[i].shape)) C:\Program Files\Python36\lib\site-packages\keras\engine\saving.py:1319: UserWarning: Skipping loading of weights for layer conv2d_2 due to mismatch in shape ((80,) vs (20,)). weight_values[i].shape))

老哥，你好，想问一下，你基于resnet50的backbone得到的mAP只有72左右吗? 官方的pytorch版本似乎有77点多，你用的跟官方的是一样的数据集吗，如果是的话，你觉得这个可能是什么原因导致的？官方的代码在pascal数据集上存在过拟合的问题吗？

In resnet.py how can I use the resnet18 backbone ? if I change the backbone to resnet18/resnet34 it didn't detect any object.
Is resnet18 faster than resnet50?

Thank you for your amazing work!
I run the code on my PC with tensorflow 2.1.0, and replace the keras package with tensorflow.compat.v1.keras, considering the different version of tensorflow. For example,

import tensorflow.compat.v1 as tf
import tensorflow.compat.v1.keras
import tensorflow.compat.v1.keras.backend as K
from tensorflow.compat.v1.keras.optimizers import Adam, SGD
import tensorflow.compat.v1.keras.preprocessing.image
# import tensorflow as tf
# import keras
# import keras.backend as K
# from keras.optimizers import Adam, SGD
# import keras.preprocessing.image

But I run into the problem at 331 line in train.py,
model.compile(optimizer=Adam(lr=1e-3), loss={'centernet_loss': lambda y_true, y_pred: y_pred})

The error is as follows:

Traceback (most recent call last):
  File "/home/m1nt/Programmer/Anaconda/envs/py37_keras/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 1619, in _create_c_op
    c_op = c_api.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: slice index -1 of dimension 0 out of bounds. for 'loss/centernet_loss_loss/strided_slice' (op: 'StridedSlice') with input shapes: [0], [1], [1], [1] and with computed input tensors: input[1] = <-1>, input[2] = <0>, input[3] = <1>.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/m1nt/Programmer/Projects/Pycharm/sec_thesis/keras-CenterNet-master/train.py", line 366, in <module>
    main()
  File "/home/m1nt/Programmer/Projects/Pycharm/sec_thesis/keras-CenterNet-master/train.py", line 331, in main
    model.compile(optimizer=Adam(lr=1e-3), loss={'centernet_loss': lambda y_true, y_pred: y_pred})
  File "/home/m1nt/Programmer/Anaconda/envs/py37_keras/lib/python3.7/site-packages/tensorflow_core/python/training/tracking/base.py", line 457, in _method_wrapper
    result = method(self, *args, **kwargs)
  File "/home/m1nt/Programmer/Anaconda/envs/py37_keras/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py", line 446, in compile
    self._compile_weights_loss_and_weighted_metrics()
  File "/home/m1nt/Programmer/Anaconda/envs/py37_keras/lib/python3.7/site-packages/tensorflow_core/python/training/tracking/base.py", line 457, in _method_wrapper
    result = method(self, *args, **kwargs)
  File "/home/m1nt/Programmer/Anaconda/envs/py37_keras/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py", line 1592, in _compile_weights_loss_and_weighted_metrics
    self.total_loss = self._prepare_total_loss(masks)
  File "/home/m1nt/Programmer/Anaconda/envs/py37_keras/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py", line 1652, in _prepare_total_loss
    per_sample_losses = loss_fn.call(y_true, y_pred)
  File "/home/m1nt/Programmer/Anaconda/envs/py37_keras/lib/python3.7/site-packages/tensorflow_core/python/keras/losses.py", line 220, in call
    y_pred, y_true)
  File "/home/m1nt/Programmer/Anaconda/envs/py37_keras/lib/python3.7/site-packages/tensorflow_core/python/ops/losses/util.py", line 79, in squeeze_or_expand_dimensions
    is_last_dim_1 = math_ops.equal(1, array_ops.shape(y_pred)[-1])
  File "/home/m1nt/Programmer/Anaconda/envs/py37_keras/lib/python3.7/site-packages/tensorflow_core/python/ops/array_ops.py", line 898, in _slice_helper
    name=name)
  File "/home/m1nt/Programmer/Anaconda/envs/py37_keras/lib/python3.7/site-packages/tensorflow_core/python/ops/array_ops.py", line 1064, in strided_slice
    shrink_axis_mask=shrink_axis_mask)
  File "/home/m1nt/Programmer/Anaconda/envs/py37_keras/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_array_ops.py", line 9535, in strided_slice
    shrink_axis_mask=shrink_axis_mask, name=name)
  File "/home/m1nt/Programmer/Anaconda/envs/py37_keras/lib/python3.7/site-packages/tensorflow_core/python/framework/op_def_library.py", line 742, in _apply_op_helper
    attrs=attr_protos, op_def=op_def)
  File "/home/m1nt/Programmer/Anaconda/envs/py37_keras/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py", line 595, in _create_op_internal
    compute_device)
  File "/home/m1nt/Programmer/Anaconda/envs/py37_keras/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 3322, in _create_op_internal
    op_def=op_def)
  File "/home/m1nt/Programmer/Anaconda/envs/py37_keras/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 1786, in __init__
    control_input_ops)
  File "/home/m1nt/Programmer/Anaconda/envs/py37_keras/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 1622, in _create_c_op
    raise ValueError(str(e))
ValueError: slice index -1 of dimension 0 out of bounds. for 'loss/centernet_loss_loss/strided_slice' (op: 'StridedSlice') with input shapes: [0], [1], [1], [1] and with computed input tensors: input[1] = <-1>, input[2] = <0>, input[3] = <1>.

I don't know how to solve it, so I want to ask for your help.
Thank you! Have a nice day!

When the model training is completed, what is the loss value?
The value of htloss, wh loss and reg loss.
My detection performance is very poor. I want to see if there is a loss problem.
Already stared .
Thank u.
: )

Has anyone have happened this case，my loss always in 5 range，while I testing my image
we got too low score and not correct bounding-box

Which the range of this Loss is fine ?

I am getting the AP scores as 0 on all classes on the validation set of my custom dataset (using --compute-val-loss). I have tried reducing the initial learning rate but it still remains zero. Any thoughts on why this is happening ?