qqwweee / keras-yolo3 Goto Github PK

thanks for you work!
i notice that when i try to convert darknet yolo model to a keras model, there is an error

so did the project miss something or something i forget？

when i use 'python yolo.py',it will remind me to 'Input image filename:',but when i input the image filename(like dog.jpg),it will occur 'NameError: name 'dog' is not defined'.can you help me?thank you. @tanakataiki @qqwweee

Hi, i wrote an mjpeg stream -> object detection -> mjpeg stream service using your yolo object class, it works perfectly fine with 12-15 fps. I would like to limit the detection to 3 classes (person, bicycle, car), i tried to train a new model with these 3 classes, using your voc_annotation.py and train.py. Modified the voc_annotation.py:
classes = ["person","bicycle","car"]
Then removed rows without annotations from 2007_test.txt, 2007_train.txt and 2007_val.txt.
The original could detect cars from far away, but the new model could not detect them, only when they were already sideways. What should i do to increase the detection range without lowering score? (its already at 0.3)

I may be wrong, but shouldn't we use darknet53.conv.74 instead of yolo3.weights when we train a customized dataset?

Hi there, I got the following error:

I'm using Tensorflow 1.5.0. Is this OK?

When I save weights during training using your original code, I got:

InvalidArgumentError (see above for traceback): Input to reshape is a tensor with 12675 values, but the requested shape requires a multiple of 3042
	 [[Node: Reshape_3 = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv2d_59/BiasAdd, Reshape_3/shape)]]

Does this mean I should save model instead of weights? May I ask why in your code "model.save_weights" is used instead of "model.save_path"?

When I saved checkpoints in train.py as keras model, I wasn't able to use it in yolo.py using load_model.

checkpoint = ModelCheckpoint(log_dir + "checkpoint.h5", monitor='val_loss', save_best_only=True)

history = model.fit([image_data, *y_true],
          np.zeros(len(image_data)),
          validation_split=.1,
          batch_size=Batch_Size,
          epochs=10000,
          callbacks=[checkpoint])

When I use yolo.py to test the trained model on images, model is loaded using

self.yolo_model = load_model(model_path, compile=False)

I got this error:

2018-05-10 05:47:25.060203: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Traceback (most recent call last):
  File "/home/jin/workspace/Intersection_TrafficFlow/detection/keras-yolo3-noaug/keras-yolo3-1/yolo.py", line 229, in <module>
    detect_img(YOLO())
  File "/home/jin/workspace/Intersection_TrafficFlow/detection/keras-yolo3-noaug/keras-yolo3-1/yolo.py", line 58, in __init__
    self.boxes, self.scores, self.classes = self.generate()
  File "/home/jin/workspace/Intersection_TrafficFlow/detection/keras-yolo3-noaug/keras-yolo3-1/yolo.py", line 80, in generate
    self.yolo_model = load_model(model_path, compile=False)
  File "/home/jin/anaconda3/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/models.py", line 243, in load_model
    model = model_from_config(model_config, custom_objects=custom_objects)
  File "/home/jin/anaconda3/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/models.py", line 317, in model_from_config
    return layer_module.deserialize(config, custom_objects=custom_objects)
  File "/home/jin/anaconda3/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/layers/__init__.py", line 55, in deserialize
    printable_module_name='layer')
  File "/home/jin/anaconda3/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 144, in deserialize_keras_object
    list(custom_objects.items())))
  File "/home/jin/anaconda3/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/engine/topology.py", line 2524, in from_config
    process_node(layer, node_data)
  File "/home/jin/anaconda3/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/engine/topology.py", line 2483, in process_node
    layer(input_tensors, **kwargs)
  File "/home/jin/anaconda3/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/engine/topology.py", line 619, in __call__
    output = self.call(inputs, **kwargs)
  File "/home/jin/anaconda3/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/layers/core.py", line 663, in call
    return self.function(inputs, **arguments)
  File "/home/jinz1/Jin/Intersection_TrafficFlow/detection/keras-yolo3-noaug/keras-yolo3-1/yolo3/model.py", line 347, in yolo_loss
NameError: name 'yolo_head' is not defined

Thank you very much for your help and for creating this repo!

I used my own data to train the model detect the specific class, and the model run well in darknet. But when I try to convert the model , I can not detect any object. I use my own cfg and my own .weight file when convert the model, and I change the path of model path and class path. Is there any place need to change?

Please see here https://github.com/zubairahmed-ai/yolov3

What kind of changes do we need to make to enable batch inference? Currently the boxes have the shape
(num_detections, 4) instead of (None, num_detections, 4)

Unable to run it on video

ImportError: No module named 'VideoCapture

Thanks for your nice work, there might exist some tiny flaw in model.py files when I exam the model summry:

input = Input(shape=(416,416,3))
model = yolo_body(input, 9, 80)
model.summary()

it will have key error about dict
so I changed it :
kk = None
try:
kk = kwargs['strides']
except:
kk = None
#some mistake here
darknet_conv_kwargs['padding'] = 'VALID' if kk==(2,2) else 'SAME'
and it works
I hope it will helpful

I don't know why this problem occurs.My train image number is 356 and val number is 46

As i trained the model with voc_annotation.py data, after finished training ,got the model of my_yolo.h5 ,
then used the test yolo.py to test the new model ,got the error as below , plz help how to fix the issue ?

replace as
#self.model_path = 'model_data/yolo.h5'
self.model_path = 'model_data/my_yolo.h5'
self.anchors_path = 'model_data/yolo_anchors.txt'
self.classes_path = 'model_data/coco_classes.txt'
self.score = 0.3
self.iou = 0.5

error

InvalidArgumentError Traceback (most recent call last)
/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
1326 try:
-> 1327 return fn(*args)
1328 except errors.OpError as e:

/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py in _run_fn(feed_dict, fetch_list, target_list, options, run_metadata)
1311 return self._call_tf_sessionrun(
-> 1312 options, feed_dict, fetch_list, target_list, run_metadata)
1313

/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py in _call_tf_sessionrun(self, options, feed_dict, fetch_list, target_list, run_metadata)
1419 self._session, options, feed_dict, fetch_list, target_list,
-> 1420 status, run_metadata)
1421

/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors_impl.py in exit(self, type_arg, value_arg, traceback_arg)
515 compat.as_text(c_api.TF_Message(self.status.status)),
--> 516 c_api.TF_GetCode(self.status.status))
517 # Delete the underlying status object from memory otherwise it stays alive

InvalidArgumentError: Input to reshape is a tensor with 12675 values, but the requested shape requires a multiple of 43095
[[Node: Reshape_3 = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv2d_59/BiasAdd, Reshape_3/shape)]]

During handling of the above exception, another exception occurred:

InvalidArgumentError Traceback (most recent call last)

I meet a strange question when I run the train.py, when I run the python train.py first, it will be appear the question ,but I run the code second, it will be ok. Can you help me fix it?(other files all run successful)

@qqwweee @tanakataiki hi,when i use 'python train.py' with my datasets(2 classes),i get the following information,cause the number of class different,some weights of convolutional layers did not load successfully，this can be trained and generate models,but the final test results deteriorated.how should I solve this problem？thank you so much.
/home/sugon/anaconda3/lib/python3.5/site-packages/keras/engine/topology.py:3462: UserWarning: Skipping loading of weights for layer conv2d_59 due to mismatch in shape ((1, 1, 1024, 21) vs (255, 1024, 1, 1)).
weight_values[i].shape))
/home/sugon/anaconda3/lib/python3.5/site-packages/keras/engine/topology.py:3462: UserWarning: Skipping loading of weights for layer conv2d_59 due to mismatch in shape ((21,) vs (255,)).
weight_values[i].shape))
/home/sugon/anaconda3/lib/python3.5/site-packages/keras/engine/topology.py:3462: UserWarning: Skipping loading of weights for layer conv2d_67 due to mismatch in shape ((1, 1, 512, 21) vs (255, 512, 1, 1)).
weight_values[i].shape))
/home/sugon/anaconda3/lib/python3.5/site-packages/keras/engine/topology.py:3462: UserWarning: Skipping loading of weights for layer conv2d_67 due to mismatch in shape ((21,) vs (255,)).
weight_values[i].shape))
/home/sugon/anaconda3/lib/python3.5/site-packages/keras/engine/topology.py:3462: UserWarning: Skipping loading of weights for layer conv2d_75 due to mismatch in shape ((1, 1, 256, 21) vs (255, 256, 1, 1)).
weight_values[i].shape))
/home/sugon/anaconda3/lib/python3.5/site-packages/keras/engine/topology.py:3462: UserWarning: Skipping loading of weights for layer conv2d_75 due to mismatch in shape ((21,) vs (255,)).
weight_values[i].shape))

When I run the python yolo.py an InvalidArgumentError occurs:

I can't use the convert.py script, since the module "configparser" isn't found.
Where do I get this module from?

I was wondering how difficult it would be to adjust your code to run on the plaidml backend (https://github.com/plaidml/plaidml), which supports AMD GPUs on Mac. plaidml is currently only compatible with Keras 2.0.8, and it seems that some of your code would need to be adapted. There are also a few lines of code that use the tf backend and not the keras backend in the model.py file, i.e. in the yolo_eval and yolo_loss functions. Compared to other implementations of yolo in keras, your code seems to be much closer to be compatible with plaidml.

I train the yolov3 model based on darknet with my own dataset. When i convert my yolo.weights to .h5 format, the error is followed:
Traceback (most recent call last):
File "convert.py", line 242, in
_main(parser.parse_args())
File "convert.py", line 140, in _main
buffer=weights_file.read(weights_size * 4))
TypeError: buffer is too small for requested array

How should i do to solve my problem?
Thanks and Regards.

I followed your instruction to run 'python yolo.py' , then I typed the test image's name. It went wrong. Like this:

@qqwweee

I got the problem:
$ python train.py
File "train.py", line 125
[*model_body.output, *y_true])
^
SyntaxError: invalid syntax

I have read the other issues #22 but got nothing helpful. I have modified my train.txt.

When I use the python yolo_video.py the frame rate is really slow(3-4s per picture), because the gpu(1080ti) is not utilized. I've installed precompiled tensorflow-gpu.
Do you have any idea how to utilize the gpu?

hi, @tanakataiki ,when I execute 'python train.py' in my personal notebook is able to train,but using the same method on the server causes the following error:
Traceback (most recent call last):
File "train.py", line 158, in
_main()
File "train.py", line 35, in _main
load_pretrained=True, freeze_body=True)
File "train.py", line 122, in create_model
model_body.load_weights(weights_path, by_name=True, skip_mismatch=True)
TypeError: load_weights() got an unexpected keyword argument 'skip_mismatch'
Is this related to the python version or other questions？

when i run python convert.py yolov3.cfg yolov3.weights model_data/yolo.h5
i meet a problem
Using TensorFlow backend.
Loading weights.
('Weights Header: ', array([ 0, 2, 0, 32013312, 0], dtype=int32))
Parsing Darknet config.
Traceback (most recent call last):
File "convert.py", line 243, in
_main(parser.parse_args())
File "convert.py", line 75, in _main
unique_config_file = unique_config_sections(config_path)
File "convert.py", line 49, in unique_config_sections
output_stream.write(line)
TypeError: unicode argument expected, got 'str'
how should i do ？

I want to use the code to detect the ID card. Right now I have some data collected and labeled. Is it OK I run train.py with only my dataset, which only has 1 class: id card?

I'm afraid that yolov3 covers multi-scale like v2 but it's multiple of 32 so Added PR.

Thanks for this great repo! I wanted to use train.py to overfit to a small sample. In fact the loss goes down to ~25 but the predicted boxes are bad. Did you manage to overfit/train with good results? Here is what I am using:

train.txt

img_000.jpg 119,349,787,844,0 811,249,1476,846,0
img_001.jpg 7,18,64,76,0 88,24,147,77,0 144,37,190,84,0 163,24,217,78,0 ,151,146,0 128,137,182,190,0 162,107,208,156,0 184,149,214,178,0 147,188,200,225,0 8,139,221,0 38,156,106,214,0 22,96,91,157,0 75,76,151,149,0
img_002.jpg 23,37,199,201,0
img_003.jpg 14,194,273,448,1 162,80,396,330,1 269,203,525,459,1
img_004.jpg 19,79,116,169,1 76,52,154,147,1 118,74,206,169,1

labels.txt

tomato
apple

when I run yolo.py, I meet the problem like title. Can you teach me how to solve the problem? Thank you.

When I trained my own data,ran train.py,this error occurred in preprocess_true_boxes.
My true_boxes shape of (33402, 20, 5) and input_shape is (416,416)
Traceback (most recent call last): File "train.py", line 161, in <module> main() File "train.py", line 151, in main y_true = preprocess_true_boxes(box_data, input_shape, anchors, len(class_names)) File "/home/sk39/workspace/cheng/keras-yolo3-master/yolo3/model.py", line 266, in preprocess_true_boxes y_true[l][b, j, i, n, 5+c] = 1 IndexError: index 15 is out of bounds for axis 4 with size 15

Could you tell me how to solve it?
Thank you!

Look forward to it!

Could you list the specific requirements just like the vision about OpenCV， Python， Tensorflow-gpu， Keras and so on. Thank your great work.

I meet a question for a long time,when I run python train.py, it will be as follows.

I try find this question,finally, I find this as follows

it is amazing!, I write the 'model_body = yolo_body(image_input, num_anchors, num_classes)' twice,it will be ok. but I don't know why. can you help me?

While converting the official tiny-YoloV3 (https://github.com/pjreddie/darknet/blob/master/cfg/yolov3-tiny.cfg) the following error occurs:

when I run train.py, I meet the problem like title. Can you teach me how to solve the problem? what's the meaning of multiple of 12675? Thank you.

what about the version of keras?and tensorflow?

@qqwweee
I mistakenly merged so wait for a bit or is it possible that you revert merge?
I am sorry for that, but it's not so big change except extra files.

Thank you very much for this great repo! I have a few questions for training.

Question 1:

I have a pretrained weight ('five_class_weight.h5) with 5 classes (car, bus, minibus, cyclist, truck) , and I'm trying to fine tune the result on 40 images with 2 classes (car, bus). Is there other changes to the files that I missed?

The pretrained weights could detect 80% cars my dataset. I used this dataset as training dataset, but after fine tune for 300 epochs it cannot detect any cars in the training dataset. My guess is it won't get worse than before if I'm doing it correctly.

First I converted weights for the 5 class model:

python convert.py five_class.cfg five_class.weights model_data/five_class_weight.h5

I'm still using the 5 classes for classes path, since this network will have a smaller change on the last layer.

train.py

```javascript
def _main():
    anchors_path = 'model_data/yolo_anchors.txt'
    classes_path = 'model_data/five_classes.txt'


def get_training_data(annotation_path, data_path, input_shape, max_boxes=100, load_previous=True):
    if load_pretrained:
           weights_path = os.path.join('model_data', 'five_class_weight.h5')

Question 2:

What are the difference between my_yolo.h5 and trained_weights.h5 ? Which one is should I use when use yolo.py for detection?

def _main():
    ......
    output_path = 'model_data/my_yolo.h5'

def train(model, image_data, y_true, log_dir='logs/'):
   ......
    model.save_weights(log_dir + 'trained_weights.h5')

yolo.py

self.model_path = 'model_data/my_yolo.h5'
self.anchors_path = 'model_data/yolo_anchors.txt'
self.classes_path = 'model_data/five_classes.txt'

Question 3: During training, I got the following warning. What does these warning mean? Would these affect the training result?

/home/jinz1/anaconda/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/engine/topology.py:3462: UserWarning: Skipping loading of weights for layer conv2d_59 due to mismatch in shape ((1, 1, 1024, 75) vs (255, 1024, 1, 1)).
  weight_values[i].shape))
/home/jinz1/anaconda/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/engine/topology.py:3462: UserWarning: Skipping loading of weights for layer conv2d_59 due to mismatch in shape ((75,) vs (255,)).
  weight_values[i].shape))
/home/jinz1/anaconda/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/engine/topology.py:3462: UserWarning: Skipping loading of weights for layer conv2d_67 due to mismatch in shape ((1, 1, 512, 75) vs (255, 512, 1, 1)).
  weight_values[i].shape))
/home/jinz1/anaconda/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/engine/topology.py:3462: UserWarning: Skipping loading of weights for layer conv2d_67 due to mismatch in shape ((75,) vs (255,)).
  weight_values[i].shape))
/home/jinz1/anaconda/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/engine/topology.py:3462: UserWarning: Skipping loading of weights for layer conv2d_75 due to mismatch in shape ((1, 1, 256, 75) vs (255, 256, 1, 1)).
  weight_values[i].shape))
/home/jinz1/anaconda/envs/keras-yolo3-noaug/lib/python3.6/site-packages/keras/engine/topology.py:3462: UserWarning: Skipping loading of weights for layer conv2d_75 due to mismatch in shape ((75,) vs (255,)).
  weight_values[i].shape))
Train on 32 samples, validate on 4 samples
Epoch 1/300
32/32 [==============================] - 3s 108ms/step - loss: 4225.3423 - val_loss: 4649.4351
...
Epoch 300/300
32/32 [==============================] - 1s 23ms/step - loss: 27.5249 - val_loss: 49.5864

I trained for more epochs later, but the detection result has lower accuracy than the pretrained model. After fine tune, 4 out of 10 cars are detected and labeled as "truck". Before fine tune, 80% cars can be correctly detected.

Epoch 1780/30000
32/32 [==============================] - 1s 25ms/step - loss: 16.4085 - val_loss: 38.6537
Epoch 01780: early stopping

Sorry for this long post of questions. Thank you very much for your help!

Hi guys!

I am currently getting 31/30/27 (mAP-50) and 55/53/49.5 (mAP) with this implementation, which is a bit lower than what the paper claims. I was wondering if anyone else experienced this and might have some intuition w.r.t. what's causing the drop?

Thanks!
Nir

Have you got the reason why
box_xy = (box_xy + xy_offset + 1) / conv_dims works

When I trained my own data,ran train.py,this error occurred.

I think there is no problem with these data, because I have successfully trained in yolo2. I also modified the classes and filters in the configuration file.
Then run python convert.py my_yolo3.cfg yolov3.weights model_data/yolo.h5.This command is no problem.But when I run train.py,this error occurred.
Is it my operation that leads to this problem?Thank you.

@qqwweee
Hello, I want to train this model more accurately just in class "person" . So I use MOT17 datasets in https://motchallenge.net/data/MOT17Det/ and these datasets are changed to VOC format. Then I use voc_annotation.py to generate 'train.txt'. In voc_annotation.py, I change classes = ["person"] because I only want to detect multiple people more accurately. And I write just "person" in /model_data/voc_classes.txt .Then, when I run train.py, I have some problems:

Traceback (most recent call last): File "/home/x/keras-yolo3/qi_train.py", line 178, in <module> _main() File "/home/x/keras-yolo3/qi_train.py", line 45, in _main load_pretrained=True, freeze_body=True) File "/home/xkeras-yolo3/qi_train.py", line 132, in create_model model_body.load_weights(weights_path, by_name=True, skip_mismatch=True) File "/home/x/anaconda3/lib/python3.6/site-packages/keras/engine/topology.py", line 2653, in load_weights reshape=reshape) File "/home/x/anaconda3/lib/python3.6/site-packages/keras/engine/topology.py", line 3439, in load_weights_from_hdf5_group_by_name reshape=reshape) File "/home/x/anaconda3/lib/python3.6/site-packages/keras/engine/topology.py", line 3140, in preprocess_weights_for_loading elif layer_weights_shape != weights[0].shape: IndexError: list index out of range

Could you please tell me how can I solve the problem? Or how to finetune the yolo.h5 in just one class("person")? Thank you very much!

thank you for your work.

The following error is shown while converting yolov3.weights to h5 format. How can I solve this?

Traceback (most recent call last):

File "convert.py", line 237, in
_main(parser.parse_args())
File "convert.py", line 134, in _main
buffer=weights_file.read(weights_size * 4))

TypeError: buffer is too small for requested array

Highly appreciate for qqwweee's work. ^_^

With same cfg(coco, 80 classes, yolov3.cfg), and same weights to darknet yolov3 (yolov3.weights), but the detection results and confidence level are much lower than darknet version. With Python 2.7, tensorflow_gpu 1.7, keras

original version result:
person: 99%
person: 99%
person: 99%
person: 97%
person: 94%
person: 92%
person: 90%
person: 79%
person: 78%
person: 66%
person: 66%
person: 65%
person: 65%
person: 59%
person: 58%
person: 51%

results from keras-yolo3:

(416, 416, 3)
Found 10 boxes for img
('person 0.48', (798, 518), (1028, 951))
('person 0.48', (1092, 33), (1234, 263))
('person 0.54', (1416, 155), (1633, 643))
('person 0.56', (1312, 19), (1470, 347))
('person 0.73', (1011, 771), (1273, 1077))
('person 0.85', (413, 697), (663, 1068))
('person 0.86', (1550, 892), (1722, 1071))
('person 0.88', (1369, 155), (1513, 634))
('person 0.89', (1632, 123), (1798, 587))
('person 0.96', (1355, 704), (1552, 1009))

here is the image for testing

9ef8628

Hi, thanks for the awesome repo. 😀
I am currently using this repo to process video, and I noticed that commit #6 caused the framerate to drop by 1/3 on average as compared to commit #5. Is it because of the ZeroPadding2D fix?

Hi,I had read the code,but I still cannot summaries the formula of the loss function of each part(bounding box,class_loss,object_loss),and beyond that I want to know the probability of the object and the class,thank you for your kindness!

Hi!
I want to train the net with no layer freezed, so I set the 'freeze_body' to False when I start training, but it raise an error of 'ran out of memory', how can I solve this problem?

And I can not understand why did you freeze almost all of the layers(except last 3 layers), is this mean that only last 3 layers can be trained when I freeze the model body? Is there any differences between this repo and Darknet in training

Thanks!

I want to train by myself, and I follow the instruction of training part. But when I run the train.py, this error happen:

$ python train.py
File "train.py", line 125
[*model_body.output, *y_true])
^
SyntaxError: invalid syntax

It seems that the * is judged as the SyntaxError... How to modify this? I am fresh to keras..