Transfer Learning for custom class number about yolov3-tf2 HOT 17 CLOSED

• fine tune means freeze darknet weights and then fine tune pre-trained weights for rest of the model
• transfer darknet means freeze all the weights from darknet and train top layers from scratch (random initialized)
• frozen means entire model is frozen, so training doesn't do anything

from yolov3-tf2.

@RyanYang1221 training from scratch can be done now with the num_classes flag.

If you want to do transfer learning you will have to customize train.py to do something like this

model = YoloV3(416, training=True, classes=2)
model_pretrained = YoloV3(416, training=True, classes=80)
model_pretrained.load_weights('pretrained_weights')
model.get_layer('yolo_darknet').set_weights(model_pretrained.get_layer('yolo_darknet').get_weights())

and then plug the transformed model in.
after its trained you can load it with num_classes=2

from yolov3-tf2.

@RyanYang1221 and @ilyes-hamrouni I am also trying to do the same with a single custom class, but I can't seem to figure out how to reference the 'pretrained_weights' and which part of train.py to imbed the following codes:

model_pretrained = YoloV3(416, training=True, classes=80)
model_pretrained.load_weights('pretrained_weights')
model.get_layer('yolo_darknet').set_weights(model_pretrained.get_layer('yolo_darknet').get_weights())

Can you share if you have progressed in this custom training ? THANKS!

I have tried referencing 'pretrained weights' as './checkpoints/yolov3.tf' but got a weights loading error - Shapes (1, 1, 1024, 18) and (1, 1, 1024, 255)

from yolov3-tf2.

I decided to go with the Darknet, its even more efficient and easy

from yolov3-tf2.

something like this should work, for me everything is fine. I have some trouble with the validation, but thats another topic.

num_classes = 6 #custom number of classes
dataset_size = 500 #amount of images
batch_size = 8
epochs = 100
classes = 'custom.classes' #file with you classnames
learning_rate = 1e-3
yolo_size = 416

model = YoloV3(yolo_size, training=True, classes=num_classes)
anchors = yolo_anchors
anchor_masks = yolo_anchor_masks

dataset = dataset.load_tfrecord_dataset(tfrecord, classes)
dataset = full_dataset.shuffle(buffer_size=1024)
dataset = full_dataset.batch(batch_size)
dataset = full_dataset.map(lambda x, y: (
        dataset.transform_images(x, yolo_size),
        dataset.transform_targets(y, anchors, anchor_masks, num_classes)))

optimizer = tf.keras.optimizers.Adam(lr=learning_rate)
loss = [YoloLoss(anchors[mask], classes=num_classes)
            for mask in anchor_masks]

   
model_pretrained = YoloV3(yolo_size, training=True, classes=80)
        model_pretrained.load_weights('./weights/yolov3.tf')
        model.get_layer('yolo_darknet').set_weights(model_pretrained.get_layer('yolo_darknet').get_weights())

model.compile(optimizer=optimizer, loss=loss)


history = model.fit(train_dataset, steps_per_epoch=dataset_size // batch_size,epochs=epochs)

from yolov3-tf2.

@zzh8829 could you guide in creating a weight_convert file for darknet (feature extractor) pretrained weights according to the model defined here?

from yolov3-tf2.

I also have the same request. I want to train my own dataset with only 2 classes. Is it possible to do transfer learning? Or could you recommend some way?

from yolov3-tf2.

I want to do transfer learning too with 2 classes only. Can someone explain to me the differences between transfer darknet, frozen and fine tune

from yolov3-tf2.

@StrongRay I'm facing the same problem, any progress?

ValueError: Layer weight shape (1, 1, 1024, 255) not compatible with provided weight shape (1, 1, 1024, 66)

from yolov3-tf2.

Nope. I did not try this for awhile. In any case, TF2.0 RC 1 was released last week. I am waiting for the dust to settle before ironing this out.

from yolov3-tf2.

Any updates on this?
Thanks

from yolov3-tf2.

This man did a great job with this yolo version is clean and easy to use , but there are some weird problems, and I guess are from tensorflow side

I ran the following line

python detect.py --image PATH_TO_IMAGE --weights ./checkpoints/yolov3_train_EPOCHNUMBER.tf --classes ./data/TXT_WITH_NAMES

(Make sure you are using the requirements.txt specs)
Everything 'worked' (I got a pretty decent customized prediction), however I'm still getting this kind of error kind of warning :

Exception ignored in: <function _CheckpointRestoreCoordinator.del at 0x7fee57e52a60>
Traceback (most recent call last):
File "/home/h/yolo_v3/yolov3-tf2/enviroment/yolito/lib/python3.7/site-packages/tensorflow/python/training/tracking/util.py", line 244, in del
.format(pretty_printer.node_names[node_id]))
File "/home/h/yolo_v3/yolov3-tf2/enviroment/yolito/lib/python3.7/site-packages/tensorflow/python/training/tracking/util.py", line 93, in node_names
path_to_root[node_id] + (child.local_name,))
File "/home/h/yolo_v3/yolov3-tf2/enviroment/yolito/lib/python3.7/site-packages/tensorflow/python/training/tracking/object_identity.py", line 76, in getitem
return self._storage[self._wrap_key(key)]
KeyError: (<tensorflow.python.training.tracking.object_identity._ObjectIdentityWrapper object at 0x7fede47f89b0>,)

Tensorflow doc has this:
https://www.tensorflow.org/tutorials/keras/save_and_restore_models

from yolov3-tf2.

@RyanYang1221 and @ilyes-hamrouni I am also trying to do the same with a single custom class, but I can't seem to figure out how to reference the 'pretrained_weights' and which part of train.py to imbed the following codes:

model_pretrained = YoloV3(416, training=True, classes=80)
model_pretrained.load_weights('pretrained_weights')
model.get_layer('yolo_darknet').set_weights(model_pretrained.get_layer('yolo_darknet').get_weights())

Can you share if you have progressed in this custom training ? THANKS!

I have tried referencing 'pretrained weights' as './checkpoints/yolov3.tf' but got a weights loading error - Shapes (1, 1, 1024, 18) and (1, 1, 1024, 255)

This is what i observed during my experiment..
The network is expecting (1, 1, 1024, 255) shape... the number of classes given as the input affects the weights..

output --> (1, 1, 1024, X)

It seems like the current code is expecting to get 80 classes as the input... I'm still looking into the code.

UPDATE:

def YoloOutput(filters, anchors, classes, name=None):
    def yolo_output(x_in):
        x = inputs = Input(x_in.shape[1:])
        x = DarknetConv(x, filters * 2, 3)
        x = DarknetConv(x, anchors * (classes + 5), 1, batch_norm=False)
        x = Lambda(lambda x: tf.reshape(x, (-1, tf.shape(x)[1], tf.shape(x)[2],
                                            anchors, classes + 5)))(x)
        return tf.keras.Model(inputs, x, name=name)(x_in)
    return yolo_output

Since there are 3 outputs from the yolov3 model yolov3_tf2/models/YoloV3 .. hence, the output is (classes + 5) x 3

from yolov3-tf2.

@bamwani It's not more efficient, especially when you have TensorRT, and you can train on the cloud with the dataset really easy. also tf.serving

from yolov3-tf2.

@RyanYang1221 and @ilyes-hamrouni I am also trying to do the same with a single custom class, but I can't seem to figure out how to reference the 'pretrained_weights' and which part of train.py to imbed the following codes:

model_pretrained = YoloV3(416, training=True, classes=80)
model_pretrained.load_weights('pretrained_weights')
model.get_layer('yolo_darknet').set_weights(model_pretrained.get_layer('yolo_darknet').get_weights())

Can you share if you have progressed in this custom training ? THANKS!
I have tried referencing 'pretrained weights' as './checkpoints/yolov3.tf' but got a weights loading error - Shapes (1, 1, 1024, 18) and (1, 1, 1024, 255)

This is what i observed during my experiment..
The network is expecting (1, 1, 1024, 255) shape... the number of classes given as the input affects the weights..

output --> (1, 1, 1024, X)
N, num_class output: x
1 18
2 21
3 24
4 27
...
80 255

It seems like the current code is expecting to get 80 classes as the input... I'm still looking into the code.

UPDATE:

def YoloOutput(filters, anchors, classes, name=None):
    def yolo_output(x_in):
        x = inputs = Input(x_in.shape[1:])
        x = DarknetConv(x, filters * 2, 3)
        x = DarknetConv(x, anchors * (classes + 5), 1, batch_norm=False)
        x = Lambda(lambda x: tf.reshape(x, (-1, tf.shape(x)[1], tf.shape(x)[2],
                                            anchors, classes + 5)))(x)
        return tf.keras.Model(inputs, x, name=name)(x_in)
    return yolo_output

Since there are 3 outputs from the yolov3 model yolov3_tf2/models/YoloV3 .. hence, the output is (classes + 5) x 3

So there is any solution to it so far?

from yolov3-tf2.

try taking a look at my last post here: #107 where I managed to add the authors suggestion into the train.py code.

from yolov3-tf2.

Here is a complete tutorial on transfer learning with custom number of classes
https://github.com/zzh8829/yolov3-tf2/blob/master/docs/training_voc.md

from yolov3-tf2.