Giter Club home page Giter Club logo

Comments (4)

github-actions avatar github-actions commented on June 2, 2024

👋 Hello @flarota, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

Status

Ultralytics CI

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

from ultralytics.

glenn-jocher avatar glenn-jocher commented on June 2, 2024

Hello!

It's quite normal for object detection models like YOLOv8 to occasionally confuse similar classes, especially if the differences are subtle and the model encounters slightly different data during inference than during training.

The scores you're seeing, where the model is confident about a wrong class and gives a much lower score to the correct class, could be influenced by a few factors:

  1. Dataset Imbalance: If one class is more prevalent in the training data, the model might lean towards predicting it more often.
  2. Inter-Class Variability: Similar classes might need more discriminative features, which could be enhanced either by tweaking the architecture or by providing more varied training samples.
  3. Model Confidence Calibration: Sometimes, the softmax layer used in final score prediction could lead to over-confident predictions. This can be addressed by techniques like temperature scaling in the softmax function to refine the confidence estimates.

You could consider reviewing your training dataset to ensure it’s as diverse and balanced as possible and look into methods of model calibration for better probability estimates. Also, experimenting with the --augment train flag and adjusting model hyperparameters could potentially improve the class discrimination.

Hope this gives you a good starting point for troubleshooting!

from ultralytics.

flarota avatar flarota commented on June 2, 2024

Hi @glenn-jocher, thank you for the reply.

I agree with you and I think the cause is a dataset problem.
But I think it's still interesting to understand why it happens and how to deal with it when you can't fix the dataset.

A collegue of mine found another significant example: in a picture, the same objects are found twice by Yolo, the first time with the "correct class" (I call it "class_A") and a second time with the "wrong class" ("class_B").
In both cases, Yolo was "sure" on the predicted class, giving to the other one a low score.

# Class_A Class_B Other classes
1-1 0,70725 0,0091712 5,4535E-05 1,1915E-09
1-2 0,15818 0,52027 1,1003E-05 2,4418E-08
2-1 0,77648 0,0081475 1,5678E-05 3,4846E-10
2-2 0,42299 0,12249 2,3135E-06 1,1411E-08

The object 1 is correctly predicted in "1-1", and incorrectly in "1-2". The same object!
Same situation in "2-1" and "2-2".

In your opinion, how this situation can happen?

Another question: in your point 3, you talked about the final softmax layer. Does Yolo have this layer? Because the sum of scores isn't 1, so I thought it ends with a sigmoid.
Is it possible to access the tensor before this final layer? I tried to identify this layer in YOLOv8x-seg model but I didn't find it, cause the complexity of the network. Could you suggest a way to do it, maybe with a piece of code?

Thanks

from ultralytics.

glenn-jocher avatar glenn-jocher commented on June 2, 2024

Hi there!

It's indeed intriguing to see how the model behaves differently for the same object within the same image. This could be due to variations in the object's context within the image, slight differences in appearance or angle in different instances, or even overlapping detections where the non-maximum suppression (NMS) hasn't fully resolved which detections to keep.

Regarding your question about the softmax layer, YOLO models typically use a sigmoid function in the final layer for object detection tasks, which explains why the scores don't sum up to 1. This is because each class score is treated independently, allowing for multi-label classification.

To access the tensor before the final sigmoid activation, you can modify the model's forward function or use hooks in PyTorch. Here’s a quick example of how you might use a forward hook to access the output before the activation:

import torch

def get_activation(name):
    def hook(model, input, output):
        print(f"{name} activation: {output.detach()}")
    return hook

model = YOLO('yolov8n.pt')  # Load your model
layer_name = 'final_conv'  # You need to specify the correct layer name
getattr(model, layer_name).register_forward_hook(get_activation(layer_name))

# Run your model to see the activations
model.predict('path/to/image.jpg')

This code sets up a hook that prints the output of a specified layer. You'll need to replace 'final_conv' with the actual name of the layer you're interested in.

Hope this helps! 😊

from ultralytics.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.