tteepe / earlybird Goto Github PK

Official Code for "EarlyBird: Early-Fusion for Multi-View Tracking in the Bird's Eye View"

Home Page: https://openaccess.thecvf.com/content/WACV2024W/RWS/html/Teepe_EarlyBird_Early-Fusion_for_Multi-View_Tracking_in_the_Birds_Eye_View_WACVW_2024_paper.html

Python 100.00%

multi-object-tracking multi-view-detection

earlybird's Introduction

Early Bird 🦅

EarlyBird: Early-Fusion for Multi-View Tracking in the Bird's Eye View

Torben Teepe, Philipp Wolters, Johannes Gilg, Fabian Herzog, Gerhard Rigoll

Usage

Getting Started

Install PyTorch with CUDA support

pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu118

Install mmcv with CUDA support

pip install mmcv==2.0.0 -f https://download.openmmlab.com/mmcv/dist/cu118/torch2.1/index.html

Install remaining dependencies
```
pip install -r requirements.txt
```

Training

python main.py fit -c configs/t_fit.yml \
    -c configs/d_{multiviewx,wildtrack}.yml

Testing

python main.py test -c model_weights/config.yaml \
    --ckpt model_weights/model-epoch=35-val_loss=6.50.ckpt

Acknowledgement

Simple-BEV: Adam W. Harley
MVDeTr: Yunzhong Hou

Cite

If you use EarlyBird, please use the following BibTeX entry.

@InProceedings{teepe2023earlybird,
    author    = {Teepe, Torben and Wolters, Philipp and Gilg, Johannes and Herzog, Fabian and Rigoll, Gerhard},
    title     = {Early{B}ird: Early-Fusion for Multi-View Tracking in the Bird's Eye View},
    booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops},
    month     = {January},
    year      = {2024},
    pages     = {102-111}
}

earlybird's People

Contributors

Stargazers

Watchers

Forkers

palmerxiao

earlybird's Issues

Where did you get "self.worldcoord_from_worldgrid_mat" from?

Thank you for your great work, it helps us a lot for our project. We really appreciate it.

We do not know how fixed parameters or matrices was designed? From the config of each dataset or manually setting?
For example, "self.worldcoord_from_worldgrid_mat = np.array([[0, 2.5, -300], [2.5, 0, -900], [0, 0, 1]])" in line 23 of "EarlyBird/EarlyBird/datasets/wildtrack_dataset.py". How to set self.worldcoord_from_worldgrid_mat like that.

And:
grid_y = pos % 480
grid_x = pos // 480
How to know the position divided by 480 but not other number.

Weights

Hello! Thank you for your excellent paper!

In my reproduction, the code "resnet = torchvision.models.resnet18(weights=torchvision.models.ResNet18_Weights.DEFAULT)" is reporting error "AttributeError: module ' AttributeError: module ' torchvision.models' has no attribute 'ResNet18_Weights'", it was solved by changing it to "resnet = torchvision.models.resnet18(pretrained=True)".

After training, run "python main.py test -c model_weights/config.yaml --ckpt model_weights/model-epoch=35-val_loss=6.50.ckpt" command, I got error "error: Parser key "config": File does not exist: '/ home/zhangkeyu/EarlyBird/EarlyBird/model_weights/config.yaml'".
Is there any file missing? Or what should I do?

I hope to reply as soon as possible, this is very important to me, thanks!

Translated with DeepL.com (free version)

Remarks wildtrack_dataset.py

Hi,

I noticed a discrepancy between the get_worldgrid_from_pos function and the instructions provided in the README regarding how positionID should be converted to world coordinates.
Current Implementation:

def get_worldgrid_from_pos(self, pos):
grid_y = pos % 480
grid_x = pos // 480
return np.array([grid_x, grid_y], dtype=int)

In this implementation:

grid_y is calculated using pos % 480.
grid_x is calculated using pos // 480.

According to the README:

The README indicates that the position ID should be interpreted on a 480x1440 grid with X-first indexing, and the formula for converting positionID to world coordinates is:

X = -3.0 + 0.025 * (ID % 480)
Y = -9.0 + 0.025 * (ID // 480)

Discrepancy:

The current implementation appears to be interpreting the grid in a Y-first manner (i.e., grid_y calculated first, then grid_x).
According to the README, the position ID should be interpreted as X-first, meaning the formula should first calculate X (which corresponds to grid_x in the code) using ID % 480 and then calculate Y (which corresponds to grid_y) using ID // 480.

This could lead to incorrect conversions between positionID and grid/world coordinates.
Suggested Fix:

To align with the README, the implementation could be updated to:

def get_worldgrid_from_pos(self, pos):
grid_x = pos % 480 # This corresponds to X in the README
grid_y = pos // 480 # This corresponds to Y in the README
return np.array([grid_x, grid_y], dtype=int)

This should ensure that the conversion between positionID and world/grid coordinates matches the intended X-first indexing described in the README.

Let me know if this makes sense or if there are any other considerations I might have missed.

Thanks!

Model Weights for Testing

Thank you for the great paper.

I have a question about whether the model weights are going to be released.

Best regards,

Results

Dear authors,

I am trying to test the model and see the results, and I tried with different backbone networks [resnet18, resnet50, swin-t] but the results I get are different from the ones mentioned in the paper, for example I am getting a low track/MOTP score. I would appreciate your advice regarding this matter and thank you for your time!

Best wishes,