Reference to the main idea: https://github.com/ahrnbom/ensemble-objdet
The idea is used for ensembling the boxes that have same image_id and category_id.
The test has been conducted using COCO 2017 val dataset and Detectron2.
In order to reproduce what I've got during the test, you can follow the below.
-
Prepare Ubuntu 20.04 in WSL(Windows Subsystem for Linux) 2.
The environment has been tested without using GPU due to WSL limitation.
The environment has been tested using GPU thanks to 21H2 update for Windows 10 Pro. -
Run the command line below to install the packages except for Detectron2.
$ conda env create -f environment_wsl.yml- Activate the environment.
$ conda activate torch- Install Detectron2.
$ git clone https://github.com/facebookresearch/detectron2.git
$ cd detectron2 && git checkout v0.6 && cd ..
$ python -m pip install -e detectron2When having any installation issue with Detectron2 please followed the instruction here.
$ wget http://images.cocodataset.org/zips/val2017.zip
$ wget http://images.cocodataset.org/annotations/annotations_trainval2017.zip
$ mkdir -p datasets/coco
$ unzip val2017.zip -d datasets/coco
$ unzip annotations_trainval2017.zip -d datasets/coco $ python evaluate_pretrained_model.py
$ ls -A1 datasets/coco
coco_instances_results_faster_rcnn_R_50_DC5_1x.json
coco_instances_results_faster_rcnn_R_50_FPN_1x.json
coco_instances_results_faster_rcnn_R_50_FPN_3x.json$ python evaluate_ensemble.py
Loaded output/coco_instances_results_faster_rcnn_R_50_DC5_1x.json
Loaded output/coco_instances_results_faster_rcnn_R_50_FPN_1x.json
Loaded output/coco_instances_results_faster_rcnn_R_50_FPN_3x.json
Saved the ensemble result to output/ensemble_results.json
Preparing for evaluating the ensemble
Loading and preparing results...
DONE (t=0.23s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
COCOeval_opt.evaluate() finished in 8.24 seconds.
Accumulating evaluation results...
COCOeval_opt.accumulate() finished in 0.87 seconds.
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.386
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.572
...Edit path for the models in config.py and repeat step 3 and 4.
| model | box AP (at IoU=0.50:0.95, area=all) | model | box AP (at IoU=0.50:0.95, area=all) | model | box AP (at IoU=0.50:0.95, area=all) |
|---|---|---|---|---|---|
| faster_rcnn_R_50_C4_1x | 33.05873724953157 | faster_rcnn_R_50_C4_1x | 33.05873724953157 | faster_rcnn_R_50_DC5_1x | 35.02630019513202 |
| faster_rcnn_R_50_DC5_1x | 35.02630019513202 | faster_rcnn_R_50_C4_3x | 35.92233833824599 | faster_rcnn_R_50_FPN_1x | 34.35279506894608 |
| faster_rcnn_R_50_FPN_1x | 34.35279506894608 | faster_rcnn_R_101_C4_3x | 38.51233125665915 | faster_rcnn_R_50_FPN_3x | 36.66724655820816 |
| Ensemble | 36.988241172963086 | Ensemble | 39.57683208823928 | Ensemble | 38.61288520614146 |
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent