megvii-research / bbn Goto Github PK

Hi,

thanks for the great work! I have a question about how to compute feature compactness as shown in Figure 6 in the original paper. What I did is to get resnet features (4096 dimensional tensor) before FC layer for each ground truth class, and compute the averaged distance with it's centroid. But the distances I got is generally greater than 50, even 100, as opposed to what's in the paper (<0.5).

Could you share how you computed the compactness for each class?

Thanks,
Issac

If I follow the default setting and run python main/train.py --cfg configs/cifar10.yaml, I cannot achieve similar results as reported in your paper.

I just achieve as below:
Epoch:200 Batch: 0/109 Batch_Loss:0.223 Batch_Accuracy:98.27%
Epoch:200 Batch:100/109 Batch_Loss:0.195 Batch_Accuracy:99.03%
---Epoch:200/200 Avg_Loss:0.220 Epoch_Accuracy:98.77% Epoch_Time: 0.13min---
------- Valid: Epoch:200 Valid_Loss:0.930 Valid_Acc:79.48%-------
--------------Best_Epoch:200 Best_Acc:79.48%--------------
-------------------Train Finished :BBN.CIFAR10.res32.200epoch-------------------
Would you like to help me to reproduce your results?
Furthermore,
What does mean imb_type in lib/dataset/imbalance_cifar.py? Does it be mentioned in your paper? What should I set to reproduce your results?

Looking forward to your reply!

What are the experimental Settings ?
What is the backbone ?
Is it ResNet-32 ?
Can you tell me ?
Thank you

I use the default cifar10.yaml, cifar100.yaml files with imbalance ratio 100 and train from scatch with this codebase. however, I only get 76.58% on CIFAR10 and 41.56% on CIFAR100, which is far from 79.82% on CIFAR10 and 42.56% on CIFAR100 in the paper. So, if there are any differences between the training setting in this codebase and the training setting corresponding to paper results on CIFAR ?

Thank you for your interesting works. Would you mind explaining the purpose of multiplying the mixed feature by 2 after concatenating them? Thanks.
https://github.com/Megvii-Nanjing/BBN/blob/7992e908842f5934f0d1ee3f430d796621e81975/lib/core/combiner.py#L56

Dear Professor,
I want to run a long tail cifar10 on the resnet-32 model with backbone

1. I want to know how to get the long-tail cifar10 dataset, can I get the long-tail data directly by run python train.py?
   or need to run imbalance_cifar.py first?

2. How should I do if I want to run the CE experiment baseline in the article?

I cannot use pytorch to do my research for some reason., so I'm trying to "translate" the code to keras form. Now I've completed the dataloader (data generator in Keras) part, but I'm stuck with putting both balance sample data and reverse sample data in the backbone model to get 2 features.
How can I use Keras "fit" or "compile" to process both forms of data? Or is it possible to use the method using Keras?

Hi，I meet "AttributeError" when running "train.py". Could you please tell me how to fix it ?

Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to ./datasets/imbalance_cifar10/cifar-10-python.tar.gz
212664376it [00:19, 38731722.29it/s]Traceback (most recent call last):
File "/xinfu/code/long_tail/BBN/main/train.py", line 69, in
train_set = eval(cfg.DATASET.DATASET)("train", cfg)
File "/xinfu/code/long_tail/BBN/lib/dataset/imbalance_cifar.py", line 25, in init
img_num_list = self.get_img_num_per_cls(self.cls_num, imb_type, imb_factor)
File "/xinfu/code/long_tail/BBN/lib/dataset/imbalance_cifar.py", line 44, in get_img_num_per_cls
img_max = len(self.data) / cls_num
AttributeError: 'IMBALANCECIFAR10' object has no attribute 'data'

I want to train my dataset with yolov5, but my dataset has a long tail problem. Some categories have little data. Can I use your method to solve this problem? if can't,please give me some advice，thanks a lot

I'm confused about the implementation of fusing features from two branches. In the combiner.py of your code, it seems that features from two branchs are mixed by concatenating. (Related codes are as follows)

        mixed_feature = 2 * torch.cat((l * feature_a, (1-l) * feature_b), dim=1)
        output = model(mixed_feature, classifier_flag=True)
        loss = l * criterion(output, label_a) + (1 - l) * criterion(output, label_b)

However, in the section 4.1 of your paper, it's said that the two features are integrated by element-wise addition.

Is the feature fusing conducted by concatenating or element-wise addition? Or am I misunderstanding anything?(Correct me if I'm wrong) Looking forward to your reply.

mixed_feature = 2 * torch.cat((l * feature_a, (1 - l) * feature_b), dim=1)
output = model(mixed_feature, classifier_flag=True)

according to the code, the target introduced in section 4.3 may can not achieve, as the feature is concatenated followed by only one classifer.

Cumulative learning strategy is proposed to shift the learning focus between the bilateral branches by controlling the weights for features produced by two branches and the classification loss L.

would you mind telling me the reason behind this change?

Hi, I have downloaded iNaturalist2018, but I couldn't download the train_2018.json and val_2018.json. Would you mind to share the two .json files? Thanks in advance!

I encounter errors when I load bbn_res32_cifar model from pre-trained res50 model . It shows that " Unexpected keys in state_dict......". How can I solve it？

Hi,

Thanks for the nice work in this repo!

Could you share your GPU setup and the training time per epoch on iNaturalist'18?

Best,
Rahul

I'm wondering whether BBN coule be introduced into one-stage object detection method, such as retinanet and yolo. Any improvement could be obtained ? Have you guys try?

Thanks

what is the mean of the "2× scheduler" in the section 5.4.2?

The second step of data sampler is "Randomly sample a class according to Pi;". If a random sample manner is implemented, it is seems that there is no use to calculate Pi for each category.
According to the code from https://github.com/Megvii-Nanjing/BBN/blob/7992e908842f5934f0d1ee3f430d796621e81975/lib/dataset/imbalance_cifar.py#L59, I think each category has equal probability to be select to train and it can't be describe as a "reverse sampling".
Is there any misunderstanding? Thanks for your reply

Hello， thank u for the effective method and clean code!
I try to change your code to run on ImageNet-LT, but get a not so good result.
The config is as follow:

NAME: 'BBN.ImageNet.coslr.res50.90epoch'
OUTPUT_DIR: './output/bbn/ImageNet'
SHOW_STEP: 50
SAVE_STEP: 30
VALID_STEP: 1
INPUT_SIZE: (224, 224)
COLOR_SPACE: 'RGB'
CPU_MODE: False

DATASET:
  ROOT: 'Dataset/ILSVRC2012'
  DATASET: 'ImageNet'
  DATA_TYPE: 'JPEG'
  TRAIN_JSON: 'datasets/ImageNet_LT/ImageNet_LT_train.txt'
  VALID_JSON: 'datasets/ImageNet_LT/ImageNet_LT_val.txt'

BACKBONE:  
  TYPE: 'bbn_res50'

MODULE:
  TYPE: 'GAP'

LOSS:
  LOSS_TYPE: 'CrossEntropy'

CLASSIFIER:
  TYPE: 'FC'
  BIAS: True

TRAIN:
  BATCH_SIZE: 256
  MAX_EPOCH: 90
  NUM_WORKERS: 16
  COMBINER:
    TYPE: 'bbn_mix'
  TENSORBOARD:
    ENABLE: False
  SAMPLER:
    TYPE: 'default'
    DUAL_SAMPLER:
      ENABLE: True
      TYPE: 'reverse'
  OPTIMIZER:
    TYPE: 'SGD'
    BASE_LR: 0.1
    MOMENTUM: 0.9
    WEIGHT_DECAY: 5e-4
  LR_SCHEDULER:
     TYPE: 'cosine'
     COSINE_DECAY_END: 0
     WARM_EPOCH: 5
    
TRANSFORMS:
  TRAIN_TRANSFORMS: ("random_resized_crop", "random_horizontal_flip", "color_jitter")
  TEST_TRANSFORMS: ("shorter_resize_for_crop", "center_crop")

TEST:
  BATCH_SIZE: 128
  NUM_WORKERS: 16
  MODEL_FILE: '/home/BBN/models/BBN.ImageNet.res50.90epoch.best_model.pth'

But only get 46.26% on val set, the baseline is about 44%
Have you run BBN on ImageNet and get some results? Can u tell me.
Thank u very much! @ZhouBoyan

Thanks for sharing the codes. I find that implement of mix_feature is different from the paper which uses addition, however, using concat in code. I wonder about the difference and connection of these two methods.

loss定义 和 bbn_mix 有点没有弄明白，感觉作用是一样的。

If I apply it to my own dataset, the above problems will occur.

Hi, could you release the pretrained models for CIFAR-10 and CIFAR-100? I cannot reproduce the results reported in your paper, which means I cannot analyze some informations with BBN

hi, i just use bbn to train in inaturalist2019, the config is same as config of iNaturalist2018.yaml.
thanks to the clean code, it is very easy to get good result with resnet50 baseline 2x.
But i found loss curve is very strage.
train loss at 67 epoch got highest.
Have you gotten the same curve? Could you explain the reason?

Hello, in your code, the feature is aggregated by concatenate
https://github.com/Megvii-Nanjing/BBN/blob/7992e908842f5934f0d1ee3f430d796621e81975/lib/core/combiner.py#L56
But in your paper, it will be aggregated by element-wise addition

I am puzzled about it and hope to get your reply

I've tried to download the pretrarined INaturalist 2018 model but the link https://pan.baidu.com/error/404.html is not working anymore. Could you please share with me working link?

Thanks in advance!

For iNaturalist:
learning rate is decay at 60th and 80th epoch by 0.1 (in paper), but lr is decay at 120 and 160 in your code

could you please check it?