About the AUC about chexpert HOT 7 CLOSED

Of course! cuz the step of the save best is 2600, I guess that you have used the pre-trained model. Can you show more details of your config of the training, like example.json? @BUAAtao

from chexpert.

I didn't change other parameters except for train.csv's path. I also trained the model by 4 1080ti.

{
"train_csv": "/data/Chexpert/CheXpert-v1.0/train.csv",
"dev_csv": "/data/Chexpert/CheXpert-v1.0/valid.csv",
"backbone": "densenet121",
"width": 512,
"height": 512,
"long_side": 512,
"fix_ratio": true,
"pixel_mean": 128.0,
"pixel_std": 64.0,
"use_pixel_std": true,
"use_equalizeHist": true,
"use_transforms_type": "Aug",
"gaussian_blur": 3,
"border_pad": "pixel_mean",
"num_classes": [1,1,1,1,1],
"batch_weight": true,
"enhance_index": [2,6],
"enhance_times": 2,
"pos_weight": [1,1,1,1,1],
"train_batch_size": 56,
"dev_batch_size": 56,
"pretrained": true,
"log_every": 10,
"test_every": 100,
"epoch": 3,
"norm_type": "BatchNorm",
"global_pool": "AVG_MAX",
"fc_bn": true,
"attention_map": "FPA",
"lse_gamma": 0.5,
"fc_drop": 0,
"optimizer": "Adam",
"criterion": "BCE",
"lr": 0.0001,
"lr_factor": 0.1,
"lr_epochs": [2],
"momentum": 0.9,
"weight_decay": 0.0,
"best_target": "auc",
"save_top_k": 3,
"save_index": [0,1,2,3,4]
}

from chexpert.

And I didn't use the pre_trained .pth, why the step is 2600 is that I used the best performance model (best3.ckpt) to test. And the best1.ckpt's step is 2000.

from chexpert.

fine.In my opinion,

1. larger batch size will get better results(I have tried).
2. the save_index is the importance of saving models for different diseases. (that means that Your preservation strategy focuses more on which diseases), the Cardiomegaly and Atelectasis are more unstable.
   3.try the "enhance_index": [2,6],
   "enhance_times": 1, in the config file.(that means oversampling) @BUAAtao

from chexpert.

Thanks. Based on your advice, I have changed "enhance_times" to 1. And get a better result as follow:
Save best is step : 8000 AUC : 0.8934086684552691

Cardiomegaly auc 0.8524423337856174
Edema auc 0.9329415310427969
Consolidation auc 0.9440104166666667
Atelectasis auc 0.84816
Pleural_Effusion auc 0.9105009191176471

I couldn't change the batch size because of the nums of GPU. I also found that he Cardiomegaly and Atelectasis were unstable and don't know how to do some changes to "save_index".Could you tell me more details about "save_index"? Then I would try a few more times to get a better restult.

from chexpert.

you can find the "save_index" is [0,1,2,3,4] in the config/example.That means the best model you saved is the higher mean of five diseases. If you set it as [0,3], it will save the model According to the average of the two results. (Cardiomegaly and Atelectasis) @BUAAtao

from chexpert.

Thanks.I changed “save_index” to [0,3] and trained the model again.The result is better. But the result is still not great.I would try other ways to improve my model.Thanks for your advice again！

from chexpert.