Fail to reimplement your paper's result for semi-supervised. about cct HOT 8 CLOSED

Hi @TyroneLi

for how many epochs did you train? and did you use more than 1 gpus?

from cct.

Hi @TyroneLi

for how many epochs did you train? and did you use more than 1 gpus?
@yassouali
Thanks for your reply. I use the default config you provided, here it is.

`{
"name": "CCT",
"experim_name": "CCT",
"n_gpu": 1,
"n_labeled_examples": 1464,
"diff_lrs": true,
"ramp_up": 0.1,
"unsupervised_w": 30,
"ignore_index": 255,
"lr_scheduler": "Poly",
"use_weak_lables":true,
"weakly_loss_w": 0.4,
"pretrained": true,

"model":{
    "supervised": false,
    "semi": true,
    "supervised_w": 1,

    "sup_loss": "CE",
    "un_loss": "MSE",

    "softmax_temp": 1,
    "aux_constraint": false,
    "aux_constraint_w": 1,
    "confidence_masking": false,
    "confidence_th": 0.5,

    "drop": 6,
    "drop_rate": 0.5,
    "spatial": true,

    "cutout": 6,
    "erase": 0.4,

    "vat": 2,
    "xi": 1e-6,
    "eps": 2.0,

    "context_masking": 2,
    "object_masking": 2,
    "feature_drop": 6,

    "feature_noise": 6,
    "uniform_range": 0.3
},


"optimizer": {
    "type": "SGD",
    "args":{
        "lr": 1e-2,
        "weight_decay": 1e-4,
        "momentum": 0.9
    }
},


"train_supervised": {
    "data_dir": "/data/voc/",
    "batch_size": 10,
    "crop_size": 320,
    "shuffle": true,
    "base_size": 400,
    "scale": true,
    "augment": true,
    "flip": true,
    "rotate": false,
    "blur": false,
    "split": "train_supervised",
    "num_workers": 8
},

"train_unsupervised": {
    "data_dir": "/data/voc/",
    "weak_labels_output": "pseudo_labels/result/pseudo_labels/",
    "batch_size": 10,
    "crop_size": 320,
    "shuffle": true,
    "base_size": 400,
    "scale": true,
    "augment": true,
    "flip": true,
    "rotate": false,
    "blur": false,
    "split": "train_unsupervised",
    "num_workers": 8
},

"val_loader": {
    "data_dir": "/data/voc/",
    "batch_size": 1,
    "val": true,
    "split": "val",
    "shuffle": false,
    "num_workers": 4
},

"trainer": {
    "epochs": 80,
    "save_dir": "saved_use_weak_labels_npy2img/",
    "save_period": 5,

    "monitor": "max Mean_IoU",
    "early_stop": 10,
    
    "tensorboardX": true,
    "log_dir": "saved_use_weak_labels_npy2img/",
    "log_per_iter": 20,

    "val": true,
    "val_per_epochs": 5
}

}`

I just ran on one single GPU with cuda 11.0. But when I change to cuda-10.0, python3.7 and pytorch1.1.0, I still cannot achieve more than 71% mIoU whatever. I donnot know which part I missed or some config I mistook. Hope you could give me some suggestions to reimplement your results. Applying multi-scale inference is able to obtain 72% mIoU but it's inconsistent with your paper's result.
Note that, I mainly want to reimplement for 1.5k supervised plus 9.0k weakly supervised generated for pretrained IRNet.

from cct.

well this is a bit weird, the config seems to be correct, pytorch can be quite mysterious sometimes :)

try pytorch 1.4 - 1.6 since these are the version I tested with (same thing for torchvision with the corresponding version)

from cct.

well this is a bit weird, the config seems to be correct, pytorch can be quite mysterious sometimes :)

try pytorch 1.4 - 1.6 since these are the version I tested with (same thing for torchvision with the corresponding version)

All right. I would try these versions. But it would take a little long time to finish. Hope I can obtain new results today. Thanks.

from cct.

well this is a bit weird, the config seems to be correct, pytorch can be quite mysterious sometimes :)

try pytorch 1.4 - 1.6 since these are the version I tested with (same thing for torchvision with the corresponding version)

I changed to pytorch1.4.0 and conducted experiment for semi(1.5k supervised and 9k weak labels), I got 72.06mIoU. However, it is still a little bit (nearly 1%) lower than your paper's result.

from cct.

hi @TyroneLi

did you also rerun the pseudo labeling process, maybe try regenerating the pseudo labels with the current setting

from cct.

hi @TyroneLi

did you also rerun the pseudo labeling process, maybe try regenerating the pseudo labels with the current setting

What's the meaning of rerun the pseudo labeling process? Do you mean rerun the pseudo labeling process with IRNet's second step to perform pseudo labeling expansion? But did you perform this in your paper's experiment?

from cct.

hi @TyroneLi

no, just rerun the same process to generate the pseudo labels as described in the readme but with the current version of pytorch & torchvision

from cct.