Hi, congratulations on your excellent work! I have read your paper carefully and has some suggestions on it.
In your paper, you compared with the results of CoTTA using the learning rate of 0.00006/8. From our experimental results, CoTTA can have similar results as yours at the first visiting if they use the same learning rate 3e-4:
Note that the results can have a +-0.2 of vibration due to the stochastic restoration. Also, the order of the data streaming can affect the results. Better to check whether you conducted a sorting on the filenames.
start adapting fog
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 400/400, 0.7 task/s, elapsed: 565s, ETA: 0s
writing results to work_dirs/res.pkl
per class results:
+---------------+-------+-------+
| Class | IoU | Acc |
+---------------+-------+-------+
| road | 94.54 | 99.22 |
| sidewalk | 66.54 | 74.63 |
| building | 80.68 | 97.04 |
| wall | 58.03 | 71.08 |
| fence | 25.57 | 27.1 |
| pole | 47.33 | 56.82 |
| traffic light | 42.42 | 48.28 |
| traffic sign | 70.22 | 90.1 |
| vegetation | 87.19 | 91.66 |
| terrain | 72.02 | 88.8 |
| sky | 97.86 | 99.12 |
| person | 63.85 | 79.33 |
| rider | 67.44 | 86.42 |
| car | 87.74 | 95.59 |
| truck | 77.07 | 81.63 |
| bus | 93.26 | 98.02 |
| train | 86.21 | 96.21 |
| motorcycle | 54.43 | 64.7 |
| bicycle | 63.06 | 86.48 |
+---------------+-------+-------+
Summary:
+--------+-------+-------+------+
| Scope | mIoU | mAcc | aAcc |
+--------+-------+-------+------+
| global | 70.29 | 80.64 | 94.0 |
+--------+-------+-------+------+
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 400/400, 0.7 task/s, elapsed: 565s, ETA: 0s
writing results to work_dirs/res.pkl
per class results:
+---------------+-------+-------+
| Class | IoU | Acc |
+---------------+-------+-------+
| road | 89.33 | 98.88 |
| sidewalk | 53.88 | 59.51 |
| building | 70.42 | 87.84 |
| wall | 35.28 | 47.38 |
| fence | 24.5 | 27.7 |
| pole | 48.73 | 61.75 |
| traffic light | 44.78 | 60.42 |
| traffic sign | 46.07 | 52.79 |
| vegetation | 40.09 | 87.76 |
| terrain | 27.55 | 65.01 |
| sky | 0.98 | 0.98 |
| person | 51.57 | 76.0 |
| rider | 42.65 | 52.77 |
| car | 78.23 | 90.77 |
| truck | 19.75 | 70.78 |
| bus | 44.25 | 46.19 |
| train | 58.46 | 60.9 |
| motorcycle | 26.96 | 30.42 |
| bicycle | 36.54 | 42.75 |
+---------------+-------+-------+
Summary:
+--------+-------+-------+-------+
| Scope | mIoU | mAcc | aAcc |
+--------+-------+-------+-------+
| global | 44.21 | 58.98 | 72.73 |
+--------+-------+-------+-------+
start adapting rain
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 400/400, 0.7 task/s, elapsed: 565s, ETA: 0s
writing results to work_dirs/res.pkl
per class results:
+---------------+-------+-------+
| Class | IoU | Acc |
+---------------+-------+-------+
| road | 86.59 | 97.76 |
| sidewalk | 57.17 | 62.53 |
| building | 90.86 | 97.03 |
| wall | 45.49 | 59.4 |
| fence | 33.19 | 38.46 |
| pole | 57.03 | 70.51 |
| traffic light | 67.61 | 76.57 |
| traffic sign | 66.47 | 81.57 |
| vegetation | 90.83 | 94.92 |
| terrain | 60.64 | 86.16 |
| sky | 97.91 | 98.45 |
| person | 53.54 | 71.4 |
| rider | 61.56 | 81.05 |
| car | 84.12 | 92.13 |
| truck | 41.19 | 60.1 |
| bus | 75.31 | 79.51 |
| train | 73.7 | 85.37 |
| motorcycle | 52.68 | 66.03 |
| bicycle | 60.39 | 75.82 |
+---------------+-------+-------+
Summary:
+--------+-------+-------+-------+
| Scope | mIoU | mAcc | aAcc |
+--------+-------+-------+-------+
| global | 66.12 | 77.62 | 93.07 |
+--------+-------+-------+-------+
start adapting snow
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 400/400, 0.7 task/s, elapsed: 566s, ETA: 0s
writing results to work_dirs/res.pkl
per class results:
+---------------+-------+-------+
| Class | IoU | Acc |
+---------------+-------+-------+
| road | 84.02 | 97.99 |
| sidewalk | 51.14 | 62.09 |
| building | 87.17 | 97.43 |
| wall | 49.85 | 62.4 |
| fence | 42.46 | 44.89 |
| pole | 60.95 | 72.05 |
| traffic light | 74.03 | 82.8 |
| traffic sign | 69.34 | 82.8 |
| vegetation | 83.19 | 87.17 |
| terrain | 4.4 | 4.99 |
| sky | 97.68 | 98.73 |
| person | 66.38 | 83.22 |
| rider | 43.74 | 80.12 |
| car | 85.6 | 95.38 |
| truck | 49.63 | 57.19 |
| bus | 59.0 | 69.45 |
| train | 85.67 | 88.14 |
| motorcycle | 23.89 | 28.61 |
| bicycle | 53.49 | 69.88 |
+---------------+-------+-------+
Summary:
+--------+-------+-------+-------+
| Scope | mIoU | mAcc | aAcc |
+--------+-------+-------+-------+
| global | 61.66 | 71.86 | 91.21 |
+--------+-------+-------+-------+