Comments (8)
machengcheng2016
commented on September 17, 2024
I notice that the ZeroshotCLIP2 does follow CLIP to utilize the "prompt ensembling" trick. However, the performance is 48.24%, still far behind the official result ~55%.
Please help me 🙏 😟
from coop.
KaiyangZhou
commented on September 17, 2024
Can you show the full log (surrounded by ``, which makes it easier to read)?
from coop.
KaiyangZhou
commented on September 17, 2024
Also make sure the Dassl package is the latest one. Please carefully check the updates shown here https://github.com/KaiyangZhou/CoOp#updates.
from coop.
machengcheng2016
commented on September 17, 2024
Yeah yeah. I did "git clone" last two weeks, so I think both Dassl and CoOP is the lastest.
The log.txt is like
***************
** Arguments **
***************
backbone:
config_file: configs/trainers/CoOp/rn50_ep50.yaml
dataset_config_file: configs/datasets/stanford_cars.yaml
eval_only: False
head:
load_epoch: None
model_dir:
no_train: False
opts: ['TRAINER.COOP.N_CTX', '16', 'TRAINER.COOP.CSC', 'False', 'TRAINER.COOP.CLASS_TOKEN_POSITION', 'end', 'DATASET.NUM_SHOTS', '1']
output_dir: output/stanford_cars/CoOp/rn50_ep50_1shots/nctx16_cscFalse_ctpend/seed2
resume:
root: /homesda/ccma/data/
seed: 2
source_domains: None
target_domains: None
trainer: CoOp
transforms: None
************
** Config **
************
DATALOADER:
K_TRANSFORMS: 1
NUM_WORKERS: 8
RETURN_IMG0: False
TEST:
BATCH_SIZE: 100
SAMPLER: SequentialSampler
TRAIN_U:
BATCH_SIZE: 32
N_DOMAIN: 0
N_INS: 16
SAME_AS_X: True
SAMPLER: RandomSampler
TRAIN_X:
BATCH_SIZE: 32
N_DOMAIN: 0
N_INS: 16
SAMPLER: RandomSampler
DATASET:
ALL_AS_UNLABELED: False
CIFAR_C_LEVEL: 1
CIFAR_C_TYPE:
NAME: StanfordCars
NUM_LABELED: -1
NUM_SHOTS: 1
ROOT: /homesda/ccma/data/
SOURCE_DOMAINS: ()
STL10_FOLD: -1
SUBSAMPLE_CLASSES: all
TARGET_DOMAINS: ()
VAL_PERCENT: 0.1
INPUT:
COLORJITTER_B: 0.4
COLORJITTER_C: 0.4
COLORJITTER_H: 0.1
COLORJITTER_S: 0.4
CROP_PADDING: 4
CUTOUT_LEN: 16
CUTOUT_N: 1
GB_K: 21
GB_P: 0.5
GN_MEAN: 0.0
GN_STD: 0.15
INTERPOLATION: bicubic
NO_TRANSFORM: False
PIXEL_MEAN: [0.48145466, 0.4578275, 0.40821073]
PIXEL_STD: [0.26862954, 0.26130258, 0.27577711]
RANDAUGMENT_M: 10
RANDAUGMENT_N: 2
RGS_P: 0.2
SIZE: (224, 224)
TRANSFORMS: ('random_resized_crop', 'random_flip', 'normalize')
MODEL:
BACKBONE:
NAME: RN50
PRETRAINED: True
HEAD:
ACTIVATION: relu
BN: True
DROPOUT: 0.0
HIDDEN_LAYERS: ()
NAME:
INIT_WEIGHTS:
OPTIM:
ADAM_BETA1: 0.9
ADAM_BETA2: 0.999
BASE_LR_MULT: 0.1
GAMMA: 0.1
LR: 0.002
LR_SCHEDULER: cosine
MAX_EPOCH: 50
MOMENTUM: 0.9
NAME: sgd
NEW_LAYERS: ()
RMSPROP_ALPHA: 0.99
SGD_DAMPNING: 0
SGD_NESTEROV: False
STAGED_LR: False
STEPSIZE: (-1,)
WARMUP_CONS_LR: 1e-05
WARMUP_EPOCH: 1
WARMUP_MIN_LR: 1e-05
WARMUP_RECOUNT: True
WARMUP_TYPE: constant
WEIGHT_DECAY: 0.0005
OUTPUT_DIR: output/stanford_cars/CoOp/rn50_ep50_1shots/nctx16_cscFalse_ctpend/seed2
RESUME:
SEED: 2
TEST:
COMPUTE_CMAT: False
EVALUATOR: Classification
FINAL_MODEL: last_step
NO_TEST: False
PER_CLASS_RESULT: False
SPLIT: test
TRAIN:
CHECKPOINT_FREQ: 0
COUNT_ITER: train_x
PRINT_FREQ: 5
TRAINER:
CG:
ALPHA_D: 0.5
ALPHA_F: 0.5
EPS_D: 1.0
EPS_F: 1.0
COCOOP:
CTX_INIT:
N_CTX: 16
PREC: fp16
COOP:
CLASS_TOKEN_POSITION: end
CSC: False
CTX_INIT:
N_CTX: 16
PREC: fp16
DAEL:
CONF_THRE: 0.95
STRONG_TRANSFORMS: ()
WEIGHT_U: 0.5
DDAIG:
ALPHA: 0.5
CLAMP: False
CLAMP_MAX: 1.0
CLAMP_MIN: -1.0
G_ARCH:
LMDA: 0.3
WARMUP: 0
ENTMIN:
LMDA: 0.001
FIXMATCH:
CONF_THRE: 0.95
STRONG_TRANSFORMS: ()
WEIGHT_U: 1.0
M3SDA:
LMDA: 0.5
N_STEP_F: 4
MCD:
N_STEP_F: 4
MEANTEA:
EMA_ALPHA: 0.999
RAMPUP: 5
WEIGHT_U: 1.0
MIXMATCH:
MIXUP_BETA: 0.75
RAMPUP: 20000
TEMP: 2.0
WEIGHT_U: 100.0
MME:
LMDA: 0.1
NAME: CoOp
SE:
CONF_THRE: 0.95
EMA_ALPHA: 0.999
RAMPUP: 300
USE_CUDA: True
VERBOSE: True
VERSION: 1
Collecting env info ...
** System info **
PyTorch version: 1.7.1
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A
OS: Linux Mint 18.3 Sylvia (x86_64)
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609
Clang version: Could not collect
CMake version: version 3.5.1
Python version: 3.7 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 9.0.176
GPU models and configuration:
GPU 0: GeForce GTX 1080 Ti
GPU 1: GeForce GTX 1080 Ti
GPU 2: TITAN Xp
GPU 3: TITAN Xp
Nvidia driver version: 430.14
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
HIP runtime version: N/A
MIOpen runtime version: N/A
Versions of relevant libraries:
[pip3] numpy==1.21.6
[pip3] torch==1.7.1
[pip3] torchvision==0.8.2
[conda] numpy 1.21.6 pypi_0 pypi
[conda] torch 1.7.1 pypi_0 pypi
[conda] torchvision 0.8.2 pypi_0 pypi
Pillow (9.1.1)
Loading trainer: CoOp
Loading dataset: StanfordCars
Reading split from /homesda/ccma/data/stanford_cars/split_zhou_StanfordCars.json
Loading preprocessed few-shot data from /homesda/ccma/data/stanford_cars/split_fewshot/shot_1-seed_2.pkl
Building transform_train
+ resize to 224x224
+ random flip
+ random resized crop
+ to torch tensor of range [0, 1]
+ normalization (mean=[0.48145466, 0.4578275, 0.40821073], std=[0.26862954, 0.26130258, 0.27577711])
Building transform_test
+ resize to 224x224
+ to torch tensor of range [0, 1]
+ normalization (mean=[0.48145466, 0.4578275, 0.40821073], std=[0.26862954, 0.26130258, 0.27577711])
***** Dataset statistics *****
Dataset: StanfordCars
# classes: 196
# train_x: 196
# val: 196
# test: 8,041
Loading CLIP (backbone: RN50)
Building custom CLIP
---------- n_cls = 196, n_ctx = 16, ctx_init = , dtype = torch.float16, ctx_dim = 512, clip_imsize = 224, cfg_imsize == 224
Initializing a generic context
Initial context: "X X X X X X X X X X X X X X X X"
Number of context words (tokens): 16
++++++++++ prompts[0] = X X X X X X X X X X X X X X X X 2000 AM General Hummer SUV.
xxxxxxxxxx tokenized_prompts.shape = torch.Size([196, 77])
++++++++++ embedding.shape = torch.Size([196, 77, 512])
Turning off gradients in both the image and the text encoder
Loading evaluator: Classification
No checkpoint found, train from scratch
Initializing summary writer for tensorboard with log_dir=output/stanford_cars/CoOp/rn50_ep50_1shots/nctx16_cscFalse_ctpend/seed2/tensorboard
epoch [1/50][5/6] time 0.628 (0.896) data 0.000 (0.163) eta 0:04:24 loss 3.6641 (4.1277) acc 18.7500 (8.1250) lr 1.000000e-05
epoch [2/50][5/6] time 0.631 (0.817) data 0.000 (0.187) eta 0:03:56 loss 3.5371 (3.2277) acc 21.8750 (20.6250) lr 2.000000e-03
epoch [3/50][5/6] time 0.634 (0.823) data 0.000 (0.190) eta 0:03:52 loss 2.7598 (2.6148) acc 40.6250 (34.3750) lr 1.998027e-03
epoch [4/50][5/6] time 0.636 (0.826) data 0.000 (0.190) eta 0:03:48 loss 2.4707 (2.3930) acc 28.1250 (36.8750) lr 1.992115e-03
epoch [5/50][5/6] time 0.635 (0.823) data 0.000 (0.187) eta 0:03:43 loss 2.3555 (2.3852) acc 31.2500 (36.8750) lr 1.982287e-03
epoch [6/50][5/6] time 0.636 (0.818) data 0.000 (0.181) eta 0:03:36 loss 2.1328 (2.2936) acc 28.1250 (37.5000) lr 1.968583e-03
epoch [7/50][5/6] time 0.639 (0.835) data 0.000 (0.196) eta 0:03:36 loss 2.4258 (2.3879) acc 31.2500 (36.2500) lr 1.951057e-03
epoch [8/50][5/6] time 0.639 (0.802) data 0.000 (0.163) eta 0:03:22 loss 2.1777 (2.1951) acc 46.8750 (45.6250) lr 1.929776e-03
epoch [9/50][5/6] time 0.639 (0.790) data 0.000 (0.150) eta 0:03:15 loss 2.1992 (2.1436) acc 50.0000 (46.2500) lr 1.904827e-03
epoch [10/50][5/6] time 0.643 (0.806) data 0.000 (0.163) eta 0:03:14 loss 2.3301 (2.3062) acc 28.1250 (35.6250) lr 1.876307e-03
epoch [11/50][5/6] time 0.644 (0.827) data 0.000 (0.184) eta 0:03:14 loss 1.6660 (2.1842) acc 50.0000 (43.7500) lr 1.844328e-03
epoch [12/50][5/6] time 0.643 (0.805) data 0.000 (0.162) eta 0:03:04 loss 2.3496 (2.4129) acc 40.6250 (40.0000) lr 1.809017e-03
epoch [13/50][5/6] time 0.645 (0.792) data 0.000 (0.147) eta 0:02:56 loss 2.4707 (2.3953) acc 40.6250 (38.7500) lr 1.770513e-03
epoch [14/50][5/6] time 0.644 (0.808) data 0.000 (0.163) eta 0:02:55 loss 1.9785 (2.1477) acc 46.8750 (45.6250) lr 1.728969e-03
epoch [15/50][5/6] time 0.644 (0.800) data 0.000 (0.156) eta 0:02:48 loss 2.2539 (2.0131) acc 40.6250 (46.2500) lr 1.684547e-03
epoch [16/50][5/6] time 0.644 (0.830) data 0.000 (0.185) eta 0:02:50 loss 1.7559 (2.1006) acc 53.1250 (48.1250) lr 1.637424e-03
epoch [17/50][5/6] time 0.645 (0.807) data 0.000 (0.162) eta 0:02:40 loss 2.1699 (2.1947) acc 43.7500 (45.6250) lr 1.587785e-03
epoch [18/50][5/6] time 0.646 (0.828) data 0.000 (0.182) eta 0:02:39 loss 2.0410 (2.4137) acc 50.0000 (41.2500) lr 1.535827e-03
epoch [19/50][5/6] time 0.644 (0.822) data 0.000 (0.177) eta 0:02:33 loss 1.8672 (1.8027) acc 43.7500 (46.8750) lr 1.481754e-03
epoch [20/50][5/6] time 0.645 (0.816) data 0.000 (0.170) eta 0:02:27 loss 2.1055 (2.1193) acc 46.8750 (43.7500) lr 1.425779e-03
epoch [21/50][5/6] time 0.647 (0.839) data 0.000 (0.192) eta 0:02:26 loss 2.2461 (2.0531) acc 53.1250 (49.3750) lr 1.368125e-03
epoch [22/50][5/6] time 0.654 (0.820) data 0.000 (0.171) eta 0:02:18 loss 1.9268 (1.8830) acc 53.1250 (53.1250) lr 1.309017e-03
epoch [23/50][5/6] time 0.654 (0.787) data 0.000 (0.138) eta 0:02:08 loss 1.8799 (2.1418) acc 40.6250 (42.5000) lr 1.248690e-03
epoch [24/50][5/6] time 0.650 (0.828) data 0.000 (0.180) eta 0:02:10 loss 2.0273 (2.1520) acc 43.7500 (45.0000) lr 1.187381e-03
epoch [25/50][5/6] time 0.661 (0.852) data 0.000 (0.199) eta 0:02:08 loss 2.1602 (2.0152) acc 31.2500 (45.0000) lr 1.125333e-03
epoch [26/50][5/6] time 0.648 (0.788) data 0.000 (0.138) eta 0:01:54 loss 1.8838 (1.9678) acc 40.6250 (45.6250) lr 1.062791e-03
epoch [27/50][5/6] time 0.651 (0.832) data 0.000 (0.181) eta 0:01:55 loss 2.0020 (1.8514) acc 37.5000 (47.5000) lr 1.000000e-03
epoch [28/50][5/6] time 0.664 (0.820) data 0.000 (0.165) eta 0:01:49 loss 2.0039 (2.1506) acc 43.7500 (41.2500) lr 9.372095e-04
epoch [29/50][5/6] time 0.649 (0.786) data 0.000 (0.133) eta 0:01:39 loss 1.7012 (1.8313) acc 53.1250 (46.8750) lr 8.746668e-04
epoch [30/50][5/6] time 0.661 (0.818) data 0.000 (0.158) eta 0:01:39 loss 1.8564 (1.9953) acc 53.1250 (48.7500) lr 8.126187e-04
epoch [31/50][5/6] time 0.681 (0.819) data 0.000 (0.157) eta 0:01:34 loss 2.3496 (1.9160) acc 37.5000 (46.2500) lr 7.513101e-04
epoch [32/50][5/6] time 0.663 (0.835) data 0.000 (0.180) eta 0:01:30 loss 1.9707 (2.0141) acc 46.8750 (48.7500) lr 6.909830e-04
epoch [33/50][5/6] time 0.671 (0.794) data 0.000 (0.135) eta 0:01:21 loss 1.4912 (1.8320) acc 59.3750 (51.8750) lr 6.318754e-04
epoch [34/50][5/6] time 0.648 (0.812) data 0.000 (0.159) eta 0:01:18 loss 2.2090 (2.1572) acc 46.8750 (48.1250) lr 5.742207e-04
epoch [35/50][5/6] time 0.649 (0.795) data 0.000 (0.145) eta 0:01:12 loss 2.0234 (1.7559) acc 40.6250 (55.6250) lr 5.182463e-04
epoch [36/50][5/6] time 0.656 (0.795) data 0.000 (0.133) eta 0:01:07 loss 1.9785 (1.9354) acc 53.1250 (50.6250) lr 4.641732e-04
epoch [37/50][5/6] time 0.651 (0.795) data 0.000 (0.142) eta 0:01:02 loss 1.7510 (1.8002) acc 37.5000 (50.0000) lr 4.122147e-04
epoch [38/50][5/6] time 0.658 (0.823) data 0.000 (0.140) eta 0:01:00 loss 1.8018 (1.9250) acc 50.0000 (46.8750) lr 3.625760e-04
epoch [39/50][5/6] time 0.657 (0.810) data 0.000 (0.154) eta 0:00:54 loss 1.7119 (1.7859) acc 53.1250 (54.3750) lr 3.154529e-04
epoch [40/50][5/6] time 0.690 (0.829) data 0.000 (0.161) eta 0:00:50 loss 2.5527 (2.0197) acc 40.6250 (48.1250) lr 2.710314e-04
epoch [41/50][5/6] time 0.664 (0.844) data 0.000 (0.182) eta 0:00:46 loss 1.9443 (2.0963) acc 40.6250 (42.5000) lr 2.294868e-04
epoch [42/50][5/6] time 0.683 (0.819) data 0.000 (0.151) eta 0:00:40 loss 1.5322 (1.8588) acc 53.1250 (53.1250) lr 1.909830e-04
epoch [43/50][5/6] time 0.650 (0.802) data 0.000 (0.144) eta 0:00:34 loss 2.5977 (2.0521) acc 37.5000 (49.3750) lr 1.556721e-04
epoch [44/50][5/6] time 0.657 (0.871) data 0.000 (0.161) eta 0:00:32 loss 1.7793 (1.8348) acc 46.8750 (53.1250) lr 1.236933e-04
epoch [45/50][5/6] time 0.650 (0.833) data 0.000 (0.182) eta 0:00:25 loss 1.6836 (2.0330) acc 50.0000 (42.5000) lr 9.517295e-05
epoch [46/50][5/6] time 0.694 (0.851) data 0.000 (0.186) eta 0:00:21 loss 1.6865 (1.9184) acc 53.1250 (50.0000) lr 7.022351e-05
epoch [47/50][5/6] time 0.709 (0.843) data 0.000 (0.173) eta 0:00:16 loss 1.5234 (1.9107) acc 56.2500 (50.6250) lr 4.894348e-05
epoch [48/50][5/6] time 0.650 (0.828) data 0.000 (0.177) eta 0:00:10 loss 1.8428 (1.9830) acc 56.2500 (48.7500) lr 3.141684e-05
epoch [49/50][5/6] time 0.658 (0.829) data 0.000 (0.176) eta 0:00:05 loss 1.7158 (1.8945) acc 56.2500 (50.6250) lr 1.771275e-05
epoch [50/50][5/6] time 0.652 (0.871) data 0.000 (0.170) eta 0:00:00 loss 1.5732 (1.8141) acc 56.2500 (50.0000) lr 7.885299e-06
Checkpoint saved to "output/stanford_cars/CoOp/rn50_ep50_1shots/nctx16_cscFalse_ctpend/seed2/prompt_learner/model.pth.tar-50"
Finished training
Do evaluation on test set
=> result
* total: 8,041
* correct: 4,209
* accuracy: 52.34%
* error: 47.66%
Elapsed: 0:04:48
from coop.
KaiyangZhou
commented on September 17, 2024
Looks like the test transform function isn't the latest one
Building transform_test
+ resize to 224x224
+ to torch tensor of range [0, 1]
+ normalization (mean=[0.48145466, 0.4578275, 0.40821073], std=[0.26862954, 0.26130258, 0.27577711])
It's supposed to do a center crop. Check this https://github.com/KaiyangZhou/Dassl.pytorch/blob/master/dassl/data/transforms/transforms.py#L323.
from coop.
KaiyangZhou
commented on September 17, 2024
Oh wait, this is the zero-shot model right? There is supposed to be no training at all.
from coop.
machengcheng2016
commented on September 17, 2024
Sorry, I just paste the log of CoOP training. The log of zeroshot.sh is like
***************
** Arguments **
***************
backbone:
config_file: configs/trainers/CoOp/rn50.yaml
dataset_config_file: configs/datasets/stanford_cars.yaml
eval_only: True
head:
load_epoch: None
model_dir:
no_train: False
opts: []
output_dir: output/ZeroshotCLIP/rn50/stanford_cars
resume:
root: ./data
seed: -1
source_domains: None
target_domains: None
trainer: ZeroshotCLIP
transforms: None
************
** Config **
************
DATALOADER:
K_TRANSFORMS: 1
NUM_WORKERS: 8
RETURN_IMG0: False
TEST:
BATCH_SIZE: 100
SAMPLER: SequentialSampler
TRAIN_U:
BATCH_SIZE: 32
N_DOMAIN: 0
N_INS: 16
SAME_AS_X: True
SAMPLER: RandomSampler
TRAIN_X:
BATCH_SIZE: 32
N_DOMAIN: 0
N_INS: 16
SAMPLER: RandomSampler
DATASET:
ALL_AS_UNLABELED: False
CIFAR_C_LEVEL: 1
CIFAR_C_TYPE:
NAME: StanfordCars
NUM_LABELED: -1
NUM_SHOTS: -1
ROOT: ./data
SOURCE_DOMAINS: ()
STL10_FOLD: -1
SUBSAMPLE_CLASSES: all
TARGET_DOMAINS: ()
VAL_PERCENT: 0.1
INPUT:
COLORJITTER_B: 0.4
COLORJITTER_C: 0.4
COLORJITTER_H: 0.1
COLORJITTER_S: 0.4
CROP_PADDING: 4
CUTOUT_LEN: 16
CUTOUT_N: 1
GB_K: 21
GB_P: 0.5
GN_MEAN: 0.0
GN_STD: 0.15
INTERPOLATION: bicubic
NO_TRANSFORM: False
PIXEL_MEAN: [0.48145466, 0.4578275, 0.40821073]
PIXEL_STD: [0.26862954, 0.26130258, 0.27577711]
RANDAUGMENT_M: 10
RANDAUGMENT_N: 2
RGS_P: 0.2
SIZE: (224, 224)
TRANSFORMS: ('random_resized_crop', 'random_flip', 'normalize')
MODEL:
BACKBONE:
NAME: RN50
PRETRAINED: True
HEAD:
ACTIVATION: relu
BN: True
DROPOUT: 0.0
HIDDEN_LAYERS: ()
NAME:
INIT_WEIGHTS:
OPTIM:
ADAM_BETA1: 0.9
ADAM_BETA2: 0.999
BASE_LR_MULT: 0.1
GAMMA: 0.1
LR: 0.002
LR_SCHEDULER: cosine
MAX_EPOCH: 200
MOMENTUM: 0.9
NAME: sgd
NEW_LAYERS: ()
RMSPROP_ALPHA: 0.99
SGD_DAMPNING: 0
SGD_NESTEROV: False
STAGED_LR: False
STEPSIZE: (-1,)
WARMUP_CONS_LR: 1e-05
WARMUP_EPOCH: 1
WARMUP_MIN_LR: 1e-05
WARMUP_RECOUNT: True
WARMUP_TYPE: constant
WEIGHT_DECAY: 0.0005
OUTPUT_DIR: output/ZeroshotCLIP/rn50/stanford_cars
RESUME:
SEED: -1
TEST:
COMPUTE_CMAT: False
EVALUATOR: Classification
FINAL_MODEL: last_step
NO_TEST: False
PER_CLASS_RESULT: True
SPLIT: test
TRAIN:
CHECKPOINT_FREQ: 0
COUNT_ITER: train_x
PRINT_FREQ: 5
TRAINER:
CG:
ALPHA_D: 0.5
ALPHA_F: 0.5
EPS_D: 1.0
EPS_F: 1.0
COCOOP:
CTX_INIT:
N_CTX: 16
PREC: fp16
COOP:
CLASS_TOKEN_POSITION: end
CSC: False
CTX_INIT:
N_CTX: 16
PREC: fp16
DAEL:
CONF_THRE: 0.95
STRONG_TRANSFORMS: ()
WEIGHT_U: 0.5
DDAIG:
ALPHA: 0.5
CLAMP: False
CLAMP_MAX: 1.0
CLAMP_MIN: -1.0
G_ARCH:
LMDA: 0.3
WARMUP: 0
ENTMIN:
LMDA: 0.001
FIXMATCH:
CONF_THRE: 0.95
STRONG_TRANSFORMS: ()
WEIGHT_U: 1.0
M3SDA:
LMDA: 0.5
N_STEP_F: 4
MCD:
N_STEP_F: 4
MEANTEA:
EMA_ALPHA: 0.999
RAMPUP: 5
WEIGHT_U: 1.0
MIXMATCH:
MIXUP_BETA: 0.75
RAMPUP: 20000
TEMP: 2.0
WEIGHT_U: 100.0
MME:
LMDA: 0.1
NAME: ZeroshotCLIP
SE:
CONF_THRE: 0.95
EMA_ALPHA: 0.999
RAMPUP: 300
USE_CUDA: True
VERBOSE: True
VERSION: 1
Collecting env info ...
** System info **
PyTorch version: 1.7.1
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A
OS: Linux Mint 18.3 Sylvia (x86_64)
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609
Clang version: Could not collect
CMake version: version 3.5.1
Python version: 3.7 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 9.0.176
GPU models and configuration:
GPU 0: GeForce GTX 1080 Ti
GPU 1: GeForce GTX 1080 Ti
GPU 2: TITAN Xp
GPU 3: TITAN Xp
Nvidia driver version: 430.14
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
HIP runtime version: N/A
MIOpen runtime version: N/A
Versions of relevant libraries:
[pip3] numpy==1.21.6
[pip3] torch==1.7.1
[pip3] torchvision==0.8.2
[conda] numpy 1.21.6 pypi_0 pypi
[conda] torch 1.7.1 pypi_0 pypi
[conda] torchvision 0.8.2 pypi_0 pypi
Pillow (9.1.1)
Loading trainer: ZeroshotCLIP
Loading dataset: StanfordCars
Reading split from /homesda/ccma/Projects/tmp/CoOp/data/stanford_cars/split_zhou_StanfordCars.json
Building transform_train
+ resize to 224x224
+ random flip
+ random resized crop
+ to torch tensor of range [0, 1]
+ normalization (mean=[0.48145466, 0.4578275, 0.40821073], std=[0.26862954, 0.26130258, 0.27577711])
Building transform_test
+ resize to 224x224
+ to torch tensor of range [0, 1]
+ normalization (mean=[0.48145466, 0.4578275, 0.40821073], std=[0.26862954, 0.26130258, 0.27577711])
***** Dataset statistics *****
Dataset: StanfordCars
# classes: 196
# train_x: 6,509
# val: 1,635
# test: 8,041
Loading CLIP (backbone: RN50)
Prompts: ['a photo of a 2000 AM General Hummer SUV.', 'a photo of a 2012 Acura RL Sedan.', 'a photo of a 2012 Acura TL Sedan.', 'a photo of a 2008 Acura TL Type-S.', 'a photo of a 2012 Acura TSX Sedan.', 'a photo of a 2001 Acura Integra Type R.', 'a photo of a 2012 Acura ZDX Hatchback.', 'a photo of a 2012 Aston Martin V8 Vantage Convertible.', 'a photo of a 2012 Aston Martin V8 Vantage Coupe.', 'a photo of a 2012 Aston Martin Virage Convertible.', 'a photo of a 2012 Aston Martin Virage Coupe.', 'a photo of a 2008 Audi RS 4 Convertible.', 'a photo of a 2012 Audi A5 Coupe.', 'a photo of a 2012 Audi TTS Coupe.', 'a photo of a 2012 Audi R8 Coupe.', 'a photo of a 1994 Audi V8 Sedan.', 'a photo of a 1994 Audi 100 Sedan.', 'a photo of a 1994 Audi 100 Wagon.', 'a photo of a 2011 Audi TT Hatchback.', 'a photo of a 2011 Audi S6 Sedan.', 'a photo of a 2012 Audi S5 Convertible.', 'a photo of a 2012 Audi S5 Coupe.', 'a photo of a 2012 Audi S4 Sedan.', 'a photo of a 2007 Audi S4 Sedan.', 'a photo of a 2012 Audi TT RS Coupe.', 'a photo of a 2012 BMW ActiveHybrid 5 Sedan.', 'a photo of a 2012 BMW 1 Series Convertible.', 'a photo of a 2012 BMW 1 Series Coupe.', 'a photo of a 2012 BMW 3 Series Sedan.', 'a photo of a 2012 BMW 3 Series Wagon.', 'a photo of a 2007 BMW 6 Series Convertible.', 'a photo of a 2007 BMW X5 SUV.', 'a photo of a 2012 BMW X6 SUV.', 'a photo of a 2012 BMW M3 Coupe.', 'a photo of a 2010 BMW M5 Sedan.', 'a photo of a 2010 BMW M6 Convertible.', 'a photo of a 2012 BMW X3 SUV.', 'a photo of a 2012 BMW Z4 Convertible.', 'a photo of a 2012 Bentley Continental Supersports Conv. Convertible.', 'a photo of a 2009 Bentley Arnage Sedan.', 'a photo of a 2011 Bentley Mulsanne Sedan.', 'a photo of a 2012 Bentley Continental GT Coupe.', 'a photo of a 2007 Bentley Continental GT Coupe.', 'a photo of a 2007 Bentley Continental Flying Spur Sedan.', 'a photo of a 2009 Bugatti Veyron 16.4 Convertible.', 'a photo of a 2009 Bugatti Veyron 16.4 Coupe.', 'a photo of a 2012 Buick Regal GS.', 'a photo of a 2007 Buick Rainier SUV.', 'a photo of a 2012 Buick Verano Sedan.', 'a photo of a 2012 Buick Enclave SUV.', 'a photo of a 2012 Cadillac CTS-V Sedan.', 'a photo of a 2012 Cadillac SRX SUV.', 'a photo of a 2007 Cadillac Escalade EXT Crew Cab.', 'a photo of a 2012 Chevrolet Silverado 1500 Hybrid Crew Cab.', 'a photo of a 2012 Chevrolet Corvette Convertible.', 'a photo of a 2012 Chevrolet Corvette ZR1.', 'a photo of a 2007 Chevrolet Corvette Ron Fellows Edition Z06.', 'a photo of a 2012 Chevrolet Traverse SUV.', 'a photo of a 2012 Chevrolet Camaro Convertible.', 'a photo of a 2010 Chevrolet HHR SS.', 'a photo of a 2007 Chevrolet Impala Sedan.', 'a photo of a 2012 Chevrolet Tahoe Hybrid SUV.', 'a photo of a 2012 Chevrolet Sonic Sedan.', 'a photo of a 2007 Chevrolet Express Cargo Van.', 'a photo of a 2012 Chevrolet Avalanche Crew Cab.', 'a photo of a 2010 Chevrolet Cobalt SS.', 'a photo of a 2010 Chevrolet Malibu Hybrid Sedan.', 'a photo of a 2009 Chevrolet TrailBlazer SS.', 'a photo of a 2012 Chevrolet Silverado 2500HD Regular Cab.', 'a photo of a 2007 Chevrolet Silverado 1500 Classic Extended Cab.', 'a photo of a 2007 Chevrolet Express Van.', 'a photo of a 2007 Chevrolet Monte Carlo Coupe.', 'a photo of a 2007 Chevrolet Malibu Sedan.', 'a photo of a 2012 Chevrolet Silverado 1500 Extended Cab.', 'a photo of a 2012 Chevrolet Silverado 1500 Regular Cab.', 'a photo of a 2009 Chrysler Aspen SUV.', 'a photo of a 2010 Chrysler Sebring Convertible.', 'a photo of a 2012 Chrysler Town and Country Minivan.', 'a photo of a 2010 Chrysler 300 SRT-8.', 'a photo of a 2008 Chrysler Crossfire Convertible.', 'a photo of a 2008 Chrysler PT Cruiser Convertible.', 'a photo of a 2002 Daewoo Nubira Wagon.', 'a photo of a 2012 Dodge Caliber Wagon.', 'a photo of a 2007 Dodge Caliber Wagon.', 'a photo of a 1997 Dodge Caravan Minivan.', 'a photo of a 2010 Dodge Ram Pickup 3500 Crew Cab.', 'a photo of a 2009 Dodge Ram Pickup 3500 Quad Cab.', 'a photo of a 2009 Dodge Sprinter Cargo Van.', 'a photo of a 2012 Dodge Journey SUV.', 'a photo of a 2010 Dodge Dakota Crew Cab.', 'a photo of a 2007 Dodge Dakota Club Cab.', 'a photo of a 2008 Dodge Magnum Wagon.', 'a photo of a 2011 Dodge Challenger SRT8.', 'a photo of a 2012 Dodge Durango SUV.', 'a photo of a 2007 Dodge Durango SUV.', 'a photo of a 2012 Dodge Charger Sedan.', 'a photo of a 2009 Dodge Charger SRT-8.', 'a photo of a 1998 Eagle Talon Hatchback.', 'a photo of a 2012 FIAT 500 Abarth.', 'a photo of a 2012 FIAT 500 Convertible.', 'a photo of a 2012 Ferrari FF Coupe.', 'a photo of a 2012 Ferrari California Convertible.', 'a photo of a 2012 Ferrari 458 Italia Convertible.', 'a photo of a 2012 Ferrari 458 Italia Coupe.', 'a photo of a 2012 Fisker Karma Sedan.', 'a photo of a 2012 Ford F-450 Super Duty Crew Cab.', 'a photo of a 2007 Ford Mustang Convertible.', 'a photo of a 2007 Ford Freestar Minivan.', 'a photo of a 2009 Ford Expedition EL SUV.', 'a photo of a 2012 Ford Edge SUV.', 'a photo of a 2011 Ford Ranger SuperCab.', 'a photo of a 2006 Ford GT Coupe.', 'a photo of a 2012 Ford F-150 Regular Cab.', 'a photo of a 2007 Ford F-150 Regular Cab.', 'a photo of a 2007 Ford Focus Sedan.', 'a photo of a 2012 Ford E-Series Wagon Van.', 'a photo of a 2012 Ford Fiesta Sedan.', 'a photo of a 2012 GMC Terrain SUV.', 'a photo of a 2012 GMC Savana Van.', 'a photo of a 2012 GMC Yukon Hybrid SUV.', 'a photo of a 2012 GMC Acadia SUV.', 'a photo of a 2012 GMC Canyon Extended Cab.', 'a photo of a 1993 Geo Metro Convertible.', 'a photo of a 2010 HUMMER H3T Crew Cab.', 'a photo of a 2009 HUMMER H2 SUT Crew Cab.', 'a photo of a 2012 Honda Odyssey Minivan.', 'a photo of a 2007 Honda Odyssey Minivan.', 'a photo of a 2012 Honda Accord Coupe.', 'a photo of a 2012 Honda Accord Sedan.', 'a photo of a 2012 Hyundai Veloster Hatchback.', 'a photo of a 2012 Hyundai Santa Fe SUV.', 'a photo of a 2012 Hyundai Tucson SUV.', 'a photo of a 2012 Hyundai Veracruz SUV.', 'a photo of a 2012 Hyundai Sonata Hybrid Sedan.', 'a photo of a 2007 Hyundai Elantra Sedan.', 'a photo of a 2012 Hyundai Accent Sedan.', 'a photo of a 2012 Hyundai Genesis Sedan.', 'a photo of a 2012 Hyundai Sonata Sedan.', 'a photo of a 2012 Hyundai Elantra Touring Hatchback.', 'a photo of a 2012 Hyundai Azera Sedan.', 'a photo of a 2012 Infiniti G Coupe IPL.', 'a photo of a 2011 Infiniti QX56 SUV.', 'a photo of a 2008 Isuzu Ascender SUV.', 'a photo of a 2012 Jaguar XK XKR.', 'a photo of a 2012 Jeep Patriot SUV.', 'a photo of a 2012 Jeep Wrangler SUV.', 'a photo of a 2012 Jeep Liberty SUV.', 'a photo of a 2012 Jeep Grand Cherokee SUV.', 'a photo of a 2012 Jeep Compass SUV.', 'a photo of a 2008 Lamborghini Reventon Coupe.', 'a photo of a 2012 Lamborghini Aventador Coupe.', 'a photo of a 2012 Lamborghini Gallardo LP 570-4 Superleggera.', 'a photo of a 2001 Lamborghini Diablo Coupe.', 'a photo of a 2012 Land Rover Range Rover SUV.', 'a photo of a 2012 Land Rover LR2 SUV.', 'a photo of a 2011 Lincoln Town Car Sedan.', 'a photo of a 2012 MINI Cooper Roadster Convertible.', 'a photo of a 2012 Maybach Landaulet Convertible.', 'a photo of a 2011 Mazda Tribute SUV.', 'a photo of a 2012 McLaren MP4-12C Coupe.', 'a photo of a 1993 Mercedes-Benz 300-Class Convertible.', 'a photo of a 2012 Mercedes-Benz C-Class Sedan.', 'a photo of a 2009 Mercedes-Benz SL-Class Coupe.', 'a photo of a 2012 Mercedes-Benz E-Class Sedan.', 'a photo of a 2012 Mercedes-Benz S-Class Sedan.', 'a photo of a 2012 Mercedes-Benz Sprinter Van.', 'a photo of a 2012 Mitsubishi Lancer Sedan.', 'a photo of a 2012 Nissan Leaf Hatchback.', 'a photo of a 2012 Nissan NV Passenger Van.', 'a photo of a 2012 Nissan Juke Hatchback.', 'a photo of a 1998 Nissan 240SX Coupe.', 'a photo of a 1999 Plymouth Neon Coupe.', 'a photo of a 2012 Porsche Panamera Sedan.', 'a photo of a 2012 Ram C/V Cargo Van Minivan.', 'a photo of a 2012 Rolls-Royce Phantom Drophead Coupe Convertible.', 'a photo of a 2012 Rolls-Royce Ghost Sedan.', 'a photo of a 2012 Rolls-Royce Phantom Sedan.', 'a photo of a 2012 Scion xD Hatchback.', 'a photo of a 2009 Spyker C8 Convertible.', 'a photo of a 2009 Spyker C8 Coupe.', 'a photo of a 2007 Suzuki Aerio Sedan.', 'a photo of a 2012 Suzuki Kizashi Sedan.', 'a photo of a 2012 Suzuki SX4 Hatchback.', 'a photo of a 2012 Suzuki SX4 Sedan.', 'a photo of a 2012 Tesla Model S Sedan.', 'a photo of a 2012 Toyota Sequoia SUV.', 'a photo of a 2012 Toyota Camry Sedan.', 'a photo of a 2012 Toyota Corolla Sedan.', 'a photo of a 2012 Toyota 4Runner SUV.', 'a photo of a 2012 Volkswagen Golf Hatchback.', 'a photo of a 1991 Volkswagen Golf Hatchback.', 'a photo of a 2012 Volkswagen Beetle Hatchback.', 'a photo of a 2012 Volvo C30 Hatchback.', 'a photo of a 1993 Volvo 240 Sedan.', 'a photo of a 2007 Volvo XC90 SUV.', 'a photo of a 2012 smart fortwo Convertible.']
Loading evaluator: Classification
Note that load_model() is skipped as no pretrained model is given
Do evaluation on test set
=> result
* total: 8,041
* correct: 3,765
* accuracy: 46.82%
* error: 53.18%
=> per-class result
* class: 0 (2000 AM General Hummer SUV) total: 44 correct: 25 acc: 56.82%
* class: 1 (2012 Acura RL Sedan) total: 32 correct: 6 acc: 18.75%
* class: 2 (2012 Acura TL Sedan) total: 43 correct: 1 acc: 2.33%
* class: 3 (2008 Acura TL Type-S) total: 42 correct: 13 acc: 30.95%
* class: 4 (2012 Acura TSX Sedan) total: 40 correct: 10 acc: 25.00%
* class: 5 (2001 Acura Integra Type R) total: 44 correct: 32 acc: 72.73%
* class: 6 (2012 Acura ZDX Hatchback) total: 39 correct: 14 acc: 35.90%
* class: 7 (2012 Aston Martin V8 Vantage Convertible) total: 45 correct: 30 acc: 66.67%
* class: 8 (2012 Aston Martin V8 Vantage Coupe) total: 41 correct: 26 acc: 63.41%
* class: 9 (2012 Aston Martin Virage Convertible) total: 33 correct: 12 acc: 36.36%
* class: 10 (2012 Aston Martin Virage Coupe) total: 38 correct: 18 acc: 47.37%
* class: 11 (2008 Audi RS 4 Convertible) total: 36 correct: 22 acc: 61.11%
* class: 12 (2012 Audi A5 Coupe) total: 41 correct: 7 acc: 17.07%
* class: 13 (2012 Audi TTS Coupe) total: 42 correct: 5 acc: 11.90%
* class: 14 (2012 Audi R8 Coupe) total: 43 correct: 38 acc: 88.37%
* class: 15 (1994 Audi V8 Sedan) total: 43 correct: 14 acc: 32.56%
* class: 16 (1994 Audi 100 Sedan) total: 40 correct: 3 acc: 7.50%
* class: 17 (1994 Audi 100 Wagon) total: 42 correct: 16 acc: 38.10%
* class: 18 (2011 Audi TT Hatchback) total: 40 correct: 16 acc: 40.00%
* class: 19 (2011 Audi S6 Sedan) total: 46 correct: 6 acc: 13.04%
* class: 20 (2012 Audi S5 Convertible) total: 42 correct: 11 acc: 26.19%
* class: 21 (2012 Audi S5 Coupe) total: 42 correct: 13 acc: 30.95%
* class: 22 (2012 Audi S4 Sedan) total: 39 correct: 28 acc: 71.79%
* class: 23 (2007 Audi S4 Sedan) total: 45 correct: 11 acc: 24.44%
* class: 24 (2012 Audi TT RS Coupe) total: 39 correct: 31 acc: 79.49%
* class: 25 (2012 BMW ActiveHybrid 5 Sedan) total: 34 correct: 3 acc: 8.82%
* class: 26 (2012 BMW 1 Series Convertible) total: 35 correct: 24 acc: 68.57%
* class: 27 (2012 BMW 1 Series Coupe) total: 41 correct: 23 acc: 56.10%
* class: 28 (2012 BMW 3 Series Sedan) total: 42 correct: 6 acc: 14.29%
* class: 29 (2012 BMW 3 Series Wagon) total: 41 correct: 21 acc: 51.22%
* class: 30 (2007 BMW 6 Series Convertible) total: 44 correct: 7 acc: 15.91%
* class: 31 (2007 BMW X5 SUV) total: 41 correct: 20 acc: 48.78%
* class: 32 (2012 BMW X6 SUV) total: 42 correct: 11 acc: 26.19%
* class: 33 (2012 BMW M3 Coupe) total: 44 correct: 12 acc: 27.27%
* class: 34 (2010 BMW M5 Sedan) total: 41 correct: 3 acc: 7.32%
* class: 35 (2010 BMW M6 Convertible) total: 41 correct: 8 acc: 19.51%
* class: 36 (2012 BMW X3 SUV) total: 38 correct: 34 acc: 89.47%
* class: 37 (2012 BMW Z4 Convertible) total: 40 correct: 30 acc: 75.00%
* class: 38 (2012 Bentley Continental Supersports Conv. Convertible) total: 36 correct: 17 acc: 47.22%
* class: 39 (2009 Bentley Arnage Sedan) total: 39 correct: 20 acc: 51.28%
* class: 40 (2011 Bentley Mulsanne Sedan) total: 35 correct: 25 acc: 71.43%
* class: 41 (2012 Bentley Continental GT Coupe) total: 34 correct: 25 acc: 73.53%
* class: 42 (2007 Bentley Continental GT Coupe) total: 46 correct: 18 acc: 39.13%
* class: 43 (2007 Bentley Continental Flying Spur Sedan) total: 44 correct: 8 acc: 18.18%
* class: 44 (2009 Bugatti Veyron 16.4 Convertible) total: 32 correct: 22 acc: 68.75%
* class: 45 (2009 Bugatti Veyron 16.4 Coupe) total: 43 correct: 32 acc: 74.42%
* class: 46 (2012 Buick Regal GS) total: 35 correct: 20 acc: 57.14%
* class: 47 (2007 Buick Rainier SUV) total: 42 correct: 3 acc: 7.14%
* class: 48 (2012 Buick Verano Sedan) total: 37 correct: 25 acc: 67.57%
* class: 49 (2012 Buick Enclave SUV) total: 42 correct: 27 acc: 64.29%
* class: 50 (2012 Cadillac CTS-V Sedan) total: 43 correct: 34 acc: 79.07%
* class: 51 (2012 Cadillac SRX SUV) total: 41 correct: 39 acc: 95.12%
* class: 52 (2007 Cadillac Escalade EXT Crew Cab) total: 44 correct: 15 acc: 34.09%
* class: 53 (2012 Chevrolet Silverado 1500 Hybrid Crew Cab) total: 40 correct: 11 acc: 27.50%
* class: 54 (2012 Chevrolet Corvette Convertible) total: 39 correct: 18 acc: 46.15%
* class: 55 (2012 Chevrolet Corvette ZR1) total: 46 correct: 19 acc: 41.30%
* class: 56 (2007 Chevrolet Corvette Ron Fellows Edition Z06) total: 37 correct: 20 acc: 54.05%
* class: 57 (2012 Chevrolet Traverse SUV) total: 44 correct: 25 acc: 56.82%
* class: 58 (2012 Chevrolet Camaro Convertible) total: 44 correct: 34 acc: 77.27%
* class: 59 (2010 Chevrolet HHR SS) total: 36 correct: 0 acc: 0.00%
* class: 60 (2007 Chevrolet Impala Sedan) total: 43 correct: 0 acc: 0.00%
* class: 61 (2012 Chevrolet Tahoe Hybrid SUV) total: 37 correct: 7 acc: 18.92%
* class: 62 (2012 Chevrolet Sonic Sedan) total: 44 correct: 21 acc: 47.73%
* class: 63 (2007 Chevrolet Express Cargo Van) total: 29 correct: 17 acc: 58.62%
* class: 64 (2012 Chevrolet Avalanche Crew Cab) total: 45 correct: 29 acc: 64.44%
* class: 65 (2010 Chevrolet Cobalt SS) total: 41 correct: 27 acc: 65.85%
* class: 66 (2010 Chevrolet Malibu Hybrid Sedan) total: 38 correct: 6 acc: 15.79%
* class: 67 (2009 Chevrolet TrailBlazer SS) total: 40 correct: 13 acc: 32.50%
* class: 68 (2012 Chevrolet Silverado 2500HD Regular Cab) total: 38 correct: 31 acc: 81.58%
* class: 69 (2007 Chevrolet Silverado 1500 Classic Extended Cab) total: 42 correct: 8 acc: 19.05%
* class: 70 (2007 Chevrolet Express Van) total: 35 correct: 8 acc: 22.86%
* class: 71 (2007 Chevrolet Monte Carlo Coupe) total: 45 correct: 1 acc: 2.22%
* class: 72 (2007 Chevrolet Malibu Sedan) total: 44 correct: 3 acc: 6.82%
* class: 73 (2012 Chevrolet Silverado 1500 Extended Cab) total: 43 correct: 0 acc: 0.00%
* class: 74 (2012 Chevrolet Silverado 1500 Regular Cab) total: 44 correct: 1 acc: 2.27%
* class: 75 (2009 Chrysler Aspen SUV) total: 43 correct: 2 acc: 4.65%
* class: 76 (2010 Chrysler Sebring Convertible) total: 40 correct: 17 acc: 42.50%
* class: 77 (2012 Chrysler Town and Country Minivan) total: 37 correct: 22 acc: 59.46%
* class: 78 (2010 Chrysler 300 SRT-8) total: 48 correct: 32 acc: 66.67%
* class: 79 (2008 Chrysler Crossfire Convertible) total: 43 correct: 29 acc: 67.44%
* class: 80 (2008 Chrysler PT Cruiser Convertible) total: 45 correct: 28 acc: 62.22%
* class: 81 (2002 Daewoo Nubira Wagon) total: 45 correct: 12 acc: 26.67%
* class: 82 (2012 Dodge Caliber Wagon) total: 40 correct: 29 acc: 72.50%
* class: 83 (2007 Dodge Caliber Wagon) total: 42 correct: 2 acc: 4.76%
* class: 84 (1997 Dodge Caravan Minivan) total: 43 correct: 30 acc: 69.77%
* class: 85 (2010 Dodge Ram Pickup 3500 Crew Cab) total: 42 correct: 12 acc: 28.57%
* class: 86 (2009 Dodge Ram Pickup 3500 Quad Cab) total: 44 correct: 12 acc: 27.27%
* class: 87 (2009 Dodge Sprinter Cargo Van) total: 39 correct: 1 acc: 2.56%
* class: 88 (2012 Dodge Journey SUV) total: 44 correct: 43 acc: 97.73%
* class: 89 (2010 Dodge Dakota Crew Cab) total: 41 correct: 19 acc: 46.34%
* class: 90 (2007 Dodge Dakota Club Cab) total: 38 correct: 2 acc: 5.26%
* class: 91 (2008 Dodge Magnum Wagon) total: 40 correct: 19 acc: 47.50%
* class: 92 (2011 Dodge Challenger SRT8) total: 39 correct: 33 acc: 84.62%
* class: 93 (2012 Dodge Durango SUV) total: 43 correct: 35 acc: 81.40%
* class: 94 (2007 Dodge Durango SUV) total: 45 correct: 2 acc: 4.44%
* class: 95 (2012 Dodge Charger Sedan) total: 41 correct: 15 acc: 36.59%
* class: 96 (2009 Dodge Charger SRT-8) total: 42 correct: 24 acc: 57.14%
* class: 97 (1998 Eagle Talon Hatchback) total: 46 correct: 27 acc: 58.70%
* class: 98 (2012 FIAT 500 Abarth) total: 27 correct: 27 acc: 100.00%
* class: 99 (2012 FIAT 500 Convertible) total: 33 correct: 24 acc: 72.73%
* class: 100 (2012 Ferrari FF Coupe) total: 42 correct: 36 acc: 85.71%
* class: 101 (2012 Ferrari California Convertible) total: 39 correct: 37 acc: 94.87%
* class: 102 (2012 Ferrari 458 Italia Convertible) total: 39 correct: 38 acc: 97.44%
* class: 103 (2012 Ferrari 458 Italia Coupe) total: 42 correct: 29 acc: 69.05%
* class: 104 (2012 Fisker Karma Sedan) total: 43 correct: 31 acc: 72.09%
* class: 105 (2012 Ford F-450 Super Duty Crew Cab) total: 41 correct: 36 acc: 87.80%
* class: 106 (2007 Ford Mustang Convertible) total: 44 correct: 29 acc: 65.91%
* class: 107 (2007 Ford Freestar Minivan) total: 44 correct: 0 acc: 0.00%
* class: 108 (2009 Ford Expedition EL SUV) total: 44 correct: 17 acc: 38.64%
* class: 109 (2012 Ford Edge SUV) total: 43 correct: 30 acc: 69.77%
* class: 110 (2011 Ford Ranger SuperCab) total: 42 correct: 20 acc: 47.62%
* class: 111 (2006 Ford GT Coupe) total: 45 correct: 38 acc: 84.44%
* class: 112 (2012 Ford F-150 Regular Cab) total: 42 correct: 32 acc: 76.19%
* class: 113 (2007 Ford F-150 Regular Cab) total: 45 correct: 3 acc: 6.67%
* class: 114 (2007 Ford Focus Sedan) total: 45 correct: 23 acc: 51.11%
* class: 115 (2012 Ford E-Series Wagon Van) total: 37 correct: 5 acc: 13.51%
* class: 116 (2012 Ford Fiesta Sedan) total: 42 correct: 25 acc: 59.52%
* class: 117 (2012 GMC Terrain SUV) total: 41 correct: 22 acc: 53.66%
* class: 118 (2012 GMC Savana Van) total: 68 correct: 31 acc: 45.59%
* class: 119 (2012 GMC Yukon Hybrid SUV) total: 42 correct: 16 acc: 38.10%
* class: 120 (2012 GMC Acadia SUV) total: 44 correct: 20 acc: 45.45%
* class: 121 (2012 GMC Canyon Extended Cab) total: 40 correct: 6 acc: 15.00%
* class: 122 (1993 Geo Metro Convertible) total: 44 correct: 43 acc: 97.73%
* class: 123 (2010 HUMMER H3T Crew Cab) total: 39 correct: 10 acc: 25.64%
* class: 124 (2009 HUMMER H2 SUT Crew Cab) total: 43 correct: 0 acc: 0.00%
* class: 125 (2012 Honda Odyssey Minivan) total: 42 correct: 37 acc: 88.10%
* class: 126 (2007 Honda Odyssey Minivan) total: 41 correct: 8 acc: 19.51%
* class: 127 (2012 Honda Accord Coupe) total: 39 correct: 8 acc: 20.51%
* class: 128 (2012 Honda Accord Sedan) total: 38 correct: 7 acc: 18.42%
* class: 129 (2012 Hyundai Veloster Hatchback) total: 41 correct: 28 acc: 68.29%
* class: 130 (2012 Hyundai Santa Fe SUV) total: 42 correct: 9 acc: 21.43%
* class: 131 (2012 Hyundai Tucson SUV) total: 43 correct: 25 acc: 58.14%
* class: 132 (2012 Hyundai Veracruz SUV) total: 42 correct: 2 acc: 4.76%
* class: 133 (2012 Hyundai Sonata Hybrid Sedan) total: 33 correct: 0 acc: 0.00%
* class: 134 (2007 Hyundai Elantra Sedan) total: 42 correct: 0 acc: 0.00%
* class: 135 (2012 Hyundai Accent Sedan) total: 24 correct: 14 acc: 58.33%
* class: 136 (2012 Hyundai Genesis Sedan) total: 43 correct: 14 acc: 32.56%
* class: 137 (2012 Hyundai Sonata Sedan) total: 39 correct: 2 acc: 5.13%
* class: 138 (2012 Hyundai Elantra Touring Hatchback) total: 42 correct: 1 acc: 2.38%
* class: 139 (2012 Hyundai Azera Sedan) total: 42 correct: 7 acc: 16.67%
* class: 140 (2012 Infiniti G Coupe IPL) total: 34 correct: 16 acc: 47.06%
* class: 141 (2011 Infiniti QX56 SUV) total: 32 correct: 19 acc: 59.38%
* class: 142 (2008 Isuzu Ascender SUV) total: 40 correct: 3 acc: 7.50%
* class: 143 (2012 Jaguar XK XKR) total: 46 correct: 19 acc: 41.30%
* class: 144 (2012 Jeep Patriot SUV) total: 44 correct: 12 acc: 27.27%
* class: 145 (2012 Jeep Wrangler SUV) total: 43 correct: 43 acc: 100.00%
* class: 146 (2012 Jeep Liberty SUV) total: 44 correct: 34 acc: 77.27%
* class: 147 (2012 Jeep Grand Cherokee SUV) total: 45 correct: 16 acc: 35.56%
* class: 148 (2012 Jeep Compass SUV) total: 42 correct: 28 acc: 66.67%
* class: 149 (2008 Lamborghini Reventon Coupe) total: 36 correct: 32 acc: 88.89%
* class: 150 (2012 Lamborghini Aventador Coupe) total: 43 correct: 18 acc: 41.86%
* class: 151 (2012 Lamborghini Gallardo LP 570-4 Superleggera) total: 35 correct: 25 acc: 71.43%
* class: 152 (2001 Lamborghini Diablo Coupe) total: 44 correct: 32 acc: 72.73%
* class: 153 (2012 Land Rover Range Rover SUV) total: 42 correct: 36 acc: 85.71%
* class: 154 (2012 Land Rover LR2 SUV) total: 42 correct: 24 acc: 57.14%
* class: 155 (2011 Lincoln Town Car Sedan) total: 39 correct: 18 acc: 46.15%
* class: 156 (2012 MINI Cooper Roadster Convertible) total: 36 correct: 12 acc: 33.33%
* class: 157 (2012 Maybach Landaulet Convertible) total: 29 correct: 12 acc: 41.38%
* class: 158 (2011 Mazda Tribute SUV) total: 36 correct: 25 acc: 69.44%
* class: 159 (2012 McLaren MP4-12C Coupe) total: 44 correct: 43 acc: 97.73%
* class: 160 (1993 Mercedes-Benz 300-Class Convertible) total: 48 correct: 32 acc: 66.67%
* class: 161 (2012 Mercedes-Benz C-Class Sedan) total: 45 correct: 2 acc: 4.44%
* class: 162 (2009 Mercedes-Benz SL-Class Coupe) total: 36 correct: 16 acc: 44.44%
* class: 163 (2012 Mercedes-Benz E-Class Sedan) total: 43 correct: 15 acc: 34.88%
* class: 164 (2012 Mercedes-Benz S-Class Sedan) total: 44 correct: 8 acc: 18.18%
* class: 165 (2012 Mercedes-Benz Sprinter Van) total: 41 correct: 38 acc: 92.68%
* class: 166 (2012 Mitsubishi Lancer Sedan) total: 47 correct: 29 acc: 61.70%
* class: 167 (2012 Nissan Leaf Hatchback) total: 42 correct: 36 acc: 85.71%
* class: 168 (2012 Nissan NV Passenger Van) total: 38 correct: 24 acc: 63.16%
* class: 169 (2012 Nissan Juke Hatchback) total: 44 correct: 38 acc: 86.36%
* class: 170 (1998 Nissan 240SX Coupe) total: 46 correct: 25 acc: 54.35%
* class: 171 (1999 Plymouth Neon Coupe) total: 44 correct: 4 acc: 9.09%
* class: 172 (2012 Porsche Panamera Sedan) total: 43 correct: 29 acc: 67.44%
* class: 173 (2012 Ram C/V Cargo Van Minivan) total: 41 correct: 2 acc: 4.88%
* class: 174 (2012 Rolls-Royce Phantom Drophead Coupe Convertible) total: 30 correct: 27 acc: 90.00%
* class: 175 (2012 Rolls-Royce Ghost Sedan) total: 38 correct: 11 acc: 28.95%
* class: 176 (2012 Rolls-Royce Phantom Sedan) total: 44 correct: 33 acc: 75.00%
* class: 177 (2012 Scion xD Hatchback) total: 41 correct: 32 acc: 78.05%
* class: 178 (2009 Spyker C8 Convertible) total: 45 correct: 15 acc: 33.33%
* class: 179 (2009 Spyker C8 Coupe) total: 42 correct: 9 acc: 21.43%
* class: 180 (2007 Suzuki Aerio Sedan) total: 38 correct: 15 acc: 39.47%
* class: 181 (2012 Suzuki Kizashi Sedan) total: 46 correct: 14 acc: 30.43%
* class: 182 (2012 Suzuki SX4 Hatchback) total: 42 correct: 24 acc: 57.14%
* class: 183 (2012 Suzuki SX4 Sedan) total: 40 correct: 0 acc: 0.00%
* class: 184 (2012 Tesla Model S Sedan) total: 38 correct: 35 acc: 92.11%
* class: 185 (2012 Toyota Sequoia SUV) total: 38 correct: 34 acc: 89.47%
* class: 186 (2012 Toyota Camry Sedan) total: 43 correct: 4 acc: 9.30%
* class: 187 (2012 Toyota Corolla Sedan) total: 43 correct: 5 acc: 11.63%
* class: 188 (2012 Toyota 4Runner SUV) total: 40 correct: 32 acc: 80.00%
* class: 189 (2012 Volkswagen Golf Hatchback) total: 43 correct: 31 acc: 72.09%
* class: 190 (1991 Volkswagen Golf Hatchback) total: 46 correct: 42 acc: 91.30%
* class: 191 (2012 Volkswagen Beetle Hatchback) total: 42 correct: 39 acc: 92.86%
* class: 192 (2012 Volvo C30 Hatchback) total: 41 correct: 29 acc: 70.73%
* class: 193 (1993 Volvo 240 Sedan) total: 45 correct: 41 acc: 91.11%
* class: 194 (2007 Volvo XC90 SUV) total: 43 correct: 20 acc: 46.51%
* class: 195 (2012 smart fortwo Convertible) total: 40 correct: 32 acc: 80.00%
* average: 47.06%
from coop.
machengcheng2016
commented on September 17, 2024
Yeah. After updating Dassl, the acc peformance is now 55.86%. It is the outdated data augmentation that cheats me.
from coop.
Related Issues (20)
- cifar100 dataset
- Question regarding the number of runs
- How to visualize classification results? HOT 1
- CoOp on cifar-100
- Question for the code x = x[torch.arange(x.shape[0]), tokenized_prompts.argmax(dim=-1)] @ self.text_projection
- error when running coop.
- `lock.acquire()` and cannot exit training
- CSC (class-specific context) CoOp/CoCoOp in "base-new" and "cross-dataset".
- Base-to-novel generalization
- I would like to ask how long is the experimental period for coop training on imagenet?
- Running zero shot for new classes
- FloatingPointError: Loss is infinite or NaN HOT 1
- About PromptLearner
- train.py --root argument error
- 论文中他、
- t中的V是如何学习得到的
- Using BLIP instead of CLIP
- At the end of the CoOp train main.sh an additional progress bar indicates the accuracy of the output, whether the test was executed in train? HOT 1
- Question about initializing the prompt vectors?
- What does it mean if I change the shot to 0？
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from coop.