yunkun-zhang / cite Goto Github PK

Hi! thanks for your great work. I tried using your code for training, but the accuracy of the training was not as high as you described in the article. can you provide the weight file that you have trained?

Very nice work, can you provide 'data/gastric_cls3_ann/train_all_0.2.txt' please?

Thx

Hi, where is the paper for this method? The link does not seem to work. Thanks.

I ran the code as instructed, but encountered the following error.
While running this code: "!mim train mmcls /configs/gastric/cite/clip-vitb16_biolinkbert-l_cb_bs128_i1000.py"
Did I make any mistake here? Can you correct it for me?
Thank you a lot for your code.

When I use clip-vitb16_biolinkbert-l_cb_bs128_i1000.py as the config to run the model,there are some mistake.
Traceback (most recent call last):
File "/home2/anaconda3/envs/CITE2/lib/python3.9/site-packages/mmcls/.mim/tools/train.py", line 205, in
main()
File "/home2/anaconda3/envs/CITE2/lib/python3.9/site-packages/mmcls/.mim/tools/train.py", line 193, in main
train_model(
File "/home2/anaconda3/envs/CITE2/lib/python3.9/site-packages/mmcls/apis/train.py", line 233, in train_model
runner.run(data_loaders, cfg.workflow)
File "/home2/anaconda3/envs/CITE2/lib/python3.9/site-packages/mmcv/runner/iter_based_runner.py", line 145, in run
iter_runner(iter_loaders[i], **kwargs)
File "/home2/anaconda3/envs/CITE2/lib/python3.9/site-packages/mmcv/runner/iter_based_runner.py", line 65, in train
outputs = self.model.train_step(data_batch, self.optimizer, **kwargs)
File "/home2/anaconda3/envs/CITE2/lib/python3.9/site-packages/mmcls/models/classifiers/base.py", line 139, in train_step
losses = self(**data)
File "/home2/anaconda3/envs/CITE2/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home2/anaconda3/envs/CITE2/lib/python3.9/site-packages/mmcv/runner/fp16_utils.py", line 119, in new_func
return old_func(*args, **kwargs)
File "/home2/anaconda3/envs/CITE2/lib/python3.9/site-packages/mmcls/models/classifiers/base.py", line 83, in forward
return self.forward_train(img, **kwargs)
File "/home2/anaconda3/envs/CITE2/lib/python3.9/site-packages/mmcls/models/classifiers/image.py", line 141, in forward_train
x = self.extract_feat(img)
File "/home2/anaconda3/envs/CITE2/lib/python3.9/site-packages/mmcls/models/classifiers/image.py", line 117, in extract_feat
x = self.neck(x)
File "/home2/anaconda3/envs/CITE2/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home2/CITE-main/models/prompt_vit.py", line 139, in forward
x = super().forward(x)
File "/home2/anaconda3/envs/CITE2/lib/python3.9/site-packages/torch/nn/modules/linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: mat1 and mat2 must have the same dtype

In the file "zeroshot_clip-vitb16_bs128_i1000.py", no arch parameter has been passed to the "head" in the code"head=dict(type='CLIPTextHead', texts=[prompt.format(t) for t in TEXTS]))".

But the class CLIPTextHead requires a parameter "arch" in the init funciton

which cause TypeError