ysmiura / ifcc Goto Github PK

Hi,
Could you please share the pre-trained DenseNet model: chexpert_densenet/model_auc14.dict.gz

Thanks

Hi there,

May I ask if the default configuration of the inference is the one you used in your experiments? I ran the infer.py as instructed but got fEnt and fEntNLI slightly different from ones shown in the paper. Is there something I'd need to modify? Thank you.

Hi,
In the article, the metrics related to clinical accuracy (chexBert-labeler) F-1, P, R are reported in their Micro average version. Most papers in the area report the Macro average, making this method difficult to compare. Could you provide the Macro average version of these metrics (Table 2)?

Thanks

Hello, ysmiura
Thanks for opening your source code. It's very nice works.
when i running the code，i have a few questions, it's mainly about the data preprocessing.
i notice the sections file createed by create_sections_file.py are IMPRESSIONS instead FINDINGS, is that right？
and if i should merge mimic_cxr_number_labeled.csv to one csv file and zip it to mimic-cxr-2.0.0-chexpert.csv.gz? and if the negbio(https://github.com/MIT-LCP/mimic-cxr/tree/master/txt/negbio) in the ifcc code is necessary？

the other question is i have applied the license of mimic_cxr, while i have not seen the mimic-cxr-2.0.0-metadata.csv.gz and mimic-cxr-2.0.0-split.csv.gz, could you please tell me where i can find it?

thank you very much,
best wishes

when trying to use the inference.py using the following parameters
!python infer.py --cuda --corpus open-i --stanza-download --splits /content/ifcc/meta.txt --cache-data cache --batch-size 24 --entity-match /content/open-i_ner.txt.gz --cider-df /content/drive/MyDrive/open-i_train-df.bin.gz --img-model densenet --img-pretrained resources/chexpert_auc14.dict.gz /content/open-i /content/ifcc/checkpoints/checkpoint_nll-bs-emnli.dict.gz resources/glove_mimic-cxr_train.512.txt.gz out_infer
we get the following error :
Unexpected exception: Traceback (most recent call last): File "infer.py", line 88, in main EpochLog.log_datasets(logger, pbar_vals, 0, 0, None, val_loader, test_loader, save=False, progress=True) File "/content/ifcc/clinicgen/log.py", line 30, in log_datasets results[split] = logger.evaluator.generate_and_eval(data_loader, prog_name) File "/content/ifcc/clinicgen/eval.py", line 668, in generate_and_eval scores, scores_detailed = self.eval(report_ids, refs, hypos, tfidf_vectorizer) File "/content/ifcc/clinicgen/eval.py", line 559, in eval mse, sde, msn, sdn = self.entity_matcher.score(ref_ids, hypos_l) File "/content/ifcc/clinicgen/eval.py", line 156, in score _, _, _, stats = self.nli.sentence_scores_bert_score(texts1, texts2, label='all', prf=self.prf) File "/content/ifcc/clinicgen/nli.py", line 221, in sentence_scores_bert_score _, _, bf = self.bert_score_model.score(bsents1, bsents2) File "/content/ifcc/clinicgen/nli.py", line 535, in score device=self.device, batch_size=self.batch_size, all_layers=self.all_layers).cpu() File "/usr/local/lib/python3.7/dist-packages/bert_score/utils.py", line 520, in bert_cos_score_idf sen_batch, model, tokenizer, idf_dict, device=device, all_layers=all_layers File "/usr/local/lib/python3.7/dist-packages/bert_score/utils.py", line 399, in get_bert_embedding model, padded_sens[i : i + batch_size], attention_mask=mask[i : i + batch_size], all_layers=all_layers, File "/usr/local/lib/python3.7/dist-packages/bert_score/utils.py", line 309, in bert_encode out = model(x, attention_mask=attention_mask, output_hidden_states=all_layers) File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) TypeError: forward() got an unexpected keyword argument 'output_hidden_states'

Hello and thank you for sharing your work!
I was wondering how can I make an inference with your provided pretrained model checkpoint.
Thank you very much in advance!

Thanks for your great work.
When i run python train.py --cuda --corpus mimic-cxr --cache-data cache --epochs 32 --batch-size 24 --cider-df mimic-cxr_train-df.bin.gz --entity-match mimic-cxr_ner.txt.gz --img-model densenet --img-pretrained resources/chexpert_auc14.dict.gz --cider-df mimic-cxr_train-df.bin.gz --bert-score distilbert-base-uncased --corpus mimic-cxr --lr-scheduler trans MIMIC_CXR_ROOT resources/glove_mimic-cxr_train.512.txt.gz out_m2trans_nll,
i got the message as below:
Unexpected exception：Traceback（most recent call last）：
File “Train.py” ，line 165， in main
save=args.log_models, progress=True)
File “/home/mayt/ifcc-master/clinicgen/log.py”, line 25, in log_datasets
logger.evaluator.setup()
File “/home/mayt/ifcc-master/clinicgen/eval.py”,line 752, in setup
model = SimpleNLI.load_model(model)
File "home/mayt/ifcc-master/clinicgen/nli.py",line 599. in load_model
bertnli.load_state_dict(states_dict)
...
RuntimeError: Error(s) in loading state_dict for BERTNLI"
Missing key(s) in state_dict:"bert.embeddings.position_ids".

because the reason of net，i load bert-base-uncased and distilbert-base-uncased locally，and the number of line aforementioned may be small changes。（i had run the ./download.sh ）

Hi,

Would you please describe what steps (transform, pad, resize, ...) have you applied to provide CNN image features?

Thanks,

HI there, thanks for your work! May I ask what version of libraries you're using? Including pytorch, torchvision, transformers, stanza ? I'm keeping seeing errors of stop Iteration and Expected object of device type cuda but got device type cpu for argument #3 'index'.

Thanks for your great work. When I am following the tutorial to reproduce the results, some problems are raised. I would be appreciated if you tell me how to fix this issue.

Step:
python train.py --cuda --corpus mimic-cxr --cache-data cache --epochs 32 --batch-size 24 --entity-match mimic-cxr_ner.txt.gz --img-model densenet --img-pretrained chexpert_densenet/model_auc14.dict.gz --bert-score distilbert-base-uncased --corpus mimic-cxr --lr-scheduler trans MIMIC_CXR_ROOT resources/glove_mimic-cxr_train.512.txt.gz out_m2trans_nll

Return:
Segmentation fault (core dumped)

Hi,
I am trying to run all the trained models on MIMIC-CXR data to get the BLEU, ROGUE metrics. How do I get that?. Do I have to get the prediction of the trained models and compute the metric separately?. Your help is much appreciated.