lsj2408 / transformer-m Goto Github PK

请问，在下游任务比如qm9上的微调脚本方便提供吗？

Great Work!
Input：atom_type + xyz
How can we get the embedding of atom and xyz?

Thanks for your amazing work!
I wonder why download link of checkpoint is not to a '*.pt' file?

Very enlightening work. Congratulations on your great achievements in the OGB Challenge! In addition, I noticed that you have made fine-tuning on the PDBbind dataset. How should you encode the protein information? Because proteins usually contain more heavy atoms, do you directly use Transformer-M to encode proteins?

Thank you for your code! It's well written~
I have a few questions on finetuning task on PDB-Bind. I sincerely look forward to your kindest reply!
1. Inputs. Which features of protein are used as input? and whether pocket data (sub sequence) or full sequence are used?
2. Model architecture. Whether protein data and ligand data are sent into seperate encoders or they are sent into the same encoder? If different encoders are used, what is the type of protein encoder? and how the extracted features of protein and ligand gathered together for later prediction?
Thank you again for your clarifications for them!
btw, may I ask when the finetuning code for PDB bind will be released? Thanks

Hi,

Would it be possible to provide the commands for training a model on QM9 from scratch? This is mentioned in appendix B5 when investigating the effectiveness of pre-training.

Kind regards,

Rob

Hi, here is what confused me. I tried to run the code provided in README.md:

but I got the following error:

evaluate.py: error: unrecognized arguments: --add-3d --num-3d-bias-kernel 128 --droppath-prob 0.1 --act-dropout 0.1 --mode-prob 0.2,0.2,0.6

It seems strange, could you help me with it?

Hi,

It's a very interesting model. Would you mind providing the code for preprocessing the data of PDBbind and fine-tuning?

Thank you for your excellent work!

May I know why you use cosine similarity to calculate loss instead of the squared error as in the paper PRE-TRAINING VIA DENOISING FOR MOLECULAR PROPERTY PREDICTION?

Thank you for your release of Transformer-M. About the settings of steps, warmup_ steps and total_ steps, do I need to divide the settings of steps by the number of GPUs used? Because I observed during training that the number of steps in each epoch of multi -gpu parallel will decrease, but the program still counts the steps according to the single card.

Hi,

Thank you for the code and the surrounding instructions!

I was trying to reproduce the results but having some difficulty making the environment work.

I installed cuda and other package versions as mentioned but torch_scatter was erroring out with "'NoneType' object has no attribute 'origin'". Looking up online, I uninstalled your recommended version and installed another one with pip install --no-index torch-scatter -f https://pytorch-geometric.com/whl/torch-1.7.0+cu110.html (even though I have PyTorch 1.7.1). But now, torch sparse decided to error out with

div(float a, Tensor b) -> (Tensor):
  Expected a value of type 'Tensor' for argument 'b' but instead found type 'int'.

  div(int a, Tensor b) -> (Tensor):
  Expected a value of type 'Tensor' for argument 'b' but instead found type 'int'.

The original call is:
  File "/nethome/yjakhotiya3/miniconda3/envs/Transformer-M/lib/python3.7/site-packages/torch_sparse/storage.py", line 316
        idx = self.sparse_size(1) * self.row() + self.col()

        row = torch.div(idx, num_cols, rounding_mode='floor')
              ~~~~~~~~~ <--- HERE
        col = idx % num_cols
        assert row.dtype == torch.long and col.dtype == torch.long

I also tried on other machines but was getting unknown cuda errors by torch distributed (which could be due to an unrelated driver version mismatch issue).

Did you encounter any of these issues or do you have any advice on how to navigate them?

Hi, Thank you for your code for finetuning qm9, I have a small question on the corresponding relationship between task_idx and specific task.
do you mean that the correspondence is as follows?


Thank you for your kindest reply.

Great piece of work.
Could you please mention, how to load your checkpoint?

Hi!

Thank you for introducing such an interesting model to us and sharing the code!

I'm trying to run the model only on 2D structures, would you mind providing a script for using only 2D structures to train the model (Like for PCQM4M-LSC-V2)?

I tried to change the dataset_name and set add_3D to false in the sample train script for 3D data in the readme file, but that doesn't work. I looked into the code and found that in the tasks/graph_prediction.py file , Class GraphPredictionTask, and load_dataset function, when calling BatchedDataDatset, when it set the dataset_version to "2D" for PCQM4M-LSC-V2, it gives the error in criterions/graph_predictions.py line 45: ori_pos = sample['net_input']['batched_data']['pos'], KeyError: 'pos'.

Thank you so much!

Hi,
I met a problem when I tried to load 'L12-old.pt' in finetuneqm9.sh , the program told me that the checkpoint's structure mismatched, how can I solve this problem?

论文作者您好！首先感谢您提供了开源的代码，对复现论文帮助很大，模型的效果也很赞！另外想请问下，不知道finetune相关的代码大概什么时候会上传呀？

Thanks for sharing the code of your enlightening work. I have a set of compounds that only have SMILES available. Would you recommend RDKit, Openbabel, or other tools for generating 3D structures as input into your model? Thank you!