mukhal / promptrank Goto Github PK

I have identified a discrepancy in the codebase, specifically in the file run_instruction_Search.py. On line 25, the script attempts to import functions evaluate_retrieval andthreaded_evalfrom src.eval_utils. However, upon closer inspection of the eval_utils module, these functions do not exist.

Additional Information:

Furthermore, I have encountered difficulties in finding instructions for training the model. In the paper, it is mentioned that training with only 128 examples outperforms certain baseline models, but no explicit instructions on how to train the model are provided.

Request for Clarification:

I kindly request clarification or additional documentation on the training procedure for the model. Specifically, information on data preparation, training parameters, and any other relevant details would be greatly appreciated.