Codes for paper EconNLI: Evaluating Large Language Models on Economics Reasoning

Our dataset can be found at the Dataset folder.

The process for dataset construction (of the training set) can be found under the DatasetConstruction folder.

The results of Table 3 can be reproduced using DatasetEvaluation/fine-tuned_model_evaluation.ipynb.

The results of Table 4 can be reproduced using DatasetEvaluation/zero-shot_model_evaluation.ipynb and DatasetEvaluation/ICL_COT_model_evaluation.ipynb.

The results of Table 5 and 6 can be reproduced using DatasetEvaluation/hallucination_detection.ipynb.