This repo contains a comprehensive metric toolset for evaluation NLG, videoQA, etc. tasks. We also offer a reading list of related papers and repositories.

MMEval is a novel LLM-based metric designed for Video Question Answering, which can utilize multimodal information to evaluate answers. You can find more in our new CVPR2024 paper CUVA and github repo.

Thank the following repositories and authors for their contributions to VideoQA Metrics, Their work has been instrumental in enhancing our project and we are grateful for their efforts.

• BLEU (ACL2002) paper
• ROUGE (ACL2004) paper
• METEOR (ACL2005) paper
• CIDEr (CVPR2015) paper
• MoverScore (EMNLP2019) paper repository
• COMET (EMNLP2020) paper
• BLEURT (ACL2020) paper repository
• UniEval (EMNLP2022) paper repository
• FunQA (2023) paper repository
• MMEval (CVPR2024) paper repository