请教下ceval数据集中计算PPL的方法 about opencompass HOT 4 CLOSED

We recommend using English or English & Chinese for issues so that we could have broader discussion.

from opencompass.

We recommend using English or English & Chinese for issues so that we could have broader discussion.

Okay, from now on, I will ask my questions in English.

from opencompass.

Given a model and an input text sequence, perplexity measures how likely the model is to generate the input text sequence.
As a metric, it can be used to evaluate how well the model has learned the distribution of the text it was trained on.

We are not predicting anything here but measuring the likelihood of generating the sequence, which is the literal meaning of perplexity.
You could refer to https://huggingface.co/docs/transformers/perplexity for more details.

from opencompass.

Given a model and an input text sequence, perplexity measures how likely the model is to generate the input text sequence.
As a metric, it can be used to evaluate how well the model has learned the distribution of the text it was trained on.

We are not predicting anything here but measuring the likelihood of generating the sequence, which is the literal meaning of perplexity. You could refer to https://huggingface.co/docs/transformers/perplexity for more details.

OK， I got it，thank you very much.

from opencompass.