Issue regarding the config_class about dnabert_2 HOT 4 CLOSED

You need to use transformers==4.29.0

from dnabert_2.

Thank you @akd13 now the above issue is resolved

from dnabert_2.

@akd13 can you help me out in understanding the below point more clearly

Some weights of BertModel were not initialized from the model checkpoint at zhihan1996/DNABERT-2-117M and are newly initialized: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

from dnabert_2.

Hey @akd13, thanks for the response.

Hey @sai-kiran7, this basically means some of the parameters in the model you are currently using does not exist in the pre-trained checkpoint. This is because the bert.pooler layer are used for transforming the last hidden state of the [CLS] token, and in our pre-training stage, we do not have any training target that relates to [CLS]. So we do not need the bert.pooler layer in the pre-training stage.

from dnabert_2.