Tokenizer model_max_length about alignment-handbook HOT 5 OPEN

I have the same exact question, I've changed the max_seq_length in config_lora.yaml to 4096, but I get these warnings:

Token indices sequence length is longer than the specified maximum sequence length for this model (2485 > 2048). Running this sequence through the model will result in indexing errors

from alignment-handbook.

Same problem here!

from alignment-handbook.

I found that it comes from here. During initialization, tokenizer does not read the max_length from the model.

As a quick hack, I was able to update it to 4096 and then reinstall alignment-handbook by doing

cd ./alignment-handbook/
python -m pip install .

from alignment-handbook.

@lewtun Would you be able to comment on here why the max_seq_len has been hard coded instead of reading from config? Any reason for this decision? Thanks

from alignment-handbook.

I'm also curious about that, especially that zephyr-7b-beta has model_max_length set to 1000000000000000019884624838656, so these models were somehow exempt from that 😐

There is max_length argument passed in config.yaml, so setting model_max_length to some hard-coded value inside the script seems pointless

from alignment-handbook.