AttributeError: 'ORTModule' object has no attribute 'resize_token_embeddings' about ort HOT 6 CLOSED

replacing
model.resize_token_embeddings(len(tokenizer))
with

model_to_resize` = model.module if hasattr(model, 'module') else model
model_to_resize.resize_token_embeddings(len(tokenizer))

src : huggingface/transformers#7146

does not give the above error message , I would like to know if this is a right fix.

from ort.

Hi Bhavya,

In general ORTModule does not forward the attributes of the underlying model. For now, yes, this is the correct fix. However, this API is subject to change as exposing the attribute .module to get the underlying model has led to issues elsewhere. Likely the name will change to something bit less friendly, e.g. ._original_module.

For HF-GPT2, you should be able to use the following repository as-is:
https://github.com/microsoft/huggingface-transformers

In the above, the ORTModule is inserted in the huggingface trainer.py script itself:
https://github.com/microsoft/huggingface-transformers/blob/c1b959563ebb677f744382ea95ca891295092187/src/transformers/trainer.py#L1109

And there's another tweak here to ensure the DDP wrapping occurs correctly for >1 gpu:
https://github.com/microsoft/huggingface-transformers/blob/c1b959563ebb677f744382ea95ca891295092187/src/transformers/trainer.py#L926

I run the model using the following launch command:
python -m torch.distributed.launch --nproc_per_node 8 huggingface-transformers/examples/pytorch/language-modeling/run_clm.py --model_name_or_path gpt2 --dataset_name wikitext --dataset_config_name wikitext-2-raw-v1 --do_train --label_smoothing 0.1 --max_steps 260 --logging_steps 1 --overwrite_output_dir --output_dir gpt2-results --logging_dir gpt2-tensorboard --per_device_train_batch_size 8 --fp16 --dataloader_num_workers 1 --ort --skip_memory_metrics

Let me know if you have other issues.

-- Suffian

from ort.

Thank you Suffian. I will try from https://github.com/microsoft/huggingface-transformers.
Bhavya

from ort.

Hi @bmedishe, is your issue resolved now?

from ort.

Yes @natke Thank you

from ort.

Great, thanks. I will close this issue. Please reach out again if you need to.

from ort.