Questions about the .pte file size about executorch HOT 4 CLOSED

The model I used is official llama2-7b pth model, which parameters size is close to 13GB. the fp32 pte file I got by using following steps:

python3 -m examples.models.llama2.export_llama -c ../llama2/consolidated.00.pth -p ../llama2/params.json

The size of llama.pte file I got is 26GB. This is already twice the weight of the original model file, is this reasonable? What are the main contents of the pte file besides the model weights?

Hi thanks for trying out llama model. Please note that by default we are converting fp16 weights to fp32 which is the reason of doubling the pte file size. The reason for this decision was we didn’t support fp16 that well before. Now that we have full support for fp16 on portable kernels, we will change the default behavior of export_llama to be exporting fp16.

BTW you can pass a -H flag to the script to export to fp16.

from executorch.

@larryliu0820 thanks for your reply!
my question is that llama2/consolidated.00.pth is a fp32 model, not half, there is no fp16 weight to fp32. but why pte file still double?
BTW, the pte file size if 16GB when I pass a -H flag to the script by using fp16.

from executorch.

@huangzhiyuan AFAIK llama2/consolidated.00.pth is a checkpoint with fp16 weights.

from executorch.

You are right. I make a mistake, and it makes sense.
Now close the issue.

from executorch.