legacyai / tf-transformers Goto Github PK

This is great work!!! I have problem with TF2+HF with too many errors, reported to TF2, I aim to switch to tf-transformers. Though library did not work in colab, I guess there are some missing files? Thanks.

Would it be possible to add bert2gpt conversion? Thanks.

You said this library is 90 times faster than HF transformers, but there is no benchmark about it.
https://github.com/legacyai/tf-transformers/tree/main/benchmarks

I was reading the code for the HF GPT2 benchmark, and it seems like key-value caching is not being used? This is pretty important for any kind of autoregressive generation and would greatly speed up the decoding time. HF models have had support for key-value caching for a while, see config arguments use_cache and past_key_values here: https://huggingface.co/docs/transformers/model_doc/gpt2#transformers.GPT2LMHeadModel.

I think it would be important for this project to re-benchmark the HF models with key-value caching enabled, as that is standard practice and without it the HF numbers are being handicapped.