Is there an offline way to download summarizer? about bert-extractive-summarizer HOT 9 OPEN

When "from summarizer import Summarizer" done, came "model = Summarizer()" !...This part could not download indeed. I want to give up...

from bert-extractive-summarizer.

You will have to download at least once from the huggingface s3 repo. Once it is downloaded, it is cached and can be used offline. You can also download/train it separately and use it as a pretrained model.

from bert-extractive-summarizer.

But how to download it separately?

from bert-extractive-summarizer.

@zysNLP It seems to be using "bert-large-uncased", so maybe you can just put your pre-downloaded version at the point where the downloaded one ends up?

The download asks for https://s3.amazonaws.com/models.huggingface.co/bert/bert-large-uncased-pytorch_model.bin where i can also download it manually

from bert-extractive-summarizer.

@Twonki Thank you very much, but where should I put the downloaded files?

from bert-extractive-summarizer.

@zysNLP I found some cached torch files under C:\Users\XXX\.cache\torch\transformers

These are rather cryptic hashes, but one is 1gb big which is about bert, and the time written matches.
Overall there are 6 hashname-files. I´m not using torch for anything else on this machine

the big one is hashed:
'54da47087cc86ce75324e4dc9bbb5f66c6e83a7c6bd23baea8b489acc8d09aa4.4d5343a4b979c4beeaadef17a0453d1bb183dd9b084f58b84c7cc781df343ae6'
if you want I can name the others too

from bert-extractive-summarizer.

@zysNLP I´m not 100% sure, but i think it´s just the normal spacy-install of the model.

You can maybe try the spacy-documentation on manual installation.

@dmmiller612 can you give a short comment on torch vs. spacy or whether I´m pointing in completely the wrong direction?

from bert-extractive-summarizer.

@Twonki Thank you! but I use ubuntu, and found nothing about .cache\torch\transformers. Maybe use command to download is well even though need a lot of time.

from bert-extractive-summarizer.

You can define the cache directory by setting paths via the environment variables TRANSFORMERS_CACHE or TORCH_HOME.

• Official Transformers Documentation
• StackOverflow Answer

from bert-extractive-summarizer.