Comments (4)
Hi @Colanim,
I'm sorry there is a typo here. The warning should be:
Warning : Empty reference sentence; Setting recall to be 0.
The reason for getting this warning is that some of the reference sentences (ground truth translations) are empty sentences.
Can you provide the scripts of how you get this message?
It might be either you have a "empty" reference sentence or something wrong with the API.
Best,
Felix
from bert_score.
hi @Colanim thank you for providing a detailed notebook to replicate the problem.
I took a look and the issue is that the nlp
package expects a list of list of strings for the references.
if you modify the code to the following, it will work correctly.
rouge.add(lp.strip(), [lg.strip()])
We will communicate with the nlp
package to see how best to improve on this input format issue.
from bert_score.
Thanks for your answer @felixgwu
Here is a Notebook reproducing the issue (using nlp
package). None of the candidate / reference are empty, but I'm still having the warning : https://colab.research.google.com/drive/1uiS89fnHMG7HV_cYxp3r-_LqJQvNNKs9?usp=sharing
from bert_score.
Hi,
Could someone help me understand why is the BERTScore recall is non zero here. Isn't zero the right score for precision as well as recall with empty candidate sentence?
score([''], ['hello how are you'], lang='en')
This is what the model returns now:
Warning: Empty candidate sentence detected; setting precision to be 0.
(tensor([0.]), tensor([0.7738]), tensor([0.]))
best,
Anupama
from bert_score.
Related Issues (20)
- BERTScore complexity ? HOT 2
- Use fine tuned model HOT 6
- get_wmt18_seg_result question HOT 3
- exception when passing numpy arrays as inputs HOT 1
- requests.exceptions.ProxyError: HTTPSConnectionPool(host='huggingface.co', port=443) HOT 2
- inputs format & running warning HOT 1
- Different roberta-large results with transformers>=4.17.0 HOT 1
- will bert-score cli command automatically use gpu? how? HOT 1
- Could you please add some models for Ukrainian language? HOT 4
- Bertscore not working for Hindi language
- A negative rescaled value HOT 10
- a little difference in results. HOT 3
- RuntimeError when using bert-base-chinese offline HOT 1
- Warning HOT 1
- BERTScore with covid-twitter-bert
- Padding token ID HOT 2
- Run bert_score on TPU (google colab) instead of gpu.
- Calculate embeddings once HOT 1
- Tensor size error with multiple refs tweets
- Can bertscore calculate semantic similarity between languages?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bert_score.