Current code_bert_score is able to handle the case wh

Thanks <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-u

Hi <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="

Getting IndexError when any of 'cands', 'refs' and 'sources' contains empty string about code-bert-score HOT 6 OPEN

neulab commented on May 25, 2024

Getting IndexError when any of 'cands', 'refs' and 'sources' contains empty string

from code-bert-score.

Comments (6)

urialon commented on May 25, 2024 1

Thanks @terryyz !
I just pushed a fix.

In case the cand or ref is an empty string, I am now setting both precision and recall to zero.
It is now working when sources are provided or not provided.

Please let us know if you have any questions or find any other issues!
Best,
Uri

from code-bert-score.

terryyz commented on May 25, 2024 1

Hi @littlepopy, the details were discussed in Appendix B.2 in the paper, where the data was collected from HumanEval-X. Besides, you may also find our paper useful. This folder may provide all the data you need 😃

from code-bert-score.

urialon commented on May 25, 2024 1

Hi @littlepopy ,
Thank you for your interest in our work.

Please see the paragraph "functional correctness experiments" in Section 3.2 in the paper https://arxiv.org/pdf/2302.05527.pdf

And also, Appendix B.2.

Let us know if you have any questions!
Uri

from code-bert-score.

littlepopy commented on May 25, 2024

Can I ask you the data provided under code-bert-score/evaluation come from where? The generation codes ### (humaneval_hyps.txt) of 13025 lines, with the corresponding NL**(humaneval_srcs.txt)** and reference code**(humaneval_refs.txt)** . And if we want to use the data, is it be promised?

from code-bert-score.

littlepopy commented on May 25, 2024

Thanks for your reply. It is very useful. Besides, thanks for all the work you have done. Wish you a good future.

…

------------------ 原始邮件 ------------------ 发件人: "Uri ***@***.***>; 发送时间: 2024年1月4日(星期四) 下午4:49 收件人: ***@***.***>; 抄送: ***@***.***>; ***@***.***>; 主题: Re: [neulab/code-bert-score] Getting IndexError when any of 'cands', 'refs' and 'sources' contains empty string (Issue #4) Hi @littlepopy , Thank you for your interest in our work. Please see the paragraph "functional correctness experiments" in Section 3.2 in the paper https://arxiv.org/pdf/2302.05527.pdf And also, Appendix B.2. Let us know if you have any questions! Uri — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

from code-bert-score.

littlepopy commented on May 25, 2024

Thanks for your reply. It is very useful. Besides, thanks for all the work you have done. Wish you a good future.

…

------------------ 原始邮件 ------------------ 发件人: "Terry ***@***.***>; 发送时间: 2024年1月4日(星期四) 下午4:48 收件人: ***@***.***>; 抄送: ***@***.***>; ***@***.***>; 主题: Re: [neulab/code-bert-score] Getting IndexError when any of 'cands', 'refs' and 'sources' contains empty string (Issue #4) Hi @littlepopy, the details were discussed in Appendix B.2 in the paper, where the data was collected from [HumanEval-X](https://huggingface.co/datasets/THUDM/humaneval-x]. Besides, you may also find our paper useful. This folder may provide all the data you need 😃 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

from code-bert-score.

Getting IndexError when any of 'cands', 'refs' and 'sources' contains empty string about code-bert-score HOT 6 OPEN

Comments (6)

Related Issues (4)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent