Comments (5)
Coopercoppers
commented on July 29, 2024
- rc_head_label应该是subj_head, obj_head, relation。你这上边写的1，1是subj_head, subj_tail，这是不对的。
- 不需要对实体进行类型划分的就设成None
- 这个可能是你的句子tokenize之后长度超过了512，检查下你的数据，看看哪个超了。
from pfn.
Yuknoshita
commented on July 29, 2024
好的，感谢您的指导
…------------------ 原始邮件 ------------------
发件人: "Coopercoppers/PFN" ***@***.***>;
发送时间: 2021年10月30日(星期六) 下午4:02
***@***.***>;
***@***.******@***.***>;
主题: Re: [Coopercoppers/PFN] 其他数据集训练出错 (Issue #5)
rc_head_label应该是subj_head, obj_head, relation。你这上边写的1，1是subj_head, subj_tail，这是不对的。
不需要对实体进行类型划分的就设成None
这个可能是你的句子tokenize之后长度超过了512，检查下你的数据，看看哪个超了。
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
from pfn.
Yuknoshita
commented on July 29, 2024
不好意思再次打扰您。
我按照您说的检查出错的句子的长度，发现我一个批次的句子都没有超过128的，但是tokenizer(x)后num_tokens=588，确实大于512.
我不是很明白tokenizer的原理，好像我将批次降低之后他仍然会超过。
这个bug改了一下午了，很是苦恼，烦请您指导一下该从哪里改，谢谢
…------------------ 原始邮件 ------------------
发件人: "Coopercoppers/PFN" ***@***.***>;
发送时间: 2021年10月30日(星期六) 下午4:24
***@***.***>;
***@***.******@***.***>;
主题: Re: [Coopercoppers/PFN] 其他数据集训练出错 (Issue #5)
Closed #5.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
from pfn.
Coopercoppers
commented on July 29, 2024
你把tokenizer后长度为588的那个句子拿出来打印一下再分析。
from pfn.
Yuknoshita
commented on July 29, 2024
这个我之前已经试过了，很奇怪的是，我的tokenizer()是对一个字符list进行的，最后得到了一个一维的输出，就是这个588的向量，在我的理解中，tokenizer应该是对每个句子进行的吧。。。
可能之前的数据处理还是有问题，谢谢您的回复
…------------------ 原始邮件 ------------------
发件人: "Coopercoppers/PFN" ***@***.***>;
发送时间: 2021年10月30日(星期六) 晚上6:07
***@***.***>;
***@***.******@***.***>;
主题: Re: [Coopercoppers/PFN] 其他数据集训练出错 (Issue #5)
你把tokenizer后长度为588的那个句子拿出来打印一下再分析。
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
from pfn.
Related Issues (20)
- Small inconsistency, or not? HOT 1
- question about eval_metric HOT 1
- When training model, is it necessary to set args.do_eval=True?
- FileNotFoundError: [Errno 2] No such file or directory: 'data/data/NYT/ner2idx.json' HOT 1
- OOM for my own bigger datasets HOT 2
- Question about entity extraction of out-of-triples HOT 1
- chinese custom dataset HOT 2
- evaluate the model with customized input HOT 2
- 请问消融实验的编码模式encoding scheme应该怎么设置呀 HOT 2
- Nest下的处理 HOT 2
- SEMEVAL dataset HOT 1
- 为什么PFN-nested model在关系训练时候可以利用实体tail信息呢？ HOT 2
- 你好，能否提供适用于nested-pfn的inference.py呢？感谢！ HOT 1
- 关于头尾实体特征和关系特征的问题 HOT 5
- 关于实验结果复现的问题 HOT 2
- 关于实体抽取和关系抽取F1分数问题 HOT 2
- 关于OOT（Out-of-triple）数据集问题 HOT 2
- why use albert-xxlarge instead of bert-base when training on some datasets? HOT 2
- Question about the tail re unit HOT 6
- 有一个报错问题，不知道是环境不对还是， HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pfn.