Comments (2)
jjy260782149
commented on June 29, 2024
今天把input_ids和labels列表长度不一样的数据删掉了，就可以正确运行了，确认是tokenizer转化的问题，应该如何解决？
from paddlenlp.
jjy260782149
commented on June 29, 2024
已发现是输入的数据处理有问题，我输入的数据通常会把连续的数字作为一个字符进行转换，这样转换后有时会分配一个id，有时会分配多个id；输入的数据应该为单个字符，不能为多个字符。
from paddlenlp.
Related Issues (20)
- [Question]: 求助，chatglm2 单卡sft内存溢出 HOT 1
- [Bug]: 'NoneType' object has no attribute 'from_pretrained' -超长文本分类任务训练报错
- [Question]: 用教程里的数据微调后，调用时无法打开inference.pdiparams HOT 1
- [Docs]: 文档挂了 readthedocs page is 404 HOT 2
- [Docs]: https://github.com/PaddlePaddle/PaddleNLP/blob/develop/applications/sentiment_analysis HOT 1
- [Bug]: PaddleNLP2.8 llama2 lora 微调后导出静态图模型报错 HOT 1
- [Bug]: The file path "examples/text_to_knowledge/nptag/deploy/python/predict.py" may encounter an IndexError. HOT 3
- [Question]: 使用ernie-layout模型推理时对预测结果中的每个entity都漏最后一个字
- [Docs]: 示例里 8010 和 8011 写反了
- [Question]: 如何使用LoRA HOT 1
- [Bug]: UNIMO模型的resize_token_embeddings方法不会修改decoder的vocab_size，导致报错 HOT 1
- [Question]: 张量并行推理内存占用异常？ HOT 5
- [Question]: PaddleNLP 报错OSError: (External) CUBLAS error(1).
- [Question]: Why is PaddleNLP /applications/ not available anymore? HOT 1
- [Question]: Inferencing using finetuned ERNIE-Layout model on Taskflow
- 【LLM】模型支持列表
- [Question]: 对ERNIE模型进行微调时，报错Pointer C should not be null.
- [Question]: 无法安装PaddleNLP HOT 2
- 文档信息抽取uie-x-base推理速度和模型加载问题咨询 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from paddlenlp.