wind91725 / gpt2-ml-finetune- Goto Github PK
View Code? Open in Web Editor NEW根据gpt2-ml中文模型finetune自己的数据集
License: Apache License 2.0
根据gpt2-ml中文模型finetune自己的数据集
License: Apache License 2.0
提示prepare_data.py也执行完成了,但是在data文件夹中只有22.json文件,没有train.tfrecord文件,并且没有错误提示。
CALLING CLOSE
DONE UPLOADING
想问下loss下降到多少是差不多收敛了,参考下你的训练结果
我看wind91725老师写的是:python train/train_wc.py --input_file=/data/home/share1/gpt2-ml-Finetune/data/train.tfrecord --output_dir=/data/home/share1/gpt2-ml-Finetune/finetune_model --init_checkpoint=/data/home/share1/gpt2-ml/models/mega/ceshi/model.ckpt-100000
实际的--init_checkpoint参数,是不是应该改为gpt2-ml的现有模型目录:models/mega/model.ckpt-220000,感谢!
定位到model.ckpt-100000,一直说找不到匹配的文件,您有出现这种情况吗?
我尝试了下使用32G-v100训练, 剩余显存可能没达到32G,可能是28G左右, 卡在这一句2020-07-11 15:00:55.541191: I tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library libcublas.so.10.0 locally
就没有输出了, 这是因为显卡剩余显存不够吗?还是说需要更好的显卡?
为什么我finetune之后的模型生成的文本更加乱了, 使用原版的模型倒是没有问题
请问输入文件这个目录在哪里呢?/data/home/share1/gpt2-ml-Finetune/data-mayun_xiugai
您好,这个程序把数据变成什么形式的了?
gpt2-ml-Finetune/finetune_model/model.ckpt-220000 is not in all_model_checkpoint_paths. Manually adding it.
不在检查路径中。。
您好,这个参数是干嘛的
小白一枚,finetune的时候,自己的数据集需要多大会比较好?1M,10M还是说需要更大的?
AttributeError: module 'tensorflow.compat.v1' has no attribute 'contrib'试过tf1.15.2和1.13.1第三步都报这个。求教
tensorflow2.x一直报错,因为 'contrib'在2.x中已经删除,降级成1.x(1.14、1.15)能运行,
开始训练后会出现一堆warring:
【Start trainning.............................................
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.init (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
W0616 06:45:01.207822 140262356580224 deprecation.py:506] From /usr/local/lib/python3.7/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.init (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
W0616 06:45:01.208348 140262356580224 deprecation.py:323] From /usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
WARNING:tensorflow:From /content/drive/MyDrive/gpt2-ml/train/dataloader.py:63: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Usetf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.experimental.AUTOTUNE)
instead. If sloppy execution is desired, usetf.data.Options.experimental_determinstic
.
W0616 06:45:01.223439 140262356580224 deprecation.py:323] From /content/drive/MyDrive/gpt2-ml/train/dataloader.py:63: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Usetf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.experimental.AUTOTUNE)
instead. If sloppy execution is desired, usetf.data.Options.experimental_determinstic
.
WARNING:tensorflow:From /content/drive/MyDrive/gpt2-ml/train/dataloader.py:81: map_and_batch (from tensorflow.python.data.experimental.ops.batching) is deprecated and will be removed in a future version.
】
开始循环训练之后会出现致命错误:
【ERROR:tensorflow:Error recorded from training_loop: module 'tensorflow._api.v1.compat.v1' has no attribute 'contrib'
E0616 06:45:48.644567 140262356580224 error_handling.py:75] Error recorded from training_loop: module 'tensorflow._api.v1.compat.v1' has no attribute 'contrib'
INFO:tensorflow:training_loop marked as finished】
Google了一圈没有找到解决办法,我猜最大的问题出现在 'tensorflow._api.v1.compat.v1' has no attribute 'contrib'上,估计修改API就好?但是找不到这段代码在哪里。
小白一个,还请大佬指点迷津。
文件格式是什么?需要分词吗
觉得奇怪的地方, train和 inference的时候使用的vocabulary 不是同一个文件的?
执行生成tfrecord训练数据这一步的时候
python prepare_data.py -input_fn /data/home/share1/gpt2-ml-Finetune/data
出错了
Traceback (most recent call last):
File "./prepare_data.py", line 17, in
from tokenization import tokenization
ModuleNotFoundError: No module named 'tokenization'
请问如何解决?谢谢
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.