essaykillerbrain / essaykiller_v2 Goto Github PK
View Code? Open in Web Editor NEW基于开源GPT2.0的初代创作型人工智能 | 可扩展、可进化
License: MIT License
essaykiller_v2's Issues
暑假社会实践
已关闭
mac不能跑吗,部署没问题了，不出结果
20多分钟了，一直不出结果
(aitext) laogege@localhost GPT2 % python scripts/demo.py -ckpt_fn finetune/trained_models/model.ckpt-280000 -min_len 1024 -samples 1
WARNING:tensorflow:From /Users/laogege/pythonwork3/aitext/EssayKiller_V2-master/LanguageNetwork/GPT2/scripts/modeling.py:141: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.Dense instead.
WARNING:tensorflow:From /Users/laogege/.pyenv/versions/aitext/lib/python3.6/site-packages/tensorflow_core/python/layers/core.py:187: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use layer.__call__ method instead.
WARNING:tensorflow:From /Users/laogege/.pyenv/versions/aitext/lib/python3.6/site-packages/tensorflow_core/python/util/dispatch.py:180: batch_gather (from tensorflow.python.ops.array_ops) is deprecated and will be removed after 2017-10-25.
Instructions for updating:
tf.batch_gather is deprecated, please use tf.gather with batch_dims=-1 instead.
模型加载好啦！🍺Bilibili干杯🍺
现在将你的作文题精简为一个句子，粘贴到这里:⬇️，然后回车
作文题目
我爱你老师
作文题目
正在生成第 1 of 1 篇文章
......
EssayKiller正在飞速写作中，请稍后......
运行这个模型需要多大的显存呀
Colab中模型文件model.ckpt-280000.data-00000-of-00001下载失败
输出信息：
/home
fatal: destination path 'EssayKiller' already exists and is not an empty directory.
/home/EssayKiller/AutoWritter/finetune
trained_models/mode [ <=> ] 3.02K --.-KB/s in 0s
Couldn't download the file :-(
model.ckpt-280000.i 100%[===================>] 70.68K 432KB/s in 0.2s
model.ckpt-280000.m 100%[===================>] 12.49M 15.0MB/s in 0.8s
模型下载完成，Git项目已构建，请继续点击下方的▶️
经倒数第二行注释的提示取消最后一行的注释。
但是下载的文件也是错误的，模型无法运行。
自己捣鼓了下，正确的应该是这样的：!python3 /home/EssayKiller/AutoWritter/scripts/down_gdrive_file.py -file_id='1ujWYTOvRLGJX0raH-f-lPZa3-RN58ZQx' -file_path='models/mega/model.ckpt-280000.data-00000-of-00001'
运行模型也没有再出现错误。
那两个可视化的前端网页挂掉了。还是要科学上网才能访问，能不能挂大陆服务器上试一下？
那两个可视化的前端网页挂掉了。还是要科学上网才能访问，能不能挂大陆服务器上试一下？
colab下载模型失败
谷歌的colab卡住了用不了了，可以看看是出了什么问题么？我对这个项目还是挺感兴趣的。
谷歌的colab卡住了用不了了，可以看看是出了什么问题么？我对这个项目还是挺感兴趣的。
最后一步缺少文件怎么解决？
百度飞桨了解一下
百度飞桨了解一下，免费GPU
creative
i have an idea if you can collaborate with me of course
email me at [email protected]
cpu跑通了，但是时间很久，gpu还是没成功
用cpu跑通demo了，但是用gpu跑不了，求救！！！
1.机器配置
操作系统：Windows 10
硬件配置环境
显卡：3070ti 8g
处理器：i9 7980xe
cuda 10.0
cudnn 7.6.5
内存 64g
软件依赖
pandas==0.24.2
regex==2019.4.14
h5py==2.9.0
numpy==1.16.2
tensorboard==1.13.1
tensorflow-gpu==1.13.1
tqdm==4.31.1
requests==2.22.0
protobuf==3.19.02.报错、解决思路、替代方案
模型加载好啦！🍺Bilibili干杯🍺
现在将你的作文题精简为一个句子，粘贴到这里:⬇️，然后回车
**********************************************作文题目**********************************************
苦练本手,方能妙手随成
**********************************************作文题目**********************************************
正在生成第 1 of 1 篇文章
......
EssayKiller正在飞速写作中，请稍后......
2022-11-27 19:19:37.206277: E tensorflow/stream_executor/cuda/cuda_blas.cc:428] failed to run cuBLAS routine: CUBLAS_STATUS_EXECUTION_FAILED
2022-11-27 19:19:37.206746: E tensorflow/stream_executor/cuda/cuda_blas.cc:2301] Internal: failed BLAS call, see log for details
Traceback (most recent call last):
File "C:\Users\ly1995\AppData\Local\conda\conda\envs\zuowen1\lib\site-packages\tensorflow_core\python\client\session.py", line 1365, in _do_call
return fn(*args)
File "C:\Users\ly1995\AppData\Local\conda\conda\envs\zuowen1\lib\site-packages\tensorflow_core\python\client\session.py", line 1350, in _run_fn
target_list, run_metadata)
File "C:\Users\ly1995\AppData\Local\conda\conda\envs\zuowen1\lib\site-packages\tensorflow_core\python\client\session.py", line 1443, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InternalError: 2 root error(s) found.
(0) Internal: Blas xGEMMBatched launch failed : a.shape=[24,11,64], b.shape=[24,11,64], m=11, n=11, k=64, batch_size=24
[[{{node sample_sequence/newslm/layer00/MatMul}}]]
[[sample_sequence/while/Identity/_1594]]
(1) Internal: Blas xGEMMBatched launch failed : a.shape=[24,11,64], b.shape=[24,11,64], m=11, n=11, k=64, batch_size=24
[[{{node sample_sequence/newslm/layer00/MatMul}}]]
0 successful operations.
0 derived errors ignored.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "d:/ly/EssayKiller_V2-master/LanguageNetwork/GPT2/scripts/demo.py", line 220, in <module>
p_for_topp: top_p[chunk_i]})
File "C:\Users\ly1995\AppData\Local\conda\conda\envs\zuowen1\lib\site-packages\tensorflow_core\python\client\session.py", line 956, in run
run_metadata_ptr)
File "C:\Users\ly1995\AppData\Local\conda\conda\envs\zuowen1\lib\site-packages\tensorflow_core\python\client\session.py", line 1180, in _run
feed_dict_tensor, options, run_metadata)
File "C:\Users\ly1995\AppData\Local\conda\conda\envs\zuowen1\lib\site-packages\tensorflow_core\python\client\session.py", line 1359, in _do_run
run_metadata)
File "C:\Users\ly1995\AppData\Local\conda\conda\envs\zuowen1\lib\site-packages\tensorflow_core\python\client\session.py", line 1384, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InternalError: 2 root error(s) found.
(0) Internal: Blas xGEMMBatched launch failed : a.shape=[24,11,64], b.shape=[24,11,64], m=11, n=11, k=64, batch_size=24
[[node sample_sequence/newslm/layer00/MatMul (defined at C:\Users\ly1995\AppData\Local\conda\conda\envs\zuowen1\lib\site-packages\tensorflow_core\python\framework\ops.py:1748) ]]
[[sample_sequence/while/Identity/_1594]]
(1) Internal: Blas xGEMMBatched launch failed : a.shape=[24,11,64], b.shape=[24,11,64], m=11, n=11, k=64, batch_size=24
[[node sample_sequence/newslm/layer00/MatMul (defined at C:\Users\ly1995\AppData\Local\conda\conda\envs\zuowen1\lib\site-packages\tensorflow_core\python\framework\ops.py:1748) ]]
0 successful operations.
0 derived errors ignored.
Original stack trace for 'sample_sequence/newslm/layer00/MatMul':
File "d:/ly/EssayKiller_V2-master/LanguageNetwork/GPT2/scripts/demo.py", line 188, in <module>
do_topk=False)
File "d:\ly\EssayKiller_V2-master\LanguageNetwork\GPT2\scripts\modeling.py", line 768, in sample
do_topk=do_topk)
File "d:\ly\EssayKiller_V2-master\LanguageNetwork\GPT2\scripts\modeling.py", line 740, in initialize_from_context
batch_size=batch_size, p_for_topp=p_for_topp, cache=None, do_topk=do_topk)
File "d:\ly\EssayKiller_V2-master\LanguageNetwork\GPT2\scripts\modeling.py", line 714, in sample_step
cache=cache,
File "d:\ly\EssayKiller_V2-master\LanguageNetwork\GPT2\scripts\modeling.py", line 499, in __init__
cache=layer_cache,
File "d:\ly\EssayKiller_V2-master\LanguageNetwork\GPT2\scripts\modeling.py", line 198, in attention_layer
attention_scores = tf.matmul(query, key, transpose_b=True)
File "C:\Users\ly1995\AppData\Local\conda\conda\envs\zuowen1\lib\site-packages\tensorflow_core\python\util\dispatch.py", line 180, in wrapper
return target(*args, **kwargs)
File "C:\Users\ly1995\AppData\Local\conda\conda\envs\zuowen1\lib\site-packages\tensorflow_core\python\ops\math_ops.py", line 2716, in matmul
return batch_mat_mul_fn(a, b, adj_x=adjoint_a, adj_y=adjoint_b, name=name)
File "C:\Users\ly1995\AppData\Local\conda\conda\envs\zuowen1\lib\site-packages\tensorflow_core\python\ops\gen_math_ops.py", line 1712, in batch_mat_mul_v2
"BatchMatMulV2", x=x, y=y, adj_x=adj_x, adj_y=adj_y, name=name)
File "C:\Users\ly1995\AppData\Local\conda\conda\envs\zuowen1\lib\site-packages\tensorflow_core\python\framework\op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "C:\Users\ly1995\AppData\Local\conda\conda\envs\zuowen1\lib\site-packages\tensorflow_core\python\util\deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "C:\Users\ly1995\AppData\Local\conda\conda\envs\zuowen1\lib\site-packages\tensorflow_core\python\framework\ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "C:\Users\ly1995\AppData\Local\conda\conda\envs\zuowen1\lib\site-packages\tensorflow_core\python\framework\ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "C:\Users\ly1995\AppData\Local\conda\conda\envs\zuowen1\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1748, in __init__
self._traceback = tf_stack.extract_stack()
2.1 关键信息抽取
(0) Internal: Blas xGEMMBatched launch failed : a.shape=[24,11,64], b.shape=[24,11,64], m=11, n=11, k=64, batch_size=24
2.2 问题分析
通过bing搜索报错信息，得知了报错原因，主要是因为显存不够造成的
2.3 想法1
既然显存不够，那就减少一些显存，让程序灵活调用显存，这样问题就解决了吧，于是我加入了如下语句
os.environ["CUDA_VISIBLE_DEVICES"] = "0"
tf_config = tf.compat.v1.ConfigProto(allow_soft_placement=True)
tf_config.gpu_options.allow_growth=True
# tf_config.gpu_options.per_process_gpu_memory_fraction = 0.6
...可还是报错，是因为显存不够吗...
替代方案1
既然gpu跑不了，那干脆不用gpu了，用cpu试试，于是我修改了以下语句
os.environ["CUDA_VISIBLE_DEVICES"] = " " #将0改为none
结果：程序跑通了，但是cpu跑肯定比gpu慢很多，跑一篇作文大概要10min，cpu占用率大概为40-50
最后一步报错 FileNotFoundError: [Errno 2] No such file or directory: 'result.txt'
【解决方法】线上版本报错result.txt找不到
这是由于模型文件没有下载完成的缘故，5个多g的文件，
清空finetune/trained_models下面的文件多运行几次（第二 步操作）试试，
或者直接下载文件到trained_models目录下面，也可试试从挂载google云盘从云盘上面复制过去。
接入 ChatGPT / GPT3.5 / GPT4 制作 WriteGPT 2.0?
如题，现在 LLM 的效果应该比这个好，是否考虑直接使用 OepnAI 的 API 加 Prompt 的形式，更新一个更强大的 WriteGPT？
22年5月18，顺利通过
GPT-2文件无效
EAST | 有效
CRNN | 有效
BERT |有效
GPT-2 | 失效，提示错误页面
烦请更新一下共享链接，因为看了B站的视频，对机器学习产生了兴趣
在此保证严格遵守约定不作任何商业或非法用途，不用来牟利，仅作学习之用。
有没有大佬知道怎么改
mv: cannot stat '/home/EssayKiller_V2/LanguageNetwork/GPT2/finetune/models/mega/*': No such file or directory
/home/EssayKiller_V2/LanguageNetwork/GPT2
模型加载中，请稍后......
python3: can't open file 'sc': [Errno 2] No such file or directory
the lens: 13
Traceback (most recent call last):
File "scripts/formatter.py", line 132, in
with open(args.org_text, 'r',encoding='UTF-8') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'result.txt'
请问国外有类似的支持微调的模型吗？
请问国外有类似的支持微调的模型吗？
finetune数据集
您好，请问fine-tune数据集，散文、优质作文的这部分可以分享一下吗？
Hi
线上colab版本最后一步错误
看了下貌似都有个问题按照其他提供的办法还是这样报错是刻意删除了部分代码防止商用还是操作有问题啊
或许可以改进一下
在线版根据#9 已经成功运行
不过生成的文章里有个神奇的东西
“优秀范文字篇六”
我
这几行无数次出现，等了20多分钟还没加载好模型，是什么问题呀？
--2022-03-30 15:38:00-- https://docs.google.com/uc?id=1A910UqSNBBi_SEoIDl15095T_5kojESO&export=download
Resolving docs.google.com (docs.google.com)... 64.233.184.138, 64.233.184.101, 64.233.184.139, ...
Connecting to docs.google.com (docs.google.com)|64.233.184.138|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘trained_models/model.ckpt-280000.data-00000-of-00001’
0K 22.8M=0s
2022-03-30 15:38:01 (22.8 MB/s) - ‘trained_models/model.ckpt-280000.data-00000-of-00001’ saved [2226]
Colab 运行出错的解决方案
我对原版的 Jupyter Notebook 做了一点修改，把下载的 Checkpoint 文件重命名了一下。
链接：https://colab.research.google.com/drive/1osIWtqrHbUu2MLy2dJn1-y1b-V9s8meL?usp=sharing
当然，生成文章的效果仍有待提高。
运行结果 demo：
colab网页版卡在第二部分中perl指令处，持续下载
网页版挂着十几个小时直到连接自动断开也没有下载完毕，期间这几行重复出现，只是resolving和connecting两行的ip地址，和下载速度在改变；右上角ram占用极低
`
--2022-06-07 12:12:38-- https://docs.google.com/uc?id=1A910UqSNBBi_SEoIDl15095T_5kojESO&export=download
Resolving docs.google.com (docs.google.com)... 74.125.31.113, 74.125.31.102, 74.125.31.101, ...
Connecting to docs.google.com (docs.google.com)|74.125.31.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘trained_models/model.ckpt-280000.data-00000-of-00001’
0K 13.5M=0s
2022-06-07 12:12:38 (13.5 MB/s) - ‘trained_models/model.ckpt-280000.data-00000-of-00001’ saved [2241]
`
`
--2022-06-07 12:12:38-- https://docs.google.com/uc?id=1A910UqSNBBi_SEoIDl15095T_5kojESO&export=download
Resolving docs.google.com (docs.google.com)... 173.194.218.138, 173.194.218.139, 173.194.218.113, ...
Connecting to docs.google.com (docs.google.com)|173.194.218.138|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘trained_models/model.ckpt-280000.data-00000-of-00001’
0K 29.5M=0s
2022-06-07 12:12:38 (29.5 MB/s) - ‘trained_models/model.ckpt-280000.data-00000-of-00001’ saved [2241]
`
需要运行多久
运行需要多久才能出文章呢我运行了十几分钟都没反应····
模型框架加载模块时卡在gdown.pl
如题，程序会卡在下载 trained_models/model.ckpt-280000.data-00000-of-00001 ，然后重复输出：
--2022-06-02 12:30:58-- https://docs.google.com/uc?id=1A910UqSNBBi_SEoIDl15095T_5kojESO&export=download
Resolving docs.google.com (docs.google.com)... 173.194.218.101, 173.194.218.138, 173.194.218.102, ...
Connecting to docs.google.com (docs.google.com)|173.194.218.101|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘trained_models/model.ckpt-280000.data-00000-of-00001’
0K 40.1M=0s
2022-06-02 12:30:58 (40.1 MB/s) - ‘trained_models/model.ckpt-280000.data-00000-of-00001’ saved [2241]
我被卡在这里已经半个多小时了，还是一直这样。如果强制停止，再继续执行第三步就会Error。
请问应该如何解决这个问题？
formatter.py 出现了gbk解码方式的报错
留着“coding=gbk”就报错无法运行
删掉能运行但没有输出结果
百年绿色长征路，峥嵘岁月报国情
最后一步报错
/home/EssayKiller/AutoWritter 模型加载中，你可以去看会儿B站...... WARNING:tensorflow:From /home/EssayKiller/AutoWritter/scripts/modeling.py:141: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version. Instructions for updating: Use keras.layers.Dense instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/layers/core.py:187: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: Please use layer.callmethod instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/util/dispatch.py:180: batch_gather (from tensorflow.python.ops.array_ops) is deprecated and will be removed after 2017-10-25. Instructions for updating:tf.batch_gatheris deprecated, please usetf.gatherwithbatch_dims=-1` instead.
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call
return fn(*args)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1350, in _run_fn
target_list, run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.OutOfRangeError: 2 root error(s) found.
(0) Out of range: Read less bytes than requested
[[{{node save/RestoreV2}}]]
[[save/RestoreV2/_301]]
(1) Out of range: Read less bytes than requested
[[{{node save/RestoreV2}}]]
0 successful operations.
0 derived errors ignored.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "scripts/demo.py", line 179, in
saver.restore(sess, args.ckpt_fn)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/saver.py", line 1290, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 956, in run
run_metadata_ptr)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1180, in _run
feed_dict_tensor, options, run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1359, in _do_run
run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1384, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: 2 root error(s) found.
(0) Out of range: Read less bytes than requested
[[node save/RestoreV2 (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
[[save/RestoreV2/_301]]
(1) Out of range: Read less bytes than requested
[[node save/RestoreV2 (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
0 successful operations.
0 derived errors ignored.
Original stack trace for 'save/RestoreV2':
File "scripts/demo.py", line 178, in
saver = tf.compat.v1.train.Saver()
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/saver.py", line 828, in init
self.build()
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/saver.py", line 840, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/saver.py", line 878, in _build
build_restore=build_restore)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/saver.py", line 508, in _build_internal
restore_sequentially, reshape)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/saver.py", line 328, in _AddRestoreOps
restore_sequentially)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/saver.py", line 575, in bulk_restore
return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/gen_io_ops.py", line 1696, in restore_v2
name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py", line 1748, in init
self._traceback = tf_stack.extract_stack()
the lens: 13
scoring api has failed...
0`
已经搞定了!大佬们能不能来个小白教程
大佬们能不能来个小白教程啊，按着clab的代码，下载谷歌的3个文件到本地，部署完各种报错
FileNotFoundError: [Errno 2] No such file or directory: 'result.txt'
Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/install_sources#common_installation_problems
for some common reasons and solutions. Include the entire stack trace
above this error message when asking for help.
Colab 模型参数设置 运行报错，所有的参数都没有改过
mv: cannot stat '/home/EssayKiller_V2/LanguageNetwork/GPT2/finetune/models/mega/*': No such file or directory
/home/EssayKiller_V2/LanguageNetwork/GPT2
模型加载中，请稍后......
WARNING:tensorflow:From /home/EssayKiller_V2/LanguageNetwork/GPT2/scripts/modeling.py:141: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.Dense instead.
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow_core/python/layers/core.py:187: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use layer.__call__ method instead.
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow_core/python/util/dispatch.py:180: batch_gather (from tensorflow.python.ops.array_ops) is deprecated and will be removed after 2017-10-25.
Instructions for updating:
tf.batch_gather is deprecated, please use tf.gather with batch_dims=-1 instead.
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call
return fn(*args)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/client/session.py", line 1350, in _run_fn
target_list, run_metadata)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.OutOfRangeError: 2 root error(s) found.
(0) Out of range: Read less bytes than requested
[[{{node save/RestoreV2}}]]
[[save/RestoreV2/_301]]
(1) Out of range: Read less bytes than requested
[[{{node save/RestoreV2}}]]
0 successful operations.
0 derived errors ignored.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "scripts/demo.py", line 179, in
saver.restore(sess, args.ckpt_fn)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 1290, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/client/session.py", line 956, in run
run_metadata_ptr)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/client/session.py", line 1180, in _run
feed_dict_tensor, options, run_metadata)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/client/session.py", line 1359, in _do_run
run_metadata)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/client/session.py", line 1384, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: 2 root error(s) found.
(0) Out of range: Read less bytes than requested
[[node save/RestoreV2 (defined at /usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
[[save/RestoreV2/_301]]
(1) Out of range: Read less bytes than requested
[[node save/RestoreV2 (defined at /usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
0 successful operations.
0 derived errors ignored.
Original stack trace for 'save/RestoreV2':
File "scripts/demo.py", line 178, in
saver = tf.compat.v1.train.Saver()
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 828, in init
self.build()
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 840, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 878, in _build
build_restore=build_restore)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 508, in _build_internal
restore_sequentially, reshape)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 328, in _AddRestoreOps
restore_sequentially)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 575, in bulk_restore
return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/ops/gen_io_ops.py", line 1696, in restore_v2
name=name)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/ops.py", line 1748, in init
self._traceback = tf_stack.extract_stack()
the lens: 13
Traceback (most recent call last):
File "scripts/formatter.py", line 132, in
with open(args.org_text, 'r',encoding='UTF-8') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'result.txt'
数据集
您好，请问使用的鲁迅数据集可以分享一下吗？
缺少文件
tensorflow.python.framework.errors_impl.NotFoundError: 2 root error(s) found.
(0) Not found: finetune/trained_models/model.ckpt-344000.data-00000-of-00001; No such file or directory
[[{{node save/RestoreV2}}]]
(1) Not found: finetune/trained_models/model.ckpt-344000.data-00000-of-00001; No such file or directory
[[{{node save/RestoreV2}}]]
[[save/RestoreV2/_301]]
这个项目一定运行一定要有 GPU？只有 CPU 能不能跑？
8 CPU/ 16G 服务器能不能跑的动？
你好
模型参数设置出错了
mv: cannot stat '/home/EssayKiller_V2/LanguageNetwork/GPT2/finetune/models/mega/*': No such file or directory
最后显示的是FileNotFoundError: [Errno 2] No such file or directory: 'result.txt'
网页版无法正常运行，😭
报错如下：
#!cat /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/tpu/tpu_feed.py
#@title #文章生成模块
作文最小篇幅 = 850 #@param {type:"number", min:800, max:1024, step:1}
生成作文篇数 = 1 #@param {type:"number", min:1, max:100, step:1}
%mv /home/EssayKiller_V2/LanguageNetwork/GPT2/finetune/models/mega/* /home/EssayKiller_V2/LanguageNetwork/GPT2/finetune/trained_models/
%cd /home/EssayKiller_V2/LanguageNetwork/GPT2/
!export TF_CPP_MIN_LOG_LEVEL=2
!echo '模型加载中，请稍后......'
!PYTHONPATH=$(pwd) python scripts/demo.py -ckpt_fn finetune/trained_models/model.ckpt-280000 -min_len
!PYTHONPATH=$(pwd) python scripts/formatter.py -org_text result.txt
模型LOADING代码已经修复好了，大家可以继续在线上生成
BUG找了很久，蛋疼
分享的模型链接失效了。请问能更新一下吗？
最近刚看到这个项目，想本地部署学习。发现模型都失效了，请问能更新下分享地址吗
EAST | 失效
CRNN | 有效
BERT |失效
GPT-2 | 失效
线上网站最后一步出错
mv: cannot stat '/home/EssayKiller_V2/LanguageNetwork/GPT2/finetune/models/mega/*': No such file or directory
/home/EssayKiller_V2/LanguageNetwork/GPT2
模型加载中，请稍后......
WARNING:tensorflow:From /home/EssayKiller_V2/LanguageNetwork/GPT2/scripts/modeling.py:141: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.Dense instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/layers/core.py:187: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use layer.__call__ method instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/util/dispatch.py:180: batch_gather (from tensorflow.python.ops.array_ops) is deprecated and will be removed after 2017-10-25.
Instructions for updating:
tf.batch_gather is deprecated, please use tf.gather with batch_dims=-1 instead.
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call
return fn(*args)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1350, in _run_fn
target_list, run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.OutOfRangeError: 2 root error(s) found.
(0) Out of range: Read less bytes than requested
[[{{node save/RestoreV2}}]]
[[save/RestoreV2/_301]]
(1) Out of range: Read less bytes than requested
[[{{node save/RestoreV2}}]]
0 successful operations.
0 derived errors ignored.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "scripts/demo.py", line 179, in
saver.restore(sess, args.ckpt_fn)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/saver.py", line 1290, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 956, in run
run_metadata_ptr)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1180, in _run
feed_dict_tensor, options, run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1359, in _do_run
run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1384, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: 2 root error(s) found.
(0) Out of range: Read less bytes than requested
[[node save/RestoreV2 (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
[[save/RestoreV2/_301]]
(1) Out of range: Read less bytes than requested
[[node save/RestoreV2 (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
0 successful operations.
0 derived errors ignored.
Original stack trace for 'save/RestoreV2':
File "scripts/demo.py", line 178, in
saver = tf.compat.v1.train.Saver()
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/saver.py", line 828, in init
self.build()
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/saver.py", line 840, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/saver.py", line 878, in _build
build_restore=build_restore)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/saver.py", line 508, in _build_internal
restore_sequentially, reshape)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/saver.py", line 328, in _AddRestoreOps
restore_sequentially)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/saver.py", line 575, in bulk_restore
return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/gen_io_ops.py", line 1696, in restore_v2
name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py", line 1748, in init
self._traceback = tf_stack.extract_stack()
the lens: 13
Traceback (most recent call last):
File "scripts/formatter.py", line 132, in
with open(args.org_text, 'r',encoding='UTF-8') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'result.txt'
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.