nicolas-ivanov / debug_seq2seq Goto Github PK
View Code? Open in Web Editor NEW[unmaintained] Make seq2seq for keras work
[unmaintained] Make seq2seq for keras work
I get the following error when running bin/train.py
1482-sraval:debug_seq2seq sraval$ python bin/train.py
Traceback (most recent call last):
File "bin/train.py", line 9, in <module>
from lib.w2v_model import w2v
File "/Users/sraval/Desktop/yooo/debug_seq2seq/lib/w2v_model/w2v.py", line 4, in <module>
from gensim.models import Word2Vec
File "/Library/Python/2.7/site-packages/gensim/__init__.py", line 6, in <module>
from gensim import parsing, matutils, interfaces, corpora, models, similarities, summarization
File "/Library/Python/2.7/site-packages/gensim/models/__init__.py", line 13, in <module>
from .word2vec import Word2Vec
File "/Library/Python/2.7/site-packages/gensim/models/word2vec.py", line 100, in <module>
from gensim.models.word2vec_inner import train_sentence_sg, train_sentence_cbow, FAST_VERSION,\
File "__init__.pxd", line 155, in init gensim.models.word2vec_inner (./gensim/models/word2vec_inner.c:9234)
ValueError: numpy.dtype has the wrong size, try recompiling
File "/home/devkite/anaconda2/lib/python2.7/site-packages/theano/configdefaults.py", line 1252, in check_mkl_openmp
raise RuntimeError('To use MKL 2018 with Theano you MUST set "MKL_THREADING_LAYER=GNU" in your environement.')
RuntimeError: To use MKL 2018 with Theano you MUST set "MKL_THREADING_LAYER=GNU" in your environement.
How to set "MKL_THREADING_LAYER=GNU"
when I try to run train.py, the error occurs:
ljy@ubuntu:~/debug_seq2seq$ python bin/train.py
Traceback (most recent call last):
File "bin/train.py", line 9, in
from lib.w2v_model import w2v
File "/home/ljy/debug_seq2seq/lib/w2v_model/w2v.py", line 4, in
from gensim.models import Word2Vec
ImportError: No module named gensim.models
But I has just install gensim by using
sudo easy_install -U gensim
Anyone could help me ?
hi there
I am trying to start learning this code, however after i download and install everything I try to run your example code but I am getting this error, could you please point me out my mistake?
thanks a lot
$ sudo python bin/train.py
INFO:gensim.utils:'pattern' package found; utils.lemmatize() is available for English
INFO:summa.preprocessing.cleaner:'pattern' package found; tag filters are available for English
Using Theano backend.
/usr/local/lib/python2.7/dist-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module.
"downsample module has been moved to the theano.tensor.signal.pool module.")
Traceback (most recent call last):
File "bin/train.py", line 10, in
from lib.nn_model.model import get_nn_model
File "/home/creangel/Downloads/keras/seq2seq/debug_seq2seq/lib/nn_model/model.py", line 4, in
from seq2seq.models import SimpleSeq2seq
File "build/bdist.linux-x86_64/egg/seq2seq/init.py", line 1, in
File "build/bdist.linux-x86_64/egg/seq2seq/cells.py", line 1, in
File "build/bdist.linux-x86_64/egg/recurrentshop/init.py", line 1, in
File "build/bdist.linux-x86_64/egg/recurrentshop/engine.py", line 1, in
ImportError: cannot import name Layer
Hi, @nicolas-ivanov
I run the training code.Maybe it contains some errors in it.I get the error like below:
`ValueError: Shape mismatch: x has 64 rows but z has 24 rows
Apply node that caused the error: Gemm{no_inplace}(Subtensor{::, int64::}.0, TensorConstant{0.20000000298}, <TensorType(float32, matrix)>, lstm_U_o_copy, TensorConstant{0.20000000298})
Toposort index: 5
Inputs types: [TensorType(float32, matrix), TensorType(float32, scalar), TensorType(float32, matrix), TensorType(float32, matrix), TensorType(float32, scalar)]
Inputs shapes: [(24, 128), (), (64, 128), (128, 128), ()]
Inputs strides: [(32768, 4), (), (512, 4), (512, 4), ()]
Inputs values: ['not shown', array(0.20000000298023224, dtype=float32), 'not shown', 'not shown', array(0.20000000298023224, dtype=float32)]
Outputs clients: [[Elemwise{Composite{(clip((i0 + i1), i2, i3) * tanh(i4))}}(TensorConstant{(1, 1) of 0.5}, Gemm{no_inplace}.0, TensorConstant{(1, 1) of 0}, TensorConstant{(1, 1) of 1}, Elemwise{Composite{((clip((i0 + i1), i2, i3) * i4) + (clip((i5 + i6), i2, i3) * tanh(i7)))}}.0)]]
HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
Apply node that caused the error: forall_inplace,cpu,scan_fn}(TensorConstant{16}, InplaceDimShuffle{1,0,2}.0, IncSubtensor{InplaceSet;:int64:}.0, DeepCopyOp.0, TensorConstant{16}, lstm_U_o, lstm_U_f, lstm_U_i, lstm_U_c)
Toposort index: 36
Inputs types: [TensorType(int64, scalar), TensorType(float32, 3D), TensorType(float32, (True, False, False)), TensorType(float32, (True, False, False)), TensorType(int64, scalar), TensorType(float32, matrix), TensorType(float32, matrix), TensorType(float32, matrix), TensorType(float32, matrix)]
Inputs shapes: [(), (16, 24, 512), (1, 64, 128), (1, 64, 128), (), (128, 128), (128, 128), (128, 128), (128, 128)]
Inputs strides: [(), (2048, 32768, 4), (32768, 512, 4), (32768, 512, 4), (), (512, 4), (512, 4), (512, 4), (512, 4)]
Inputs values: [array(16), 'not shown', 'not shown', 'not shown', array(16), 'not shown', 'not shown', 'not shown', 'not shown']
Outputs clients: [[], [], [InplaceDimShuffle{0,1,2}(forall_inplace,cpu,scan_fn}.2)]]
HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.`
How can I solve it ?
Hi nicolas,
first really thanks for your work. when I run your code, I cannot get meaningful results, all I got is like
NFO:lib.nn_model.train:[why ?] -> [i ' . . $$$ . $$$ $$$ $$$ $$$ as as as as i i]
INFO:lib.nn_model.train:[who ?] -> [i ' . . $$$ . $$$ $$$ $$$ $$$ as as as as i i]
INFO:lib.nn_model.train:[yeah ?] -> [i ' . . $$$ . $$$ $$$ $$$ $$$ as as as as i i]
INFO:lib.nn_model.train:[what is it ?] -> [i ' . . $$$ . $$$ $$$ $$$ $$$ as as as as as i]
INFO:lib.nn_model.train:[why not ?] -> [i ' . . $$$ . $$$ $$$ $$$ $$$ as as as as i i]
INFO:lib.nn_model.train:[really ?] -> [i ' . . $$$ . $$$ $$$ $$$ $$$ as as as as i i]
INFO:lib.nn_model.train:[huh ?] -> [i ' . . $$$ . $$$ $$$ $$$ $$$ as as as as i i]
INFO:lib.nn_model.train:[yes ?] -> [i ' . . $$$ . $$$ $$$ $$$ $$$ as as as as i i]
INFO:lib.nn_model.train:[what ' s that ?] -> [i ' . . $$$ . $$$ $$$ $$$ $$$ as as as as as i]
INFO:lib.nn_model.train:[what are you doing ?] -> [i ' . . $$$ . $$$ $$$ $$$ $$$ as as as as as i]
INFO:lib.nn_model.train:[what are you talking about ?] -> [i ' . . $$$ . $$$ $$$ $$$ $$$ as as as as as i]
INFO:lib.nn_model.train:[what happened ?] -> [i ' . . $$$ . $$$ $$$ $$$ $$$ as as as as as i]
INFO:lib.nn_model.train:[hello ?] -> [i ' . . $$$ . $$$ $$$ $$$ $$$ as as as as i i]
INFO:lib.nn_model.train:[where ?] -> [i ' . . $$$ . $$$ $$$ $$$ $$$ as as as as i i]
INFO:lib.nn_model.train:[how ?] -> [i ' . . $$$ . $$$ $$$ $$$ $$$ as as as as i i]
INFO:lib.nn_model.train:[excuse me ?] -> [i ' . . $$$ . $$$ $$$ $$$ $$$ as as as as i i]
INFO:lib.nn_model.train:[who are you ?] -> [i ' . . $$$ . $$$ $$$ $$$ $$$ as as as as as i]
INFO:lib.nn_model.train:[what do you want ?] -> [i ' . . $$$ . $$$ $$$ $$$ $$$ as as as as as i]
INFO:lib.nn_model.train:[what ' s wrong ?] -> [i ' . . $$$ .
or
NFO:lib.nn_model.train:[what are you talking about ?] -> [i ' . . . . . . . . , , , , , ,]
INFO:lib.nn_model.train:[what happened ?] -> [i ' . . . . . . . . , , , , , ,]
INFO:lib.nn_model.train:[hello ?] -> [i ' . . . . . . . . , , , , , ,]
INFO:lib.nn_model.train:[where ?] -> [i ' . . . . . . . . , , , , , ,]
INFO:lib.nn_model.train:[how ?] -> [i ' . . . . . . . . , , , , , ,]
INFO:lib.nn_model.train:[excuse me ?] -> [i ' . . . . . . . . , , , , , ,]
INFO:lib.nn_model.train:[who are you ?] -> [i ' . . . . . . . . , , , , , ,]
could you sharing your opinion with me? really appreciate
I am getting the following error while prediction. The model predicts fine when the input and output sequence length is equal.
Input sequence length = 16
Output sequence length = 6
Could you help?
Traceback (most recent call last):
File "/home/jaimita/debug_seq2seq/bin/train.py", line 39, in <module>
learn()
File "/home/jaimita/debug_seq2seq/bin/train.py", line 36, in learn
train_model(nn_model, w2v_model, dialog_lines_for_nn, index_to_token)
File "/home/jaimita/debug_seq2seq/lib/nn_model/train.py", line 96, in train_model
log_predictions(test_sentences, nn_model, w2v_model, index_to_token)
File "/home/jaimita/debug_seq2seq/lib/nn_model/train.py", line 21, in log_predictions
prediction = predict_sentence(sent, nn_model, w2v_model, index_to_token)
File "/home/jaimita/debug_seq2seq/lib/nn_model/predict.py", line 47, in predict_sentence
tokens_sequence = _predict_sequence(input_sequence, nn_model, w2v_model, index_to_token, diversity)
File "/home/jaimita/debug_seq2seq/lib/nn_model/predict.py", line 34, in _predict_sequence
predictions = nn_model.predict(X, verbose=0)[0]
File "build/bdist.linux-x86_64/egg/keras/models.py", line 661, in predict
File "build/bdist.linux-x86_64/egg/keras/models.py", line 322, in _predict_loop
File "build/bdist.linux-x86_64/egg/keras/backend/theano_backend.py", line 384, in __call__
File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 871, in __call__
storage_map=getattr(self.fn, 'storage_map', None))
File "/usr/local/lib/python2.7/dist-packages/theano/gof/link.py", line 314, in raise_with_op
reraise(exc_type, exc_value, exc_trace)
File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 859, in __call__
outputs = self.fn()
File "/usr/local/lib/python2.7/dist-packages/theano/scan_module/scan_op.py", line 963, in rval
r = p(n, [x[0] for x in i], o)
File "/usr/local/lib/python2.7/dist-packages/theano/scan_module/scan_op.py", line 952, in <lambda>
self, node)
File "theano/scan_module/scan_perform.pyx", line 405, in theano.scan_module.scan_perform.perform (/home/jaimita/.theano/compiledir_Linux-3.19--generic-x86_64-with-Ubuntu-14.04-trusty-x86_64-2.7.6-64/scan_perform/mod.cpp:4316)
File "/usr/local/lib/python2.7/dist-packages/theano/gof/link.py", line 314, in raise_with_op
reraise(exc_type, exc_value, exc_trace)
File "theano/scan_module/scan_perform.pyx", line 397, in theano.scan_module.scan_perform.perform (/home/jaimita/.theano/compiledir_Linux-3.19--generic-x86_64-with-Ubuntu-14.04-trusty-x86_64-2.7.6-64/scan_perform/mod.cpp:4193)
ValueError: Input dimension mis-match. (input[0].shape[0] = 6, input[1].shape[0] = 16)
Apply node that caused the error: Elemwise{Add}[(0, 1)](InplaceDimShuffle{1,0,2}.0, InplaceDimShuffle{1,0,2}.0)
Toposort index: 31
Inputs types: [TensorType(float32, 3D), TensorType(float32, 3D)]
Inputs shapes: [(6, 32, 128), (16, 32, 128)]
Inputs strides: [(16384, 512, 4), (512, 8192, 4)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[Subtensor{int64:int64:int8}(Elemwise{Add}[(0, 1)].0, ScalarFromTensor.0, ScalarFromTensor.0, Constant{1})]]
HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
Apply node that caused the error: forall_inplace,cpu,scan_fn}(TensorConstant{6}, IncSubtensor{InplaceSet;:int64:}.0, IncSubtensor{Set;:int64:}.0, IncSubtensor{InplaceSet;:int64:}.0, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, <TensorType(float32, vector)>, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, InplaceDimShuffle{1,0,2}.0, InplaceDimShuffle{x,0}.0, InplaceDimShuffle{x,0}.0, InplaceDimShuffle{x,0}.0, InplaceDimShuffle{x,0}.0, InplaceDimShuffle{x,0}.0, InplaceDimShuffle{x,0}.0)
Toposort index: 384
Inputs types: [TensorType(int8, scalar), TensorType(float32, 3D), TensorType(float32, (True, False, False)), TensorType(float32, (True, False, False)), TensorType(float32, matrix), TensorType(float32, matrix), TensorType(float32, vector), TensorType(float32, matrix), TensorType(float32, matrix), TensorType(float32, matrix), TensorType(float32, matrix), TensorType(float32, matrix), TensorType(float32, matrix), TensorType(float32, matrix), TensorType(float32, matrix), TensorType(float32, matrix), TensorType(float32, matrix), TensorType(float32, matrix), TensorType(float32, matrix), TensorType(float32, 3D), TensorType(float32, row), TensorType(float32, row), TensorType(float32, row), TensorType(float32, row), TensorType(float32, row), TensorType(float32, row)]
Inputs shapes: [(), (6, 32, 128), (1, 32, 128), (1, 32, 128), (128, 128), (128, 1), (1,), (128, 128), (128, 128), (128, 128), (128, 128), (128, 128), (128, 128), (128, 128), (128, 128), (128, 128), (128, 128), (128, 128), (128, 128), (32, 6, 128), (1, 128), (1, 1), (1, 128), (1, 128), (1, 128), (1, 128)]
Inputs strides: [(), (16384, 512, 4), (16384, 512, 4), (16384, 512, 4), (512, 4), (4, 4), (4,), (512, 4), (512, 4), (512, 4), (512, 4), (512, 4), (512, 4), (512, 4), (512, 4), (512, 4), (512, 4), (512, 4), (512, 4), (512, 16384, 4), (512, 4), (4, 4), (512, 4), (512, 4), (512, 4), (512, 4)]
Inputs values: [array(6, dtype=int8), 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', array([ -2.43138842e-14], dtype=float32), 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', array([[ -2.43138842e-14]], dtype=float32), 'not shown', 'not shown', 'not shown', 'not shown']
Outputs clients: [[forall_inplace,cpu,scan_fn}(TensorConstant{6}, forall_inplace,cpu,scan_fn}.0, Alloc.0, IncSubtensor{InplaceSet;:int64:}.0, IncSubtensor{Set;:int64:}.0, IncSubtensor{InplaceSet;:int64:}.0, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, <TensorType(float32, matrix)>, InplaceDimShuffle{x,0}.0, InplaceDimShuffle{x,0}.0, InplaceDimShuffle{x,0}.0, InplaceDimShuffle{x,0}.0)], [], []]
HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
Process finished with exit code 1
hi there
I am trying to start learning this code, however after i download and install everything I try to run your example code but I am getting this error, could you please point me out my mistake?
INFO:lib.nn_model.model:Initializing NN model with the following params: INFO:lib.nn_model.model:Input dimension: 256 (token vector size) INFO:lib.nn_model.model:Hidden dimension: 512 INFO:lib.nn_model.model:Output dimension: 20001 (token dict size) INFO:lib.nn_model.model:Input seq length: 16 INFO:lib.nn_model.model:Output seq length: 6 INFO:lib.nn_model.model:Batch size: 32 Traceback (most recent call last): File "bin/train.py", line 36, in <module> learn() File "bin/train.py", line 29, in learn nn_model = get_nn_model(token_dict_size=len(index_to_token)) File "/Users/xiao/WorkSpace/Dev/3rd/debug_seq2seq/lib/nn_model/model.py", line 29, in get_nn_model depth=1 File "/Users/xiao/anaconda3/envs/dl/lib/python3.6/site-packages/seq2seq/models.py", line 81, in SimpleSeq2Seq output = decoder(x) File "/Users/xiao/anaconda3/envs/dl/lib/python3.6/site-packages/recurrentshop-1.0.0-py3.6.egg/recurrentshop/engine.py", line 452, in __call__ File "/Users/xiao/anaconda3/envs/dl/lib/python3.6/site-packages/recurrentshop-1.0.0-py3.6.egg/recurrentshop/engine.py", line 917, in num_states File "/Users/xiao/anaconda3/envs/dl/lib/python3.6/site-packages/recurrentshop-1.0.0-py3.6.egg/recurrentshop/engine.py", line 128, in num_states File "/Users/xiao/anaconda3/envs/dl/lib/python3.6/site-packages/recurrentshop-1.0.0-py3.6.egg/recurrentshop/cells.py", line 171, in build_model File "/Users/xiao/anaconda3/envs/dl/lib/python3.6/site-packages/keras/engine/base_layer.py", line 431, in __call__ self.build(unpack_singleton(input_shapes)) File "/Users/xiao/anaconda3/envs/dl/lib/python3.6/site-packages/keras/layers/core.py", line 861, in build constraint=self.kernel_constraint) File "/Users/xiao/anaconda3/envs/dl/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper return func(*args, **kwargs) File "/Users/xiao/anaconda3/envs/dl/lib/python3.6/site-packages/keras/engine/base_layer.py", line 252, in add_weight constraint=constraint) File "/Users/xiao/anaconda3/envs/dl/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 400, in variable v = tf.Variable(value, dtype=tf.as_dtype(dtype), name=name) File "/Users/xiao/anaconda3/envs/dl/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 235, in __init__ constraint=constraint) File "/Users/xiao/anaconda3/envs/dl/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 355, in _init_from_args initial_value, name="initial_value", dtype=dtype) File "/Users/xiao/anaconda3/envs/dl/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1014, in convert_to_tensor as_ref=False) File "/Users/xiao/anaconda3/envs/dl/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1104, in internal_convert_to_tensor ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref) File "/Users/xiao/anaconda3/envs/dl/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 235, in _constant_tensor_conversion_function return constant(v, dtype=dtype, name=name) File "/Users/xiao/anaconda3/envs/dl/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 214, in constant value, dtype=dtype, shape=shape, verify_shape=verify_shape)) File "/Users/xiao/anaconda3/envs/dl/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 496, in make_tensor_proto "Cannot create a tensor proto whose content is larger than 2GB.") ValueError: Cannot create a tensor proto whose content is larger than 2GB.
File: lib/nn_model/train.py
Line: 68
def save_model(nn_model):
#model_full_path = os.path.join(DATA_PATH, 'nn_models', NN_MODEL_PATH)
nn_model.save_weights(NN_MODEL_PATH, overwrite=True)
I noticed the function 'save_model(nn_model)' in lib/nn_model/train.py doesn't work so far. It seems there is an example to save/load model using HDF5 and json:
json_string = model.to_json()
open('my_model_architecture.json', 'w').write(json_string)
model.save_weights('my_model_weights.h5')
(elsewhere...)
model = model_from_json(open('my_model_architecture.json').read())
model.load_weights('my_model_weights.h5')
Does the method work?
I am very happy to find this repository that shows how to train a seq2seq model.
But this repository maybe a little old, many of its dependency packages should be updated:
So could the author can spend a little time to update this repository? I will appreciate if the author could update the repository.
Reading git+git://github.com/datalogai/recurrentshop.git
Download error on git+git://github.com/datalogai/recurrentshop.git: unknown url type: git+git -- Some packages may not be found!
Reading https://pypi.python.org/simple/recurrentshop/
Couldn't find index page for 'recurrentshop' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Hi guys.
I'm running Nicolas code and I have some concerns about the end-of-sentence symbol, i.e. "$$$"
What I understand is Nicolas put "$$$" at the end of each sentence. So my questions are:
Thank you in advance
debug_seq2seq/lib/nn_model/model.py
Line 35 in eb2147f
Here an uninitialized model is stored and then loaded, potentially overwriting already existing models.
I am getting the error
Could not install packages due to an EnvironmentError: [Errno 21] Is a directory: '/home/ram/.local/lib/python3.7/site-packages/pip-19.0.1.dist-info/METADATA'
When I'm running your code with python bin\train.py
machine restarts.
After debugging I found out that it occurs in the SimpleSeq2Seq model creation.
Machine config:
I used tensorflow 1.2 and theano 0.9.0 for keras backend and have got the same problem.
Other models start successfully (e.g. cifar10, mnist).
Hi,
I am getting this error when I call python train.py,
Traceback (most recent call last):
File "train.py", line 10, in
from lib.nn_model.model import get_nn_model
File "/home/afo214/tensorflow/vrp/seqTOseq/debug_seq2seq/lib/nn_model/model.py", line 4, in
from seq2seq.models import SimpleSeq2seq
File "build/bdist.linux-x86_64/egg/seq2seq/init.py", line 1, in
File "build/bdist.linux-x86_64/egg/seq2seq/cells.py", line 1, in
ImportError: cannot import name weight
Any idea?
andy1028@andy1028-Envy:/media/andy1028/data1t/os_prj/github/debug_seq2seq$ python bin/train.py
INFO:gensim.utils:Pattern library is not installed, lemmatization won't be available.
INFO:summa.preprocessing.cleaner:'pattern' package not found; tag filters are not available for English
Using TensorFlow backend.
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcurand.so locally
Traceback (most recent call last):
File "bin/train.py", line 10, in
from lib.nn_model.model import get_nn_model
File "/media/andy1028/data1t/os_prj/github/debug_seq2seq/lib/nn_model/model.py", line 4, in
from seq2seq.models import SimpleSeq2seq
ImportError: cannot import name SimpleSeq2seq
I had the done:
andy1028@andy1028-Envy:/media/andy1028/data1t/os_prj/github/debug_seq2seq/seq2seq$ git remote -v
origin https://github.com/farizrahman4u/seq2seq.git (fetch)
origin https://github.com/farizrahman4u/seq2seq.git (push)
andy1028@andy1028-Envy:/media/andy1028/data1t/os_prj/github/debug_seq2seq/seq2seq$ sudo python3 setup.py install
running install
running bdist_egg
running egg_info
writing seq2seq.egg-info/PKG-INFO
writing requirements to seq2seq.egg-info/requires.txt
writing dependency_links to seq2seq.egg-info/dependency_links.txt
writing top-level names to seq2seq.egg-info/top_level.txt
reading manifest file 'seq2seq.egg-info/SOURCES.txt'
writing manifest file 'seq2seq.egg-info/SOURCES.txt'
.....
andy1028@andy1028-Envy:/media/andy1028/data1t/os_prj/github/debug_seq2seq$ python bin/train.py
INFO:gensim.utils:Pattern library is not installed, lemmatization won't be available.
INFO:summa.preprocessing.cleaner:'pattern' package not found; tag filters are not available for English
Using TensorFlow backend.
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcurand.so locally
INFO:lib.dialog_processor:Loading corpus data...
INFO:lib.dialog_processor:/var/lib/try_seq2seq/corpora_processed/movie_lines_cleaned_m1.txt and /var/lib/try_seq2seq/words_index/w_idx_movie_lines_cleaned_m1.txt exist, loading files from disk
INFO:main:-----
INFO:lib.w2v_model.w2v:Loading model from /var/lib/try_seq2seq/w2v_models/movie_lines_cleaned_w5_m1_v256.bin
INFO:gensim.utils:loading Word2Vec object from /var/lib/try_seq2seq/w2v_models/movie_lines_cleaned_w5_m1_v256.bin
INFO:gensim.utils:setting ignored attribute syn0norm to None
INFO:gensim.utils:setting ignored attribute cum_table to None
INFO:lib.w2v_model.w2v:Model "movie_lines_cleaned_w5_m1_v256.bin" has been loaded.
INFO:main:-----
INFO:lib.nn_model.model:Initializing NN model with the following params:
INFO:lib.nn_model.model:Input dimension: 256 (token vector size)
INFO:lib.nn_model.model:Hidden dimension: 512
INFO:lib.nn_model.model:Output dimension: 20001 (token dict size)
INFO:lib.nn_model.model:Input seq length: 16
INFO:lib.nn_model.model:Output seq length: 6
INFO:lib.nn_model.model:Batch size: 32
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:925] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
I tensorflow/core/common_runtime/gpu/gpu_init.cc:102] Found device 0 with properties:
name: GeForce GTX 950M
major: 5 minor: 0 memoryClockRate (GHz) 1.124
pciBusID 0000:01:00.0
Total memory: 4.00GiB
Free memory: 3.68GiB
I tensorflow/core/common_runtime/gpu/gpu_init.cc:126] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_init.cc:136] 0: Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:839] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 950M, pci bus id: 0000:01:00.0)
Traceback (most recent call last):
File "bin/train.py", line 37, in
learn()
File "bin/train.py", line 30, in learn
nn_model = get_nn_model(token_dict_size=len(index_to_token))
File "/media/andy1028/data1t/os_prj/github/debug_seq2seq/lib/nn_model/model.py", line 29, in get_nn_model
depth=1
File "build/bdist.linux-x86_64/egg/seq2seq/models.py", line 77, in SimpleSeq2Seq
File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 308, in add
output_tensor = layer(self.outputs[0])
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 514, in call
self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 572, in add_inbound_node
Node.create_node(self, inbound_layers, node_indices, tensor_indices)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 149, in create_node
output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
File "build/bdist.linux-x86_64/egg/recurrentshop/engine.py", line 305, in call
File "build/bdist.linux-x86_64/egg/recurrentshop/engine.py", line 51, in
File "/usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py", line 1175, in rnn
state_size = int(states[0].get_shape()[-1])
TypeError: int returned non-int (type NoneType)
andy1028@andy1028-Envy:/media/andy1028/data1t/os_prj/github/debug_seq2seq$
Tried to reproduce your results, but fail. After doing minor fixes re changed case, etc, I get:
INFO:lib.nn_model.train:Full-data-pass iteration num: 1
/home/akhavr/src/seq2seq/.env/local/lib/python2.7/site-packages/keras/models.py:610: UserWarning: The "show_accuracy" argument is deprecated, instead you should pass the "accuracy" metric to the model at compile time:
`model.compile(optimizer, loss, metrics=["accuracy"])`
warnings.warn('The "show_accuracy" argument is deprecated, '
Traceback (most recent call last):
File "bin/train.py", line 37, in <module>
learn()
File "bin/train.py", line 33, in learn
train_model(nn_model, w2v_model, dialog_lines_for_nn, index_to_token)
File "/home/akhavr/src/seq2seq/debug_seq2seq/lib/nn_model/train.py", line 87, in train_model
nn_model.fit(X_train, Y_train, batch_size=TRAIN_BATCH_SIZE, nb_epoch=1, show_accuracy=True, verbose=1)
File "/home/akhavr/src/seq2seq/.env/local/lib/python2.7/site-packages/keras/models.py", line 627, in fit
sample_weight=sample_weight)
File "/home/akhavr/src/seq2seq/.env/local/lib/python2.7/site-packages/keras/engine/training.py", line 1052, in fit
batch_size=batch_size)
File "/home/akhavr/src/seq2seq/.env/local/lib/python2.7/site-packages/keras/engine/training.py", line 983, in _standardize_user_data
exception_prefix='model target')
File "/home/akhavr/src/seq2seq/.env/local/lib/python2.7/site-packages/keras/engine/training.py", line 111, in standardize_input_data
str(array.shape))
Exception: Error when checking model target: expected recurrentcontainer_2 to have shape (None, 6, 512) but got array with shape (32, 6, 20001)
What I'm doing wrong?
When I train SimpleSeq2Seq I get the error :
optional_input_placeholder = _to_list(_OptionalInputPlaceHolder().inbound_nodes[0].output_tensors)[0]
AttributeError: '_OptionalInputPlaceHolder' object has no attribute 'inbound_nodes'
from recurrentshop . I am using Python 2.7.13
Thanks
Natan Katz
G:\Anaconda2\lib\site-packages\gensim\utils.py:840: UserWarning: detected Windows; aliasing chunkize to chunkize_serial
warnings.warn("detected Windows; aliasing chunkize to chunkize_serial")
G:\Anaconda2\lib\site-packages\gensim\utils.py:1015: UserWarning: Pattern library is not installed, lemmatization won't be available.
warnings.warn("Pattern library is not installed, lemmatization won't be available.")
INFO:summa.preprocessing.cleaner:'pattern' package not found; tag filters are not available for English
Using Theano backend.
INFO:lib.dialog_processor:Loading corpus data...
INFO:lib.dialog_processor:H:/EclipseWorkspace/NetFault_Analysis/keras-seq2seq/debug_seq2seq-master/corpora_processed\movie_lines_cleaned_m1.txt and H:/EclipseWorkspace/NetFault_Analysis/keras-seq2seq/debug_seq2seq-master/words_index\w_idx_movie_lines_cleaned_m1.txt exist, loading files from disk
INFO:main:-----
INFO:lib.w2v_model.w2v:Loading model from H:/EclipseWorkspace/NetFault_Analysis/keras-seq2seq/debug_seq2seq-master/w2v_models\movie_lines_cleaned_w5_m1_v128.bin
INFO:gensim.utils:loading Word2Vec object from H:/EclipseWorkspace/NetFault_Analysis/keras-seq2seq/debug_seq2seq-master/w2v_models\movie_lines_cleaned_w5_m1_v128.bin
INFO:gensim.utils:setting ignored attribute syn0norm to None
INFO:gensim.utils:setting ignored attribute cum_table to None
INFO:gensim.utils:loaded H:/EclipseWorkspace/NetFault_Analysis/keras-seq2seq/debug_seq2seq-master/w2v_models\movie_lines_cleaned_w5_m1_v128.bin
INFO:lib.w2v_model.w2v:Model "movie_lines_cleaned_w5_m1_v128.bin" has been loaded.
INFO:main:-----
INFO:lib.nn_model.model:Initializing NN model with the following params:
INFO:lib.nn_model.model:Input dimension: 128 (token vector size)
INFO:lib.nn_model.model:Hidden dimension: 128
INFO:lib.nn_model.model:Output dimension: 20001 (token dict size)
INFO:lib.nn_model.model:Input seq length: 16
INFO:lib.nn_model.model:Output seq length: 6
INFO:lib.nn_model.model:Batch size: 32
G:\Anaconda2\lib\site-packages\keras\engine\topology.py:379: UserWarning: Theregularizers
property of layers/models is deprecated. Regularization losses are now managed via thelosses
layer/model property.
warnings.warn('Theregularizers
property of layers/models '
Traceback (most recent call last):
File "H:\EclipseWorkspace\NetFault_Analysis\keras-seq2seq\debug_seq2seq-master\bin\train.py", line 37, in
learn()
File "H:\EclipseWorkspace\NetFault_Analysis\keras-seq2seq\debug_seq2seq-master\bin\train.py", line 30, in learn
nn_model = get_nn_model(token_dict_size=len(index_to_token))
File "H:\EclipseWorkspace\NetFault_Analysis\keras-seq2seq\debug_seq2seq-master\lib\nn_model\model.py", line 30, in get_nn_model
depth=1
File "build\bdist.win-amd64\egg\seq2seq\models.py", line 73, in SimpleSeq2Seq
File "build\bdist.win-amd64\egg\recurrentshop\engine.py", line 198, in add
File "G:\Anaconda2\lib\site-packages\keras\models.py", line 332, in add
output_tensor = layer(self.outputs[0])
File "G:\Anaconda2\lib\site-packages\keras\engine\topology.py", line 546, in call
self.build(input_shapes[0])
File "build\bdist.win-amd64\egg\recurrentshop\cells.py", line 121, in build
File "build\bdist.win-amd64\egg\recurrentshop\engine.py", line 83, in init
File "G:\Anaconda2\lib\site-packages\keras\initializations.py", line 95, in orthogonal
a = np.random.normal(0.0, 1.0, flat_shape)
File "mtrand.pyx", line 1636, in mtrand.RandomState.normal (numpy\random\mtrand\mtrand.c:20676)
File "mtrand.pyx", line 242, in mtrand.cont2_array_sc (numpy\random\mtrand\mtrand.c:7401)
MemoryError
I'm using anaconda2(python2 win 64 bit cpu only) and my colleague is using ubuntu with GPU (python 2). Does anybody meet this problem? How can I solve this? Many thanks.
andy1028@andy1028-Envy:/media/andy1028/data1t/os_prj/github/debug_seq2seq$ python bin/train.py
INFO:gensim.utils:Pattern library is not installed, lemmatization won't be available.
INFO:summa.preprocessing.cleaner:'pattern' package not found; tag filters are not available for English
Using TensorFlow backend.
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcurand.so locally
INFO:lib.dialog_processor:Loading corpus data...
INFO:lib.dialog_processor:/var/lib/try_seq2seq/corpora_processed/movie_lines_cleaned_m1.txt and /var/lib/try_seq2seq/words_index/w_idx_movie_lines_cleaned_m1.txt exist, loading files from disk
INFO:main:-----
INFO:lib.w2v_model.w2v:Loading model from /var/lib/try_seq2seq/w2v_models/movie_lines_cleaned_w5_m1_v256.bin
INFO:gensim.utils:loading Word2Vec object from /var/lib/try_seq2seq/w2v_models/movie_lines_cleaned_w5_m1_v256.bin
INFO:gensim.utils:setting ignored attribute syn0norm to None
INFO:gensim.utils:setting ignored attribute cum_table to None
INFO:lib.w2v_model.w2v:Model "movie_lines_cleaned_w5_m1_v256.bin" has been loaded.
INFO:main:-----
INFO:lib.nn_model.model:Initializing NN model with the following params:
INFO:lib.nn_model.model:Input dimension: 256 (token vector size)
INFO:lib.nn_model.model:Hidden dimension: 512
INFO:lib.nn_model.model:Output dimension: 20001 (token dict size)
INFO:lib.nn_model.model:Input seq length: 16
INFO:lib.nn_model.model:Output seq length: 6
INFO:lib.nn_model.model:Batch size: 32
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:925] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
I tensorflow/core/common_runtime/gpu/gpu_init.cc:102] Found device 0 with properties:
name: GeForce GTX 950M
major: 5 minor: 0 memoryClockRate (GHz) 1.124
pciBusID 0000:01:00.0
Total memory: 4.00GiB
Free memory: 3.75GiB
I tensorflow/core/common_runtime/gpu/gpu_init.cc:126] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_init.cc:136] 0: Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:839] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 950M, pci bus id: 0000:01:00.0)
Traceback (most recent call last):
File "bin/train.py", line 37, in
learn()
File "bin/train.py", line 30, in learn
nn_model = get_nn_model(token_dict_size=len(index_to_token))
File "/media/andy1028/data1t/os_prj/github/debug_seq2seq/lib/nn_model/model.py", line 29, in get_nn_model
depth=1
File "/usr/local/lib/python2.7/dist-packages/seq2seq/models.py", line 76, in SimpleSeq2Seq
model.add(encoder)
File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 276, in add
layer.create_input_layer(batch_input_shape, input_dtype)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 370, in create_input_layer
self(x)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 514, in call
self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 572, in add_inbound_node
Node.create_node(self, inbound_layers, node_indices, tensor_indices)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 149, in create_node
output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
File "build/bdist.linux-x86_64/egg/recurrentshop/engine.py", line 296, in call
File "build/bdist.linux-x86_64/egg/recurrentshop/engine.py", line 51, in
TypeError: can only concatenate tuple (not "list") to tuple
andy1028@andy1028-Envy:/media/andy1028/data1t/os_prj/github/debug_seq2seq$ git pull
Already up-to-date.
andy1028@andy1028-Envy:/media/andy1028/data1t/os_prj/github/debug_seq2seq$
I was trying to run this code on Python 2.7, Ubuntu 16.04. But it looks like i'm having some import error. This is my error's log:
"Traceback (most recent call last):
File "bin/train.py", line 10, in
from lib.nn_model.model import get_nn_model
File "/home/gmo/debug_seq2seq/lib/nn_model/model.py", line 4, in
from seq2seq.models import SimpleSeq2seq
File "build/bdist.linux-x86_64/egg/seq2seq/init.py", line 1, in
File "build/bdist.linux-x86_64/egg/seq2seq/cells.py", line 1, in
File "build/bdist.linux-x86_64/egg/recurrentshop/init.py", line 1, in
File "build/bdist.linux-x86_64/egg/recurrentshop/engine.py", line 3, in
ImportError: cannot import name initializers"
Can anyone tell me how to fix this? Thanks
there is the error I got:
INFO:gensim.utils:detected Windows; aliasing chunkize to chunkize_serial
INFO:summa.preprocessing.cleaner:'pattern' package not found; tag filters are no t available for English
INFO:lib.dialog_processor:Loading corpus data...
INFO:lib.dialog_processor:data/try_seq2seq\corpora_processed\movie_lines_cleaned_10k_m5.txt and data/try_seq2seq\words_index\w_idx_movie_lines_cleaned_10k_m5.txt exist, loading files from disk
INFO:__main__:-----
INFO:lib.w2v_model.w2v:Loading model from data/try_seq2seq\w2v_models\movie_lines_cleaned_10k_w5_m5_v128.bin
INFO:gensim.utils:loading Word2Vec object from data/try_seq2seq\w2v_models\movie_lines_cleaned_10k_w5_m5_v128.bin
INFO:gensim.utils:setting ignored attribute syn0norm to None
INFO:gensim.utils:setting ignored attribute cum_table to None
INFO:lib.w2v_model.w2v:Model "movie_lines_cleaned_10k_w5_m5_v128.bin" has been loaded.
INFO:__main__:-----
INFO:lib.nn_model.model:Initializing NN model with the following params:
INFO:lib.nn_model.model:Input dimension: 128 (token vector size)
INFO:lib.nn_model.model:Hidden dimension: 256
INFO:lib.nn_model.model:Output dimension: 1768 (token dict size)
INFO:lib.nn_model.model:Input seq length: 8
INFO:lib.nn_model.model:Output seq length: 8
INFO:lib.nn_model.model:Batch size: 64
INFO:lib.nn_model.model:Model is built
INFO:__main__:-----
INFO:lib.nn_model.train:Full-data-pass iteration num: 1
Epoch 1/1
Traceback (most recent call last):
File "bin/train.py", line 37, in <module>
learn()
File "bin/train.py", line 33, in learn
train_model(nn_model, w2v_model, dialog_lines_for_nn, index_to_token)
File "E:\work\github\nicolas-ivanov\debug_seq2seq\lib\nn_model\train.py", line 84, in train_model
nn_model.fit(X_train, Y_train, batch_size=TRAIN_BATCH_SIZE, nb_epoch=1, show_accuracy=True, verbose=1)
File "build\bdist.win-amd64\egg\keras\models.py", line 489, in fit
File "build\bdist.win-amd64\egg\keras\models.py", line 210, in _fit
File "D:\soft\tool\Anaconda\lib\site-packages\theano-0.7.0-py2.7.egg\theano\compile\function_module.py", line 871, in __call__
storage_map=getattr(self.fn, 'storage_map', None))
File "D:\soft\tool\Anaconda\lib\site-packages\theano-0.7.0-py2.7.egg\theano\gof\link.py", line 314, in raise_with_op
reraise(exc_type, exc_value, exc_trace)
File "D:\soft\tool\Anaconda\lib\site-packages\theano-0.7.0-py2.7.egg\theano\compile\function_module.py", line 859, in __call__
outputs = self.fn()
File "D:\soft\tool\Anaconda\lib\site-packages\theano-0.7.0-py2.7.egg\theano\gof\op.py", line 876, in rval
r = p(n, [x[0] for x in i], o)
File "D:\soft\tool\Anaconda\lib\site-packages\theano-0.7.0-py2.7.egg\theano\tensor\subtensor.py", line 2160, in perform
out[0] = inputs[0].__getitem__(inputs[1:])
IndexError: index 8 is out of bounds for axis 0 with size 8
Apply node that caused the error: AdvancedSubtensor(Subtensor{int64::}.0, Subtensor{int64}.0, Subtensor{int64}.0)
Toposort index: 486
Inputs types: [TensorType(float64, 3D), TensorType(int64, vector), TensorType(int64, vector)]
Inputs shapes: [(8L, 64L, 1768L), (512L,), (512L,)]
Inputs strides: [(905216L, 14144L, 8L), (8L,), (8L,)]
Inputs values: ['not shown', 'not shown', 'not shown']
Outputs clients: [[Shape_i{1}(AdvancedSubtensor.0), Elemwise{Sub}[(0, 1)](AdvancedSubtensor.0, AdvancedSubtensor.0)]]
Backtrace when the node is created:
File "build\bdist.win-amd64\egg\keras\models.py", line 70, in weighted
filtered_y_pred = y_pred[weights.nonzero()[:-1]]
Hello,
I am getting this error when trying to run the example:
ModuleNotFoundError: No module named 'utils.utils'
SimpleSeq2seq has updated to capital letter. This needs to be updated.
I came up with the following sanity check to ensure that the implementation and word embeddings etc are good.
I created a dataset of 100,000 lines, that has the following 6 lines repeated over and over again:
hi . $$$
hi , joey . $$$
hello ? $$$
who are you ? $$$
what are you doing ? $$$
nothing much . you ? $$$
I then ran your code with the following parameters and model:
TOKEN_REPRESENTATION_SIZE = 32 # word2vec parameter
HIDDEN_LAYER_DIMENSION = 4096 # number of nodes in each LSTM layer
seq2seq = Seq2seq(
batch_input_shape=(SAMPLES_BATCH_SIZE, INPUT_SEQUENCE_LENGTH, TOKEN_REPRESENTATION_SIZE),
hidden_dim = HIDDEN_LAYER_DIMENSION,
output_length=ANSWER_MAX_TOKEN_LENGTH,
output_dim=token_dict_size,
depth=2,
dropout=0.25,
peek=True
)
opt=adagrad(clipvalue=50)
model.compile(loss='sparse_categorical_crossentropy', optimizer=opt, metrics=["accuracy"])
After 10 data passes, my result look like this:
INFO:lib.nn_model.train:[hi. ] -> [$$$ doing who who $$$ $$$ $$$]
INFO:lib.nn_model.train:[hello ?] -> [$$$ doing who who $$$ $$$ $$$]
INFO:lib.nn_model.train:[who are you ?] -> [$$$ doing who who $$$ $$$ $$$]
INFO:lib.nn_model.train:[what are you doing ?] -> [$$$ doing who who $$$ $$$ $$$]
So basically, the sanity check fails. The model can't even learn the answer to these 6 lines, even though they were repeated so many times. Does anyone know why this is happening? What could be the problem?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.