deeppavlov / ner Goto Github PK

Named Entity Recognition

License: Apache License 2.0

Python 74.38% Jupyter Notebook 25.62%

nlp-machine-learning named-entity-recognition neural-network deep-learning natural-language-understanding natural-language-processing

ner's Introduction

This repository is outdated please move to https://github.com/deepmipt/DeepPavlov

Neural Networks for Named Entity Recognition

In this repo you can find several neural network architectures for named entity recognition from the paper "Application of a Hybrid Bi-LSTM-CRF model to the task of Russian Named Entity Recognition" https://arxiv.org/pdf/1709.09686.pdf, which is inspired by LSTM+CRF architecture from https://arxiv.org/pdf/1603.01360.pdf.

NER class from ner/network.py provides methods for construction, training and inference neural networks for Named Entity Recognition.

We provide pre-trained CNN model for Russian Named Entity Recognition. The model was trained on three datatasets:

Gareev corpus [1] (obtainable by request to authors)
FactRuEval 2016 [2]
NE3 (extended Persons-1000) [3, 4]

The pre-trained model can recognize such entities as:

Persons (PER)
Locations (LOC)
Organizations (ORG)

An example of usage of the pre-trained model is provided in example.ipynb.

Remark: at training stage the corpora were lemmatized and lowercased. So text must be tokenized and lemmatized and lowercased before feeding it into the model.

The F1 measure for presented model along with other published solution provided in the table below:

Models	Gareev’s dataset	Persons-1000	FactRuEval 2016
Gareev et al. [1]	75.05
Malykh et al. [5]	62.49
Trofimov [6]		95.57
Rubaylo et al. [7]			78.13
Sysoev et al. [8]			74.67
Ivanitsky et al. [9]			87.88
Mozharova et al. [10]		97.21
Our (Bi-LSTM+CRF)	87.17	99.26	82.10

Usage

Installing

The toolkit is implemented in Python 3 and requires a number of packages. To install all needed packages use:

$ pip3 install -r requirements.txt

$ pip3 install git+https://github.com/deepmipt/ner

Warning: there is no GPU version of TensorFlow specified in the requirements file

Command-Line Interface

The simplest way to use pre-trained Russian NER model is via command line interface:

$ echo "На конспирологическом саммите в США глава Федерального Бюро Расследований сделал невероятное заявление" | ./ner.py

На O
конспирологическом O
саммите O
в O
США B-LOC
глава O
Федерального B-ORG
Бюро I-ORG
Расследований I-ORG
сделал O
невероятное O
заявление O

And for interactive usage simply type:

$ ./ner.py

Usage as module

>>> import ner
>>> extractor = ner.Extractor()
>>> for m in extractor("На конспирологическом саммите в США глава Федерального Бюро Расследований сделал невероятное заявление"):
...     print(m)
Match(tokens=[Token(span=(32, 35), text='США')], span=Span(start=32, end=35), type='LOC')
Match(tokens=[Token(span=(42, 54), text='Федерального'), Token(span=(55, 59), text='Бюро'), Token(span=(60, 73), text='Расследований')], span=Span(start=42, end=73), type='ORG')

Training

To see how to train the network and what format of data is required see training_example.ipynb jupyter notebook.

Literature

[1] - Rinat Gareev, Maksim Tkachenko, Valery Solovyev, Andrey Simanovsky, Vladimir Ivanov: Introducing Baselines for Russian Named Entity Recognition. Computational Linguistics and Intelligent Text Processing, 329 -- 342 (2013).

[2] - https://github.com/dialogue-evaluation/factRuEval-2016

[3] - http://ai-center.botik.ru/Airec/index.php/ru/collections/28-persons-1000

[4] - http://labinform.ru/pub/named_entities/descr_ne.htm

[5] - Reproducing Russian NER Baseline Quality without Additional Data. In proceedings of the 3rd International Workshop on ConceptDiscovery in Unstructured Data, Moscow, Russia, 54 – 59 (2016)

[6] - Rubaylo A. V., Kosenko M. Y.: Software utilities for natural language information retrievial. Almanac of modern science and education, Volume 12 (114), 87 – 92.(2016)

[7] - Sysoev A. A., Andrianov I. A.: Named Entity Recognition in Russian: the Power of Wiki-Based Approach. dialog-21.ru

[8] - Ivanitskiy Roman, Alexander Shipilo, Liubov Kovriguina: Russian Named Entities Recognition and Classification Using Distributed Word and Phrase Representations. In SIMBig, 150 – 156. (2016).

[9] - Mozharova V., Loukachevitch N.: Two-stage approach in Russian named entity recognition. In Intelligence, Social Media and Web (ISMW FRUCT), 2016 International FRUCT Conference, 1 – 6 (2016)

ner's People

Contributors

Stargazers

Watchers

ner's Issues

Add test results to readme

@mu-arkhipov Please add to readme bib reference and link to the paper on arxiv and the table that gives comparison of the model with previous results.

Error on sentences without "normal" tokens

When you pass an empty string or string without normal tokens (for example, ascii emoticons), exception is raised:

>>> import ner
>>> extractor = ner.Extractor()
>>> list(extractor(':)')) # works fine — empty list is returned
>>> list(extractor('')) # fails
>>> list(extractor('|*!*|')) # fails

Maybe these cases should return an empty generator too? Trace:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/universome/pyvenvs/zoo/lib/python3.6/site-packages/ner/extractor/extractor.py", line 54, in __call__
    tags = self.network.predict_for_token_batch([tokens_lemmas])[0]
  File "/home/universome/pyvenvs/zoo/lib/python3.6/site-packages/ner/network.py", line 379, in predict_for_token_batch
    batch_x, _ = self.corpus.tokens_batch_to_numpy_batch(tokens_batch)
  File "/home/universome/pyvenvs/zoo/lib/python3.6/site-packages/ner/corpus.py", line 205, in tokens_batch_to_numpy_batch
    max_token_len = max([len(token) for utt in batch_x for token in utt])
ValueError: max() arg is an empty sequence

Could not find a version that satisfies the requirement

When I try to execute the command sudo pip3 install -r ./requirements.txt --user with dependencies from this repository:

numpy==1.13.1
tensorflow==1.3.0
pymorphy2==0.8
pymorphy2-dicts==2.4.393442.3710985
tqdm==4.19.5
requests==2.18.4
gensim==2.3.0

I get the error:

Could not find a version that satisfies the requirement tensorflow==1.3.0 (from -r ./requirements.txt (line 2)) (from versions: 1.13.0rc1, 1.13.0rc2, 1.13.1, 1.13.2, 1.14.0rc0, 1.14.0rc1, 1.14.0, 2.0.0a0, 2.0.0b0, 2.0.0b1)
No matching distribution found for tensorflow==1.3.0 (from -r ./requirements.txt (line 2))

Weblink for pretrained model in ner/ner/extractor/extractor.py is probably wrong

Please, take a look at this line
model_url='http://lnsigo.mipt.ru/export/ner/ner_model_total_rus.tar.gz'
Probably the correct link is http://lnsigo.mipt.ru/export/models/ner/ner_model_total_rus.tar.gz

Unable to evaluate on custom entities

I have other dataset then CoNLL 2003, where i have my own custom entites. But i am unable to evaluate model as it is unable to find phrases in training set. any solution?

Using results from training_example.ipynb

How do I use the results to predict on new sentences. Do I save the architecture and trained model file. Want something similar to “example.ipynb”, but on English data.

FileNotFoundError on Extractor()

On first time init, i had a ReadError after auto-downloading and extracting ner_model_total_rus.tar.gz:

~/speech/lib/python3.5/tarfile.py in open(cls, name, mode, fileobj, bufsize, **kwargs)
   1560                         fileobj.seek(saved_pos)
   1561                     continue
-> 1562             raise ReadError("file could not be opened successfully")
   1563 
   1564         elif ":" in mode:

ReadError: file could not be opened successfully

Then, on calling extractor = ner.Extractor() i haveFileNotFoundError: [Errno 2] No such file or directory: '/home/user/semantic_venv/lib/python3.5/site-packages/ner/extractor/../model/params.json'

Lower F1 in FactRuEval 2016

Code from training_example achieve only 55 F1 in FactRuEval 2016 (82.10 F1 in your report).
What could be the problem?

Parameters for Bi-LSTM + CRF, Bi-LSTM + CRF + external word embedding news

Hello there, great work!

Could you please specify the exact parameters that should be used in order to replicate your experiments from the paper, for both "Bi-LSTM + CRF" and "Bi-LSTM + CRF + external word embedding news" models? The model_params for instantiating the NER network and the learning_params for training.

Thanks!

TypeError: init() got an unexpected keyword argument 'file'

Hi, thanks for the great and a long-awaited tool.

I used to have tensorflow 1.7.0 for CPU installed (initially from provided binaries, then built from source).

In both cases I've bumped into the following issue:

$ python3
Python 3.5.2 (default, Nov 23 2017, 16:37:01) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ner
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>$ python3
Python 3.5.2 (default, Nov 23 2017, 16:37:01) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow
/usr/local/lib/python3.5/dist-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
>>> tensorflow.__version__
'1.7.0'
>>> import ner
>>> 

  File "/usr/local/lib/python3.5/dist-packages/ner/__init__.py", line 1, in <module>
    from .extractor import Extractor
  File "/usr/local/lib/python3.5/dist-packages/ner/extractor/__init__.py", line 1, in <module>
    from .extractor import Extractor
  File "/usr/local/lib/python3.5/dist-packages/ner/extractor/extractor.py", line 11, in <module>
    from ner.network import NER
  File "/usr/local/lib/python3.5/dist-packages/ner/network.py", line 17, in <module>
    import tensorflow as tf
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/__init__.py", line 24, in <module>
    from tensorflow.python import *  # pylint: disable=redefined-builtin
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/__init__.py", line 52, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/core/framework/graph_pb2.py", line 16, in <module>
    from tensorflow.core.framework import node_def_pb2 as tensorflow_dot_core_dot_framework_dot_node__def__pb2
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/core/framework/node_def_pb2.py", line 16, in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/core/framework/attr_value_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/core/framework/tensor_pb2.py", line 16, in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/core/framework/resource_handle_pb2.py", line 41, in <module>
    options=None, file=DESCRIPTOR),
TypeError: __init__() got an unexpected keyword argument 'file'

However, after importing tensorflow first, things do work.

$ python3
Python 3.5.2 (default, Nov 23 2017, 16:37:01) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow
/usr/local/lib/python3.5/dist-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
>>> tensorflow.__version__
'1.7.0'
>>> import ner
>>>

Prior to noticing the previous hack, I had simply installed the older tensorflow version (I forgot which one, sorry), and that fixed it.

unimplementederror: tensor array has size zero, but element shape [?,256] is not fully defined. currently only static shapes are supported when packing zero-size tensorarray

I am using tensorflow 1.14.
when I try to run training, i encountered this error.
please help me to resolve this.

IndexError: string index out of range

Awkward IndexError on specific strings: Сергее-вичу gives the error while Сергее-вич works fine.

Code to reproduce:

import ner

extractor = ner.Extractor(model_url='http://lnsigo.mipt.ru/export/models/ner/ner_model_total_rus.tar.gz')
list(extractor('Сергее-вич')) # works!
list(extractor('Сергее-вичу')) # fails

Trace:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/universome/pyvenvs/zoo/lib/python3.6/site-packages/ner/extractor/extractor.py", line 54, in __call__
    tags = self.network.predict_for_token_batch([tokens_lemmas])[0]
  File "/home/universome/pyvenvs/zoo/lib/python3.6/site-packages/ner/network.py", line 379, in predict_for_token_batch
    batch_x, _ = self.corpus.tokens_batch_to_numpy_batch(tokens_batch)
  File "/home/universome/pyvenvs/zoo/lib/python3.6/site-packages/ner/corpus.py", line 219, in tokens_batch_to_numpy_batch
    x['capitalization'][n, :len(utt)] = [tok[0].isupper() for tok in utt]
  File "/home/universome/pyvenvs/zoo/lib/python3.6/site-packages/ner/corpus.py", line 219, in <listcomp>
    x['capitalization'][n, :len(utt)] = [tok[0].isupper() for tok in utt]
IndexError: string index out of range

Examples are not reproduced (lower accuracy)

Just installed NER from Gtihub. Run file training_example.ipynb from https://github.com/deepmipt/ner/blob/master/training_example.ipynb, then the results in example notebook looks bad:

Epoch 4
Eval on valid:
processed 54612 tokens with 5942 phrases; found: 314 phrases; correct: 13.

precision:  4.14%; recall:  0.22%; FB1:  0.42


Eval on train:
processed 217662 tokens with 23499 phrases; found: 1416 phrases; correct: 47.

precision:  3.32%; recall:  0.20%; FB1:  0.38

	LOC: precision:  2.36%; recall:  0.21%; F1:  0.39 635

	MISC: precision:  0.00%; recall:  0.00%; F1:  0.00 0

	ORG: precision:  8.45%; recall:  0.38%; F1:  0.73 284

	PER: precision:  1.61%; recall:  0.12%; F1:  0.23 497


Eval on valid:
processed 54612 tokens with 5942 phrases; found: 316 phrases; correct: 13.

precision:  4.11%; recall:  0.22%; FB1:  0.42

	LOC: precision:  0.69%; recall:  0.05%; F1:  0.10 144

	MISC: precision:  0.00%; recall:  0.00%; F1:  0.00 0

	ORG: precision:  4.23%; recall:  0.22%; F1:  0.42 71

	PER: precision:  8.91%; recall:  0.49%; F1:  0.93 101


Eval on test:
processed 49888 tokens with 5648 phrases; found: 258 phrases; correct: 15.

precision:  5.81%; recall:  0.27%; FB1:  0.51

	LOC: precision:  2.73%; recall:  0.18%; F1:  0.34 110

	MISC: precision:  0.00%; recall:  0.00%; F1:  0.00 0

	ORG: precision:  9.76%; recall:  0.24%; F1:  0.47 41

	PER: precision:  7.48%; recall:  0.49%; F1:  0.93 107

This result is wrong. Please help me!!

Examples are not reproduced

Just installed NER from Gtihub.

The results in example notebook looks corrupted:

На B-PER
конспирологическом I-PER
саммите O
в B-PER
США I-PER
глава O
Федерального O
Бюро O
Расследований O
сделал O
невероятное O
заявление O

Moreover, re-running example.ipynb leads to downloading the model again (MD5 sum is wrong). Perhaps, the model was changed on MIPT website?

URL for model download not working

http://lnsigo.mipt.ru/export/ner/ner_model_total_rus.tar.gz

Excited to try out your package!

ValueError: max() arg is an empty sequence

mldl@mldlUB1604:/ub16_prj/ner$ python3 ner.py
Downloading from http://lnsigo.mipt.ru/export/models/ner/ner_model_total_rus.tar.gz to model/ner_model_total_rus.tar.gz
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 44.2M/44.3M [01:07<00:00, 696kB/s]Extracting model/ner_model_total_rus.tar.gz archive into model/
/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gradients_impl.py:95: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
"Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 44.3M/44.3M [01:20<00:00, 696kB/s]
Traceback (most recent call last):
File "ner.py", line 61, in
print_predict(query)
File "ner.py", line 55, in print_predict
tags = network.predict_for_token_batch([tokens_lemmas])[0]
File "/home/mldl/ub16_prj/ner/ner/network.py", line 379, in predict_for_token_batch
batch_x, _ = self.corpus.tokens_batch_to_numpy_batch(tokens_batch)
File "/home/mldl/ub16_prj/ner/ner/corpus.py", line 205, in tokens_batch_to_numpy_batch
max_token_len = max([len(token) for utt in batch_x for token in utt])
ValueError: max() arg is an empty sequence
mldl@mldlUB1604:/ub16_prj/ner$

UnimplementedError: TensorArray has size zero, but element shape <unknown> is not fully defined. Currently only static shapes are supported when packing zero-size TensorArrays.

Hi there!

I tried running your notebook example as is, just increasing epochs to 20 instead of 5, and I got the error below. Running with 5 epochs worked fine.

I think it has something to do with running the Adam optimizer, as defined the method below, from network.py.

I guess it's running the optimizer code after 10 epochs, and there's something incompatible there with my tensorflow version. I'm running 1.4.1 version.

def get_train_op(self, loss, learning_rate, learnable_scopes=None, lr_decay_rate=None):
        global_step = tf.Variable(0, trainable=False)
        try:
            n_training_samples = len(self.corpus.dataset["train"])
        except TypeError:
            n_training_samples = 1024
        batch_size = tf.shape(self._x_w)[0]
        decay_steps = tf.cast(n_training_samples / batch_size, tf.int32)
        if lr_decay_rate is not None:
            learning_rate = tf.train.exponential_decay(learning_rate,
                                                       global_step,
                                                       decay_steps=decay_steps,
                                                       decay_rate=lr_decay_rate,
                                                       staircase=True)
            self._learning_rate_decayed = learning_rate
        variables = self.get_trainable_variables(learnable_scopes)

        # For batch norm it is necessary to update running averages
        extra_update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
        with tf.control_dependencies(extra_update_ops):
            train_op = tf.train.AdamOptimizer(learning_rate).minimize(loss, global_step=global_step, var_list=variables)
        return train_op

Epoch 10

UnimplementedError Traceback (most recent call last)
~/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
1322 try:
-> 1323 return fn(*args)
1324 except errors.OpError as e:

~/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/client/session.py in _run_fn(session, feed_dict, fetch_list, target_list, options, run_metadata)
1301 feed_dict, fetch_list, target_list,
-> 1302 status, run_metadata)
1303

~/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py in exit(self, type_arg, value_arg, traceback_arg)
472 compat.as_text(c_api.TF_Message(self.status.status)),
--> 473 c_api.TF_GetCode(self.status.status))
474 # Delete the underlying status object from memory otherwise it stays alive

UnimplementedError: TensorArray has size zero, but element shape is not fully defined. Currently only static shapes are supported when packing zero-size TensorArrays.
[[Node: gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGatherV3 = TensorArrayGatherV3[dtype=DT_FLOAT, element_shape=, _device="/job:localhost/replica:0/task:0/device:GPU:0"](gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGrad/TensorArrayGradV3, rnn/TensorArrayUnstack/range, gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGrad/gradient_flow)]]
[[Node: Adam/update/_174 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_2413_Adam/update", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

During handling of the above exception, another exception occurred:

UnimplementedError Traceback (most recent call last)
in ()
4 'batch_size': 8,
5 'learning_rate_decay': 0.707}
----> 6 results = net.fit(**learning_params)

~/repositorios/russian-ner/ner/network.py in fit(self, batch_gen, batch_size, learning_rate, epochs, dropout_rate, learning_rate_decay)
242 self.train_writer.add_summary(summary)
243
--> 244 self._sess.run(self._train_op, feed_dict=feed_dict)
245 if self.verbouse:
246 self.eval_conll('valid', print_results=True)

~/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/client/session.py in run(self, fetches, feed_dict, options, run_metadata)
887 try:
888 result = self._run(None, fetches, feed_dict, options_ptr,
--> 889 run_metadata_ptr)
890 if run_metadata:
891 proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

~/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
1118 if final_fetches or final_targets or (handle and feed_dict_tensor):
1119 results = self._do_run(handle, final_targets, final_fetches,
-> 1120 feed_dict_tensor, options, run_metadata)
1121 else:
1122 results = []

~/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/client/session.py in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)
1315 if handle is None:
1316 return self._do_call(_run_fn, self._session, feeds, fetches, targets,
-> 1317 options, run_metadata)
1318 else:
1319 return self._do_call(_prun_fn, self._session, handle, feeds, fetches)

~/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
1334 except KeyError:
1335 pass
-> 1336 raise type(e)(node_def, op, message)
1337
1338 def _extend_graph(self):

Caused by op 'gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGatherV3', defined at:
File "/home/pedro/anaconda3/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/pedro/anaconda3/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/pedro/anaconda3/lib/python3.6/site-packages/ipykernel_launcher.py", line 16, in
app.launch_new_instance()
File "/home/pedro/anaconda3/lib/python3.6/site-packages/traitlets/config/application.py", line 658, in launch_instance
app.start()
File "/home/pedro/anaconda3/lib/python3.6/site-packages/ipykernel/kernelapp.py", line 477, in start
ioloop.IOLoop.instance().start()
File "/home/pedro/anaconda3/lib/python3.6/site-packages/zmq/eventloop/ioloop.py", line 177, in start
super(ZMQIOLoop, self).start()
File "/home/pedro/anaconda3/lib/python3.6/site-packages/tornado/ioloop.py", line 888, in start
handler_func(fd_obj, events)
File "/home/pedro/anaconda3/lib/python3.6/site-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/home/pedro/anaconda3/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 440, in _handle_events
self._handle_recv()
File "/home/pedro/anaconda3/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 472, in _handle_recv
self._run_callback(callback, msg)
File "/home/pedro/anaconda3/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 414, in _run_callback
callback(*args, **kwargs)
File "/home/pedro/anaconda3/lib/python3.6/site-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/home/pedro/anaconda3/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 283, in dispatcher
return self.dispatch_shell(stream, msg)
File "/home/pedro/anaconda3/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 235, in dispatch_shell
handler(stream, idents, msg)
File "/home/pedro/anaconda3/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 399, in execute_request
user_expressions, allow_stdin)
File "/home/pedro/anaconda3/lib/python3.6/site-packages/ipykernel/ipkernel.py", line 196, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "/home/pedro/anaconda3/lib/python3.6/site-packages/ipykernel/zmqshell.py", line 533, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
File "/home/pedro/anaconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2698, in run_cell
interactivity=interactivity, compiler=compiler, result=result)
File "/home/pedro/anaconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2802, in run_ast_nodes
if self.run_code(code, result):
File "/home/pedro/anaconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2862, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 16, in
net = NER(corp, **model_params)
File "/home/pedro/repositorios/russian-ner/ner/network.py", line 184, in init
self._train_op = self.get_train_op(loss, learning_rate_ph, lr_decay_rate=learning_rate_decay_ph)
File "/home/pedro/repositorios/russian-ner/ner/network.py", line 375, in get_train_op
train_op = tf.train.AdamOptimizer(learning_rate).minimize(loss, global_step=global_step, var_list=variables)
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/training/optimizer.py", line 343, in minimize
grad_loss=grad_loss)
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/training/optimizer.py", line 414, in compute_gradients
colocate_gradients_with_ops=colocate_gradients_with_ops)
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 581, in gradients
grad_scope, op, func_call, lambda: grad_fn(op, *out_grads))
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 353, in _MaybeCompile
return grad_fn() # Exit early
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 581, in
grad_scope, op, func_call, lambda: grad_fn(op, *out_grads))
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/ops/tensor_array_grad.py", line 186, in _TensorArrayScatterGrad
grad = g.gather(indices)
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/ops/tensor_array_ops.py", line 361, in gather
element_shape=element_shape)
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 4158, in _tensor_array_gather_v3
flow_in=flow_in, dtype=dtype, element_shape=element_shape, name=name)
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
op_def=op_def)
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1470, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

...which was originally created as op 'rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3', defined at:
File "/home/pedro/anaconda3/lib/python3.6/runpy.py", line 193, in run_module_as_main
"main", mod_spec)
[elided 19 identical lines from previous traceback]
File "", line 16, in
net = NER(corp, **model_params)
File "/home/pedro/repositorios/russian-ner/ner/network.py", line 138, in init
sequence_lengths)
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/contrib/crf/python/ops/crf.py", line 170, in crf_log_likelihood
log_norm = crf_log_norm(inputs, sequence_lengths, transition_params)
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/contrib/crf/python/ops/crf.py", line 137, in crf_log_norm
dtype=dtypes.float32)
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 614, in dynamic_rnn
dtype=dtype)
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 728, in dynamic_rnn_loop
for ta, input in zip(input_ta, flat_input))
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 728, in
for ta, input in zip(input_ta, flat_input))
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/util/tf_should_use.py", line 107, in wrapped
return _add_should_use_warning(fn(*args, **kwargs))
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/ops/tensor_array_ops.py", line 414, in unstack
indices=math_ops.range(0, num_elements), value=value, name=name)
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/util/tf_should_use.py", line 107, in wrapped
return _add_should_use_warning(fn(*args, **kwargs))
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/ops/tensor_array_ops.py", line 442, in scatter
name=name)
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 4553, in _tensor_array_scatter_v3
flow_in=flow_in, name=name)
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/pedro/virtualenv/machine-learning/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
op_def=op_def)

UnimplementedError (see above for traceback): TensorArray has size zero, but element shape is not fully defined. Currently only static shapes are supported when packing zero-size TensorArrays.
[[Node: gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGatherV3 = TensorArrayGatherV3[dtype=DT_FLOAT, element_shape=, _device="/job:localhost/replica:0/task:0/device:GPU:0"](gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGrad/TensorArrayGradV3, rnn/TensorArrayUnstack/range, gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGrad/gradient_flow)]]
[[Node: Adam/update/_174 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_2413_Adam/update", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]