while running tutorials, TypeError: 'NoneType' object is not iterable about adanet HOT 8 OPEN

Could it be that tf.estimator.EvalSpec did not support None steps in TF 1.8?

from adanet.

@cweill
i changed it to 100 (the default number for tf.estimator.EvalSpec). it stayed almost the same, except print out evaluation progressbar.

before: (steps=None):

INFO:tensorflow:Starting evaluation at 2018-11-07-09:00:11
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from C:\Users\dell\AppData\Local\Temp\tmp6brw31q8\model.ckpt-5000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Finished evaluation at 2018-11-07-09:00:14
INFO:tensorflow:Saving dict for global step 5000: accuracy = 0.8413, average_loss = 0.46480888, global_step = 5000, loss = 0.4642294

TypeError Traceback (most recent call last)
in ()
6 eval_spec=tf.estimator.EvalSpec(
7 input_fn=input_fn("test", training=False, batch_size=BATCH_SIZE),
----> 8 steps=None))

TypeError: 'NoneType' object is not iterable

after: (steps=100):

INFO:tensorflow:Starting evaluation at 2018-11-08-01:13:41
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from C:\Users\dell\AppData\Local\Temp\tmp1mw6x8zx\model.ckpt-5000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
> INFO:tensorflow:Evaluation [10/100]
INFO:tensorflow:Evaluation [20/100]
...
INFO:tensorflow:Evaluation [90/100]
INFO:tensorflow:Evaluation [100/100]
INFO:tensorflow:Finished evaluation at 2018-11-08-01:13:43
INFO:tensorflow:Saving dict for global step 5000: accuracy = 0.8421875, average_loss = 0.47057664, global_step = 5000, loss = 0.47057664

TypeError Traceback (most recent call last)
in ()
6 eval_spec=tf.estimator.EvalSpec(
7 input_fn=input_fn("test", training=False, batch_size=BATCH_SIZE),
----> 8 steps=100))

TypeError: 'NoneType' object is not iterable

from adanet.

@yuezhilanyi: Could you please copy the full log from start to end? It's not clear what is raising this TypeError.

In the meantime you can use Colab to run the tutorials. For examples: https://colab.research.google.com/github/tensorflow/adanet/blob/master/adanet/examples/tutorials/adanet_objective.ipynb

from adanet.

@cweill

Here is the full log.
full_log.txt

from adanet.

Tensorflow v1.10.0 improved local run behavior in tf.estimator.train_and_evaluate which does not reload checkpoints for evaluation, see release.md.

It maybe the reason that adanet doesn't work well on tf 1.8.0.

from adanet.

Can you try upgrading to TF 1.9+? We recently bumped our minimum version to 1.9.

from adanet.

@cweill

Sorry I can't, for it is for production sometimes also.

But I tried on my own laptop with no gpu support and TRAIN_STEPS=1000:

• when using TF 1.8, the same issues occured;
• when using TF 1.9, it succeeded

I run the whole adanet_objective.ipynb sucessfully.

from adanet.

Unfortunately, I won't be able to look into TF 1.8 support for a while. If you figure out the source of the issue, feel free to submit a PR.

from adanet.