'tf_convert_data.py' make only 'voc_2007.train.tfrecord' file in ./tfrecords.
how can I make this 'voc_2007_test.tfrecord'?
$ sudo python eval_ssd_network.py --eval_dir=${EVAL_DIR} --dataset_dir=${DATASET_DIR} --dataset_name=pascalvoc_2007 --dataset_split_name=test --model_name=ssd_300_vgg --checkpoint_path=${CHECKPOINT_PATH} --batch_size=1
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcudnn.so.6 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.so.8.0 locally
===========================================================================
Training | Evaluation flags:
===========================================================================
{'batch_size': 1,
'checkpoint_path': './checkpoints/VGG_VOC0712_SSD_300x300_ft_iter_120000.ckpt',
'dataset_dir': './VOC2007/',
'dataset_name': 'pascalvoc_2007',
'dataset_split_name': 'test',
'eval_dir': './logs/',
'eval_image_size': None,
'eval_resize': 4,
'gpu_memory_fraction': 0.04,
'keep_top_k': 200,
'master': '',
'matching_threshold': 0.5,
'max_num_batches': None,
'model_name': 'ssd_300_vgg',
'moving_average_decay': None,
'nms_threshold': 0.45,
'num_classes': 21,
'num_preprocessing_threads': 4,
'preprocessing_name': None,
'remove_difficult': True,
'select_threshold': 0.01,
'select_top_k': 400,
'wait_for_checkpoints': False}
===========================================================================
SSD net parameters:
===========================================================================
{'anchor_offset': 0.5,
'anchor_ratios': [[2, 0.5],
[2, 0.5, 3, 0.3333333333333333],
[2, 0.5, 3, 0.3333333333333333],
[2, 0.5, 3, 0.3333333333333333],
[2, 0.5],
[2, 0.5]],
'anchor_size_bounds': [0.15, 0.9],
'anchor_sizes': [(21.0, 45.0),
(45.0, 99.0),
(99.0, 153.0),
(153.0, 207.0),
(207.0, 261.0),
(261.0, 315.0)],
'anchor_steps': [8, 16, 32, 64, 100, 300],
'feat_layers': ['block4', 'block7', 'block8', 'block9', 'block10', 'block11'],
'feat_shapes': [(38, 38), (19, 19), (10, 10), (5, 5), (3, 3), (1, 1)],
'img_shape': (300, 300),
'no_annotation_label': 21,
'normalizations': [20, -1, -1, -1, -1, -1],
'num_classes': 21,
'prior_scaling': [0.1, 0.1, 0.2, 0.2]}
===========================================================================
Training | Evaluation dataset files:
===========================================================================
['./VOC2007/voc_2007_test.tfrecord']
INFO:tensorflow:Evaluating ./checkpoints/VGG_VOC0712_SSD_300x300_ft_iter_120000.ckpt
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/contrib/training/python/training/evaluation.py:298: init (from tensorflow.python.training.summary_io) is deprecated and will be removed after 2016-11-30.
Instructions for updating:
Please switch to tf.summary.FileWriter. The interface and behavior is the same; this is just a rename.
INFO:tensorflow:Starting evaluation at 2017-03-30-08:37:50
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:910] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
I tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 0 with properties:
name: TITAN X (Pascal)
major: 6 minor: 1 memoryClockRate (GHz) 1.531
pciBusID 0000:01:00.0
Total memory: 11.90GiB
Free memory: 3.24GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 0: Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: TITAN X (Pascal), pci bus id: 0000:01:00.0)
INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.NotFoundError'>, ./VOC2007/voc_2007_test.tfrecord
[[Node: pascalvoc_2007_data_provider/parallel_read/ReaderReadV2 = ReaderReadV2[_device="/job:localhost/replica:0/task:0/cpu:0"](pascalvoc_2007_data_provider/parallel_read/TFRecordReaderV2, pascalvoc_2007_data_provider/parallel_read/filenames)]]
Caused by op u'pascalvoc_2007_data_provider/parallel_read/ReaderReadV2', defined at:
File "eval_ssd_network.py", line 350, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 44, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "eval_ssd_network.py", line 150, in main
shuffle=False)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/slim/python/slim/data/dataset_data_provider.py", line 83, in init
seed=seed)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/slim/python/slim/data/parallel_reader.py", line 234, in parallel_read
reader_kwargs=reader_kwargs).read(filename_queue)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/slim/python/slim/data/parallel_reader.py", line 132, in read
enqueue_ops.append(self._common_queue.enqueue(reader.read(queue)))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/io_ops.py", line 191, in read
return gen_io_ops._reader_read_v2(self._reader_ref, queue_ref, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_io_ops.py", line 410, in _reader_read_v2
queue_handle=queue_handle, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2327, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1226, in init
self._traceback = _extract_stack()
NotFoundError (see above for traceback): ./VOC2007/voc_2007_test.tfrecord
[[Node: pascalvoc_2007_data_provider/parallel_read/ReaderReadV2 = ReaderReadV2[_device="/job:localhost/replica:0/task:0/cpu:0"](pascalvoc_2007_data_provider/parallel_read/TFRecordReaderV2, pascalvoc_2007_data_provider/parallel_read/filenames)]]
W tensorflow/core/framework/op_kernel.cc:993] Out of range: PaddingFIFOQueue '_2_batch/padding_fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/padding_fifo_queue, batch/n)]]
W tensorflow/core/framework/op_kernel.cc:993] Out of range: PaddingFIFOQueue '_2_batch/padding_fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/padding_fifo_queue, batch/n)]]
W tensorflow/core/framework/op_kernel.cc:993] Out of range: PaddingFIFOQueue '_2_batch/padding_fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/padding_fifo_queue, batch/n)]]
W tensorflow/core/framework/op_kernel.cc:993] Out of range: PaddingFIFOQueue '_2_batch/padding_fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/padding_fifo_queue, batch/n)]]
W tensorflow/core/framework/op_kernel.cc:993] Out of range: PaddingFIFOQueue '_2_batch/padding_fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/padding_fifo_queue, batch/n)]]
W tensorflow/core/framework/op_kernel.cc:993] Out of range: PaddingFIFOQueue '_2_batch/padding_fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/padding_fifo_queue, batch/n)]]
W tensorflow/core/framework/op_kernel.cc:993] Out of range: PaddingFIFOQueue '_2_batch/padding_fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/padding_fifo_queue, batch/n)]]
W tensorflow/core/framework/op_kernel.cc:993] Out of range: PaddingFIFOQueue '_2_batch/padding_fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/padding_fifo_queue, batch/n)]]
W tensorflow/core/framework/op_kernel.cc:993] Out of range: PaddingFIFOQueue '_2_batch/padding_fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/padding_fifo_queue, batch/n)]]
W tensorflow/core/framework/op_kernel.cc:993] Out of range: PaddingFIFOQueue '_2_batch/padding_fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/padding_fifo_queue, batch/n)]]
W tensorflow/core/framework/op_kernel.cc:993] Out of range: PaddingFIFOQueue '_2_batch/padding_fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/padding_fifo_queue, batch/n)]]
W tensorflow/core/framework/op_kernel.cc:993] Out of range: PaddingFIFOQueue '_2_batch/padding_fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/padding_fifo_queue, batch/n)]]
W tensorflow/core/framework/op_kernel.cc:993] Out of range: PaddingFIFOQueue '_2_batch/padding_fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_FLOAT, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_INT64, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/padding_fifo_queue, batch/n)]]
I tensorflow/core/kernels/logging_ops.cc:79] AP_VOC07/mAP[0]
I tensorflow/core/kernels/logging_ops.cc:79] AP_VOC12/mAP[0]
Traceback (most recent call last):
File "eval_ssd_network.py", line 350, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 44, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "eval_ssd_network.py", line 324, in main
session_config=config)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/slim/python/slim/evaluation.py", line 207, in evaluate_once
config=session_config)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/training/python/training/evaluation.py", line 430, in evaluate_once
session.run(eval_ops, feed_dict)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py", line 478, in exit
self._close_internal(exception_type)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py", line 511, in _close_internal
self._sess.close()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py", line 739, in close
self._sess.close()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py", line 827, in close
self._coord.join()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/coordinator.py", line 386, in join
six.reraise(*self._exc_info_to_raise)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/queue_runner_impl.py", line 234, in _run
sess.run(enqueue_op)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 767, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 965, in _run
feed_dict_string, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1015, in _do_run
target_list, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1035, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: ./VOC2007/voc_2007_test.tfrecord
[[Node: pascalvoc_2007_data_provider/parallel_read/ReaderReadV2 = ReaderReadV2[_device="/job:localhost/replica:0/task:0/cpu:0"](pascalvoc_2007_data_provider/parallel_read/TFRecordReaderV2, pascalvoc_2007_data_provider/parallel_read/filenames)]]
Caused by op u'pascalvoc_2007_data_provider/parallel_read/ReaderReadV2', defined at:
File "eval_ssd_network.py", line 350, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 44, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "eval_ssd_network.py", line 150, in main
shuffle=False)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/slim/python/slim/data/dataset_data_provider.py", line 83, in init
seed=seed)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/slim/python/slim/data/parallel_reader.py", line 234, in parallel_read
reader_kwargs=reader_kwargs).read(filename_queue)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/slim/python/slim/data/parallel_reader.py", line 132, in read
enqueue_ops.append(self._common_queue.enqueue(reader.read(queue)))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/io_ops.py", line 191, in read
return gen_io_ops._reader_read_v2(self._reader_ref, queue_ref, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_io_ops.py", line 410, in _reader_read_v2
queue_handle=queue_handle, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2327, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1226, in init
self._traceback = _extract_stack()
NotFoundError (see above for traceback): ./VOC2007/voc_2007_test.tfrecord
[[Node: pascalvoc_2007_data_provider/parallel_read/ReaderReadV2 = ReaderReadV2[_device="/job:localhost/replica:0/task:0/cpu:0"](pascalvoc_2007_data_provider/parallel_read/TFRecordReaderV2, pascalvoc_2007_data_provider/parallel_read/filenames)]]