Hi,
I have managed to succesfully setup the code and the dependencies on my machine. I was also successfully able to run the command
python src/experiment_deblurring.py print_config
But when I move on to the next step and try to run this command
python src/experiment_deblurring.py with experiment_name=experiment_a image_name=barbara elemental.optimal_DNCNN_experiment_a
I get an error
DataLossError (see above for traceback): not an sstable (bad magic number)
[[Node: save/RestoreV2_4 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_4/tensor_names, save/RestoreV2_4/shape_and_slices)]]
[[Node: save/RestoreV2_19/_63 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_134_save/RestoreV2_19", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"]()]]
Upon some googling I found out that this error can be a result of the model file not being compatible with my system. I am currently using tensorflow 1.3.0 for the GPU as suggested in the readme. What should I do?
The full stacktrace of the error message is posted below:
2018-04-09 17:11:20.068781: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating
TensorFlow device (/gpu:0) -> (device: 0, name: TITAN X (Pascal), pci bus id: 0000:01:00.0)
INFO:tensorflow:Restoring parameters from models/DNCNN__gaussian_0.02__40-40-
1__128/model.ckpt
INFO - tensorflow - Restoring parameters from models/DNCNN__gaussian_0.02__40-40-
1__128/model.ckpt
2018-04-09 17:11:20.266064: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.266064: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.266119: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.266084: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.266242: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.266339: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.266498: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.266534: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.266666: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.266672: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.266690: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.267034: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.267252: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.267297: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.267314: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.267884: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.268141: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.268219: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.268291: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.268315: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.268376: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.268515: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.268811: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.268954: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.268967: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.268997: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.269120: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.269249: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.269332: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.269358: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.269375: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.269409: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.269562: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
2018-04-09 17:11:20.269663: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: not an sstable (bad magic number)
ERROR - deblurring - Failed after 0:00:04!
Traceback (most recent calls WITHOUT Sacred internals):
File "src/experiment_deblurring.py", line 365, in main
cnn_func = init_cnn_func() if elemental['denoising_prior'] == 'CNN' else None
File "/home/uzair/Denoising_Raeid/learn_prox_ops/src/experiment_ingredients.py", line 185, in init_cnn_func
nn_deployer = Deployer(FLAGS)
File "/home/uzair/Denoising_Raeid/learn_prox_ops/src/tf_solver.py", line 77, in init
saver.restore(self.sess, opt.model_path)
File "/home/uzair/.local/lib/python3.4/site-packages/tensorflow/python/training/saver.py", line 1560, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/home/uzair/.local/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 895, in run
run_metadata_ptr)
File "/home/uzair/.local/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 1124, in _run
feed_dict_tensor, options, run_metadata)
File "/home/uzair/.local/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 1321, in _do_run
options, run_metadata)
File "/home/uzair/.local/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 1340, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.DataLossError: not an sstable (bad magic number)
[[Node: save/RestoreV2_4 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_4/tensor_names, save/RestoreV2_4/shape_and_slices)]]
[[Node: save/RestoreV2_19/_63 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_134_save/RestoreV2_19", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"]]
Caused by op 'save/RestoreV2_4', defined at:
File "src/experiment_deblurring.py", line 344, in
@ex.automain
File "/home/uzair/.local/lib/python3.4/site-packages/sacred/experiment.py", line 130, in automain
self.run_commandline()
File "/home/uzair/.local/lib/python3.4/site-packages/sacred/experiment.py", line 242, in run_commandline
return self.run(cmd_name, config_updates, named_configs, {}, args)
File "/home/uzair/.local/lib/python3.4/site-packages/sacred/experiment.py", line 187, in run
run()
File "/home/uzair/.local/lib/python3.4/site-packages/sacred/run.py", line 223, in call
self.result = self.main_function(*args)
File "/home/uzair/.local/lib/python3.4/site-packages/sacred/config/captured_function.py", line 47, in captured_function
result = wrapped(*args, **kwargs)
File "src/experiment_deblurring.py", line 365, in main
cnn_func = init_cnn_func() if elemental['denoising_prior'] == 'CNN' else None
File "/home/uzair/.local/lib/python3.4/site-packages/sacred/config/captured_function.py", line 47, in captured_function
result = wrapped(*args, **kwargs)
File "/home/uzair/Denoising_Raeid/learn_prox_ops/src/experiment_ingredients.py", line 185, in init_cnn_func
nn_deployer = Deployer(FLAGS)
File "/home/uzair/Denoising_Raeid/learn_prox_ops/src/tf_solver.py", line 76, in init
saver = tf.train.Saver()
File "/home/uzair/.local/lib/python3.4/site-packages/tensorflow/python/training/saver.py", line 1140, in init
self.build()
File "/home/uzair/.local/lib/python3.4/site-packages/tensorflow/python/training/saver.py", line 1172, in build
filename=self._filename)
File "/home/uzair/.local/lib/python3.4/site-packages/tensorflow/python/training/saver.py", line 688, in build
restore_sequentially, reshape)
File "/home/uzair/.local/lib/python3.4/site-packages/tensorflow/python/training/saver.py", line 407, in _AddRestoreOps
tensors = self.restore_op(filename_tensor, saveable, preferred_shard)
File "/home/uzair/.local/lib/python3.4/site-packages/tensorflow/python/training/saver.py", line 247, in restore_op
[spec.tensor.dtype])[0])
File "/home/uzair/.local/lib/python3.4/site-packages/tensorflow/python/ops/gen_io_ops.py", line 663, in restore_v2
dtypes=dtypes, name=name)
File "/home/uzair/.local/lib/python3.4/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
op_def=op_def)
File "/home/uzair/.local/lib/python3.4/site-packages/tensorflow/python/framework/ops.py", line 2630, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/home/uzair/.local/lib/python3.4/site-packages/tensorflow/python/framework/ops.py", line 1204, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
DataLossError (see above for traceback): not an sstable (bad magic number)
[[Node: save/RestoreV2_4 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_save/Const_0_0, save/RestoreV2_4/tensor_names, save/RestoreV2_4/shape_and_slices)]]
[[Node: save/RestoreV2_19/_63 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_134_save/RestoreV2_19", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"]]