Comments (8)
Hi,
1- I haven't seen this error, sorry.
2- Those files just summarize whatever is in the test_####k folders. You should have 1 folder per checkpoint saved. It looks like in your case, you only have the 10000 iteration saved or available. This summary is only done for the "out2*" files (meaning per slide averages), for first 3 labels, macro and micro averages. They show the AUC and CIs. It's just a summary but you can change the code easily if you need to extract other info. Labels "1", "2" and "3" would be whatever order was in your "sorted" folder (before conversion to TFRecord), in alphabetical order.
3- These are the probabilities generated for each tile. You have the tile name and in the brackets, the probabilities generated for the 3 classes (you have an additional class (the first one) called the "background" class which is part of the original inception). The number after the bracket is the corrected probability (ignoring the background class) for the expected label.
HTH
N.
from deeppath.
Thanks.
Bill
from deeppath.
The error that I described above is fixed by changing line 66 in nc_inception_eval.py from:
tf.initialize_all_variables()
to
tf.global_variables_initializer()
ref. here. initialize_all_variables() is deprecated and apparently is broken in newer tensorflow releases.
Bill
from deeppath.
Thanks for the follow up, Bill.
from deeppath.
I am encountering additional errors in attempting to train for mutation classification using ...01_training/xClasses/bazel-bin/inception/imagenet_train (output below). I am presuming it is a similar deprecation issue. I'm running tensorflow-gpu 1.13.1 for compatibility with the latest CUDA. I encountered many issues trying to install earlier CUDA versions.
It appears to not like the labels component, but they look OK to me. Here is an example:
feature {
key: "image/class/label"
value {
int64_list {
value: 0
value: 1
value: 0
value: 0
value: 0
value: 0
value: 0
value: 0
value: 0
value: 1
value: 1
}
}
}
Do I, perhaps, build inception differently for multiclass? If you have any suggestions, would appreciate your help.
Thanks,
Bill
Partial output from failing multiclass training:
WARNING:tensorflow:From /home/bcliffor/DeepPATH/DeepPATH_code/01_training/xClasses/bazel-bin/inception/imagenet_train.runfiles/inception/inception/inception_train.py:339: start_queue_runners (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version. Instructions for updating: To construct input pipelines, use the
tf.data` module.
2019-04-29 16:14:14.696263: I tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library libcublas.so.10.0 locally
2019-04-29 16:14:24.091919: W tensorflow/core/framework/op_kernel.cc:1401] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: image/class/label. Can't parse serialized Example.
2019-04-29 16:14:24.091920: W tensorflow/core/framework/op_kernel.cc:1401] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: image/class/label. Can't parse serialized Example.
2019-04-29 16:14:24.091920: W tensorflow/core/framework/op_kernel.cc:1401] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: image/class/label. Can't parse serialized Example.
2019-04-29 16:14:24.091935: W tensorflow/core/framework/op_kernel.cc:1401] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: image/class/label. Can't parse serialized Example.
ERROR:tensorflow:Exception in QueueRunner: Key: image/class/label. Can't parse serialized Example.
[[{{node batch_processing/ParseSingleExample_1/ParseSingleExample}}]]
ERROR:tensorflow:Exception in QueueRunner: Key: image/class/label. Can't parse serialized Example.
[[{{node batch_processing/ParseSingleExample/ParseSingleExample}}]]
ERROR:tensorflow:Exception in QueueRunner: Key: image/class/label. Can't parse serialized Example.
[[{{node batch_processing/ParseSingleExample_2/ParseSingleExample}}]]
ERROR:tensorflow:Exception in QueueRunner: Key: image/class/label. Can't parse serialized Example.
[[{{node batch_processing/ParseSingleExample_3/ParseSingleExample}}]]
Exception in thread QueueRunnerThread-batch_processing/batch_join/fifo_queue-batch_processing/batch_join/fifo_queue_enqueue_3:
Traceback (most recent call last):
File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
self.run()
File "/usr/lib/python3.5/threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
File "/home/bcliffor/dp/lib/python3.5/site-packages/tensorflow/python/training/queue_runner_impl.py", line 257, in _run
enqueue_callable()
File "/home/bcliffor/dp/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1257, in _single_operation_run
self._call_tf_sessionrun(None, {}, [], target_list, None)
File "/home/bcliffor/dp/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Key: image/class/label. Can't parse serialized Example.
[[{{node batch_processing/ParseSingleExample_3/ParseSingleExample}}]]
Exception in thread QueueRunnerThread-batch_processing/batch_join/fifo_queue-batch_processing/batch_join/fifo_queue_enqueue_1:
Traceback (most recent call last):
File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
self.run()
File "/usr/lib/python3.5/threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
File "/home/bcliffor/dp/lib/python3.5/site-packages/tensorflow/python/training/queue_runner_impl.py", line 257, in _run
enqueue_callable()
File "/home/bcliffor/dp/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1257, in _single_operation_run
self._call_tf_sessionrun(None, {}, [], target_list, None)
File "/home/bcliffor/dp/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Key: image/class/label. Can't parse serialized Example.
[[{{node batch_processing/ParseSingleExample_1/ParseSingleExample}}]]`
from deeppath.
Hi,
There is an extra "background" class in inception, so an additional one will be automatically added during the processing (you can check the original github page: https://github.com/tensorflow/models/blob/master/research/inception/README.md)
Just be careful also when you convert to TFRecord. You can double check everything is fine with the name of the test TFRecord files. the "true" label appears as a binary at the end of the file name. If you have 10 labels, the label text file must have exactly 10 lines (no empty line at the top or at the bottom)
HTH,
Best,
Nicolas
from deeppath.
from deeppath.
Hi - Glad this is solved.
I indeed used "multiClasses/nc_imagenet_eval.py" originally, but the "xClasses/nc_imagenet_eval.py" should work too (if you use that one, you would need to add the " --mode='1_sigmoid'" option).
At some I will probably delete the "multiClasses/nc_imagenet_eval.py" from the code and just keep the second xClass version.
Best,
Nicolas
from deeppath.
Related Issues (20)
- train/test/folder image structure for fine-tuning via Inception V3 for mutation classification HOT 1
- Which exact WSIs are selected for EGFR prediction? HOT 4
- About Transfer Learning - Frozen weights?
- Exact way of doing tile aggregation by averaging output probability HOT 1
- mrxs images HOT 1
- stuck at 0b_tileLoop_deepzoom6.py when tiles a very large tiff image. HOT 2
- Clarification about Labels HOT 2
- normalize tile HOT 1
- How to compute the loss HOT 1
- How to specify certain slides in a split HOT 2
- Multi-label mutation data splits
- In Pytorch
- How can I use my data to build my json file?
- Use 0f_ HeatMap_ nClasses_ Overall.py has no result output
- Questions about cv2.findContours() HOT 1
- Color classification problem in 3-Analyze the outcome HOT 7
- Multiple Instance Learning HOT 1
- Reference image for Vahadane normalization HOT 2
- JSON file HOT 2
- Issue running bazel
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deeppath.