If folder names include Japanese language, an abnormal termination with the following msg occurs when 2D CNN training is executed (ver 0.82).

{'Image Folder': 'C:/Users/urakubo/Desktop/Dojo_Devel/Example2CNNテスト/TrainingImages', 'Segmentation Folder': 'C:/Users/urakubo/Desktop/Dojo_Devel/Example2CNNテスト/GroundTruth', 'Model Folder': 'C:\Users\urakubo\Desktop\UNI-EM0.82_CPU\data\model_tensorflow', 'Batch Size': '1', 'Loss Function': 'softmax', 'Maximal Epochs': '2000', 'Display Frequency': '200', 'Augmentation': 'fliplr, flipud, transpose', 'Save Parameters': 'C:\Users\urakubo\Desktop\UNI-EM0.82_CPU\data\parameters\Training_2D.pickle', 'Load Parameters': 'C:\Users\urakubo\Desktop\UNI-EM0.82_CPU\data\parameters\Training_2D.pickle', 'U depth': '8', 'N res blocks': '9', 'N highway units': '9', 'N dense blocks': '5', 'N dense layers': '5', 'Network': 'unet'}
Target file to check color type : C:/Users/urakubo/Desktop/Dojo_Devel/Example2CNNテスト/TrainingImages\0049.png
Traceback (most recent call last):
File "UNI-EM\segment_2D_DNN\TrainingTab.py", line 192, in _ExecuteTraining
File "UNI-EM\segment_2D_DNN\ExecuteTraining.py", line 58, in init
AttributeError: 'NoneType' object has no attribute 'shape'

Dojo rarely fails to appear in the built-in web browser. This is a problem of the web browser, but not Dojo itself. This seems to occur under windows 10 only in GPU-equipped PCs but either live python or standalone versions. We have not found how to suppress it fundamentally. Below the movie is shown.

Successful launch of Dojo
https://www.dropbox.com/s/ta9q9vc9cckincu/ice_video_20190629-185924.webm?dl=0

Launch failure of Dojo in the built-in web browser
https://www.dropbox.com/s/aal44l598u33tt0/ice_video_20190629-185604.webm?dl=0

If problem occurs, please remove cache files in the following folder at first:
C:\Users%USERNAME%\AppData\Local\main

Once Dojo launches under the "gpu-disable" mode of Chromium, this failure disappears even after we comment out the correspondent command (below).

main.py, L: 23 (Ver 0.84)

os.environ['QTWEBENGINE_CHROMIUM_FLAGS'] = '--disable-gpu'

system folder

• Credit.py
• DojoMenu.py
• FileMenu.py
• MainWindow.py
• Script.py
• func_persephonep.py
• FileSystem.py ... FileManager.py
• Params.py

dojoio folder

• DialogOpenDojoFolder.py
• DojoFileIO.py
• ExportIdDialog.py
• ExportImageDialog.py
• DB.py
• DialogGenerateDojoFolder.py
• ExportImgSeg.py
• ImportImgSeg.py
• SaveChanges.py
• np2imgo.py
• np2sego.py
• np2sego_dummy.py

train.zip

Hi,

I try to train a new network, however Images and Mask do not match (sorting error?). So it takes the wrong mask for the image.
See zip file.

Best, Ron

2. The 2D CNNs now accepts png/jpg images. In future, I will extend
   UNI-EM to accept multipage tiff images and mp4 videos.
   I found the "load_images" function in translate.py (line 262), and I
   will modify some around it.
   Is this right? Do you have any suggestion?

This is the right place!
The current file i/o is implemented in tensorflow and uses "lazy execution". Because GPU memory (and CPU memory) is typically smaller than a typical image dataset, it does not read all images at once, but rather the current training batch. Of course, it will buffer them in CPU memory if possible. The function tf.image.decode_jpeg and everything that follows will define a computation graph and handle all that buffering and you will not experience a memory full error, however it has the disadvantage, that you can only read RGB (3 channel) or maybe RGBT (4 channel) images. And to my knowledge there is no implementation of a TIFF or MOV reader in tensorflow. In that case one has to rewrite the whole load_images procedure (might be a bit difficult to do) or write a tool that converts tiff and mov to png (easier). But that is just my opinion.

Could the trained weights of FFN (using SNEMI3D dataset) be shared? It is taking too long for training.
Thanks!

Image folder dialog is launched with a delay if it has a large number of thumbnail images.
Generation of the thumbnail images should run in another thread.

In the english verison of "Example workflow 2: Membrane segmentation using 3D Flood Filling Networks (FFN)"
In the "Training" section, it says :
"Check "Sparse Z" if the z-pitch (nm/pixel) of the EM image is smaller than the xy-pitch (nm/pixel)."
However in the Japanese version it says to check "Sparse Z" if the z-pitch is greater than the xy-pitch. We think the english version should be changed.

Dojo adjust func should have a function to fill enclosed areas.

User cannot select a target image folder through a "Open folder" dialog if it has already been open.

The folder management system should also manage Tensorflow model folder and ffn folder.

FFN inference got shut down if "ffn/0/0/seg-0_0_0*" files exist in Linux.

・Inference時、ffn/0/0/　にseg-0_0_0.・・・ファイルが存在すると
プログラムが落ちます。消去するか、名前を変更すれば問題ないように思います。

1. I tested your CNNs with many EM images, and the error below appears
   in the inference process if a 511x511 size test image is given, even
   if the resnet was trained with a pair of 511x511 size images.
   I feel, there is a bit complicated rule beyond the even/odd pixel number.
   I would ask your suggestion about the size restriction. I should at
   least describe it in the github.

I found that in the current implementation (at least for unet) the size for the input/output of the CNN must be exponent of 2, e.g. 128, 256, 512, 1024... otherwise the up- and down-sampling layers mismatch. During training you can use any image size, because the augmentation procedure cut out squares of a fixed size (patch_size=256, see line 26 in https://github.com/tbullmann/imagesegmentation-tensorflow/blob/master/translate.py ). During test/prediction the full size of the image is used, and the size of the CNN is scaled accordingly. Therefore, in the current implementation the images must be have width/height of an exponent of 2, e.g. 128, 256, 512, 1024...
A solution would be to extend images to the next largest exponent of 2, e.g. images of 200x500 to an image 256x512) by mirroring the content and then crop the output of the CNN to the original size.