Giter Club home page Giter Club logo

image_restoration's Introduction

Image Restoration: denoising and splitting

Welcome to the Image Restoration exercises. In this part of the course, we will explore how to use deep learning to denoise images, with examples of widely used algorithm for both supervised and unsupervised denoising. We will also explore the difference between unstructured and structured noise, or between UNet (which you are familiar with by now) and VAE architectures (see COSDD exercise)!

Finally, we have bonus exercises for those wanted to explore more denoising algorithms or image splitting!

Setup

Please run the setup script to create the environment for these exercises and download data.

source setup.sh

Exercises

  1. Context-aware restoration
  2. Noise2Void
  3. Correlated and Signal Dependent Denoising (COSDD)
  4. DenoiSplit

Bonus

image_restoration's People

Contributors

neptunes5thmoon avatar cateek avatar ben-salmon avatar msschwartz21 avatar fjug avatar funkey avatar

image_restoration's Issues

Feedback from Morgan and Diane

In notebook 3, I remember the students struggling a bit with the model configuration task since there are so many possible parameters. Can we give them a list of the essential parameters that they need to consider and focus on?


Universal tasks

<div class="alert alert-danger">
Please use kernel <kernel name here>
</div>
  • Check for any references to slack. We are going to use Element this year.

Feedback

Hi Larissa @neptunes5thmoon,

Great notebooks! Very minor nitpicking feedback:

exercise1.ipynb

  1. Small typo in acquisitions in Here the high SNR images are acquistions of Human U2OS cells taken
  2. In the line here

"Since the image pairs were synthetically created in this example, they are already aligned perfectly.",

one could add an extra line saying that ...

"for real data, usually, low and high image pairs are not pixel-level aligned."

  1. Small typo in CSBDeep in

"To train a denoising network we will use the CSB Deep Repo"

  1. Small typo. Instead of:

"This part will no tasks to fill in in this exercise",

we can say:

"This part will have no tasks to fill in, but go through ..."

  1. The data comes from the BBBC dataset as mentioned in the introduction, but then why do we say later that the data is Tribolium?

" First we download some example data, consisting of low-SNR and high-SNR 3D images of Tribolium"

  1. Maybe since the dataset which is downloaded is 277 MiB and could take time to download, we could add a line to the order of:

"... Please wait for a few minutes for the data to be downloaded."

  1. Could change:
    print("low image size =", y.shape)
    to
    print("low SNR image size =", y.shape)
  2. Could explain the generated graph in:

"Visualize results Plot the test stack pair and the predicted restored stack (middle)."

  1. Could change the italics formatting from:
    This is _not_ done in this notebook
    to This is <i>not</i> done in this notebook

exercise2.ipynb

  1. Typo in microsecond:

"As you can see the last image, which is the average of 4 images with 5$\mu s$ scantime"

improvment ideas from 2022 edition

Timing

CARE notebook: ~90 min till we went over questions (cause it was lunchtime) ca. 10:30-12:00
N2N notebook: ~90 min (ca. 15:30-17:00) for students to work on, ca. 30 min to discuss in the group (3 students had finished the notebook at that point)
N2V notebook: ~30 min (ca.17:30-18:00), then discuss solutions in ca 5 min

Problems/Ideas for improvement:

  • Small bug: logdir needs to exist for tensorboard to work
  • some students had other notebooks from previous exercises running and then got cryptic OOM errors
  • for N2N: students struggled with understanding that the SEM data comes as one tiff stack of several images and then should be resaved as single images for training
  • N2N: clarify difference between model.predict and model.keras_model.predict (model.predict under the hood splits the image into patches to predict on, maybe more)
  • N2V: in addition to the assert that the training/test data is there, check that the folders do not contain other tif files - or switch implementation to load_imgs
  • useful addition would be a task to look at patch size vs receptive field size

2024 Review

Finished reviewing (looked at ben/denoisplit as per mattermost).
01 and 04 need some work I think but for the other ones only minor comments:

Installation

  • end up in a different directory after running source setup.sh
  • recommended is putting data on our s3 bucket and download from there
  • dependencies of denoiSplit not installed

01_CARE

  • missing colored boxes for checkpoints, exercises etc. in the exercise notebook
  • still got TODOs in there (most importantly: missing scaffolding for some tasks)
  • import for augment_batch and normalize is at the very beginning and defined by you. Maybe add a comment that makes clear that it's provided by you and add a cell with ?augment_batch and ?normalize at task 1, similarly for denormalize in task 3
  • agree that calculation of means and stds should go in the function
  • they already know about data loaders in principle
  • the unet should come from here: https://github.com/dlmbl/dlmbl-unet
  • IMO importing torch.nn and torch.optim would be better than importing with as nn
  • I got NaNs in my loss when my lr was too high
  • something with the training is not working for me - validation loss is always the same

02_Noise2Void

  • config explanation: I think axes being "YX" warrants more explanation (e.g. they might get 2D but be thrown for a loop by it not being "XY" )
  • getting some warnings during training
  • Task2: tensorboard is already started
  • in export: TypeError: CAREamist.export_to_bmz() got an unexpected keyword argument 'path' (parameter is called path_to_archive)
  • Task 5: detail but data is already downloaded, so change text to "loaded in the following cell"

03_COSDD

  • put explanation for dimensions before or in the task instead of after
  • still has two TODOs (figure and pretrained model)
  • in tensorboard metrics: 20 min in text instead of 10 min
  • if you can add tensorboard logs for the pretrained models that'd be great
  • Bonus
    • import is wrong
    • also still TODO: add pretrained
    • device is not defined

04_DenoiSplit

  • sys path extension isn't right
  • missing dependency albumentations
  • I assume this isn't done? Very light on details of what's happening
  • exercise version has answers to questions

05 Noise2Noise

  • text: "regression exercises" -> restoration exercises
  • Task1: typo vizualize -> visualize

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.