Comments (17)
Also, would you be interested in contributing this @old-school-kid ?
from keras-cv.
Hey @old-school-kid any chance I could get a little more context as to how people tend to use this in the CV domain? Particularly w/ deep models.
Thanks
from keras-cv.
Hi @LukeWood
SLIC isn't used in any CV model for image augmentation, afaik. But I have seen a lot of work using that as a pre-processing layer in my domain (material science) and even in medical imaging. In material science it helps in grain boundary detection and finding hairline fractures.
Do you want me to put links to some papers here using the preprocessing techniques? TIA.
from keras-cv.
That would actually be great. I'm really interested to see how it's used in both material science and medical imaging!
Thanks @old-school-kid
from keras-cv.
In Material Science
- Aluminum alloy microstructural segmentation Citation- 8
- Detection of microstructures in low carbon steel Citation- 8
- Classification of Bainitic Sub-Classes in Steel
In Medical Imaging
- A supervoxel‐based segmentation method for prostate MR images Citation- 36
- Detection and diagnosis of colitis on computed tomography using deep convolutional neural networks Citation- 40
- Segmentation and objective assessment of atrial scars Citation- 34
In Object Detection (Old methods, pre-2017)
- Salient object detection with high-level prior based on Bayesian fusion
- Salient object detection based on global multi-scale superpixel contrast
- Salient object detection method for breast tumor in ultrasound images based on absorbing Markov chain
from keras-cv.
Also, would you be interested in contributing this @old-school-kid ?
Sure, would love to.
from keras-cv.
What is the final goal in your material science domain? Semantic segmentation? Or something different?
from keras-cv.
E.g. I am taking a look at https://github.com/Scientific-Computing-Lab-NRCN/MLography but I don't know if we still need some intermediate preprocessing like SLIC before the network learning stack.
from keras-cv.
@bhack
Mostly Semantic segmentation, yes.
E.g. I am taking a look at https://github.com/Scientific-Computing-Lab-NRCN/MLography but I don't know if we still need some intermediate preprocessing like SLIC before the network learning stack.
I went through the repo. They have directly fed the image to an UNET which is fine, but generally we go for noise reduction and clustering (to separate matrix and grains) and this has proven to achieve better results.
This repo throws more light on this.
from keras-cv.
You repo is a little bit old and I am not an expert in the material science domain. But I remember that a few year ago, at ECCV 2018, there was a paper proposing a learning/differentiable approach ((currently 118 citations):
https://varunjampani.github.io/ssn/
But also if it was differentiable I don't know how much it is still popular today learning this intermediate representation.
from keras-cv.
@old-school-kid I'm fairly sure this is a good fit and is something that we'd be interested in hosting as long as the contribution is well written & maintainable.
So if you're interested please prepare a PR.
from keras-cv.
You repo is a little bit old and I am not an expert in the material science domain. But I remember that a few year ago, at ECCV 2018, there was a paper proposing a learning/differentiable approach ((currently 118 citations):
https://varunjampani.github.io/ssn/
That was a nice article. Thank you for sharing! While that can be used as a intermediate representation in end-to-end learning, SLIC is just used as a pre-processing layer and in no way it is fused with learning. But the article you shared looks promising too.
But also if it was differentiable I don't know how much it is still popular today learning this intermediate representation.
The papers I have shared above under Material Science are from 2019, 2020, 2021. Moreover apart from research, it is used in industry as it can easily segment matrix in microstructures.
from keras-cv.
Moreover apart from research, it is used in industry as it can easily segment matrix in microstructures
I think that this part could be a little bit out of the scope if not strictly functional to the learning step for this specific library.
But I still think that it is valid for a generic CV library (e.g. PIL, scikit image, Opencv, etc..).
from keras-cv.
This is in scope. The real question is whether or not the impact is high enough to justify maintaining the layer.
Medical image segmentation is an important domain with a lot of promise. SLIC is clearly showing promise there, and in other image segmentation areas.
This is especially in scope if the SLIC layer is differentiable. Then we can also use it as a preprocessing layer, or as it's used in https://varunjampani.github.io/ssn/.
The cost of maintaining a specialized layer like this is pretty low. So I am comfortable accepting it if @old-school-kid is willing to contribute a well refined implementation. If you do decide to work on it, feel free to add me as the reviewer on the PR 👍 .
from keras-cv.
In the last part of @old-school-kid comment It was quite clear that he mentioned that It Is useful also as is, without considering if It is useful or not in the learning pipeline.
For that specific part of the comment I think, but I could be wrong as I don't fully understand the Keras-cv policy, that we are not going to collect CV operations as is if they are not functional to the learning step or to populate metrics/visualizations.
Also more in general what I claimed Is that I think, just by a popularity/resources ratio point of view that this intermediate rappresentation, also when differentiable, It isn't not so popular anymore.
Instead if we are evaluating the non differentiable version I think that It could be useful to review this as a baseline in the context of modern proposed solutions for unsupervised/self-supervised/semi-supervised image segmentation papers (e.g. https://arxiv.org/abs/2007.09990 and other works).
from keras-cv.
Implementation Ref.
https://albumentations.ai/docs/api_reference/augmentations/transforms/#albumentations.augmentations.transforms.Superpixels
from keras-cv.
Lets close this our as stale for now until theres a strong use case. If we end up trying to tackle a segmentation competition and can't compete without this, we should reprioritize.
from keras-cv.
Related Issues (20)
- Wrong bounding boxes in the visualization of `tfds.datasets.kitti` HOT 2
- Error when training YOLOV8 with jax backend HOT 10
- Bad YOLOv8 performance when compared to Ultralytics implementation HOT 6
- Using large dataset from a TF record the model doesnt train anything HOT 3
- How to contribute pretrained models? HOT 2
- Add DINOv2 HOT 4
- take over video swin checkpoints HOT 11
- FasterRCNN and ROIAlign are Non-Functional in Master HOT 1
- DropBlock2D not working with TensorFlow backend in graph mode on Keras 3 HOT 1
- Detecting more than 100 objects in a single image using retinanet HOT 1
- Image Classifier Task Can't Export to Tensorflow saved model format or to TFLite HOT 6
- Can't Select Activation / Output for DeepNetV3Plus, broken example for 1 class
- How could I segment object without need to resize the whole image for it
- Attempting to load SegFormer preset "segformer_b0_imagenet" results in 403 error
- Inference on ONNX YOLOv8 model HOT 2
- YOLOV8Backbone inconsistent output tensor shapes on Torch backend
- Fine tuned model unable to detect objects HOT 1
- Issues I am Facing while training KerasCV YOLO model on my own dataset HOT 2
- Update Keras-CV object detection tutorial from Keras 2 to Keras 3 for new YOLO version (at least YOLOv9 or newer) HOT 4
- StableDiffusion.text_to_image() casuses an excaption in Colab HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from keras-cv.