Comments (7)
- Would like to ask if this could work with a data set that has no defined category (example vases could be mixed in with mugs in the same dataset)
This code provides a means for doing supervised semantic segmentation. So basically, you can use any dataset which has meshes and per edge segmentation labels for training the network. I am not sure if this is what you are getting at: but say you take the set of "aliens" (4 possible segments) and "chairs" (3 possible segments) from this repo (coseg segmentation). I provided code to train each one as a separate network. But you could train one network which could recieve chairs or aliens and predict a label from 1-7. The first 4 being aliens , and the last 3 being chairs.
- Are the number of segmented parts fixed? Or can there be a unlimited number of segments?
You can train any number of segments you like (assume you have enough data , network capacity and GPU power). The number of segments is defined by the number of classes in the 2d-cross entropy loss.
- Can it be trained to segment finer details? Example i have a wooden plank, it will be segmented into 6 different surfaces. And if the wooden plank had holes drilled, the segments would increase by 1 for each hole. (i think this term is called surface segmentation? if it exists)
Not sure I completely understood, but same thing applies as above. With enough data labels you should be able to learn to do a fine grained segmentation.
Also, make sure to check out the segmentation wiki.
from meshcnn.
Hi,
allow me to rephrase and clarify some of my questions:
Yes i can combine your set of "aliens" and "chairs" but would it still work if i do not explicitly categorize them as "aliens" and "chairs" when training/testing?. So for example when predicting an unknown (but is actually an alien) object with legs, would the legs be then be segmented as a chair's legs? what about not having a fixed set of parts? https://www.youtube.com/watch?v=6JQdNzsw7jA perhaps the first 20seconds of the video helps to bring across what i'm trying to say
in addition, for your 'aliens' data set you have labeled them 1-4 corresponding to their different parts, e.g. head, body, tail etc. am i able to train the model in such a way that it doesnt have this constraint? i can just say that this section of the alien is a part and that section is another part, and the model is able to segment the object into its different parts
from meshcnn.
Yes i can combine your set of "aliens" and "chairs" but would it still work if i do not explicitly categorize them as "aliens" and "chairs" when training/testing?.So for example when predicting an unknown (but is actually an alien) object with legs, would the legs be then be segmented as a chair's legs? what about not having a fixed set of parts? https://www.youtube.com/watch?v=6JQdNzsw7jA perhaps the first 20seconds of the video helps to bring across what i'm trying to say
So as for explicitly knowing whether it's a leg that came from an alien vs. chair, it depends how the ground truth-label is built. For example, if we said that 1-3 are the segments for the chair and then 4-7 are the alien segments, so the category of the object is implied by the segmentation labels. Since predicting segment 1-3 must mean that the object is from the class chair. However, this constraint is of course not required in test time, since the network will just predict the segment.
About grouping them together. You would have to train the network to possibly predict a max number of segments (say 10). but that doesn't mean that all the segments must be present in each mesh model. The network will predict a probability the edge belongs to a particular segmentation ID, so that can be any value (for example, for one particular mesh it could predict most of the edges segment 9 and the rest 8). There is no constraint that all segments should be present in each mesh, only that each edge should predict a probability for all segments. For segments that are not present, this probability should be close to 0.
in addition, for your 'aliens' data set you have labeled them 1-4 corresponding to their different parts, e.g. head, body, tail etc. am i able to train the model in such a way that it doesn't have this constraint? i can just say that this section of the alien is a part and that section is another part, and the model is able to segment the object into its different parts
It is technically possible, but I am not sure the meaning of such a thing. I mean, what exactly determines which part corresponds across different shapes? In the case of very similar shapes which have similar semantic parts, I think this could make sense (e.g., different legged animals, humans, etc.). Otherwise I am not sure.
All in all it seems you are looking for something unsupervised, or at least, weakly supervised. Which this code as is doesn't support. But you are welcome to take the layers we provided and build an unsupervised segmentation approach :)
from meshcnn.
I see, i kind of understand what you are trying to say.
So it would be close to impossible to label the alien model parts with 1-3 and then the chair parts with 1-4 and then throw the model a object and say hey, segment it into different parts. The part labeled '1' would have to at least have similarities across the training sets instead of it being just a generic part then.
perhaps the solution i am looking for lies in the unsupervised training. I'll check that out as well then.
Oh. 1 more question. After creating your models in blender, how did you annotate them?
Thanks for your kind reply!
from meshcnn.
I see, i kind of understand what you are trying to say.
So it would be close to impossible to label the alien model parts with 1-3 and then the chair parts with 1-4 and then throw the model a object and say hey, segment it into different parts. The part labeled '1' would have to at least have similarities across the training sets instead of it being just a generic part then.
Exactly. Another option is to use MeshCNN as a binary classifier, to predict the boundaries between segments. This should be transferable across different shapes.
Oh. 1 more question. After creating your models in blender, how did you annotate them?
Did you check out the scripts in the Segmentation Wiki?
from meshcnn.
Exactly. Another option is to use MeshCNN as a binary classifier, to predict the boundaries between segments. This should be transferable across different shapes.
That sounds like a plausible way to segment the parts that i'm looking for. Any advice on how to proceed?
from meshcnn.
I think you can try to work with the data I have first, to see if it will work first on a single shape class (i.e., just aliens). And then you can extend it to see if it will work on aliens and humans, for example. You should search for all boundary edges (edges which have different segmentation IDs in their 1-ring, called the gemm data-structure in the code).
from meshcnn.
Related Issues (20)
- assert no zero face area
- Does the class "Mesh" have an attribute about face?
- Undestanding .eseg file
- Proper TensorBoard Usage
- How to prepare data for a custom dataset?
- cannot install pytorch=1.2.0 HOT 2
- How to save a segmented (colored) mesh in .obj format?
- IndexError: index out of range
- Using my own obj files for training
- The problem when I run meshcnn classification HOT 1
- Problems with date preprocessing.
- help in figuring out how human_seg dataset was created
- Cannot load pre-trained weights
- Request for further documentation of code options on the meshcnn wiki for the Classification task
- Numpy error? HOT 2
- A small spelling mistake in the paper
- setting an array element with a sequence
- Looking for the data set of bunny.obj
- Dependencies from environment HOT 1
- AssertionError
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from meshcnn.