Use pychubby in android using Chaquopy sdk about pychubby HOT 21 CLOSED

@bharatkhadodra1996 thank you for your interest!

So full transparency, I have no experience with Chaquopy. However, you can access new_lf.img and it will give you the actual image as a numpy array. Hope that helps!

from pychubby.

Ok so I tried your given solution. I converted the new_lf.img to byte_array and passed it to my android file. When I try to create a bitmap with that byte array, null bitmap is created. So there maybe some issue with byte_array.

from pychubby.

Hmm, again, I am not an expert on Chaquopy. Maybe consider creating an issue in their issue tracker: https://github.com/chaquo/chaquopy/issues and ask how to work with images that are stored in a numpy array.

from pychubby.

Ok I solve that issue by saving file to device. One more issue I faced is my app crash because PyChubby is not able to handle some images. Maybe Dlib is not able to detecting the face or something I do not know the clear reason because there is no error log message about this. So can you tell me what can be the reason and how can I identify the error ?

from pychubby.

I think one possible reason is that the image is too big. Try resizing it (making it smaller)

let me know if it helps:)

@bharatkhadodra1996

Another possibility is that it did not find any face in the image and then this exception would be raised
https://github.com/jankrepl/pychubby/blob/master/pychubby/detect.py#L221-L222

from pychubby.

Image is not too big size, its around 100 kb.
I traced my code and I found that below line causing an error.
lf = LandmarkFace.estimate(img, model_path=None, n_upsamples=1, allow_multiple=False)

This is the error I got = There are some duplicates.
Below is the image I am using.

I do not understand image is very clear and it is very easy to identify face landmarks then why it is throwing such error.

from pychubby.

Hmm, interesting.

This is the part of the code that raises that exception:

It basically means that there are duplicate landmark points detected by dlib. Honestly, I don't remember whether this would lead to some crazy problems inside of pychubby if it is just one or two duplicates. However, maybe dlib just failed to do the detection correctly and there are way more duplicates. But what you can do is to

• clone the repo and cd to it
• pip install -e . (editable installation
• comment out those lines and just see whether something breaks + inspect what the 68 detected landmarks look like lf.points

Hope that helps

from pychubby.

I actually just checked and it seems like there is just a single duplicate landmark in your image. So I guess in this specific case it would make sense to not raise the ValueError. Again, feel free to adjust it.

I am a bit afraid to allow for duplicates in general because if there are 2 duplicate landmark points and there is an action that moves each of the duplicates to a new position that is different then it will basically become "unsolvable". Hope that makes sense:)

from pychubby.

Okay so basically you want me to clone pychubby project to my account. Comment the lines in detect.py file and then use it.
Is that correct ?

from pychubby.

Okay so basically you want me to clone pychubby project to my account. Comment the lines in detect.py file and then use it. Is that correct ?

yes:)

from pychubby.

Ok so I forked this project and made necessary changes. But I do not know how to use it now ? I mean I was writing pip install pychubby for the original project. So what I need to write to use this forked project ?

from pychubby.

https://packaging.python.org/en/latest/tutorials/installing-packages/#installing-from-a-local-src-tree

from pychubby.

Thank you for the response. It helped me a lot in my project. After completing all the process, my application size is around 200 MB. From 200 MB, 150+ MB is occupied by PyChubby. So Do you have any idea if I want to reduce the size or if I can use lite version of PyChubby ?

from pychubby.

Thank you for the response. It helped me a lot in my project. After completing all the process, my application size is around 200 MB. From 200 MB, 150+ MB is occupied by PyChubby. So Do you have any idea if I want to reduce the size or if I can use lite version of PyChubby ?

So I would guess the space is taken by your dependencies (things you have in your virtual environment)
So you would probably have to remove some dependencies (e.g. matplotlib) and make sure to modify the source code of pychubby accordingly - assuming your project does not really need them. Not trivial.

Also, a completely different paradigm would be to write a HTTP server around pychubby and make it possible to send requests to it. You can host it on the cloud. This way the actual processing would be done on the server rather than on the cellphone. However, that would be a lot of work and it would require a lot of domain knowledge. Also maybe you would not wanna go for this solution since it would require constant internet connection.

from pychubby.

Thank you for the response. It helped me a lot in my project. After completing all the process, my application size is around 200 MB. From 200 MB, 150+ MB is occupied by PyChubby. So Do you have any idea if I want to reduce the size or if I can use lite version of PyChubby ?

So I would guess the space is taken by your dependencies (things you have in your virtual environment) So you would probably have to remove some dependencies (e.g. matplotlib) and make sure to modify the source code of pychubby accordingly - assuming your project does not really need them. Not trivial.

Also, a completely different paradigm would be to write a HTTP server around pychubby and make it possible to send requests to it. You can host it on the cloud. This way the actual processing would be done on the server rather than on the cellphone. However, that would be a lot of work and it would require a lot of domain knowledge. Also maybe you would not wanna go for this solution since it would require constant internet connection.

Ok I will try this solution once my application is completed.
One thing I noticed that when I compress my JPG file, its face landmark points get changed. That's why I got the previous error "There are some duplicates".
If I do not compress the image file, application takes too much time to process and sometimes crashes. So do you have any idea or solution how this situation can be handled ?

from pychubby.

@bharatkhadodra1996 I think you just need to find the right balance between speed and quality. The bigger the image the slower PyChubby is going to be. So try to find the image resolution that represents the right tradeoff for you

from pychubby.

Yes that's right. Currently I am working on it.
I want to add one more feature to this project which is currently not available in PyChubby.
This feature will whiten teeth part. So do you have any plan to add such feature ?

from pychubby.

The teeth whitening is definitely not something PyChubby can do (it just does warping). I would encourage you to look into generative models that could take your original image + prompt and generate a new one. Or a bit more oldschool approach would be using semantic segmentation to find the pixels that correspond to teeth and then manually overwrite those pixels:D

from pychubby.

@jankrepl I was thinking that I am getting the landmarks of face. So it is easy to recognize the teeth area using those landmark points. So can I do something with that in python ?

from pychubby.

Well, getting the mouth landmarks will still leave you with the problem of trying to segment things like gums, lips, teeth...

Anyway, as mentioned above this is a problem unrelated to puchubby. I suggest we close this thread and in case you have another specific question related to pychubby feel free to open a new issue:)

Good luck @bharatkhadodra1996

from pychubby.

Ok sure. And thank you for all the help. @jankrepl

from pychubby.