Comments (10)
Hey @zdoek001! I'm a bot here to help you out with any bugs, questions, or contributions you have. Let's tackle this issue together while we wait for a human maintainer.
To create a black and white mask of a face without optimizing it, you can use the mediapipe_face_mesh
function. This function processes an image to detect face landmarks and creates a mask where the face area is white and the rest is black.
Here is the relevant code:
def mediapipe_face_mesh(
image: Image.Image, confidence: float = 0.3
) -> PredictOutput[int]:
import mediapipe as mp
mp_face_mesh = mp.solutions.face_mesh
draw_util = mp.solutions.drawing_utils
drawing_styles = mp.solutions.drawing_styles
w, h = image.size
with mp_face_mesh.FaceMesh(
static_image_mode=True, max_num_faces=20, min_detection_confidence=confidence
) as face_mesh:
arr = np.array(image)
pred = face_mesh.process(arr)
if pred.multi_face_landmarks is None:
return PredictOutput()
preview = arr.copy()
masks = []
for landmarks in pred.multi_face_landmarks:
draw_util.draw_landmarks(
image=preview,
landmark_list=landmarks,
connections=mp_face_mesh.FACEMESH_TESSELATION,
landmark_drawing_spec=None,
connection_drawing_spec=drawing_styles.get_default_face_mesh_tesselation_style(),
)
points = np.array(
[[land.x * w, land.y * h] for land in landmarks.landmark], dtype=int
)
outline = cv2.convexHull(points).reshape(-1).tolist()
mask = Image.new("L", image.size, "black")
draw = ImageDraw.Draw(mask)
draw.polygon(outline, fill="white")
masks.append(mask)
bboxes = create_bbox_from_mask(masks, image.size)
preview = Image.fromarray(preview)
return PredictOutput(bboxes=bboxes, masks=masks, preview=preview)
Additionally, you can use the mask_preprocess
function from the adetailer
library to preprocess the mask. Here is an example:
from PIL import Image
from adetailer.mask import mask_preprocess
# Preprocess the mask
processed_masks = mask_preprocess(masks, kernel=0, x_offset=0, y_offset=0, merge_invert='NONE')
This function will generate a black and white mask of the face detected in the image, and you can further manipulate the mask using additional utility functions if required [1][2].
To continue talking to Dosu, mention @dosu.
from adetailer.
Yes, there is an API available for the mediapipe_face_mesh
function. This function processes an image to detect face landmarks and creates a mask where the face area is white and the rest is black.
Here is the relevant code:
def mediapipe_face_mesh(
image: Image.Image, confidence: float = 0.3
) -> PredictOutput[int]:
import mediapipe as mp
mp_face_mesh = mp.solutions.face_mesh
draw_util = mp.solutions.drawing_utils
drawing_styles = mp.solutions.drawing_styles
w, h = image.size
with mp_face_mesh.FaceMesh(
static_image_mode=True, max_num_faces=20, min_detection_confidence=confidence
) as face_mesh:
arr = np.array(image)
pred = face_mesh.process(arr)
if pred.multi_face_landmarks is None:
return PredictOutput()
preview = arr.copy()
masks = []
for landmarks in pred.multi_face_landmarks:
draw_util.draw_landmarks(
image=preview,
landmark_list=landmarks,
connections=mp_face_mesh.FACEMESH_TESSELATION,
landmark_drawing_spec=None,
connection_drawing_spec=drawing_styles.get_default_face_mesh_tesselation_style(),
)
points = np.array(
[[land.x * w, land.y * h] for land in landmarks.landmark], dtype=int
)
outline = cv2.convexHull(points).reshape(-1).tolist()
mask = Image.new("L", image.size, "black")
draw = ImageDraw.Draw(mask)
draw.polygon(outline, fill="white")
masks.append(mask)
bboxes = create_bbox_from_mask(masks, image.size)
preview = Image.fromarray(preview)
return PredictOutput(bboxes=bboxes, masks=masks, preview=preview)
This function is part of the adetailer
repository and can be used to generate face masks from images [1].
To continue talking to Dosu, mention @dosu.
from adetailer.
mediapipe_face_mesh
@dosu
Is there an API?
from adetailer.
What are you trying to get the mask image for?
from adetailer.
What are you trying to get the mask image for?
Once I have the mask image, I will put it in the mask of the img2img. When it is generated, the facial area is excluded so that it can be processed later in other developments
Is there any way to obtain this mask? Via API。@Bing-su
from adetailer.
Can't you just draw the mask image directly in img2img inpaint? I don't see the need for a mask.
from adetailer.
Can't you just draw the mask image directly in img2img inpaint? I don't see the need for a mask.
If I'm using the Web UI, that can be drawn, but I'm using the Web UI API for the request, as a step in the program, so the implementation of the mask generation needs to be automated
from adetailer.
The WebUI has this feature so you can get inpaint masks, but it's probably not possible with the API.
from adetailer.
! [chrome_xJgSkCvP8z] (https://private-user-images.githubusercontent.com/37621276/339330273-6f9b3991-5d6b-4815-b200-ede0336efb97.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTgyNzYyNTcsIm5iZiI6MTcxODI3NTk1NywicGF0aCI6Ii8zNzYyMTI3Ni8zMzkzMzAyNzMtNmY5YjM5OTEtNWQ2Yi00ODE1LWIyMDAtZWRlMDMzNmVmYjk3LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MTMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjEzVDEwNTIzN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWE3NTJkZWI3MzE2Nzg2YjUwNjllODAwY2I1ZDNlOTMwMmE5NzFjZTUxOWYwOThmYzY4Y2ZjODUyOGU3ZjEwZGUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.Lc1pCMfV4zJmKd5wZ5vU3y9vhr9PTSFxsiGk_gTh8MI) The WebUI has this feature so you can get inpaint masks, but it's probably not possible with the API.
Ok, thanks for the help, is this button in adetailer, please? Or what location? Didn't find it
from adetailer.
from adetailer.
Related Issues (20)
- [Bug]: setting live preview to 0 (real time update) breaks detection mask drawing HOT 1
- [Bug]: ADeatailer not working???? HOT 2
- [Bug]: ADtailer toolbar is not displayed even after restarting the web ui HOT 3
- [Feature Request]: Option to use hires fix settings HOT 2
- [Feature Request]: Add Use same sampler option HOT 2
- [Bug]: detection box color went from red to blue HOT 5
- [Bug]: Adetailer's inpainting conflicts with Tiled VAE HOT 2
- [Bug]: Inpainting - Use Separate Steps - not working HOT 2
- [Report]: what is going on with the output? HOT 2
- [Bug]: ADetailer act like it is Disabled HOT 5
- getting this weird error HOT 2
- [Bug]: ADetailer crashes because of a NoneType error HOT 5
- [Feature Request]: Mask in shape of the face (not square) HOT 3
- [Feature Request]: Is requiring mediapipe>=0.10.13 too strict? HOT 4
- [Bug]: When Using [SEP] in Adetailer Prompt, only 1 face's prompt is added to the image metadata HOT 4
- [Feature Request]: how to use skip_img2img in webuiapi?skip_img2img is not in args
- [Feature Request]: How to use code with prompt instead of ComfyUI and SDA1111 HOT 3
- [Bug]: ADetailer breaks Regional Prompter after the first batch HOT 4
- [Feature Request]: Option to return the detected area as a image mask HOT 1
- [Feature Request]: step1、 img2img step2、 adetailer repair the image in one step by API
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from adetailer.