de-code / python-tf-bodypix Goto Github PK
View Code? Open in Web Editor NEWA Python implementation of the bodypix model.
License: MIT License
A Python implementation of the bodypix model.
License: MIT License
This app is really cool. It's got 100% of the functionality I need on Linux to replicate Zoom's BG subtraction on Mac/Win.
Now, I want a feature where the face mask is used to crop the output image from ReplaceBackgroundApp.get_output_image() and then rescale that to the output image size.
So the face would be "tracked" regardless of where it goes in the frame ala AutoFrame from Nvidia (https://youtu.be/GRFjfGH87Dk?t=55)
Assign this to me
Hi,
On the bodypix github page, the javascript bodypix version shows a keypoint skeleton which seems to be an allPoses field returned from the segmentPerson() call. Is it possible to get these pose locations from the python api? (ie a list of keypoints with their (x,y) locations)
Hello,
When I change the model to MOBILENET_FLOAT_100_STRIDE_8, I received the error below. I only used the sample code in the repo and only changed the model. The MOBILENET_FLOAT_75_STRIDE_16 and MOBILENET_FLOAT_50_STRIDE_16 were working. I appreciate your help in advance.
Users/takas/bodyPartSeg/bodypix/main.py
Traceback (most recent call last):
File "c:\Users\takas\bodyPartSeg\bodypix\main.py", line 23, in <module>
result = bodypix_model.predict_single(image_array)
File "C:\Users\takas\bodyPartSeg\bodypix\bodypix\lib\site-packages\tf_bodypix\model.py", line 437, in predict_single
tensor_map = self.predict_fn(model_input_image)
File "C:\Users\takas\bodyPartSeg\bodypix\bodypix\lib\site-packages\tf_bodypix\model.py", line 134, in __call__
return self.predict_fn(
File "C:\Users\takas\bodyPartSeg\bodypix\bodypix\lib\site-packages\tf_bodypix\model.py", line 87, in __call__
result = self.wrapped(*args, **kwargs)
File "C:\Users\takas\bodyPartSeg\bodypix\bodypix\lib\site-packages\tensorflow\python\eager\polymorphic_function\monomorphic_function.py", line 1474, in __call__
return self._call_impl(args, kwargs)
File "C:\Users\takas\bodyPartSeg\bodypix\bodypix\lib\site-packages\tensorflow\python\eager\wrap_function.py", line 243, in _call_impl
return super(WrappedFunction, self)._call_impl(
File "C:\Users\takas\bodyPartSeg\bodypix\bodypix\lib\site-packages\tensorflow\python\eager\polymorphic_function\monomorphic_function.py", line 1492, in _call_impl
return self._call_with_flat_signature(args, kwargs, cancellation_manager)
File "C:\Users\takas\bodyPartSeg\bodypix\bodypix\lib\site-packages\tensorflow\python\eager\polymorphic_function\monomorphic_function.py", line 1541, in _call_with_flat_signature
return self._call_flat(args, self.captured_inputs, cancellation_manager)
File "C:\Users\takas\bodyPartSeg\bodypix\bodypix\lib\site-packages\tensorflow\python\eager\polymorphic_function\monomorphic_function.py", line 1745, in _call_flat
return self._build_call_outputs(self._inference_function.call(
File "C:\Users\takas\bodyPartSeg\bodypix\bodypix\lib\site-packages\tensorflow\python\eager\polymorphic_function\monomorphic_function.py", line 378, in call
outputs = execute.execute(
File "C:\Users\takas\bodyPartSeg\bodypix\bodypix\lib\site-packages\tensorflow\python\eager\execute.py",
line 52, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.UnimplementedError: Graph execution error:
Detected at node 'MobilenetV1/Conv2d_7_depthwise/BatchNorm/batchnorm_1/add_1/conv' defined at (most recent call last):
File "c:\Users\takas\bodyPartSeg\bodypix\main.py", line 14, in <module>
bodypix_model = load_model(download_model(
File "C:\Users\takas\bodyPartSeg\bodypix\bodypix\lib\site-packages\tf_bodypix\model.py", line 608, in
load_model
predict_fn = load_model_and_get_predict_function(model_path)
File "C:\Users\takas\bodyPartSeg\bodypix\bodypix\lib\site-packages\tf_bodypix\model.py", line 579, in
load_model_and_get_predict_function
return load_using_tfjs_graph_converter_and_get_predict_function(model_path)
File "C:\Users\takas\bodyPartSeg\bodypix\bodypix\lib\site-packages\tf_bodypix\model.py", line 564, in
load_using_tfjs_graph_converter_and_get_predict_function
tf_fn = tfjs_graph_converter.api.graph_to_function_v2(graph)
File "C:\Users\takas\bodyPartSeg\bodypix\bodypix\lib\site-packages\tfjs_graph_converter\api.py", line
420, in graph_to_function_v2
return wrapped_import.prune(
Node: 'MobilenetV1/Conv2d_7_depthwise/BatchNorm/batchnorm_1/add_1/conv'
Current kernel implementation does not support dilations, received [1 2 2 1]
[[{{node MobilenetV1/Conv2d_7_depthwise/BatchNorm/batchnorm_1/add_1/conv}}]] [Op:__inference_pruned_766]
Is there a way to perform batch prediction to leverage GPU? I believe it is a functionality in the JS version, but I am not sure how to do that in this python version.
Using with Windows, the "os.path.join(model_base_path, weights_manifest_path)" in line 81 of download.py generates a wrong URL with "" separator.
Changing to "model_base_path+"/"+weights_manifest_path" corrected the problem
When I specify the part names ['left_upper_arm_front', 'right_upper_arm_front']
, the mask highlights the back of the sitting lady's arms. When I switch to ['left_upper_arm_back', 'right_upper_arm_back']
, the mask highlights the front of the arms instead. Same issue for lower arms.
Hi
using the code from repo readme, arises error on get_pose function as follow:
~/.venvs/tf/lib/python3.8/site-packages/tf_bodypix/bodypix_js_utils/multi_person/decode_pose.py in getDisplacement(edgeId, point, displacements)
33 # LOGGER.debug('point=%s, edgeId=%s, numEdges=%s', point, edgeId, numEdges)
34 return Vector2D(
---> 35 y=displacements[point.y, point.x, edgeId],
36 x=displacements[point.y, point.x, numEdges + edgeId]
37 )
IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices
OS: Linux base
Python: 3.8
Installation method: pip install tf-bodypix[all]
clean environment installation: false
Thanks for this great repository. I face a weird problem and could not solve it. I run the Python API code, When it wants to download the model an error is raised. By default, the BodyPixModelPaths.MOBILENET_FLOAT_50_STRIDE_16 is download. After downloading the model architecture in the "download_model" function, it can not open it as a JSON file. The error is:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
After simple checking, I found the error comes from the following code in the "download_model" function.
with open(local_model_json_path, 'r', encoding='utf-8') as model_json_fp: model_json = json.load(model_json_fp)
I think the modle.json file which download is not correct. Can you help me how to fix it?
Thanks a lot for your help
Right now, only PIL is supported for the --background argument.
Change ReplaceBackgroundApp.get_image_source() check if the video extension is a video and use a video loading API (maybe scikit-video or ffmpeg-python?) to lazy load frames and return them as the background image
The original TensorFlowJS code includes net.segmentMultiPerson
and net.segmentMultiPersonParts
, which allow code to more easily distinguish people when they appear in an image together.
I started looking at the code to see if this is something I could port, but I just don't understand TensorFlow or the existing implementations well enough to figure out how.
Please consider this for a future enhancement.
Thanks much.
There is an error occured when I run "bodypix_model = load_model(download_model(
BodyPixModelPaths.MOBILENET_RESNET50_FLOAT_STRIDE_32".
"Expectation: URL fetch failure on https://storage.googleapis.com/tfjs-models/savedmodel/bodypix/resnet50/float/model-stride32.json:404 Not Found
@de-code I got inspired to split auto-track into a separate mode and just add it to this project.
I like the effect a lot with my 120deg FOV Spedal camera because I can move around the room and it (mostly) tracks my face which is very convenient for long boring meetings. I know that we talked about adding it to layered-vision but it just was easiest to add the additional mode in this project.
Enjoy!
Ben
Getting a Value error on some images while not in some.
Entire Traceback:
Traceback (most recent call last):
File "main.py", line 297, in <module>
res = run_face_id(face_obj, img)
File "main.py", line 28, in run_face_id
text5 = face_obj.check_background()
File "main.py", line 219, in check_background
result = self.bodypix_model.predict_single(self.img)
File "/home/vardan/face_id/lib/python3.7/site-packages/tf_bodypix/model.py", line 293, in predict_single
model_input_image, padding = self.get_padded_and_resized(image, model_input_size)
File "/home/vardan/face_id/lib/python3.7/site-packages/tf_bodypix/model.py", line 287, in get_padded_and_resized
model_input_size.width
File "/home/vardan/face_id/lib/python3.7/site-packages/tf_bodypix/bodypix_js_utils/util.py", line 79, in pad_and_resize_to
resized = tf.image.resize([padded], [target_height, target_width])[0]
File "/home/vardan/face_id/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py", line 201, in wrapper
return target(*args, **kwargs)
File "/home/vardan/face_id/lib/python3.7/site-packages/tensorflow/python/ops/image_ops_impl.py", line 1546, in resize_images_v2
skip_resize_if_same=False)
File "/home/vardan/face_id/lib/python3.7/site-packages/tensorflow/python/ops/image_ops_impl.py", line 1226, in _resize_images_common
raise ValueError('\'size\' must be a 1-D int32 Tensor')
ValueError: 'size' must be a 1-D int32 Tensor
The size of one of the images in question was (145, 226, 3) by width, height, and channels. Is there a minimum size requirement that may be causing it?
I'm trying to implement bodypix into ComfyUI
I'm having issue of error
"SavedModel file does not exist"
Cannot get it run on windows machine
I want to run this code on raspberry pi 4. In the first step, I should convert this model to tflite model. I use the following link https://www.tensorflow.org/lite/convert/index to convert the model but it raises the following error.
OSError: SavedModel file does not exist at: ./models/mobilenet-float16-stride16/{saved_model.pbtxt|saved_model.pb}
I also use the following code for model conversion
python -m tf_bodypix \ convert-to-tflite \ --model-path \ "https://storage.googleapis.com/tfjs-models/savedmodel/bodypix/mobilenet/float/075/model-stride16.json" \ --optimize \ --quantization-type=float16 \ --output-model-file "./mobilenet-float16-stride16.tflite"
it works and the model convert to tflite but when I want to use it in tensorflow lite based on this link https://www.tensorflow.org/lite/guide/inference, I found that the input model dimension is array([1, 1, 1, 3], dtype=int32). Can you help me to solve this problem?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.