konstantinosang / pykinect2-mapper-functions Goto Github PK
View Code? Open in Web Editor NEWICoordinateMapper functions using PyKinectV2 wrapper for python
License: MIT License
ICoordinateMapper functions using PyKinectV2 wrapper for python
License: MIT License
I am using python 3.7 64bit.
i did not do pip install since it seems to not work in that way. rather using the given files directly via import.
However the issue is that running mapper.py
is very slow. 0.5 fps is the frame rate. I think waitKey(3000) is the reason. But that still does not fix the issue:
When i set show = False that improves but still slow and not smooth 25/30fps. Rather somewhere around 8/10fps.
Any solution to this issue? When i do not call depth_2_color_space the speed is just back to 25/30fps i can say.
Hello. When I use the function depth_point_2_world_point(), I find there may be a bug.
As you can see in these picture, no matter what the input coordinates are, the output X and y change all the time, but the Z value remains unchanged at 20.48.
I really want to know why this problem occurs. Thanks very much.
[hello] @KonstantinosAng
Actually I am using kinect/pykinect2 lib to connect my kinect v2 with python. I am getting the depth image also and I did some processing also in opencv but I am facing issue when I am going to print the captured frame matrix it shows me the same values in in entire column so I am not able to detect any object coming in front of my kinect sensor. also I am attaching the screenshot of matrix output.
And my second issue is about frame resolution , In kinect/pykinect2 i am not able to reduce the resolution properly. if i want 15x15 resolution then it converts height only 15 and width is 120. when i am going to increase the resolution above 120x120 then it work properly but below 120x120 it only decreases the height and the width is 120 as it is .
my code:
import cv2
from pykinect2 import PyKinectV2
from pykinect2 import PyKinectRuntime
import ctypes
import _ctypes
import pygame
import numpy as np
from PIL import Image
MARGIN =10
WIDTH = 10
HEIGHT = 10
mog = cv2.createBackgroundSubtractorMOG2(history=2000, varThreshold=16, detectShadows=True)
class DepthRuntime(object):
def init(self): # constructor initialize(assign values) to the data members of the class
x = pygame.init() # initialize all imported pygame modules
# Used to manage how fast the screen updates
self._clock = pygame.time.Clock()
# Loop until the user clicks the close button.
self._done = False
# Used to manage how fast the screen updates
self._clock = pygame.time.Clock()
# Kinect runtime object, we want only color and body frames
self._kinect = PyKinectRuntime.PyKinectRuntime(PyKinectV2.FrameSourceTypes_Depth)
# back buffer surface for getting Kinect infrared frames, 8bit grey, width and height equal to the Kinect color frame size
self._frame_surface = pygame.Surface((self._kinect.depth_frame_desc.Width, self._kinect.depth_frame_desc.Height), 0, 24)
# here we will store skeleton data
self._bodies = None
# Set the width and height of the screen [width, height]
self._screen = pygame.display.set_mode( (120,120),pygame.HWSURFACE | pygame.DOUBLEBUF | pygame.RESIZABLE, 32)
self._infoObject = pygame.display.Info()
print(self._infoObject)
pygame.display.set_caption("Kinect for Windows v2 Depth")
def draw_Depth_newframe(self, frame, target_surface):
if frame is None: # some usb hub do not provide the infrared image. it works with Kinect studio though
return
target_surface.lock()
f7= np.uint8(frame.clip(1, 4000)/ 16.) #clip to change the minimum value to 0 and maximum value to 250
frame8bit = np.dstack((f7,f7,f7)) #change all RGB values to the same value
address = self._kinect.surface_as_array(target_surface.get_buffer())
ctypes.memmove(address,frame8bit.ctypes.data, frame8bit.size) #Moving data (frame) to that particular address
#frame = cv2.cvtColor(frame8bit, cv2.COLOR_GRAY2RGB)
# cv2.imshow('KINECT Video Stream', frame)
frame8bit = frame8bit.astype(np.uint8)
frame8bit = cv2.resize(frame8bit, (20,20))
new_Arr = cv2.cvtColor(frame8bit, cv2.COLOR_BGR2GRAY)
print(new_Arr)
fgmask = mog.apply(new_Arr)
# threshold this and clean it up using dilation with a elliptical mask
fgthres = cv2.threshold(fgmask.copy(), 128, 255, cv2.THRESH_BINARY)[1]
fgdilated = cv2.dilate(fgthres, kernel=cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3, 3)), iterations=3)
bgmodel = mog.getBackgroundImage()
cv2.imshow("bgmodel",bgmodel)
cv2.imshow("fgdilated",fgdilated)
cv2.imshow("fgthres",fgthres)
cv2.imshow("new_Arr", new_Arr)
# print(new_Arr.ndim)
# print(new_Arr.size)
# print(new_Arr.shape)
#frame = frame.astype(np.uint8)
#reshaped_arr = np.reshape(frame8bit,(1,651264))
del address
target_surface.unlock()
def run(self):
# -------- Main Program Loop -----------
while not self._done:
# --- Main event loop
for event in pygame.event.get(): # User did something
if event.type == pygame.QUIT: # If user clicked close
self._done = True # Flag that we are done so we exit this loop
elif event.type == pygame.VIDEORESIZE: # window resized
self._screen = pygame.display.set_mode((0,0),pygame.HWSURFACE | pygame.DOUBLEBUF | pygame.RESIZABLE, 32)
# --- Getting frames and drawing
if self._kinect.has_new_depth_frame():
frame = self._kinect.get_last_depth_frame()
self.draw_Depth_newframe(frame, self._frame_surface)
frame = None
self._screen.blit(self._frame_surface, [0,0])
pygame.display.update()
# --- Go ahead and update the screen with what we've drawn.
pygame.display.flip()
# --- Limit to 60 frames per second
self._clock.tick(1)
# Close our Kinect sensor, close the window and quit.
self._kinect.close()
pygame.quit()
main = "Kinect v2 Depth"
game = DepthRuntime()
game.run()
please help me in this issues.
Thank you
Thanks for compiling this set of useful mapper functions!
I think the value 4 in this line
PyKinect2-Mapper-Functions/mapper.py
Line 58 in bc63d39
should be changed to 1
Hello, I can't display the picture when I use uint16 in the depth of About Aligned Image.
It can only be displayed if it is changed to uint8, but it does not seem to be very complete. Do you have any ideas?
Thank you
Hello, when I use mapper, the color_2_depth_space containing imshow will not have a window display.
Can you help me? Thank you
as you can see in this picture, this is my code.
and there will always be an error:
Traceback (most recent call last):
File "video_kinect.py", line 462, in <module>
x,y = color_point_2_depth_point(a._kinect,_DepthSpacePoint,a._kinect._depth_frame_data,[int(center_x),int(center_y)])
File "video_kinect.py", line 365, in color_point_2_depth_point
return [int(depth_x), int(depth_y)]
OverflowError: cannot convert float infinity to integer
I really can not understand why this error occurs. Thank you!
I have already acquired rgb-d dataset from kinect2. Can this library be used for already acquired images?
First, I know that this is for PyKinect2 and it doesn't work in the original one. I wish to do something similar for the original PyKinect, but all information on how to do it has seemed to have vanished. I'm asking here as you seem to have figured out how to do it in this version and I'm wondering if you could point me in the right direction.
SDK for 1.8: https://learn.microsoft.com/en-us/previous-versions/windows/kinect-1.8/hh855347(v=ieb.10)
The pykinect docs seem to have been lost at some point.
Thanks for these functions! They are really useful.
However, in the depth_2_color_space function, the returned depth gets converted to uint8 which I think is not optimal. The original data is in uint16 range and numpy astype may cause loss in precision, even plain erroneous depth value.
Best regards!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.