wieden-kennedy / cinder-kcb2 Goto Github PK
View Code? Open in Web Editor NEWCinder wrapper for Kinect 2 Common Bridge
Cinder wrapper for Kinect 2 Common Bridge
Hi there,
When I compiled the code to run the samples. it showed the following errors:
Error 1
error C2039: 'DeviceOptions' : is not a member of 'Kinect2'
Error 2
error C2228: left of '.enableColor' must have class/struct/union
Error 3
error C2660: 'Kinect2::Device::start' : function does not take 1 arguments
Error 4
error C3861: 'DeviceOptions': identifier not found
Error 5
error C2039: 'connectFrameEventHandler' : is not a member of 'Kinect2::Device'
The errors refers to the following lines of the code:
mDevice->start(Kinect2::DeviceOptions().enableColor()); //enable specific sources
mDevice->connectFrameEventHandler([&](const Kinect2::Frame& frame)
{
mFrame = frame;
});
I am not sure if it is something related to the project configuration , or it is the "Kinect2" header which has an issue.
So I would like to know if you can help me to solve it ?
Thanks in advance.
Regards,
/ Abdullah
Hello!
In the BodyApp example the function getState() for the Hand object,is always returning 0 (Unknown State) and LowConfidenceTracking, no matter what. Maybe there's an error getting the info from the SDK?
I'm using the block in an app that has an override on shutdown(). Occasionally the app won't shut all the way down, and if I pause it in the debugger, it's usually somewhere in the update loop / idling a thread somewhere, e.g. on this line. I'm managing the Kinect in a class, and in that class' destructor I'm disconnecting all handlers that I previously connected, and calling Device::stop(). This is intermittent so I'm sorry I can't be more specific... just wanted to get it up here while I'm thinking about it.
The sample PointCloudApp works as expected when I compile and run from Visual Studio 2013. However, when I try to run the built .exe, the app runs but the point cloud never draws. Tried from both Cinder release and master.
Hey,
I can compile the BodyApp example but TinderBox seems to have trouble resolving the PLATFORMTARGET variable when creating a new project with KCB2 (copy) selected.
"Unable to copy from C:/cinder/blocks/Cinder-KCB2/lib/$(PlatformTarget)/KCBv2.dll ..."
I get a periodic lag in my Kinect2 data when using this API. This doesn't occur with the main Kinect2 API (non KCB) and it is definitely in the kinect data coming in (my app continues to run at 60fps as the Kinect2 data halts and then starts up again).
On my Quadcore laptop, the library hogs my CPU to 100%, even when Kinect isn't connected.
Is this supported? I am getting errors both when trying to use the block (Unable to copy from .... blocks/Cinder-KCB2/lib/$(PlatformTarget)/$(Configuration) etc) and when building, the faceApp for example:
Error 1 error C2039: 'Channel8uRef' : is not a member of 'cinder'
\cinder-kcb2\src\kinect2.h 64 1 FaceApp
Error 14 error C2039: 'vec2' : is not a member of 'cinder' \cinder-kcb2\src\kinect2.h 72 1 FaceApp
I copied the repo into the blocks folder, then launched the tinderbox exe from the master branch. I'm not very familiar with Windows tools, as I usually work in Linux, so could be an error on my side.
Just wanted to report this, and this is a simple fix.
According to the MSDN docs:
"The BodyIndexFrame Class represents a frame that is computed based on the depth image. This image tells you which depth or infrared pixels belong to tracked people and which belong to the background. The pixel values in this frame are 8-bit unsigned integers, where 0-5 map directly to the BodyData index in the BodyFrame Class. Values greater than the value obtained from BodyFrameSource.BodyCount Property indicate that the pixel is part of the background, not associated with a tracked body. This frame is useful for green screening applications..."
https://msdn.microsoft.com/en-us/library/dn782033.aspx
There doesn't seem to be a way to access this BodyFrameSource.BodyCount from the KCB2 API. BodyFrame::getBodies().size() always returns 6, for example, regardless whether any bodies are visible. Is there another way?
MayaCam.h is removed from latest code and replaced with CameriaUi.h
Kinect.h is also out of sync with cinder master code.
I have done some changes to PointCloud and kinect.h.
The app is showing blank image. I would like to know what is wrong in my code.
Kinect.h - Replaced addVertices to addPositions
PointCloudApp.cpp is as follows ๐
/*
*
class PointCloudApp : public ci::app::App
{
public:
void draw() override;
void mouseDrag(ci::app::MouseEvent event) override;
void resize() override;
void setup() override;
void update() override;
private:
ci::Channel16uRef mChannelDepth;
Kinect2::DeviceRef mDevice;
ci::Surface8uRef mSurfaceColor;
ci::Surface32fRef mSurfaceDepthToCameraTable;
ci::Surface32fRef mSurfaceDepthToColorTable;
long long mTimeStamp;
long long mTimeStampPrev;
void loadGlsl();
ci::gl::GlslProgRef mGlslProg;
ci::gl::TextureRef mTextureColor;
ci::gl::TextureRef mTextureDepth;
ci::gl::TextureRef mTextureDepthToCameraTable;
ci::gl::TextureRef mTextureDepthToColorTable;
ci::gl::VboMeshRef mVboMesh;
ci::CameraUi mMayaCam;
ci::CameraPersp camera;
float mFrameRate;
bool mFullScreen;
ci::params::InterfaceGlRef mParams;
};
using namespace ci;
using namespace ci::app;
using namespace std;
void PointCloudApp::draw()
{
gl::viewport(getWindowSize());
gl::clear();
gl::setMatrices(camera);
gl::enableAlphaBlending();
gl::enableDepthRead();
gl::enableDepthWrite();
if (mSurfaceColor) {
if (mTextureColor) {
mTextureColor->update(*mSurfaceColor);
}
else {
mTextureColor = gl::Texture::create(*mSurfaceColor);
}
mTextureColor->bind(0);
}
if (mChannelDepth) {
if (mTextureDepth) {
gl::ScopedTextureBind scopeTextureBind(mTextureDepth->getTarget(), mTextureDepth->getId());
glTexSubImage2D(mTextureDepth->getTarget(), 0, 0, 0,
mTextureDepth->getWidth(), mTextureDepth->getHeight(),
GL_RED_INTEGER, GL_UNSIGNED_SHORT, mChannelDepth->getData());
}
else {
mTextureDepth = gl::Texture::create(
mChannelDepth->getWidth(), mChannelDepth->getHeight(),
gl::Texture::Format().dataType(GL_UNSIGNED_SHORT).internalFormat(GL_R16UI));
}
mTextureDepth->bind(1);
}
if (mSurfaceDepthToCameraTable && !mTextureDepthToCameraTable) {
mTextureDepthToCameraTable = gl::Texture::create(*mSurfaceDepthToCameraTable);
mTextureDepthToCameraTable->bind(2);
}
if (mSurfaceDepthToColorTable) {
if (mTextureDepthToColorTable) {
mTextureDepthToColorTable->update(*mSurfaceDepthToColorTable);
}
else {
mTextureDepthToColorTable = gl::Texture::create(
*mSurfaceDepthToColorTable,
gl::Texture::Format().dataType(GL_FLOAT));
}
mTextureDepthToColorTable->bind(3);
}
gl::ScopedGlslProg scopeGlsl(mGlslProg);
gl::setDefaultShaderVars();
mGlslProg->uniform("uTextureColor", 0);
mGlslProg->uniform("uTextureDepth", 1);
mGlslProg->uniform("uTextureDepthToCameraTable", 2);
mGlslProg->uniform("uTextureDepthToColorTable", 3);
gl::draw(mVboMesh);
if (mTextureColor) {
mTextureColor->unbind();
}
if (mTextureDepth) {
mTextureDepth->unbind();
}
if (mTextureDepthToCameraTable) {
mTextureDepthToCameraTable->unbind();
}
if (mTextureDepthToColorTable) {
mTextureDepthToColorTable->unbind();
}
mParams->draw();
}
void PointCloudApp::loadGlsl()
{
try {
mGlslProg = gl::GlslProg::create(gl::GlslProg::Format()
.vertex(loadAsset("cloud.vert"))
.fragment(loadAsset("cloud.frag")));
}
catch (gl::GlslProgCompileExc ex) {
console() << "GLSL Error: " << ex.what() << endl;
quit();
}
catch (gl::GlslNullProgramExc ex) {
console() << "GLSL Error: " << ex.what() << endl;
quit();
}
catch (...) {
console() << "Unknown GLSL Error" << endl;
quit();
}
}
void PointCloudApp::mouseDrag(MouseEvent event)
{
bool middle = event.isMiddleDown() || (event.isMetaDown() && event.isLeftDown());
bool right = event.isRightDown() || (event.isControlDown() && event.isLeftDown());
mMayaCam.mouseDrag(event.getPos(), event.isLeftDown() && !middle && !right, middle, right);
}
void PointCloudApp::resize()
{
//CameraPersp cam = mMayaCam.getCamera();
mMayaCam = CameraUi(&camera, getWindow());
camera.setAspectRatio(getWindowAspectRatio());
gl::enableVerticalSync();
}
void PointCloudApp::setup()
{
gl::enable(GL_TEXTURE_2D);
mFrameRate = 0.0f;
mFullScreen = false;
mTimeStamp = 0L;
mTimeStampPrev = mTimeStamp;
loadGlsl();
mDevice = Kinect2::Device::create();
mDevice->start();
mDevice->connectColorEventHandler([&](const Kinect2::ColorFrame& frame)
{
mSurfaceColor = frame.getSurface();
});
mDevice->connectDepthEventHandler([&](const Kinect2::DepthFrame& frame)
{
mChannelDepth = frame.getChannel();
mTimeStamp = frame.getTimeStamp();
});
//////////////////////////////////////////////////////////////////////////////////////////////
ivec2 sz = Kinect2::DepthFrame().getSize();
vector<vec2> vertices;
for (int32_t x = 0; x < sz.x; ++x) {
for (int32_t y = 0; y < sz.y; ++y) {
vertices.push_back(vec2(x, y) / vec2(sz));
}
}
gl::VboRef vbo = gl::Vbo::create(GL_ARRAY_BUFFER, vertices.size() * sizeof(vec2), &vertices[0], GL_STATIC_DRAW);
geom::BufferLayout layout;
layout.append(geom::Attrib::POSITION, 2, sizeof(vec2), 0);
vector<pair<geom::BufferLayout, gl::VboRef>> vertexArrayBuffers = { make_pair(layout, vbo) };
mVboMesh = gl::VboMesh::create(vertices.size(), GL_POINTS, vertexArrayBuffers);
//////////////////////////////////////////////////////////////////////////////////////////////
mParams = params::InterfaceGl::create("Params", ivec2(200, 120));
mParams->addParam("Frame rate", &mFrameRate, "", true);
mParams->addParam("Full screen", &mFullScreen).key("f");
mParams->addButton("Load GLSL", [&]() { loadGlsl(); }, "key=g");
mParams->addButton("Quit", [&]() { quit(); }, "key=q");
resize();
}
void PointCloudApp::update()
{
mFrameRate = getAverageFps();
if (mFullScreen != isFullScreen()) {
setFullScreen(mFullScreen);
mFullScreen = isFullScreen();
}
if (!mSurfaceDepthToCameraTable) {
mSurfaceDepthToCameraTable = mDevice->mapDepthToCameraTable();
}
if ((mTimeStamp != mTimeStampPrev) && mSurfaceColor && mChannelDepth) {
mTimeStampPrev = mTimeStamp;
mSurfaceDepthToColorTable = Surface32f::create(mChannelDepth->getWidth(), mChannelDepth->getHeight(), false, SurfaceChannelOrder::RGB);
vector<ivec2> positions = mDevice->mapDepthToColor(mChannelDepth);
vec2 sz(Kinect2::ColorFrame().getSize());
Surface32f::Iter iter = mSurfaceDepthToColorTable->getIter();
vector<ivec2>::iterator v = positions.begin();
while (iter.line()) {
while (iter.pixel()) {
iter.r() = (float)v->x / sz.x;
iter.g() = 1.0f - (float)v->y / sz.y;
iter.b() = 0.0f;
++v;
}
}
}
}
CINDER_APP(PointCloudApp, RendererGl(RendererGl::Options().coreProfile()), [](App::Settings* settings)
{
settings->prepareWindow(Window::Format().size(1280, 960).title("Point Cloud App"));
settings->setFrameRate(60.0f);
})
Hi,
I was wondering if it'd be useful to expose this "depth-to-camera-space" lookup table in your block. This lookup table can be used to map the depth frame pixels to 3d positions in camera space. Converted to a texture, it can be used to "inverse-project" the depth map in a vertex shader.
I currently am using the following function (the name & the assert call might not be ideal):
ci::Surface32f Device::getDepthFrameToCameraSpaceTable() const {
PointF* tableEntries = nullptr;
uint32_t tableCount;
GetDepthFrameToCameraSpaceTable( mKinect, &tableCount, &tableEntries );
Vec2i size = Frame::getDepthSize();
assert( size.x * size.y == tableCount );
ci::Surface32f lookupSurface( size.x, size.y, false );
Surface32f::Iter iter = lookupSurface.getIter();
size_t i = 0;
while( iter.line() ) {
while( iter.pixel() ) {
iter.r() = tableEntries[i].X;
iter.g() = tableEntries[i].Y;
iter.b() = 1.0f;
++i;
}
}
return lookupSurface;
}
Used in the vertex shader, it would look something like this:
vDepth = texture( uDepth, ciTexCoord0 ).r;
pos.xy = vDepth * texture( uDepthToCameraSpace, ciTexCoord0 ).rg; //or texelFetch
pos.z = vDepth;
Just a suggestion. Feel free to close this "issue".
Hey!
I was wondering if you were planning on adding the ability to get the Floor Clip Plane in this cinder block.
Here is the API reference for IBodyFrame: https://msdn.microsoft.com/en-us/library/microsoft.kinect.kinect.ibodyframe.aspx
And here is the floor clip plane: https://msdn.microsoft.com/en-us/library/microsoft.kinect.kinect.ibodyframe.get_floorclipplane.aspx
I am happy to add it in myself, but I was wondering if this would be a different type of frame, that I would connect an event handler to etc, or if this should be built into the existing Body data.
Hi Guys,
I am trying to compile the project using Cinder 0.8.6 but I am getting lots of errors of datatypes not included in Cinder and being used in KinectV2.h
For example the file "cinder/gl/VboMesh.h" doesn't exist in Cinder.
The types Channel16uRef, Surface8uRef ..etc don't exist in Cinder
Any help would be really appropriated
Thanks
The boolean operators on channels & surfaces have been removed: the implicit shared_ptr pattern has been replaced with the typedef'ed shared_ptr refs. As such, all the if-statements checking if a channel or a surface exist now fail on compile.
Just a heads up...
This error occurs on Line 745 in Kinect2.cpp and halts compiling of the BasicApp (syntax error). Also an error where this is called:
function "Kinect2::Device::Process::operator=(const Kinect2::Device::Process &)" (declared implicitly) cannot be referenced -- it is a deleted function [project name]
Is there something I'm missing?
Cinder v 0.9.0
Visual Studio 2015
Windows 10
Hello,
I'm having some trouble getting a few of the samples to build. These are the specific errors I'm getting for each. The BodyApp sample however builds fine.
I'll add that I'm just getting my feet wet in Cinder. Any suggestions would be much appreciated.
BasicApp
1> BasicApp.cpp
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(71): error C2039: 'viewport' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(71): error C3861: 'viewport': identifier not found
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(72): error C2039: 'clear' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(72): error C3861: 'clear': identifier not found
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(73): error C2039: 'setMatricesWindow' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(73): error C3861: 'setMatricesWindow': identifier not found
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(74): error C2039: 'enableAlphaBlending' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(74): error C3861: 'enableAlphaBlending': identifier not found
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(78): error C2039: 'draw' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(78): error C2660: 'BasicApp::draw' : function does not take 3 arguments
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(82): error C2039: 'draw' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(82): error C2660: 'BasicApp::draw' : function does not take 3 arguments
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(86): error C2039: 'draw' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(86): error C2660: 'BasicApp::draw' : function does not take 3 arguments
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(90): error C2039: 'draw' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(90): error C2660: 'BasicApp::draw' : function does not take 3 arguments
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(98): error C2039: 'enable' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(98): error C3861: 'enable': identifier not found
FaceApp
1> FaceApp.cpp
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\faceapp\src\faceapp.cpp(80): error C2039: 'draw' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\faceapp\src\faceapp.cpp(80): error C2660: 'FaceApp::draw' : function does not take 3 arguments
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\faceapp\src\faceapp.cpp(114): error C2039: 'draw' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\faceapp\src\faceapp.cpp(114): error C2660: 'FaceApp::draw' : function does not take 1 arguments
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\faceapp\src\faceapp.cpp(126): error C2039: 'drawStrokedRect' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\faceapp\src\faceapp.cpp(126): error C3861: 'drawStrokedRect': identifier not found
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\faceapp\src\faceapp.cpp(128): error C2039: 'drawSolidCircle' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\faceapp\src\faceapp.cpp(128): error C3861: 'drawSolidCircle': identifier not found
PointCloudApp
1> PointCloudApp.cpp
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\pointcloud\src\pointcloudapp.cpp(39): fatal error C1083: Cannot open include file: 'cinder/MayaCamUI.h': No such file or directory
1> Kinect2.cpp
Cheers,
Stephen
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.