Faulty depth measurements near object edges about ros_astra_camera HOT 34 OPEN

I can confirm this issue affects both ASTRA S and ASTRA cameras.

from ros_astra_camera.

This was not the case with an older driver I got from the Orbbec website (the link is down now).

Could you provide the link of older driver? we will check it .

from ros_astra_camera.

I too can confirm that there is a significant difference between the older driver that was hosted on dropbox and this one. I am attaching the old driver for your reference:
OpenNI-2-Linux-x64-Orbbec3D.tar.gz

from ros_astra_camera.

These artifacts are a challenge for navigating with the TurtleBot. It generally works but with noteably lower performance due to the extra noise needing to be rejected. And the avoidance of phantom obstacles.

Here's a screenshot from my debugging.

It would be really great to add the filters to the released version of the driver.

from ros_astra_camera.

A bugfix for this is required before we can consider integrating the Orbbec Astra with our products. Any idea when we could expect a fix for this?

from ros_astra_camera.

@bit-pirate The binary releases of our openni2 drivers include this filter. You can find them as part of Astra SDK's openni2 folder or at https://3dclub.orbbec3d.com/t/full-openni-2-environment-download/34. Note that ARM builds exclude the filter due to the limited processing power of most ARM CPUs. If you need additional support on this please contact [email protected].

from ros_astra_camera.

@tfoote is the OpenNI2 bundle linked above by @JoshBlake included in our ROS package for the Astra? "Our ROS package" being your fork, if I understand the current situation correctly.

from ros_astra_camera.

All my changes are merged upstream. https://github.com/tfoote/ros_astra_camera/network I did the release from my branch for speed only. All development and the next release should come from the orbbec main branch.

from ros_astra_camera.

Looks like it the filter is currently in the filterlibrary branch and I guess will be eventually merged Looks much better with that.

Any chance to get any info on how this filter works?

from ros_astra_camera.

@AlexReimann the branch of filerlibrary is a temporary solution , that provider function same to master with extra filter driver . but it will not merged to the master. you can use codes by reference to README.

from ros_astra_camera.

So what's the planned solution?

from ros_astra_camera.

Yup, waiting for a solution as well... any news?

from ros_astra_camera.

Bump ._.'

from ros_astra_camera.

@JoshBlake @ob-tim-liu it would be very important for us and our products, like the TurtleBot, to get a fix for this edge noise into the released driver. Could you let us know about the progress?

from ros_astra_camera.

now, we provider the pull request to tfoot about orbbec driver with soft filter library.
before merge this request , you can use the 'filterlibrary' branch , the only different is
astra_openni2 in the local dir.

刘华
座机：0755-86329228
邮箱：[email protected]
手机：13603061506
深圳奥比中光科技有限公司 www.orbbec.com.cn
地址：深圳市南山区粤兴三道8号**地质大学产学研基地中地大楼A808

From: Marcus Liebhardt
Date: 2016-11-04 14:42
To: orbbec/ros_astra_camera
CC: ob-tim-liu; Mention
Subject: Re: [orbbec/ros_astra_camera] Faulty depth measurements near object edges (#7)
@JoshBlake @ob-tim-liu it would be very important for us and our products, like the TurtleBot, to get a fix for this edge noise into the released driver. Could you let us know about the progress?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

from ros_astra_camera.

In case people are wondering, @ob-tim-liu is talking about tfoote/OpenNI2#1 . Once that is merged, the master branch in this repo should do the same as the filterlibrary branch. The only difference is that the filterlibrary branch contains a copy of tfoote/OpenNI2, whereas master uses tfoote/OpenNI2 as an external dependency (which is better, of course).

from ros_astra_camera.

Understood. Thanks for the update, guys.

@tfoote could you notify us here when that PR has been merged and the package re-released? Then we will test it again (and hopefully close the issue).

from ros_astra_camera.

@tfoote bounced this back

So.. bump :p

from ros_astra_camera.

@ob-tim-liu as @tfoote mentioned in his comment in the open PR (tfoote/OpenNI2#1), Orbbec should start release their ROS S/W drivers from their own, official repo.

Could you please do the required changes and release the important changes in the filterlibrary branch?

from ros_astra_camera.

@bit-pirate i have change the link of OpenNI2 from tfoote's github to orbbec, you can use the latest release on master branch

from ros_astra_camera.

Thanks @ob-tim-liu ! @AlexReimann please verify.

from ros_astra_camera.

Latest deb still doesn't have the filter (not released yet?) and the master branch here doesn't show any depth image since the last commit with our Astras.

If I roll-back the master branch one commit, the filter is in and works.

So just need to release the deb and fix that issue?

from ros_astra_camera.

The release deb is still at version 0.1.5. So the commits from Sep 22 to Dec 16 are not released yet.

I'd say, 1. fix depth image issue and 2. release the softfilter changes and depth image bugfix with version 0.1.6.

from ros_astra_camera.

Also, the deb is still released from tfoote/ros_astra_camera, not from this repo. Given the low responsiveness of the Orbbec people, I think I would even prefer it if somebody other than them would maintain the deb releases. We would just need somebody to volunteer for the job.

from ros_astra_camera.

Also, the filter library is not really a fix, but a horrible workaround. It's so computationally expensive that it's impossible to do on an ARM board, and since it's baked into the OpenNI driver instead of a standalone node you can't run it on an external PC either.

This is what I have observed so far:

• The random pixels in the depth image only occur near object edges, and only if depth registration is enabled in the driver.
• They seem truly random, not interpolated between foreground and background.

Depth registration has to reproject each depth value from the depth into the rgb frame. My theory is that the random pixels occur in all pixels that are visible from the rgb frame, but not the depth frame (hence near object edges). Probably they just allocated some uninitialized memory for the registered depth image, then overwrote each pixel for which they can compute a depth value, leaving the rest uninitialized (i.e. "random").

The proper (and easy!) fix would be to just initialize the registered depth image with "NaN". If Orbbec could upgrade their firmware to do just that, we wouldn't need the filter library any more, and the Astra would finally become usable.

from ros_astra_camera.

I don't know how I can help, but I wanted to add another data point (and to make sure what I'm seeing isn't a new issue) I get something like this when the astra is too close to an object or it is looking at a surface that is out of range:

Here's a gif of me moving it and show it in a glass door and if I put my hand close: https://gfycat.com/MelodicBlondKudu

Notice the left-right smears in the depth image.

from ros_astra_camera.

from ros_astra_camera.

@ob-tim-liu Most users use the prebuilt debian packages in which the filter library is not included. A new release which includes the filter here: https://github.com/tfoote/OpenNI2/pull/1/files would give most users a much better experience with the astra when used with ROS.

from ros_astra_camera.

@mintar we get the problem also when there is no depth registration

Update: The problem was one of our engineers was using the wrong repository / branch

Besides that we run into a new problem case:

In this case we get noise depth pixels up to a certain pixel height (you can clearly see that in the lower right corner) in the areas where the sensor can't get a proper depth value. This looks to me like some memory initialization or corruption problem.

I agree on @mintar that fixing this by fixing the firmware or driver properly is the way to go instead of hiding the problem by using some filter.

@ob-tim-liu Please could you let someone make sure you are properly initializing stuff (I'm humbly suggesting memcheck)?

I will do some more test over the day and see if there is a way around this.

Update: The problem was one of our engineers was using the wrong repository / branch

from ros_astra_camera.

Hello all;
After having problems with the astra driver from indigo repositories and having issues, I have compiled this driver from source testing the flag -DFILTER to both values ON and OFF on filter branch of this repository. Seems to work better. In any case, any plans to move these changes to master?
Is an alternative fix planned for the package in ROS or an alternative solution in Astra FW?

Thanks,
Alberto

from ros_astra_camera.

Is there a bug fix?

from ros_astra_camera.

@aswinsarang Please use the filter library from the above comments.

from ros_astra_camera.

The problem stated 4 posts above was because someone used a branch / repository without the filter library.

from ros_astra_camera.

I wonder if anyone confirmed that the problem was actually a not-initialized buffer or not.

And about the filter...where can I find the source code or the algorithm to see if we can do any better?

from ros_astra_camera.