Stereo camera support about flightgoggles HOT 5 CLOSED

Hi all,

We've just pushed a patch for v2.0.3. It adds an example launch file for stereo and also allows for adjusting the camera resolution from ROS params. To update:

# Pull new flightgoggles source code
cd ~/catkin_ws/src/flightgoggles
git pull
# Force flightgoggles to redownload binary at build time
catkin clean
catkin build

To run teleoperation with sample stereo pipeline:

roslaunch flightgoggles stereoTeleopExample.launch # WARNING! May crash! Read "Does FlightGoggles currently support Stereo?" FAQ for more info.

Current workarounds for the GPU backbuffer restriction include decreasing the default camera resolution, physically rotating your display, attaching a vertical display to your rendering computer, or using AWS-based renderering.

@koriavinash1 could you try again after updating to v2.0.3 and reducing your camera resolution using the new ROS param-based method? If it still crashes, please post a player.log from the renderer.

@strombom, you do bring up a good point! Many monitors nowadays have ultra-wide resolutions. It is certainly possible to arrange the cameras side by side. Very old versions of FlightGoggles < ~v1.5.0 used to stack cameras side by side. However, we moved to vertical stacking in order to increase memory access efficiency and frame rate. Vertical concatenation and splitting of images is an order of magnitude faster since each image occupies a contiguous block of memory. Unfortunately, engineering always has tradeoffs. In this case, we decided to value faster framerate over slightly tighter render configuration restrictions.

Once things stabilize a bit, we'll be re-releasing the code for the FlightGoggles rendering backend. Once that happens, anyone would be free to implement (and pull request) other workarounds for the GPU backbuffer issue.

Best,
-Winter

from flightgoggles.

Hi @ashwinvk94, I'm sorry for the inconvenience. This behavior is a known issue. Here's some info about it from the FAQ:

Q. Does FlightGoggles currently support Stereo?
Yes! Simply pass render_stereo:=true to the FlightGoggles launch file. Note: as of v2.0.2 the combined vertical resolution of the two stereo cameras must not be greater than the vertical resolution of the largest monitor attached to your computer. In other words, you must have a monitor with >=1536 vertical resolution to render stereo at the default camera resolution. A current workaround is to decrease the default camera resolution, or rotate or attach a vertical display to your rendering computer.

As an explanation for the current restriction, FlightGoggles uses the GPU backbuffer for storing rendered images. However, the size of this buffer is limited by the screen resolution. Thus, concatenated images cannot be larger than the size of the GPU backbuffer. This issue will be patched in a later release.

We hope to patch out this restriction by sometime Monday/Tuesday.

from flightgoggles.

Hello @Winter-Guerra ,
I'm unable to load stereo images, launch with "render_stereo:=true" freezes an entire system. Image size is also reduced so that it fits monitor's vertical resolution.
system specification:
GPU: Titan V
RAM: 64 GB
What might be possible reason?

Thank you

from flightgoggles.

@Winter-Guerra
Would it be possible to render the images side-by-side instead of top-bottom?
Monitor width>=2048 must be more common than monitor height>=1536?

from flightgoggles.

Closing since stereo is implemented. Please open a new issue for new problems.

from flightgoggles.