baptistehudyma / rgb-d-slam Goto Github PK

When a match is found with an old feature (better covariance then the currently observed features, and not matched for a long time), retry the matching operation to capture older features.

Ideas:

• Find a pose using only old features (loop closing pose)
• update the position of the observed features with the loop closure

The conversion of depth map to a boxed matrix representation takes 1/3 of the global performances, and should be optimized.
It is used only for primitive detection.

Two options:

• Extract primitives with an undistorted depth map, and project them in RGB camera space
• Optimize the matrix creation by removing unnecessary loops

We need an utils function to compute a point to cylinder distance.

The variance of the x and y of detected points is constant.
It should be a model of error that increases when getting close to the borders, where the pixels are most distorted.

The output variance of our program does not contain a rotation variance for the trajectory.
We should compute it and keep it along side the position variance.

When no depth is available for a feature, use a form of triangulation to estimate it's missing parameters (usually: depth)

Points features can be stored as Inverse Depth Represantation.

Convert the current cylinder representation to a superquadric form.

• Separate treatment of places and cylinders
• Change the primitive projection functions

During the optimization process, a feature with a better variance should be more important to the final result.

A feature with a worse covariance should not be taken as seriously during optimization.

Levenberg Marquardt functor uses "match._worldFeatureCovariance.x()" instead of converting it to screen space

When features are lost and cannot be tracked into the local map, transfer them to a global map.

The global map must:

• Be updated efficiently (eg: when a loop closing occurs)
• Allow for local and global descriptors (to detect loop closures)
• Allow for the update of the local map when a global optimization is found
• be limited in RAM. We do not want to fill the computer's memory

The global map could be a :

• A mesh, handled as in a video game
• An octree (res 1-5m), where each cubes can have a dedicated descriptor
• A keyframe based graph: every SLAM out there are using this format, with bundle adjustment, but I find it inelegant

For now, parameters are in absolute untis (eg: pixels).
They should be reworked to be relative (eg: to the image size, feature density, etc).

This can remove some parameters.

Detect when a feature has a motion relative to the camera, to exclude it from the pose optimization process.

• Detect moving points
• Detect moving lines
• Detect moving planes
• Detect moving superquadrics

-> Iterative RANSAC has some promising results

Optimize all containers uses

• replace maps by sorted vectors when needed
• reserve container size beforhand
• Reuse allocated containers

Add a cylinder to plane, as well as a plane to cylinder matching capability.

Display the observer trajectory in the output map file

For now, lines are only detected (as a preliminary work).
They should be added to the feature system :

• Tracked
• Matched
• Handled in local map

Replace the pinhole camera model with a generic class that allow the user to use any type of camera as long as we can define generic mathematical models for those cameras.

It's already possible to do that for pinhole and fisheye cameras, maybe more

Allow user to add IMU's measurements, GPS, ... any kind of data feed.
Add a generic class to implement the specificity of a given data feed, and add them generically to the optimization process.

Using the detected proto-planes features, search for closed shapes that qualifies as superquadrics.

• Use a RANSAC fitting based on L-M optimization

For now, the RANSAC process uses the same optimization as the global optimization.
Using a dedicated PNP algorithm could improves performances.

If two map points (or stagged an map point) are identified as being the same, they should be merged.

• Implement a system to merge two points in the local map
• Merge their covariance matrix
• Merge their tracking values

The current system cannot detect points at the same coordinates, but modifying the _keypointMaskDiameter parameter to 1 can produce some overlapped points (for testing only)

We should use detected primitives in world space in the RANSAC and global optimization to improve performances and reliability.

• Implement a primitive projection system
• Add primitives to the optimization process

The motion tracking model is not acting as expected (erratic and large unrealistic movements).
It should be replaced by a more stable model (Extend Kalman filter is a good candidate)

There is a case where is_visible returns false when a plan is in camera view, but not one of it's boundary points are.
The method should check if the boundary itself is in view, and not just the points that defines it.

• Detect the borders of an observed plane

• store border points with the plane

• update those points with new detections of planes

• Find a way to store the points coordinates relative to the plane parameters

Most of the "last" used in the program mean "latest", so some renaming should be done.

English is not my first language !

Tracked primitives should be added to the map file.
They should be handled as meshes, to keep a consistent mask and shapes.

Primitives are stored as a camera base representation (in a camera centered space).
To use them in global optimization, we should transform the planes and cylinders (for now) into a world representation.

• Need to think about a way to handle the primitive pixel mask in world space
• Need to adapt the matching technique to fit the world transformation

When tracking is lost, we should create a new global map AND be able to reattach the previous global map if a link between the two is found

Predict the new pose variance after extrapolating a new position with the motion model