apriltag_plnd's People
Forkers
leelaserapriltag_plnd's Issues
Merging with vision landing project
Hi @chobitsfan,
I'm the current maintainer of RosettaDrone and we are looking forward to contribute to an opensource visual based precision landing project.
I reviewed your implementation and understood everything (code is clean).
Your implementation has this pros:
- Uses Apriltag
- Computes a landing point (offset) independent of the marker centers.
- Sends out the data to compute the yaw
Vision landing's track_targets has this pros:
- gstreamer support...but probably slower then just working with the latest decoded frame
- Uses an input video queue to reduce latency (receives new frames in background while doing the image processing at the same time in other thread)...but I believe the V4L implementation does something similar under the hood (?). It's important to always have the next frame decoded in memory ready to be processed as soon as the previous detection process finishes. Unnecessary frames will be dropped (wasted resources), but the overall latency is optimized.
- outputs AR images/video to debug.
- threshold filter based on historical detections (tells if we can trust the detection). Does the Apriltag library output wrong detections? If so, maybe we can just trust all detections (?) and we don't need to filter again. Anyway, the filters should be applied using the results of multiple markers.
- filter wrong pose estimations, for example, when a marker is detected with negative z value, ie. supposedly being behind the camera.
- documentation (how to calibrate the camera for correct pose estimation)
I believe both projects should be merged somehow so a maintainer community can be built around.
Of course, they are completely different implementations, but the final goal is exactly the same.
Before proposing a project merge strategy, I would like to see the code of the "frontend" where you process the marker data sent via IPC from the backend "capture.c".
You are probably doing there stuff like filtering out detection errors, maybe generating AR images for debugging and other stuff that is done in the equivalent vision landing python script.
About the scope:
We are also interested in doing some extrapolation to:
- Compute the absolute target position to deal with latencies: we need to track the drone position when the image was captured and add the displacement of the drone since this timestamp.
- Correct the absolute target position considering the last computed absolute positions: since the target could be moving, we could do a linear extrapolation based on the last two positions or a b-spline extrapolation considering tree or more points.
- Also a PID could be used to smooth the drone landing motion
Anyway, I believe the merged project should just focus on returning the position of the target relative to the camera, including the error filters required to compute a robust and consistent landing target position based on all markers and maybe also considering previous computations.
The rest I mentioned above could be implemented in the flight controller.
BTW, do you know if this has been implemented?
For RosettaDrone, we will need a independent implementation (apart from the FC), so it would be ideal to use a shared library for this (a second layer separated from the flight controller and from the target tracking/capture layer).
Video output with AR drawings for debugging
Did you implement video output generation for debugging on your "front end"?
If not, I could start implementing this.
This code should probably go in the backend, since we have the input image there.
Camera simulator
Are you using any camera simulator?
If not, I can provide a simple Python based OpenGL simulator and provide instructions.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.