petworm / larvio Goto Github PK
View Code? Open in Web Editor NEWA lightweight, accurate and robust monocular visual inertial odometry based on Multi-State Constraint Kalman Filter.
A lightweight, accurate and robust monocular visual inertial odometry based on Multi-State Constraint Kalman Filter.
@PetWorm
Dear Sir,
Thank you for publishing this work !!
While trying to run the ROS example we receive the following error:
user@user-HP-ProDesk-600-G4-DM:~/LARVIO/ros_wrapper$ roslaunch larvio larvio_euroc.launch
... logging to /home/user/.ros/log/4efc391c-0333-11eb-bbb7-3822e23158ce/roslaunch-user-HP-ProDesk-600-G4-DM-27946.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.
started roslaunch server http://user-HP-ProDesk-600-G4-DM:46333/
PARAMETERS
NODES
/qxc_robot/
system (nodelet/nodelet)
ROS_MASTER_URI=http://localhost:11311
process[qxc_robot/system-1]: started with pid [27961]
type is larvio/SystemNodelet
[ INFO] [1601535646.772076998]: System: Finish loading ROS parameters...
[ INFO] [1601535646.795484764]: System Manager: Finish creating ROS IO...
Images from now on will be utilized...
Dynamic initialization success !
[qxc_robot/system-1] process has died [pid 27961, exit code -11, cmd /opt/ros/melodic/lib/nodelet/nodelet standalone larvio/SystemNodelet ~imu:=/imu0 ~cam0_image:=/cam0/image_raw __name:=system __log:=/home/user/.ros/log/4efc391c-0333-11eb-bbb7-3822e23158ce/qxc_robot-system-1.log].
log file: /home/user/.ros/log/4efc391c-0333-11eb-bbb7-3822e23158ce/qxc_robot-system-1*.log
all processes on machine have died, roslaunch will exit
shutting down processing monitor...
... shutting down processing monitor complete
done
Please advise.
Thank you
What is the license of your code?
Hi! I'm trying to learn LARVIO implementration, but i have poor knowledge in the field of inertial sensors. Could You tell me what books can help in understanding the work?
thanks for your great job on LARVIO!in dynamic initialization,the covariance is not considered for imu_state in LARVIO? could you have any ideas about the obtian of covariance from your dynamic initialization?
I have been attempting to modify the provided example codes to run "Real-time" with an imu and raspberry pi cam on a NVIDIA Jetson Nano dev board. So far I have managed to fill the imu buffer and gather imaged properly but after some digging in the code it is returning an error at "not enough features; move device around". The imu and camera are attached to the same device so their movement is correlated but it is not able to track features through the frames.
As there are no examples of how to get the algorithm running in "real time" I was wondering if anyone could help with how to structure the information and feed it to the algorithm correctly!
Thank you for any help you can provide, I can provide any information that would be helpful.
Hi,
Great work!
Do you by any chance have results regarding running time stats for both the frontend and backend part and CPU consumption for when you run LARVIO on the TX2 or whether it has the same running time as the pc? At the moment I am able to run it on a Jetson Nano XS at 30 Hz.
Also, I just tried to run LARVIO on my pc, but the best I get for the backend it 11ms. What did you do to get it to approx. 8 ms (i.e how many features, and whether you disabled some options on the yaml file)?
I would like to reach the 50 Hz on the Jetson or at least try to optimize as much as possible (I am thinking to offload the frontend to the GPU maybe by using OpenCV with GPU).
If you could give me any hint on which direction I should take that would be very helpful.
Thanks a lot :)
Best,
Ilyass
亲爱的邱博:
您试过用larvio跑kaist 数据集吗?我尝试了下,没有成功,一上来就发散得很快。
定位到是这里
buildOpticalFlowPyramid(
img_, curr_pyramid_,
Size(processor_config.patch_size, processor_config.patch_size),
processor_config.pyramid_levels, true, BORDER_REFLECT_101,
BORDER_CONSTANT, false);
出了问题
我的配置文件是按照euroc.yaml改的,但是图像的分辨率比较低(640x400)。
请问有可能是分辨率太低的原因吗,如果不是的话,会是什么原因呢
===========================================
using FEJ...
estimating td... initial td = 0
estimating extrinsic...
not calibrating imu instrinsic online...
Hybrid MSCKF...Maximum number of feature in state is 30
features augmented into state will use 1d idp
===========================================
Images from now on will be utilized...
Inclinometer-initializer completed by using 181 imu data !!!
*** stack smashing detected ***: ./larvio terminated
Aborted (core dumped)
Should I add someting else ?
Hello,
Thanks for your great work!
I saw you update MSCKF feature/SLAM feature/new SLAM feature in one step. Will it perform better if we update these features step by step?
Best,
Oliver
Hello,
I would like to know if it is possible to run this algorithm with the realsense d435i.
I tried it but the results are far from being good. May be is something wrong with the configuration that I'm using...
Do you have any configuration for this camera like you have for the mynteye?
Best,
Filipe
in zupt-update. the residual is about "prev - cur", but the jacobian is about " cur - prev", is there a mistake?
Hi,
Are there any reference papers for the ZUPT algorithm applied to MSCKF?
邱博士:
您好!我有一个问题想要请教你,原谅我英文不好,所以选择中文交流。
我看到你在imu和图像的时间戳处理上,选择以imu时间作为基准,然后通过特征点坐标预测的方式得到当前时刻的视觉观测,我看到很多其他工作是选择以图像时间戳为基准,通过插值的方式得到图像时间对应的imu测量。我想问一下这两种方式有什么优缺点或者异同吗?
祝安
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp: In member function ‘bool larvio::StaticInitializer::tryIncInit(const std::vectorlarvio::ImuData&, larvio::MonoCameraMeasurementPtr)’:
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:20:22: error: ISO C++ forbids declaration of ‘feature’ with no type [-fpermissive]
for (const auto& feature : img_msg->features)
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:20:32: warning: range-based ‘for’ loops only available with -std=c++11 or -std=gnu++11
for (const auto& feature : img_msg->features)
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:21:29: error: request for member ‘id’ in ‘feature’, which is of non-class type ‘const int’
init_features[feature.id] = Vector2d(feature.u, feature.v);
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:21:52: error: request for member ‘u’ in ‘feature’, which is of non-class type ‘const int’
init_features[feature.id] = Vector2d(feature.u, feature.v);
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:21:63: error: request for member ‘v’ in ‘feature’, which is of non-class type ‘const int’
init_features[feature.id] = Vector2d(feature.u, feature.v);
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:30:20: error: ISO C++ forbids declaration of ‘feature’ with no type [-fpermissive]
for (const auto& feature : img_msg->features) {
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:30:30: warning: range-based ‘for’ loops only available with -std=c++11 or -std=gnu++11
for (const auto& feature : img_msg->features) {
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:31:27: error: request for member ‘id’ in ‘feature’, which is of non-class type ‘const int’
curr_features[feature.id] = Vector2d(feature.u, feature.v);
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:31:50: error: request for member ‘u’ in ‘feature’, which is of non-class type ‘const int’
curr_features[feature.id] = Vector2d(feature.u, feature.v);
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:31:61: error: request for member ‘v’ in ‘feature’, which is of non-class type ‘const int’
curr_features[feature.id] = Vector2d(feature.u, feature.v);
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:32:36: error: request for member ‘id’ in ‘feature’, which is of non-class type ‘const int’
if (init_features.find(feature.id) != init_features.end()) {
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:33:32: error: request for member ‘u’ in ‘feature’, which is of non-class type ‘const int’
Vector2d vec2d_c(feature.u, feature.v);
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:33:43: error: request for member ‘v’ in ‘feature’, which is of non-class type ‘const int’
Vector2d vec2d_c(feature.u, feature.v);
^
/home/liujiang/Code/LARVIO/src/StaticInitializer.cpp:34:48: error: request for member ‘id’ in ‘feature’, which is of non-class type ‘const int’
TUM corridor1 的参数能共享一份嘛,邱博
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.