catkin_make failed with sophus, fmt, absl problem, need to check the version of them. THANKS!

Hi, thanks for open-sourcing the code.
We were trying to implement DSOL using both, the iros22 and main branch, and it seems to be running quite slow as it is taking multiple seconds to process each image frame from the tartan_air gascola dataset. We have tried different launch file parameters such as, changing the frequency and wait time, turning off visualization and logging as well. We have even tried it on different processors (going up to i9 11th gen.), but the time taken do not change drastically. I am attaching a summary snapshot of the parameters and output as tested on MacBook Air using the Parallels virtual machine with default configuration for your reference. The times increase as we move ahead in the dataset sequence.

---------Parameters-----------

SUMMARY
========

PARAMETERS
 * /dsol_data/adjust/affine: False
 * /dsol_data/adjust/c2: 4
 * /dsol_data/adjust/dof: 4
 * /dsol_data/adjust/grad_factor: 1.0
 * /dsol_data/adjust/max_iters: 6
 * /dsol_data/adjust/max_levels: 3
 * /dsol_data/adjust/max_outliers: 0
 * /dsol_data/adjust/min_depth: 0.1
 * /dsol_data/adjust/rel_change: 0.01
 * /dsol_data/adjust/stereo: True
 * /dsol_data/align/affine: False
 * /dsol_data/align/c2: 4
 * /dsol_data/align/dof: 4
 * /dsol_data/align/grad_factor: 1.0
 * /dsol_data/align/max_iters: 6
 * /dsol_data/align/max_levels: 0
 * /dsol_data/align/max_outliers: 1
 * /dsol_data/align/min_depth: 0.1
 * /dsol_data/align/rel_change: 0.01
 * /dsol_data/align/stereo: False
 * /dsol_data/cloud_max_depth: 50.0
 * /dsol_data/data_dir: /media/psf/robo/c...
 * /dsol_data/data_max_depth: 100.0
 * /dsol_data/end: 0
 * /dsol_data/freq: 10.0
 * /dsol_data/log: 5
 * /dsol_data/motion_alpha: 0.5
 * /dsol_data/odom/init_align: True
 * /dsol_data/odom/init_depth: False
 * /dsol_data/odom/init_stereo: True
 * /dsol_data/odom/marg: False
 * /dsol_data/odom/min_track_per_kf: 10
 * /dsol_data/odom/min_track_ratio: 0.35
 * /dsol_data/odom/num_kfs: 4
 * /dsol_data/odom/num_levels: 4
 * /dsol_data/odom/reinit: True
 * /dsol_data/odom/vis_min_depth: 0.5
 * /dsol_data/reverse: False
 * /dsol_data/save: 
 * /dsol_data/select/cell_size: 16
 * /dsol_data/select/max_grad: 64
 * /dsol_data/select/max_ratio: 0.6
 * /dsol_data/select/min_grad: 4
 * /dsol_data/select/min_ratio: 0.4
 * /dsol_data/select/nms_size: 1
 * /dsol_data/select/reselect: True
 * /dsol_data/select/sel_level: 1
 * /dsol_data/start: 0
 * /dsol_data/stereo/half_cols: 3
 * /dsol_data/stereo/half_rows: 2
 * /dsol_data/stereo/match_level: 3
 * /dsol_data/stereo/min_depth: 0.5
 * /dsol_data/stereo/min_zncc: 0.8
 * /dsol_data/stereo/refine_size: 1
 * /dsol_data/tbb: 1
 * /dsol_data/vis: False
 * /dsol_data/wait_ms: 8
 * /rosdistro: noetic
 * /rosversion: 1.15.14

NODES
  /
    dsol_data (dsol/sv_dsol_node_data)

auto-starting new master
process[master]: started with pid [944472]
ROS_MASTER_URI=http://localhost:11311/

setting /run_id to 1955a36a-03d2-11ed-bda9-0b7ac2dd2c25
process[rosout-1]: started with pid [944483]
started core service [/rosout]
process[dsol_data-2]: started with pid [944490]
[ INFO] [1657843643.404443861]: tartan_air(dir=/media/psf/robo/catkin_ws/src/dsol/tartan_air/P001, size=382, dtypes=[image, depth, intrin, pose])
[ INFO] [1657843643.405418708]: Data range: [0, 382)
[ INFO] [1657843643.405431500]: Reverse: false
[ INFO] [1657843643.405734408]: Freq: 10
[ INFO] [1657843643.405745949]: Max depth: 100
[ INFO] [1657843643.416015278]: DirectOdometry(
	OdomCfg(tbb=1, log=5, vis=false, marg=false, num_kfs=4, num_levels=4, min_track_ratio=0.35, vis_min_depth=0.5, reinit=true, init_depth=false, init_stereo=true, init_align=true)
	Camera(w=0, h=0, fxycxy=[0 0 0 0], b=0, scale=1)
	PixelSelector(cfg=SelectCfg(sel_level=1, cell_size=16, min_grad=4, max_grad=64, nms_size=1, min_ratio=0.4, max_ratio=0.6, reselect=true), grid_border=1)
	StereoMatcher(cfg=StereoCfg(half_rows=2, half_cols=3, match_level=3, refine_size=1, min_depth=0.5, min_zncc=0.8))
	FrameAligner(cfg=DirectOptmCfg(init_level=0, max_iters=6, max_xs=0.1), DirectCostCfg(affine=false, stereo=false, c2=4, dof=4, max_outliers=1, grad_factor=1, min_depth=0.1))
	BundleAdjuster(cfg=DirectOptmCfg(init_level=0, max_iters=6, max_xs=0.1), DirectCostCfg(affine=false, stereo=true, c2=4, dof=4, max_outliers=0, grad_factor=1, min_depth=0.1))
	KeyframeWindow(size=0/4))
[ INFO] [1657843643.416354519]: wait_ms: 8
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20220714 17:07:23.416422 944490 ocv.cpp:122] Press 's' to step, 'r' to play, 'p' to pause, 'space' to toggle play/pause, 'esc' to quit
[ INFO] [1657843643.480953533]: motion_alpha: 0.5
[ INFO] [1657843643.481054614]: === 0 ===
[ INFO] [1657843643.503414512]: Camera(w=640, h=480, fxycxy=[320 320 320 240], b=0.25, scale=1)
[ INFO] [1657843643.524947808]: T_c0_w:
    0.984318    -0.176402 -2.77556e-17      49.6318
-2.77556e-17 -4.44089e-16            1      3.05804
   -0.176402    -0.984318 -4.44089e-16      14.8371
           0            0            0            1

---------Output-----------

I20220714 16:41:46.706459 939354 odom.cpp:554] Manager: dsol
[All_Keyframe    ] n: 2        | last: 2.117205403s   | mean: 1.0637994855s  | min: 10.393568ms    | max: 2.117205403s   | sum: 2.127598971s   |
[All_Tracking    ] n: 16       | last: 353.075553ms   | mean: 455.432054ms   | min: 175.646us      | max: 608.229543ms   | sum: 7.286912867s   |
[K0_SetOccMask   ] n: 2        | last: 66.784us       | mean: 61.5345us      | min: 56.285us       | max: 66.784us       | sum: 123.069us      |
[K1_SelectPixels ] n: 2        | last: 987.885us      | mean: 1.034462ms     | min: 987.885us      | max: 1.081039ms     | sum: 2.068924ms     |
[K2_InitPoints   ] n: 2        | last: 2.539954ms     | mean: 4.32093ms      | min: 2.539954ms     | max: 6.101906ms     | sum: 8.64186ms      |
[K3_InitDepths   ] n: 2        | last: 102.405us      | mean: 262.8235us     | min: 102.405us      | max: 423.242us      | sum: 525.647us      |
[K5_BundleAdjust ] n: 1        | last: 2.111605888s   | mean: 2.111605888s   | min: 2.111605888s   | max: 2.111605888s   | sum: 2.111605888s   |
[K5_InitAlign    ] n: 2        | last: 16.623us       | mean: 18.9565us      | min: 16.623us       | max: 21.29us        | sum: 37.913us       |
[K6_InitPatches  ] n: 2        | last: 1.885864ms     | mean: 2.297835ms     | min: 1.885864ms     | max: 2.709806ms     | sum: 4.59567ms      |
[P0_ConvertGray  ] n: 32       | last: 104.821us      | mean: 121.526us      | min: 104.404us      | max: 331.253us      | sum: 3.888838ms     |
[T0_MakePyramid  ] n: 32       | last: 137.484us      | mean: 157.364us      | min: 129.651us      | max: 285.216us      | sum: 5.035655ms     |
[T1_TrackFrame   ] n: 15       | last: 352.938069ms   | mean: 485.6488415ms  | min: 345.37753ms    | max: 608.0996ms     | sum: 7.284732625s   |
[ INFO] [1657842106.706868622]: OdomStatus(add_kf=false, remove_kf=false, total_kfs=2)
[ INFO] [1657842106.707184794]: === 16 ===
I20220714 16:41:46.745659 939354 odom.cpp:286] Track frame with window size: 2
I20220714 16:41:47.085711 939354 odom.cpp:295] DirectStatus(num_kfs=2, num_points=819, num_levels=4, num_iters=0/17, num_costs=598, cost=1.58e+05, converged=true)
I20220714 16:41:47.085772 939354 odom.cpp:298] 0: KeyframeStatus(pixels=1055, depths=1040, patches=4184 | info_bad= 15, info_uncert=660, info_ok=256, info_max=124)
I20220714 16:41:47.085788 939354 odom.cpp:298] 1: KeyframeStatus(pixels=1049, depths=1041, patches=4170 | info_bad= 10, info_uncert=600, info_ok=332, info_max=107)
I20220714 16:41:47.085803 939354 odom.cpp:358] win 2, tracked 394, depths 1040.5, ratio: 37.87%
[ INFO] [1657842107.085848365]: OdomStatus(add_kf=false, remove_kf=false, total_kfs=2)
[ INFO] [1657842107.086061798]: === 17 ===
I20220714 16:41:47.127633 939354 odom.cpp:286] Track frame with window size: 2
I20220714 16:41:47.495066 939354 odom.cpp:295] DirectStatus(num_kfs=2, num_points=819, num_levels=4, num_iters=0/19, num_costs=576, cost=1.81e+05, converged=true)
I20220714 16:41:47.495116 939354 odom.cpp:298] 0: KeyframeStatus(pixels=1055, depths=1040, patches=4184 | info_bad= 15, info_uncert=660, info_ok=256, info_max=124)
I20220714 16:41:47.495131 939354 odom.cpp:298] 1: KeyframeStatus(pixels=1049, depths=1041, patches=4170 | info_bad= 10, info_uncert=600, info_ok=332, info_max=107)
I20220714 16:41:47.495144 939354 odom.cpp:358] win 2, tracked 366, depths 1040.5, ratio: 35.18%
[ INFO] [1657842107.495186976]: OdomStatus(add_kf=false, remove_kf=false, total_kfs=2)
[ INFO] [1657842107.495372705]: === 18 ===

Are we missing any tunable configuration parameters? Any help would be appreciated.

Thank you

Hi, thanks for opensourcing the code.
Does the code support monocular sensor configuration?

The complete error log is shown below：

/home/XXXXXXXX/catkin/catkin_dsol/src/dsol-main/sv/dsol/frame.cpp:21:37: required from here
/usr/local/include/fmt/core.h:1017:9: error: static assertion failed: Cannot format argument. To make type T formattable provide a formatter specialization: https://fmt.dev/latest/api.html#formatting-user-defined-types
1017 | formattable,
| ^~~~~~~~~~~
make[2]: *** [dsol-main/sv/dsol/CMakeFiles/sv_dsol_frame.dir/build.make:63: dsol-main/sv/dsol/CMakeFiles/sv_dsol_frame.dir/frame.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:2367: dsol-main/sv/dsol/CMakeFiles/sv_dsol_frame.dir/all] Error 2
make: *** [Makefile:141: all] Error 2

It seems that there are some variables requiring being type conversion. Unfortunately, fmt coding is not what i familiar with.

By the way, could you please tell me which fmt and abseil version do you use? I also find several Version compatibility Issues about them.

Thanks anyway.

Everything is incredible with the original config file, no mather precision or CPU-cost.

BUT when turn on the affine option in config file, the trajectory just crash immediately.

Is there anything i neglect to change along with affine?

As the image shown above, the direction is from rightBottom to leftTop, the green line means groud_ture and red means the error trajectory.

As for the terminal output, "kf=true" appear in every frame, and the point tracking fails in every frame.

Hi, Chao Qu.

I found that the default configuration of DSOL doesn't use marginalization, it makes sense because DSOL performs worse with marginalization ON. But why?

Theoretically it should performs better. Does direct method just conflict with marginalization?
( It is just my casual surmise since DM-VIO perform better with a custom marginalization )

I found that DSOL use the weighting function from DVO rather than using huber like original DSO, why?

Is that because you have test them and found it better using huber?

Also, i found that you simplify the weighting function (the variable σ), have you tested the difference?

I'm trying to run DSOL in a cheap platform which doesn't have L2 cache. Without doubt the perform sucks totally.
Is there any way to decrease the cache occupation?
I'm considering resize the image input to 240p, and change the data type of features from double to float. Is it reasonable or is there any better way?
As you guys see, i care about occupation more than precision.

hi, thank you for open sourcing your excellent work. i am derivating the jacobians used in pba, and having problems about the term: "Vector3d dp1_dq0 = T10_fej.translation(); " , in function AdjustCost::CalcJacGeo( ) of adjust.cpp, , according to my derivation, this term is something like "dp1_dq0 = -T10_fej.linear()*point0.pt()/q0", when i run my modification, the optimization won't converge. may i ask for some detailed derivetions about this term? thanks a lot.

Hey, nice work!
After glancing over your paper, I thought about optimizing my LDSO fork with your DSO changes for a better framerate.
Apparently, the keyframe creation process seems to be the most time-consuming, so that would be one change I'd like to try out.
I know where the keyframe selection is in LDSO, but could you point me towards the right code section in your repo so that I could easily implement it in my repo? Thanks a lot!

With the example code you support,it seems both node_odom and node_data need to provide either depth or stereo image, and odom.cpp will check init_depth || init_stereo. I try to run it with monocular dataset, and the program immediately exit.

Hi, thanks for your great work.
I use catkin_make to build the code, but get the following error:

I install fmt using the following command:

git clone --depth 1 --branch 8.1.0 https://github.com/fmtlib/fmt.git
cd fmt
cmake -DCMAKE_POSITION_INDEPENDENT_CODE=TRUE -DCMAKE_CXX_STANDARD=17 -DFMT_TEST=False
sudo make install

could you give me suggestion?

catkin build error:
dsol_ws/src/dsol/sv/dsol/image.h:14:11: error: ‘cv::Rect {aka class cv::Rect_<int>}’ has no member named ‘empty’ if (roi.empty()) return false;

Hi, i'm very confusing about pba jacobian.
I thought that PBA use additive approach, not inverse compositional, the error function is I1(W(x0;T^w_1,T^w_0,q0)) - I0(x0). So i direct calculate partial err/ partial T^w_1 and partial err/partial T^w_0. But it's diff from CalcJacGeo function.
hope for your response!

Hi, when I run DSOL based on Ubuntu 20.04 Ros Noetic, I encounter the below problem. Do you know how to solve it? Thanks for your time~

ROS_MASTER_URI=http://localhost:11311

process[dsol_data-1]: started with pid [284373]
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20230125 18:18:55.030283 284373 dataset.cpp:734] Could not infer dataset name from dir '/home/cpy/Datasets/dataset/sequences/00'
[ERROR] [1674641935.030472459]: Invalid dataset at: /home/cpy/Datasets/dataset/sequences/00
[dsol_data-1] process has died [pid 284373, exit code -11, cmd /home/cpy/catkin_ws_DSOL/devel/lib/dsol/sv_dsol_node_data __name:=dsol_data __log:=/home/cpy/.ros/log/63cd045a-9c94-11ed-80d5-fd604bd6d0fd/dsol_data-1.log].
log file: /home/cpy/.ros/log/63cd045a-9c94-11ed-80d5-fd604bd6d0fd/dsol_data-1*.log
all processes on machine have died, roslaunch will exit
shutting down processing monitor...
... shutting down processing monitor complete
done

Thanks for opensourcing the code.
You mentioned about advanced features can you explain about those features and if i combine inertial data with DSO lite is it going to be computationally cheaper than okvis or kimera.
Thank you

Hi,

I'm thinking how to accelerate DSOL further more, then i realize that DSOL uses lie group(9 DOF) for residual computing just like original DSO, however quoternion uses only 4 DOF.

So i wonder would it faster using quoternion? Since it can decrease the date amount thus increase the cache hit?

[ 0%] Building CXX object dsol/sv/util/CMakeFiles/sv_util_eigen.dir/eigen.cpp.o
[ 0%] Building CXX object dsol/sv/util/CMakeFiles/sv_util_cmap.dir/cmap.cpp.o
[ 0%] Building CXX object dsol/sv/util/CMakeFiles/sv_util_ocv.dir/ocv.cpp.o
[ 1%] Building CXX object dsol/sv/util/CMakeFiles/sv_util_summary.dir/summary.cpp.o
[ 1%] Building CXX object dsol/sv/util/CMakeFiles/sv_util_metric.dir/metric.cpp.o
[ 2%] Building CXX object dsol/sv/dsol/CMakeFiles/sv_dsol_image.dir/image.cpp.o
/root/catkin_ws/src/dsol/sv/util/cmap.cpp:1:10: fatal error: sv/util/cmap.h: No such file or directory
#include "sv/util/cmap.h"
^~~~~~~~~~~~~~~~
compilation terminated.
/root/catkin_ws/src/dsol/sv/util/eigen.cpp:1:10: fatal error: sv/util/eigen.h: No such file or directory
#include "sv/util/eigen.h"
^~~~~~~~~~~~~~~~~
compilation terminated.
dsol/sv/util/CMakeFiles/sv_util_cmap.dir/build.make:75: recipe for target 'dsol/sv/util/CMakeFiles/sv_util_cmap.dir/cmap.cpp.o' failed
make[2]: *** [dsol/sv/util/CMakeFiles/sv_util_cmap.dir/cmap.cpp.o] Error 1
dsol/sv/util/CMakeFiles/sv_util_eigen.dir/build.make:75: recipe for target 'dsol/sv/util/CMakeFiles/sv_util_eigen.dir/eigen.cpp.o' failed
make[2]: *** [dsol/sv/util/CMakeFiles/sv_util_eigen.dir/eigen.cpp.o] Error 1
/root/catkin_ws/src/dsol/sv/util/ocv.cpp:1:10: fatal error: sv/util/ocv.h: No such file or directory
#include "sv/util/ocv.h"
^~~~~~~~~~~~~~~
compilation terminated.
CMakeFiles/Makefile2:3853: recipe for target 'dsol/sv/util/CMakeFiles/sv_util_cmap.dir/all' failed
make[1]: *** [dsol/sv/util/CMakeFiles/sv_util_cmap.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
CMakeFiles/Makefile2:3827: recipe for target 'dsol/sv/util/CMakeFiles/sv_util_eigen.dir/all' failed
make[1]: *** [dsol/sv/util/CMakeFiles/sv_util_eigen.dir/all] Error 2
[ 2%] Built target gazebo_ros_gencfg
[ 2%] Building CXX object dsol/sv/ros1/CMakeFiles/sv_ros1_msg_conv.dir/msg_conv.cpp.o
dsol/sv/util/CMakeFiles/sv_util_ocv.dir/build.make:75: recipe for target 'dsol/sv/util/CMakeFiles/sv_util_ocv.dir/ocv.cpp.o' failed
make[2]: *** [dsol/sv/util/CMakeFiles/sv_util_ocv.dir/ocv.cpp.o] Error 1
CMakeFiles/Makefile2:3931: recipe for target 'dsol/sv/util/CMakeFiles/sv_util_ocv.dir/all' failed
make[1]: *** [dsol/sv/util/CMakeFiles/sv_util_ocv.dir/all] Error 2
/root/catkin_ws/src/dsol/sv/util/summary.cpp:1:10: fatal error: sv/util/summary.h: No such file or directory
#include "sv/util/summary.h"
^~~~~~~~~~~~~~~~~~~
compilation terminated.
dsol/sv/util/CMakeFiles/sv_util_summary.dir/build.make:75: recipe for target 'dsol/sv/util/CMakeFiles/sv_util_summary.dir/summary.cpp.o' failed
make[2]: *** [dsol/sv/util/CMakeFiles/sv_util_summary.dir/summary.cpp.o] Error 1
CMakeFiles/Makefile2:3879: recipe for target 'dsol/sv/util/CMakeFiles/sv_util_summary.dir/all' failed
make[1]: *** [dsol/sv/util/CMakeFiles/sv_util_summary.dir/all] Error 2
/root/catkin_ws/src/dsol/sv/dsol/image.cpp:1:10: fatal error: sv/dsol/image.h: No such file or directory
#include "sv/dsol/image.h"
^~~~~~~~~~~~~~~~~
compilation terminated.
/root/catkin_ws/src/dsol/sv/util/metric.cpp:1:10: fatal error: sv/util/metric.h: No such file or directory
#include "sv/util/metric.h"
^~~~~~~~~~~~~~~~~~
compilation terminated.
dsol/sv/dsol/CMakeFiles/sv_dsol_image.dir/build.make:75: recipe for target 'dsol/sv/dsol/CMakeFiles/sv_dsol_image.dir/image.cpp.o' failed
make[2]: *** [dsol/sv/dsol/CMakeFiles/sv_dsol_image.dir/image.cpp.o] Error 1
CMakeFiles/Makefile2:4036: recipe for target 'dsol/sv/dsol/CMakeFiles/sv_dsol_image.dir/all' failed
make[1]: *** [dsol/sv/dsol/CMakeFiles/sv_dsol_image.dir/all] Error 2
dsol/sv/util/CMakeFiles/sv_util_metric.dir/build.make:75: recipe for target 'dsol/sv/util/CMakeFiles/sv_util_metric.dir/metric.cpp.o' failed
make[2]: *** [dsol/sv/util/CMakeFiles/sv_util_metric.dir/metric.cpp.o] Error 1
CMakeFiles/Makefile2:4010: recipe for target 'dsol/sv/util/CMakeFiles/sv_util_metric.dir/all' failed
make[1]: *** [dsol/sv/util/CMakeFiles/sv_util_metric.dir/all] Error 2
In file included from /opt/ros/melodic/include/ros/duration.h:50:0,
from /opt/ros/melodic/include/ros/time.h:57,
from /opt/ros/melodic/include/ros/serialization.h:34,
from /opt/ros/melodic/include/geometry_msgs/Pose.h:14,
from /root/catkin_ws/src/dsol/sv/ros1/./msg_conv.h:3,
from /root/catkin_ws/src/dsol/sv/ros1/msg_conv.cpp:1:
/root/catkin_ws/src/dsol/sv/util/math.h:8:10: fatal error: sv/util/eigen.h: No such file or directory
#include "sv/util/eigen.h"
^~~~~~~~~~~~~~~~~
compilation terminated.
dsol/sv/ros1/CMakeFiles/sv_ros1_msg_conv.dir/build.make:75: recipe for target 'dsol/sv/ros1/CMakeFiles/sv_ros1_msg_conv.dir/msg_conv.cpp.o' failed
make[2]: *** [dsol/sv/ros1/CMakeFiles/sv_ros1_msg_conv.dir/msg_conv.cpp.o] Error 1
CMakeFiles/Makefile2:4673: recipe for target 'dsol/sv/ros1/CMakeFiles/sv_ros1_msg_conv.dir/all' failed
make[1]: *** [dsol/sv/ros1/CMakeFiles/sv_ros1_msg_conv.dir/all] Error 2
Makefile:145: recipe for target 'all' failed
make: *** [all] Error 2
Invoking "make -j8 -l8" failed

Hi, thank you for sharing great results!

I would like to run this source code but found out there is no calibration file for the data you are sharing.
Could you share the calibration file of the sample data you sharing?

Thank you in advance!