按教程在Jetpack 5.0.2上部署后，运行demo发现相机打不开
2024-05-06 12:25:47,732[2084:2084][camera_data_manager.py:195] Camera: 0, try cap: souphttpsrc timeout=0 location=http://127.0.0.1:38000/v1/camera/0 ! jpegparse ! nvjpegdec ! video/x-raw(memory:NVMM) ! nvvidconv ! video/x-raw,width=1920,height=1080,format=I420 ! tee name=t
t. ! queue ! nvjpegenc ! shmsink socket-path=/tmp/camera_jpeg_0 sync=false
t. ! queue ! appsink sync=false
2024-05-06 12:25:47,733[2084:2084][camera_data_manager.py:199] Camera: 0, can not open
2024-05-06 12:25:47,734[2084:2084][camera_data_manager.py:195] Camera: 1, try cap: souphttpsrc timeout=0 location=http://127.0.0.1:38000/v1/camera/1 ! jpegparse ! nvjpegdec ! video/x-raw(memory:NVMM) ! nvvidconv ! video/x-raw,width=1920,height=1080,format=I420 ! tee name=t
t. ! queue ! nvjpegenc ! shmsink socket-path=/tmp/camera_jpeg_1 sync=false
t. ! queue ! appsink sync=false
2024-05-06 12:25:47,734[2084:2084][camera_data_manager.py:199] Camera: 1, can not open
2024-05-06 12:25:47,734[2084:2084][camera_data_manager.py:195] Camera: 2, try cap: souphttpsrc timeout=0 location=http://127.0.0.1:38000/v1/camera/2 ! jpegparse ! nvjpegdec ! video/x-raw(memory:NVMM) ! nvvidconv ! video/x-raw,width=1920,height=1080,format=I420 ! tee name=t
t. ! queue ! nvjpegenc ! shmsink socket-path=/tmp/camera_jpeg_2 sync=false
t. ! queue ! appsink sync=false
2024-05-06 12:25:47,734[2084:2084][camera_data_manager.py:199] Camera: 2, can not open
2024-05-06 12:25:47,734[2084:2084][camera_data_manager.py:195] Camera: 3, try cap: souphttpsrc timeout=0 location=http://127.0.0.1:38000/v1/camera/3 ! jpegparse ! nvjpegdec ! video/x-raw(memory:NVMM) ! nvvidconv ! video/x-raw,width=1920,height=1080,format=I420 ! tee name=t
t. ! queue ! nvjpegenc ! shmsink socket-path=/tmp/camera_jpeg_3 sync=false
t. ! queue ! appsink sync=false
2024-05-06 12:25:47,735[2084:2084][camera_data_manager.py:199] Camera: 3, can not open

@w111liang222 , Thanks for making good software!

I have just built CPU-version using the auto-ipu docker image. This is because I have RTX 2080Ti which is not SM80+.

I managed to get the software running, but I don't see 3D cuboids when in detect+SLAM mode.

Perhaps the detection is not running at all?

I have attached screenshots of the log and the web UI.

我已经成功运行demo，但我把demo中detection里面lidar输入删除后，就没有detected objects,然后看了一下detection的代码，sensor_inference/object_infer.py,在这没看到单目3D的算法，只看到了点云检测的部分，Fusion部分我看介绍中谈到的是单目3D与点云检测的融合，但我看了sensor_fusion/fusion.py，感觉是Trafficlight和object Detection的融合，具体是在哪里做的单目3D算法，能告知一下吗，没有太看懂

Do you support other brands of radar? Is there a reference document?

Hi, I have run LCD with your dataset. I can visit the web UI at http://localhost:1234/, but can not see something like pointclouds in the web.

Is there something wrong with my configuration?

The running result is as follows:

$ tools/scripts/start_system.sh
run server python script
dpkg-query: no packages found matching nvidia-l4t-core
Running on board: IPC, machine: x86_64, jetpack: 20.04
2023-07-05 15:42:00.743[77059:77059][SystemUtils.cpp:34] Failed to setschedparam: Operation not permitted
[Errno 2] No such file or directory: '~/work/licence/licences'
2023-07-05 15:42:00.811[77061:77061][SystemUtils.cpp:34] Failed to setschedparam: Operation not permitted
2023-07-05 15:42:00,898[77061:77061][module_manager.py: 47] pipeline: Source connect to SLAM
2023-07-05 15:42:00,898[77061:77061][module_manager.py: 47] pipeline: SLAM connect to Sink
2023-07-05 15:42:00,898[77061:77061][module_manager.py: 71] start to setup module: SLAM
2023-07-05 15:42:00.898[77061:77067][SystemUtils.cpp:34] Failed to setschedparam: Operation not permitted
2023-07-05 15:42:00.898[77061:77067][localization.cpp:13] Localization: release static resources
2023-07-05 15:42:00.898[77061:77067][hdl_graph_slam_nodelet.cpp:123] releasing hdl graph nodelet...
2023-07-05 15:42:00,898[77061:77067][        slam.py: 51] SLAM use sensors: ['IMU', '1-Ouster-OS1-128']
2023-07-05 15:42:00,899[77061:77061][module_manager.py: 73] setup module: SLAM, done
2023-07-05 15:42:00,899[77061:77061][module_manager.py: 71] start to setup module: Sink
iKD-Tree Multi thread started 
2023-07-05 15:42:00,905[77068:77068][sink_template.py: 58] deamon UdpSink starts
2023-07-05 15:42:00.906[77061:77067][prefiltering_nodelet.cpp:39] downsample: VOXELGRID 0.2
2023-07-05 15:42:00.906[77068:77061][SystemUtils.cpp:34] Failed to setschedparam: Operation not permitted
2023-07-05 15:42:00.906[77061:77067][prefiltering_nodelet.cpp:78] outlier_removal: NONE
2023-07-05 15:42:00.906[77061:77067][hdl_graph_slam_nodelet.cpp:52] initializing hdl graph nodelet...
2023-07-05 15:42:00.906[77061:77067][graph_slam.cpp:56] construct solver: lm_var
SOLVER FACTORY WARNING: Unable to create solver lm_var
2023-07-05 15:42:00.906[77061:77067][graph_slam.cpp:63] error : failed to allocate solver!!
gn_var_cholmod       CHOLMOD 	Gauss-Newton: Cholesky solver using CHOLMOD (variable blocksize)
gn_fix3_2_cholmod    CHOLMOD 	Gauss-Newton: Cholesky solver using CHOLMOD (fixed blocksize)
gn_fix6_3_cholmod    CHOLMOD 	Gauss-Newton: Cholesky solver using CHOLMOD (fixed blocksize)
gn_fix7_3_cholmod    CHOLMOD 	Gauss-Newton: Cholesky solver using CHOLMOD (fixed blocksize)
lm_var_cholmod       CHOLMOD 	Levenberg: Cholesky solver using CHOLMOD (variable blocksize)
lm_fix3_2_cholmod    CHOLMOD 	Levenberg: Cholesky solver using CHOLMOD (fixed blocksize)
lm_fix6_3_cholmod    CHOLMOD 	Levenberg: Cholesky solver using CHOLMOD (fixed blocksize)
lm_fix7_3_cholmod    CHOLMOD 	Levenberg: Cholesky solver using CHOLMOD (fixed blocksize)2023-07-05 15:42:00,906[77061:77061][module_manager.py: 73] setup module: Sink, done

dl_var_cholmod       CHOLMOD 	Dogleg: Cholesky solver using CHOLMOD (variable blocksize)
gn_var_csparse       CSparse 	Gauss-Newton: Cholesky solver using CSparse (variable blocksize)
gn_fix3_2_csparse    CSparse 	Gauss-Newton: Cholesky solver using CSparse (fixed blocksize)
gn_fix6_3_csparse    CSparse 	Gauss-Newton: Cholesky solver using CSparse (fixed blocksize)
gn_fix7_3_csparse    CSparse 	Gauss-Newton: Cholesky solver using CSparse (fixed blocksize)
lm_var_csparse       CSparse 	Levenberg: Cholesky solver using CSparse (variable blocksize)
lm_fix3_2_csparse    CSparse 	Levenberg: Cholesky solver using CSparse (fixed blocksize)
lm_fix6_3_csparse    CSparse 	Levenberg: Cholesky solver using CSparse (fixed blocksize)
lm_fix7_3_csparse    CSparse 	Levenberg: Cholesky solver using CSparse (fixed blocksize)2023-07-05 15:42:00,906[77061:77061][module_manager.py: 71] start to setup module: Source

dl_var_csparse       CSparse 	Dogleg: Cholesky solver using CSparse (variable blocksize)
gn_pcg               PCG 	Gauss-Newton: PCG solver using block-Jacobi pre-conditioner (variable blocksize)
gn_pcg3_2            PCG 	Gauss-Newton: PCG solver using block-Jacobi pre-conditioner (fixed blocksize)
gn_pcg6_3            PCG 	Gauss-Newton: PCG solver using block-Jacobi pre-conditioner (fixed blocksize)
gn_pcg7_3            PCG 	Gauss-Newton: PCG solver using block-Jacobi pre-conditioner (fixed blocksize)
lm_pcg               PCG 	Levenberg: PCG solver using block-Jacobi pre-conditioner (variable blocksize)
lm_pcg3_2            PCG 	Levenberg: PCG solver using block-Jacobi pre-conditioner (fixed blocksize)
lm_pcg6_3            PCG 	Levenberg: PCG solver using block-Jacobi pre-conditioner (fixed blocksize)
lm_pcg7_3            PCG 	Levenberg: PCG solver using block-Jacobi pre-conditioner (fixed blocksize)
-------------
2023-07-05 15:42:00,910[77061:77061][player_data_manager.py:135] player have 201 frames, duration 19.971323 s
2023-07-05 15:42:01.007[77070:77061][SystemUtils.cpp:34] Failed to setschedparam: Operation not permitted
 * Serving Flask app 'module.source.player_data_manager' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
2023-07-05 15:42:01,009[77061:77061][source_manager.py: 35] Source, start to setup: lidar
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:38000
Press CTRL+C to quit
2023-07-05 15:42:01.012[77061:77061][lidar_driver_wrapper.cpp:34] lidar 0:Ouster-OS1-64 open success
2023-07-05 15:42:01.012[77061:77061][lidar_driver_wrapper.cpp:46] set lidar 0:Ouster-OS1-64 external parameter [-0.03368380293250084, -0.660768985748291, 1.7658969163894653, 177.67449951171875, 135.47140502929688, -177.0221710205078]
2023-07-05 15:42:01.012[77061:77061][lidar_driver_wrapper.cpp:58] set lidar 0:Ouster-OS1-64 range filter [-72.0, -72.0, -2.0, 72.0, 72.0, 4.0]
2023-07-05 15:42:01.012[77061:77061][lidar_driver_wrapper.cpp:70] set lidar 0:Ouster-OS1-64 exclude [-2.13, -2.01, -5.0, 2.92, 0.5, 5.0]
2023-07-05 15:42:01.015[77061:77061][lidar_driver_wrapper.cpp:34] lidar 1:Ouster-OS1-128 open success
2023-07-05 15:42:01.015[77061:77061][lidar_driver_wrapper.cpp:46] set lidar 1:Ouster-OS1-128 external parameter [0.0, 0.0, 1.88961923122406, 179.99972534179688, 179.9855194091797, -177.86497497558594]
2023-07-05 15:42:01.015[77061:77061][lidar_driver_wrapper.cpp:58] set lidar 1:Ouster-OS1-128 range filter [-72.0, -72.0, -2.0, 72.0, 72.0, 4.0]
2023-07-05 15:42:01.015[77061:77061][lidar_driver_wrapper.cpp:70] set lidar 1:Ouster-OS1-128 exclude [-3.37, -3.02, -4.15, 3.16, 2.33, 4.7]
2023-07-05 15:42:01.017[77061:77061][lidar_driver_wrapper.cpp:34] lidar 2:Ouster-OS1-64 open success
2023-07-05 15:42:01.017[77061:77061][lidar_driver_wrapper.cpp:46] set lidar 2:Ouster-OS1-64 external parameter [-0.09995567798614502, 0.6768492460250854, 1.7785784006118774, 1.0974562168121338, -44.20530319213867, 2.35439395904541]
2023-07-05 15:42:01.017[77061:77061][lidar_driver_wrapper.cpp:58] set lidar 2:Ouster-OS1-64 range filter [-72.0, -72.0, -2.0, 72.0, 72.0, 4.0]
2023-07-05 15:42:01.017[77061:77061][lidar_driver_wrapper.cpp:70] set lidar 2:Ouster-OS1-64 exclude [-3.37, -3.14, -4.92, 3.16, 1.44, 4.58]
2023-07-05 15:42:01,017[77061:77061][source_manager.py: 38] Source, setup: lidar, done
2023-07-05 15:42:01,018[77061:77061][source_manager.py: 35] Source, start to setup: camera
2023-07-05 15:42:01,048[77061:77061][camera_data_manager.py:207] camera 0, try cap: souphttpsrc timeout=0 location=http://127.0.0.1:38000/v1/camera/0 ! jpegparse ! jpegdec ! video/x-raw  ! videoscale ! video/x-raw,width=1920,height=1080  ! videoconvert ! video/x-raw,width=1920,height=1080,format=I420 ! tee name=t
    t. ! queue ! jpegenc ! shmsink socket-path=/tmp/camera_jpeg_0 sync=false
    t. ! queue ! appsink sync=false 
127.0.0.1 - - [05/Jul/2023 15:42:01] "GET /v1/camera/0 HTTP/1.1" 200 -
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=0, duration=-1
2023-07-05 15:42:01,090[77061:77061][camera_data_manager.py:217] camera 0 open success
2023-07-05 15:42:01,090[77061:77061][camera_data_manager.py:207] camera 1, try cap: souphttpsrc timeout=0 location=http://127.0.0.1:38000/v1/camera/1 ! jpegparse ! jpegdec ! video/x-raw  ! videoscale ! video/x-raw,width=1920,height=1080  ! videoconvert ! video/x-raw,width=1920,height=1080,format=I420 ! tee name=t
    t. ! queue ! jpegenc ! shmsink socket-path=/tmp/camera_jpeg_1 sync=false
    t. ! queue ! appsink sync=false 
127.0.0.1 - - [05/Jul/2023 15:42:01] "GET /v1/camera/1 HTTP/1.1" 200 -
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=0, duration=-1
2023-07-05 15:42:01,109[77061:77061][camera_data_manager.py:217] camera 1 open success
2023-07-05 15:42:01,109[77061:77061][camera_data_manager.py:207] camera 2, try cap: souphttpsrc timeout=0 location=http://127.0.0.1:38000/v1/camera/2 ! jpegparse ! jpegdec ! video/x-raw  ! videoscale ! video/x-raw,width=1920,height=1080  ! videoconvert ! video/x-raw,width=1920,height=1080,format=I420 ! tee name=t
    t. ! queue ! jpegenc ! shmsink socket-path=/tmp/camera_jpeg_2 sync=false
    t. ! queue ! appsink sync=false 
127.0.0.1 - - [05/Jul/2023 15:42:01] "GET /v1/camera/2 HTTP/1.1" 200 -
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=0, duration=-1
2023-07-05 15:42:01,127[77061:77061][camera_data_manager.py:217] camera 2 open success
2023-07-05 15:42:01,127[77061:77061][camera_data_manager.py:207] camera 3, try cap: souphttpsrc timeout=0 location=http://127.0.0.1:38000/v1/camera/3 ! jpegparse ! jpegdec ! video/x-raw  ! videoscale ! video/x-raw,width=1920,height=1080  ! videoconvert ! video/x-raw,width=1920,height=1080,format=I420 ! tee name=t
    t. ! queue ! jpegenc ! shmsink socket-path=/tmp/camera_jpeg_3 sync=false
    t. ! queue ! appsink sync=false 
127.0.0.1 - - [05/Jul/2023 15:42:01] "GET /v1/camera/3 HTTP/1.1" 200 -
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=0, duration=-1
2023-07-05 15:42:01,172[77061:77061][camera_data_manager.py:217] camera 3 open success
2023-07-05 15:42:01,172[77061:77061][source_manager.py: 38] Source, setup: camera, done
2023-07-05 15:42:01,172[77061:77061][source_manager.py: 35] Source, start to setup: radar
2023-07-05 15:42:01,172[77061:77061][source_manager.py: 38] Source, setup: radar, done
2023-07-05 15:42:01,172[77061:77061][source_manager.py: 35] Source, start to setup: ins
2023-07-05 15:42:01.172[77061:77061][ins_driver_wrapper.cpp:19] ins open success
2023-07-05 15:42:01.172[77061:77061][ins_driver_wrapper.cpp:34] set ins external parameter [0.09695542603731155, 1.2899638414382935, -0.11553408205509186, 87.35172271728516, -0.02054435946047306, -0.8822724223136902]
2023-07-05 15:42:01.173[77061:77061][SystemUtils.cpp:23] Failed to setschedparam: Operation not permitted
2023-07-05 15:42:01.173[77061:77061][SystemUtils.cpp:23] Failed to setschedparam: Operation not permitted
2023-07-05 15:42:01.173[77061:77061][ins_driver_wrapper.cpp:43] ins start success
2023-07-05 15:42:01.173[77061:77112][ins_driver.cpp:317] start ins com receving, device /dev/ttyUSB0
2023-07-05 15:42:01.173[77061:77113][ins_driver.cpp:378] start ins gps receving
2023-07-05 15:42:01.173[77061:77111][ins_driver.cpp:278] start ins udp receving, port 9888
2023-07-05 15:42:01,173[77061:77061][source_manager.py: 38] Source, setup: ins, done
2023-07-05 15:42:01,173[77061:77061][module_manager.py: 73] setup module: Source, done
2023-07-05 15:42:01.173[77061:77114][SystemUtils.cpp:34] Failed to setschedparam: Operation not permitted
2023-07-05 15:42:01.173[77061:77113][ins_driver.cpp:381] No GPSD running
2023-07-05 15:42:01,173[77061:77114][manager_template.py: 80] SLAM, FPS: 1.8
2023-07-05 15:42:01.173[77061:77115][SystemUtils.cpp:34] Failed to setschedparam: Operation not permitted
2023-07-05 15:42:01,173[77061:77116][data_manager_template.py: 57] Camera: 0: acquire_loop start
2023-07-05 15:42:01.174[77061:77116][SystemUtils.cpp:34] Failed to setschedparam: Operation not permitted
2023-07-05 15:42:01,173[77061:77061][  perception.py: 34] start success
2023-07-05 15:42:01,173[77061:77118][data_manager_template.py: 57] Camera: 1: acquire_loop start
2023-07-05 15:42:01.176[77061:77118][SystemUtils.cpp:34] Failed to setschedparam: Operation not permitted
2023-07-05 15:42:01,174[77061:77119][data_manager_template.py: 57] Camera: 2: acquire_loop start
2023-07-05 15:42:01.176[77061:77119][SystemUtils.cpp:34] Failed to setschedparam: Operation not permitted
2023-07-05 15:42:01,176[77061:77120][data_manager_template.py: 57] Camera: 3: acquire_loop start
2023-07-05 15:42:01.176[77061:77120][SystemUtils.cpp:34] Failed to setschedparam: Operation not permitted
[ WARN:1] global ../modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
[ WARN:2] global ../modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
[ WARN:3] global ../modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
[ WARN:4] global ../modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
2023-07-05 15:42:01.178[77061:77136][SystemUtils.cpp:34] Failed to setschedparam: Operation not permitted
2023-07-05 15:42:01.178[77061:77137][SystemUtils.cpp:34] Failed to setschedparam: Operation not permitted
2023-07-05 15:42:01.178[77061:77138][SystemUtils.cpp:34] Failed to setschedparam: Operation not permitted
2023-07-05 15:42:01.178[77061:77139][SystemUtils.cpp:34] Failed to setschedparam: Operation not permitted
2023-07-05 15:42:01,178[77061:77115][manager_template.py: 80] Source, FPS: 1.8
2023-07-05 15:42:03,197[77061:77115][manager_template.py: 80] Source, FPS: 1.1
/!\ gevent_zeromq BUG /!\ catching up after missing event (RECV) /!\
/!\ gevent_zeromq BUG /!\ catching up after missing event (RECV) /!\
2023-07-05 15:42:05,217[77061:77115][manager_template.py: 80] Source, FPS: 0.8
2023-07-05 15:42:07,238[77061:77115][manager_template.py: 80] Source, FPS: 0.7
/!\ gevent_zeromq BUG /!\ catching up after missing event (RECV) /!\
2023-07-05 15:42:09,258[77061:77115][manager_template.py: 80] Source, FPS: 0.6
/!\ gevent_zeromq BUG /!\ catching up after missing event (RECV) /!\

Thanks to your great job! I want to combine this project with autoware planning core, and i found every main nodes was written by python. Do you have some ideas to convert them to C++? Thank you!

ModuleNotFoundError: No module named 'cpp_utils_ext'
Traceback (most recent call last):
File "web_backend/server.py", line 36, in <module>
from perception_server import PerceptionServer
File "/home/nvidia/znqc/work/lidar-slam-detection/web_backend/perception_server.py", line 5, in <module>
from jsonrpc.backend.flask import api
ModuleNotFoundError: No module named 'jsonrpc.backend'

I use the rosbag_proxy tool to send rosbag messages to LSD,But it seems that it can only run in Acquisition mode, not in slam mode

Thanks for your great job.
I'm new to slam. I only want to use camera and imu to realize location and mapping for demo experiment, and do not need pointcloud.(To be honest, I don't have lidar)

If the input of radar data is removed from the code, can it run normally and get proper result?
what is the sub-module of such function?

Hi, I'm trying to test your program following the instructions here : https://github.com/w111liang222/lidar-slam-detection/blob/main/docs/slam.md
When I launch the LSD and open localhost:config , I cannot change the setting about mapping & localization. I just see the waiting page, as the picture below

你好，我在用激光雷达相机联合标定，完成点对标定以后，进入微调环节，明显对不上，微调的幅度完全修正不过来，我不明白这是怎么回事。另外，问一下，在联合标定之前需要先单独标定激光雷达吗？激光雷达标定的坐标系是什么

请问调整obj的显示ui如何才能生效？

你好，我在验证lidar坐标转像素坐标系的效果，请问像素坐标的是怎么约定的？

box2d_corner 这个坐标系

Hello, your work is excellent. I would like to use ROS to test the object detection performance of this code in a real-world scenario. What should I do? I don't have a GPU, and I'm using the Helios lidar and RealSense camera. I'm using the non-CUDA version (auto-ipu) of the code. Docker is already able to receive information from the lidar and camera, but it seems that object detection is not running. Is it because I haven't calibrated it? Below is the terminal information.

When I open http://localhost/editor When there is no 'merge map' option under the file, I have already pulled the latest library. The following is the output log information at startup

root@dns:/home/znqc/work/lidar-slam-detection# tools/scripts/start_system.sh
run server python script
dpkg-query: no packages found matching nvidia-l4t-core
Running on board: IPC, machine: x86_64, jetpack: 20.04
2023-04-28 09:42:16,937[27174:27174][module_manager.py: 47] pipeline: Source connect to SLAM
2023-04-28 09:42:16,937[27174:27174][module_manager.py: 47] pipeline: SLAM connect to Sink
2023-04-28 09:42:16,937[27174:27174][module_manager.py: 71] start to setup module: SLAM
2023-04-28 09:42:16.937[27174:27182][localization.cpp:13] Localization: release static resources
2023-04-28 09:42:16.937[27174:27182][hdl_graph_slam_nodelet.cpp:122] releasing hdl graph nodelet...
2023-04-28 09:42:16,937[27174:27182][ slam.py: 51] SLAM use sensors: ['IMU', '0-RS-LiDAR-16', 'flir:0']
2023-04-28 09:42:16,937[27174:27174][module_manager.py: 73] setup module: SLAM, done
2023-04-28 09:42:16,937[27174:27174][module_manager.py: 71] start to setup module: Sink
iKD-Tree Multi thread started
2023-04-28 09:42:16.942[27174:27182][prefiltering_nodelet.cpp:39] downsample: VOXELGRID 0.15
2023-04-28 09:42:16.942[27174:27182][prefiltering_nodelet.cpp:78] outlier_removal: NONE
2023-04-28 09:42:16.942[27174:27182][hdl_graph_slam_nodelet.cpp:51] initializing hdl graph nodelet...
2023-04-28 09:42:16.942[27174:27182][graph_slam.cpp:56] construct solver: lm_var

Using CSparse poseDim -1 landMarkDim -1 blockordering 0

2023-04-28 09:42:16.942[27174:27182][loop_detector.hpp:50] initializing loop detection...
2023-04-28 09:42:16.942[27174:27182][registrations.cpp:57] registration: FAST_VGICP
2023-04-28 09:42:16.942[27174:27182][registrations.cpp:34] registration: FAST_GICP
2023-04-28 09:42:16,950[27188:27188][sink_template.py: 58] deamon UdpSink starts
2023-04-28 09:42:16,950[27174:27174][module_manager.py: 73] setup module: Sink, done
2023-04-28 09:42:16,950[27174:27174][module_manager.py: 71] start to setup module: Source
2023-04-28 09:42:16,954[27174:27174][player_data_manager.py:135] /home/znqc/work/slam_mapping_demo
2023-04-28 09:42:16,954[27174:27174][player_data_manager.py:136] player have 1942 frames, duration 194.099473 s

• Serving Flask app 'module.source.player_data_manager'
• Debug mode: off
  2023-04-28 09:42:17,007[27174:27174][source_manager.py: 35] Source, start to setup: lidar
  WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
• Running on http://127.0.0.1:38000
  Press CTRL+C to quit
  2023-04-28 09:42:17.009[27174:27174][lidar_driver_wrapper.cpp:34] lidar 0:RS-LiDAR-16 open success
  2023-04-28 09:42:17.009[27174:27174][lidar_driver_wrapper.cpp:46] set lidar 0:RS-LiDAR-16 external parameter [0.0, 0.0, 0.0, 178.35, 0.0, 0.0]
  2023-04-28 09:42:17.009[27174:27174][lidar_driver_wrapper.cpp:58] set lidar 0:RS-LiDAR-16 range filter [-200.0, -200.0, -50.0, 200.0, 200.0, 50.0]
  2023-04-28 09:42:17.009[27174:27174][lidar_driver_wrapper.cpp:70] set lidar 0:RS-LiDAR-16 exclude [-1.0, -1.0, -2.5, 1.0, 1.0, 2.5]
  2023-04-28 09:42:17,009[27174:27174][source_manager.py: 38] Source, setup: lidar, done
  2023-04-28 09:42:17,010[27174:27174][source_manager.py: 35] Source, start to setup: camera
  2023-04-28 09:42:17,015[27174:27174][camera_data_manager.py:207] camera flir:0, try cap: souphttpsrc timeout=0 location=http://127.0.0.1:38000/v1/camera/0 ! jpegparse ! jpegdec ! video/x-raw ! videoscale ! video/x-raw,width=1024,height=768 ! videoconvert ! video/x-raw,width=1024,height=768,format=I420 ! tee name=t
  t. ! queue ! jpegenc ! shmsink socket-path=/tmp/camera_jpeg_0 sync=false
  t. ! queue ! appsink sync=false
  127.0.0.1 - - [28/Apr/2023 09:42:17] "GET /v1/camera/0 HTTP/1.1" 200 -
  [ WARN:[email protected]] global /tmp/opencv-4.5.5/modules/videoio/src/cap_gstreamer.cpp (1405) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=0, duration=-1
  2023-04-28 09:42:17,041[27174:27174][camera_data_manager.py:217] camera flir:0 open success
  2023-04-28 09:42:17,041[27174:27174][source_manager.py: 38] Source, setup: camera, done
  2023-04-28 09:42:17,041[27174:27174][source_manager.py: 35] Source, start to setup: radar
  2023-04-28 09:42:17,041[27174:27174][source_manager.py: 38] Source, setup: radar, done
  2023-04-28 09:42:17,041[27174:27174][source_manager.py: 35] Source, start to setup: ins
  2023-04-28 09:42:17.041[27174:27174][ins_driver_wrapper.cpp:19] ins open success
  2023-04-28 09:42:17.041[27174:27174][ins_driver_wrapper.cpp:34] set ins external parameter [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
  2023-04-28 09:42:17.042[27174:27174][ins_driver_wrapper.cpp:43] ins start success
  2023-04-28 09:42:17.042[27174:27213][ins_driver.cpp:278] start ins udp receving, port 9888
  2023-04-28 09:42:17.042[27174:27214][ins_driver.cpp:317] start ins com receving, device /dev/ttyIMU_GPS0
  2023-04-28 09:42:17.042[27174:27215][ins_driver.cpp:378] start ins gps receving
  2023-04-28 09:42:17,042[27174:27174][source_manager.py: 38] Source, setup: ins, done
  2023-04-28 09:42:17,042[27174:27174][module_manager.py: 73] setup module: Source, done
  2023-04-28 09:42:17.042[27174:27215][ins_driver.cpp:381] No GPSD running
  2023-04-28 09:42:17,042[27174:27216][manager_template.py: 80] SLAM, FPS: 4.8
  2023-04-28 09:42:17,042[27174:27218][data_manager_template.py: 57] Camera: flir:0: acquire_loop start
  2023-04-28 09:42:17,042[27174:27174][ perception.py: 34] start success
  [ WARN:[email protected]] global /tmp/opencv-4.5.5/modules/videoio/src/cap_gstreamer.cpp (1405) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
  2023-04-28 09:42:17,044[27174:27217][manager_template.py: 80] Source, FPS: 4.7
  2023-04-28 09:42:17.158[27174:27184][IMU_Processing.hpp:362] IMU Initial Done
  2023-04-28 09:42:17.158[27174:27184][laserMapping.cpp:974] FastLio undistort points is empty
  2023-04-28 09:42:18,044[27174:27219][module_manager.py:116] system status Running, running thread/process information:
  2023-04-28 09:42:18,044[27174:27219][module_manager.py:120] Thread 27174:Perception daemon False ident 0x7f0b7fd44c00
  2023-04-28 09:42:18,045[27174:27219][module_manager.py:120] Thread 27182:FastLIO daemon True ident 0x7f0b5bec6700
  2023-04-28 09:42:18,045[27174:27219][module_manager.py:120] Thread 27216:SLAM daemon True ident 0x7f0b267fc700
  2023-04-28 09:42:18,045[27174:27219][module_manager.py:120] Thread 27217:Source daemon True ident 0x7f0b25ffb700
  2023-04-28 09:42:18,045[27174:27219][module_manager.py:120] Thread 27218:Py-Cflir:0 daemon True ident 0x7f0b257fa700
  2023-04-28 09:42:18,045[27174:27219][module_manager.py:120] Thread 27221:zerorpc daemon True ident 0x7f0b0f7fe700
  2023-04-28 09:42:18,045[27174:27219][module_manager.py:120] Thread 27226:Py-Jflir:0 daemon True ident 0x7f0b0ffff700
  2023-04-28 09:42:21.471[27174:27187][hdl_graph_slam_nodelet.cpp:231] map update, average score 0.0, best score 0.19276465879244892, inlier ratio 0.7278611, dx 0.9071208, da 31.995329
  2023-04-28 09:42:21.471[27174:27187][hdl_graph_slam_nodelet.cpp:236] map initializing, odometry not stable, resetting...
  2023-04-28 09:42:22.372[27174:27187][hdl_graph_slam_nodelet.cpp:231] map update, average score 0.0, best score 0.12182340285386119, inlier ratio 0.84300596, dx 0.23566943, da 34.099247
  2023-04-28 09:42:22.372[27174:27187][hdl_graph_slam_nodelet.cpp:234] map initializing, do not allow only rotation, resetting...
  2023-04-28 09:42:22.972[27174:27187][hdl_graph_slam_nodelet.cpp:231] map update, average score 0.0, best score 0.08593199736664416, inlier ratio 0.86002994, dx 0.22220375, da 35.935818
  2023-04-28 09:42:22.972[27174:27187][hdl_graph_slam_nodelet.cpp:234] map initializing, do not allow only rotation, resetting...
  2023-04-28 09:42:26.076[27174:27187][hdl_graph_slam_nodelet.cpp:231] map update, average score 0.0, best score 0.08094863986928584, inlier ratio 0.91425645, dx 1.5321901, da 3.5760264
  2023-04-28 09:42:26.076[27174:27187][hdl_graph_slam_nodelet.cpp:239] map initialize success
  2023-04-28 09:42:28,148[27174:27219][module_manager.py:137] Process 27188:UdpSink CPU 0.0%, Mem 132 MB
  2023-04-28 09:42:28,249[27174:27219][module_manager.py:137] Process 27189:playback CPU 0.0%, Mem 153 MB
  2023-04-28 09:42:28,349[27174:27219][module_manager.py:137] Process 27174:Perception CPU 0.0%, Mem 272 MB
  2023-04-28 09:42:28.375[27174:27187][hdl_graph_slam_nodelet.cpp:231] map update, average score 0.08094863986928584, best score 0.07137393419893548, inlier ratio 0.9357572, dx 1.5206559, da 12.783187
  2023-04-28 09:42:30.277[27174:27187][hdl_graph_slam_nodelet.cpp:231] map update, average score 0.07616128703411065, best score 0.04000307471355479, inlier ratio 0.9788242, dx 1.0415105, da 32.237644
  2023-04-28 09:42:31.979[27174:27187][hdl_graph_slam_nodelet.cpp:231] map update, average score 0.06410854959392537, best score 0.047140600227384206, inlier ratio 0.76601595, dx 1.1509631, da 31.542225
  2023-04-28 09:42:31.979[27174:27187][hdl_graph_slam_nodelet.cpp:252] map update, keep finding better keyframe
  2023-04-28 09:42:32.079[27174:27187][hdl_graph_slam_nodelet.cpp:231] map update, average score 0.06410854959392537, best score 0.047140600227384206, inlier ratio 0.76601595, dx 1.2203963, da 34.50612
  2023-04-28 09:42:32.079[27174:27187][hdl_graph_slam_nodelet.cpp:252] map update, keep finding better keyframe
  2023-04-28 09:42:32.181[27174:27187][hdl_graph_slam_nodelet.cpp:231] map update, average score 0.06410854959392537, best score 0.047140600227384206, inlier ratio 0.76601595, dx 1.295662, da 36.685013
  2023-04-28 09:42:32.181[27174:27187][hdl_graph_slam_nodelet.cpp:252] map update, keep finding better keyframe

Hi there, I found that in mapping/fastlio/src/preprocess.cpp [velodyne_handler] function, you set the timestamp of each point in a scan to 0? That may be a little different from offical-fastlio. In fastlio, they will use imu to do forward broadcast and undistor every point in a scan. This funcation is located at **IMU_Processing.hpp" line 219 sort(xxxx).
Maybe you forgot?

如果增加轮速数据结合imu数据应该能在隧道或楼宇之间等rtk失锁状态有较好的约束效果吧？

Thank you for open source this excellent work, it is worthy of my serious study. I noticed that the data you provided are all in python's .pkl format. If I want to test other SLAM datasets such as NCLT, do I have to process them into .pkl files first? Or do you have other ideas?Looking forward to your reply (^▽^)

@w111liang222 切换到非实时模式，总是会卡死，不成功。帮忙看一下，谢谢！
2024-05-27 07:00:01,840[10667:10667][module_manager.py: 47] pipeline: Source connect to Sink
2024-05-27 07:00:01,840[10667:10667][module_manager.py: 71] start to setup module: Sink
2024-05-27 07:00:01,851[10667:10667][module_manager.py: 73] setup module: Sink, done
2024-05-27 07:00:01,852[10667:10667][module_manager.py: 71] start to setup module: Source
2024-05-27 07:00:01,852[10667:10667][source_manager.py: 31] Source, start to setup: lidar
2024-05-27 07:00:01,853[10676:10676][sink_template.py: 50] deamon UdpSink starts
2024-05-27 07:00:01.858[10667:10667][lidar_driver_wrapper.cpp:34] lidar 0:RS-Helios open success
2024-05-27 07:00:01.858[10667:10667][lidar_driver_wrapper.cpp:46] set lidar 0:RS-Helios external parameter [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
2024-05-27 07:00:01.858[10667:10667][lidar_driver_wrapper.cpp:58] set lidar 0:RS-Helios range filter [-72.0, -72.0, -2.0, 72.0, 72.0, 4.0]
2024-05-27 07:00:01.858[10667:10667][lidar_driver_wrapper.cpp:70] set lidar 0:RS-Helios exclude [-0.5, -0.5, 0.0, 0.5, 0.5, 0.0]
2024-05-27 07:00:01,858[10667:10667][source_manager.py: 34] Source, setup: lidar, done
2024-05-27 07:00:01,859[10667:10667][source_manager.py: 31] Source, start to setup: camera
11111 rtsp://admin:[email protected]:554/Streaming/Channels/101
22222 0
33333
44444
55555
66666 ,width=1920,height=1080
2024-05-27 07:00:01,917[10667:10667][camera_data_manager.py:201] Camera: rtsp://admin:[email protected]:554/Streaming/Channels/101, try cap: rtspsrc location=rtsp://admin:[email protected]:554/Streaming/Channels/101 latency=0 ! decodebin ! videoflip method=0 ! video/x-raw ! videoscale ! video/x-raw,width=1920,height=1080 ! videoconvert ! video/x-raw,width=1920,height=1080,format=I420 ! tee name=t
t. ! queue leaky=2 ! shmsink socket-path=/tmp/camera_I420_0 sync=false
t. ! queue ! jpegenc ! shmsink socket-path=/tmp/camera_jpeg_0 sync=false
t. ! queue ! appsink sync=false

[ WARN:[email protected]] global /tmp/opencv-4.5.5/modules/videoio/src/cap_gstreamer.cpp (1374) open OpenCV | GStreamer warning: unable to query duration of stream
[ WARN:[email protected]] global /tmp/opencv-4.5.5/modules/videoio/src/cap_gstreamer.cpp (1405) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=0, duration=-1
2024-05-27 07:00:02,213[10667:10667][camera_data_manager.py:209] Camera: rtsp://admin:[email protected]:554/Streaming/Channels/101, run service: third_party/x86_64/launch_rtsp_server -p 554 -m /cam "shmsrc is-live=true socket-path=/tmp/camera_I420_0 do-timestamp=1 ! video/x-raw,width=1920,height=1080,framerate=(fraction)0,format=I420 ! queue ! x264enc speed-preset=1 threads=8 tune=zerolatency sliced-threads=false bitrate=976 ! h264parse ! rtph264pay pt=96 name=pay0 " &
2024-05-27 07:00:02,232[10667:10667][camera_data_manager.py:211] Camera: rtsp://admin:[email protected]:554/Streaming/Channels/101 open success
2024-05-27 07:00:02,233[10667:10667][source_manager.py: 34] Source, setup: camera, done
2024-05-27 07:00:02,233[10667:10667][source_manager.py: 31] Source, start to setup: radar
2024-05-27 07:00:02,234[10667:10667][source_manager.py: 34] Source, setup: radar, done
2024-05-27 07:00:02,234[10667:10667][source_manager.py: 31] Source, start to setup: ins
2024-05-27 07:00:02.234[10667:10667][ins_driver_wrapper.cpp:18] INS open success
2024-05-27 07:00:02.235[10667:10667][ins_driver_wrapper.cpp:29] set INS extrinsic parameter [0.09695542603731155, 1.2899638414382935, -0.11553408205509186, 87.35172271728516, -0.02054435946047306, -0.8822724223136902]
2024-05-27 07:00:02.235[10667:10753][ins_driver.cpp:338] start ins udp receving, port 9888
2024-05-27 07:00:02.235[10667:10754][ins_driver.cpp:383] start ins com receving, device /dev/ttyUSB0
2024-05-27 07:00:02.235[10667:10755][ins_driver.cpp:442] start ins gps receving
2024-05-27 07:00:02.236[10667:10667][ins_driver_wrapper.cpp:34] INS start success
2024-05-27 07:00:02.236[10667:10755][ins_driver.cpp:445] No GPSD running
2024-05-27 07:00:02,236[10667:10667][source_manager.py: 34] Source, setup: ins, done
2024-05-27 07:00:02,236[10667:10667][module_manager.py: 73] setup module: Source, done
2024-05-27 07:00:02,239[10667:10759][data_manager_template.py: 53] Lidar: 0-RS-Helios: acquire_loop start
2024-05-27 07:00:02.239[10667:10758][lidar_driver_wrapper.cpp:92] lidar 0:RS-Helios start success
2024-05-27 07:00:02.239[10667:10761][lidar_driver.cpp:24] start lidar: RS-Helios, receveing, port 6699
2024-05-27 07:00:02,239[10667:10667][ perception.py: 34] start success
stream ready at rtsp://0.0.0.0:554/cam
2024-05-27 07:00:02,240[10667:10762][data_manager_template.py: 53] Camera: rtsp://admin:[email protected]:554/Streaming/Channels/101: acquire_loop start
2024-05-27 07:00:02,287[10667:10758][camera_data_manager.py:313] JPEG: rtsp://admin:[email protected]:554/Streaming/Channels/101, try cap: shmsrc is-live=true socket-path=/tmp/camera_jpeg_0 do-timestamp=1 ! queue ! image/jpeg,width=1920,height=1080,framerate=(fraction)0 ! appsink sync=false
[ WARN:[email protected]] global /tmp/opencv-4.5.5/modules/videoio/src/cap_gstreamer.cpp (1405) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
2024-05-27 07:00:02,298[10667:10758][camera_data_manager.py:320] JPEG: rtsp://admin:[email protected]:554/Streaming/Channels/101 open success
2024-05-27 07:00:02,301[10667:10770][data_manager_template.py: 53] JPEG: rtsp://admin:[email protected]:554/Streaming/Channels/101: acquire_loop start
2024-05-27 07:00:02,302[10667:10758][manager_template.py: 80] Source, FPS: 1.1
2024-05-27 07:00:03.239[10667:10759][lidar_driver_wrapper.cpp:138] 0:RS-Helios No lidar data comes
2024-05-27 07:00:03,240[10667:10759][data_manager_template.py: 62] Lidar: 0-RS-Helios, acquire FPS: 0.5
2024-05-27 07:00:03,261[10667:10760][module_manager.py:116] system status Running, running thread/process information:
2024-05-27 07:00:03,261[10667:10760][module_manager.py:120] Thread 10667:Perception daemon False ident 0x7f0bc5f8d7c0
2024-05-27 07:00:03,262[10667:10760][module_manager.py:120] Thread 10758:Source daemon True ident 0x7f0b3a7fc700
2024-05-27 07:00:03,262[10667:10760][module_manager.py:120] Thread 10759:Py-L0-RS-Helios daemon True ident 0x7f0b39ffb700
2024-05-27 07:00:03,262[10667:10760][module_manager.py:120] Thread 10762:Py-Crtsp://admin:[email protected]:554/Streaming/Channels/101 daemon True ident 0x7f0b38ff9700
2024-05-27 07:00:03,262[10667:10760][module_manager.py:120] Thread 10763:zerorpc daemon True ident 0x7f0b17fff700
2024-05-27 07:00:03,262[10667:10760][module_manager.py:120] Thread 10770:Py-Jrtsp://admin:[email protected]:554/Streaming/Channels/101 daemon True ident 0x7f0af7fff700
2024-05-27 07:00:03,304[10667:10758][manager_template.py: 80] Source, FPS: 1.0
2024-05-27 07:00:04.240[10667:10759][lidar_driver_wrapper.cpp:138] 0:RS-Helios No lidar data comes
2024-05-27 07:00:04,240[10667:10759][data_manager_template.py: 62] Lidar: 0-RS-Helios, acquire FPS: 0.7
2024-05-27 07:00:05.241[10667:10759][lidar_driver_wrapper.cpp:138] 0:RS-Helios No lidar data comes
2024-05-27 07:00:05,241[10667:10759][data_manager_template.py: 62] Lidar: 0-RS-Helios, acquire FPS: 0.9
2024-05-27 07:00:06.242[10667:10759][lidar_driver_wrapper.cpp:138] 0:RS-Helios No lidar data comes
2024-05-27 07:00:06,242[10667:10759][data_manager_template.py: 62] Lidar: 0-RS-Helios, acquire FPS: 0.9
2024-05-27 07:00:07.243[10667:10759][lidar_driver_wrapper.cpp:138] 0:RS-Helios No lidar data comes
2024-05-27 07:00:07,243[10667:10759][data_manager_template.py: 62] Lidar: 0-RS-Helios, acquire FPS: 1.0
2024-05-27 07:00:08.243[10667:10759][lidar_driver_wrapper.cpp:138] 0:RS-Helios No lidar data comes
2024-05-27 07:00:08,244[10667:10759][data_manager_template.py: 62] Lidar: 0-RS-Helios, acquire FPS: 1.0
2024-05-27 07:00:09.244[10667:10759][lidar_driver_wrapper.cpp:138] 0:RS-Helios No lidar data comes
2024-05-27 07:00:09,244[10667:10759][data_manager_template.py: 62] Lidar: 0-RS-Helios, acquire FPS: 1.0
2024-05-27 07:00:10.245[10667:10759][lidar_driver_wrapper.cpp:138] 0:RS-Helios No lidar data comes
2024-05-27 07:00:10,245[10667:10759][data_manager_template.py: 62] Lidar: 0-RS-Helios, acquire FPS: 1.0
2024-05-27 07:00:10,818[10667:10667][ http_sink.py: 92] http query FPS: 0.1
2024-05-27 07:00:10,818[10667:10667][ http_sink.py: 98] http client connect
2024-05-27 07:00:10,819[10667:10779][ http_sink.py: 72] http sink process loop start
jpeg encoder, worker 0 loop starts
jpeg encoder, worker 1 loop starts
jpeg encoder, worker 2 loop starts
jpeg encoder, worker 3 loop starts
jpeg encoder, worker 4 loop starts
jpeg encoder, worker 5 loop starts
jpeg encoder, worker 6 loop starts
jpeg encoder, worker 7 loop starts
2024-05-27 07:00:11,067[10667:10667][ http_sink.py: 92] http query FPS: 2.0
2024-05-27 07:00:11.246[10667:10759][lidar_driver_wrapper.cpp:138] 0:RS-Helios No lidar data comes
2024-05-27 07:00:11,246[10667:10759][data_manager_template.py: 62] Lidar: 0-RS-Helios, acquire FPS: 1.0
2024-05-27 07:00:12.247[10667:10759][lidar_driver_wrapper.cpp:138] 0:RS-Helios No lidar data comes
2024-05-27 07:00:12,247[10667:10759][data_manager_template.py: 62] Lidar: 0-RS-Helios, acquire FPS: 1.0
2024-05-27 07:00:13.248[10667:10759][lidar_driver_wrapper.cpp:138] 0:RS-Helios No lidar data comes
2024-05-27 07:00:13,248[10667:10759][data_manager_template.py: 62] Lidar: 0-RS-Helios, acquire FPS: 1.0
2024-05-27 07:00:13,367[10667:10760][module_manager.py:137] Process 10676:UdpSink CPU 0.0%, Mem 130 MB
2024-05-27 07:00:13,468[10667:10760][module_manager.py:137] Process 10667:Perception CPU 0.0%, Mem 311 MB
2024-05-27 07:00:14.248[10667:10759][lidar_driver_wrapper.cpp:138] 0:RS-Helios No lidar data comes
2024-05-27 07:00:14,249[10667:10759][data_manager_template.py: 62] Lidar: 0-RS-Helios, acquire FPS: 1.0
2024-05-27 07:00:14,590[10667:10667][ perception.py: 51]
camera:

• extrinsic_parameters:
  • 0
  • 0
  • 0
  • 0
  • 0
  • 0
    index: 0
    intrinsic_parameters:
  • 0
  • 0
  • 0
  • 0
  • 0
  • 0
  • 0
  • 0
    name: rtsp://admin:[email protected]:554/Streaming/Channels/101
    output_height: 1080
    output_width: 1920
    static_extrinsic_parameters:
  • 0
  • 0
  • 0
  • 0
  • 0
  • 0
    stream:
    host: ''
    port: 554
    sink: rtsp
    undistortion: false
    detection:
    freespace:
    max_height: 2.5
    min_height: 0.2
    mode: learning
    range:
    • -40
    • 40
    • -40
    • 40
      resolution: 0.4
      sensor_input:
  • 0-Ouster-OS1
  • 1-Ouster-OS1
  • 2-Ouster-OS1
  • '0'
    input:
    data_path: /home/znqc/work/demo_data
    mode: offline
    ins:
    device: /dev/ttyUSB0
    extrinsic_parameters:
  • 0.09695542603731155
  • 1.2899638414382935
  • -0.11553408205509186
  • -0.8822724223136902
  • -0.02054435946047306
  • 87.35172271728516
    imu_extrinsic_parameters:
  • 0
  • 0
  • 0
  • -0.8526138067245483
  • -0.0389305055141449
  • 89.72308349609375
    ins_fix:
    precision: 0.5
    stable_time: 0
    status: 42
    use: true
    ins_float:
    precision: 5
    stable_time: 10
    status: -1
    use: true
    ins_normal:
    precision: 20
    stable_time: 10
    status: -1
    use: true
    ins_type: 6D
    port: 9888
    relay:
    destination: 192.168.1.101
    use: false
    lidar:
• exclude:
  • -0.5
  • -0.5
  • 0
  • 0.5
  • 0.5
  • 0
    extrinsic_parameters:
  • 0
  • 0
  • 0
  • 0
  • 0
  • 0
    name: RS-Helios
    port: 6699
    range:
  • -72
  • -72
  • -2
  • 72
  • 72
  • 4
    output:
    localization:
    UDP:
    destination: 192.168.1.101
    port: 9001
    use: false
    object:
    cyclist: true
    freespace: true
    pedestrian: true
    traffic_cone: true
    use: true
    vehicle: true
    point_cloud:
    destination: 192.168.1.101
    use: false
    protocol:
    CAN:
    baud: 500000
    device: can0
    use: false
    UDP:
    destination: 192.168.1.101
    port: 9000
    use: true
    pipeline:
• • Source
  • Sink
    radar: []
    roi:
• contour: []
  is_included: true
  slam:
  localization:
  colouration: false
  key_frames_interval:
  • 4
  • 20
    map_path: lp_log/map
    map_resolution: 0.2
    sensor_input: []
    mapping:
    gravity_constraint: false
    ground_constraint: true
    key_frames_interval:
  • 4
  • 20
    key_frames_range: 1000
    loop_closure: true
    map_resolution: 0.2
    sensor_input:
  • 1-Ouster-OS1
  • IMU
    method: FastLIO
    mode: mapping
    origin:
    altitude: 0
    latitude: 0
    longitude: 0
    use: true

2024-05-27 07:00:14,671[10667:10758][manager_template.py: 94] Source loop try to stop
2024-05-27 07:00:14,672[10667:10758][source_manager.py: 49] Source, try stop capture: player
2024-05-27 07:00:14,672[10667:10758][source_manager.py: 51] Source, stop capture: player, done
2024-05-27 07:00:14,673[10667:10758][source_manager.py: 49] Source, try stop capture: lidar
2024-05-27 07:00:15.249[10667:10759][lidar_driver_wrapper.cpp:138] 0:RS-Helios No lidar data comes

你好,我在调用3D单目视觉检测算法,我发现1.3.1版本没有打开,当我打开以后引起加载报错,请帮忙看一下.我想在GPU上跑双传感器融合的版本.我的GPU是特斯拉P4,


引起报错的位置

Hi, I have installed the latest NVIDIA Container Toolkit, but when I run the command "bash sensor_inference/pytorch_model/export/generate_trt.sh", I meet the problem:

Is there any advice? Thanks!

你好，我分别在目标检测模式下和数据采集模式下 通过rtsp接入相机进行标定，空格键保存大于10张，但是交点检测没有反应，状态一栏一直是空，一直无法到下一步，请帮忙看一下

Hi, when I open the localhost:config GUI keeps loading and not displaying, both the cuda version and the non-cuda version I've tried, could you please tell me what's causing this?

Hi, I want to follow the doc to run slam with the demo dataset. But in sensor configuration there're no camera and rs-lidar-16:

By the way, where can I set the dataset path?

When I run tools/scripts/start_system.sh on the embedded platform, the following error is displayed
The File "/ usr/lib/python3.8 multiprocessing/process. Py", line 315, in _bootstrap
self.run()
The File "/ usr/lib/python3.8 multiprocessing/process. Py", line 108, in the run
self._target(*self._args, **self._kwargs)
File "web_backend/server.py", line 17, in perception_main
from module.perception import Perception
File "/root/lidar-slam-detection/module/perception.py", line 8, in
import calibration.calibration
File "/root/lidar-slam-detection/calibration/calibration.py", line 4, in
from sensor_driver.common_lib.cpp_utils import (
File "/root/lidar-slam-detection/sensor_driver/common_lib/cpp_utils.py", line 1, in
import cpp_utils_ext
ModuleNotFoundError: No module named 'cpp_utils_ext'
Traceback (most recent call last):
File "web_backend/server.py", line 36, in
from perception_server import PerceptionServer
File "/root/lidar-slam-detection/web_backend/perception_server.py", line 8, in
from message_server import MessageServer
File "/root/lidar-slam-detection/web_backend/message_server.py", line 7, in
from zerocm import ZCM
ImportError: dynamic module does not define module export function (PyInit_zerocm)
Is that something I didn't install

目前一直卡在map initializing, odometry not stable, resetting...这个阶段，这个应该怎么做？

After evaluating keyframe_updater->is_update(odom, need_update, must_update, dx, da), where must_update is determined to be true, it fails to pass the subsequent conditional statement

In this case, despite must_update being true, the best_frame value fails to get updated, resulting in the execution of the subsequent conditional statement.

This situation could potentially lead to the construction of keyframes using outdated best_frame data.

In addition, regarding the concepts of fitness_score and inlier_ratio mentioned in the first image, as well as the calculations within the if statement, here is some information:

Additionally, I don't fully understand the meaning of fitness_score and inlier_ratio in the first image, as well as the calculations within the if statement. Is there any relevant documentation available? I noticed in the calculation function for fitness_score this statement: if (horizon_dist <= 10.0 && p1.z < floor_height_max && p2.z < floor_height_max && fabs(p1.z - p2.z) > 0.25). Does this statement preserve ground points? Also, what is the purpose of lines 221-224 in the first image, as they don't appear to be used later on?

Thx for ur great job.
I check the api of slam.h and slam.py but do not understand how do u visualize the update keyframe when mapping. Cause the odom of the keyframe will be update when backend finishs the optimization, I think you should continuely monitor the keyframe queue. But when I look around the api, the "update_odom" will deque all the keyframe from mKeyframeQueue. In such case, the mKeyframeQueue will be empty. So I think the "update_odom" should only be run after the mapping is finished?

The Rabosense rader message could not be transmitted, Instead, it is transmitted to Ros.

你好，我用的1.3.1版本，切换到目标检测模式后，可输出结果，但是好像没有用到显卡，检测帧率很低（速度慢），请问应该怎么调试以下，我的docker用的gpu版本

un server python script
dpkg-query: no packages found matching nvidia-l4t-core
Running on board: IPC, machine: x86_64, jetpack: 20.04
2024-05-27 08:39:01,478[26853:26853][ infer.py: 23] image object detection is deactived
2024-05-27 08:39:01,490[26853:26853][module_manager.py: 47] pipeline: Source connect to Detect
2024-05-27 08:39:01,491[26853:26853][module_manager.py: 47] pipeline: Detect connect to Sink
2024-05-27 08:39:01,491[26853:26853][module_manager.py: 71] start to setup module: Detect
2024-05-27 08:39:01,506[26853:26853][object_filter.py: 25] class mapping: {'vehicle': 1, 'pedestrian': 2, 'cyclist': 3, 'traffic_cone': 4}
2024-05-27 08:39:01,506[26853:26853][module_manager.py: 73] setup module: Detect, done
2024-05-27 08:39:01,507[26853:26853][module_manager.py: 71] start to setup module: Sink
2024-05-27 08:39:01,513[26853:26853][module_manager.py: 73] setup module: Sink, done
2024-05-27 08:39:01,514[26853:26853][module_manager.py: 71] start to setup module: Source
2024-05-27 08:39:01,514[26874:26874][sink_template.py: 58] deamon UdpSink starts
2024-05-27 08:39:01,523[26853:26853][player_data_manager.py:135] player have 201 frames, duration 19.971323 s
/usr/local/lib/python3.8/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py:65: UserWarning: Specified provider 'OpenVINOExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider'
warnings.warn(

• Serving Flask app 'module.source.player_data_manager'
• Debug mode: off
  2024-05-27 08:39:01,724[26853:26853][source_manager.py: 35] Source, start to setup: lidar
  WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
• Running on http://127.0.0.1:38000
  Press CTRL+C to quit
  2024-05-27 08:39:01.738[26853:26853][lidar_driver_wrapper.cpp:34] lidar 0:Ouster-OS1-64 open success
  2024-05-27 08:39:01.738[26853:26853][lidar_driver_wrapper.cpp:46] set lidar 0:Ouster-OS1-64 external parameter [-0.03368380293250084, -0.660768985748291, 1.7658969163894653, 177.67449951171875, 135.47140502929688, -177.0221710205078]
  2024-05-27 08:39:01.738[26853:26853][lidar_driver_wrapper.cpp:58] set lidar 0:Ouster-OS1-64 range filter [-72.0, -72.0, -2.0, 72.0, 72.0, 4.0]
  2024-05-27 08:39:01.738[26853:26853][lidar_driver_wrapper.cpp:70] set lidar 0:Ouster-OS1-64 exclude [-2.13, -2.01, -5.0, 2.92, 0.5, 5.0]
  2024-05-27 08:39:01.745[26853:26853][lidar_driver_wrapper.cpp:34] lidar 1:Ouster-OS1-128 open success
  2024-05-27 08:39:01.745[26853:26853][lidar_driver_wrapper.cpp:46] set lidar 1:Ouster-OS1-128 external parameter [0.0, 0.0, 1.88961923122406, 179.99972534179688, 179.9855194091797, -177.86497497558594]
  2024-05-27 08:39:01.745[26853:26853][lidar_driver_wrapper.cpp:58] set lidar 1:Ouster-OS1-128 range filter [-72.0, -72.0, -2.0, 72.0, 72.0, 4.0]
  2024-05-27 08:39:01.745[26853:26853][lidar_driver_wrapper.cpp:70] set lidar 1:Ouster-OS1-128 exclude [-3.37, -3.02, -4.15, 3.16, 2.33, 4.7]
  2024-05-27 08:39:01.751[26853:26853][lidar_driver_wrapper.cpp:34] lidar 2:Ouster-OS1-64 open success
  2024-05-27 08:39:01.751[26853:26853][lidar_driver_wrapper.cpp:46] set lidar 2:Ouster-OS1-64 external parameter [-0.09995567798614502, 0.6768492460250854, 1.7785784006118774, 1.0974562168121338, -44.20530319213867, 2.35439395904541]
  2024-05-27 08:39:01.751[26853:26853][lidar_driver_wrapper.cpp:58] set lidar 2:Ouster-OS1-64 range filter [-72.0, -72.0, -2.0, 72.0, 72.0, 4.0]
  2024-05-27 08:39:01.751[26853:26853][lidar_driver_wrapper.cpp:70] set lidar 2:Ouster-OS1-64 exclude [-3.37, -3.14, -4.92, 3.16, 1.44, 4.58]
  2024-05-27 08:39:01,751[26853:26853][source_manager.py: 38] Source, setup: lidar, done
  2024-05-27 08:39:01,751[26853:26853][source_manager.py: 35] Source, start to setup: camera
  2024-05-27 08:39:01,807[26853:26853][camera_data_manager.py:207] camera 0, try cap: souphttpsrc timeout=0 location=http://127.0.0.1:38000/v1/camera/0 ! jpegparse ! jpegdec ! video/x-raw ! videoscale ! video/x-raw,width=1920,height=1080 ! videoconvert ! video/x-raw,width=1920,height=1080,format=I420 ! tee name=t
  t. ! queue ! jpegenc ! shmsink socket-path=/tmp/camera_jpeg_0 sync=false
  t. ! queue ! appsink sync=false
  127.0.0.1 - - [27/May/2024 08:39:01] "GET /v1/camera/0 HTTP/1.1" 200 -
  [ WARN:[email protected]] global /tmp/opencv-4.5.5/modules/videoio/src/cap_gstreamer.cpp (1405) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=0, duration=-1
  2024-05-27 08:39:01,911[26853:26853][camera_data_manager.py:217] camera 0 open success
  2024-05-27 08:39:01,911[26853:26853][camera_data_manager.py:207] camera 1, try cap: souphttpsrc timeout=0 location=http://127.0.0.1:38000/v1/camera/1 ! jpegparse ! jpegdec ! video/x-raw ! videoscale ! video/x-raw,width=1920,height=1080 ! videoconvert ! video/x-raw,width=1920,height=1080,format=I420 ! tee name=t
  t. ! queue ! jpegenc ! shmsink socket-path=/tmp/camera_jpeg_1 sync=false
  t. ! queue ! appsink sync=false
  127.0.0.1 - - [27/May/2024 08:39:01] "GET /v1/camera/1 HTTP/1.1" 200 -
  [ WARN:[email protected]] global /tmp/opencv-4.5.5/modules/videoio/src/cap_gstreamer.cpp (1405) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=0, duration=-1
  2024-05-27 08:39:01,965[26853:26853][camera_data_manager.py:217] camera 1 open success
  2024-05-27 08:39:01,965[26853:26853][camera_data_manager.py:207] camera 2, try cap: souphttpsrc timeout=0 location=http://127.0.0.1:38000/v1/camera/2 ! jpegparse ! jpegdec ! video/x-raw ! videoscale ! video/x-raw,width=1920,height=1080 ! videoconvert ! video/x-raw,width=1920,height=1080,format=I420 ! tee name=t
  t. ! queue ! jpegenc ! shmsink socket-path=/tmp/camera_jpeg_2 sync=false
  t. ! queue ! appsink sync=false
  127.0.0.1 - - [27/May/2024 08:39:04] "GET /v1/camera/2 HTTP/1.1" 200 -
  [ WARN:[email protected]] global /tmp/opencv-4.5.5/modules/videoio/src/cap_gstreamer.cpp (1405) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=0, duration=-1
  2024-05-27 08:39:04,374[26853:26853][camera_data_manager.py:217] camera 2 open success
  2024-05-27 08:39:04,374[26853:26853][camera_data_manager.py:207] camera 3, try cap: souphttpsrc timeout=0 location=http://127.0.0.1:38000/v1/camera/3 ! jpegparse ! jpegdec ! video/x-raw ! videoscale ! video/x-raw,width=1920,height=1080 ! videoconvert ! video/x-raw,width=1920,height=1080,format=I420 ! tee name=t
  t. ! queue ! jpegenc ! shmsink socket-path=/tmp/camera_jpeg_3 sync=false
  t. ! queue ! appsink sync=false
  127.0.0.1 - - [27/May/2024 08:39:04] "GET /v1/camera/3 HTTP/1.1" 200 -
  [ WARN:[email protected]] global /tmp/opencv-4.5.5/modules/videoio/src/cap_gstreamer.cpp (1405) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=0, duration=-1
  2024-05-27 08:39:04,427[26853:26853][camera_data_manager.py:217] camera 3 open success
  2024-05-27 08:39:04,428[26853:26853][source_manager.py: 38] Source, setup: camera, done
  2024-05-27 08:39:04,428[26853:26853][source_manager.py: 35] Source, start to setup: radar
  2024-05-27 08:39:04,428[26853:26853][source_manager.py: 38] Source, setup: radar, done
  2024-05-27 08:39:04,428[26853:26853][source_manager.py: 35] Source, start to setup: ins
  2024-05-27 08:39:04.428[26853:26853][ins_driver_wrapper.cpp:19] ins open success
  2024-05-27 08:39:04.428[26853:26853][ins_driver_wrapper.cpp:34] set ins external parameter [0.09695542603731155, 1.2899638414382935, -0.11553408205509186, 87.35172271728516, -0.02054435946047306, -0.8822724223136902]
  2024-05-27 08:39:04.429[26853:26853][ins_driver_wrapper.cpp:43] ins start success
  2024-05-27 08:39:04.429[26853:26952][ins_driver.cpp:317] start ins com receving, device /dev/ttyUSB0
  2024-05-27 08:39:04.429[26853:26951][ins_driver.cpp:278] start ins udp receving, port 9888
  2024-05-27 08:39:04,429[26853:26853][source_manager.py: 38] Source, setup: ins, done
  2024-05-27 08:39:04.429[26853:26953][ins_driver.cpp:378] start ins gps receving
  2024-05-27 08:39:04,429[26853:26853][module_manager.py: 73] setup module: Source, done
  2024-05-27 08:39:04.429[26853:26953][ins_driver.cpp:381] No GPSD running
  2024-05-27 08:39:04,430[26853:26954][manager_template.py: 80] Detect, FPS: 0.2
  2024-05-27 08:39:04,431[26853:26956][data_manager_template.py: 57] Camera: 0: acquire_loop start
  2024-05-27 08:39:04,431[26853:26958][data_manager_template.py: 57] Camera: 1: acquire_loop start
  2024-05-27 08:39:04,432[26853:26853][ perception.py: 34] start success
  2024-05-27 08:39:04,432[26853:26959][data_manager_template.py: 57] Camera: 2: acquire_loop start
  2024-05-27 08:39:04,433[26853:26960][data_manager_template.py: 57] Camera: 3: acquire_loop start
  [ WARN:[email protected]] global /tmp/opencv-4.5.5/modules/videoio/src/cap_gstreamer.cpp (1405) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
  [ WARN:[email protected]] global /tmp/opencv-4.5.5/modules/videoio/src/cap_gstreamer.cpp (1405) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
  [ WARN:[email protected]] global /tmp/opencv-4.5.5/modules/videoio/src/cap_gstreamer.cpp (1405) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
  [ WARN:[email protected]] global /tmp/opencv-4.5.5/modules/videoio/src/cap_gstreamer.cpp (1405) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
  2024-05-27 08:39:04,481[26853:26955][manager_template.py: 80] Source, FPS: 0.2
  2024-05-27 08:39:04,568[26853:26853][ http_sink.py: 92] http query FPS: 0.2
  2024-05-27 08:39:04,568[26853:26853][ http_sink.py: 98] http client connect
  2024-05-27 08:39:04,569[26853:26984][ http_sink.py: 72] http sink process loop start
  2024-05-27 08:39:05,112[26853:26853][ http_sink.py: 92] http query FPS: 1.0
  2024-05-27 08:39:05,396[26853:26955][manager_template.py: 80] Source, FPS: 0.6
  2024-05-27 08:39:05,474[26853:26957][module_manager.py:116] system status Running, running thread/process information:
  2024-05-27 08:39:05,474[26853:26957][module_manager.py:120] Thread 26853:Perception daemon False ident 0x7f77608ca0c0
  2024-05-27 08:39:05,475[26853:26957][module_manager.py:120] Thread 26954:Detect daemon True ident 0x7f76d2ffd700
  2024-05-27 08:39:05,475[26853:26957][module_manager.py:120] Thread 26955:Source daemon True ident 0x7f76d27fc700
  2024-05-27 08:39:05,475[26853:26957][module_manager.py:120] Thread 26956:Py-C0 daemon True ident 0x7f76d1ffb700
  2024-05-27 08:39:05,475[26853:26957][module_manager.py:120] Thread 26958:Py-C1 daemon True ident 0x7f76d0ff9700
  2024-05-27 08:39:05,475[26853:26957][module_manager.py:120] Thread 26959:Py-C2 daemon True ident 0x7f76affff700
  2024-05-27 08:39:05,475[26853:26957][module_manager.py:120] Thread 26960:Py-C3 daemon True ident 0x7f76af7fe700
  2024-05-27 08:39:05,476[26853:26957][module_manager.py:120] Thread 26961:zerorpc daemon True ident 0x7f76aeffd700
  2024-05-27 08:39:05,476[26853:26957][module_manager.py:120] Thread 26976:Py-J0 daemon True ident 0x7f76ae7fc700
  2024-05-27 08:39:05,476[26853:26957][module_manager.py:120] Thread 26977:Py-J1 daemon True ident 0x7f7663fff700
  2024-05-27 08:39:05,476[26853:26957][module_manager.py:120] Thread 26978:Py-J2 daemon True ident 0x7f76637fe700
  2024-05-27 08:39:05,476[26853:26957][module_manager.py:120] Thread 26979:Py-J3 daemon True ident 0x7f7662ffd700
  2024-05-27 08:39:05,477[26853:26957][module_manager.py:120] Thread 26984:HttpSink daemon True ident 0x7f7643fff700
  2024-05-27 08:39:05,824[26871:26871][ infer_base.py: 99] lidarDet inference time is: 414.6 ms
  2024-05-27 08:39:06,018[26873:26873][detect_template.py: 63] tracker process time is: 186.8 ms
  2024-05-27 08:39:06,038[26853:26954][manager_template.py: 80] Detect, FPS: 0.4
  2024-05-27 08:39:06,228[26871:26871][ infer_base.py: 99] lidarDet inference time is: 381.2 ms
  2024-05-27 08:39:06,257[26853:26954][manager_template.py: 80] Detect, FPS: 2.5
  jpeg encoder, worker 0 loop starts
  jpeg encoder, worker 1 loop starts
  jpeg encoder, worker 2 loop starts
  jpeg encoder, worker 3 loop starts
  jpeg encoder, worker 4 loop starts
  jpeg encoder, worker 5 loop starts
  2024-05-27 08:39:06,261[26853:26955][manager_template.py: 80] Source, FPS: 5.4
  2024-05-27 08:39:06,616[26871:26871][ infer_base.py: 99] lidarDet inference time is: 362.6 ms
  2024-05-27 08:39:06,631[26853:26956][data_manager_template.py: 66] Camera: 0, acquire FPS: 4.9
  2024-05-27 08:39:06,645[26853:26960][data_manager_template.py: 66] Camera: 3, acquire FPS: 6.1
  2024-05-27 08:39:07,064[26871:26871][ infer_base.py: 99] lidarDet inference time is: 426.9 ms
  2024-05-27 08:39:07,429[26871:26871][ infer_base.py: 99] lidarDet inference time is: 358.4 ms
  2024-05-27 08:39:07,908[26853:26853][ http_sink.py: 92] http query FPS: 0.7
  2024-05-27 08:39:07,910[26853:26955][manager_template.py: 80] Source, FPS: 3.0
  2024-05-27 08:39:07,915[26853:26954][manager_template.py: 80] Detect, FPS: 1.5
  2024-05-27 08:39:07,955[26853:26960][data_manager_template.py: 66] Camera: 3, acquire FPS: 3.4
  2024-05-27 08:39:07,956[26853:26958][data_manager_template.py: 66] Camera: 1, acquire FPS: 5.0
  2024-05-27 08:39:08,339[26871:26871][ infer_base.py: 99] lidarDet inference time is: 414.1 ms
  2024-05-27 08:39:07,960[26853:26959][data_manager_template.py: 66] Camera: 2, acquire FPS: 5.2
  2024-05-27 08:39:08,368[26853:26956][data_manager_template.py: 66] Camera: 0, acquire FPS: 2.7
  2024-05-27 08:39:08,371[26853:26955][manager_template.py: 80] Source, FPS: 2.6
  2024-05-27 08:39:08,799[26871:26871][ infer_base.py: 99] lidarDet inference time is: 448.1 ms
  2024-05-27 08:39:08,806[26853:26958][data_manager_template.py: 66] Camera: 1, acquire FPS: 3.1
  2024-05-27 08:39:08,809[26853:26956][data_manager_template.py: 66] Camera: 0, acquire FPS: 2.5
  2024-05-27 08:39:08,832[26853:26960][data_manager_template.py: 66] Camera: 3, acquire FPS: 2.3
  2024-05-27 08:39:08,835[26853:26959][data_manager_template.py: 66] Camera: 2, acquire FPS: 3.2
  2024-05-27 08:39:08,843[26853:26955][manager_template.py: 80] Source, FPS: 2.4
  2024-05-27 08:39:09,174[26871:26871][ infer_base.py: 99] lidarDet inference time is: 349.9 ms
  2024-05-27 08:39:09,570[26871:26871][ infer_base.py: 99] lidarDet inference time is: 387.9 ms
  2024-05-27 08:39:09,598[26853:26959][data_manager_template.py: 66] Camera: 2, acquire FPS: 2.2
  2024-05-27 08:39:09,601[26853:26853][ http_sink.py: 92] http query FPS: 0.6
  2024-05-27 08:39:09,604[26853:26960][data_manager_template.py: 66] Camera: 3, acquire FPS: 1.8
  2024-05-27 08:39:09,609[26853:26956][data_manager_template.py: 66] Camera: 0, acquire FPS: 1.9
  2024-05-27 08:39:09,616[26853:26955][manager_template.py: 80] Source, FPS: 1.8
  2024-05-27 08:39:09,638[26853:26958][data_manager_template.py: 66] Camera: 1, acquire FPS: 5.5
  2024-05-27 08:39:09,678[26853:26853][module_manager.py: 91] system is started
  2024-05-27 08:39:09,711[26853:26955][manager_template.py: 80] Source, FPS: 6.2
  2024-05-27 08:39:09,856[26853:26958][data_manager_template.py: 66] Camera: 1, acquire FPS: 5.1
  2024-05-27 08:39:09,957[26853:26955][manager_template.py: 80] Source, FPS: 5.1
  2024-05-27 08:39:10,050[26853:26959][data_manager_template.py: 66] Camera: 2, acquire FPS: 7.4
  2024-05-27 08:39:10,050[26871:26871][ infer_base.py: 99] lidarDet inference time is: 426.7 ms
  2024-05-27 08:39:10,053[26853:26958][data_manager_template.py: 66] Camera: 1, acquire FPS: 5.1
  2024-05-27 08:39:10,056[26853:26955][manager_template.py: 80] Source, FPS: 7.6
  2024-05-27 08:39:10,231[26853:26955][manager_template.py: 80] Source, FPS: 6.7
  2024-05-27 08:39:10,318[26853:26958][data_manager_template.py: 66] Camera: 1, acquire FPS: 6.9
  2024-05-27 08:39:10,335[26853:26959][data_manager_template.py: 66] Camera: 2, acquire FPS: 7.8
  2024-05-27 08:39:10,455[26871:26871][ infer_base.py: 99] lidarDet inference time is: 381.5 ms
  2024-05-27 08:39:10,485[26853:26955][manager_template.py: 80] Source, FPS: 5.3
  2024-05-27 08:39:10,598[26853:26959][data_manager_template.py: 66] Camera: 2, acquire FPS: 5.8
  2024-05-27 08:39:10,600[26853:26958][data_manager_template.py: 66] Camera: 1, acquire FPS: 5.2
  2024-05-27 08:39:10,627[26853:26960][data_manager_template.py: 66] Camera: 3, acquire FPS: 5.6
  2024-05-27 08:39:10,631[26853:26853][ http_sink.py: 92] http query FPS: 0.8
  2024-05-27 08:39:10,689[26853:26956][data_manager_template.py: 66] Camera: 0, acquire FPS: 6.1
  2024-05-27 08:39:10,872[26871:26871][ infer_base.py: 99] lidarDet inference time is: 392.7 ms
  2024-05-27 08:39:10,898[26853:26955][manager_template.py: 80] Source, FPS: 3.9
  2024-05-27 08:39:10,919[26853:26959][data_manager_template.py: 66] Camera: 2, acquire FPS: 4.5
  2024-05-27 08:39:10,920[26853:26956][data_manager_template.py: 66] Camera: 0, acquire FPS: 5.2
  2024-05-27 08:39:10,921[26853:26958][data_manager_template.py: 66] Camera: 1, acquire FPS: 4.2
  2024-05-27 08:39:10,921[26853:26960][data_manager_template.py: 66] Camera: 3, acquire FPS: 4.5
  2024-05-27 08:39:11,031[26853:26853][ http_sink.py: 92] http query FPS: 1.6
  2024-05-27 08:39:11,281[26871:26871][ infer_base.py: 99] lidarDet inference time is: 383.5 ms
  2024-05-27 08:39:11,307[26853:26955][manager_template.py: 80] Source, FPS: 3.2
  2024-05-27 08:39:11,330[26853:26960][data_manager_template.py: 66] Camera: 3, acquire FPS: 3.5
  2024-05-27 08:39:11,725[26871:26871][ infer_base.py: 99] lidarDet inference time is: 432.8 ms
  2024-05-27 08:39:11,750[26853:26959][data_manager_template.py: 66] Camera: 2, acquire FPS: 2.8
  2024-05-27 08:39:11,755[26853:26958][data_manager_template.py: 66] Camera: 1, acquire FPS: 2.7
  2024-05-27 08:39:12,109[26871:26871][ infer_base.py: 99] lidarDet inference time is: 364.7 ms
  2024-05-27 08:39:12,131[26853:26956][data_manager_template.py: 66] Camera: 0, acquire FPS: 3.0
  2024-05-27 08:39:12,148[26853:26955][manager_template.py: 80] Source, FPS: 2.2
  2024-05-27 08:39:12,497[26871:26871][ infer_base.py: 99] lidarDet inference time is: 377.0 ms
  2024-05-27 08:39:12,518[26853:26853][ http_sink.py: 92] http query FPS: 1.2
  2024-05-27 08:39:12,529[26853:26956][data_manager_template.py: 66] Camera: 0, acquire FPS: 2.8
  2024-05-27 08:39:12,531[26853:26960][data_manager_template.py: 66] Camera: 3, acquire FPS: 2.1
  2024-05-27 08:39:12,531[26853:26958][data_manager_template.py: 66] Camera: 1, acquire FPS: 2.0
  2024-05-27 08:39:12,538[26853:26959][data_manager_template.py: 66] Camera: 2, acquire FPS: 2.1
  2024-05-27 08:39:12,965[26871:26871][ infer_base.py: 99] lidarDet inference time is: 451.8 ms
  2024-05-27 08:39:12,990[26853:26955][manager_template.py: 80] Source, FPS: 1.7
  2024-05-27 08:39:13,326[26871:26871][ infer_base.py: 99] lidarDet inference time is: 355.6 ms
  2024-05-27 08:39:13,327[26853:26954][manager_template.py: 80] Detect, FPS: 7.1
  2024-05-27 08:39:13,351[26853:26956][data_manager_template.py: 66] Camera: 0, acquire FPS: 2.0
  2024-05-27 08:39:13,366[26853:26959][data_manager_template.py: 66] Camera: 2, acquire FPS: 1.6
  2024-05-27 08:39:13,367[26853:26958][data_manager_template.py: 66] Camera: 1, acquire FPS: 1.6
  ^CProcess UdpSink:
  Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
  self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
  self._target(*self._args, **self._kwargs)
  File "/home/znqc/work/new/1.3.1/lidar-slam-detection/module/sink/sink_template.py", line 63, in _run
  data_dict = self.input_queue.get(block=True, timeout=0.1)
  File "/usr/lib/python3.8/multiprocessing/queues.py", line 107, in get
  if not self._poll(timeout):
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 257, in poll
  return self._poll(timeout)
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 424, in _poll
  r = wait([self], timeout)
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 931, in wait
  ready = selector.select(timeout)
  File "/usr/lib/python3.8/selectors.py", line 415, in select
  fd_event_list = self._selector.poll(timeout)
  KeyboardInterrupt
  Process Process-1:1:
  Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
  self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
  self._target(*self._args, **self._kwargs)
  File "/home/znqc/work/new/1.3.1/lidar-slam-detection/sensor_inference/infer_base.py", line 94, in _infer_thread
  result = self.process(data_dict)
  File "/home/znqc/work/new/1.3.1/lidar-slam-detection/sensor_inference/lidar_infer.py", line 102, in process
  cls_preds, box_preds, label_preds = pointpillar_infer(self.engine, lidar_data)
  File "/home/znqc/work/new/1.3.1/lidar-slam-detection/sensor_inference/lidar_infer.py", line 53, in pointpillar_infer
  return pointpillar_infer_onnx(engine, input_tuple)
  File "/home/znqc/work/new/1.3.1/lidar-slam-detection/sensor_inference/lidar_infer.py", line 38, in pointpillar_infer_onnx
  cls_preds, box_preds, label_preds = engine.run(None, {'voxel_features': input_tuple[0],
  File "/usr/local/lib/python3.8/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 217, in run
  return self._sess.run(output_names, input_feed, run_options)
  KeyboardInterrupt
  KeyboardInterrupt
  2024-05-27T08:39:13Z
  Process fusion:
  2024-05-27 08:39:13,764[26853:26960][data_manager_template.py: 66] Camera: 3, acquire FPS: 1.5
  Process tracker:
  Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
  self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
  self._target(*self._args, **self._kwargs)
  File "/home/znqc/work/new/1.3.1/lidar-slam-detection/sensor_fusion/fusion.py", line 220, in _run_thread
  super()._run_thread()
  File "/home/znqc/work/new/1.3.1/lidar-slam-detection/module/detect/detect_template.py", line 52, in _run_thread
  input_dict = self.input_queue.get(block=True, timeout=0.1)
  File "/usr/lib/python3.8/multiprocessing/queues.py", line 107, in get
  if not self._poll(timeout):
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 257, in poll
  return self._poll(timeout)
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 424, in _poll
  r = wait([self], timeout)
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 931, in wait
  ready = selector.select(timeout)
  File "/usr/lib/python3.8/selectors.py", line 415, in select
  fd_event_list = self._selector.poll(timeout)
  KeyboardInterrupt
  KeyboardInterrupt
  Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
  self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
  self._target(*self._args, **self._kwargs)
  File "/home/znqc/work/new/1.3.1/lidar-slam-detection/sensor_fusion/tracker.py", line 81, in _run_thread
  super()._run_thread()
  File "/home/znqc/work/new/1.3.1/lidar-slam-detection/module/detect/detect_template.py", line 52, in _run_thread
  input_dict = self.input_queue.get(block=True, timeout=0.1)
  File "/usr/lib/python3.8/multiprocessing/queues.py", line 107, in get
  if not self._poll(timeout):
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 257, in poll
  return self._poll(timeout)
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 424, in _poll
  r = wait([self], timeout)
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 931, in wait
  ready = selector.select(timeout)
  File "/usr/lib/python3.8/selectors.py", line 415, in select
  fd_event_list = self._selector.poll(timeout)
  KeyboardInterrupt
  2024-05-27T08:39:13Z
  2024-05-27 08:39:13,764[26853:26954][manager_template.py: 80] Detect, FPS: 4.7
  Traceback (most recent call last):
  File "web_backend/server.py", line 100, in
  http_server.serve_forever()
  File "/usr/local/lib/python3.8/dist-packages/gevent/baseserver.py", line 400, in serve_forever
  self._stop_event.wait()
  File "src/gevent/event.py", line 163, in gevent._gevent_cevent.Event.wait
  File "src/gevent/_abstract_linkable.py", line 521, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait
  File "src/gevent/_abstract_linkable.py", line 487, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
  File "src/gevent/_abstract_linkable.py", line 490, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
  File "src/gevent/_abstract_linkable.py", line 442, in gevent._gevent_c_abstract_linkable.AbstractLinkable._AbstractLinkable__wait_to_be_notified
  File "src/gevent/_abstract_linkable.py", line 451, in gevent._gevent_c_abstract_linkable.AbstractLinkable._switch_to_hub
  File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_greenlet_primitives.py", line 65, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_gevent_c_greenlet_primitives.pxd", line 35, in gevent._gevent_c_greenlet_primitives._greenlet_switch
  KeyboardInterrupt
  2024-05-27 08:39:13,777[26853:26955][manager_template.py: 80] Source, FPS: 1.5
  2024-05-27 08:39:13,781[26853:26959][camera_data_manager.py:228] camera 2, get invalid data
  2024-05-27 08:39:13,781[26853:26960][camera_data_manager.py:228] camera 3, get invalid data
  2024-05-27 08:39:13,781[26853:26958][camera_data_manager.py:228] camera 1, get invalid data
  2024-05-27 08:39:13,781[26853:26956][camera_data_manager.py:228] camera 0, get invalid data
  Process Process-1:
  Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
  self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
  self._target(*self._args, **self._kwargs)
  File "web_backend/server.py", line 25, in perception_main
  start_perception_server(perception, "ipc:///tmp/perception")
  File "web_backend/server.py", line 14, in start_perception_server
  s.run()
  File "/usr/local/lib/python3.8/dist-packages/zerorpc/core.py", line 176, in run
  self._acceptor_task.get()
  File "src/gevent/greenlet.py", line 826, in gevent._gevent_cgreenlet.Greenlet.get
  File "src/gevent/greenlet.py", line 814, in gevent._gevent_cgreenlet.Greenlet.get
  File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_greenlet_primitives.py", line 65, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_gevent_c_greenlet_primitives.pxd", line 35, in gevent._gevent_c_greenlet_primitives._greenlet_switch

Would love to be able to use this on a standard windows laptop.

Hi:
Thanks for open source this great work. But I can not the demo. Here is a screen shot of my localhost in firefox.
Please have a look.

Can you post the link to the LSD paper? I want to learn more in detail.

Hello, do you have a training framework for open-source lidar detection and how to use my own data for training？ And what is the accuracy of the improved pointpillar？

Hi, I'm using your docker 15liangwang/auto-ipu to run LSD, because i didn't have powerful enough GPU, but when i run the script in the way you write in README.md

bash sensor_inference/pytorch_model/export/generate_trt.sh

It output the error message like that:

sensor_inference/pytorch_model/export/generate_trt.sh: line 4: /usr/src/tensorrt/bin/trtexec: No such file or directory
sensor_inference/pytorch_model/export/generate_trt.sh: line 12: polygraphy: command not found

Everything is ok except this command, looking forward to your reply, thand you.

在orin上编译项目，按照文档doc/nvidia/README.md配置完环境后，按照tools/scripts/install_dependency.sh流程python zcm/python/setup.py bdist_wheel过程无法通过，报错信息如下：[1/1] Cythonizing zcm/python/zerocm.pyx
performance hint: zcm/python/zerocm.pyx:81:5: Exception check on 'handler_cb' will always require the GIL to be acquired. Possible solutions:
1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
2. Use an 'int' return type on the function to allow an error code to be returned.
performance hint: zcm/python/zerocm.pyx:86:5: Exception check on 'handler_cb_raw' will always require the GIL to be acquired. Possible solutions:
1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
2. Use an 'int' return type on the function to allow an error code to be returned.
performance hint: zcm/python/zerocm.pyx:90:5: Exception check on 'handler_cb_deprecated' will always require the GIL to be acquired. Possible solutions:
1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
2. Use an 'int' return type on the function to allow an error code to be returned.
performance hint: zcm/python/zerocm.pyx:95:5: Exception check on 'handler_cb_raw_deprecated' will always require the GIL to be acquired. Possible solutions:
1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
2. Use an 'int' return type on the function to allow an error code to be returned.

Error compiling Cython file:

...
subs.handler = handler
subs.msgtype = None
sig = signature(handler)
selected_handler_cb = handler_cb_raw_deprecated if len(sig.parameters) == 2 else handler_cb_raw
while True:
subs.sub = zcm_try_subscribe(self.zcm, channel.encode('utf-8'), selected_handler_cb, <void*> subs)
^

zcm/python/zerocm.pyx:124:76: Cannot assign type 'void (*)(const zcm_recv_buf_t *, const char *, void *) except * nogil' to 'zcm_msg_handler_t'. Exception values are incompatible. Suggest adding 'noexcept' to type 'void (const zcm_recv_buf_t *, const char *, void *) except * nogil'.

Error compiling Cython file:

...
subs.handler = handler
subs.msgtype = msgtype
sig = signature(handler)
selected_handler_cb = handler_cb_deprecated if len(sig.parameters) == 2 else handler_cb
while True:
subs.sub = zcm_try_subscribe(self.zcm, channel.encode('utf-8'), selected_handler_cb, <void*> subs)
^

zcm/python/zerocm.pyx:136:76: Cannot assign type 'void (*)(const zcm_recv_buf_t *, const char *, void *) except * nogil' to 'zcm_msg_handler_t'. Exception values are incompatible. Suggest adding 'noexcept' to type 'void (const zcm_recv_buf_t *, const char *, void *) except * nogil'.
Traceback (most recent call last):
File "zcm/python/setup.py", line 21, in
ext_modules=cythonize([
File "/usr/local/lib/python3.8/dist-packages/Cython/Build/Dependencies.py", line 1154, in cythonize
cythonize_one(*args)
File "/usr/local/lib/python3.8/dist-packages/Cython/Build/Dependencies.py", line 1321, in cythonize_one
raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: zcm/python/zerocm.pyx

你好楼主，web窗口camera显示不稳定，我在程序内部获取是正常的如print（sensor.read()），我的窗口大小是1920 1080的，我调整了640 480 仍然有这个问题，我怀疑有可能是web_ui的问题。我的相机是264编码

补充一下，配置窗口，连接表示符，也在同步闪烁

Hi, I want to follow the the doc to open one pcd file.
But as the figure showed, it is blank:

激光雷达完成标定后，点云检测近处10米左右检测正常，目标有少许离开地面0.1-0.2米，10-20米检测的物体都漂在半空中，目标的高度差异很大。检测目标为人，激光雷达安装高度是1.4米，32线。不确定是哪里的问题。我看标定完了，系数z也是1.4米和实际也一致。

hello ,thank you for your project. there was a problem with my test. the map editor can find the map flie. but I get an error when I open the map file.

2023-09-20 10:08:08.633[723:832][map_loader.cpp:92] Localization: failed to load map meta data, /media/lp_log/map/odometry/map_info.txt
2023-09-20 10:08:08.633[723:832][localization.cpp:92] Map Loader: error to load map: /media/lp_log/map

can reltime show lidar data