What assumption(s) does your deskewing option have? about kiss-icp HOT 6 CLOSED

Hello all, sorry for the late reply, and thanks for the positive feedback! Really encouraging.

The C++ will be available soon :) We are still waiting for the paper review to finish….

About extra datasets

Even if the datasets you want to contribute are a bit specific…. I don't think we/anyone cares. The python-dataloader I envision since a while (also check vdbfusion) is a self-contain, ugly-as-f**ck, python module, which is only in charge to parse the data. As long as don't add a ton of dependencies, this does not really overcharge the system at all. You can also always guard this "optional" dependencies. For example, I did not wanted to inject Open3D as dependency, since each platform has its own ways to struggle with this. And I don't want to troubleshoot open3D installation problems :P. If you check the visaulizer code, you will find some nice tricks on how to do this at runtime ;)

class RegistrationVisualizer(StubVisualizer):
    # Public Interaface ----------------------------------------------------------------------------
    def __init__(self):
        try:
            self.o3d = importlib.import_module("open3d")
        except ModuleNotFoundError as err:
            print(f'open3d is not installed on your system, run "pip install open3d"')
            exit(1)

And then on the class, you can use self.o3d the same way you would use o3d, ie: self.source = self.o3d.geometry.PointCloud().

So at the end it doesn't really matter how dirty, data-depenendt, etc the data loader is. As long as there is one use case, I guess the community will always appreciate it and build on top of those.

About temporal assumptions

@jhpauls I guess you are mostly right. And I apologize you had to reverse-engineer this, it was never the intention. Hopefully when the full code is out there you wouldn't need to guess. I'm planning to also add documentation after releasing the code on how to implement custom dataloaders and I will point this information with the proper credit ;) Thanks.

About deskewing v2.0

I find your opinion very interesting since for me it's quite counterintuitive: The same point observed in front of the vehicle might have a 1.0 m different position compared to being observed in the back of the same vehicle a bit later when driving with 20 m/s. Do you rely so much more on temporally closer (and similarly distorted) scans that this temporally far contradiction is overruled when the odometry is calculated? Did you compare the effect of deskewing with ground truth poses?

You are right, and indeed when the speed of the vehicle is quite fast, not having a proper deskewing (our case) is your enemy. Although on average, will never be that "severe", since you do not only use those "extreme" points. Again, for mapping, your map would look a bit ugly, but for odometry estimation, which is odometry estimation, not full-blown-slam-localization-system-bullet-proof 😅 , having a simply deskewing seems to work just fine. Of course, kiss-icp is just a baseline to build a real SLAM system on top of it. And not just using it as it is on a final real-world application. We tried to simplify it to its minimum to have a solid starting point(but not perfect). On our experience, aiming for perfection is what complicates (sometimes) the generalization of the system ;)
I hope this reply your concern.

from kiss-icp.

Hello @jhpauls, thanks for such interesting question.

The assumptions are:

• You are using a LiDAR sensor, where each point in the scan is taken at different scanning times. By looking at your picture, it looks like you are using a rotating LiDAR sensor. So no problem here.
• You must have timestamp data available, if you are using the rosbag, then we try to read from the t or timestamp field this information. If you are using the generic dataloader (the one that reads from folders) then, we can't be sure of which type of sensor you are using, and the timestamp field is ignored. Meaning that, if you have files on your system, you will need to create a custom dataloader and provide the way to read or compute the timestamp information. Examples on how to do this can be seen on the MulRan dataloader (for ousters) and on the Kitti-raw dataloader for Velodyne.
• The last assumption, which is the strongest one, is that the sensor is moving with “constant” velocity. Because we use the previous estimation to deskew the point cloud. For ground-base vehicles, this mostly holds true. Due to the inertia of the robot, for example, a car can't really change its velocity in 0.1 seconds (if we assume a 10Hz LiDAR). That being said, for more "shaky" scenarios this assumption might not hold, and indeed the deskewing might not be perfect. But turns out that in those scenarios deskewing might not be necessary (for odometry estimation)

A word about deskewing:

Historically, the robotics community has been worrying(in my humble opinion) way too much about deskewing. While it's true that indeed is a problem present on the data inevitable for most LiDAR sensors, for odometry-estimation(that is, what we do) is not always so relevant. Most LiDAR nowadays can operate at 10 or even 20 Hz, which basically means that the distortion occurred in the scan might not be your worst nightmare. Nevertheless, for mapping applications, I do see that this is a more problematic situation. Also keep in mind that not all the points in the scan are affected in the same way, the distortion present between two consecutive timestamps is not the same if you compare the one from the first and last timestamp.

PS: Remember to pass the --deskew flag when processing your data

from kiss-icp.

Wow, that's a comprehensive answer worth waiting for! 👌

Indeed, we are using a rotating lidar (Velodyne VLS-128 Alpha Prime) and we do have precise information for each pose as well as for each point that I would like to use. As far as I see, you right now also assume a fix scan frequency, which is just fine for us. But that means you only consider timestamps within the scan.

For the intra-scan timing information, thanks for pointing towards a custom dataloader. Indeed, I was able to take a look at the MulRan / Kitti-raw dataloaders and implement my own one that works just as expected. 👍
As this is a custom data format, I do not think a PR makes a lot of sense, especially since it introduces a special version of pypcd as dependency. For my findings and anyone else who wants to implement a custom dataloader, I'll leave a few hints at the bottom.
The results are just fine now:

About the deskewing

I find your opinion very interesting since for me it's quite counterintuitive: The same point observed in front of the vehicle might have a 1.0 m different position compared to being observed in the back of the same vehicle a bit later when driving with 20 m/s. Do you rely so much more on temporally closer (and similarly distorted) scans that this temporally far contradiction is overruled when the odometry is calculated? Did you compare the effect of deskewing with ground truth poses?

How to implement a custom dataloader with timing information

If anyone else is trying to implement a custom dataloader with timing information, you might find the following reverse-engineered assumptions useful:

• timestamps are returned by the dataloader next to the points and need to match the points returned in size and order
• timestamps are assumed to be in the interval 0.0 to 1.0 and are then converted to something like -0.05 to 0.05 in deskew.py:35
  • if your scan is slightly larger (e.g. due to firing timings), that does not hurt as long as you make a 360° turn from 0.0 to 1.0
• during deskewing (in the C++ code), there is a temporal linearity(?) assumption so you need to sort the points according to the timestamps or you will get strange artifacts (bendy lampposts etc)

@nachovizzo et al., please correct me with those assumptions if I'm wrong. :)

PS: Besides the discussion about the influence of deskewing I would consider the issue solved. Feel free to close the issue if you want.

from kiss-icp.

Amazing! Thanks for so much details. I'm curious @jhpauls where are you find the C++ source code? It seems not availble in the repo (as all python code too, but python at least we can see with pip show -f kiss-icp :) )

Also I have a working prototype of a Custom Data loader from a pcap of one of the Lidar sensor on the market, so I'm curious to contribute to the repo if it makes sense to integrate more Lidar formats.

And I can say that kiss-icp results are amazing with high density lidars and very long drives for 10Gb worth of data in the city in a loop may have smth like 2-3 meters of a drift, which super cool!

Looking forward to work with a source code!

from kiss-icp.

Ah sorry, I did not have access to the C++ code, but was simply looking at the Python code that comes with the pip package. But given the preprocessing in Python, my knowledge about firing timing and the pattern in the artifacts when using unsorted timestamps I was able to draw some conclusion about the underlying deskewing approach. 🤷

I guess working with .pcap files is extremely sensor specific since you need the whole unpacking logic of each sensor, right?
Maybe a better way would be to support timing fields in .pcd files (afaik they are not standardized with proper resolution and I am not aware of any nice library that reads them) or even .las/.laz files. But I mean, you can barely complain about this. kiss-icp already supports quite a bunch of formats and datasets. 😅

from kiss-icp.

Thanks, I see. Waiting for the C++ then. But I also was satisfied with source code in Python lib and agree, the datasets supported are staggering.

from kiss-icp.