mbrossar / ai-imu-dr Goto Github PK

Hi, @mbrossar. I want to use my data(rosbag file) to train the network, firstly I need to transfer my data to your input format. The rosbag file has IMU raw data and GPS data, I'm a little confused of the the following pairs:

(ax, ay, az) (af, al, au)
(wx, wy, wz) (wf, wl, wu)
(vn, ve, vf) (vf, vl, vu)

What's the difference with them and which one is the sensor raw data?

Hi,

First thanks for your great work and open source.

I have downloaded the code and run it on my laptop, it seems like the location results get similar results whether or not training P0 and Q (I comment the #iekf.set_learned_covariance(torch_iekf) in main_kitti.py which the Q and P0 running in initial value) , so i want to ask it is true or i need to comment or change some other code?

#6

Hi Martin,

Hope you are doing good. I am willing to train the neural network with my own data sequence(prepared data sequences in kitti format) and ended up in some errors.Please can you give your inputs on below :

I have installed torch version 1.4.0 but I did not have Cuda installed in my laptop.Is cuda is mandatory to run the training?

Best Regards,
Harish

Hey! thanks for your work!
Excuse me for taking your time with, maybe, a silly question, but i can't figure out derivation of H Jacobian in your "RINS-W: Robust Inertial Navigation System on Wheels". I've got no problems with A and C. But following general method i always got same HVel,V as yours, but without Rt multiplication:
[0 I ...] vs [0 R' ...]
[Gx 0...] vs [R'*Gx 0 ...]
Can't figure out where R' comes from... Will appriciate any help.
Although this question is not about this particular code but it is about this particular method (same derivation should be used for velocity update in ai-imu paper) and your answer will be extremely helpful.

Thanks in advance!

Hello,

I keep getting an error that says modulenotfounderror: liegroups. I was wondering what this is.

I found on the internet that this is apart of the utils package however it is not referenced anywhere in your code.

Could you please explain where this is coming from?

Hey,

I was wondering where I can find the training figure results? If there aren't any can you please send me the code to create the trained figure results?

Thank you,
Sorry for so many questions

Hello again!
Using RIEKF framework (on SE2(3) group as in preliminary paper from RINS-W) i've noticed that with ZUPT, LAT, UP pseudo measurements uncertainty of position reaches huge numbers with order of 1e+5(6..7..). Which can be explained by the fact that we never measure position, but only predict it. But... with time (~20sec with 100hz) this huge numbers give negative influence on filter's stability. Did you experience this issue in your experiments? or maybe it just caused by my mistakes in filter architecture... Thank you very much for help!

Hi,

First thanks for your great work and open source.

I have one question in theory. In your algorithm, only the null lateral and vertical velocity assumptions are utilized, but no constraints on the forward direction. I could not understand how the bias in the moving forward direction is estimated and how the estimation error in that direction can be corrected. I mean the state does not violate the 2 assumptions when the errors in that direction are very large. But those errors are not observed in your experiments.

Could you give me some advise about how to explain those phenomenons?

Thanks a lot.

Yanming

Hi, I know the loss is the the relative error as it's described in the paper:

We stress the loss function consists of the relative translation error trel

But how is it calculated in the code? I'm confused of the following code in compute_delta_p(Rot, p) function. What's the seq_lengths? why does it consist of 100, 200, .., 800 ?

    seq_lengths = [100, 200, 300, 400, 500, 600, 700, 800]
    k_max = int(Rot.shape[0] / step_size) - 1

    for k in range(0, k_max):
        idx_0 = k * step_size
        for seq_length in seq_lengths:
            if seq_length + distances[idx_0] > distances[-1]:
                continue
            idx_shift = np.searchsorted(distances[idx_0:], distances[idx_0] + seq_length)
            idx_end = idx_0 + idx_shift

            list_rpe[0].append(idx_0)
            list_rpe[1].append(idx_end)

        idxs_0 = list_rpe[0]
        idxs_end = list_rpe[1]
        delta_p = Rot[idxs_0].transpose(-1, -2).matmul(
            ((p[idxs_end] - p[idxs_0]).float()).unsqueeze(-1)).squeeze()
        list_rpe[2] = delta_p
    return list_rpe

Hello. I recently read your paper and I am interested in implementing your dead reckoning algorithm on an RC car equipped with a raspberry pi, GPS, and IMU. I was wondering if you had any suggestions for where to start if I wanted to modify the code to work with data I record from the car to train the neural network and then test the algorithm in real time. From glancing through the code, my intuition is that I need to change the datasets.py class and it would be nearly plug and play, but I thought that I would get your insight first.

Il subsiste ce dogme, sur la propagation des incertitudes, il faut bien les calculer une à une pour avoir la prétention d'estimer une position. Avant tout il faut et il suffit de mesurer les covariances associées aux données extereos.

thanks for making your work public, i follow your instructions but something going wrong.....

maosongwang@T480s:~/work/ai-imu-dr/src$ python3 main_kitti.py
IEKF nets loaded
Test filter on sequence: 2011_09_30_drive_0028_extract
Traceback (most recent call last):
File "main_kitti.py", line 483, in
launch(KITTIArgs)
File "main_kitti.py", line 31, in launch
test_filter(args, dataset)
File "main_kitti.py", line 437, in test_filter
to_numpy=True)
File "/home/maosongwang/work/ai-imu-dr/src/utils.py", line 9, in prepare_data
t, ang_gt, p_gt, v_gt, u = dataset.get_data(dataset_name)
File "/home/maosongwang/work/ai-imu-dr/src/dataset.py", line 77, in get_data
pickle_dict = self[self.datasets.index(i) if type(i) != int else i]
File "/home/maosongwang/work/ai-imu-dr/src/dataset.py", line 57, in getitem
mondict = self.load(self.path_data_save, self.datasets[i])
File "/home/maosongwang/work/ai-imu-dr/src/dataset.py", line 148, in load
pickle_dict = pickle.load(file_pi)
ModuleNotFoundError: No module named 'liegroups'

In APPENDIX A , 'As (15) are measurements expressed in the robot’s frame, they lend themselves to the Right IEKF methodology'. But I don't know why it use RIKEF? In other words, when to use LIEKF and when to use RIEKF. Thank you!

hi, training parameters url is not found, please check it (https://cloud.mines-paristech.fr/index.php/s/5wjlZpZPEgV8l9p/download). And I am sorry that I have not had time to read the paper carefully. I would like to ask if this method is suitable for general scenes or for the more stringent requirements of model parameters. Thank you.

Dear Martin

First thanks for your great work.

I want to know the batchsize in your code, could you show me please.

Thank you so much!

I'm looking forward to your reply.

first , i have ModuleNotFoundError: No module named 'liegroups'
and then i install liegroups through https://github.com/utiasSTARS/liegroups
and then it failed in ModuleNotFoundError: No module named 'future'

Thanks for sharing the code. However, I got the following error:
../data/2011_09_26_drive_0056_extract.p
../data/2011_09_26_drive_0084_extract.p
../data/2011_09_26_drive_0086_extract.p
../data/2011_10_03_drive_0034_extract.p
Traceback (most recent call last):
File "/home/hao/Desktop/ai-imu-dr/src/main_kitti.py", line 482, in
dataset = KITTIDataset(args)
File "/home/hao/Desktop/ai-imu-dr/src/main_kitti.py", line 119, in init
super(KITTIDataset, self).init(args)
File "/home/hao/Desktop/ai-imu-dr/src/dataset.py", line 56, in init
self.set_normalize_factors()
File "/home/hao/Desktop/ai-imu-dr/src/dataset.py", line 96, in set_normalize_factors
pickle_dict = self.load(self.path_data_save, dataset)
File "/home/hao/Desktop/ai-imu-dr/src/dataset.py", line 151, in load
pickle_dict = pickle.load(file_pi)
AttributeError: Can't get attribute 'SO3' on <module 'liegroups.torch.so3' from '/home/.conda/envs/myenv/lib/python3.6/site-packages/liegroups/torch/so3.py'>

I have installed liegroups package and passed the pytest. Just wonder whether anyone else has encountered the same error before.

Thanks!

Hello,

I was wondering what the loss_train variable does inside of train_torch_filter. I would also like to know where the checkpoint is in your code. If I needed to stop running the code for some reason, I just wanted to make sure that there is a checkpoint saving where I have stopped.

Thanks

Hi,

First thanks for your great work and open source.

I have run your code on my laptop, and the result is really cool.
But I found somewhere your code not same with the paper. In the utils_numpy_filter.py, the way to calc the H_v_imu is different.


Could you please give me some explanation？

Thank you very much.

Hello, I tried to replace the input with wx, wy, wz, ax, ay, az, roll, pitch, yall, the code is:
if iekf.mes_net.training:
u = dataset.add_noise(u)
u = u.numpy()
u = np.hstack((u, ang_gt)) #6+3=9
u = torch.from_numpy(u)
but when I run the code, there is an error:
RuntimeError: Given groups=1, weight of size 32 9 5, expected input[1, 6, 2000] to have 9 channels, but got 6 channels instead

I don't know how to deal with it, could you give me some suggestion, thank you!

Hi,

First thanks for your great work and open source.

I have downloaded your code and run it on my laptop, the result is really cool. But I notice your another article "RINS-W: Robust Inertial Navigation System on Wheels" , I have not found the code.
So, if you can upload the code of the article "RINS-W: Robust Inertial Navigation System on Wheels", I will be deeply grateful .

Thank you very much.

Hello Martin, Can you give me your point of view about this publication?https://ieeexplore.ieee.org/abstract/document/8668708,
Regards, EO.

PS : I see the point, this is as if they have continued with the derivations included the commands,

See the equations or the pool of equations 2,3 and 4.

This should be another advance in the way of doing with the odometric model but I cannot find any bibliographic references according to the so called new equations of such a best method.

I need information about the fields in the pickle files;
'ang_gt'
'name'
'p_gt'
'sm_gt'
't'
't0'
'u'
'v_gt'

Thanks in advance.

some pickle files do not include factors declared in the read_data function, such as ‘t0’.

Hello,

Is there an error metric equation in your code? If there is, where can I find and print the result from the error metric. I would like to make some comparisons and will need to check out the exact number from the result.

Thanks

Hi, I am quite interested in your work and your results look awesome on Kitti Dataset. I think the Pseudo-measurement derived from car motion plays an important role in the success. And I am wondering whether it is possible to extend your work to MAV Dataset like EuRoC dataset. Since MAV motion is more complex and there is no Pseudo-measurement here, I am trying to look for an alternative observation model. Currently, I am trying to fuse the translation direction ( scale is unknown) and rotation observation observed by visual information into your work, but find some trouble due to difference in frame rate of observation (from camera, 20 Hz) and IMU rate (200 Hz).

Would you please give me some advice about this? I will really appreciate it.

Cheers

Hello,

I am researching an imu project and have been reading your paper as reference for my research. I was hoping you could clarify where (path_data_save) is saving in dataset.py line 64. If you could get back to me as soon as possible that would be great!

Thank you very much for enlighting work and published code.

In many other works on orientation estimation I noticed there is an additional "re-normalization" step of the rotation matrix, making sure it is unitary and the norm of the columns are all one.

I havn't noticed any such stage in the proposed algorithm.

How can the code guarantee that the estimated rotation matrix, e.g, R_IMU, is indeed a "proper rotation" matrix?

I looked at the resulting "z direction" (last column of the matrix) and noticed that its norm significantly deviates from one.

How can I rectify this?

Thanks,
-Shai

Hello, Martin!
If i want to fuse GPS measurements of position\velocity (which are left invariant in nature) with this RIEKF estimated state should i just do LIEKF propagation\update considering RIEKF state as "previous step" (N-1) state? or there is some special method to do this? Thank you very much for help!
UPD: or maybe it is pretty correct to just use method you showed in #22 with Y(X)=p

Hello ， I want to know how to get poses from the pure IMU integration .It doesn't seem to appear in your paper.Thank you !

Hello ， in this paper AI-IMU Dead-Reckoning, the Fig. 1 in this paper.
I want to know how to draw the pure IMU integration (cyan) .
Thank you !

Thank you for your great work!

I have some questions about what you mean by "00-11 KITTI". I have not used KITTI dataset before and I see there are multiple categories of datasets e.g. City, Residential and so on, and each category has many sequences. I would like to ask which of them are "00-11 KITTI"?

Thank you very much!

Hi,

First thanks for your great work and open source.

I have downloaded your code and run it on my laptop, the result is really cool. But I notice that both the KITTI dataset and your another article "RINS-W: Robust Inertial Navigation System on Wheels" 's dataset the IMU gyro and accelermeter precision is really low, it break its limition and reach a good result using iekf and training methond.

Now I am working on high precision IMU navigation such as fog imu(bias stablity < 0.05 deg/h) so I just want to ask : are you think about the proposed method will have good affect on high precision IMU . I want to try, but not for sure.

Ayant un peu travaillé sur ce sujet, je pense que c'est vraiment très ennuyeux de ne pas posséder un modèle odométrique pour faire changer lesdites covariances et surtout il faut faire attention à bien ordonner les choses.

Bonjour Martin, pouvez-vous m'en dire plus sur la trajectoire avec l'aide d'un ladar, de quoi s'agit t'il?

Dear Martin Brossar, first of all thank you for such a great paper and making the code public. I have spent couple weeks on getting it run with the kitti dataset. But i am not able to do so. In one of the issues you had solved the same. But it is to vain, I get the below error irrespective of which kitti dataset i try. i have pointed the directory to the top level of kitti dataset, the path looks like

/dataset/kitti/2011_09_26/2011_09_26_drive_0019_sync

But still it throws the below error.

Hi,
I really like the simplicity of the formulation and the use of "pseudo measurements" to guide the filtering process.

I would like to extend the proposed method to other types of motions, for example a "pendulum like" motion where the IMU frame velocity is mostly in the up/dn direction and zero for lateral /forward motion. Moreover, there is a periodicity to the rotation, velocity and position - they can be assumed as having zero mean.

Is there a way to get a more detailed description of how the measurements Jacobian matrix H_n is calculated?

Thank you!

Dear Martin,

First thanks for your great work and open source. This is an interesting approach!

I have one question in theory. In your paper, noise parameters Qn and Nn will be learned by a neural network. I have downloaded your project from GitHub, and tried to consider the Qn and Nn parameters use the default values and fixed. I have used both your trained model and my own trained model. However, there seems to be no obvious difference between the two methods.

Could you give me some advise about how to explain those phenomenons?

Thanks a lot.

C.Chengtao

Hi,

First thanks for your great work and open source.
I have a question about the IMU state initialization.
In your code,
b_acc=np.zeros((N,3))
Then, you use acc = Rot_prev.dot(u[3:6] - b_acc_prev) + self.g to calculate the current acc.
In fact, u[3:6] contains the acceleration of gravity,but the b_acc_prev does not. I cant understand why.
In my opinion, b_acc_prev should be init as np(0,0,-9.80665).

Hello,

I found your paper very interesting and thanks for sharing the codes! I test the filters by running main_kitti.py, and got similar results as in paper. However, I didn't find in codes where you load the trained model for test. I deleted the provided training parameters (delta_p.p, iekfnets.p, normalize_factors.p), and it still generates the same results. Did you hard coded your trained parameters somewhere?

Thanks for the work. What's the license for this repo?

Hello,
you have computed the tranlation error and rotation error with odometry development kit, but I can not download it. Could you please provide me with an alternative link that I can use to download the development kit? Thank you very much!

Hi,
Thanks for sharing this great work!
I read your paper and found the results in Table 1,which shows Translation Error and Rotational Error. But the data in your project code is 100 Hz so we got estimated poses with 100 Hz, while Kitti Visual Odometry benchmark(11 training sequences) is 10 Hz, they are not correctly synchronized although just downsample the raw data(100 Hz) to 10Hz. Therefore I don not know how to compute correctly translation error / rotation error with Odometry development kit from http://www.cvlibs.net/datasets/kitti/eval_odometry.php.
Can you explain in detail how the results in the Table 1 are calculated?
Thank you so much for reply~

I am wondering where you found the specification values regarding the imu of the Rt3003 in your paper. Here you say that the imu has a gyro and bias stability of 36 deg/h and 1 mg, respectively. However in the datasheet of the Rt3003 v3 (here) it claims to have a gyro stability of 2 deg/h and bias stability of 2 μg, which would be much better.

So where did you find these values and why are they so different from the official datasheet?

Thanks for you sharing, I'm puzzled with the derivation of velocity in the car frame and measurement matrix H equations.

Could you please provide the full derivation of equations (14) and (37), any help appreciated.

Hello, thanks for your open source code.It's very helpful for me. But i cna't download reformated pickle format of the 00-11 KITTI IMU raw data and training parameters in the link you provided. Would you provide another available link for me? Thank you very much.

Hi,
path_data_base = "./media/mines/46230797-4d43-4860-9b76-ce35e699ea47/KITTI/raw",
I want to know if this path needs to be manually created locally. What data is placed under this path?

Thank you so much in advance.

Hi, first thanks for your great work. I have a question about how to generate ground truth data from Complex Urban LiDAR Data Set? Looking for your reply, thanks!