Test loftr in inloc about image-matching-toolbox HOT 12 OPEN

Anyhow, thanks for your response. I will try LoFTR-OT and response the results here.

from image-matching-toolbox.

Hi @xunfeng2zkj ,

Could you detail what your issue is?

from image-matching-toolbox.

Can I cite this result and this repo in my paper？

from image-matching-toolbox.

@xunfeng2zkj ,

Do you mean for Loftr? I think in the original paper, they are not using this repo, so I would rather stick to the paper numbers to compare to Loftr on InLoc.

Regarding citation, if you use it to run your own methods, yes you can cite this repo.

On the other hand, I remember I have obtained better results for LoFTR using this repo :
(48.5 / 73.2 / 83.8 | 55.0 / 75.6 / 83.2)

Can you try with the config I used:

inloc:
    <<: *default
    match_threshold: 0.5
    npts: 4096
    imsize: 1024
    pairs: 'pairs-query-netvlad40-temporal.txt'
    rthres: 48
    skip_matches: 20

from image-matching-toolbox.

Hi, I have also tested LoFTR but only get this result.

This is my setting:

Maybe I need to use the indoor weights?

from image-matching-toolbox.

Hi @ewrfcas ，

As far as I remember their outdoor weights in general worked for me. Have you tried to reproduce loftr on hpatches first #20 (comment) ? Just incase there are some setup issues with Loftr. And I will probably try this again on InLoc once I find some time to check it. For now you can just tuning a bit the parameters like ransac_thres or match_threshold and see first how they change the performance.

from image-matching-toolbox.

Thanks for your reply. I have tested HPatches with the AUC of 3px:64.58, 5px:74.77, 10px:84.32.
Besides, I read the LoFTR paper and find that they used LoFTR-OT, while I used LoFTR-DS. Would it cause the performance gap?

from image-matching-toolbox.

Hi, I have tried another setting: confidence threshold=0.2, and achieve

This result is better than the previous one with a 0.5 threshold:

Any comments?

from image-matching-toolbox.

I remember that I removed the code "try...except..." for the debugging.
Would it have any influence on the final results?

from image-matching-toolbox.

Hi @ewrfcas,

For the HPatches, this looks good. According to zju3dv/LoFTR#65, they are using a finetuned version of Loftr-ot. To be honest, I don't remember whether I have tried Loftr-ot yet.
Also I have recently pull the latest version of LoFTR so maybe also something has been updated since the last time I run loftr myself using immatch.
To be honest, I can not comment much on this without investigate it myself..but I am not sure how long it will take before I get back to you on this.

On the other hand, I can not guarantee that the way I evaluated LoFTR , e.g., how I do the quantization, is the same as what the LoFTR authors did. If you want to compare to LoFTR, I would recommend to use their released number, but you can still use this repo to evaluate your own methods.

from image-matching-toolbox.

hey guys, i'm using the default setting of this repo to test loftr in Inloc, but the speed is really slow. the tqdm says 6 hours are needed, is that normal ? Thanks! btw, i'm running on a single A100, loftr can run fast on Megadepth with its own repo.

Also, the evaluation doc has some misleading information, in the data tree, the 'InLoc' folder should has a subfolder named 'database', but not 'dataset', according to the 'pairs....txt'.
cc @ewrfcas @GrumpyZhou @xunfeng2zkj

from image-matching-toolbox.

There's ~14k pairs for inloc inference pairs. You can check the time used for matching image pairs only and check whether this is normal.

Thx for pointing out the eval doc mistake. I will update.

from image-matching-toolbox.