Comments (12)
Sure 👍.. Just ping me when anything is needed to be done from my side :)
from image-matching-toolbox.
Sounds good! Just ping me here, or at email: [email protected]
from image-matching-toolbox.
I have looked at the code structure and it seems to be very good. One thing, which I need to discuss with the other IMC orgs, is that for the stereo what your code does in order to generate "keypoints", violates the pairwise assumption. And if one does it anyway, then it would be only rational to perform some kind of optimization or bundle adjustment.
So maybe the correct way is for us to change the expected format to be suitable for pairwise methods.
I will update you soon.
from image-matching-toolbox.
Hi @ducha-aiki ,
Actually, I had them roughly coded (80%) some time ago, but unfortunately i never get time to start testing them 😅😅😅.
How about i clean my local code first and push to another branch first? Probably this (latest next) weekend.
Then you can help me in testing them and improving, especially you seem really familiar with the benchmark.
And thanks for offering the help, that's definitely appreciated !
from image-matching-toolbox.
Hi @ducha-aiki ,
Sorry for the late update.. I finally got time to clean some code and push them.
Now they are in the new branch dev_imc.
I scratched a development log dev_imc.md.
I have wrote some of my ideas about how i want to organize the datasets, outputs. Feel free to change them or add comments there.
Anything regarding the data, code (namings, code lint, logic) are open for discussion and changes.
Just ping me if anything is unclear to you!
Regarding the time schedule, no pressure and no rush at all.
And thank you again for offering to improve this repo together :)
from image-matching-toolbox.
OK, here is an update: I modified the benchmark code in the branch (not yet merged), which allows submission of "pairwise" keypoints, which are unique for every image pair.
ubc-vision/image-matching-benchmark#42
The caveat: this works only for the stereo.
For the multiview we are thinking about the following options: just concat the "pairwise" keypoints and resp. modify match indexes. However, this would rely on COLMAP ability to merge the duplicating disjoint keypoints.
The second option is to allow participants to merge/BA the keypoints themselves and submit - as usual - single keypoint list per image.
from image-matching-toolbox.
The template (with LoFTR) to generate such "pairwise" submission is here https://github.com/ducha-aiki/imc2021-sample-kornia-submission/blob/loftr/extract_loftr_matches.py
It does not include RANSAC though
from image-matching-toolbox.
Hi @ducha-aiki ,
Thanks for the update, i think for stereo task it's good to have the support of pairwise matching.
Regarding multiview task, this repository already has the support for those methods that are not keypoint-based and see the implementation in localize_sfm_helper.py.
What i am doing there is:
- Extract matches for all pairs
- For each image, collect its set of keypoints by finding all of the points that are involved in the matches extracted from all image pairs.
- Merge those keypoints by quantizing the very nearby keypoints w.r.t a quantize threshold.
So i need to first extract all the matches, then i compute the keypoints.
This heuristic solution is validated on Patch2Pix to improve significantly the localization accuracy on Aachen Day and Night original version. I also applied this to Loftr, which also has this issue and results are: 89.1 / 94.9 / 98.3 87.8 / 93.9 / 100.0. If i just feed the raw keypoints that are concatenated, the performance will be really bad, which kinda refers that colmap is not handling those nearby keypoints well. While this solution is not perfect, it is rather handy since it doesn't bring much computation burden during evaluation.
So i hope this also applies to the multiview task in IMC.
Another thing, it seems i can not open the pairwise example: https://github.com/ducha-aiki/imc2021-sample-kornia-submission/blob/loftr/extract_loftr_matches.py.
from image-matching-toolbox.
Oh, so we basically can recommend your solution as an official one for the multiview. The remaining thing (for us at IMC) is to tell the code to take pairwise matches for stereo and per-image matching in mutiview, which we will do next week.
May ask then you to submit Patch2Pix entry to IMC, to make sure that everything is working is required?
Regarding the example - I have merged it to the master branch, sorry :)
Here it is https://github.com/ducha-aiki/imc2021-sample-kornia-submission/blob/master/extract_loftr_matches.py
from image-matching-toolbox.
Offtopic: We have recently added LoFTR to kornia
https://kornia.readthedocs.io/en/latest/_modules/kornia/feature/loftr/loftr.html#LoFTR
I think, it would be good for everyone to add Patch2pix there as well - for users it would be easy to use, for you - potentially more users. What do you think?
from image-matching-toolbox.
Sure, i can submit Patch2Pix when you want to examine your updated benchmark.
One thing about the quantization solution you need to notice is that this sacrifice the accuracy during the quantization to have the repeatability, maybe when you write the documentation, you can inform the user what they should expect here.
When you say Patch2Pix, you mean the refinement part or the whole network when combined with NCNet or both?
I also think it's good to have it added to IMC. It might also help me to understand more where it works and where not.
from image-matching-toolbox.
I mean both - as a end-to-end matching module, and as a way to refine correspondences as well :)
from image-matching-toolbox.
Related Issues (20)
- LoFTR has a bug in position_encoding.py
- COTR no matches found in hpatches test HOT 3
- Test loftr in inloc HOT 12
- Where is the alignments in InLoc? HOT 1
- Missing RobotCar/queries HOT 9
- robotcar-season eval run very slowly HOT 3
- Test loftr in aachen HOT 2
- python setup.py develop failed HOT 1
- what is the meaning of $colmap path ? HOT 1
- The possibilities of python to C++ HOT 1
- Feature point annotation tool HOT 10
- Errors in loading state_dict for patch2pix HOT 1
- FileNotFoundError: no 'cameras.bin' when test loftr on aachen
- Images illumination change evaluation
- I have a question
- Installation Problems, HOT 1
- Pycolmap Error not found during install HOT 1
- IMMATCH ModuleNotFoundError HOT 1
- Download.sh for pretrained weights DO NOT WORK HOT 1
- Patch2Pix AttributeError: 'Conv4d' object has no attribute 'weight'
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from image-matching-toolbox.