This repository contains PyTorch implementations of the camera models used in the COLMAP structure-from-motion pipeline.

The camera models support automatic differentiation for project and backproject functions. Which for some reason are called map and unmap in this repo.

This code was mainly developed for my own research purposes.

Just git clone this repository to your project folder.

git clone https://github.com/DaniilSinitsyn/colmap_cameras_pytorch.git

impot torch
from colmap_cameras_pytorch.colmap_cameras import model_selector

# Model defined as a string in colmap cameras.txt
camera_txt = "SIMPLE_RADIAL 100 100 100 50 50 0.3"
camera_model = camera_txt.split()[0]
camera_params = torch.tensor([float(x) for x in camera_txt.split()[1:]])

# Create model based on the colmap string
model = model_selector(camera_model, camera_params)

# project 3d points onto image
pts3d = torch.rand(10, 3)
points_2d, valid = model.map(points3d)
# unproject 2d points to the ray
points_3d = model.unmap(points_2d)

As everything is differentiable, you can optimize camera parameters using PyTorch's optimizers.

model.require_grad = True
optimizer = torch.optim.Adam([model._data], lr=0.01)

for _ in range(iterations):
    optimizer.zero_grad()
    ...
    loss = ...
    loss.backward()
    optimizer.step()

By default camera's center is fixed. If you want to optimize it too:

model.OPTIMIZATION_FIX_CENTER = False

There are in total 4 flags that can be set for each camera:

• OPTIMIZATION_FIX_FOCALS: Fix focal lengthes (default: False)
• OPTIMIZATION_FIX_CENTER: Fix principal point (default: True)
• OPTIMIZATION_FIX_EXTRA: Fix extra parameters (default: False)
• ROOT_FINDING_MAX_ITERATIONS: Number of iterations for root finding (default: 50)

All camera models are supported:

To use a specific camera model you can import it directly from the colmap_cameras.models module.

import torch
from colmap_cameras_pytorch.colmap_cameras.models import Pinhole

image_shape = torch.tensor([[100, 100]]).float()
params = torch.tensor([100, 100, 50, 50]).float()
model = Pinhole(params, image_shape)

apps.refit_model is a simple script that uses Gauss-Newton optimization to fit one camera model to another.

python3 -m apps.refit_model --input_camera "SIMPLE_RADIAL 100 100 100 50 50 0.3"  --output_camera "RADIAL_FISHEYE" --iterations 20

Some camera models require solving polynomial roots. For high-order polynomials, the only way to do this is to use a numerical solver.

I don't like the fact that automatic differentiation goes through Newton's method or the QR algorithm.

This repo contains an extention of torch.autograd.Function for Newton's method and Companion matrix root solver.

To run tests:

python3 -m tests.run_tests -v

• Add remap app, that generates remaps alongside with a class to run them.
• Estimate image area where camera is valid for each model. (Basically to check whether distortion is monotonic)
• Visualisation util for the previous point.