mistermap / undeepvo Goto Github PK

View Code? Open in Web Editor NEW

11.0 11.0 12.0 5.74 MB

Realization of UnDeepVO method

Python 2.25% Jupyter Notebook 97.75%

undeepvo's People

Contributors

Stargazers

Watchers

Forkers

ilinvalery kichkun dinarkino kshpv i-khokhlov hitf vorkorro14 liubamboo satoshirobatofujimoto essamgoda nkadiadrian

undeepvo's Issues

U-Net model is not correct

class LastUpBlock(nn.Module):
    def __init__(self, in_channels, out_channels):
        super().__init__()

        self.convs = nn.Sequential(
            nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1),
            nn.ReLU(),
            nn.Conv2d(out_channels, out_channels, kernel_size=3, padding=1),
            nn.ReLU(),
        )

    def forward(self, x):
        out = self.convs(x)

        return out

should be

class LastUpBlock(nn.Module):
    def __init__(self, in_channels, out_channels):
        super().__init__()

        self.convs = nn.Sequential(
            nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1),
            nn.ReLU(),
            nn.Conv2d(out_channels, out_channels, kernel_size=1, padding=0),
        )

    def forward(self, x):
        out = self.convs(x)

        return out

Сделать supervised depth regression

Проверить temporal и spatial photometric loss

Test MonoDepth2

Запустить Monodepth2 и сравнить графики лосов

Решение depth net вырождается в константу по глубине

Изменять интрисики камер при изменении размера изображения (reshape, crop)

train more than one sequence

How to train more than one sequence?
this train only 08 sequence what if I want to train more sequences (if this possible)
dataset = pykitti.odometry(MAIN_DIR, '08', frames=range(0, 340, 1))

Запустить нейронку с маленьким размером датасета

Нормализовать входные изображения

Посчитать Mean и std

Взять метрики из monodepth и тестовый датасет и прогнать на нашей модели

AssertionError: scalar should be 0D

Hello, Why am I getting four dimensional scalar data? How do you do that?

monodepth2 data crop? data resize?

Улучшить вывод в tensorboard и mlflow

Вывести left и right глубину и картинки
Вывусти все лоссы в tensorboard и в mlflow
Graph
Вывести траекторию tensorboard
Артифакты в mlflow
Добавить вывод гипер параметров

ConnectionError: HTTPConnectionPool(host='329801-ilinvalery.tmweb.ru', port=5001): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=undeepvo (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8650e619e8>: Failed to establish a new connection: [Errno -2] Name or service not known',))

how to solve this issue

DisparityConsistencyLoss:  lambda = 0.85
PoseLoss: lambda_pose= ; lambda_rotation= ;
SpatialPhotometricConsistencyLoss: lambda = ;

TemporalPhotometricConsistencyLoss: lambda =0.85;

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-32-840e18527b00> in <module>()
     26 criterion = UnsupervisedCriterion(dataset_manager.get_cameras_calibration("cuda:0"),
     27                                   lambda_position, lambda_rotation, lambda_s,
---> 28                                   lambda_disparity, lambda_registration)
     29 handler = TrainingProcessHandler(mlflow_tags={"name": "MishaNotebook"}, 
     30                                  mlflow_parameters=mlflow_parameters)

2 frames
/content/undeepvo/criterion/spatial_photometric_consistency_loss.py in __init__(self, lambda_s, left_camera_matrix, right_camera_matrix, transform_from_left_to_right, window_size, reduction, max_val)
     17 
     18         self.l1_loss = torch.nn.L1Loss()
---> 19         self.SSIM_loss = kornia.losses.SSIM(window_size=self.window_size, reduction=self.reduction,
     20                                             max_val=self.max_val)
     21 

AttributeError: module 'kornia.losses' has no attribute 'SSIM'

Использовать ground truth позиции в предсказании stereo

lr sheduler

"The learning rate started from 0.001 and decreased by half for every 1/5 of total iterations"

Добавить возможность выбора через сколько итераций/эпох снижать, либо же что-то умное использовать из торча