zqpei / patchmatch_inpainting Goto Github PK

View Code? Open in Web Editor NEW

49.0 3.0 10.0 3.53 MB

Implementation of PatchMatch for image inpainting in cpp

License: Other

CMake 2.00% Python 7.04% C++ 84.04% C 5.99% Shell 0.94%

patchmatch inpainting opencv3 exemplar-based-image-inpainting

patchmatch_inpainting's Introduction

PatchMatch for inpainting

Introduction

This repository borrows most of the code from younesse-cv. However that repository is C style, which cannot be compiled using opencv 3.x. What I did is wrapping the code using opencv 3.x API in C++ style. Thanks zvezdochiot for suggestion.

Dependencies

cmake > 2.8
opencv 3.x
g++-4.7

How to use

Download this repository

git clone https://github.com/ZQPei/patchmatch_inpainting.git

Compile

mkdir build
cd build
cmake ..
make
cd ..

Run
As what does in run.sh.

for i in 0 1 2; do
  ./build/patchmatch image_files/inpainting/image/image_0000$i.png \
                     image_files/inpainting/mask/mask_0000$i.png \
                     image_files/inpainting/output/output_0000$i.png \
                     image_files/inpainting/metrics.log \
                     $i;
done

Or simple:

  ./build/patchmatch image_files/forest/forest.bmp \
                     image_files/forest/forest_mask.png \
                     image_files/forest/forest_inpaint.png

Demo

mask	masked image	inpainting image using PatchMatch	origin image

References

https://github.com/younesse-cv/PatchMatch

patchmatch_inpainting's People

Contributors

Stargazers

Watchers

Forkers

imageprocessing-electronicpublications asdlei99 zengxianyu xiongup cmf588124 vtalker jiachen0212 chenshiren168 guagod xifeng205

patchmatch_inpainting's Issues

g++-5?

$ g++ --version
g++ (Debian 4.7.2-5) 4.7.2

All works.

Bad alloc!

patchmatch_inpainting/source/main.cpp

Lines 63 to 65 in 0bea9c6

 int ** mask = (int **)calloc(int(height), sizeof(int*)); 

 for (int i = 0; i<width; i++) 

 mask[i] = (int *)calloc(int(width), sizeof(int));

Maybe?:

	int ** mask = (int **)calloc(int(height), sizeof(int*));
	for (int i = 0; i < height; i++)
            mask[i] = (int *)calloc(int(width), sizeof(int));

See:

patchmatch_inpainting/source/main.cpp

Lines 127 to 130 in 0bea9c6

 //free memory 

 for (int i = 0; i < height; ++i) 

 free(mask[i]); 

 free(mask);

name?

patchmatch_inpainting/CMakeLists.txt

Lines 27 to 28 in 53c3610

 ADD_EXECUTABLE(main ./source/main.cpp ./source/inpaint.cpp ./source/maskedimage.cpp ./source/nearestneighborfield.cpp ./source/qualitymesures.cpp) 

 TARGET_LINK_LIBRARIES(main opencv_core opencv_imgproc opencv_highgui)

main? Maybe inpaintcv?

Version for maintainer?

Maybe add to https://github.com/ZQPei/patchmatch_inpainting/releases :

0.1 "simpe version"

fix #1: bad alloc
fix #2: name tools
fix #3: simple tools
fix #4: change LICENSE
fix #5: g++ version

Take output of neural network as reference along with neighbour pixels

Hello @ZQPei and @zvezdochiot Thank you for your work!
I wonder if is it possible to improve results of deep learning approach with patch-match algorithm. It seems to me neural network should better understand the content of a gap in image. Unfortunately methods that we have tried output quite blurred texture. In contrast patch match is good at giving sharp textures.
How do you think will patchmatch work on neural network output? And where should I start in order to modify the code to take not only neighbour pixels but also pixels under the mask?

Here are some examples where neural network could improve in-painting results:
left – original; center – neural network; right – patchmatch (this repo);

notice border of column

deformation of road

smth brown in the water and creepy copy of the guy's hand

Thank you in advance for any help and thoughts!

First prepare a large test set of 1000 images or more. For simpility, just copy the existing test images for 400 times, the same to the masks. Rename then as image_00001.png, image_00002.png, etc.
Replace the main function as what I did at the very beginning. It was tested on 12000 images with 6 group of masks. You may need modify the code below a little bit to run on your test images properly.

If you find your memory usage growing continuously, that is the implicit problem hidden in the process function which will cause segmentation fault.

int main(int argc, char** argv) {

	char image_path[100];
	char mask_path[100];
	char masked_path[100];
	char output_path[100];

	double psnr_mean[6] = {0,};
	double ssim_mean[6] = {0,};
	double time_mean[6] = {0,};



	for(int i=0;i<6;++i){
		double psnr_total = 0.0;
		double ssim_total = 0.0;
		double time_total = 0.0;
		for(int j=i*2000;j<(i+1)*2000;++j){
			sprintf(image_path, "../image_files/inpainting/image/image_%05d.png", j);
			sprintf(mask_path, "../image_files/inpainting/mask/mask_%05d.png", j);
			sprintf(masked_path, "../image_files/inpainting/masked_image/masked_image_%05d.png", j);
			sprintf(output_path, "../image_files/inpainting/output/output_%05d.png", j);

			process(image_path, mask_path, output_path, &psnr_total, &ssim_total, &time_total);
		}
		psnr_mean[i] = psnr_total/2000.0;
		ssim_mean[i] = ssim_total/2000.0;
		time_mean[i] = time_total/2000.0;
		printf("[%02d%% - %02d%%] average psnr: %lf\taverage ssim: %lf\taverage time: %lf\n", i*10, (i+1)*10, psnr_mean[i], ssim_mean[i], time_mean[i]);

		FILE * fp = fopen("../image_files/inpainting/result.txt", "a");
		fprintf(fp, "[%02d%% - %02d%%] average psnr: %lf\taverage ssim: %lf\taverage time: %lf\n", i*10, (i+1)*10, psnr_mean[i], ssim_mean[i], time_mean[i]);
		fclose(fp);
	}

	for(int i=0;i<6;++i){
		printf("[%02d%% - %02d%%] average psnr: %lf\taverage ssim: %lf\taverage time: %lf\n", i*10, (i+1)*10, psnr_mean[i], ssim_mean[i], time_mean[i]);
	}


	return 0;
}

args?

patchmatch_inpainting/source/main.cpp

Lines 137 to 163 in 53c3610

 int main(int argc, char** argv) { 

 char image_path[100]; 

 char mask_path[100]; 

 char masked_path[100]; 

 char output_path[100]; 

 double psnr_total = 0.0; 

 double ssim_total = 0.0; 

 double time_total = 0.0; 

 for (int j = 0; j<3; ++j) { 

 sprintf(image_path, "./image_files/inpainting/image/image_%05d.png", j); 

 sprintf(mask_path, "./image_files/inpainting/mask/mask_%05d.png", j); 

 sprintf(masked_path, "./image_files/inpainting/masked_image_reverse/masked_image_%05d.png", j); 

 sprintf(output_path, "./image_files/inpainting/output/output_%05d.png", j); 

 process(image_path, mask_path, output_path, &psnr_total, &ssim_total, &time_total); 

 } 

 double psnr_mean = psnr_total / 3; 

 double ssim_mean = ssim_total / 3; 

 double time_mean = time_total / 3; 

 printf("average psnr: %lf\taverage ssim: %lf\taverage time: %lf\n", psnr_mean, ssim_mean, time_mean); 

 return 0; 

 }

Maybe?:

int main(int argc, char** argv)
{
    double psnr_total = 0.0;
    double ssim_total = 0.0;
    double time_total = 0.0;
    if (argc != 4)
    {
        printf("Usage: %s input.png mask.png output.png", argv[0]);
    }
    else
    {
        process(argv[1], argv[2], argv[3], &psnr_total, &ssim_total, &time_total);
        printf("average psnr: %lf\taverage ssim: %lf\taverage time: %lf\n", psnr_total, ssim_total, time_total);
    }

    return 0;
}

	int mask = (int )calloc(int(height), sizeof(int*));
	for (int i = 0; i<width; i++)
	mask[i] = (int *)calloc(int(width), sizeof(int));

	//free memory
	for (int i = 0; i < height; ++i)
	free(mask[i]);
	free(mask);

	ADD_EXECUTABLE(main ./source/main.cpp ./source/inpaint.cpp ./source/maskedimage.cpp ./source/nearestneighborfield.cpp ./source/qualitymesures.cpp)
	TARGET_LINK_LIBRARIES(main opencv_core opencv_imgproc opencv_highgui)

	int main(int argc, char** argv) {

	char image_path[100];
	char mask_path[100];
	char masked_path[100];
	char output_path[100];


	double psnr_total = 0.0;
	double ssim_total = 0.0;
	double time_total = 0.0;
	for (int j = 0; j<3; ++j) {
	sprintf(image_path, "./image_files/inpainting/image/image_%05d.png", j);
	sprintf(mask_path, "./image_files/inpainting/mask/mask_%05d.png", j);
	sprintf(masked_path, "./image_files/inpainting/masked_image_reverse/masked_image_%05d.png", j);
	sprintf(output_path, "./image_files/inpainting/output/output_%05d.png", j);

	process(image_path, mask_path, output_path, &psnr_total, &ssim_total, &time_total);
	}
	double psnr_mean = psnr_total / 3;
	double ssim_mean = ssim_total / 3;
	double time_mean = time_total / 3;
	printf("average psnr: %lf\taverage ssim: %lf\taverage time: %lf\n", psnr_mean, ssim_mean, time_mean);


	return 0;
	}

zqpei / patchmatch_inpainting Goto Github PK

patchmatch_inpainting's Introduction

PatchMatch for inpainting

Introduction

Dependencies

How to use

Demo

References

patchmatch_inpainting's People

Contributors

Stargazers

Watchers

Forkers

patchmatch_inpainting's Issues

Recommend Projects

Recommend Topics

Recommend Org