Hi, I tried this demo with model u provide, but the result is not good. However, the result u showed below are quite good. I wonder it is my personal problem or the trained model is not as same as what you use to generate these samples?

hi,
Thanks very much for your work.Here I have a question: in GatedConv2d, why does sigmoid function add to the conved inp instead of conved mask.
Thank you

@joyoungjoo
Could you please tell some details about how to make sketch like below:

I used HED to make sketch after histogram equalization, but still can't get facial sketch.

Looking forward to your reply.

Traceback (most recent call last):
File "demo.py", line 11, in
from model import Model
File "/home/anbanglee/SC-FEGAN-master/model.py", line 4, in
from ops import *
File "/home/anbanglee/SC-FEGAN-master/ops.py", line 9
"""
^
SyntaxError: invalid syntax
？
what's wrong ?i use python3

QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
Failed to open framebuffer /dev/fb0 (No such file or directory)
linuxfb: Failed to initialize screen
Segmentation fault (core dumped)

DeepfillV2 had a good performance in situation showed in Fig 6. But there is no comparison with it. Is there obvious visualization gap between these two methods?

I am a new player in this filed. I don't have many experiance about this. So sorry for bothing you.

Thank you~

hello, in Style transfer and image restoration,instancenorm is often used ,why network uses local signal normalization？ thanks

ImportError: No module named ui.ui
?

The authors mentioned that the refined stage of the Coarse-Refined network has blurred the image in Chapter 4.1 of the paper, and they gave the reason as

'We discovered that the reason for this is because L1 loss about output of refined network is always smaller than of coarse network'.

I'm not quite understand it. I would really appreciate it if someone could give me a more detailed explanation.

What the code (in ops.py) currently does is:

1. apply the same transposed convolution (TC) to the input x twice,
   results in x1, x2 with x1 == x2. In code, x1 corresponds to deconv and x2 to g.
2. to both outputs of the TC, a different learned bias is added: x1 += b1, x2 += b2
3. leaky relu is applied x1 = lrelu(x1, 0.2)
4. x2 is overwritten: g = tf.nn.sigmoid(deconv)
5. x1 = x1 * x2 which is equivalent to
   lrelu(conv(x) + b1, 0.2) * sigmoid(lrelu(conv(x) + b1), 0.2))

Two things seem kind of strange and raised following questions:

1. Why do we apply the same convolution to create the mask and features? Shouldn't we use two different convolutions as we do with gated convolution? Also, why the different biases then?
2. Shouldn't we apply sigmoid to g instead of deconv?

Thanks in advance!

For convenience i posted the relevant code below:

def gate_deconv(input_, output_shape, k_h=5, k_w=5, d_h=2, d_w=2, stddev=0.02,
       name="deconv", training=True):
    with tf.variable_scope(name):
        # filter : [height, width, output_channels, in_channels]
        w = tf.get_variable('w', [k_h, k_w, output_shape[-1], input_.get_shape()[-1]],
                  initializer=tf.random_normal_initializer(stddev=stddev))

        deconv = tf.nn.conv2d_transpose(input_, w, output_shape=output_shape,
                    strides=[1, d_h, d_w, 1])

        biases = tf.get_variable('biases1', [output_shape[-1]], initializer=tf.constant_initializer(0.0))
        deconv = tf.reshape(tf.nn.bias_add(deconv, biases), deconv.get_shape())
        deconv = tf.nn.leaky_relu(deconv)

        g = tf.nn.conv2d_transpose(input_, w, output_shape=output_shape,
                    strides=[1, d_h, d_w, 1])
        b = tf.get_variable('biases2', [output_shape[-1]], initializer=tf.constant_initializer(0.0))
        g = tf.reshape(tf.nn.bias_add(g, b), deconv.get_shape())
        g = tf.nn.sigmoid(deconv)

        deconv = tf.multiply(g,deconv)

        return deconv, g

does all the images has to be 512 by 512 pixels for it to work? if so what software I should use to reshape a 4k face into 512 by 512? and maybe Train code + update the code so that it accept all resolution instead of stretching it out.

Not sure if this was supposed to be straightfoward but it took me a while to figure out that you need a checkpoint file that references models listed in google drive to run demo.py

I had to create a checkpoint file with the following properties
model_checkpoint_path: "SC-FEGAN.ckpt"
all_model_checkpoint_paths: "SC-FEGAN.ckpt"

Running this on MacOS version 10.14.3

when i run i am getting error ModuleNotFoundError: No module named 'utils.config'

Appreciate if anyone would let me know how can we extracted only masked part of image from the edited image in the UI ?

Could you help me how to train the model code?

I have changed the yml file to CKPT_DIR: '/home/mustafa/ckpt' inside this dir there is model file SC-FEGAN.ckpt.data-00000-of-00001 and index file SC-FEGAN.ckpt.index
error is

Model loaded from /home/mustafa/ckpt....start
Traceback (most recent call last):
  File "demo.py", line 234, in <module>
    ex = Ex(model, config)
  File "demo.py", line 22, in __init__
    self.model.load_demo_graph(config)
  File "/media/mustafa/ubuntu_backup/Projects/gan/SC-FEGAN/model.py", line 111, in load_demo_graph
    var_value = tf.contrib.framework.load_variable(ckpt_path, from_name)
  File "/home/mustafa/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/contrib/framework/python/framework/checkpoint_utils.py", line 79, in load_variable
    reader = load_checkpoint(checkpoint_dir)
  File "/home/mustafa/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/contrib/framework/python/framework/checkpoint_utils.py", line 62, in load_checkpoint
    "given directory %s" % filepattern)
ValueError: Couldn't find 'checkpoint' file or checkpoints in given directory /home/mustafa/ckpt

As the title said. Thank you.

I'm using 0.12.0 of tensorflow and it gives the above error,i tried removing the axis argument then it says no 'values' parameter found

File: model.py
method : build_demo_graph

Can't start example:

root@ubuntu:~/experiment/SC-FEGAN-master# python3 demo.py
Traceback (most recent call last):
  File "demo.py", line 11, in <module>
    from model import Model
  File "/root/experiment/SC-FEGAN-master/model.py", line 4, in <module>
    from ops import *
  File "/root/experiment/SC-FEGAN-master/ops.py", line 9
    """
      ^
SyntaxError: invalid syntax

I am running via CPU:

root@ubuntu:~/experiment/SC-FEGAN-master# cat demo.yaml
INPUT_SIZE: 512
BATCH_SIZE: 1

#GPU_NUM: 1

# directories
CKPT_DIR: './ckpt/SC-FEGAN.ckpt'

Hi! I want to test your neural network, but I'm getting a 48-line error. Installed all the dependencies, as in your manual. Python 3.6.7. PyQT 5.

File ui/ui.py, error:
self.pushButton.clicked.connect(Form.open)

If I tried to read the model code and convert it to from 2D to 3D, will it work?
I mean is it doable?

Hi，I have already run this program and the output is quite good！ Now I want to understand the code and make some changes by myself. But I‘m a student and my python is not that good. I wonder if there is more explanations for your code？ Thank you very much！！

Hi, thanks for sharing your code

I get this error when try to run
!python demo.py

i use google colab

qt.qpa.xcb: could not connect to display
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, webgl, xcb.

Is the trained model file authored and released under certain open source license?

I'm trying to give the model a custom mask but the doesn't give the expected results even tho the model give a good result with provided mask (using the mask button).

    def make_mask(self, pts, mslice,i):
        if len(pts)>0:
            mask = np.zeros((512,512,3))
            for pt in pts:
                cv2.line(mask,pt['prev'],pt['curr'],(255,255,255),12)
            mask = np.asarray(mask[:,:,0]/255,dtype=np.uint8)
            mask = np.expand_dims(mask,axis=2)
            mask = np.expand_dims(mask,axis=0)
    else:
        # loading the mask from matlab file {1=hole} 
        array2D= scipy.io.loadmat('data.mat')['Holes_Vol']

        # resize the 2d matrix
        mask = cv2.resize(mslice, (512, 512), interpolation=cv2.INTER_CUBIC)

        #  convert the matrix to 3 chanels (3D) (RGB)
        mask = np.repeat(mask[:, :, np.newaxis]*255, 3, axis=2)

        # The below line are taken from the 3 line above else
        mask = np.asarray(mask[:,:,0]/255,dtype=np.uint8)
        mask = np.expand_dims(mask,axis=2)
        mask = np.expand_dims(mask,axis=0)
        
    return mask 

The code before the modification:

any suggestions?

In your Google drive, there is only two files which are SC-FEGAN.ckpt.data-00000-of-00001 and SC-FEGAN.ckpt.index.

How can I get SC-FEGAN.ckpt.
I can't run demo.py because that file doesn't in your Google drive.

qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, webgl, xcb.

?

It would be helpful if you clarify the version of your dependencies.

First, my apologies: I am extremely new to working with Python. Let me give you a quick overview of what I did:

0. Run Windows 10 (64x) with Python 3.7.2 (64bit)
1. Created a new virtual environment
2. Used pip to install Tensorflow (1.13.0rc2), numpy (1.16.1), PyQT5 (5.12), PyAML (3.13) and opencv-python (4.0.0.21) in the virtual environment
3. Downloaded code and model
4. Created folder "ckpt" within the "SC-FEGAN-master" folder and placed the model there

Now, if I run:
(venv) C:\WINDOWS\system32>python C:\Users\Jan\Downloads\SC-FEGAN-master\demo.py

I get:

File "C:\Users\Jan\Downloads\SC-FEGAN-master\demo.py", line 229, in
config = Config('demo.yaml')
File "C:\Users\Jan\Downloads\SC-FEGAN-master\utils\config.py", line 10, in init
assert os.path.exists(filename), "ERROR: Config File doesn't exist."
AssertionError: ERROR: Config File doesn't exist.`>

If I run:
(venv) C:\WINDOWS\system32>python C:\Users\Jan\Downloads\SC-FEGAN-master\ui\ui.py

I get:

Traceback (most recent call last):
File "C:\Users\Jan\Downloads\SC-FEGAN-master\ui\ui.py", line 84, in
ui.setupUi(Form)
File "C:\Users\Jan\Downloads\SC-FEGAN-master\ui\ui.py", line 47, in setupUi
self.pushButton.clicked.connect(Form.open)
AttributeError: 'QWidget' object has no attribute 'open'

Any idea what the problem could be?

Training code is necessary to understand what loss functions were used, what was the model architecture etc. It would be a nice idea to add training code too.

Traceback (most recent call last):
File "C:\Users\Noah\AppData\Local\Programs\Python\Python37\lib\site-packages\t
ensorflow\python\pywrap_tensorflow.py", line 58, in
from tensorflow.python.pywrap_tensorflow_internal import *
File "C:\Users\Noah\AppData\Local\Programs\Python\Python37\lib\site-packages\t
ensorflow\python\pywrap_tensorflow_internal.py", line 28, in
_pywrap_tensorflow_internal = swig_import_helper()
File "C:\Users\Noah\AppData\Local\Programs\Python\Python37\lib\site-packages\t
ensorflow\python\pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, descript
ion)
File "C:\Users\Noah\AppData\Local\Programs\Python\Python37\lib\imp.py", line 2
42, in load_module
return load_dynamic(name, filename, file)
File "C:\Users\Noah\AppData\Local\Programs\Python\Python37\lib\imp.py", line 3
42, in load_dynamic
return _load(spec)
ImportError: DLL load failed: Det går inte att hitta den angivna modulen.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "demo.py", line 11, in
from model import Model
File "C:\Users\Noah\Documents\SC-FEGAN-master\model.py", line 1, in
import tensorflow as tf
File "C:\Users\Noah\AppData\Local\Programs\Python\Python37\lib\site-packages\t
ensorflow_init_.py", line 24, in
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-im
port
File "C:\Users\Noah\AppData\Local\Programs\Python\Python37\lib\site-packages\t
ensorflow\python_init_.py", line 49, in
from tensorflow.python import pywrap_tensorflow
File "C:\Users\Noah\AppData\Local\Programs\Python\Python37\lib\site-packages\t
ensorflow\python\pywrap_tensorflow.py", line 74, in
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "C:\Users\Noah\AppData\Local\Programs\Python\Python37\lib\site-packages\t
ensorflow\python\pywrap_tensorflow.py", line 58, in
from tensorflow.python.pywrap_tensorflow_internal import *
File "C:\Users\Noah\AppData\Local\Programs\Python\Python37\lib\site-packages\t
ensorflow\python\pywrap_tensorflow_internal.py", line 28, in
_pywrap_tensorflow_internal = swig_import_helper()
File "C:\Users\Noah\AppData\Local\Programs\Python\Python37\lib\site-packages\t
ensorflow\python\pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, descript
ion)
File "C:\Users\Noah\AppData\Local\Programs\Python\Python37\lib\imp.py", line 2
42, in load_module
return load_dynamic(name, filename, file)
File "C:\Users\Noah\AppData\Local\Programs\Python\Python37\lib\imp.py", line 3
42, in load_dynamic
return _load(spec)

What's the problem here? All the dependencies are installed. I use Python 3.7.3

Hi. The Code and its working process is good.
I am getting good results for images of size 512.
If i considered images of size 1024,first i resized them to 512 and performing the tatto removal in an image. The result is of blurry if i resize them to the size 1024. What is the best way to get good quality of the image for higher resultion..

Thanks in advance.
SandhyaLaxmi

flake8 testing of https://github.com/JoYoungjoo/SC-FEGAN on Python 3.7.1

$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

./ops.py:12:20: F821 undefined name 'x'
        x = tf.pad(x, [[0,0], [p, p], [p, p], [0,0]], mode=padding)
                   ^
./utils/config.py:23:21: F821 undefined name 'DictAsMember'
            value = DictAsMember(value)
                    ^
2     F821 undefined name 'x'
2

E901,E999,F821,F822,F823 are the "showstopper" flake8 issues that can halt the runtime with a SyntaxError, NameError, etc. These 5 are different from most other flake8 issues which are merely "style violations" -- useful for readability but they do not effect runtime safety.

• F821: undefined name name
• F822: undefined name name in __all__
• F823: local variable name referenced before assignment
• E901: SyntaxError or IndentationError
• E999: SyntaxError -- failed to compile a file into an Abstract Syntax Tree

Hi,
I downloaded the model and located in the correct position as you mentioned. However, I got this error when I run demo.py file.
ValueError: Couldn't find 'checkpoint' file or checkpoints in given directory ./ckpt

Where should I download it?
Please help me how to resolve this issue.
Thank you.

Could you please send me a complete code(including training code)?

Hello is the model is same in the paper result? i am thinking the model you provided not actual model. Is it same model or not can you answer it ?

Hi,

Thanks for your great work.
In your code,
https://github.com/run-youngjoo/SC-FEGAN/blob/master/ops.py#L50
this line states g = tf.nn.sigmoid(deconv)
However, IMHO the sigmoid should be applied to g itself to construct the gate
It should be g = tf.nn.sigmoid(g) right?
Did I misunderstand anything?

Thanks for your great work again!

help me open this neural network

IMMEDIATELY CREATE A WEBSITE WITH THIS NEURAL NETWORK AND DOWNLOAD IT FOR FREE!!!!!!!!!!!!!!!!!!!

how do i ?

run
$ python3 demo.py

got

After commenting out GPU_NUM in demo.yaml, this happens:

Traceback (most recent call last):
File "demo.py", line 230, in
os.environ["CUDA_VISIBLE_DEVICES"] = str(config.GPU_NUM)
File "/Users/francesco.cardinale/forge/SC-FEGAN/utils/config.py", line 21, in getattr
value = self._cfg_dict[name]
KeyError: 'GPU_NUM'

Instead of commenting the line as in the readme, setting it to 0 works just fine

Hi,
When I visited your google driver as you mentioned, there were 2 ckpt files. The ckpt.data file was not worked because of the wrong file. The ckpt.index file is loaded successfully. When I run demo.py file, I got this issue.
"tensorflow.python.framework.errors_impl.NotFoundError: generator/gconv1_ds/kernel not found in checkpoint file"

It seems that the model doesn't have such named layers, or tensorflow can't find such layers in the model.
Please help me how to resolve this issue.
Thank you.

When will the training code be released? Thanks！

The license file is missing.

This code don't crush program if fileName contains Cyrillic symbols

stream = open(fileName, "rb")
bytes = bytearray(stream.read())
numpyarray = np.asarray(bytes, dtype=np.uint8)
stream.close()
image = QPixmap(fileName)
#mat_img = cv2.imread(fileName)
mat_img = cv2.imdecode(numpyarray, cv2.IMREAD_UNCHANGED)
if image.isNull():
QMessageBox.information(self, "Image Viewer",
"Cannot load %s." % fileName)
return