Hello

I want to add loss function of Sigma RMSE
I'm using this for fitting series of weather data(Temp, WindSpeed, etc)
Used it by custom Loss Function, but It can be useful to other people

Actually tried to PR to Keras(keras-team/keras#6996)
fchollet sent me here

After I read CONTRIBUTING.md, I'm writing this issue for PR

Thanks.

I'm trying to fix the examples to fit keras-2, however I got the following error when I try to run the cifar10_densenet.py:

Traceback (most recent call last):
File "examples/cifar10_densenet.py", line 38, in
input_shape=img_dim, weights=None)
File "/home/ubuntu/Github/keras-contrib/keras_contrib/applications/densenet.py", line 117, in DenseNet
dropout_rate, weight_decay)
File "/home/ubuntu/Github/keras-contrib/keras_contrib/applications/densenet.py", line 487, in __create_dense_net
dropout_rate=dropout_rate, weight_decay=weight_decay)
File "/home/ubuntu/Github/keras-contrib/keras_contrib/applications/densenet.py", line 378, in __dense_block
x = merge(x_list, mode='concat', concat_axis=concat_axis)
File "/usr/local/lib/python2.7/dist-packages/keras/legacy/layers.py", line 456, in merge
name=name)
File "/usr/local/lib/python2.7/dist-packages/keras/legacy/layers.py", line 117, in init
self(input_tensors, mask=input_masks)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 554, in call
output = self.call(inputs, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/keras/legacy/layers.py", line 210, in call
return K.concatenate(inputs, axis=self.concat_axis)
File "/usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py", line 1525, in concatenate
return tf.concat([to_dense(x) for x in tensors], axis)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/array_ops.py", line 1000, in concat
dtype=dtypes.int32).get_shape(
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 669, in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 176, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 165, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py", line 367, in make_tensor_proto
_AssertCompatible(values, dtype)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py", line 302, in _AssertCompatible
(dtype.name, repr(mismatch), type(mismatch).name))
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

Many guys have got the similar error when they call tf.concat(axis, [x1,x2,x3]) instead of tf.concat([x1,x2,x3], axis). However this is not the case here. Any one has any solution?

Hi,

I am trying to run this segment of code from the Deconvolution3D example:

model = Sequential()
model.add(Deconvolution3D(3, 3, 3, 3, output_shape=(None, 3, 14, 14, 14), padding='valid', input_shape=(3, 12, 12, 12)))

but am experiencing this error "TypeError: init() got multiple values for keyword argument 'output_shape'". Is this a bug?

Thank you!

Hi,
I want to use difference of structural similarity as the loss function. This is my code
from keras.models import Model
from keras.layers import Input, Conv2D
from keras.optimizers import Adam
from keras_contrib.losses import DSSIMObjective
input_shape = (1,33,33)
n1 = 64
n2 = 32
n3 = 1
f1 = 9
f2 = 1
f3 = 5
x = Input(shape = input_shape)
c1 = Conv2D(n1, (f1, f1), padding="same", kernel_initializer="he_normal", activation="relu")(x)
c2 = Conv2D(n2, (f2, f2), padding="same", kernel_initializer="he_normal", activation="relu")(c1)
c3 = Conv2D(n3, (f3, f3), padding="same", kernel_initializer="he_normal")(c2)
model = Model(inputs=x, outputs=c3)
adam = Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-8)
model.compile(loss=DSSIMObjective, metrics=['mse'], optimizer=adam)

When I tried to run the program, I got the following error at the line model.compile(loss=DSSIMObjective, metrics=['mse'], optimizer=adam)
File "/home/ldm/Desktop/SRCNN/SRCNN-hdf5/SSIM", line 23, in <module>
model.compile(loss=DSSIMObjective, metrics=['mse'], optimizer=adam)

File "/home/ldm/anaconda2/envs/SRCNN/lib/python2.7/site-packages/keras/engine/training.py", line 899, in compile
sample_weight, mask)

File "/home/ldm/anaconda2/envs/SRCNN/lib/python2.7/site-packages/keras/engine/training.py", line 441, in weighted
ndim = K.ndim(score_array)

File "/home/ldm/anaconda2/envs/SRCNN/lib/python2.7/site-packages/keras/backend/theano_backend.py", line 216, in ndim
return x.ndim

AttributeError: DSSIMObjective instance has no attribute 'ndim'

To be clear, I am using Keras 2 and theano 0.9 as backend. I suspect I didn't use DSSIMObjective in the right way, how should I use it? @farizrahman4u
Thanks in advance.

According to this discussion, this line must be
expected_output = K.eval(KCTF.depth_to_space(input_tensor, scale=scale_factor))

The fact that the theano implementation returns the same result as the tensorflow wrapper shows that is is correct.

Now it's not using TensorFlow's depth_to_space if backend is Theano.

And the test name (test_subias_pixel_upscaling) also looks wrong.

See this post.

Hi,

First of all, you are all life savers -- I needed Deconvolution3D to try and make a video-GAN!

I'm having a bit of trouble understanding the doc string examples (using tensorflow backend but applies to both). Specifically, when trying to figure out the appropriate kernel/strides it is suggested to "use a dummy input and see the actual output shape." But if the specified output shape is wrong, the .predict won't run. So I can't use this technique to debug my layer.

This is mianly me not understanding deconvolutional math, but any suggestions on how to use the function to gain understanding would be appreciated!

For reference, I'm referring to

 # apply a 3x3x3 transposed convolution

    #  with stride 2x2x2 and 3 output filters on a 12x12x12 image:
    model = Sequential()
    model.add(Deconvolution3D(3, 3, 3, 3, output_shape=(None, 25, 25, 25, 3),
                              strides=(2, 2, 2),
                              padding='valid',
                              input_shape=(12, 12, 12, 3)))
    model.summary()
    # we can predict with the model and print the shape of the array.
    dummy_input = np.ones((32, 12, 12, 12, 3))
    preds = model.predict(dummy_input)
    print(preds.shape)  # (None, 25, 25, 25, 3)

Hi,
I don't know if the STNs could be considered relevant enough to include them in this repo; however, I currently have an implementation of the STN layer in this gist. My original repository with a test case on the cluttered MNIST dataset can be found here. It is currently working with the tf backend (one can run the mnist_cluttered_example.ipynb). However, I recently moved everything into K tensors (in the K_tensors branch) but the meshgrid from theano is failing. Looking at the CONTRIBUTING.md it seems like you only accept PRs that work with both back-ends, is this the case? Also, if someone is interested in having a look at the failing K_meshgrid function in the theano backend and helping me fix it, it would be awesome.

Thank you.

I have updated the image preprocessing to allow for FCN's, so I imagine this would fit better in Keras. However, I've also updated it to accept 'BGR' image format (Does the transform at the same time it resizes) and load grayscale to I/16 mode instead of L, however this could become an arg to be passed in also.

If this isn't accepted into the main library and is deemed to fit best here, then what would be the best way to allow it to work nicely with Keras?

Since it would add new lines, so keeping up to date with the main library would be a problem (i.e. merging new additions into the contrib), but overriding is hard (for users who don't want to edit keras by hand) unless it is built in.

I imagine we add another arg to the keras version which stores a list of custom object names, then loads from that list instead of the builtin functions, allowing users to override keras functions with contrib ones.

So, how should we handle situations like this?
All of the above is related to image preprocessing only.

I get an error when trying to import keras-contrib

import keras_contrib

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/abhai/keras-contrib/keras_contrib/__init__.py", line 2, in <module>
    from . import backend
  File "/home/abhai/keras-contrib/keras_contrib/backend/__init__.py", line 14, in <module>
    from .tensorflow_backend import *
  File "/home/abhai/keras-contrib/keras_contrib/backend/tensorflow_backend.py", line 14, in <module>
    from keras.backend.common import floatx, _EPSILON, image_dim_ordering, reset_uids
ImportError: cannot import name image_dim_ordering

Just a small inconsistency - the backend for tensorflow input variable is ssizes while for theano it's strides. Not sure if it matters if they're the same name but seems like it's easier in the long run if they are.

Also curious why cols are before rows?

Thank you for adding this function.

I feel like we should do a PR push (i.e., Twitter) to get people contributing and interested here. Thoughts?

Keras-2 is nearing completion so we should start porting everything to work with the new Keras-2 APIs. I will create a branch for this purpose. Please state what you're working on in this thread so people don't duplicate effort.

I think we'd better demonstrate the purpose of this new keras-contrib repository in the README.md. From my point of view, the purpose is to:

1. provide an official keras extension repository for the keras community, which contains newest layers, activations, loss functions, optimizers, and so on.
2. keep the original keras repository clean, only the most valuable functions will be finally merged into the original keras repository.

Any suggestions?

Hi all,
I am facing issues while importing Cropping3D class from keras_contrib on centos 6.5 with python 2.7.13 .

I had installed KERAS-CONTRIB as follows -
export PYTHONPATH="$HOME/SOFTWARES/KERAS-CONTRIB/master-29042017/gnu/lib/python2.7/site-packages"
python setup.py install --prefix="$HOME/SOFTWARES/KERAS-CONTRIB/master-29042017/gnu"|tee keras_logs

The import works fine from the keras-contrib source folder-
[user@gpu1 keras-contrib]$ python
Python 2.7.13 (default, Apr 24 2017, 13:28:09)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import tensorflow as tf
import keras.backend as K
Using TensorFlow backend.
from keras.models import Model
from keras.layers import Input
from keras.layers.advanced_activations import LeakyReLU
from keras.layers.normalization import BatchNormalization
from keras.layers.core import Activation,Dense,Reshape,Flatten
from keras.layers.convolutional import Cropping3D
from keras_contrib.layers.convolutional import Deconvolution3D
from keras.layers.convolutional import Convolution3D,Convolution2D

But from any other folder , i get error-

[user@gpu1 keras-contrib]$ cd ..
[user@gpu1 SRC]$ python
Python 2.7.13 (default, Apr 24 2017, 13:28:09)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-11)] on linux2
[user@gpu1 ~]$ python
Python 2.7.13 (default, Apr 24 2017, 13:28:09)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import tensorflow as tf
import keras.backend as K
from keras.models import Model
from keras.layers import Input
from keras.layers.advanced_activations import LeakyReLU
from keras.layers.normalization import BatchNormalization
from keras.layers.core import Activation,Dense,Reshape,Flatten
from keras.layers.convolutional import Cropping3D
from keras_contrib.layers.convolutional import Deconvolution3D
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named keras_contrib.layers.convolutional
from keras.layers.convolutional import Convolution3D,Convolution2D

It seems that Deconvolution3D is not the part of egg file.
I don't want to run my simulations from keras-contrib folder. How could i fix this issue?

the two modes in crf are join mode and marginal mode,but i am not clear about the marginal mode,and i use this mode to train the data, the result is almost same to the softmax, anyone can explain it?

I'm getting a lot of weird errors like pip install Theano failing tonight. Not sure if you guys also get all those emails about travis troubleshooting some problems with sudo tonight. Everything seems to work fine if you hit restart job on whatever subjob fails. I guess just a heads up and I'll close this if everything goes away eventually.

Hi,
I would like to propose another type of image mask generator.
I'm aware of this and of 'Example of transforming images and masks together', so I tried to mix both functions (transform image and mask, provide a pipeline of transformations).
Glad to hear any feedback on this proposition and the code.

Whenever I try to import the CRF class, I get the following error:

Traceback (most recent call last):
  File "keras_test2.py", line 12, in <module>
    from keras_contrib.layers import CRF
  File "build/bdist.macosx-10.12-x86_64/egg/keras_contrib/__init__.py", line 13, in <module>
  File "build/bdist.macosx-10.12-x86_64/egg/keras_contrib/optimizers/__init__.py", line 1, in <module>
  File "build/bdist.macosx-10.12-x86_64/egg/keras_contrib/optimizers/ftml.py", line 8, in <module>
  File "build/bdist.macosx-10.12-x86_64/egg/keras_contrib/optimizers/ftml.py", line 34, in FTML
AttributeError: 'module' object has no attribute 'legacy_get_updates_support'

In the code of dssim, the covariance is not calculated properly.

See : keras-team/keras#4292 (comment)

(I'll fix it this week, just to leave a trace.)

Looking at the code of BatchRenormalization , I think there is a serious issue there. This section, for example, inside the call() function:

# update r_max and d_max
t_val = K.get_value(self.t)
r_val = self.r_max_value / (1 + (self.r_max_value - 1) * np.exp(-t_val))
d_val = self.d_max_value / (1 + ((self.d_max_value / 1e-3) - 1) * np.exp(-(2 * t_val)))
t_val += float(self.t_delta)

The main issue is that the code uses Python/Numpy operations, rather than Keras/TensorFlow operations. Note that the layer call() function is executed only once when the model is created. It does not get called repeatedly for every batch. The call() function should create a TF/Theano computation graph rather than do the computation directly in Python/Numpy.

What's happening there is that it calls K.get_value(), which internally creates a TF session and runs it to get the float value of self.t. Then, r_val and d_val and calculated in Python code and used in subsequent update ops. Since r_val and d_val are computed only once when the model is created, they'll have values of 1 and 0 respectively and will remain so. In other words, if I'm reading this correctly, this layer is operating as a regular batch normalization layer, not as a batch re-normalization layer.

I didn't run the code to confirm so I could be missing something. If so, I'd appreciate being corrected.

when I use the keras-contrib, meet the "module has no attribution interfaces.legacy_get_updates_support " ,why?

I set up a BILSTM-CRF model for sequence labelling, very similar to this example (https://github.com/farizrahman4u/keras-contrib/blob/master/examples/conll2000_chunking_crf.py).

from keras.layers  import Dense, Masking
from keras.layers import Dropout
from keras.layers import Bidirectional
from keras.layers import LSTM
from keras.models import Sequential
from keras.optimizers import Adam
from keras_contrib.layers import CRF

self._model = Sequential(name='core_sequential')
self._model.add(Masking(mask_value=0., input_shape=(None, input_size)))

self._model.add(Dropout(dropout_rate,name='dropout_layer_1'))
self._model.add(Bidirectional(LSTM(units=hidden_unit_size,
                                     return_sequences=True,
                                     activation="tanh", 
                                     name="lstm_layer"),
                                     name='birnn_layer'))
crf = CRF(2, sparse_target=True)
self._model.add(crf)
self._model.compile(optimizer=Adam(lr=lr), loss=crf.loss_function, metrics=[crf.accuracy])

self._model.fit(x_train, y_train, validation_data=(x_val, y_val), epochs=epochs, batch_size=batch_size,
                            verbose=verbose, shuffle=shuffle)

self._model.save(filename)

The model was trained successfully and I also successfully called the predict_classes() function, as soon as I trained it and I had the object after training...

Then I tried to load it back, like it is mentioned in the wiki "A Common "Gotcha", importing the additional layer, before calling load_model() function :

from keras.models import load_model
from keras_contrib.layers import CRF

self._model = load_model(filename)

The error message is:

[...]
  File "/Users/kiddo/anaconda/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 140, in deserialize_keras_object
    list(custom_objects.items())))
  File "/Users/kiddo/anaconda/lib/python3.6/site-packages/keras/models.py", line 1202, in from_config
    layer = layer_module.deserialize(conf, custom_objects=custom_objects)
  File "/Users/kiddo/anaconda/lib/python3.6/site-packages/keras/layers/__init__.py", line 54, in deserialize
    printable_module_name='layer')
  File "/Users/kiddo/anaconda/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 133, in deserialize_keras_object
    ': ' + class_name)
**ValueError: Unknown layer: CRF**

So, I changed to:

from keras_contrib.layers import CRF

self._model = load_model(filename, custom_objects={'CRF':CRF})

The new error says that the loss is missing also:

  File "/Users/kiddo/anaconda/lib/python3.6/site-packages/keras/engine/training.py", line 751, in compile
    loss_function = losses.get(loss)
  File "/Users/kiddo/anaconda/lib/python3.6/site-packages/keras/losses.py", line 96, in get
    return deserialize(identifier)
  File "/Users/kiddo/anaconda/lib/python3.6/site-packages/keras/losses.py", line 88, in deserialize
    printable_module_name='loss function')
  File "/Users/kiddo/anaconda/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 157, in deserialize_keras_object
    ':' + function_name)
**ValueError: ('Unknown loss function', ':loss')**

I tried extending the custom_objects dict with ':loss':CRF.CRF.loss_function', but I still have the same error...

Any idea about that?

With Theano (git version 8f3f254 due to compiler issues) the CRF layer is broken. Unfortunately I am unable to test this with the current stable Theano. Theano git runs other recurrent models without problems.

% python conll2000_chunking_crf.py
Using Theano backend.
[...]
Epoch 1/10
8936/8936 [==============================] - 15s - loss: nan - viterbi_acc: 0.0098 - val_loss: nan - val_viterbi_acc: 0.0093
Epoch 2/10
8936/8936 [==============================] - 15s - loss: nan - viterbi_acc: 0.0098 - val_loss: nan - val_viterbi_acc: 0.0093
[...]

With the TensorFlow backend, the example runs perfectly on my machine.

This is an issue for layers/models that help with upsampling, deconvolution, super resolution, and other cases where your output layer is larger than your input layer. One specific possibility:

• Atrous transpose convolutions with tf python test

Others can add additional options.

I'm just starting to get back into Keras after some time away, and there is some old code I wanted to contribute:

keras-team/keras#1625

import time
from keras.callbacks import Callback
class TimedStopping(Callback):
    '''Stop training when enough time has passed.
    # Arguments
        seconds: maximum time before stopping.
        verbose: verbosity mode.
    '''
    def __init__(self, seconds=None, verbose=0):
        super(Callback, self).__init__()

        self.start_time = 0
        self.seconds = seconds
        self.verbose = verbose

    def on_train_begin(self, logs={}):
        self.start_time = time.time()

    def on_epoch_end(self, epoch, logs={}):
        if time.time() - self.start_time > self.seconds:
            self.model.stop_training = True
            if self.verbose:
                print('Stopping after %s seconds.' % self.seconds)

Is something like this helpful or is there already code that does this now?

Contributions w.r.t. the following layers and examples are welcomed:

• Layers
  • Layer Normalization
  • PELU [DONE!]
  • Batch Renormalization
• Examples
  • DenseNets [DONE!]
  • Residual of Residual Networks [DONE!]
  • Wide ResNets[DONE!]
  • ResNet in ResNet

I'm trying to use the CRF with the Functional (Model) API, but I'm not sure how to proceed. I keep getting No data provided for "crf_1". Need data for each key in: ['crf_1']

The general idea is:

    label_td = TimeDistributed(Dense(params["n_labels"], activation="tanh"))(drop)

    crf = CRF(params["n_labels"])
    labels_out = crf(label_td)

    model = Model(inputs=[data_in], outputs=[labels_out])
    model.compile(loss=crf.loss_function,
                  optimizer="adam",
                  metrics=[crf.accuracy])

but that doesn't work. Unfortunately I'm unable to rewrite to a sequential model, because the Masking layer complains (I'm feeding it embeddings directly, instead of using an Embedding layer).

Hi,
I am using DSSIMObjective as loss function. And my code is:
n1 = 64
n2 = 32
n3 = 1
f1 = 9
f2 = 1
f3 = 5
model = Sequential()
model.add(Conv2D(n1, (f1, f1), padding='same', input_shape=input_shape, activation='relu'))
model.add(Conv2D(n2, (f2, f2), padding='same', activation='relu'))
model.add(Conv2D(n3, (f3, f3), padding='same'))
adam = Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-8)
model.compile(loss = DSSIMObjective(), metrics=['mse'], optimizer=adam)
model.fit(X, y, batch_size=128, epochs=n_epoch, shuffle='batch')

I got the following error at the line of model.fit(X, y, batch_size=128, epochs=n_epoch, shuffle='batch')

ValueError: cannot reshape array of size 139392 into shape (128,15488,3,3) Apply node that caused the error: Reshape{4}(Images2Neibs{valid}.0, Join.0) Toposort index: 113 Inputs types: [TensorType(float32, matrix), TensorType(int64, vector)] Inputs shapes: [(15488, 9), (4,)] Inputs strides: [(36, 4), (8,)] Inputs values: ['not shown', array([ 128, 15488, 3, 3])] Outputs clients: [[InplaceDimShuffle{0,x,1,2,3}(Reshape{4}.0)]]
Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
File "/home/ldm/anaconda2/envs/SRCNN/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 94, in execfile
builtins.execfile(filename, *where)
File "/home/ldm/Desktop/SRCNN/SRCNN-hdf5/train.py", line 74, in <module>
model.compile(loss = DSSIMObjective(), metrics=['mse'], optimizer=adam)
File "/home/ldm/anaconda2/envs/SRCNN/lib/python2.7/site-packages/keras/models.py", line 766, in compile
**kwargs)
File "/home/ldm/anaconda2/envs/SRCNN/lib/python2.7/site-packages/keras/engine/training.py", line 899, in compile
sample_weight, mask)
File "/home/ldm/anaconda2/envs/SRCNN/lib/python2.7/site-packages/keras/engine/training.py", line 430, in weighted
score_array = fn(y_true, y_pred)
File "build/bdist.linux-x86_64/egg/keras_contrib/losses.py", line 39, in __call__
patches_pred = KC.extract_image_patches(y_pred, kernel, kernel, 'valid', self.dim_ordering)
File "build/bdist.linux-x86_64/egg/keras_contrib/backend/theano_backend.py", line 167, in extract_image_patches
patches = KTH.reshape(patches, (batch, num_channels, KTH.shape(patches)[0] // num_channels, patch_size, patch_size))
File "/home/ldm/anaconda2/envs/SRCNN/lib/python2.7/site-packages/keras/backend/theano_backend.py", line 713, in reshape
y = T.reshape(x, shape)
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.

BTW, if I use mse as loss function, the program can run properly with:
model.compile(loss = losses.mean_squared_error, metrics=['mse'], optimizer=adam)
Both my training (X) and validation (y) data have a shape of (21884,1,33,33).
@farizrahman4u

There are a number of good model implementations in third party repositories, and I think keras-contrib may be the most sensible place for them to go. However, each will need an example and a way to train, or a single script that works for all models. For that reason I propose a tf-slim style setup for vision based classification and semantic segmentation that can eliminate some of that overhead for users and provide more general utility than the keras/applications folder.

The following may be aspects of a model folder:

• have a script equivalent to train_image_classifier.py explained in the tf-slim readme.
• train_image_classifier might support both image labeling and semantic segmentation
• model test folder, perhaps in a folder models placed in https://github.com/farizrahman4u/keras-contrib/tree/master/tests/keras_contrib

One such example model implementation is keras-resnet. The author is interested in submitting a pull request to keras-contrib.

I know that deep-learning-models exists but there are benefits keras-contrib could eventually provide:

• provide way to train the models from scratch
• community administrators (doesn't rely on availability of fcollet)
• code and documentation to convert trained models from other deep learning apis like caffe

Is there interest in this type of contribution?
Would the ideal place to put such a thing be in kerascontrib/keras-contrib/models?

With Python 3.5.2 and nltk 3.2.4

(train_x, _, train_y), (test_x, _, test_y), (vocab, _, class_labels) = conll2000.load_data()
Downloading data from https://raw.githubusercontent.com/nltk/nltk_data/gh-pages/packages/corpora/conll2000.zip
606208/756607 [=======================>......] - ETA: 
/home/ymeng/.keras/datasets/conll2000.zip
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-137-4c42ae7b7d79> in <module>()
----> 1 (train_x, _, train_y), (test_x, _, test_y), (vocab, _, class_labels) = conll2000.load_data()

/usr/local/lib/python3.5/dist-packages/keras_contrib/datasets/conll2000.py in load_data(path, min_freq)
     12     print(path)
     13     archive = ZipFile(path, 'r')
---> 14     train = _parse_data(archive.open('conll2000/train.txt'))
     15     test = _parse_data(archive.open('conll2000/test.txt'))
     16     archive.close()

/usr/local/lib/python3.5/dist-packages/keras_contrib/datasets/conll2000.py in _parse_data(fh)
     28 def _parse_data(fh):
     29     string = fh.read()
---> 30     data = [[row.split() for row in sample.split('\n')] for sample in string.strip().split('\n\n')]
     31     fh.close()
     32     return data

TypeError: a bytes-like object is required, not 'str'

Hi,
when run "pascal_voc.py oascal_voc_setup" occur some error ;
**Exception originated from within Sacred.
Traceback (most recent calls):
File "C:\Program Files\python3.5\python-3.5.3.amd64\lib\sacred\experiment.py",
line 242, in run_commandline
return self.run(cmd_name, config_updates, named_configs, {}, args)
File "C:\Program Files\python3.5\python-3.5.3.amd64\lib\sacred\experiment.py",
line 187, in run
run()
File "C:\Program Files\python3.5\python-3.5.3.amd64\lib\sacred\run.py", line 2
23, in call
self.result = self.main_function(*args)
File "C:\Program Files\python3.5\python-3.5.3.amd64\lib\sacred\config\captured
_function.py", line 47, in captured_function
result = wrapped(*args, **kwargs)
File "pascal_voc.py", line 732, in pascal_voc_setup
pascal_berkeley_root)
File "C:\Program Files\python3.5\python-3.5.3.amd64\lib\sacred\config\captured
_function.py", line 47, in captured_function
result = wrapped(*args, kwargs)
File "pascal_voc.py", line 683, in convert_pascal_berkeley_augmented_mat_annot
ations_to_png
pascal_berkeley_root)
File "C:\Program Files\python3.5\python-3.5.3.amd64\lib\sacred\config\captured
_function.py", line 44, in captured_function
wrapped.logger.debug("Started")
AttributeError: 'NoneType' object has no attribute 'debug'

i update sacred and keras versin is 2

I'm training and testing DenseNetFCN on Pascal VOC2012

Could I get advice on next steps to take to debug and improve the results?

To do so I'm using the train.py training script in my fork of Keras-FCN in conjunction with DenseNetFCN in keras-contrib with #46 applied, which for DenseNetFCN really mostly changes the formatting for pep8, though the regular densenet is more heavily modified.

I use Keras-FCN because we don't have an FCN training script here in keras-contrib yet, though I plan to adapt & submit this here once things work properly. While the papers don't publish results on Pascal VOC, the original DenseNet has fairly state of the art results on ImageNet & cifar10/cifar100, and DenseNetFCN performed well on CamVid and the Gatech dataset. Considering this I expected that perhaps DenseNetFCN might fail to get state of the art results, but I figured it could most likely in a worst case get over 50% mIOU and around 70-80% pixel accuracy since it has many similarities in common with ResNet, and performed quite well on the much smaller CamVid dataset.

DenseNet FCN configuration I'm using

    return densenet.DenseNetFCN(input_shape=(320, 320, 3),
                                weights=None, classes=classes,
                                nb_layers_per_block=4,
                                growth_rate=13,
                                dropout_rate=0.2)

This is very close to the configuration FC-DenseNet56 from the 100 layers tiramisu aka DenseNetFCN paper.

Sparse training accuracy

Here is what I'm seeing as I train with the Adam optimizer and loss rate of 0.1:

lr: 0.100000
Epoch 2/450
366/366 [==============================] - 287s - loss: 62.5131 - 
sparse_accuracy_ignoring_last_label: 0.3657
[...snip...]
lr: 0.100000
Epoch 148/450
366/366 [==============================] - 286s - loss: 82.2138 - 
sparse_accuracy_ignoring_last_label: 0.3375

Similar networks and training scripts verified for a baseline

1. I've successfully trained AtrousFCNResNet50_16s from scratch up to around 7x% pixel accuracy on the pascal VOC2012 test set.
2. AtrousFCNResNet50_16s with resnet from the keras pretrained weights as fchollet provided downloaded using the Keras-FCN get_weights_path and transfer_FCN script trained without issue to 0.56025 mIOU and around 8x% accuracy.
3. I've been able to train plain DenseNet on cifar 10 up to expected levels of accuracy published in the original papers.
4. I've been able to train DenseNetFCN on a single image, and predict pixels with 99% accuracy on the image itself with all augmentation disabled, and mid 9x% accuracy on augmented versions of the training image.
   • This should at least demonstrate that the network can be trained, and the labels aren't saved incorrectly. I know (4) isn't a valid experiment for final conclusions, it just helps eliminate a variety of possible bugs.

Verifying training scripts with AtrousFCNResNet50_16s

For comparison, AtrousFCNResNet50_16s test set training results, which can be brought to 0.661 mIOU with augmented pascal voc,

I also trained AtrousFCNResNet50_16s from scratch with the results below:

PASCAL VOC trained with pretrained imagenet weights
IOU:
[ 0.90873648  0.74772504  0.44416247  0.57239141  0.50728778  0.51896323
  0.69891196  0.66111323  0.64380596  0.19145411  0.49733934  0.32720705
  0.5488089   0.49649298  0.6157158   0.75780816  0.35492963  0.57446371
  0.32721105  0.63200183  0.53067634]
meanIOU: 0.550343
pixel acc: 0.896132
150.996609926s used to calculate IOU.

Download links

Pascal VOC
http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar  

Augmented 
http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/semantic_contours/benchmark.tgz

Automated VOC download script

Thanks

@titu1994 and @aurora95, since I'm using your respective DenseNetFCN and Keras-FCN implementations, could I get any comments or advice you might have on this? I'd appreciate your thoughts.

All, thanks for giving this a look, as well as your consideration and advice!

Guys, we are 2 bugs away from passing all tests :

• A shape mismatch issue in PELU which shows up in both backends
• A dim-ordering related issue in Deconv3d which shows up in TF backend alone

It would be highly appreciated if any of you could volunteer for fixing one or both of these. A Red Travis badge can damage the credibility of the whole repo (not just the modules that are failing, since people don't go through the test logs.)

Thanks.

Guys, what do you think about adding a Hot Dog/Not Hot Dog example? Images are available in MS COCO dataset.

Cheers.

Hi @farizrahman4u,

I have a few scripts can be used to build keras models of recent papers. Some of them are :

Finished models with weights

Inception v4, Inception-Resnet v1, Inception-Resnet v2 (weights available for Inception v4 - via https://github.com/kentsommer/keras-inceptionV4, and therefore he should add a PR)

Wide Residual Networks (weights for WRN-16-8 and WRN-28-8 for theano backend are available)

Residual of Residual Networks (weights for RoR-WRN-40-2 is available for theano backend)

DenseNets (weights for DenseNet-40-12 available for theano backend)

Custom Callbacks

Snapshot Ensembles (ensemble weights for CIFAR 10 (Wide ResNet - 16 - 4) and CIFAR 100 (Wide ResNet - 16 - 4) are available for theano backend)

My question is : are they worth the addition to the applications folder? And if so, should the weights be hosted in the original repositories or over here?

Moreover, for snapshot ensemble, the example is more important than the custom callback (snapshot ensemble can be used for any model, it simply requires 3 different callbacks to work). The main difficulty is how to use the ensemble model weights to predict (and or simply calculate accuracy).

So should an example be added as well in the examples directory of keras_contrib?

https://github.com/flyyufelix/cnn_finetune has densenet and resnet weights

Hi,

I tried to save the architecture of a model that contains a SReLU activation function using JSON, but I got the following error:

Traceback (most recent call last):
File "Test2.py", line 37, in
open("Test_" + modelname + "_Architecture.json", 'w').write(model.to_json())
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/keras/engine/topology.py", line 2619, in to_json
return json.dumps(model_config, default=get_json_type, **kwargs)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/json/init.py", line 237, in dumps
**kw).encode(obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/json/encoder.py", line 198, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/json/encoder.py", line 256, in iterencode
return _iterencode(o, 0)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/keras/engine/topology.py", line 2616, in get_json_type
raise TypeError('Not JSON Serializable:', obj)
TypeError: ('Not JSON Serializable:', <keras.initializers.RandomUniform object at 0x11714f9e8>)

However I was able to save the entire model using H5.
Is there a way to save the model using JSON?

Hey there, so I've been silently working on a few ideas I've had about a sensible dataset format and I'm currently looking for some feedback before I open a PR here. Could someone check out my version of the MS-COCO dataset and tell me what I would need to add/change in order to make it worthy of a merge?

For instance:

1. It would make more sense if there were separate files for each dataset, like keras-contrib already is.
2. I'm in favor of a base, abstract dataset class with methods, like flow, and attributes like IDS/CLASS_NAMES that should be overriden. I could use some help with that as well though, I'm not sure how that should work.
3. I'd like to remove MSCOCOReduced as a separate class and introduce the possibility of dataset subsets instead, as a feature of the class itself.
4. Transformations of the data should be separate from the definition of the dataset itself. As soon as a common format is finalized, applying transformations to diverse datasets should be a piece of cake.
5. Maybe turn the Dataset class (or the output of "flow") into a Sequence? That way it could be combined with keras-transform

I'm especially interested in your opinions regarding my use of a configuration file instead of feeding raw parameter values through code. I prefer my way visually and I think it's more readable but on the other hand it might be harder to debug.

Thanks a lot, I'm looking forward to your feedback!

EDIT: My code is for semantic segmentation but I've intentionally kept the title vague, this discussion can accommodate any kind of task.

Is it a good idea to define some rules to review a PR?
For example, because we are a lot of collaborators, it could interesting to have more than one approval to merge a PR.
What do you think?

If someone wants to start tackling the issue of making all the tests pass with the CNTK backend that would be amazing!

I'm working on some code that loads ms coco and pascal voc for segmentation, but it would currently involve steps that depend on several other python libraries, including tf only steps. It could be used to create examples for the DenseNetFCN and #46 segmentation models.

I'm creating an example that trains on these keras-contrib models but with the dependency limitations mentioned above, and I can modify it for the keras-contrib code structure. However, this brings up the following questions:

• Would keras-contrib accept a contribution with those limitations?
• What would be the procedure?
• Is someone interested in adapting for more general operation?

If it makes the most sense for me to just keep everything separate, that's okay.

Now I'm using the CRF layer very usefully. Thanx for developed it :)
But I found some inconvenience part, which is saving and loading the model.

I wanted to load the CRF layer with load_model(model_path).

Other layers in keras-contrib can automatically load the model with load_model().
because get_custom_objects().update({'CosineDense': CosineDense}) infrom to origin-keras about the custom layer.

However, the CRF Layer cannot contain this code, because CRF contains custom loss function
Like this code

model.compile(loss=crf.loss_function,
               optimizer='adam',
               metrics=[crf.accuracy])

I also tried this code too but it's not working

model.save(path, custom_objects={'CRF':CRF})
load_model(path, custom_objects={'CRF':CRF})

File "model.py", line 16, in __init__
    self.model = load_model(model_path, custom_objects={'CRF': CRF})
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/keras/models.py", line 264, in load_model
    sample_weight_mode=sample_weight_mode)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/keras/models.py", line 781, in compile
    **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/keras/engine/training.py", line 681, in compile
    loss_function = losses.get(loss)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/keras/losses.py", line 102, in get
    return deserialize(identifier)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/keras/losses.py", line 94, in deserialize
    printable_module_name='loss function')
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 159, in deserialize_keras_object
    ':' + function_name)
ValueError: Unknown loss function:loss

Please give me some idea for saving CRF layer and loss function :)

Environment:

• TensorFlow 1.0.1
• Keras 2.0.1,

Issue:

• When I try and import PELU, I get this trace.

ImportErrorTraceback (most recent call last)
<ipython-input-1-fd71cf66dcca> in <module>()
     12 from sklearn.pipeline import Pipeline
     13 
---> 14 from keras_contrib.layers.advanced_activations import PELU
     15 
     16 LABEL_COLUMN = "item_interest"

/usr/local/lib/python2.7/dist-packages/keras_contrib/__init__.py in <module>()
      1 from __future__ import absolute_import
----> 2 from . import backend
      3 from . import datasets
      4 from . import layers
      5 from . import preprocessing

/usr/local/lib/python2.7/dist-packages/keras_contrib/backend/__init__.py in <module>()
     12     from .theano_backend import *
     13 elif K.backend() == 'tensorflow':
---> 14     from .tensorflow_backend import *

/usr/local/lib/python2.7/dist-packages/keras_contrib/backend/tensorflow_backend.py in <module>()
     13 import os
     14 import warnings
---> 15 from keras.backend.common import floatx, _EPSILON, image_data_format, reset_uids
     16 from keras.backend.tensorflow_backend import _preprocess_conv3d_input
     17 from keras.backend.tensorflow_backend import _preprocess_conv3d_kernel

ImportError: cannot import name reset_uids

Hi, I was wondering if it is alright to add ResNeXt model building code from the paper Aggregated Residual Transformations for Deep Neural Networks without pre-trained weights for CIFAR-10 or CIFAR100.

I'm currently using my GPU for another research project, but since ResNeXt models aren't currently available for Keras, I was hoping to add them to contrib so that someone else could use it to train on CIFAR or even ImageNet.

We can update the code later on to point to the weights for CIFAR.

hi,i check the paper Second-order Convolutional Neural Networks https://arxiv.org/pdf/1703.06817.pdf, it use the Covariance Descriptor Unit to replace full-connected layer.
Can add the Covariance Descriptor Unit? Thanks.

Hello,

I've tried to use the CRF layer as an output to my Bi-LSTM CRF model and it trains well and shows reasonable accuracy by using crf.accuracy instead of "accuracy", however when I do model.predict() or predict_classes or predict_proba it always outputs 0, am I doing anything wrong?
(Tried on both Tensorflow and Theano)

crf = CRF(len(classes), sparse_target=True)

batch_size = 32
model = Sequential()
model.add(Bidirectional(LSTM(64, return_sequences=True, dropout=0.5, recurrent_dropout=0.5),input_shape=(SEQ_LENGTH, 300)))
model.add(crf)

early_stopping = EarlyStopping(monitor='val_loss', patience=1)

model.compile('adam', loss=crf.loss_function, metrics=[crf.accuracy, 'accuracy'])

Edit: I've tested the CoNLL example and it works, however for some reason my model doesn't and always outputs an array of zeros (Even on training data itself), even though the viterbi accuracy is > 95%

@linxihui Can you help please?

We have models for segmentation including DenseNetFCN, but we need a training example.

I've tested Keras-FCN. It works well as a starting point, and the author is happy to see it incorporated into keras-contrib. I also have a fork with DensenetFCN and #46 integrated.

A few questions before a pull request is created:

1. Is there interest in this?
2. Where would be the appropriate place and manner to integrate SegDataGemerator, which performs training/augmentation on segmentation data?
3. What about auxiliary functions such as conversion of pretrained weights, perhaps that should go directly in the example script?

@titu1994 It looks like your PRs from #28 and #69 are causing travis to fail on some of your changes in test_normalization.py in keras-contrib. Would you mind either looking for a fix or creating a PR that disables the relevant code?

https://travis-ci.org/farizrahman4u/keras-contrib
https://travis-ci.org/farizrahman4u/keras-contrib/jobs/278090084

Hi,

Thanks for sharing these new implementations.
I tried training fcn densenet with camvid dataset but ended up with following error.

ValueError: Error when checking target: expected reshape_2 to have 4 dimensions, but got array with shape (367, 50176, 12)

Could you please let me know what am I doing wrong here?