neil454 / deep-motion Goto Github PK

I am facing some problems running the train.py file. I have followed your instructions but an error has occurred regarding the pool4 in get_unet_2.
Error:
ValueError: Negative dimension size caused by subtracting 2 from 1 for 'max_pooling2d_3/MaxPool' (op: 'MaxPool') with input shapes: [?,1,32,128].

The input shape I'm using is the same as you gave: (6, 128, 384).

When only run two lines:
from FI_unet import get_unet_2
model = get_unet_2((6, 128, 384))

it shows error of:

when run:

from FI_unet import get_unet_2
model = get_unet()

it shows error of:

Note that I use:
Python 3.5
Tensorflow 0.10.0
Keras 1.1.2

The urls used in batch_video_download.py for the youtube-8M files no longer source any files which categories did you source your original training files from ?

I am trying to work out any image pre-processing that was needed for the KITTI data, and wondered if there was any particular reason for the 20:404 array range?

Did you manually resize the images before ingesting or are you "cropping" the X array?

Hi Neil,

I have couple of queries regarding the training data.
I understand that the training set should be of dimensions 128x384.

However in train.py, the input given for training is,

• X_val_KITTI.npy
• y_val_KITTI.npy

I would like to know how the training dataset is loaded exactly inside these files.

I want to train for higher resolution [720p] videos.
But i am not understanding how to modify train.py & how to load the training data for the same.

Please consider this request as high priority.
Waiting for your guidance.

It seems the split zip files containing the model's weights cannot be properly extracted.
The following command was used to create the merged zip:

zip -s- weights.zip -O weightsCombined.zip

The resulting zip could not be extracted (it creates a damaged file).

$ unzip -FF weightsCombo.zip
Archive: weightsCombo.zip
inflating: weights_unet2_finetune_youtube_100epochs.hdf5
error: invalid compressed data to inflate

Further testing shows that the weights.zip apparently contains no (useful) data. Joining with the another command reveals problems with the file.

$ zip -FF weights.zip --out weights_fixed.zip
Fix archive (-FF) - salvage what can
Found end record (EOCDR) - says expect 4 splits
Scanning for entries...
Found spanning marker - expected as this is split (multi-disk) archive...
copying: weights_unet2_finetune_youtube_100epochs.hdf5 (212788492 bytes)
zip warning: entry truncated: weights_unet2_finetune_youtube_100epochs.hdf5
zip warning: expected compressed/stored size 212788492, actual 134217621
zip warning: unexpected signature 50 4b 0a 0c on disk 2 at 33332149

   zip warning: skipping this signature...
   zip warning: unexpected signature 50 4b 0b 06 on disk 2 at 36919392

   zip warning: skipping this signature...
   zip warning: unexpected signature 50 4b 03 07 on disk 2 at 59441625

   zip warning: skipping this signature...
   zip warning: unexpected signature 50 4b 05 0b on disk 3 at 1861669

   zip warning: skipping this signature...

Central Directory found...
no local entry: weights_unet2_finetune_youtube_100epochs.hdf5
EOCDR found ( 4 11462122)...
zip warning: zip file empty

While trying deep-motion convert I kept getting errors about dependencies not being installed on my computer , despite having installed all the libraries mentioned in the readme , now that seems to be gone , but now i get this.

GStreamer-CRITICAL **: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed

(python:6153): GStreamer-CRITICAL **: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed
GStreamer: Error opening bin: no source element for URI "/media/neil/Neil's"
Traceback (most recent call last):
File "fps_convert.py", line 89, in

when trying to run fps convert , the error report mentions a folder in /media , is that something you have in your computer, that the program depends on ?

Hi,
I was trying to test the model but I get the following errors:

Using TensorFlow backend.
/home/cm72ein/Project/DeepMotion/deep-motion-master/src/FI_unet.py:59: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(32, (3, 3), padding="same", activation="relu")
conv1 = Convolution2D(32, 3, 3, activation='relu', border_mode='same')(inputs)
/home/cm72ein/Project/DeepMotion/deep-motion-master/src/FI_unet.py:61: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(32, (3, 3), padding="same", activation="relu")
conv1 = Convolution2D(32, 3, 3, activation='relu', border_mode='same')(bn1)
/home/cm72ein/Project/DeepMotion/deep-motion-master/src/FI_unet.py:65: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(64, (3, 3), padding="same", activation="relu")
conv2 = Convolution2D(64, 3, 3, activation='relu', border_mode='same')(pool1)
/home/cm72ein/Project/DeepMotion/deep-motion-master/src/FI_unet.py:67: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(64, (3, 3), padding="same", activation="relu")
conv2 = Convolution2D(64, 3, 3, activation='relu', border_mode='same')(bn2)
/home/cm72ein/Project/DeepMotion/deep-motion-master/src/FI_unet.py:71: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(128, (3, 3), padding="same", activation="relu")
conv3 = Convolution2D(128, 3, 3, activation='relu', border_mode='same')(pool2)
/home/cm72ein/Project/DeepMotion/deep-motion-master/src/FI_unet.py:73: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(128, (3, 3), padding="same", activation="relu")
conv3 = Convolution2D(128, 3, 3, activation='relu', border_mode='same')(bn3)
Traceback (most recent call last):
File "train.py", line 202, in
main()
File "train.py", line 85, in main
model = get_unet_2(input_shape=(6, 128, 384))
File "/home/cm72ein/Project/DeepMotion/deep-motion-master/src/FI_unet.py", line 75, in get_unet_2
pool3 = MaxPooling2D(pool_size=(2, 2))(bn3)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 619, in call
output = self.call(inputs, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/keras/layers/pooling.py", line 158, in call
data_format=self.data_format)
File "/usr/local/lib/python2.7/dist-packages/keras/layers/pooling.py", line 221, in _pooling_function
pool_mode='max')
File "/usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py", line 3657, in pool2d
data_format=tf_data_format)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/nn_ops.py", line 1958, in max_pool
name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_nn_ops.py", line 2806, in _max_pool
data_format=data_format, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2958, in create_op
set_shapes_for_outputs(ret)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2209, in set_shapes_for_outputs
shapes = shape_func(op)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2159, in call_with_requiring
return call_cpp_shape_fn(op, require_shape_fn=True)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/common_shapes.py", line 627, in call_cpp_shape_fn
require_shape_fn)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/common_shapes.py", line 691, in _call_cpp_shape_fn_impl
raise ValueError(err.message)
ValueError: Negative dimension size caused by subtracting 2 from 1 for 'max_pooling2d_3/MaxPool' (op: 'MaxPool') with input shapes: [?,1,32,128].

Hi Neil,
Is it possible for you to attach a readme in order to reproduce training of a model? Or can you possibly upload a pretrained model in order to investigate the potential when interpolation of frames using neural networks?

Hi Neil,
I am working on video frame interpolation, your project seems to be very interesting. But I have trouble when using your network. Due to the different version of software packages and some other unexpected errors. As the deadline is tomorrow for my project, I wonder whether it is possible for you to run a 5-frame-video and generate 8 times more frames than the original ones (using 3 times of function: double_vid_fps( )). The goal of this work is only to compare the tracking results of different methods. Of course, we will credit this video on you and cite your paper in our later paper.

output.mp4.zip

There is no license included in this repository, and there is no statement about allowed usage in the README.md file. This makes it extremely risky to use this code even for educational purposes. I would appreciate the addition of a license or some guidance on allowed usage.

Hi Neil,
I was going through your sample code for frame interpolation.
However, I could not download the dataset you have used to train your model for interpolation.
"http://kitti.is.tue.mpg.de/kitti/raw_data/"

Can you please let me know how i can get the dataset?

Best regards,
Ashritha

http://kitti.is.tue.mpg.de/kitti/raw_data/2011_09_26_calib.zip give 404