Comments (6)
lecode-official
commented on July 18, 2024
Same here, I have tried to install Keras 1.1.2 and Tensorflow 0.12.0 (0.10.0 is not available via PyPI anymore). Then I have tried to convert the code to the current Keras and Tensorflow versions:
inputs = Input((6, self.frame_height, self.frame_width))
conv1 = Conv2D(32, (3, 3), activation='relu', padding='same', data_format="channels_first")(inputs)
bn1 = BatchNormalization()(conv1)
conv1 = Conv2D(32, (3, 3), activation='relu', padding='same', data_format="channels_first")(bn1)
bn1 = BatchNormalization()(conv1)
pool1 = MaxPooling2D(pool_size=(2, 2), data_format="channels_first")(bn1)
conv2 = Conv2D(64, (3, 3), activation='relu', padding='same', data_format="channels_first")(pool1)
bn2 = BatchNormalization()(conv2)
conv2 = Conv2D(64, (3, 3), activation='relu', padding='same', data_format="channels_first")(bn2)
bn2 = BatchNormalization()(conv2)
pool2 = MaxPooling2D(pool_size=(2, 2), data_format="channels_first")(bn2)
conv3 = Conv2D(128, (3, 3), activation='relu', padding='same', data_format="channels_first")(pool2)
bn3 = BatchNormalization()(conv3)
conv3 = Conv2D(128, (3, 3), activation='relu', padding='same', data_format="channels_first")(bn3)
bn3 = BatchNormalization()(conv3)
pool3 = MaxPooling2D(pool_size=(2, 2), data_format="channels_first")(bn3)
conv4 = Conv2D(256, (3, 3), activation='relu', padding='same', data_format="channels_first")(pool3)
bn4 = BatchNormalization()(conv4)
conv4 = Conv2D(256, (3, 3), activation='relu', padding='same', data_format="channels_first")(bn4)
bn4 = BatchNormalization()(conv4)
pool4 = MaxPooling2D(pool_size=(2, 2), data_format="channels_first")(bn4)
conv5 = Conv2D(512, (3, 3), activation='relu', padding='same', data_format="channels_first")(pool4)
bn5 = BatchNormalization()(conv5)
conv5 = Conv2D(512, (3, 3), activation='relu', padding='same', data_format="channels_first")(bn5)
bn5 = BatchNormalization()(conv5)
pool5 = MaxPooling2D(pool_size=(2, 2), data_format="channels_first")(bn5)
conv5_2 = Conv2D(512, (3, 3), activation='relu', padding='same', data_format="channels_first")(pool5)
bn5_2 = BatchNormalization()(conv5_2)
conv5_2 = Conv2D(512, (3, 3), activation='relu', padding='same', data_format="channels_first")(bn5_2)
bn5_2 = BatchNormalization()(conv5_2)
up5_2 = concatenate([UpSampling2D(size=(2, 2))(bn5_2), bn5], axis=1)
conv6_2 = Conv2D(512, (3, 3), activation='relu', padding='same', data_format="channels_first")(up5_2)
bn6_2 = BatchNormalization()(conv6_2)
conv6_2 = Conv2D(512, (3, 3), activation='relu', padding='same', data_format="channels_first")(bn6_2)
bn6_2 = BatchNormalization()(conv6_2)
up6 = concatenate([UpSampling2D(size=(2, 2))(bn6_2), bn4], axis=1)
conv6 = Conv2D(256, (3, 3), activation='relu', padding='same', data_format="channels_first")(up6)
bn6 = BatchNormalization()(conv6)
conv6 = Conv2D(256, (3, 3), activation='relu', padding='same', data_format="channels_first")(bn6)
bn6 = BatchNormalization()(conv6)
up7 = concatenate([UpSampling2D(size=(2, 2))(bn6), bn3], axis=1)
conv7 = Conv2D(128, (3, 3), activation='relu', padding='same', data_format="channels_first")(up7)
bn7 = BatchNormalization()(conv7)
conv7 = Conv2D(128, (3, 3), activation='relu', padding='same', data_format="channels_first")(bn7)
bn7 = BatchNormalization()(conv7)
up8 = concatenate([UpSampling2D(size=(2, 2))(bn7), bn2], axis=1)
conv8 = Conv2D(64, (3, 3), activation='relu', padding='same', data_format="channels_first")(up8)
bn8 = BatchNormalization()(conv8)
conv8 = Conv2D(64, (3, 3), activation='relu', padding='same', data_format="channels_first")(bn8)
bn8 = BatchNormalization()(conv8)
up9 = concatenate([UpSampling2D(size=(2, 2))(bn8), bn1], axis=1)
conv9 = Conv2D(32, (3, 3), activation='relu', padding='same', data_format="channels_first")(up9)
bn9 = BatchNormalization()(conv9)
conv9 = Conv2D(32, (3, 3), activation='relu', padding='same', data_format="channels_first")(bn9)
bn9 = BatchNormalization()(conv9)
conv10 = Conv2D(3, (1, 1), activation='sigmoid')(bn9)But that does not work either, because I always get to the problem, that the upscaling is incorrect (I am not very familar with Keras, so I guess some internal implementations have changed, which changed the dimensions of the tensors):
Traceback (most recent call last):
File "/home/david/Repositories/tisr-masters-thesis/tisr/__main__.py", line 16, in <module>
application.run()
File "/home/david/Repositories/tisr-masters-thesis/tisr/application.py", line 89, in run
command.run(command_line_arguments)
File "/home/david/Repositories/tisr-masters-thesis/tisr/commands/predict_deep_motion_command.py", line 89, in run
with InferenceModel(command_line_arguments.checkpoint_file_name, 384, 128) as inference_model:
File "/home/david/Repositories/tisr-masters-thesis/tisr/neural_network/deep_motion_model.py", line 141, in __enter__
self.initialize()
File "/home/david/Repositories/tisr-masters-thesis/tisr/neural_network/deep_motion_model.py", line 88, in initialize
up5_2 = concatenate([UpSampling2D(size=(2, 2))(bn5_2), bn5], axis=1)
File "/usr/local/lib/python3.5/dist-packages/keras/layers/merge.py", line 641, in concatenate
return Concatenate(axis=axis, **kwargs)(inputs)
File "/usr/local/lib/python3.5/dist-packages/keras/engine/topology.py", line 594, in __call__
self.build(input_shapes)
File "/usr/local/lib/python3.5/dist-packages/keras/layers/merge.py", line 354, in build
'Got inputs shapes: %s' % (input_shape))
ValueError: A `Concatenate` layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 1024, 8, 12), (None, 512, 8, 24)]
Any help will be greatly appreciated.
from deep-motion.
lecode-official
commented on July 18, 2024
Okay, sorry, I just found the following comment: #2 (comment), which explains how to fix this. In that case the data_format as in my previous post, does not seem to be necessary. This is the code I used to successfully convert the code to work on Tensorflow 1.6.0 and Keras 2.1.6:
inputs = Input((6, self.frame_height, self.frame_width))
conv1 = Conv2D(32, (3, 3), activation='relu', padding='same')(inputs)
bn1 = BatchNormalization()(conv1)
conv1 = Conv2D(32, (3, 3), activation='relu', padding='same')(bn1)
bn1 = BatchNormalization()(conv1)
pool1 = MaxPooling2D(pool_size=(2, 2))(bn1)
conv2 = Conv2D(64, (3, 3), activation='relu', padding='same')(pool1)
bn2 = BatchNormalization()(conv2)
conv2 = Conv2D(64, (3, 3), activation='relu', padding='same')(bn2)
bn2 = BatchNormalization()(conv2)
pool2 = MaxPooling2D(pool_size=(2, 2))(bn2)
conv3 = Conv2D(128, (3, 3), activation='relu', padding='same')(pool2)
bn3 = BatchNormalization()(conv3)
conv3 = Conv2D(128, (3, 3), activation='relu', padding='same')(bn3)
bn3 = BatchNormalization()(conv3)
pool3 = MaxPooling2D(pool_size=(2, 2))(bn3)
conv4 = Conv2D(256, (3, 3), activation='relu', padding='same')(pool3)
bn4 = BatchNormalization()(conv4)
conv4 = Conv2D(256, (3, 3), activation='relu', padding='same')(bn4)
bn4 = BatchNormalization()(conv4)
pool4 = MaxPooling2D(pool_size=(2, 2))(bn4)
conv5 = Conv2D(512, (3, 3), activation='relu', padding='same')(pool4)
bn5 = BatchNormalization()(conv5)
conv5 = Conv2D(512, (3, 3), activation='relu', padding='same')(bn5)
bn5 = BatchNormalization()(conv5)
pool5 = MaxPooling2D(pool_size=(2, 2))(bn5)
conv5_2 = Conv2D(512, (3, 3), activation='relu', padding='same')(pool5)
bn5_2 = BatchNormalization()(conv5_2)
conv5_2 = Conv2D(512, (3, 3), activation='relu', padding='same')(bn5_2)
bn5_2 = BatchNormalization()(conv5_2)
up5_2 = concatenate([UpSampling2D(size=(2, 2))(bn5_2), bn5], axis=1)
conv6_2 = Conv2D(512, (3, 3), activation='relu', padding='same')(up5_2)
bn6_2 = BatchNormalization()(conv6_2)
conv6_2 = Conv2D(512, (3, 3), activation='relu', padding='same')(bn6_2)
bn6_2 = BatchNormalization()(conv6_2)
up6 = concatenate([UpSampling2D(size=(2, 2))(bn6_2), bn4], axis=1)
conv6 = Conv2D(256, (3, 3), activation='relu', padding='same')(up6)
bn6 = BatchNormalization()(conv6)
conv6 = Conv2D(256, (3, 3), activation='relu', padding='same')(bn6)
bn6 = BatchNormalization()(conv6)
up7 = concatenate([UpSampling2D(size=(2, 2))(bn6), bn3], axis=1)
conv7 = Conv2D(128, (3, 3), activation='relu', padding='same')(up7)
bn7 = BatchNormalization()(conv7)
conv7 = Conv2D(128, (3, 3), activation='relu', padding='same')(bn7)
bn7 = BatchNormalization()(conv7)
up8 = concatenate([UpSampling2D(size=(2, 2))(bn7), bn2], axis=1)
conv8 = Conv2D(64, (3, 3), activation='relu', padding='same')(up8)
bn8 = BatchNormalization()(conv8)
conv8 = Conv2D(64, (3, 3), activation='relu', padding='same')(bn8)
bn8 = BatchNormalization()(conv8)
up9 = concatenate([UpSampling2D(size=(2, 2))(bn8), bn1], axis=1)
conv9 = Conv2D(32, (3, 3), activation='relu', padding='same')(up9)
bn9 = BatchNormalization()(conv9)
conv9 = Conv2D(32, (3, 3), activation='relu', padding='same')(bn9)
bn9 = BatchNormalization()(conv9)
conv10 = Conv2D(3, (1, 1), activation='sigmoid')(bn9)from deep-motion.
clausmichele
commented on July 18, 2024
Can you expand on this? I copied and paste your code into the definition of get_unet_2but it still gives me an error:
ValueError: Negative dimension size caused by subtracting 2 from 1 for 'max_pooling2d_3/MaxPool' (op: 'MaxPool') with input shapes: [?,1,32,128].
from deep-motion.
lecode-official
commented on July 18, 2024
Sorry, I should have elaborated a little more. The problem ist, that by default Keras uses TensorFlow channel ordering, i.e. height, width, channels. But in this case the author changed the channel ordering to Theano, i.e. channels, height, width. Neil probably set that in the Keras configuration file, so everyone checking out his code didn't have that configuration. But you can add the following two lines at the top of your code to switch the channel ordering just for this code:
import keras.backend
keras.backend.set_image_dim_ordering('th')This should do the trick. Hope that helps.
from deep-motion.
SpaceMonkeyForever
commented on July 18, 2024
@lecode-official I got fps_convert.py by following what you said, so thanks for that.
I'm downloading the pre-trained model because I can't get "train" to work and the data download script doesn't work any more because the KITTI data was moved on the server. I think I can skip training by using the pre-trained model.
The issue is that the model expects input with dimensions 384x128 and I want it to work with any video file. My understanding is it should work fine because the weights don't depend on dims of the input since it's a conv net (the same mask is applied regardless of the input size). However, Keras doesn't accept that. Is there any way around it?
To be clear, I'm doing:
model = get_unet_2((6, height, width))
model.load_weights("./../model_weights/weights_unet2_finetune_youtube_100epochs.hdf5") # throws error
where "height" and "width" are taken from the file I'm reading and are not equal to 384x128.
I get the following error:
ValueError: Dimension 0 in both shapes must be equal, but are 192 and 384. Shapes are [192] and [384]. for 'Assign_2' (op: 'Assign') with input shapes: [192], [384].
192 is the width I'm giving to get_unet_2 which doesn't match 384.
from deep-motion.
SpaceMonkeyForever
commented on July 18, 2024
@lecode-official I see now that they mentioned the input dimensions cannot be changed because of batch normalisation in the model. I don't understand why this matters tho. How does batch normalisation on the hidden layers tie the dimensions of the input and the weights trained together?
from deep-motion.
Related Issues (14)
- Reproduce results HOT 2
- raw_data_downloader.sh not working HOT 1
- Vid lists in batch_video_download.py nolonger valid
- Unable to download the KITTI dataset
- Training with new dataset
- does deep-motion depends on unspecified configurations or dependencies ? HOT 15
- Archive files for model weights possibly corrupt HOT 10
- Error in get_unet_2 function HOT 3
- Sample video generation HOT 1
- Why the 20:404 offset in the image array? HOT 2
- Problem training HOT 1
- Input data error
- No license specified HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deep-motion.