Should not the input_shape = (16,112,112,16) instead of input_shape = (112,112,16,3) ?

Can the model takes webcam input?
Thanks in advance.

您好，非常感谢你的分享，我安装了对应版本的tensorflow和keras，当我运行video_demo.py时，加载模型时出现下面的问题：
Traceback (most recent call last):
File "video_demo.py", line 64, in
main()
File "video_demo.py", line 22, in main
load_model('.\results\weights_c3d.h5')
File "C:\Anaconda2\envs\py3\envs\keras\lib\site-packages\keras\models.py", line 233, in load_model
with h5py.File(filepath, mode='r') as f:
File "C:\Anaconda2\envs\py3\envs\keras\lib\site-packages\h5py_hl\files.py", line 312, in init
fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
File "C:\Anaconda2\envs\py3\envs\keras\lib\site-packages\h5py_hl\files.py", line 142, in make_fid
fid = h5f.open(name, flags, fapl=fapl)
File "h5py_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py\h5f.pyx", line 78, in h5py.h5f.open
esults\weights_c3d.h5', errno = 22, error message = 'Invalid argument', flags = 0, o_flags = 0)
作者你有遇到过吗？非常感谢

训练集准确率和验证集准确率差这么多，而且训练集准确率快到1了，难到不是过拟合了吗

Hi@TianzhongSong
博主你好，我现在想用你的网络训练自己的数据集（51类），我已经将数据集制作成需要的格式，并生成了train_list.txt和test_list.txt,我将models.py中的nb_classes改为了51；将train_c3d.py中的num_classes也改为了51，我在WIN10下训练，没有找到train_c3d.py下定义的root_path，我将root_path改为视频类别所在的目录，但是在运行train_c3d.py时，model.fit_generator处报错：
Epoch 1/6 2018-10-12 21:58:30.598905: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2 2018-10-12 21:58:30.600815: I tensorflow/core/common_runtime/process_util.cc:69] Creating new thread pool with default inter op setting: 8. Tune using inter_op_parallelism_threads for best performance. 2018-10-12 21:58:33.877366: E tensorflow/core/common_runtime/executor.cc:697] Executor failed to create kernel. Invalid argument: Value for attr 'data_format' of "NDHWC" is not in the list of allowed values: "NHWC", "NCHW" ; NodeDef: MklInputConversion/_237 = _MklInputConversion[T=DT_FLOAT, _kernel="MklOp", data_format="NDHWC", _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv3d_1/convolution, conv3d_1/Reshape, DMT/_10, conv3d_1/Reshape:1); Op<name=_MklInputConversion; signature=input_0:T, input_1:T, mkl_input_0:uint8, mkl_input_1:uint8 -> output_0:T, output_1:T, mkl_output_0:uint8, mkl_output_1:uint8; attr=T:type,allowed=[DT_HALF, DT_FLOAT, DT_DOUBLE, DT_UINT8, DT_INT8, DT_UINT16, DT_INT16, DT_INT32, DT_INT64, DT_COMPLEX64, DT_COMPLEX128]; attr=data_format:string,default="NHWC",allowed=["NHWC", "NCHW"]> [[Node: MklInputConversion/_237 = _MklInputConversion[T=DT_FLOAT, _kernel="MklOp", data_format="NDHWC", _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv3d_1/convolution, conv3d_1/Reshape, DMT/_10, conv3d_1/Reshape:1)]] Traceback (most recent call last): File "L:/match/2018jiangsupostgraduatecv/C3D-keras-master/train_c3d.py", line 195, in <module> main() File "L:/match/2018jiangsupostgraduatecv/C3D-keras-master/train_c3d.py", line 173, in main verbose=1) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper return func(*args, **kwargs) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\engine\training.py", line 1415, in fit_generator initial_epoch=initial_epoch) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\engine\training_generator.py", line 213, in fit_generator class_weight=class_weight) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\engine\training.py", line 1215, in train_on_batch outputs = self.train_function(ins) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\backend\tensorflow_backend.py", line 2666, in __call__ return self._call(inputs) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\backend\tensorflow_backend.py", line 2635, in _call session) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\backend\tensorflow_backend.py", line 2587, in _make_callable callable_fn = session._make_callable_from_options(callable_opts) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\tensorflow\python\client\session.py", line 1414, in _make_callable_from_options return BaseSession._Callable(self, callable_options) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\tensorflow\python\client\session.py", line 1368, in __init__ session._session, options_ptr, status) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 519, in __exit__ c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.InvalidArgumentError: Value for attr 'data_format' of "NDHWC" is not in the list of allowed values: "NHWC", "NCHW" ; NodeDef: MklInputConversion/_237 = _MklInputConversion[T=DT_FLOAT, _kernel="MklOp", data_format="NDHWC", _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv3d_1/convolution, conv3d_1/Reshape, DMT/_10, conv3d_1/Reshape:1); Op<name=_MklInputConversion; signature=input_0:T, input_1:T, mkl_input_0:uint8, mkl_input_1:uint8 -> output_0:T, output_1:T, mkl_output_0:uint8, mkl_output_1:uint8; attr=T:type,allowed=[DT_HALF, DT_FLOAT, DT_DOUBLE, DT_UINT8, DT_INT8, DT_UINT16, DT_INT16, DT_INT32, DT_INT64, DT_COMPLEX64, DT_COMPLEX128]; attr=data_format:string,default="NHWC",allowed=["NHWC", "NCHW"]> [[Node: MklInputConversion/_237 = _MklInputConversion[T=DT_FLOAT, _kernel="MklOp", data_format="NDHWC", _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv3d_1/convolution, conv3d_1/Reshape, DMT/_10, conv3d_1/Reshape:1)]] Exception ignored in: <bound method BaseSession._Callable.__del__ of <tensorflow.python.client.session.BaseSession._Callable object at 0x0000027A1EE51D30>> Traceback (most recent call last): File "K:\Anaconda\envs\classifyvedio\lib\site-packages\tensorflow\python\client\session.py", line 1398, in __del__ self._session._session, self._handle, status) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 519, in __exit__ c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.InvalidArgumentError: No such callable handle: 2723102589000 我在网上没找到解决方案，所以想请教我这种做法有哪里出错了？十分感谢

I wonder why is the input transposed:

x = np.transpose(x, (0,2,3,1,4))

in both the val and the training stacks generation

Im not able to download it since its in baidu . Can you please upload it to dropbox?

How to output the accuracy of a single input video??

Hello @TianzhongSong

generator_output = next(self._generator)

File "/ds_data/C3D-keras-master/train_c3d.py", line 116, in generator_train_batch
x_train, x_labels = process_batch(new_line[a:b],img_path,train=True)
File "/ds_data/C3D-keras-master/train_c3d.py", line 71, in process_batch
img = imgs[symbol + j]
IndexError: list index out of range

Traceback (most recent call last):
File "/ds_data/C3D-keras-master/train_c3d.py", line 180, in
main()
File "/ds_data/C3D-keras-master/train_c3d.py", line 171, in main
validation_steps=val_samples // batch_size,verbose=None)
File "/usr/local/lib/python3.5/dist-packages/keras/legacy/interfaces.py", line 87, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/keras/engine/training.py", line 2011, in fit_generator
generator_output = next(output_generator)
StopIteration

Process finished with exit code 1

help me to solve this error <3

你好，我按照你您出的方法进行训练了，训练是能够跑起来的，但是非常慢，我的显卡是1080ti，训练了一天之后，现在第二个epoch快要跑完了，但是准确率还是只有2%，并没有提高的迹象，loss已经降到了4左右，请问我可能是哪里弄错了，谢谢您。 @TianzhongSong

Hi,

I tried to run your code but it shown the following error in generator_val_batch
https://github.com/TianzhongSong/C3D-keras/blob/master/train_c3d.py#L138

test_data = np.transpose(test_data, (0, 2, 3, 1, 4))

UnboundLocalError: local variable 'test_data' referenced before assignment.
Do you have any idea?

你好！当我用我自己的数据去训练C3D时，我替换了test_file.txt，train_file.txt，classind.txt，但是当我执行make_label时，无法生成对应的test_list.txt。
显示的是：OSError：文件名、目录名或卷标语法不正确。：‘F：/model/C3D/imgs/\n’
这个的意思是没有对应的行为类别下的图片吗，但是我在vedio2img.py中已经转换了对应class的图片了。

what is this part in the def process_batch means?
i'm new in this feild could you explain it? thank you!!

if train: crop_x = random.randint(0, 15) crop_y = random.randint(0, 58) is_flip = random.randint(0, 1) for j in range(16): img = imgs[symbol + j] image = cv2.imread(img_path + path + '/' + img) image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) image = cv2.resize(image, (171, 128)) if is_flip == 1: image = cv2.flip(image, 1) batch[i][j][:][:][:] = image[crop_x:crop_x + 112, crop_y:crop_y + 112, :] labels[i] = label else: for j in range(16): img = imgs[symbol + j] image = cv2.imread(img_path + path + '/' + img) image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) image = cv2.resize(image, (171, 128)) batch[i][j][:][:][:] = image[8:120, 30:142, :] labels[i] = label

inputs[..., 0] -= 99.9
inputs[..., 1] -= 92.1
inputs[..., 2] -= 82.6
inputs[..., 0] /= 65.8
inputs[..., 1] /= 62.3
inputs[..., 2] /= 60.3
请问这一步的作用是什么？

hi @TianzhongSong ,
Using TensorFlow backend.

Layer (type) Output Shape Param #

input_1 (InputLayer) (None, 112, 112, 16, 3) 0

conv3d_1 (Conv3D) (None, 112, 112, 16, 64) 5248

max_pooling3d_1 (MaxPooling3 (None, 56, 56, 16, 64) 0

conv3d_2 (Conv3D) (None, 56, 56, 16, 128) 221312

max_pooling3d_2 (MaxPooling3 (None, 28, 28, 8, 128) 0

conv3d_3 (Conv3D) (None, 28, 28, 8, 128) 442496

max_pooling3d_3 (MaxPooling3 (None, 14, 14, 4, 128) 0

conv3d_4 (Conv3D) (None, 14, 14, 4, 256) 884992

max_pooling3d_4 (MaxPooling3 (None, 7, 7, 2, 256) 0

conv3d_5 (Conv3D) (None, 7, 7, 2, 256) 1769728

max_pooling3d_5 (MaxPooling3 (None, 4, 4, 1, 256) 0

flatten_1 (Flatten) (None, 4096) 0

dense_1 (Dense) (None, 2048) 8390656

dropout_1 (Dropout) (None, 2048) 0

dense_2 (Dense) (None, 2048) 4196352

dropout_2 (Dropout) (None, 2048) 0

dense_3 (Dense) (None, 101) 206949

activation_1 (Activation) (None, 101) 0

Total params: 16,117,733
Trainable params: 16,117,733
Non-trainable params: 0

2019-03-14 10:46:39.491224: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2019-03-14 10:46:39.491251: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2019-03-14 10:46:39.491257: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2019-03-14 10:46:39.491261: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2019-03-14 10:46:39.491265: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
self.run()
File "/usr/lib/python3.5/threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
File "/home/ltroot/.local/lib/python3.5/site-packages/keras/utils/data_utils.py", line 568, in data_generator_task
generator_output = next(self._generator)
File "/ds_data/C3D-keras-master/train_c3d.py", line 117, in generator_train_batch
x_train, x_labels = process_batch(new_line[a:b], img_path, train=True)
File "/ds_data/C3D-keras-master/train_c3d.py", line 72, in process_batch
img = imgs[symbol + j]
IndexError: list index out of range

Epoch 1/5
Traceback (most recent call last):
File "/ds_data/C3D-keras-master/train_c3d.py", line 182, in
main()
File "/ds_data/C3D-keras-master/train_c3d.py", line 173, in main
verbose=1)
File "/home/ltroot/.local/lib/python3.5/site-packages/keras/legacy/interfaces.py", line 87, in wrapper
return func(*args, **kwargs)
File "/home/ltroot/.local/lib/python3.5/site-packages/keras/engine/training.py", line 2011, in fit_generator
generator_output = next(output_generator)
StopIteration

Process finished with exit code 1
please look into this issue

I would like to ask about the normalization of the input:

            inputs[..., 0] -= 99.9
            inputs[..., 1] -= 92.1
            inputs[..., 2] -= 82.6
            inputs[..., 0] /= 65.8
            inputs[..., 1] /= 62.3
            inputs[..., 2] /= 60.3

I am not sure if these numbers are correct, I have computed the mean and it looks different:

>> mean_dataset_16.mean(axis=0).mean(axis=0)
array([ 90.48464058,  97.57829463, 102.25286134])

and here is the std deviation:

[70.59445308, 69.45308667, 71.02353467]

Am I missing something or there is an issue here?

@TianzhongSong

Thank you for sharing!

I want to know the version of Keras, because because i had a problem, namely, "Importerror cannot import name Max_Pool3D".
This may not be a code problem, so i want to change my version of Keras.

Thank you so much!

Hi TianzhongSong:
Can you tell me how to make train_list.txt and test_list.txt ?You are handmade. Not necessarily the best recognition rate for every 16 entries
ApplyEyeMakeup/v_ApplyEyeMakeup_g01_c01 1 0
ApplyEyeMakeup/v_ApplyEyeMakeup_g01_c01 17 0
ApplyEyeMakeup/v_ApplyEyeMakeup_g01_c01 33 0
ApplyEyeMakeup/v_ApplyEyeMakeup_g01_c01 49 0
ApplyEyeMakeup/v_ApplyEyeMakeup_g01_c01 65 0
ApplyEyeMakeup/v_ApplyEyeMakeup_g01_c01 81 0
ApplyEyeMakeup/v_ApplyEyeMakeup_g01_c01 97 0

your sincerely
Yongsheng Qian

您好，非常感谢你的代码，我在train_c3d.py中没有看到root_path的定义，请问这个应该怎么解决吗