pomonam / attentioncluster Goto Github PK

Hi,

Thanks a lot for your code! Could you provide some general guidelines on how extracted video/audio features from any dataset should be prepared to train the frame-level AttentionClusterModule? Thanks. :)

Hello Juhan Bae,

This repo for Attention Cluster is nice, after reading the corresponding paper I have a question about the number of clusters. It seems that you implement the attention cluster with only one attention map for each modality, while in the official paper there are many clusters(many attention maps) for each modality (rgb & audio), so maybe a loop should be introduced in class AttentionClusterModel() in frame_level_model.py ? :- )

Best wishes,
Skye

Hi,

I believe that the following line is unnecessary:

Maybe L101 is unnecessary as well. In my understanding, the paper proposes normalisation only right after the shifting operation. So, I believe that only the normalisation in L99 is useful.

I can understand the normalisation in L101 as is common to normalise the whole vector and not only within the cluster (similarly to VLAD), but L87 seems pretty redundant.

Let me know what you think. Thanks.

The article mentions that "when we train the model, we can randomly sample a part of the features from the local feature set, but use all the features during testing." Why can we use only a part of the features for training, but use all the features during testing?

Thank you!

I run your code and download dataset from kaggle manually but I got this error :

tensorflow.python.framework.errors_impl.InvalidArgumentError: Name: , Feature list 'audio' is required but could not be found.  Did you mean to include it in feature_list_dense_missing_assumed_empty or feature_list_dense_defaults?
	 [[{{node train_input/ParseSingleSequenceExample_1/ParseSingleSequenceExample}}]]

How can I fix this issue?