antoine77340 / loupe Goto Github PK

Hi, firstly thank you for making this code public! I am currently looking to reproduce some of the experiments done in the NetVLAD paper. To do this I'd like to use the VLAD layer as defined here and appended to the end of a VGG16 network as they do in the paper. The output shape of a feature from VGG16, minus the final classification layer is 7,7,512. However, I can't figure out how to pass this to the VLAD layer as defined here. There seems to be roughly four input parameters: feature_size, max_samples, cluster_size, output_dim

The paper describes an overview of the system: "Formally, given N D-dimensional local image descriptors as input, and K cluster centres (“visual words”) as VLAD parameters, the output VLAD image representation V is K×D-dimensional. For convenience we will write V as a K ×D matrix, but this matrix is converted into a vector and, after normalization, used as the image representation"

Feature size: should this be the flattened feature dimensions, i.e. 7x7x512 or just 512, is this the D from the paper?
max_samples: I can't find an explicit mention of this parameter anywhere apart from this code.
cluster_size: number of clusters (K in the original paper)
output dimensions: presumably this is KxD?

Some clarification of this would really be appreciated

`the process of intra normalization in NetFV

    fv2 = tf.reshape(fv2,[-1,self.cluster_size*self.feature_size])      
    fv2 = tf.nn.l2_normalize(fv2,1)
    fv2 = tf.reshape(fv2,[-1,self.cluster_size*self.feature_size])
    fv2 = tf.nn.l2_normalize(fv2,1)`

should remove the first reshape?

Hello. This is a very useful project. Do you mind if I rewrite the code and adapt LOUPE to keras version instead?

The error raised when I run this code:
'''
import loupe as lp
import tensorflow as tf
x = tf.placeholder("float", [None,1000,2048])
NetVLAD = lp.NetVLAD(feature_size=2048, max_samples=1000, cluster_size=64,
output_dim=2048, gating=True, add_batch_norm=True,
is_training=True)
NetVLAD.forward(x)
'''
I think this x.shape is #batch_size dot #max_sample dot #feature_size. Should I change line 126 in loupe.py into
'''
cluster_weights = tf.get_variable("cluster_weights",
[1, self.feature_size, self.cluster_size],
initializer = tf.random_normal_initializer(
stddev=1 / math.sqrt(self.feature_size)))
''' ?
But it also lead to other error, can you help me?
Thank you very much!