Details about giqa HOT 12 CLOSED

1. path/to/activation means you need to give a path that denotes the activation file.
2. It's correct to download the InceptionV3 model used in FID, because we also use it to extract features.
3. For testing, the path/to/dataset denotes the test data folder. (Since you have already built the model.)

from giqa.

Hello, I used your method to operate the vg dataset, because the image size is different,l adjusted the image to 500 x400 size and got the relevant knn data.
When I choose pca=0.95 and activate operation, its output value is 825, when pca=0.9, its output value is 525, when pca=0.85, the output value is 386. What does this value represent?
When I run knn_score.py measurement gan generate images,pca=0.95 & k= 7000,the score of sample_2557.jpg is:23.120098128922876;pca=0.90 & k= 7000,the score of sample_2557.jpg is:28.22968146838341; pca=0.85 & k=7000,the score of sample_2557.jpg is:33.52534136281287.And when k=3500, score wasn't as high as k=7000,score distribution around 12.
What does this score means? I don't understand it.How did you get your table9? And how can I evaluate my data with your method?
Besides, my behaviors are
“python write_act.py VGomg --act_path activation/vg54.pkl --pca_rate 0.95 --pca_path pca/ pca95-vg54.pkl --gpu 1”
“python knn_score.py data/lostvg --act_path activation/vg54.pkl --pca_path pca/pca95-vg54.pkl --K 3500 --output_file out/knn98/35005495vg.txt --gpu 1”
I will continue to operate your gmm experiment, thank you very much for your help, thank you

from giqa.

1. the output value denotes the remaining dimension from PCA, please refer to our paper for more details.
2. the score is a “relative” score, compare the number among different models is meaningless, the correct way to use the score is to compare two image's scores from the same model(KNN or GMM), and the higher score denotes the image has better quality.
3. For table9 in our paper, first these are evaluated on our LGIQA dataset, you may download it from "readme". LGIQA dataset contains pairs of images which has annotated which has the higher quality, you can compare it with the score you get, than you may get the accuracy. I really suggest you read our paper more carefully, especially Sec3.2 and Sec5.0, it may improve your understanding of our work.
4. Besides, I think VisualGenorm dataset is kind of complicated, I haven't tried on this dataset, I think for this kind of complicated dataset, it's better to use all the images to get the activation, for KNN_score, I am not sure what's the best K number, a bigger K cost more time than small K, it's worth trying a smaller K, besides, it's also worth trying the GMM_score, it needs an additional time for building GMM, but the inference is much faster compared with KNN.

from giqa.

@cientgu Hello, I used my own data and the GMM model, but got a very large negative score value like this, is this normal?

from giqa.

@cientgu Hello, I used my own data and the GMM model, but got a very large negative score value like this, is this normal?

Only analysis of the value of the score is meaningless, but you can use the score to compare which is better and which is worse on the same dataset, I think the number is normal. Notice one more thing, there should be enough data in your own dataset, otherwise, it may be inaccurate.

from giqa.

@cientgu Hello, I used my own data and the GMM model, but got a very large negative score value like this, is this normal?

Only analysis of the value of the score is meaningless, but you can use the score to compare which is better and which is worse on the same dataset, I think the number is normal. Notice one more thing, there should be enough data in your own dataset, otherwise, it may be inaccurate.

Thank you for your reply, just to confirm one more thing, when the score is negative, the image quality with a score of -8202388 is higher than that with a score of -8929917, right?
In other words, I should not compare the absolute value but the real value of the score, right?

from giqa.

Hello, I am sorry to bother you again could you provide your evaluation database to me ? I downloaded your data set through the link, but it doesn't give a corresponding score.
I used my own datasets to train the GMM model to score, but the score was too low, I wanted to test whether the model I got by training my own data set was consistent with your manual evaluation criteria. To see if there is a problem with my own model.
Thank you for your help.
This is the fraction of the partial vg dataset I got, where the m component is 50:
score of sample_2557.jpg is:
-331.1316756187852
score of sample_2283.jpg is:
-633.6856968862959
score of sample_2106.jpg is:
-274.1916243507116
score of sample_540.jpg is:
-333.962998405286
score of sample_66.jpg is:
-334.99846765170224
score of sample_434.jpg is:
-538.2433860777709
score of sample_2387.jpg is:
-509.22751200925626
score of sample_3122.jpg is:
-717.7854063733386
score of sample_710.jpg is:
-518.359456909934
score of sample_1136.jpg is:
-843.3530016434424
score of sample_1158.jpg is:
-533.8602747861478
score of sample_662.jpg is:
-651.9051215940399
score of sample_697.jpg is:
-796.5913233245227
score of sample_3185.jpg is:
-651.4458333069384
score of sample_748.jpg is:
-483.165836370943
score of sample_751.jpg is:
-443.9178540298226
what's more，I get the mu and sigma are both 0， so ,I feel like I might be wrong

from giqa.

@cientgu Hello, I used my own data and the GMM model, but got a very large negative score value like this, is this normal?

Only analysis of the value of the score is meaningless, but you can use the score to compare which is better and which is worse on the same dataset, I think the number is normal. Notice one more thing, there should be enough data in your own dataset, otherwise, it may be inaccurate.

Thank you for your reply, just to confirm one more thing, when the score is negative, the image quality with a score of -8202388 is higher than that with a score of -8929917, right?
In other words, I should not compare the absolute value but the real value of the score, right?

Yes, you are right.

from giqa.

Hello, I am sorry to bother you again could you provide your evaluation database to me ? I downloaded your data set through the link, but it doesn't give a corresponding score.
I used my own datasets to train the GMM model to score, but the score was too low, I wanted to test whether the model I got by training my own data set was consistent with your manual evaluation criteria. To see if there is a problem with my own model.
Thank you for your help.
This is the fraction of the partial vg dataset I got, where the m component is 50:
score of sample_2557.jpg is:
-331.1316756187852
score of sample_2283.jpg is:
-633.6856968862959
score of sample_2106.jpg is:
-274.1916243507116
score of sample_540.jpg is:
-333.962998405286
score of sample_66.jpg is:
-334.99846765170224
score of sample_434.jpg is:
-538.2433860777709
score of sample_2387.jpg is:
-509.22751200925626
score of sample_3122.jpg is:
-717.7854063733386
score of sample_710.jpg is:
-518.359456909934
score of sample_1136.jpg is:
-843.3530016434424
score of sample_1158.jpg is:
-533.8602747861478
score of sample_662.jpg is:
-651.9051215940399
score of sample_697.jpg is:
-796.5913233245227
score of sample_3185.jpg is:
-651.4458333069384
score of sample_748.jpg is:
-483.165836370943
score of sample_751.jpg is:
-443.9178540298226
what's more，I get the mu and sigma are both 0， so ,I feel like I might be wrong

I will update it in the next two days.

from giqa.

Hello, I am sorry to bother you again could you provide your evaluation database to me ? I downloaded your data set through the link, but it doesn't give a corresponding score.
I used my own datasets to train the GMM model to score, but the score was too low, I wanted to test whether the model I got by training my own data set was consistent with your manual evaluation criteria. To see if there is a problem with my own model.
Thank you for your help.
This is the fraction of the partial vg dataset I got, where the m component is 50:
score of sample_2557.jpg is:
-331.1316756187852
score of sample_2283.jpg is:
-633.6856968862959
score of sample_2106.jpg is:
-274.1916243507116
score of sample_540.jpg is:
-333.962998405286
score of sample_66.jpg is:
-334.99846765170224
score of sample_434.jpg is:
-538.2433860777709
score of sample_2387.jpg is:
-509.22751200925626
score of sample_3122.jpg is:
-717.7854063733386
score of sample_710.jpg is:
-518.359456909934
score of sample_1136.jpg is:
-843.3530016434424
score of sample_1158.jpg is:
-533.8602747861478
score of sample_662.jpg is:
-651.9051215940399
score of sample_697.jpg is:
-796.5913233245227
score of sample_3185.jpg is:
-651.4458333069384
score of sample_748.jpg is:
-483.165836370943
score of sample_751.jpg is:
-443.9178540298226
what's more，I get the mu and sigma are both 0， so ,I feel like I might be wrong

I have updated some test images and results, please refer to readme.

from giqa.

Hello, I input the testset , run get_gmm, get the mu and sigma in the “calculate_activation_statistics “are both 0, such as : mu：[0.0.0....0.0.0.]. What’s the reason?
Is this the mean and sigma of my test image orthe training set? I'm confused about that. Thank you very much

from giqa.

Hello, I input the testset , run get_gmm, get the mu and sigma in the “calculate_activation_statistics “are both 0, such as : mu：[0.0.0....0.0.0.]. What’s the reason?
Is this the mean and sigma of my test image orthe training set? I'm confused about that. Thank you very much

The test set only contains 50 test images (not the training dataset!). Use it to test the gmm_score.py file with the pre-built GMM model on the LSUN-cat dataset, which I have put here(https://drive.google.com/drive/folders/17fAzhyQGXwgSJYO1PhmbnSl72FAE4VCJ). We did not provide code to test "extract features" or "get gmm". If you have more questions, please email me at [email protected].

from giqa.