about label/score normalization about hyperiqa HOT 5 CLOSED

also, i find that the target network contains 5 fc layers in the code, while 4 fc layers claimed in the paper.

from hyperiqa.

Actually, we didn't do this normalization in cross database evaluation. This is because the final criterion SRCC calculates only the correlation ratio between two vectors and thus has no relationship with vector scale.
However, it seems to be true that normalizing labels to the same scale in single database evaluation is more reasonable, since we use a fixed lr for training all databases, while it might be better to use different lr for training datasets with different label ranges.

from hyperiqa.

Thanks for your kind remind, there indeed exists a little difference about fc numbers from the paper and our code. However, it seems using whether 4 or 5 fc layers doesn't affect the model performance much, this is probably due to the target net has already learned sufficient quality representation in several front layers. You can also change the fc numbers by your own to see if the performance changes accordingly.

from hyperiqa.

Thanks for your reply.
i think it's important to normalize label to get consistent label distribution in cross database evaluation. as for using different lr for training different database, it does not matter because the gradient from L1 loss has nothing to do with the scale.
just a restatement of my ideas, and there's no need to answer. thanks for sharing such a great work.

from hyperiqa.

Our pleasure ; )

from hyperiqa.