how to detect novel class with AE-WTN? about ae-wtn HOT 4 CLOSED

Hi @XiaoJiNu, thanks for your interest. The novel classes are detected using the classification weights transformed from large-scale classification network (this was trained on novel classes but not the detection network) using AE-WTN. The box regression part is class-agnostic, meaning that it only outputs 1x4 regression values to be used for all classes including the novel ones.

from ae-wtn.

@xternalz thanks for your reply, I understood.

from ae-wtn.

Hi, I'm trying to use your work for my custom dataset. While using your model I wonder is box regression is truly trained to class agnostic way to the whole object. Isn't it more precise to say box regression is trained to be class agnostic to the training dataset? I'm quite confused about how regression can be trained class agnostic to the novel dataset while didn't seen at training time.

from ae-wtn.

When a detector is trained to perform box regression for many (seen) object categories in a class agnostic fashion, its box regression network can somehow generalize to unseen object categories in another dataset. This is related to learning generic objectness. Several works have also shown that class-agnostic box regression generalizes well to unseen object categories or object categories which do not have box annotations:

• What leads to generalization of object proposals?. ArXiv 2020.
• R-FCN-3000 at 30FPS: Decoupling Detection and Classification. CVPR 2018.

We never claimed anything about the box regression being class-agnostic to novel categories and/or datasets.

from ae-wtn.