Questions about comprehension of this paper about adaptive-compositional-modules HOT 2 CLOSED

Hi, thanks for your interest.

For (1)(2), please refer to section 4.1
In short, we are using learnable weight coefficients lambda to decide where to add the adapter/whether we need to add an adapter. c is the initialization value of lambda, as a hyperparameter to tune.

lambda is changed/learned during the training process, and yeah, it is the weight for its corresponding module. After training, in each transformer layer, we select the module with the biggest weight. (i) If the selected one is an old module, the new task will share this module with the corresponding old tasks. (ii) If the selected one is a new module, we add a new module in this layer for the new task.

For (3), adapters for the old task are fixed during the decision stage.
During the training stage, if some of the old tasks are replayed (see the last paragraph of section 4.2 for details), (i) For the setting of similar tasks, we updated all adapter modules of the replayed tasks. (ii) For the setting of dissimilar tasks, we updated the adapter modules of the replayed tasks that are shared with the current task.
(In the code, this is related to the option "--partial transfer")

(4) Yes. And we also assume we know the identifiers for all adapter baselines.

from adaptive-compositional-modules.

Thank you for your clear reply. I have understood !

from adaptive-compositional-modules.