Channel/Spatial/Element wise Attention Modules about keras-cv HOT 7 CLOSED

II think that in this specific field you need to always find a balance between popupalirity, state of the art, sedimentation and the available human and computing resources you have at a specific point on the time dimension in your community.
So you need to define a policy with an equilibrium over all these tensions expecially if you want try to growth your code contributor community and not just your user base.

E.g. we are still investing resoruces for resnext-rs and waiting for the citation/popularity threshold for the next STOA:
https://arxiv.org/abs/2201.03545

Often an high popupalirity component/model is needed to be maintained just cause it is a reccurent baseline for new accademic work.

Another addition dimension/tension Is the computing resources required by a model and its own components.

from keras-cv.

I have not extensively checked about these in the model gardens components, but just picking the First One in the list:

https://github.com/tensorflow/models/blob/master/official/vision/beta/projects/yolo/modeling/layers/nn_blocks.py#L1170

from keras-cv.

Currently we are prioritizing components that are required to achieve state of the art results on specific tasks. I.e., imagenet1k classification, COCO object detection, etc. Any chance I could get some guidance as to where these components excel?

Thanks

from keras-cv.

@LukeWood
The above list isn't meant to be the replacement for the current priority list. The point is whether these cv components are fit in keras-cv or not.

KerasCV is a repository of modular building blocks (layers, ... .

It can be considered as a to-do list, an interested contributor can get references from here in the coming days. Let me know if I miss something. Also, I think a few components should be already on the current priority list, for example, Squeeze-and-Excitation or CBAM.

from keras-cv.

@innat @LukeWood
I have a two part question about this?

1. Are we adding modules like Spatial Attention Module and Channel Attention Module or should we be adding the whole model (example CBAM as whole) as an application with parameters to change configuration of the model?
2. As @bhack pointed out we cannot solely depend on state of the art and we have to take into account the popularity and its "promosing"prospects. Soif you could throw some light on that area too.

Could you post these separately so people could be assigned if interested and maybe some of the model could be prioritized.

from keras-cv.

Ah, also can SWIN transformer be added to the list?

from keras-cv.

@old-school-kid
about (1). I would prefer these as modules. For example, if I want to experiment with my custom model and channel-wise attention module I should do that with the API call.
about (2). I agree with those points. But it's not clear to me how to define promising things here. And what role tfa.layers can serve here.

I didn't make a separate post regarding the above modules because I was not sure (and still) whether it's welcomed or we may need some discussion before approaching. The discussion thread wasn't created when I posted it. Maybe that's the right place. But I think, if it's welcomed, the interested contributor may pick up their interest from the above list and send PR.

For Swin-Transformer, it's been asked, #15545

from keras-cv.