Dali backend support feature to handle init function about dali_backend HOT 4 OPEN

cc @szalpal @jantonguirao

from dali_backend.

Hi @farzanehnakhaee70,

Could you tell us more about what do you mean by:

Currently dali_backend just works with post and preprocessing for the functions which do not require initialization

In the case of DALI if you load the processing model all the necessary operators' initialization happens. In most cases, it is just the memory allocation. Do you have any particular operation in mind which may require extended initialization?

from dali_backend.

Hi @JanuszL ,
Thanks a lot for your answer.
The problem for my side is I'm wondering if I can use dali backend for transformer like models and import the tokenizer in the initialization part and afterward use the initialized tokenizer during the processing (as it takes time for initializing the tokenizers during each execution). Is it possible?

from dali_backend.

Hi @farzanehnakhaee70,

DALI is initialized once when the model is loaded. Then when the inference request is processed the pipeline is just run,
However I'm not sure if DALI is capable of tokenization. You may consider writing a custom plugin for DALI https://github.com/triton-inference-server/dali_backend/blob/main/docs/examples/dali_plugin/README.md.

from dali_backend.