runtime parameters like resize shape from config.pbtxt about dali_backend HOT 3 CLOSED

Hi @dumbPy !

Apologies for the late response.

I envision 2 scenarios you can follow:

1. You want to have a single model.dali (i.e. serialized DALI pipeline) model for all of your YOLO models. In this scenario, since the DALI pipeline is already serialized, your best chance would be to use fn.external_source and feed the model with the input. Unfortunately, Triton does not make it easy to pass constants specified in config.pbtxt. However, you could work this around. If you set up a Python Backend model before the DALI model, Python model might have knowledge about the shape of the YOLO model. This way, Python model can take the user input and pass it (at the minimal cost) to DALI together with the resize parameter.
2. The other option is to keep unserialized DALI model and then deploy it. Obviously, it's not something that can be always convenient. However, if you have a deployment step before running the server and choosing the YOLO input shape, you can parameterize DALI pipeline serialization:

@pipeline_def
def pipe(rx, ry):
    inp = fn.external_source(...)
    resized = fn.resize(inp, resize_x=rx, resize_y=ry, ...)
    return resized

[...]
def serialize():
    args = args.parse_args()  # argparse in one way to pass the cmdline parameters...
    p = pipe(rx=args.rx, ry=args.ry, ...)
    p.serialize(...)

I hope this is going to help.

from dali_backend.

Hi @szalpal
thank you, and sorry for my late reply 😅

Yes, Solution 2 is what I ended up going with since it's doesn't take much time to compile, while deploying a model.
Thanks a lot..

from dali_backend.

could you elaborate on option 2 please? do we serialize DALI pipeline on-the-fly in the triton container?

from dali_backend.