Unable to run purge with Celery release candidate 5.3.0rc1 about pyramid_celery HOT 8 CLOSED

Note the error You must provide the paste --ini argument is coming from pyramid-celery here

while the got an unexpected keyword argument 'ini' is coming from Celery

from pyramid_celery.

I'm wondering if a possible path forward is instead of using a preload options, which seem to add these options to all commands,

celery_app.user_options['preload'].add()

e.g.,

you could instead specify which particular commands should use your ini and ini_var options as mentioned in https://github.com/celery/celery/blob/e7b47a62d789557cf18ed0e56e2dfb99a51a62f7/docs/userguide/extending.rst#adding-new-command-line-options

        ini_option = Option(
            ('--ini', '-i',),
            help='Paste ini configuration file.'
        )
        ini_var_option = Option(
            ('--ini-var',),
            help='Comma separated list of key=value to pass to ini.'
        )
        # TODO: Add "shell" to list if Celery developers will add the "allow_extra_args" 
        # context flag to shell command  
        # https://github.com/celery/celery/blob/f3a2cf45a69b443cac6c79a5c85583c8bd91b0a3/celery/bin/shell.py#L82
        for option in ['worker','beat']:
            celery_app.user_options[option].add(
                ini_option
            )
            celery_app.user_options[option].add(
                ini_var_option
            )

This approach does seem to be working fine for me because Celery's Beat and Worker command already specify

context_settings={
    'allow_extra_args': True
}

See for example: https://github.com/celery/celery/blob/f3a2cf45a69b443cac6c79a5c85583c8bd91b0a3/celery/bin/beat.py#L11

I left a TODO above because, currently this isn't working for the shell command because Celery does not (yet) specify allow_extra_args in its context settings. See e.g., https://github.com/celery/celery/blob/f3a2cf45a69b443cac6c79a5c85583c8bd91b0a3/celery/bin/shell.py#L82

I've verified this solution works for worker and beat commands already and the Celery docs suggest it will work for events command too. If a feature request were made to Celery for the change to shell command to add the allow_extra_args, do you think this could provide a path forward?

from pyramid_celery.

Also, correct me if I am wrong, while shell does need ini interaction, to get all the registered tasks, it doesn't seem like purge functionality would require ini interaction.

from pyramid_celery.

I've created a feature request ticket in Celery celery/celery#8259

from pyramid_celery.

@sontek, can you comment on the proposed approach?

from pyramid_celery.

The Celery project has now officially release their 5.3.1 https://pypi.org/project/celery/5.3.1

This inability to configure and run purge or shell commands with pyramid_celery persists with Celery v5.3.1

from pyramid_celery.

@sontek my pull request to Celery has been merged bringing back preload options support for purge and shell. See celery/celery#8374

I believe no code changes will be needed in pyramid-celery. We still need to confirm once a new release of Celery is available.

from pyramid_celery.

Purge, shell, inspect etc have been restored to accepting ini options from Pyramid-Celery as a result of resolution of celery/celery#8374

from pyramid_celery.