Deprecate global parameters? about luigi HOT 6 CLOSED

I don't agree that we should remove them. We use them heavily for some things (switching to test mode for a general base task, changing fair scheduler pools etc.). However I think it makes sense to differentiate the interface for how to declare them vs how to declare "local" parameters (I don't like is_global=True) . Being "global" I think it makes more sense to have something like this in the global scope:

luigi.register_global('pool', IntegerParameter(default=5))

class MyTask(Task):
    def run(self):
         pool = luigi.global('pool')
         ...

This will clean up a lot of the global vs. local parameter code used within tasks and make it clearer what a task parameter and a global is. Not sure about providing the type as a parameter in register_global() above, but I think it can be convenient from a command line parsing perspective.

What do you think?

from luigi.

I think that makes sense, a lot of the problems are because they are defined in the task scope.

what about

luigi.global.pool = IntegerParameter(default=5)

maybe weird to modify internals like that

from luigi.

+1 to deprecating. We only use them in a few places that could easily be changed. We tend to use the Luigi configuration for globally scoped configurations.

from luigi.

I have to give -1, at least to both of the suggested replacements as they stand.
I usually do something like this:

some_global_param = luigi.Parameter(..., default=some_default)

class SomeTask(..):
  param = some_global_param
  other_param_here

class SomeOtherTask(..):
  params_here

Registering a global param strikes me as distasteful because it's modifying internal luigi state, and it seems to violate a bit of the sandboxing principle.

Perhaps defining the use-cases might be worthwhile?

from luigi.

Can anyone remind me how create global parameter using client.cfg?

from luigi.

Parameter system is completely rewritten and there are no globals anymore. Closing this. See docs for usage of the new system.

from luigi.