[BUG] Rogue string expansion about merlin HOT 13 CLOSED

I see, good point. It's an easy fix, I'll run some tests to make sure what we think is happening is happening.

from merlin.

To be clear, you should be able to use env variables in all sections except the steps. These are definitely used in the batch section. I mean they should all be replaced at run , except for the steps.

from merlin.

Well the point is to replace them immediately, like paths to flux in the batch section. There are not part of a step so they won't be replaced by the env when they are run. Also in variables in the env section, those should be replaced at run, if a user wants the variable to remain unchanged , it should be in a step.

from merlin.

Ah ok, I think leaving them alone (and exposed) at run time will actually fix another (related) bug/feature. I was having issues using ${SYS_TYPE} in the batch section for flux w/ run-workers and was having a hard time tracking it down. Not sure yet if it's related, but there was some odd behavior

from merlin.

I think we need to remove the expanduser and expandvars references by Merlin outside of the workers

I propose we leave user and environment variables alone and let the workers take care of them as they would at run time

from merlin.

@lucpeterson okay, so even symbols like ~ won't work then

from merlin.

we should check whether having "~" in a spec file would work or not

from merlin.

i bring this up to address Timo's desire to use environment variables in spec descriptions for extra flexibility

as it stands now, environment variables are expanded when "merlin run" is executed and filled out in the spec, and therefore hard-coded into the step. this prevents using things like ${SYS_TYPE} within steps

from merlin.

Ohhhh I see the problem. Has nothing to do with ~. We should leave env vars alone, to instead be interpreted by each step's shell and not meddled with by merlin.

from merlin.

#247

from merlin.

Do we have to explicitly replace them or can they be replaced by the native shell? Can we center around an example?

from merlin.

I think this is semantics, by replaced I mean made available at run with the run env variables. So yes, as long as they are in the shell when run it is fine.

from merlin.

Fixed with #247

from merlin.