ko1nksm / shdotenv Goto Github PK

Describe the bug

By default shdotenv skip current defined variable names.
It's a right logic to prefer inherited environment.
But in group of values taken from file it prefer first value. Should be the last.
When a developer overrides values ​​in an .env file, he believes that the lower ones override the higher ones.

To Reproduce

Steps to reproduce the behavior:

# make file
$ cat <<EOF > example.env 
SSH_HOST=oldest
SSH_HOST=old
SSH_HOST=actual
EOF

case 1: use it like settings reader

# good
$ SSH_HOST=overrided ./shdotenv --noexport -e example.env

# bad (undefined behavior)
$ SSH_HOST=overrided ./shdotenv --noexport -e example.env --overload
SSH_HOST='oldest'
SSH_HOST='old'
SSH_HOST='actual'

# bad (unexpected value)
$ ./shdotenv --noexport -e example.env
SSH_HOST='oldest'

case 2: use it like runner

# good
$ SSH_HOST=overrided ./shdotenv -e example.env -- sh -c 'echo $SSH_HOST'
overrided

# bad
$ ./shdotenv -e example.env -- sh -c 'echo $SSH_HOST'
oldest

ok, may be --overload is solution? No it only revert results

# bad (--overload works like --ignore-environment, but it not set here)
$ SSH_HOST=overrided ./shdotenv -e example.env --overload -- sh -c 'echo $SSH_HOST'
actual

# good
$ ./shdotenv -e example.env --overload -- sh -c 'echo $SSH_HOST'
actual

No way to make universal usage for select 'overrided' when override and 'actual' otherwise.

Expected behavior

#  --overload enabled by default, no ignore environment
$ SSH_HOST=overrided ./shdotenv --noexport -e example.env

#  --overload no ignore environment
$ SSH_HOST=overrided ./shdotenv --noexport -e example.env --overload

#  --overload enabled by default, print last value
$ SSH_HOST=overrided ./shdotenv --noexport -e example.env --ignore-environment
SSH_HOST='actual'

#  --overload enabled by default, print last value
$ ./shdotenv --noexport -e example.env
SSH_HOST='actual'

$ ./shdotenv -e example.env -- sh -c 'echo $SSH_HOST'
actual

$ SSH_HOST=overrided ./shdotenv -e example.env  --overload -- sh -c 'echo $SSH_HOST'
overrided

$ SSH_HOST=overrided ./shdotenv -e example.env --ignore-environment -- sh -c 'echo $SSH_HOST'
actual

# new mode for print all
$ ./shdotenv --noexport -e example.env --no-overload
SSH_HOST='oldest'
SSH_HOST='old'
SSH_HOST='actual'

Environment (please complete the following information):

• OS: Mac OS X 10.13.6
• GNU Awk 5.1.1, API: 3.1 (GNU MPFR 4.1.0, GNU MP 6.2.1)

Additional context

• fix --overload to overload only env file values, not parent env. The --ignore-environment is works fine
• enable --overload by default. because it confusing developers.
• add option to disable overload feature. like --no-overload or --overload no or --all to print all

Rename the option name to allow support for formats other than shell. Also rename the SHDOTENV_SHELL environment variable to the SHDOTENV_FORMAT.

Is your feature request related to a problem? Please describe.

# test.env
KEY1=value1
KEY2=value2

$ ./shdotenv --noexport --env test.env
KEY1='value1'
KEY2='value2'

When KEY1 is already defined in my shell, it's removed from the output:

$ KEY1='blaaat' ./shdotenv --noexport --env test.env
KEY2='value2'

I don't want that. I want the result always to have both keys.

Describe the solution you'd like
I want to cascade multiple dotenv files into one file but without using the env vars in my shell.

Describe alternatives you've considered

I can use env -i shdotenv ... but not sure if that is expected.

Additional context

I would like to store binary strings like this: https://stackoverflow.com/a/46458707/1049833 - but it doesn't seem to be supported by shdotenv because "using without quotes is not allowed".

Is there any way around this?

We plan to remove options that are not used interactively in v1.0.0.

To be removed

• -s, --shell (Deprecated)
• -o, -n, -g.
• -k, --keyonly (Rename to --name-only)

Currently

Usage: shdotenv [OPTION]... [--] [[COMMAND | export] [ARG]...]

  If the COMMAND is specified, it will load .env files and run the command.
  If the COMMAND is omitted, it will output the result of interpreting .env
  files. It can be safely loaded into the shell (For example, using eval).

Options:
  -d, --dialect DIALECT     Specify the .env dialect [default: posix]
                                posix, ruby, node, python,
                                php, go, rust, docker
  -f, --format FORMAT       Output in the specified format [default: sh]
                                sh, fish
  -e, --env ENV_PATH        Location of the .env file [default: .env]
                              Multiple -e options are allowed
                              If the ENV_PATH is "-", read from stdin
  -i, --ignore-environment  Ignore the current environment variables
      --overload            Overload predefined environment variables
      --noexport            Do not append "export" prefix
      --grep PATTERN        Output only names that match the regexp pattern
      --name-only           Output only environment variable names
  -q, --quiet               Suppress all output (useful for test .env files)
  -v, --version             Show the version and exit
  -h, --help                Show this message and exit

  Deprecated: (to be removed in the next version)
  -s, --shell SHELL         Use the -f (--format) option instead
  -k, --keyonly             Use the --name-only option instead
  -o, -n, -g                Use long options instead

Usage: shdotenv export [-n | -p] [--] [NAME]...
  Exports environment variables in posix-compliant .env format.

  -n  List only environment variable names
  -p  Append "export" prefix to environment variable names

  This will be output after the .env files is loaded. If you do not want
  to load it, specify "-e /dev/null". This is similar to "export", "env"
  and "printenv" commands, but quoting correctly and exports only portable
  environment variable name that are valid as identifier for posix shell.

Is your feature request related to a problem? Please describe.

I'm using shdotenv in a special Laradock project structure, where I merge Laradock's and my environment variables into one:

eval "$($DIR/shdotenv --dialect docker --overload --env laradock.env.example || echo "exit $?")"
set -a && . ./.env && set +a

However, shdotenv cannot expand the tilde character, if I put the value in quote or not, doesn't matter:

eval "$($DIR/shdotenv --dialect posix --overload --env .env || echo "exit $?")"

Output I'm getting:

APACHE_SSL_PATH=~/.laradock/ssl/apache2

Expected output:

APACHE_SSL_PATH=/home/user/.laradock/ssl/apache2

Describe the solution you'd like

It would be nice to have a flag, where it can expand $HOME shorthand (~) to absolute paths.

Describe alternatives you've considered

The only working solution is to use the good old bash env variable parsing trick:

set -a && . ./.env && set +a

But with this, I loose every control over the file and I have to format it very carefully. It works with my files, but not with the ones from various repositories.

Additional context

None.

Hello! Great library!

I have a question, is there a support for variables that has special values like secret keys that don't have single qoutes?

Example:

GENERATED_SECRET_KEY=dAz31q82*E0d

Unless I understood the docs wrong, shdotenv export ignores -e /dev/null
Ex:

$ echo "VAR1=foo" > .env
$ shdotenv -e /dev/null -e .env export
HOME='/root'
PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
PWD='/tmp'
SHLVL='2'
TERM='xterm'
VAR1='foo'

Expected behavior

$ echo "VAR1=foo" > .env
$ shdotenv -e /dev/null -e .env export
VAR1='foo'

Environment:

• OS: linux (alpine:3.17 container)
• awk version: GNU awk 5.1.1

Additional context
I would like to use shdotenv to manipulate an env file ensuring that file is valid and merging files, for example:

echo "VAR1=foo" > .existing.env
MYOUTPUT=$(echo 'value that I have no idea on how to escape')
shdotenv export MYOUTPUT | shdotenv -e /dev/null -e existing.env -e - export

Of course I can do shdotenv export MYOUTPUT >> existing.env and rely that values would be overwritten due their order, but I would prefer to use shdotenv to keep the env file cleaner.

I recommend using # dotenv directive to specify dialect, but this may not be acceptable for your projects. Although it is possible to specify by the --dialect option, it is tedious to specify it every time, so I will add the ability to specify it by .shdotenv file.

Currently README instructs

$ wget https://github.com/ko1nksm/shdotenv/releases/latest/download/shdotenv -O $HOME/bin/shdotenv
$ chmod +x $HOME/bin/shdotenv

However, $HOME/bin is not a standard location - not available on Ubuntu Linux or macOS based on my test.

~/.local/bin enjoys at least some standard support.

I would propose changing README to recommend this best practice. I can update it with a PR unless there are some concerns around this.

Is your feature request related to a problem? Please describe.
I like the portability and functions of this project, but would like an option to export without quotes, without escaping anything. My current usecase is converting dotenv files to json.

With values in the format "var=value" i can easily do this:

echo -e "var1=val1\nvar2=val2" | jq -Rn '{Parameters:[(inputs | split("=")) | {(.[0]): (.[1:] | join("="))}] | add}'
{
  "Parameters": {
    "var1": "val1",
    "var2": "val2"
  }
}

Describe the solution you'd like

I would like a flag "--no-quotes" which outputs without quotes. Everything after the equals should be provided as is without any quoting.

Describe alternatives you've considered

I could use sed to replace the single quotes or do it in jq, but it gets ugly. I think it would be a good feature to output in a more standard dotenv format.

The python project outputs by default without quotes

$ ▶ dotenv list
myvar=myvalue
myconcatvar=concat/myvalue/woo

I am happy to work on a PR for this if you are agreeable.

Is your feature request related to a problem? Please describe.

Generating dotenv compatible file based on current environment variables.

Describe the solution you'd like

 # saves sorted list of env variable names/values to dotenv compatible format
shdotenv > .env

# accepts space separated key list as arguments
shdotenv KEY_1 KEY_2 > .env 

Describe alternatives you've considered

env | sort > .env

These alternative solutions are generally:

• not posix compliant
• not dotenv compliant, like missing quotes around special characters
• not simply abstracted
• have bugs with special characters

Additional context

I am initializing my environment based on a shell script that I source, that takes various inputs. After I initialize my environment I would like to export it in a way that other programs like IDEs can references those same values.

https://stackoverflow.com/questions/60756020/print-environment-variables-sorted-by-name-including-variables-with-newlines