Describe the bug
When you want to pass module parameter that is not in the format key=value, but is a flag, such as --no-cloudwatch-metrics in this module, you will see an error like this:

2024-04-11 11:38:01 [INFO] [gruntwork-install] Executing /tmp/gruntwork-script-modules/fail2ban/install.sh --no-cloudwatch-metrics no-cloudwatch-metrics
2024-04-11 11:38:01 [ERROR] [configure-fail2ban] Unrecognized argument: no-cloudwatch-metrics

The script is not prepared to accept a parameter other than one that has the format key=value. Look here.

To Reproduce

gruntwork-install --module-name 'fail2ban' --module-param 'no-cloudwatch-metrics' --repo https://github.com/gruntwork-io/module-security --tag 'v0.65.5'

Expected behavior
It should work fine with all existing parameters.

Describe the solution you'd like
The bootstrap script is hardcoded to install binaries into /usr/local/bin, and to download user-data into /etc/user-data. This prevents running this script without root permissions, even though none of these artifacts need to be installed system-wide.

Describe alternatives you've considered

• Manually install the prerequisites (fetch and gruntwork-installer), but this is error-prone.
• Modify the bootstrap script to use alternate directories (I have done this, PR incoming).

Additional context
This is a minor change that can be accomplished by overriding the default values for two environment variables. It may not be widely used, and would be easily discovered by advanced users upon inspection of the script, so it does not need associated CLI args or even documentation.

A user provided this error log to illustrate that they couldn't use the installer to install openvpn-admin on a VM.

gruntwork-install --binary-name openvpn-admin --repo https://github.com/gruntwork-io/terraform-aws-openvpn --tag v0.5.4

Error:

2021-07-02 12:08:08 [INFO] [gruntwork-install] Installing openvpn-admin...

2021-07-02 12:08:09 [INFO] [gruntwork-install] Moving /tmp/gruntwork-script-modules/openvpn-admin_linux_amd64 to /usr/local/bin/openvpn-admin and setting execute permissions

mv: cannot stat '/tmp/gruntwork-script-modules/openvpn-admin_linux_amd64': No such file or directory

We suspect that this has to do with the downloaded file being in the wrong place (i.e., the native system instead of the VM), or something like that. Is this a case we should support?

Workaround: Allow the script to complete and error out. Manually scp or otherwise locate the file and move it to the right place, adding execute permissions.

bootstrap-gruntwork-installer.sh installs fetch into /usr/local/bin. Depending on the OS and the user (e.g., root), this directory is not always on the PATH. As a result, the call to fetch in gruntwork-install may fail.

We should either (1) call fetch from an absolute path in gruntwork-install or (2) install fetch into something more likely to be on the PATH, such as /usr/bin.

As I understand, to upgrade a tool we need to re-run the same command than for the installation, replacing the version.

It will be good for the sake of simplicity and to avoid the rot of the tool versions, to have a way to upgrade easily the different tools.

It could be a command: gruntwork upgrade. It will require of course some metadata regarding the list of the tools installed.

Did you considered using existing package managers (brew, apt-get, ...) ? There are solving lots of problems than new coming packing managers will face sooner or later.

I understand than some of your tools are private. Brew seems to provide a way to support this.

This issue is not valid - gruntworks modules are published via GitHub releases, not packages.

Hello 👋

My name is Peter and I work at FreeAgent. We use Gruntworks extensively - thanks for your great work, it helps us a lot! I've got one quick request to raise however:

Currently the gruntwork-installer requires a GITHUB_OAUTH_TOKEN to be able to access the gruntworks modules as they live in private GitHub repositories.

This GITHUB_OAUTH_TOKEN currently needs to be a "Private Access Token" (link) with repo access. This means that the token has full read & write access to all repositories that that user has access to.

Describe the solution you'd like
Following the principle of least privilege, would it be possible to alter the installer to only need a GITHUB_OAUTH_TOKEN with read:package access.

The binaries that are eventually installed (in the use cases I've come across) are published as packages to GitHub, but I think the gruntworks-installer fails because it queries the tags before installing (logs at the bottom).

This might require adding an optional parameter to manually specify the GitHub release?

Describe alternatives you've considered
None

Additional context
The motivation behind wanting not to have a repo scoped token in our pipeline is that we'd like to allow dependabot to create PR and run CI, but want to grant as few permissions to that workflow as possible to mitigate against a malicious package stealing tokens.

Failure log when using a GITHUB_OAUTH_TOKEN scoped to only package:read:

#8 0.830 2022-04-26 10:49:41 [INFO] [gruntwork-install] Installing gruntkms...
#8 1.048 [fetch] time="2022-04-26T10:49:41Z" level=error msg="\nReceived an HTTP 404 Response when attempting to query the repo for its tags.\n\nThis means that either no GitHub repo exists at the URL provided, or that you don't have permission to access it.\nIf the URL is correct, you may need to pass in a --github-oauth-token.\n\nUnderlying error message:\nReceived HTTP Response 404 while fetching releases for GitHub URL https://api.github.com/repos/gruntwork-io/gruntkms/tags?per_page=100. Full HTTP response: {\"message\":\"Not Found\",\"documentation_url\":\"https://docs.github.com/rest/reference/repos#list-repository-tags\"}\n\n"
#8 ERROR: process "/bin/sh -c curl -LSs https://raw.githubusercontent.com/gruntwork-io/gruntwork-installer/master/bootstrap-gruntwork-installer.sh | bash /dev/stdin --version \"$GRUNTWORK_INSTALLER_VERSION\" &&   gruntwork-install --repo \"[https://github.com/gruntwork-io/gruntkms\](https://github.com/gruntwork-io/gruntkms/)" --binary-name \"gruntkms\" --tag \"$GRUNTKMS_VERSION\"" did not complete successfully: exit code: 1

Behavior:

$ docker run --rm -it amazonlinux /bin/bash
bash-4.2# curl -Ls https://raw.githubusercontent.com/gruntwork-io/gruntwork-installer/master/bootstrap-gruntwork-installer.sh | bash /dev/stdin --version v0.0.22
Installing gruntwork-install...
Installing fetch version v0.3.2 to /usr/local/bin/fetch
Downloading https://github.com/gruntwork-io/fetch/releases/download/v0.3.2/fetch_linux_amd64 to /tmp/gruntwork-bootstrap-download-FsgPxG
Got expected status code 200
Moving /tmp/gruntwork-bootstrap-download-FsgPxG to /usr/local/bin/fetch
/dev/stdin: line 71: sudo: command not found
bash-4.2# sudo
bash: sudo: command not found

From line 71 of bootstrap-gruntwork-installer.sh:

sudo mv -f "$tmp_path" "$file"

Looks like a check needs to be made on all sudo commands to not use sudo on Amazon Linux 2.

A customer recently had an issue with curl failing to download fetch. The error is:

Step 10/24 : RUN curl -Ls https://raw.githubusercontent.com/gruntwork-io/gruntwork-installer/master/bootstrap-gruntwork-installer.sh | bash /dev/stdin --version v0.0.23
 ---> Running in b606ba274890
Installing gruntwork-install...
Installing fetch version v0.3.2 to /usr/local/bin/fetch
Downloading https://github.com/gruntwork-io/fetch/releases/download/v0.3.2/fetch_linux_amd64 to /tmp/gruntwork-bootstrap-download-JAaGkA
ERROR: Expected status code 200 but got 302 when downloading https://github.com/gruntwork-io/fetch/releases/download/v0.3.2/fetch_linux_amd64
The command '/bin/sh -c curl -Ls https://raw.githubusercontent.com/gruntwork-io/gruntwork-installer/master/bootstrap-gruntwork-installer.sh | bash /dev/stdin --version v0.0.23' returned a non-zero code: 1

Upon investigation, the error turned out to be a network monitoring proxy that was in the middle of the SSL connection causing curl to fail SSL validation. This was being swallowed silently.

Gruntwork installer needs to install an updated version of fetch. It is also possible that fetch will need to be patched after gruntwork install is installing the updated version. I am not sure if the most recent fetch release is patched.

Fetch details:

• CVE-2022-27664

Installed version: 1.16.2
Patched version: 1.18.6
Paths: /usr/local/bin/fetch

The current one line command for the installation is working (curl -Ls https://raw.githubusercontent.com/gruntwork-io/gruntwork-installer/master/bootstrap-gruntwork-installer.sh | bash /dev/stdin --version 0.0.11)

But if you want to install the latest version (currently 0.0.16), it is not working. It seems that, starting from the 0.0.14, the version start with the letter "v". The following command is working:
curl -Ls https://raw.githubusercontent.com/gruntwork-io/gruntwork-installer/master/bootstrap-gruntwork-installer.sh | bash /dev/stdin --version v0.0.16

Is it possible to install this on a machine without sudo? basically in a docker image (eg jenkins jnlp slave image). Related to #37

Describe the bug
When installing cloud-nuke, I specified a tag that didn't exist (I omitted v from v0.16.2. The messaging was not crystal clear and led me astray:

2022-08-01 09:44:53 [INFO] [gruntwork-install] Moving /tmp/gruntwork-script-modules/cloud-nuke_darwin_amd64 to /usr/local/bin/cloud-nuke and setting execute permissions

There shouldn't be any attempt to move a non-existent binary. Even better, we might be able to detect that there was no tag specified and message that to make it really clear.

To Reproduce

$ gruntwork-install --binary-name 'cloud-nuke' --repo 'https://github.com/gruntwork-io/cloud-nuke' --tag '0.16.2'
2022-08-01 09:44:52 [INFO] [gruntwork-install] Installing cloud-nuke...
2022-08-01 09:44:53 [INFO] [gruntwork-install] Moving /tmp/gruntwork-script-modules/cloud-nuke_darwin_amd64 to /usr/local/bin/cloud-nuke and setting execute permissions
mv: rename /tmp/gruntwork-script-modules/cloud-nuke_darwin_amd64 to /usr/local/bin/cloud-nuke: No such file or directory

Expected behavior
Tell the user that there is no such tag.
Check to see if the binary exists before attempting to move it and message accordingly.

Every now and then GitHub API fails with GitHub API error Failed to download file at the url https://api.github.com/repos/xxx/xxx/zipball/xxx. Received HTTP Response 502.. To make the installer more resilient to those transient errors, an optional retry should be added.

Because gruntwork-installer uses fetch, one option could be adding the retry to fetch instead.

When calling gruntwork-install in a Dockerfile and attempting to download a private repo, we currently echo this INFO level log line to STDOUT:

2020-10-06 23:48:56 [INFO] [gruntwork-install] Repository is not public. GITHUB_OAUTH_TOKEN environment variable is required.

In my case, given my setup, this log line was dark red - so it seemed like an error.

The first couple of times, I mistook this as meaning that gruntwork-install was not finding the GITHUB_OAUTH_TOKEN it was looking for, when in fact it's just confirming that because you're attempting to access a private repo, the token is in fact required.

Does anyone else find this potentially confusing? If so, perhaps we should modify the messaging to be more explicit - or consider instead stating that a valid-looking GITHUB_OAUTH_TOKEN was found, etc.

It seems like using ssh auth for git is much more scalable for teams, as you don't have to set an oauth token anywhere.

Hello.

I have a problem with running command:

    "type": "shell",
    "inline": [
      "curl -LsS  https://raw.githubusercontent.com/gruntwork-io/gruntwork-installer/master/bootstrap-gruntwork-installer.sh | bash /dev/stdin --version v0.0.35"
    ]},

Looks like it stucks, I see no output from curl, packer try run next commands and from next shell provisioner but fails because command gruntwork-install is not available.

Reading the logs it looks like this tool shells out to curl, which prints redirect URLs as it follows them.

Using curl --silent should prevent this, or using a tool like github-release download: https://github.com/aktau/github-release

The Gruntwork Installer is still using our old, incorrect bash pattern of declaring variables:

local readonly foo="bar"

Combining local and readonly this way doesn't actually work. We need to change to:

local -r foo="bar"

Or, in the case of arrays:

local -ar foo=("aaa" "bbb")

This is legacy code that never got used.

That said, we should standardize on a common install location for all Gruntwork utilities. Perhaps /opt/gruntwork/<repo>/<module>?

A typical gruntwork-installer configuration in circle.yml looks like this:

...
- curl -Ls https://raw.githubusercontent.com/gruntwork-io/gruntwork-installer/master/bootstrap-gruntwork-installer.sh | bash /dev/stdin --version "v0.0.14"
    - gruntwork-install --module-name "gruntwork-module-circleci-helpers" --repo "https://github.com/gruntwork-io/module-ci" --tag "v0.3.8"
    - gruntwork-install --binary-name "docs-generator" --repo "https://github.com/gruntwork-io/module-ci" --tag "v0.3.8"
...

test:
  override:
    - run-go-tests --path test

The question here is where did that run-go-tests function come from? it'd be nice to answer this systematically. For example, imagine this revised circle.yml:

...
- gruntwork-install --alias "cci" --module-name "gruntwork-module-circleci-helpers" --repo "https://github.com/gruntwork-io/module-ci" --tag "v0.3.8"
...
test:
  override:
    - cci.run-go-tests --path test

Now it's clear exactly where this came from.

For users browsing the OS, perhaps include a cci.help function that explains which gruntwork-installer command was used to install this.