minamijoyo / hcledit Goto Github PK

This example file:

$ cat /tmp/attr.hcl 

# This resource is pasted directly from the hcledit readme
resource "foo" "bar" {
  attr1 = "val1"
  nested {
    attr2 = "val2"
  }
}

# And I expect this module to work with nesting the same way as the above resource
module "foo" {
  nested = {
    attr2 = "val2"
  }
}

Retrieving a nested value from the resource works as expected:

$ cat /tmp/attr.hcl | hcledit attribute get resource.foo.bar.nested.attr2
"val2"

But retrieving a similary nested value from the module doesn't work:

$ cat /tmp/attr.hcl | hcledit attribute get module.foo.nested.attr2
[no output at all]

Note that I can get the nested map, I just can't seem to recurse inside it:

$ cat /tmp/attr.hcl | hcledit attribute get module.foo.nested
{
    attr2 = "val2"
  }

$ hcledit version
0.2.10
# On an Apple Silicon Macbook, if that matters

Similar to how JQ works having '*' or some alternative (maybe "--all"), to let you output the whole file with all of its resources and blocks.
The uses could be to check if a file contains no errors, or to use as a formatter for hcl config files.

Hi @minamijoyo,

Great work on this CLI tool for editing HCL2 files! 👍

I myself am also doing some work using the hclwrite package. However, I'm completely new to Go and I'm nowhere near your level looking at your code.

What I struggle to do/achieve, is something similar to your example:

$ cat tmp/block.hcl | hcledit block append resource.foo.bar block1.label1 --newline
resource "foo" "bar" {
  attr1 = "val1"

  block1 "label1" {
  }
}

resource "foo" "baz" {
  attr1 = "val2"
}

Where I can see that it's possible to specify nested blocks, to be able to append to their bodies.

I'm creating a "console" application, in which I need to be able to add blocks/set attributes (amongst other operations from the hclwrite package) within specific bodies (blocks). Do you have any guidance for me how to do this, more than looking at your code (which I've done but fail to understand exactly what I'd need to implement).

For example, I have the following function:

func AddHCL2Block(hcl2Configuration []byte, typeName string, labels []string) []byte {
	parsedConfig, diags := hclwrite.ParseConfig(hcl2Configuration, "HCL2configuration", hcl.Pos{Line: 1, Column: 1})
	if diags.HasErrors() {
		log.Fatalf("Errors: %s", diags)
	}

	parsedConfig.Body().AppendNewBlock(typeName, labels)

	return parsedConfig.Bytes()
}

This appends the block to the root body, but I'd like to add a parameter to the function which allows me to specify which body (block) I'd like to append the new block to, just as you can, by passing a properly formatted string to your hcledit block append command.

Your input would be very much appreciated, since I'm quite far into a project and have been struggling with this all along. Unfortunately I've not been able to solve this on my own, I've been struggling with this for weeks, I don't even now where to start. Now when I discovered your tool, I got my hopes up since there's someone that've actually done what I'm trying to do, so it for sure can't be impossible.

Crossing my fingers here. 🤞

Best regards

Given:

#main.tf
locals {
  a = "foo"
  b = "bar"
  c = {"nums":[1,2]}
}

It would be nice to be able to do something like:

hcledit -f main.tf attribute list locals

to get:

locals.a
locals.b
locals.c.nums[0]
locals.c.nums[1]

$ hcledit version
0.2.9

$ cat ~/.terraformrc

provider_installation {
    dev_overrides {
        "mycompany/provider" = "/Users/me/go/bin/"
    }

    filesystem_mirror {
        path = "./terraform.d/plugins"
        include = ["mycompany.com/*/*"]
    }

    direct {
        exclude = ["mycompany.com/*/*"]
    }
}

$ cat ~/.terraformrc | hcledit block get dev_overrides
failed to parse input: -:3,9-10: Invalid argument name; Argument names must not be quoted.

$ cat ~/.terraformrc | hcledit block list
failed to parse input: -:3,9-10: Invalid argument name; Argument names must not be quoted.

However, the official docs say to quote the attribute name.

https://developer.hashicorp.com/terraform/cli/config/config-file#development-overrides-for-provider-developers

It would be nice to have exit codes to the get commands based on the matching like grep does.

$ hcledit block get module.namespace -f main.tf
module "namespace" {
  source = "[email protected]:xxxxxxxxx/platform_terraform_modules.git//k8s/namespace?ref=7.15.2"
  project_name = var.project_name
  env          = var.environment
  region       = var.region
  owners       = var.owners
}
$ echo $?
0
$ hcledit block get module.namasdfasdfasdespace -f main.tf
$ echo $?                                                 
0

Thank you :)

I was hoping to use hcledit to extract and manipulate AWS provider attributes within our terraform config.

However, it looks like the lack of provider names or address indexing prevents this from working? I could also be missing something on my end, though.

The example file I'm working with is a main.tf file with 4 different AWS provider blocks. Each provider has a different profile specified, but I can only interact with the first of these from hcledit:

$ hcledit -f main.tf attribute get provider.aws.profile
"prod"

$ grep -A 1 aws main.tf
provider "aws" {
  profile = "prod"
--
provider "aws" {
  profile = "dev"
--
provider "aws" {
  profile = "qa"
--
provider "aws" {
  profile = "ops"

Can I add a non-existent block?

for example.

test.hcl

test { }
i would like to get below result

test { } test2{ }

Is it possible to publish a windows exe as part of the release artifacts?

I am having issues building the GO lib as my corporate laptop blocks the GO package manager downloads

Consider following input:

test.hcl:

b1 {
  b2 l1 {
  }
}

Get nested block - works as expected

$ cat test.hcl | hcledit block get b1.b2.l1
b2 l1 {
}

Removal of nested block

Expected output:

b1 {
}

Actual output:

b1 {
  b2 l1 {
  }
}

Issue is present for single and multiple nested blocks. Use Case: Removal of all terraform.before_hook.* and terraform.after_hook.* in a terragrunt configuration before running it in automation in order to prevent execution of scripts on CI/CD agent host.

I tried to add the findLongestMatchingBlocks to unformattedBlockRemoveFilter and got the matching block. However, I would need a hint on how to remove those longest matching blocks from the resulting file:

Looking to be able to enable support for auto-updating a user's ~/.terraformrc file, assuming it exists.

The official docs state:

As a convenience for provider development, Terraform supports a special additional block dev_overrides in provider_installation blocks. The contents of this block effectively override all of the other configured installation methods, so a block of this type must always appear first in the sequence: [code example]

I see that there is support for append, but what about adding support for prepend?

Given the following config:

a = {
  b = "c"
}

I'd like to add another key/value pair to the a attribute, so that it ends up looking like this:

a = {
  b = "c"
  d = "e"
}

The type of d could be arbitrary, so instead of "e", we could have, e.g. 1, true, or ["e", "f"].

Is there a way to achieve this with hcledit?

Because redirecting stdout directly to file that we want to modify will remove the content
cat versions.tf | hcledit attribute set terraform.required_version '"=> 0.12.26"' > versions.tf
👎

I'm forced to do some cp/mv tricks:
cat versions.tf | hcledit attribute set terraform.required_version '"=> 0.12.26"' > tmp.tf && mv tmp.tf versions.tf

Would love to see inplace feature like in sed:

 -i extension
        Edit files in-place, saving backups with the specified extension.  If a
        zero-length extension is given, no backup will be saved.  It is not rec-
        ommended to give a zero-length extension when in-place editing files, as
        you risk corruption or partial content in situations where disk space is
        exhausted, etc. 

When I ran the command below against the same file multiple times, this creates multiples blocks

hcledit -f elb.tf  -u block append resource.aws_lb.api-aws access_logs

elb.tf

resource "aws_lb" "api-aws" {
  enable_cross_zone_load_balancing = false
  internal                         = true
  load_balancer_type               = "network"
  name                             = "api-aws"
  subnet_mapping {
    subnet_id = data.aws_subnet.sa-east-1a.id
  }
  subnet_mapping {
    subnet_id = data.aws_subnet.sa-east-1b.id
  }
  subnet_mapping {
    subnet_id = data.aws_subnet.sa-east-1c.id
  }
}

#First interaction result

resource "aws_lb" "api-aws" {
  enable_cross_zone_load_balancing = false
  internal                         = true
  load_balancer_type               = "network"
  name                             = "api-aws"
  subnet_mapping {
    subnet_id = data.aws_subnet.sa-east-1a.id
  }
  subnet_mapping {
    subnet_id = data.aws_subnet.sa-east-1b.id
  }
  subnet_mapping {
    subnet_id = data.aws_subnet.sa-east-1c.id
  }
  access_logs {
  }
}

#Second interaction result

resource "aws_lb" "api-aws" {
  enable_cross_zone_load_balancing = false
  internal                         = true
  load_balancer_type               = "network"
  name                             = "api-aws"
  subnet_mapping {
    subnet_id = data.aws_subnet.sa-east-1a.id
  }
  subnet_mapping {
    subnet_id = data.aws_subnet.sa-east-1b.id
  }
  subnet_mapping {
    subnet_id = data.aws_subnet.sa-east-1c.id
  }
  access_logs {
  }
  access_logs {
  }
}

And goes on

Is it expected for this component?

https://manpages.org/jo

Is it possible to edit an item within a list? Let's say I had the following

resource "foo" "bar" {
  my_field = [
    {
      baz = "item"
    },
    {
      test = "item2"
   }
  ]
}

I can't figure out how to even run hcledit attribute get on baz i.e. hcledit attribute get resource.foo.bar.my_field[0].baz or hcledit attribute get resource.foo.bar.my_field.0.baz don't work

test.tf

resource "foo" "bar" {
  attr1 = "val1"
  nested {
    attr2 = "val2"
  }
}

Expected this but the last 2 statements returned empty strings

$ cat test.tf | hcledit attribute get resource.foo.bar.nested.attr2
"val2"
$ cat test.tf | hcledit attribute get resource.foo.bar.attr1
"val1"
$ cat test.tf | hcledit attribute get resource.foo.bar.nested
{
  attr2 = "val2"
}
$ cat test.tf | hcledit attribute get resource.foo.bar
{
  attr1 = "val1"
  nested {
    attr2 = "val2"
  }
}

Nice job on hcledit and thanks for sharing this tool with the community.

This was not fixed by #31.

Version

# hcledit version
0.2.10

Comment causes issues

body get parent shows all the rows

# hcledit body get -f /etc/nomad.d/client.hcl client
client {
  meta {
    consul            = "client"
    primary_workload  = "ingress"
    availability_zone = "eu-west-1c" # This is a comment
  }
}

Comment causes the third line to dissapear from body get parent.child

# hcledit body get -f /etc/nomad.d/client.hcl client.meta
consul           = "client"
primary_workload = "ingress"

Now remove the comment and it parses the full three lines

# hcledit body get -f /etc/nomad.d/client.hcl  client.meta
consul            = "client"
primary_workload  = "ingress"
availability_zone = "eu-west-1c"

Given the following hcl file file.hcl (notice no spaces surrounding the equals sign):

resource "foo" "bar" {
  attr1="val1"
}

hcledit attribute append resource.foo.bar.attr2 '"val2"' -f file.hcl --newline outputs the following (notice spaces were added surrounding the equals sign for attr1):

resource "foo" "bar" {
  attr1 = "val1"

  attr2 = "val2"
}

This is not always a desirable behavior, e.g. when a single small change is done to a large unformatted hcl file, the diff will appear huge, even though most of it is only formatting changes.

Can we have a flag (e.g. --no-format or similar) that makes hcledit not apply formatting? (The user can always opt to apply formatting manually after running hcledit.)

cat terraform.tf
# The default provider configuration; resources that begin with `aws_` will use
# it as the default, and it can be referenced as `aws`.
provider "aws" {
  region = "us-east-1"
}

# Additional provider configuration for west coast region; resources can
# reference this as `aws.west`.
provider "aws" {
  alias  = "west"
  region = "us-west-2"
}

cat main.tf | hcledit attribute set provider.aws.assume_role.role_arn '"arn:aws:iam::222:role/terraform"'
# The default provider configuration; resources that begin with `aws_` will use
# it as the default, and it can be referenced as `aws`.
provider "aws" {
  region = "us-east-1"
  assume_role {
    role_arn = "arn:aws:iam::222:role/terraform"
  }
}

# Additional provider configuration for west coast region; resources can
# reference this as `aws.west`.
provider "aws" {
  alias  = "west"
  region = "us-west-2"
  assume_role {
    role_arn = "arn:aws:iam::111:role/terraform"
  }
}

Note that only the first provider block got updated but not the second one. Is there some way to continue the editing to also affect the second block?

Hi,
I'm trying to set attribute in the following file

data "terraform_remote_state" "state" {
  config = {
    workspaces = {
      name = "tt-test"
    }
  }
}

but without the luck. No output there. The command I used was
cat ./remote-states.tf | /scripts/hcledit attribute get data.terraform_remote_state.state.config.workspaces.name
The only thing I could get is

> cat ./remote-states.tf | /scripts/hcledit attribute get data.terraform_remote_state.state.config
{
    workspaces = {
      name = "tt-test"
    }
  }

Is it possible to do by hcledit?

Thanks a lot for this tool!!!

Given an empty file new.tf, I want to start writing into it like this:

$ hcledit block append "" module.stack -f new.tf

This is not supported, right?

It would be handy to have such an option to script the whole creation of HCL using hcledit.

Great tool! We noticed that there isn't yet a build that works on Apple Silicon Macs (arm64).

I've opened a PR that should hopefully add a supported build: #36

Version

hcledit version
0.2.10

Example

• repro.tf

resource "foo" "bar" {
  attr = "value"

  dynamic "baz" {
    for_each = var.qux != "" ? [1] : []
    content {
      dyn_attr = "value"
    }
  }
}

Check if dynamic block can be addressed:

hcledit -f ./repro.tf block list
resource.foo.bar

hcledit -f ./repro.tf attribute get resource.foo.bar.attr
"value"

hcledit -f ./repro.tf attribute get resource.foo.bar.dynamic
hcledit -f ./repro.tf attribute get resource.foo.bar.dynamic.baz

Terraform cloud requires a cloud block with the following format:

terraform {
  cloud {
    organization = "example_corp"
    ## Required for Terraform Enterprise; Defaults to app.terraform.io for Terraform Cloud
    hostname = "app.terraform.io"

    workspaces {
      tags = ["app"]
    }
  }
}

I am able to append the empty cloud block, and then the organization attribute.

# hcledit -u -f main.tf block append terraform cloud
# hcledit -u -f main.tf attribute append terraform.cloud.organization '"$ORG_NAME"'
# hcledit -f main.tf block get terraform
terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = ">=3.41.0"
    }
  }
  cloud {
    organization = "$ORG_NAME"
  }
}

Next, I try to append the nested workspaces block:

# hcledit -u -f main.tf block append terraform.cloud workspaces
# hcledit -f main.tf block get terraform
terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = ">=3.41.0"
    }
  }
  cloud {
    organization = "$ORG_NAME"
  }
}

I expect to see it appended, but it is not. I also get no output that there was a problem.

Like tfupdate, it would be nice to update the hcl using hcledit without knowing the exact file to update.

E.g.

tfupdate terraform -v 0.12.16 -r ./

Here is a current command of hcledit

hcledit attribute append resource.foo.bar.nested.attr3 '"val3"' --newline --update --file main.tf

Perhaps a directory argument and/or recursive flag could be added with --file as optional?

hcledit attribute append resource.foo.bar.nested.attr3 '"val3"' --newline --update -r ./

I would be awesome if you could make it available through Go.
So I could just import this library in my own application and use this great magic !

Hi. Have you tested your binary under dockerized Alpine?

I'm using following installation and it doesn't work.

(...)
	&& curl -L "$( curl -Ls https://api.github.com/repos/minamijoyo/hcledit/releases/latest | grep -o -E "https://.+?_linux_amd64.tar.gz" )" \
    -o hcledit.tar.gz \
	&& tar -xf hcledit.tar.gz hcledit \
	&& mv hcledit /usr/bin/hcledit \
  && chmod +x /usr/bin/hcledit \
  && chown $(id -u):$(id -g) /usr/bin/hcledit \
(...)

I'm getting bash: ./hcledit: No such file or directory error.

Can you please advise?

I have the following HCL file:

locals {
  workload_1 = {
    vpc0 = {
      cidr_block     = "192.168.1.0/24"
      extended_cidrs = {}
      name_prefix    = "workload"
    }
  }
}

And receive the following outputs:

$ cat test.hcl | hcledit block get locals
locals {
  workload_1 = {
    vpc0 = {
      cidr_block     = "192.168.1.0/24"
      extended_cidrs = {}
      name_prefix    = "workload"
    }
  }
}
$ cat test.hcl | hcledit block get locals.workload_1
$ cat test.hcl | hcledit block get locals.workload_1.vpc0
$ 

No results are returned for locals.workload_1. Not sure why. Thanks!

Take for example this version.tf:

terraform {
  required_version = ">= 1.0.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 3.0"
    }
  }
}

I can retrieve the aws provider just fine:

❯ cat versions.tf | hcledit attribute get terraform.required_providers.aws
{
      source  = "hashicorp/aws"
      version = ">= 3.0"
}

But I cannot retrieve the version attribute of it:

❯ cat versions.tf | hcledit attribute get terraform.required_providers.aws.version

Setting this attribute doesn't work either:

❯ cat versions.tf | hcledit attribute set terraform.required_providers.aws.version "< 4.0"

terraform {
  required_version = ">= 1.0.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 3.0"
    }
  }
}

Am I doing something wrong or can't hcledit go this deep?

Hi!

I'm trying to use hcledit to parse module input variables of type object(), with inline comments. Looking like this:

variable "test" {
  type = object({
    foo = string     # The foo variable is used to control the foo'nes
    bar = number     # The number of foos
  })
}

But cat variables.tf | hcledit attribute get "variable.test.type" will only output the first key in the object:

object({
    foo = string

If I remove the two inline comments, the output of hcledit is correct:

object({
    foo = string
    bar = number
  })

We are using these comments to document complex variables and need to have them inline.

If you try to set a negative value for a number, it will be recognized as a flag. Is there any way to escape it?

I'm trying to programmatically modify tfvars files, but it appears attribute append doesn't work for them.

E.g. given this file:

owner = [
  "tom",
  "dave"
]

I was hoping I could append a new item with:

hcledit -f terraform.tfvars attribute append owner '"me"'

but this just replaces the whole list to give owner = "me"

It'd be great if this program supported append/rm/set operations just on terraform vars in tfvars files.

Get does seem to work already - I haven't tried the others yet since I need append.

I'm creating a projects.auto.tfvars file.
I used the following command to create an empty object
hcledit -f test.tf -u attribute append testing '{}'
This successfully creates
testing = {}
But now I'd like to create the following

testing = {
  testing2 = {}
}

I tried this with the following command but get no output
hcledit -f test.tf -u attribute append testing.testing2 '{}'
Am I possibly just doing something wrong or is this a feature you'd need to implement into the code?

Looks like hcledit has an issue with parsing the ~> construct:
failed to build expression at the parse phase: failed to parse input: generated_by_buildExpression:1,12-14: Invalid escape sequence; The symbol "~" is not a valid escape sequence selector.
Greater/smaller works as expected though.

We use this version construct all throughout our code ( & gitops process)

Hi. Great piece of work, I really appreciate it.

But technically correct HCL is causing some minor issues, e.g.

  list = [
    "item1",
    "item2",
  ]

Is outputted exactly like that with hcledit attribute get:

[
    "item1",
    "item2",
  ]

When I later parse it with jq -r '.[] it throws error parse error: Expected another array element at line 4, column 3.

Output from hcledit attribute get shouldn't show the comma after the last element of a list.

I can't tell if there's a way to address a block that looks like this:

provider "registry.terraform.io/hashicorp/aws" {
  version     = "3.38.0"
  constraints = "~> 3.38"
  hashes = [
    "h1:qKEjN/EM56XT46vGY33eoq7nD6JuGqRqFp7tkzTrRM0=",
  ]
}

The block can be listed, and the result looks like this:

$ cat .terraform.lock.hcl | hcledit block list
provider.registry.terraform.io/hashicorp/aws

But I can't use that (as is) to show the block:

$ cat .terraform.lock.hcl | hcledit block get provider.registry.terraform.io/hashicorp/aws

Probably the two dots are being parsed as part of the syntax for addresses. If I change those two to underscores, block get prints the block just fine:

$ cat .terraform.lock.hcl | hcledit block get provider.registry_terraform_io/hashicorp/aws
provider "registry_terraform_io/hashicorp/aws" {
  version     = "3.38.0"
  constraints = "~> 3.38"
  hashes = [
    "h1:qKEjN/EM56XT46vGY33eoq7nD6JuGqRqFp7tkzTrRM0=",
  ]
}

So clearly I need to escape those dots. But I can't figure out how. None of these work:

$ cat .terraform.lock.hcl | hcledit block get "provider.registry\.terraform\.io/hashicorp/aws"

$ cat .terraform.lock.hcl | hcledit block get "provider.registry\\.terraform\\.io/hashicorp/aws"

$ cat .terraform.lock.hcl | hcledit block get provider.'"registry.terraform.io/hashicorp/aws"'

$ cat .terraform.lock.hcl | hcledit block get provider."'registry.terraform.io/hashicorp/aws'"

Can you please add to the readme how to escape dots correctly?

Terraform supports having multiple provider blocks, which is useful in multi-account setups. Unfortunately, it appears as though hcledit stops after the first match.

It would be useful if hcledit handled all matches, or if it supported taking an index if there are multiple matches.

Command:
hcledit attribute append 'provider.aws.default_tags' "{tags = local.module_tags}" -u -f providers.tf

Expected:

provider "aws" {
  region  = local.aws_region
  alias   = "develop"
  profile = "develop"
  default_tags = { tags = local.module_tags }
}

provider "aws" {
  region  = local.aws_region
  alias   = "production"
  profile = "production"
  default_tags = { tags = local.module_tags }
}

Actual result:

provider "aws" {
  region  = local.aws_region
  alias   = "develop"
  profile = "develop"
  default_tags = { tags = local.module_tags }
}

provider "aws" {
  region  = local.aws_region
  alias   = "production"
  profile = "production"
}

Similar to the -r flag in jq, it would be nice to just get the value without quotes to make value comparison easier.