alecthomas / hcl Goto Github PK

View Code? Open in Web Editor NEW

46.0 3.0 10.0 119 KB

Parsing, encoding and decoding of HCL to and from Go types and an AST.

License: MIT License

Go 98.66% Shell 1.34%

golang hcl

hcl's Introduction

Parsing, encoding and decoding of HCL to and from Go types

This package provides idiomatic Go functions for marshalling and unmarshalling HCL, as well as an AST

It supports the same tags as the Hashicorp hcl2 gohcl package, but is much less complex.

Unlike gohcl it also natively supports time.Duration, time.Time, encoding.TextUnmarshaler and json.Unmarshaler.

It is HCL1 compatible and does not support any HCL2 specific features.

Design

HCL -> AST -> Go -> AST -> HCL

Mapping can start from any point in this cycle.

Marshalling, unmarshalling, parsing and serialisation are all structurally isomorphic operations. That is, HCL can be deserialised into an AST or Go, or vice versa, and the structure on both ends will be identical.

HCL is always parsed into an AST before unmarshaling and, similarly, Go structures are always mapped to an AST before being serialised to HCL.

Between	And	Preserves
HCL	AST	Structure, values, order, comments.
HCL	Go	Structure, values, partial comments (via the `help:""` tag).
AST	Go	Structure, values.

Schema reflection

HCL has no real concept of schemas (that I can find), but there is precedent for something similar in Terraform variable definition files. This package supports reflecting a rudimentary schema from Go, where the value for each attribute is one of the scalar types number, string or boolean. Lists and maps are typed by example.

Here's an example schema.

// A string field.
str = string
num = number
bool = boolean
list = [string]

// A map.
map = {
  string: number,
}

// A block.
block "name" {
  attr = string
}

// Repeated blocks.
block_slice "label0" "label1" {
  attr = string
}

Comments are from help:"" tags. See schema_test.go for details.

Struct field tags

The tag format is as with other similar serialisation packages:

hcl:"[<name>][,<option>]"

The supported options are:

Tag	Description
`attr` (default)	Specifies that the value is to be populated from an attribute.
`block`	Specifies that the value is to populated from a block.
`label`	Specifies that the value is to populated from a block label.
`optional`	As with attr, but the field is optional.
`remain`	Specifies that the value is to be populated from the remaining body after populating other fields. The field must be of type `[]hcl.Entry`.

Additionally, a separate help:"" tag can be specified to populate comment fields in the AST when serialising Go structures.

Position

Any block with a field named Pos of the type hcl.Position will have that field populated with positional information:

Pos Position `hcl:"-"`

hcl's People

Contributors

Stargazers

Watchers

Forkers

forkkit kalfonso lyonlai hamdanjaveed juliaogris bjoerndemeyer mightyguava envien kingcrypto38 datadog

hcl's Issues

Extra remaining fields in the presence of a slice of structs

I ran into an issue when trying to obtain remaining fields for the example below, the second nested field (your) was returned as part of remaining fields.

HCL:

name = "hello"
nested {
  name = "my"
}
nested {
  name = "your"
}
message1 = "wonderful"
message2 = world

type remainStruct struct {
	Name   string          `hcl:"name"`
	Nested []*remainNested `hcl:"nested,optional"`
	Remain []*Entry        `hcl:",remain"`
}

type remainNested struct {
	Name string `hcl:"name"`
}

The image below shows the third remaining field in the unmarshalled object:

Support recursive string interpolation

For compatibility, eg.

"foo ${firstName("User Name")}"

Parser fails to parse negative numbers

Example

See this playground for an example.

Expectation

The unmarshalled struct has the field NegativeInt set to -1

Actual

Parser fails to parse the number.

I think this is happening because there's no word boundary at the beginning of the number for this lexer rule to match. The negative sign might be messing with it. If I take the negative sign out it works, and if I test the regex without the beginning \b it also works.

Default annotation not working after v0.1.8

Hello,

To begin with, thanks for your work on this package. It's great you have comment & time.Duration support !

I tried to use it as a replacement of hashicorp/hcl in one of my project. But based on my unit tests, default annotation (such as hcl:"connection_string" default:"postgresql://user:password@localhost:5432/postgres") stopped working between v0.1.8 & v0.1.9.

When using v0.1.8 of this package, tests are successful, default values are correctly set. link to PR & link to CI

github.com/Stoakes/go-pkg/config	[no test files]
=== RUN   Test_FileExportCommand
--- PASS: Test_FileExportCommand (0.00s)
PASS
coverage: 86.1% of statements
ok  	github.com/Stoakes/go-pkg/config/cmd	0.004s	coverage: 86.1% of statements

When using v0.1.9 or higher, tests fails, default values aren't set anymore. link to PR & link to CI

github.com/Stoakes/go-pkg/config	[no test files]
=== RUN   Test_FileExportCommand
    builder_test.go:115: hcl test error. Expected "mode = false
        
        Local {
          connection_string = "***localhost:5432/postgres"
        }
        
        remote {
          address = ""
          useTLS = false
        }
        " got "Local {
        }
        
        remote {
          address = ""
        }
        "
--- FAIL: Test_FileExportCommand (0.00s)
FAIL
coverage: 86.1% of statements
FAIL	github.com/Stoakes/go-pkg/config/cmd	0.005s
FAIL

Tell me if I can help.

Is there an equivalent to `hil.Interpolate` in v2?

It looks like the hil package was removed in the v2 release in this commit. I can't seem to find an equivalent function in the docs or v2 release.

Is there one that I'm missing or should this be implemented manually?
Thanks!

Export withSchema

For my HCL config files it would be easier for the end users if the help comments were also written to the output of Marshal. There already is a marshal option withSchema that allows that but it is not exported. Please consider exporting this marshal option.

Construct schema from third party tagged with json

We'd like to construct schema from third party type tagged with json. Constructing the schema fails currently:

panic: struct v1alpha1.TiKVServerConfig used as attribute, is it missing a "block" tag?

Example structure: https://github.com/pingcap/tidb-operator/blob/master/pkg/apis/pingcap/v1alpha1/tikv_config.go

Derived HCL attributes

Hi,

We have a Terraform template with some logic that is becoming hard to understand. We'd like to move the logic to Go and provide the processed data as variable in the struct to be generated as HCL. In order to simplify logic inside Terraform template and move it over to Go but would like to have derived attributes that are used only in the HCL output. Does this already exist?

Go struct:

type MyStruct struct {
    Val string `json:"val,omitempty" hcl:"val"`
    Roles string `json:"roles,omitempty" hcl:"roles,optional,output"`
}

output doesn't exist, just to show the intention or how this could be likely used. Roles value is the result of processing Val.

input hcl

val = "a"

output hcl

v := &MyStruct{
    val: "a",
}

val = "a"
roles = "a,a_owners"

help needed

How do I convert terraform files in the directory an convert it to json files( output it to output.json ) using this library?

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

chore(deps): update all non-major dependencies (github.com/alecthomas/assert/v2, github.com/alecthomas/participle/v2, github.com/alecthomas/repr, go, golangci-lint)
chore(deps): update actions/checkout action to v4

Detected dependencies

github-actions

.github/workflows/ci.yml

actions/checkout v2

actions/checkout v2

gomod

go.mod

go 1.18

github.com/alecthomas/assert/v2 v2.2.0

github.com/alecthomas/participle/v2 v2.0.0-beta.5

github.com/alecthomas/repr v0.1.0

hermit

bin/hermit

go 1.19.1

golangci-lint 1.46.2

Check this box to trigger a request for Renovate to run again on this repository