Hi !

I've deployed a vault server in development version with a root token being "root".

If i do:

export VAULT_TOKEN="root"
export VAULT_ADDR=http://10.100.10.8:8200

vault token lookup
Key                 Value
---                 -----
accessor            gbNXF7H5gb82nF2UsVLk5T1t
creation_time       1723126368
creation_ttl        0s
display_name        token
entity_id           n/a
expire_time         <nil>
explicit_max_ttl    0s
id                  root
issue_time          2024-08-08T14:12:48.477231813Z
meta                <nil>
num_uses            0
orphan              true
path                auth/token/create
policies            [root]
renewable           false
ttl                 0s
type                service

vault kv get -mount=stfc-techops-staging-k8s test-injection
================ Secret Path ================
stfc-techops-staging-k8s/data/test-injection

======= Metadata =======
Key                Value
---                -----
created_time       2024-08-08T14:13:18.864066203Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            1

======= Data =======
Key           Value
---           -----
csi_driver    ok
injector      ok

But if i use vkv, i get:

vkv export -p stfc-techops-staging-k8s --show-values --format yaml
[ERROR] not authenticated, perhaps not a valid token: Error making API request.

URL: GET http://10.100.10.8:8200/v1/auth/token/lookup-self
Code: 403. Errors:

* 2 errors occurred:
        * permission denied
        * invalid token

.

That error message is usually thrown when NO authentication has been provided, which makes little sense. I even tried:

VKV_LOGIN_COMMAND="echo root"

Without success !

Just an idea, I'm wondering if there could be a different to generate the exported variable based on the vault secret path+name, or on some other key-value in the secret, instead of using the "key" of the secret as the exported variable name?

For example:

├── dev
│   └── token
│       └── value=************
│   
├── prd
│   ├── token
│   │   └── value=************
│   │   
│   └── key
│       └── value=************
├── os
│   └── value=************
│   
└── test
    └── value=************

Running -f export will only result in one (non-deterministic) export value=... being printed because all of the secrets are in the format of value=foo/bar/etc.

I'm imagining something like -f export-flatten --field=value that instead creates exports based on the flattened secret path+name, so it would generate something like:

export dev_token=...
export prd_token=...
export prd_key=...
export os=...
export test=...

and -f export-name --field=value that instead creates exports based on thesecret name:

export token=...
export key=...
export os=...
export test=...

Tried using template but couldn't get the recursion to work properly.

Alternatively, a way to generate exports based on a meta-kv in the secret itself:

kv put engine/kv/dev/token value=abc _vkv_export=my_token

Which when exported with-f export gets transformed into

export my_token=abc

Hi,

Running v0.7.0, and noticed that secret values are getting interpreted rather than passed as is.
Example:

"value": "password<"

is JSON rendered as:
"value": "password\u003c"

When using base as format, if newline (\n) is in value, this is also interpreted resulting in the below incorrect syntax

          │   ├── example-keystore-jks_base64=/u3+7QQAAAYtIxCqRAAAFATCCBP0wDgYKKwAAAAIAAAABAAAAAQABMYBBAEqAhEBAQUABIIE6Z9q
            │   │   w2lvG+76o2JhexbzP2qA8EnUwzRorbDkbgRvTrsp3dMFBEnYqFurMXJMVnY8qffoD4OC19OvMkFg
            │   │   /am4W0GdZV05AQh+OTmEgDPfyVNKZCosmGl5Zr6zYuxSR1a9jVOjIPh+KIgDFy/Q021S7R+1fHrR
            │   │   6eKX5JDtl3TWDSphgInCsNN+h8LVk+nC6DtJoAQSqC6+t+b1SRzvVOWk+EmnRHTLULmjG7zTCNx0

Hello. This may be user error, but I'm trying to import from a json file that I created using vkv snapshot save. I actually have 4 of these files, one for each KV engine that I have.

When importing, the top-level paths are lost. Here's an example of the data:

{
  "toplevel/": {
    "sublevel": {
      "AWS_ACCESS_KEY_ID": "laskdfjalskdjfl",
      "AWS_DEFAULT_REGION": "us-east-1",
      "AWS_SECRET_ACCESS_KEY": "kasdhflksdjfls",

My import command looks like this:

./vkv import -f vault-export-2024-02-15/cicd.yaml -p cicd

Then, when I see the secrets in Vault, toplevel is gone, and so the path will be like cicd/sublevel/.

I think I'm probably doing something wrong. Any help would be appreciated. Thanks.

https://github.com/d6o/GoTree#usage

output:

␉��␉��␉�� devices/
␉��   ␉��␉��␉�� 984/
␉��       ␉��␉��␉�� bmcusers/
␉��           ␉��␉��␉�� monitoring
␉��␉��␉�� hpe
␉��␉��␉�� test

Expose http server so vkv can be used in gitlab/github CI service container

• support namespaces via VAULT_NAMESPACE
• list / copy / move / delete / compare sub commands
• MAX_SECRET_LENGTH
• Matrix testing for os and vault version
• multiple root paths
• View secrets for a certain policy / or print out policy information
• support kv v1
• Remove sub path var
• add option fornamespace listing
• Each secret pro line
• flag for printing version, and metadata if v2, enabling via flag
• add markdown table as output format
• rename flags to json and yaml, avoid the word secrets
• better gitlab pages and readme

• new output format which shows the token policies recursive for each kv path using self-capabilities api call
• cli flag for showing secrets version
• cli flag for showing secrets metadata
• cli flag for showing the links to each secret
• shell completion
• man page
• cli flag which lists all kv engines and use can interactively choose
• cli flag for browsing through kv engines and on right side u see the tree view (maybe using something like bubbletea or fzf)
• better github pages https://github.com/alex-shpak/hugo-book
• create an icon
• add template snippet for generating the corresponding tf code that recreates the complete kv engine
• harmonize gif images and font sizes
• only build gifs when tag
• test document behaviour with not sufficient token policy
• always test against last 3 mayor versions
• support KVv1
• import command which takes vkv json or yaml output (flags: -f, -d, STDIN, --force, dry-run/preview)
• Separate files in printer dir with tests
• add commands for encrypting exported secrets with pgp and importing them again (decrypting)
• Subcommand for mirroring secrets between two emgines in intervalls (daemon) takes a congig file
• Flag for showing previous secret versions
• Warn if masked secrets are imported
• Handle Engine path in vkv import
• encrypt/decrypt secrets using sops and vault
• Vault backup subcommand (export alle kv engines from all visble ns, flags: namespace, list engines, …)
• Vault restore - restores all exported kv engines
• Docker & k8s cronjob integration
• #118
• use conventional commits and group in changelog in goreleaser
• quick start guide
• write proper testutils (create/delete ns, engines, spinup vault, ...)
• usage with tf for secret importing without being in state
• List minimum required policy for all features
• add custom metadata to markdown output
• switch to https://pkg.go.dev/github.com/hashicorp/vault/api#Client.KVv2 and https://pkg.go.dev/github.com/hashicorp/vault/api#Client.KVv1 and see if it is better

Can we use the sub-command vkv import to write data into an existing engine path and do not overwrite the existing data ?

see #283

received via email

Hi Tom,

I’d like to thank you for you work on this vkv swiss knife tool. It works like a charm for kv secret engines.

However, while testing, I ran into a problem with the following error.

$>  vkv export -p garr -f=json

panic: interface conversion: interface {} is nil, not map[string]interface {}
goroutine 1 [running]:
[github.com/FalcoSuessgott/vkv/pkg/vault.(*Vault).IsKVv1(0xafe1fd](http://github.com/FalcoSuessgott/vkv/pkg/vault.(*Vault).IsKVv1(0xafe1fd)?, {0x7ffd2ade928f?, 0xc0004d18c0?})
       /home/runner/work/vkv/vkv/pkg/vault/kv.go:120 +0x179
[github.com/FalcoSuessgott/vkv/pkg/vault.(*Vault).ListKeys(0xc000050618](http://github.com/FalcoSuessgott/vkv/pkg/vault.(*Vault).ListKeys(0xc000050618), {0x7ffd2ade928f, 0x3}, {0x0, 0x0})
       /home/runner/work/vkv/vkv/pkg/vault/kv.go:72 +0xec
[github.com/FalcoSuessgott/vkv/pkg/vault.(*Vault).ListRecursive(0x7ffd2ade928f](http://github.com/FalcoSuessgott/vkv/pkg/vault.(*Vault).ListRecursive(0x7ffd2ade928f)?, {0x7ffd2ade928f, 0x3}, {0x0, 0x0}, 0x0)
       /home/runner/work/vkv/vkv/pkg/vault/kv.go:31 +0xbc
[github.com/FalcoSuessgott/vkv/cmd/export.(*exportOptions).buildMap(0xc00058ea10](http://github.com/FalcoSuessgott/vkv/cmd/export.(*exportOptions).buildMap(0xc00058ea10), 0x0?)
       /home/runner/work/vkv/vkv/cmd/export/export.go:194 +0x96
[github.com/FalcoSuessgott/vkv/cmd/export.NewExportCmd.func1(0xc0005c4100](http://github.com/FalcoSuessgott/vkv/cmd/export.NewExportCmd.func1(0xc0005c4100)?, {0xaeeff5?, 0x4?, 0xaeef01?})
       /home/runner/work/vkv/vkv/cmd/export/export.go:87 +0x405
[github.com/spf13/cobra.(*Command).execute(0xc00041cf00](http://github.com/spf13/cobra.(*Command).execute(0xc00041cf00), {0xc00050c8a0, 0x3, 0x3})
       [/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:983](mailto:/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:983) +0xabc
[github.com/spf13/cobra.(*Command).ExecuteC(0xc0005ca000)](http://github.com/spf13/cobra.(*Command).ExecuteC(0xc0005ca000))
       [/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:1115](mailto:/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:1115) +0x3ff
[github.com/spf13/cobra.(*Command).Execute(..](http://github.com/spf13/cobra.(*Command).Execute(..).)       [/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:1039](mailto:/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:1039)
[github.com/FalcoSuessgott/vkv/cmd.Execute({0xc020ac](http://github.com/FalcoSuessgott/vkv/cmd.Execute(%7B0xc020ac)?, 0xc00004c730?})
       /home/runner/work/vkv/vkv/cmd/root.go:82 +0x2b
main.main()
       /home/runner/work/vkv/vkv/main.go:13 +0x25 

I found out this was because “garr” secret engine is of type “generic” and that was causing the panic. We adopted Vault couple of years ago and the initial and heavily used secret engines were created of type “generic” which also is the same as KV1 type.

Path                                    Type                     Accessor                            Description
----                                        ----                        --------                                   -----------im/                                      generic                 generic_22e0c17f            n/a
inv/                                     generic                 generic_0deb1fc1            n/a
is/                                        generic                 generic_5f453afd             n/a
is2/                                     kv                          kv_8a47f636                     n/a
japan/                                generic                 generic_1c952757           n/a
garr/                                    generic                generic_f5e72d8f            n/a
paas/                                  generic                 generic_bd6e9dc3           n/a

I would like to request if you can also help include “generic” secret engine type as well to vkv?

If you could also point out to the section of code to make changes, I’ll give it a try and build it locally.

Also, I noticed that if I choose json or yaml format, the secrets are not hashed but are displayed in plain text.

Thanks in advance for your help. Much appreciated.

This should also work :

vkv -p <vault-namespace>/path

Hello

Is there a way to export and then import the metadata ?

I see it in base and markdown but don't see the metadata neither in json nor in yaml.

Is it possible?

Hi,

As the title implies, JSON format currently does not have version number displayed unlike base output format. It would be great to know the version number of the secret when outputting in JSON.

Furthermore, do you have any plans to support extracting all versions of a secret in JSON format? This is highly beneficial if running a search against the export in JSON.

You would know:

• The Secret name and its version
• If the value changed in later versions

As a workaround I modify this line and apprend something like:

			if err != nil && !strings.Contains(err.Error(), "no secrets in") {
				return nil, err
			}

Can you maybe add an option to ignore these kind of error or something to not interrupt the recursive process? Thanks

pass a template file using golangs template syntax and render the secrets

continue in recursive read instead of returning err

https://goreleaser.com/customization/homebrew/?h=brew#limitations

brews:
  - repository:
      owner: vkv
      name: homebrew-tap
    directory: Formula
    goarm: "7"
    homepage: https://github.com/FalcoSuessgott/vkv
    description: vkv enables you to list, compare, move, import, document, backup & encrypt secrets from a HashiCorp Vault KV engine
    license: MIT
    test: |
      system "#{bin}/vkv -v"
    dependencies:
      - name: go
        type: optional
    extra_install: |-
      bash_completion.install "completions/vkv.bash" => "vkv"
      zsh_completion.install "completions/vkv.zsh" => "_vkv"
      fish_completion.install "completions/vkv.fish"
      man1.install "manpages/vkv.1.gz"

Use same as in markdown

• print URL use vault addr env var (add as option to export)
• use regex as param and output // line per line that match either key/value

> vkv export --path secret                                            
secret/ [desc=key/value secret storage] [type=kv2]
├── admin [v=1] [key=value]
│   └── sub=********
├── demo [v=1]
│   └── foo=***
└── sub/
    ├── demo [v=1]
    │   ├── demo=***********
    │   ├── password=******
    │   └── user=*****
    └── sub2 <---  delimiter missing
        └── demo [v=2] [admin=false key=value]
            ├── admin=***
            ├── foo=***
            ├── password=********
            └── user=****

either remove the delimiters for all (because we already have indents) or fix it

Hi Falco! Thanx for your OpenSource!
After you merged root and subpath - I think now it is creating a wrong URL for request
When I run .\vkv.exe -p /root/sub1/sub2/sub3
I getting error :

Code: 403. Errors:

• 1 error occurred:
  * permission denied
  https://sog-vault.avp.ru:8200/v1/metadata/root/sub1/sub2/sub3?list=true

The problem is - the metadata part should be in between /root/ and /sub/
like: https://sog-vault.avp.ru:8200/v1/root/metadata/sub1/sub2/sub3?list=true
according to https://www.vaultproject.io/api/secret/kv/kv-v2#list-secrets

Originally posted by @krot4u in #6 (comment)

'vkv import' does not have an '-e' flag like the 'vkv export' command does to support KV-engines with special characters.

Am I missing something to properly import files with a engine path that contains '/' in the path?

export.json

{
    "secrets/v1": {
        "test01/": {
            "test02/": {
                "test03": {
                    "key": "test-secret"
                }
            }
        }
    }
}

vkv import -d -p secrets/v1 -f test_staging_v1.json
reading secrets from test_staging_v1.json
parsing secrets from JSON

preview:

secrets/v1/
└── v1
    └── test01
        └── test02
            └── test03
                └── key=***********

apply changes by using the --force flag

The reason for using import for this, is due to having modfied/removed secrets from the exported file, to import into a new vault environment.

Thanks!

ps this tool is amazing, thanks for the hard work.

$ vkv snapshot save -d backup1
created backup1
created backup1/kv-project1.yaml
[ERROR] could not read secrets from kv-project2/: no secrets in kv-project2 found.

In vkv 0.6.0, if there are no secrets in the KV, the program quit and stop going through other kv.

When listing all secrets via it fails when it hits a path with no secrets:

./vkv export --path secret/<path>
[ERROR] no secrets in secret/<path> found

Maybe there is a way around this other than creating a secret in the failing path?