vault-fix is a CLI utility and python package that helps exporting and importing secrets to and from Vault instances. You can use this either to load fixture files for local development (its original purpose). Or to migrate data from Vault instance to another, while secrets may be encrypted and/or piped to another vault-fix instance so the data is not persisted.
vault-fix was created to address an issue with the default mode of Vault instances in dev mode, for local development. Vault will start with ephemeral storage, i.e. in-memory, mounting a volume will not make it persistent. If you want to have persistent data, you'd have to provision a mount and a volume. However, this will make your local test environment more stateful, which is not always desirable. Plus a normal Vault instance will can "seal" itself to protect itself from attackers, which is not something you normally want to deal with during development.
Instead you may want to load a known fixture, containing a curated set of secrets that you don't want to manually set every time you restarted vault. In other words, a fixture. This allows you to start from a clean slate every time you test or debug. You can automate the loading or dumping of secrets, and/or use the CLI.
pip install vault-fixFinding out how this works:
vault-fix --help
Usage: vault-fix [OPTIONS] COMMAND [ARGS]...
Load or dump data?
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --install-completion Install completion for the current shell. │
│ --show-completion Show completion for the current shell, to copy it or customize the installation. │
│ --help Show this message and exit. │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ──────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ dump Load up, and dump secrets to and from Vault. │
│ load Load up, and dump secrets to and from Vault. │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯Specific to dumping fixtures:
vault-fix dump --help
Usage: vault-fix dump [OPTIONS] MOUNT PATH
Load up, and dump secrets to and from Vault.
╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ * mount TEXT Vault mount [default: None] [required] │
│ * path TEXT Vault path within the mount [default: None] [required] │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ * --token -t TEXT Vault access token. [default: None] [required] │
│ --host -H TEXT Vault hostname [default: localhost] │
│ --port -P INTEGER Vault network port. [default: 8200] │
│ --tls --no-tls Enable or disable TLS [default: tls] │
│ --verbose -v INTEGER Specify verbosity level by passing more 1 or more -v -vv │
│ -vvv's │
│ [default: 0] │
│ --file -f TEXT Output file, stdout if not specified [default: -] │
│ --password -p TEXT Password to encrypt the dumped fixture, or none for plain text │
│ output. │
│ --pretty --no-pretty Pretty print the output (if JSON formatted [default: pretty] │
│ --serializer [json|yaml] Which serializer do you prefer? [default=yaml] [default: yaml] │
│ --help Show this message and exit. │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯Specific to loading fixtures:
vault-fix load --help
Usage: vault-fix load [OPTIONS] MOUNT PATH
Load up, and dump secrets to and from Vault.
╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ * mount TEXT Vault mount [default: None] [required] │
│ * path TEXT Vault path within the mount [default: None] [required] │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ * --token -t TEXT Vault access token. [default: None] [required] │
│ --host -H TEXT Vault hostname [default: localhost] │
│ --port -P INTEGER Vault network port. [default: 8200] │
│ --tls --no-tls Enable or disable TLS [default: tls] │
│ --verbose -v INTEGER Specify verbosity level by passing more 1 or more -v -vv │
│ -vvv's │
│ [default: 0] │
│ --file -f TEXT Input file, assumes stdin if not specified [default: -] │
│ --password -p TEXT Password to decrypt the dumped fixture, or none for plain │
│ text input. │
│ --deserializer [json|yaml|auto] Which deserializer does the fixture file require? │
│ [default: auto] │
│ --help Show this message and exit. │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯Dump secrets from a locally running vault instance:
vault-fix dump secret / --no-tlsOutput will be printed to stdout, you can specify -f FILE or direct output to a file, like:
vault-fix dump secret / --no-tls > my-fixture.yamlIf you want your secrets encrypted, pass -p to get a password prompt, or pass the password on the command line (not safe).
vault-fix dump secret / --no-tls -pOnly secrets will be encrypted, the paths will be in plain text.
If you want your secrets dumped in JSON format instead of the default YAML format, pass --serializer json
vault-fix dump secret / --no-tls --serializer jsonLoad secrets from a file to a locally running vault instance:
vault-fix load secret / --no-tls -f my-fixture.jsonIf the fixture is encrypted, you need to pass the -p parameter, or you will get a runtime error.
Load secrets from a file to a locally running vault instance:
cat my-fixture.json | vault-fix load secret / --no-tls --deserializer jsonWhich brings us to this command, that allow you to migrate secrets between vault instances:
vault-fix dump secret / -H vault.dev.yourdomain.com | vault-fix load secret / --no-tlsOne of the best things about this utility is that you can automatically load fixtures to a local vault dev server, e.g. during application startup.
from hvac import Client
from vault_fix.load import load_fixture_from_file
from vault_fix.serializers.yaml import yaml_deserializer
# Vault docker container running on your local machine in dev mode, with ephemeral storage.
# Assuming the following defaults
VAULT_ADDR = "http://vault:8200"
VAULT_TOKEN = "root"
VAULT_TLS_ENABLED = False
VAULT_MOUNT = "secret"
FIXTURE_PATH = "../vault_fixture_local_dev.yaml"
def load_vault_secrets() -> None:
print(f"Attempting to import vault fixture from {FIXTURE_PATH}")
client = Client(url=VAULT_ADDR, token=VAULT_TOKEN, verify=VAULT_TLS_ENABLED)
try:
with open(FIXTURE_PATH, "rt") as fixture_fh:
load_fixture_from_file(
hvac=client, fixture=fixture_fh, mount_point=VAULT_MOUNT, deserializer=yaml_deserializer
)
print(f"Imported vault fixture from {FIXTURE_PATH}")
except OSError:
print(f"Can't read fixture file from {FIXTURE_PATH}")- The path parameter specifies the path in the vault server you want to dump. Or the path you would like to load to a server from the fixture file. Meaning you can select a subset of secrets to dump or load from servers or fixtures respectively.
- vault-fix does not dump or import metadata, including previous versions of secrets.
Checkout the project, make a virtual env with hatch and install dependencies.
git checkout [email protected]:SnijderC/vault-fix.git
cd vault-fix
pre-commit install
pip install hatch
hatch shellIf you're in a hatch shell, exit it first, then:
hatch run test:pytestThis will test vault-fix against Python 3.9 - 3.11. If you don't have all of those, they will be skipped. You can install them with pyenv:
pyenv install 3.9 3.10 3.11
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent