Quickstart

This guide will run you through some of the features of vkv.

Prerequisites

In order to perform all of the described tasks, you will need the following tools:

• a Linux/MacOS Shell
• docker installed and running (alternatively vault CLI can be used)
• vkv installed (follow https://falcosuessgott.github.io/vkv/02_installation/)

Spin up a development Vault server

First, we setup a development Vault server.

Open a terminal and run:

docker run -p 8200:8200 hashicorp/vault server -dev -dev-root-token-id=root

You should then be able to visit http://127.0.0.1:8200 in your browser and see a Vault login page.

The root token is root.

Verify connection

Once you have exported the required environment variables, you can verify your connection with the vault CLI by running:

export VAULT_ADDR="http://127.0.0.1:8200"
export VAULT_TOKEN="root"
vault status
Key             Value
---             -----
Seal Type       shamir
Initialized     true
Sealed          false
Total Shares    1
Threshold       1
Version         1.12.1
Build Date      2022-10-27T12:32:05Z
Storage Type    inmem
Cluster Name    vault-cluster-1bbeabe8
Cluster ID      405e99ab-5f8c-18ca-dafb-228c91add2f4
HA Enabled      false

If vault status returned an output like this you good to go to the next step

Write secrets to Vault using vault

In a development Vault server a KVv2 under secret/ is enabled by default. We want to write some secrets using vault:

vault kv put -mount=secret admin username=user password=passw0rd
vault kv metadata put -mount=secret -custom-metadata=key=value admin
vault kv put -mount=secret db/prod env=prod username=user password=passw0rd-prod
vault kv put -mount=secret db/dev env=dev username=user password=passw0rd-dev

List secrets using vkv

vkv requires at least VAULT_ADDR and VAULT_TOKEN if the vault status command works, vkv will also work.

We can now use vkv to list all of our secrets recursively:

vkv export --path secret
secret/ [desc=key/value secret storage] [type=kv2]
├── admin [v=1] [key=value]
│   └── sub=********
├── demo [v=1]
│   └── foo=***
└── sub
    ├── demo [v=1]
    │   ├── demo=***********
    │   ├── password=******
    │   └── user=*****
    └── sub2
        └── demo [v=2] [admin=false key=value]
            ├── admin=***
            ├── foo=***
            ├── password=********
            └── user=****

Here are some explanations: * vkv prints the description and the engine type + version

• vkv masks the secrets per default, you can disable this by using --show-values or VKV_EXPORT_SHOW_VALUES=true

• vkv limits the length of the secrets per default to 12 for readability purposes (You can set you own value length by using --max-value-length=XX or VKV_EXPORT_MAX_VALUE_LENGTH=XX)

• v1 indicates the secret version (disable by using --show-version) or VKV_EXPORT_SHOW_VERSION=false

• [key=value] represents the custom metadata that we added to the secret in step 3. (disable by --show-metadata or (VKV_EXPORT_SHOW_METADATA=false)

• every secret is a hyperlink which links to the secret in the Vault UI (disable with --with-hyperlink=false or VKV_EXPORT_WITH_HYPERLINK). Works only for supported terminals (see https://github.com/savioxavier/termlink/blob/master/termlink.go#L80)

This output format is the default format called base. vkv has many other useful output formats.

You can see them all using this oneliner:

for f in base yaml json markdown policy export; do
echo -n "\n===> Output Format: $f <===\n"
vkv export -p secret --format=$f;
done

===> Output Format: base <===
vkv export --path secret
secret/ [desc=key/value secret storage] [type=kv2]
├── admin [v=1] [key=value]
│   └── sub=********
├── demo [v=1]
│   └── foo=***
└── sub
    ├── demo [v=1]
    │   ├── demo=***********
    │   ├── password=******
    │   └── user=*****
    └── sub2
        └── demo [v=2] [admin=false key=value]
            ├── admin=***
            ├── foo=***
            ├── password=********
            └── user=****

===> Output Format: yaml <===
secret/:
  admin:
    password: '********'
    username: '****'
  db/:
    dev:
      env: '***'
      password: '************'
      username: '****'
    prod:
      env: '****'
      password: '*************'
      username: '****'


===> Output Format: json <===
{
  "secret/": {
    "admin": {
      "password": "********",
      "username": "****"
    },
    "db/": {
      "dev": {
        "env": "***",
        "password": "************",
        "username": "****"
      },
      "prod": {
        "env": "****",
        "password": "*************",
        "username": "****"
      }
    }
  }
}

===> Output Format: markdown <===
|      PATH      |   KEY    |    VALUE     | VERSION | METADATA  |
|----------------|----------|--------------|---------|-----------|
| secret/admin   | password | ********     |       1 | key=value |
|                | username | ****         |         |           |
| secret/db/dev  | env      | ***          |       1 |           |
|                | password | ************ |         |           |
|                | username | ****         |         |           |
| secret/db/prod | env      | ****         |       1 |           |
|                | password | ************ |         |           |
|                | username | ****         |         |           |

===> Output Format: policy <===
PATH            CREATE  READ    UPDATE  DELETE  LIST    ROOT
secret/admin    ✖       ✖       ✖       ✖       ✖       ✔
secret/db/dev   ✖       ✖       ✖       ✖       ✖       ✔
secret/db/prod  ✖       ✖       ✖       ✖       ✖       ✔

===> Output Format: export <===
export env='prod'
export password='passw0rd-prod'
export username='user'

Most of these formats, offer various commandline flags, such as --show-secrets, --only-paths, --only-keys, --max-value-length to modify the output. These flags can also be set through environment variables:

VKV_EXPORT_FORMAT=JSON VKV_EXPORT_SHOW_VALUES=true vkv export -p secret
{
  "secret/": {
    "admin": {
      "password": "passw0rd",
      "username": "user"
    },
    "db/": {
      "dev": {
        "env": "dev",
        "password": "passw0rd-dev",
        "username": "user"
      },
      "prod": {
        "env": "prod",
        "password": "passw0rd-prod",
        "username": "user"
      }
    }
  }
}

Import secrets using vkv

Meanwhile vkv export can be used to store secrets, vkv import is used to import secrets from a vkv export command (either yaml or json format is accepted).

Knowing this, we can copy a secret engine to another secret engine:

vkv export -p secret -f=yaml | vkv import - -p copy
reading secrets from STDIN
parsing secrets from YAML
writing secret "copy/db/dev"
writing secret "copy/db/prod"
writing secret "copy/admin"
successfully imported all secrets

result:

copy/ [type=kv2]
├── admin [v=1] [key=value]
│   └── sub=********
├── demo [v=1]
│   └── foo=***
└── sub
    ├── demo [v=1]
    │   ├── demo=***********
    │   ├── password=******
    │   └── user=*****
    └── sub2
        └── demo [v=2] [admin=false key=value]
            ├── admin=***
            ├── foo=***
            ├── password=********
            └── user=****

Or even to another Vault instance:

vkv export -p secret -f=yaml | VAULT_ADDR="..." VAULT_TOKEN="..." vkv import - -p copy
[...]

The - tells vkv to read the secrets from STDIN. You cal also specify a file using the --file parameter.

vkv will create the KVv2 engine if it doesn't exist. If the engine indeed exists, vkv will error unless --force is used.

You can also copy sub-paths to other engines:

vkv export -p secret/admin -f=yaml --show-values| vkv import - -p admin
reading secrets from STDIN
parsing secrets from YAML
writing secret "admin/admin"
successfully imported all secrets

result:

admin/
└── admin [v=1]
    ├── password=********
    └── username=****

Create KVv2 Snapshots using vkv

vkv enables you to create and restore snapshots of all KVv2 engines in all namespaces of a Vault instance (requires an appropiate token + policy):

Consider the following namespaces and KVv2 engines on a Vault Enterprise instance:

# list all namespaces
vkv list namespaces --all
sub
sub/sub2
test
test/test2
test/test2/test3

# list all engines with their respective namespace as the prefix
vkv list engines --all --include-ns-prefix
secret
secret_2
sub/sub2/sub_sub2_secret
sub/sub2/sub_sub2_secret_2
sub/sub_secret
sub/sub_secret_2
test/test2/test3/test_test2_test3_secret
test/test2/test3/test_test2_test3_secret_2

You can create a snapshot of those KVv2 engines by running:

vkv snapshot save --destination vkv-export-$(date '+%Y-%m-%d')
created vkv-export-2022-12-29
created vkv-export-2022-12-29/secret.yaml
created vkv-export-2022-12-29/secret_2.yaml
created vkv-export-2022-12-29/sub
created vkv-export-2022-12-29/sub/sub_secret_2.yaml
created vkv-export-2022-12-29/sub/sub_secret.yaml
created vkv-export-2022-12-29/sub/sub2
created vkv-export-2022-12-29/sub/sub2/sub_sub2_secret.yaml
created vkv-export-2022-12-29/sub/sub2/sub_sub2_secret_2.yaml
created vkv-export-2022-12-29/test
created vkv-export-2022-12-29/test/test2
created vkv-export-2022-12-29/test/test2/test3
created vkv-export-2022-12-29/test/test2/test3/test_test2_test3_secret.yaml
created vkv-export-2022-12-29/test/test2/test3/test_test2_test3_secret_2.yaml

As you can see: vkv exported all engines and wrote them to the specified directory:

vkv-export-2022-12-29/
├── secret_2.yaml
├── secret.yaml
├── sub
│   ├── sub2
│   │   ├── sub_sub2_secret_2.yaml
│   │   └── sub_sub2_secret.yaml
│   ├── sub_secret_2.yaml
│   └── sub_secret.yaml
└── test
    └── test2
        └── test3
            ├── test_test2_test3_secret_2.yaml
            └── test_test2_test3_secret.yaml

5 directories, 8 files

whereas one file is the JSON output of a single KVv2 engine:

cat vkv-export-2022-12-29/secret.yaml
{
  "admin": {
    "sub": "password"
  },
  "demo": {
    "foo": "bar"
  },
  "sub/": {
    "demo": {
      "demo": "hello world",
      "password": "s3cre5",
      "user": "admin"
    },
    "sub2/": {
      "demo": {
        "admin": "key",
        "foo": "bar",
        "password": "password",
        "user": "user"
      }
    }
  }
}

You could .tar.gz those directories and save those encrypted files in a secure fashion.

Restore vkv snapshots

In order to restore a vkv snapshot the snapshot restore command is invoked:

# no KVv2 engines configured
vkv list engines --all --include-ns-prefix
[ERROR] no engines found.

# restore a snapshot
vkv snapshot restore --source vkv-export-2022-12-29
[root] restore engine: secret
[root] writing secret "secret/admin"
[root] writing secret "secret/demo"
[root] writing secret "secret/sub/demo"
[root] writing secret "secret/sub/sub2/demo"
[root] restore engine: secret_2
[root] writing secret "secret_2/admin"
[root] writing secret "secret_2/demo"
[root] writing secret "secret_2/sub/demo"
[root] writing secret "secret_2/sub/sub2/demo"
[root] restore namespace: "sub"
[sub] restore namespace: "sub2"
[sub/sub2] restore engine: sub_sub2_secret
[sub/sub2] writing secret "sub_sub2_secret/admin"
[sub/sub2] writing secret "sub_sub2_secret/demo"
[sub/sub2] writing secret "sub_sub2_secret/sub/demo"
[sub/sub2] writing secret "sub_sub2_secret/sub/sub2/demo"
[sub/sub2] restore engine: sub_sub2_secret_2
[sub/sub2] writing secret "sub_sub2_secret_2/admin"
[sub/sub2] writing secret "sub_sub2_secret_2/demo"
[sub/sub2] writing secret "sub_sub2_secret_2/sub/sub2/demo"
[sub/sub2] writing secret "sub_sub2_secret_2/sub/demo"
[sub] restore engine: sub_secret
[sub] writing secret "sub_secret/admin"
[sub] writing secret "sub_secret/demo"
[sub] writing secret "sub_secret/sub/demo"
[sub] writing secret "sub_secret/sub/sub2/demo"
[sub] restore engine: sub_secret_2
[sub] writing secret "sub_secret_2/sub/demo"
[sub] writing secret "sub_secret_2/sub/sub2/demo"
[sub] writing secret "sub_secret_2/admin"
[sub] writing secret "sub_secret_2/demo"
[root] restore namespace: "test"
[test] restore namespace: "test2"
[test/test2] restore namespace: "test3"
[test/test2/test3] restore engine: test_test2_test3_secret
[test/test2/test3] writing secret "test_test2_test3_secret/sub/sub2/demo"
[test/test2/test3] writing secret "test_test2_test3_secret/admin"
[test/test2/test3] writing secret "test_test2_test3_secret/demo"
[test/test2/test3] writing secret "test_test2_test3_secret/sub/demo"
[test/test2/test3] restore engine: test_test2_test3_secret_2
[test/test2/test3] writing secret "test_test2_test3_secret_2/admin"
[test/test2/test3] writing secret "test_test2_test3_secret_2/demo"
[test/test2/test3] writing secret "test_test2_test3_secret_2/sub/demo"
[test/test2/test3] writing secret "test_test2_test3_secret_2/sub/sub2/demo"

# verify engines have been created
vkv list engines --all --include-ns-prefix
secret
secret_2
sub/sub2/sub_sub2_secret
sub/sub2/sub_sub2_secret_2
sub/sub_secret
sub/sub_secret_2
test/test2/test3/test_test2_test3_secret
test/test2/test3/test_test2_test3_secret_2

Please note, that vkv snapshot restore is an experimental feature, you should always double check vkvs beahviour on a development Vault before running it against a production Vault