ecadlabs / signatory Goto Github PK

If a signing request is received, and signatory does not recognize the kind or op signatory should log an error including the actual bytes that are unrecognized.

By logging the unrecognized op/kind, we inform the signatory operator in diagnosing the issue.

We want integration tests for common signing scenarios.

Start by targeting the file-based vault, and add the others in due course.

Integration tests should be written in go.

When a vault is configured, but the corresponding PKH is not listed in the tezos hash.

Current behavior when a vault contains tz1U8bnXWZqLQ8d3ZMMa1HztzxP24rykZk8z but that PKH is not listed in the signatory configuration tezos: hash:

Current behavior:

Current behavior when a vault contains tz1fNiSGiq7X7Br1H5pgPJWJSGouBaRAw8Qb and the tezos: hash also contains that same PKH:

Folloiwng list output has two entries.

• tz1fNiSGiq7X7Br1H5pgPJWJSGouBaRAw8Qb is in the signatory configuration tezos: hash, and is displayed as Active:true
• tz2QyuDVHo7dVCjec27JnakGZNZRFUE7qDAP is not in the config, but exists only in the vault.

$ ./signatory-cli list -c ledger_signatory.yaml 
INFO[0000] Initializing vault                            vault=ledger vault_name=ledger
Public Key Hash:    tz1fNiSGiq7X7Br1H5pgPJWJSGouBaRAw8Qb
Vault:              Ledger
ID:                 bip32-ed25519/44'/1729'/0'/0'
Active:             true
Allowed Operations: [block endorsement generic]
Allowed Kinds:      []

Public Key Hash:    tz2QyuDVHo7dVCjec27JnakGZNZRFUE7qDAP
Vault:              Ledger
ID:                 secp256k1/44'/1729'/0'/1'
Active:             false

Proposed behavior:

Add a property to the output called status, remove the *DISABLED* text.

$ signatory-cli -c ./examples/yubihsm.yaml list
INFO[0000] Initializing vault                            vault=yubihsm vault_name=yubi
Public Key Hash:    tz1U8bnXWZqLQ8d3ZMMa1HztzxP24rykZk8z
Status:           FOUND_NOT_CONFIGURED
Vault:              YubiHSM
ID:                 cbf7

Current behavior when a vault contains tz1U8bnXWZqLQ8d3ZMMa1HztzxP24rykZk8z and the tezos: hash also contains that same PKH:

$ signatory-cli -c ./examples/yubihsm.yaml list
INFO[0000] Initializing vault                            vault=yubihsm vault_name=yubi
Public Key Hash:    tz1U8bnXWZqLQ8d3ZMMa1HztzxP24rykZk8z
Status:           ACTIVE
Vault:              YubiHSM
ID:                 cbf7
Allowed Operations: [block endorsement]
Allowed Kinds:      []

Add a file based backend so that signatory will read the tezos secret keys from the .tezos-client directory, and load said keys so that signatory can sign operations.

Support for non-encrypted and encrypted keys is required. A method to provide the decryption passphrase for encrypted keys will be required.

Hi

After setting Signatory up (with Azure)
I'm getting the following error:

Error signing request: xxxx is not listed in config

I can get the pubkey hash using curl
curl 127.0.0.1:6733/keys/***

Am I missing something?

Logging of the operation bytes is attractive for audit/investigation purposes, but might be too much data for some operators. Lets add a top level yaml configuration setting to log operations called log_payloads.

This will cause signatory to log all payloads as an Info level log message once before attempting to sign the request. Useful if we get a request that is denied by policy, the operator can inspect the operation payload to see its purpose.

A test that generates a new key. Goal is to make sure that signatory continues to be able to import keys into yubihsms.

Hardware requirement, a yubihsm (no emulators please).

• Imports it into a yubihsm
• Verifies that signatory and get the expected public key back from yubihsm
• Verify that signatory can sign an operation with the new key
• Remove the key from the yubihsm (tidy up, we don't want to fill the yubihsm with junk keys over time)

Coverage report is produced with the commands:

go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out -o coverage.html

Presently Signatory does not offer any form of authentication for incoming requests.

The tezos-client does have an authentication scheme, and we want to support that scheme so that tezos-client (which the baker and endorser also uses) can use Signatory as a remote signer while also securing the connection.

See: https://tezos.gitlab.io/user/key-management.html#secure-the-connection for details.

See: https://www.robustperception.io/exposing-the-software-version-to-prometheus

Labels;

• branch
• goversion
• revision (git sha)
• version (release tag)

Hook up to CI system

Setup CodeClimate or other static analysis job for Signatory

Add the ability to configure signatory to only sign specific type of operations.

Here are the 3 operations type that would be useful to support initially.

• block
• endorsement
• generic

By default signatory should only support block and endorsement operations and filter ou regular transactions

Update all log lines to use fields.

• msg (arbitrary informational text)
• pkh (tz3Tm6UTWmPAZJaNSPAQNiMiyFSHnRXrkcHj) - PKH stands for "Public Key Hash"
• vault (azure)
• vault_name (ex vaultname91)
• op ("endorsement", block, generic)
• kind (ballot, transaction, reveal etc..)

While we are doing this work, we should also refactor some errors into their own type/values over liberal use of fmt.Errorf()

Log examples; We have logs as follows

INFO[0473] Signing for key: tz3Tm6UTWmPAZJaNSPAQNiMiyFSHnRXrkcHj
INFO[0473] Signing operation with Azure vaultname91 vault
INFO[0476] Signed endorsement successfully

Using fields so they present as below will make logs much more useful, especially when ingested to a log store.

INFO[0473] msg=Requesting signing operation vault=azure vault_name=vaultname91 pkh=tz3Tm6UTWmPAZJaNSPAQNiMiyFSHnRXrkcHj 
INFO[0476] msg=Signed endorsement successfully vault=azure vault_name=vaultname91 pkh=tz3Tm6UTWmPAZJaNSPAQNiMiyFSHnRXrkcHj 

Ability to specify one or more tezos account addresses as approved transfer destinations

Key use cases:

1. baker to hot wallet
2. Exchange to users registered/known/approved wallet addresses
3. hot wallet to delegators

Adding chain_id= into logs will be useful, as signatory can sign requests for different testnets. Faucet keys work on all Tezos testnets. But the chain_id differs, so it's convenient to have that in logs.

Examples of log lines would benefit from chain_id=

time="2020-04-23T18:26:52Z" level=info msg="Requesting signing operation" op=endorsement pkh=tz1Kj83aLgNPRqEFfPvHpJGu5ws6cssrAuzm vault=File vault_name=local_file_keys
time="2020-04-23T18:26:52Z" level=info msg="About to sign raw bytes" op=endorsement pkh=tz1Kj83aLgNPRqEFfPvHpJGu5ws6cssrAuzm raw=029caecab930cd6c829669793b90f76a861cf7cff2aefab3c4780843f54bfc62c89a710d3f000005954b vault=File vault_name=local_file_keys
time="2020-04-23T18:26:52Z" level=info msg="Signed endorsement successfully" op=endorsement pkh=tz1Kj83aLgNPRqEFfPvHpJGu5ws6cssrAuzm vault=File vault_name=local_file_keys

When a dupe operation with the same level is received, the watermark implementation prints an error to the log:

ERRO[3045] Error signing request: Not safe to sign

Improve this error to include:

• Vault name
• PKH
• Level
• ChainID

Convert the existing metric for vault_sign_request_duration_microseconds to a histogram.

Current metrics presents as follows;

vault_sign_request_duration_microseconds_sum{vault="Yubi"} 83.59501900000001
vault_sign_request_duration_microseconds_count{vault="Yubi"} 1

Please also add the vault_name label for the new histogram metric.

Make a simple load test so we can capture latency data for each backend, and compare performance.

We should use prometheus/grafana to present the data as a graph.

Add promteheus metrics to count requests, latency, and errors

Propose to add labels for:

• algorithm
• backend (Ex: googlekms, azure, yubihsm etc..)

Add support for https://cloud.google.com/hsm/ backend

• Implement signing
• Document Set up
• Document key import process and limitations
• Integration Tests

I would consider adding the support for SoftHSM, could be an economic and fast test environment for Signatory (or even a free solution for small bakers)

• Run unit tests
• Build and push docker images to http://hub.docker.com/r/ecadlabs/signatory
• Add code climate SAST and code coverage

Ability to specify what methods on the smart contract the operation is invoking

Add support for using a Ledger hardware wallet, running the Tezos Baking ledger app, for baking.

Source code for the ledger apps (both regular wallet and baking app) is available here: https://github.com/obsidiansystems/ledger-app-tezos

The Tezos client already supports ledger integration directly, but there is value and demand to support the ledger in Signatory, and use the remote signer capability of the tezos-client instead.

The Ledger device can have two "apps" installed, the regular wallet, that can sign any operation, but it prompts the user for approvals, and the Baker app, that only signs endorsements and blocks (and possibly ballots??). To install the baker app, use the Ledger Live Desktop app, and enable experimental mode. Then go to manager and search for Tezos. You should see two options as seen in the screenshot below:

Related resources:

https://medium.com/@obsidian.systems/tezos-baking-v2-0-0-for-the-ledger-nano-s-testing-tezos-amendment-proposals-c10aced06824
https://www.tezos.org.sg/guide-to-using-the-tezos-node/
TestPad Scripts for Signatory: https://ecadlabs.ontestpad.com/project/23/

Dalphanet has new operation tags.

Docker image is: tezos/tezos:dalpha-release

Example Logs from signatory

time="2020-08-04T20:54:44Z" level=error msg="tezos: unknown or unimplemented operation tag: 111" pkh=tz1No1K2BygWXDp5Qs87KxNYaZmUNH3wxfJH raw=03b38e95aaa47c14f217356d1c0551926c5c77c951fc17568d60e462f245a956916b00229af7dc83c2130bd13ba9ed4f05534423384d2dec09cd0280ade2040000f4d4d983a6ae060c89cc60830a90b7dadd53cb42df583cefd3c41fe0e8e59d716f00229af7dc83c2130bd13ba9ed4f05534423384d2dae0ace02e8d9a905ce0280bd9b8a5400f4d4d983a6ae060c89cc60830a90b7dadd53cb42df583cefd3c41fe0e8e59d71582a0000002100f4d4d983a6ae060c89cc60830a90b7dadd53cb42df583cefd3c41fe0e8e59d71
time="2020-08-04T20:54:44Z" level=error msg="Error signing request: tezos: unknown or unimplemented operation tag: 111"

Investigate supporting the new NetHSM device as a Signatory backend: https://www.nitrokey.com/products/nethsm

When signatory starts up and is configured for azure, attempt to connect or "ping" the API to test that the credentials work.

This will be helpful to users configuring for the first time. Nice to know if authentication works without actually trying to sign something.

Add flag to print version and build info to the signatory and signatory-cli commands.

Testing/verifcation note;

The output from the version flag should be consistent with the signatory_build_info prometheus build metric.

Would you mind adding ARM docker container or Binaries to the release?

Thanks

Ability to determine a group of humans whose approval is required for key operations (transfers or voting).

To be determined:

• communication channel (e.g. Telegram, Slack, SMS)
• handling operation timeouts: how long to wait for a response before the operation is rejected due to lack of response (ultimately this could be configured by user; could initially start with a default setting)

Add Azure Key Vault.

• Document key generation options
• Document key import options
• Document supported algorithms (Understood to be Secp256k1 and P-256)

Add support to use AWS' CloudHSM as a backend signer.
https://aws.amazon.com/cloudhsm/

• AWS imlementation
• AWS CloudHSM Setup documentation
• Integration test against AWS backend
• Permanent test-net baker setup using AWS as a backend

Add a config validation method to help new users with trivial config errors.

Ability to allow transactions only during certain times of day; if transfer is within this window, allow it

• Find out if we an delete https://hub.docker.com/r/ecadlabs/signatory so anyone doing docker pull gets an error instead of a stale docker image.
• Remove publishing from CI/CD to docker hub
• Make sure publishing to ghcr works properly
• Make sure anon (non-authenticated) docker pulls work from ghcr
• Add note to release notes that docker hub is gone
• Grep all docs to update refs to docker hug to be ghcr

Problem Statment

If a signatory config has multiple backends setup, that contains the secret for an address, it is not possible to specify which backend or backends should be used, and (if more than one) in what order.

User Story:

As a signatory operator with multiple vaults configured in my signatory.yaml file, I want to set preference on an address in the signatory config so that it will only use specified backends for signing.

If I have an address such as tz3MhmeqpudUqEX8PYTbNDF3CVcnnjNQoo8N and I know that the private key for this address is only in backend yubi_abc I want to "pin" my configuration so that signatory only uses that backend and no others.

Is restricting what type of operation the signer performs (via magic byte) planned for this implementation?

While doing a technical review with yubico staff, they told us we could support YubiKeys as a HSM for users that only have two or three keys.

It would work a bit differently to the yubiHSM support. It's an interesting idea, as the cost of a yubikey is far less than a yubiHSM. It is also a lot cheaper than a Ledger device. Making it an attractive & inexpensive HSM solution for small bakers. The Yubkikey arguably has a better form factor than a ledger for this type of use.

Some unknowns;

• Does the YubiKey require human interaction for signings? (endorsements, bakes)
• Does the YubiKey require human intervention when powered on?
• What would provisioning of the key for this type of use look like?
• What would private key import look like?
• What would integration with the yubikey look like?

Add YubiHSM support

Goal: to operate signatory with each backend so that we have continious first hand experience of operating signatory as a baker with every backend. This allows us to find any operational corner cases, regressions or oddities first, before users of signatory has a chance to encounter such issued.

For each current and next Tezos testnet, we want to run one baker, with a modest amount of stake, to operate using each Signatory backend.

• Publish a table on signatory website; Make each tz1 address a link to tzkt indexer. This table should be updated when a testnet is retired, and also when a new testnet is added.

Implement liveness and readiness probes along with the appropriate signal handling.

Include an integration test that covers situation where daemon is sent signal to shutdown while a signature request is in flight. Daemon should set readiness to false, but wait for in flight operations to drain.

As part of the same job, I suggest we put metrics on its own HTTP port, and put the liveness readiness checks on that port under the /healthz/ slug. This will keep these concerns separate from the main offerings of the service.

Ability to set a limit to the number of transactions allowed in a certain time period; user can specify both the number of transactions and the applicable time period

Ability to add smart contract methods (extracted from params) to an approve_list or deny_list

This feature will allow smart contract owners granular control on the types and properties of operations they sign using signatory.

To implement this, signatory will need to have the ability to parse and inspect Michelson. The operator will want to write some form of rules that apply to the entry points.

We can consider this issue blocked until we have the ability to parse Michelson into an AST in go.

Blocked by #191

Counters and histograms for;

• signing operations
• labels
  • vault
  • vault_name
  • op
  • opkind
• errors
  • backend error types (azure, use http codes? 401,403 etc...)

Create a companion grafana dashboard to help operators make the most of these metrics easily
Documentation on use, prometheus quickstart config snippets

Ability to determine a maximum and/or minimum amount of Tez tokens in a particular transaction; if the tez amount in the transaction is withing the max/min range, it should be signed (allowed), otherwise, it should be denied.

Design work required;

• single transactions Versus batch transactions (further grooming needed, let's make some test cases).

When a batch operation is sent, we need to define the treatment. Do we sum up the tez in all transactions in the batch and evaluate that? Or do we check if the individual transactions are within the range, and reject if any are outside?

Doc how to take a keypair from ~/.tezos-client and import it into the YubiHSM2 device.

This is targeted at existing bakers that want to keep their address and want to move key storage to a HSM.