attestantio / dirk Goto Github PK

Hey! I'm wondering how to handle pruning of the database for information on slashing protection since dirk doesn't provide any options for it.

First, I'd like to know whether there's a reason for not having such an option? If not, up to which point is it safe to prune the database then?

Thanks!

Problem

The codahale/hdrhistogram repo has been transferred under the github HdrHstogram umbrella with the help from the original author in Sept 2020 (new repo url https://github.com/HdrHistogram/hdrhistogram-go). The main reasons are to group all implementations under the same roof and to provide more active contribution from the community as the original repository was archived several years ago.

The dependency URL should be modified to point to the new repository URL. The tag "v0.9.0" was applied at the point of transfer and will reflect the exact code that was frozen in the original repository.

If you are using Go modules, you can update to the exact point of transfer using the @v0.9.0 tag in your go get command.

go mod edit -replace github.com/codahale/hdrhistogram=github.com/HdrHistogram/[email protected]

Performance Improvements

From the point of transfer, up until now (mon 16 aug 2021), we've released 3 versions that aim support the standard HdrHistogram serialization/exposition formats, and deeply improve READ performance.
We recommend to update to the latest version.

Looks like the last release for dirk is from 2021-12-10: https://github.com/attestantio/dirk/releases. Is there another planned release with all the latest commits?

Hi attestant team!

Overview
I wanted to start a conversation around adding an isSlashable api point which receives as input the same params as the signing api but instead just returns true/ false for if the attestation/ proposal is slashable.

Why?
Specifically for SSV.Network we use a remote signer that's connected to the node, during the consensus phase one of the validations steps is to check a proposed attestation/ proposal are not slashable before the other peers decided on it.
It would be helpful to have that supported by dirk, after which, we can build an adapter for it to connect to the ssv node.

How?
The api end point should look like the signing endpoint, difference is the result it returns.
Ideally this can also be made into an EIP to extend on 3030.

Would love your thoughts/ comments

After a new validator got activated on mainnet, Vouch tried to access the account on Dirk, which failed to obtain the account from ethdo. Listing all accounts in ethdo, the account that Dirk failed to obtain was visible there.

Here is the log from Dirk:

{"level":"trace","service":"fetcher","impl":"mem","path":"account-01/validators/007","time":"2021-02-26T08:59:59+01:00","message":"Fetching account"}
{"level":"trace","service":"fetcher","impl":"mem","path":"account-01/validators/007","time":"2021-02-26T08:59:59+01:00","message":"Fetching wallet"}
{"level":"trace","service":"fetcher","impl":"mem","path":"account-01/validators/007","time":"2021-02-26T08:59:59+01:00","message":"Wallet found in cache"}
{"level":"warn","service":"fetcher","impl":"mem","path":"account-01/validators/007","error":"no account with name \"validators/007\"","time":"2021-02-26T08:59:59+01:00","message":"Account not found"}
{"level":"warn","service":"signer","impl":"standard","error":"failed to obtain account by name: no account with name \"validators/007\"","result":"denied","time":"2021-02-26T08:59:59+01:00","message":"Did not obtain account; denied"}

After a simple restart of Dirk, it was able to find the account and sign messages with it.

I could be reading the code incorrectly, but I don't think the current slashing protection covers the "surrounded" vote case.

Here is the tail of the log

{"level":"info","version":"1.0.2","time":"2021-03-30T17:44:04Z","message":"Starting dirk"}
SIGILL: illegal instruction
PC=0x107f8a4 m=0 sigcode=2

goroutine 0 [idle]:
runtime: unknown pc 0x107f8a4
stack: frame={sp:0x7fff3c084c80, fp:0x0} stack=[0x7fff3b886528,0x7fff3c085560)
00007fff3c084b80: e7a278e38f6be830 dc54b77391b667d2
00007fff3c084b90: 0000000000000000 0000000000000000
00007fff3c084ba0: 0000000000000000 0000000000000000
00007fff3c084bb0: 0000000000000000 0000000000000000
00007fff3c084bc0: 0000000000000000 0000000000000004
00007fff3c084bd0: 0000000000004000 0000000000000000
00007fff3c084be0: 0000000000000007 0000000000000000
00007fff3c084bf0: 0000000000000072 00000000023cd008
00007fff3c084c00: 0000000000003fff 0000000000004030
00007fff3c084c10: ffffffffffffff88 0000000000000100
00007fff3c084c20: 0000013000000401 0000000000000020
00007fff3c084c30: 0000000000000004 000000720000007b
00007fff3c084c40: 000000770000007c 0000000000000000
00007fff3c084c50: 0000000000000005 00007fbc59493c40
00007fff3c084c60: 0000000000004000 00000000000003ff
00007fff3c084c70: ffffffffffffff88 0000000000000200
00007fff3c084c80: <000000000107f882 00000000023e1570
00007fff3c084c90: 0000000001f50208 00007fff3c084ce8
00007fff3c084ca0: 0000000000000004 000000000107f501
00007fff3c084cb0: 00000000023dd590 000000000105ad0e
00007fff3c084cc0: 00000000000001ff 00000000023dd590
00007fff3c084cd0: 0000000000000800 0000000000000000
00007fff3c084ce0: 0000000000000002 0000000000000000
00007fff3c084cf0: 0000000000000000 0000000000000000
00007fff3c084d00: 0000000000000000 0000000000000000
00007fff3c084d10: 0000000002f112ed 00007fff3c084e10
00007fff3c084d20: 00007fff3c084d68 0000000000479816 <runtime.pcdatavalue+134>
00007fff3c084d30: 000000000176e2f0 0000000001ed58c0
00007fff3c084d40: 000000000009320b 0000000000000004
00007fff3c084d50: 00007fff3c085200 0000000001f50210
00007fff3c084d60: 0000000000000008 54b165b8181ac726
00007fff3c084d70: 1ad4fdb87d39fbb4 55c5243a9a0aca61
runtime: unknown pc 0x107f8a4
stack: frame={sp:0x7fff3c084c80, fp:0x0} stack=[0x7fff3b886528,0x7fff3c085560)
00007fff3c084b80: e7a278e38f6be830 dc54b77391b667d2
00007fff3c084b90: 0000000000000000 0000000000000000
00007fff3c084ba0: 0000000000000000 0000000000000000
00007fff3c084bb0: 0000000000000000 0000000000000000
00007fff3c084bc0: 0000000000000000 0000000000000004
00007fff3c084bd0: 0000000000004000 0000000000000000
00007fff3c084be0: 0000000000000007 0000000000000000
00007fff3c084bf0: 0000000000000072 00000000023cd008
00007fff3c084c00: 0000000000003fff 0000000000004030
00007fff3c084c10: ffffffffffffff88 0000000000000100
00007fff3c084c20: 0000013000000401 0000000000000020
00007fff3c084c30: 0000000000000004 000000720000007b
00007fff3c084c40: 000000770000007c 0000000000000000
00007fff3c084c50: 0000000000000005 00007fbc59493c40
00007fff3c084c60: 0000000000004000 00000000000003ff
00007fff3c084c70: ffffffffffffff88 0000000000000200
00007fff3c084c80: <000000000107f882 00000000023e1570
00007fff3c084c90: 0000000001f50208 00007fff3c084ce8
00007fff3c084ca0: 0000000000000004 000000000107f501
00007fff3c084cb0: 00000000023dd590 000000000105ad0e
00007fff3c084cc0: 00000000000001ff 00000000023dd590
00007fff3c084cd0: 0000000000000800 0000000000000000
00007fff3c084ce0: 0000000000000002 0000000000000000
00007fff3c084cf0: 0000000000000000 0000000000000000
00007fff3c084d00: 0000000000000000 0000000000000000
00007fff3c084d10: 0000000002f112ed 00007fff3c084e10
00007fff3c084d20: 00007fff3c084d68 0000000000479816 <runtime.pcdatavalue+134>
00007fff3c084d30: 000000000176e2f0 0000000001ed58c0
00007fff3c084d40: 000000000009320b 0000000000000004
00007fff3c084d50: 00007fff3c085200 0000000001f50210
00007fff3c084d60: 0000000000000008 54b165b8181ac726
00007fff3c084d70: 1ad4fdb87d39fbb4 55c5243a9a0aca61

goroutine 1 [syscall]:
runtime.cgocall(0xf6d740, 0xc0008bfda8, 0xc0008bfdc8)
/usr/local/go/src/runtime/cgocall.go:133 +0x5b fp=0xc0008bfd78 sp=0xc0008bfd40 pc=0x4283cb
github.com/herumi/bls-eth-go-binary/bls._Cfunc_blsInit(0xf600000005, 0x0)
_cgo_gotypes.go:417 +0x4d fp=0xc0008bfda8 sp=0xc0008bfd78 pc=0x8ee42d
github.com/herumi/bls-eth-go-binary/bls.Init(0x5, 0xa, 0x40)
/go/pkg/mod/github.com/herumi/[email protected]/bls/bls.go:63 +0x40 fp=0xc0008bfe00 sp=0xc0008bfda8 pc=0x8f0750
github.com/wealdtech/go-eth2-types/v2.InitBLS(0x200, 0x40)
/go/pkg/mod/github.com/wealdtech/go-eth2-types/[email protected]/bls.go:22 +0x2e fp=0xc0008bfe28 sp=0xc0008bfe00 pc=0x8fd84e
main.main()
/app/main.go:124 +0x246 fp=0xc0008bff88 sp=0xc0008bfe28 pc=0xf62f56
runtime.main()
/usr/local/go/src/runtime/proc.go:203 +0x1fa fp=0xc0008bffe0 sp=0xc0008bff88 pc=0x45c5ea
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1373 +0x1 fp=0xc0008bffe8 sp=0xc0008bffe0 pc=0x48ce51

goroutine 102 [select]:
go.opencensus.io/stats/view.(*worker).start(0xc00017a180)
/go/pkg/mod/[email protected]/stats/view/worker.go:276 +0x100
created by go.opencensus.io/stats/view.init.0
/go/pkg/mod/[email protected]/stats/view/worker.go:34 +0x68

rax 0x7fc
rbx 0x23dd590
rcx 0x1f50210
rdx 0xc999e990f3f29c6d
rdi 0x107f4b0
rsi 0x7fff3c084ce0
rbp 0x107f501
rsp 0x7fff3c084c80
r8 0x3
r9 0x1f50208
r10 0x23cb010
r11 0xc999e990f3f29c6d
r12 0x4
r13 0x7fff3c084ce8
r14 0x1f50208
r15 0x23e1570
rip 0x107f8a4
rflags 0x10206
cs 0x33
fs 0x0
gs 0x0

Operating System: Linux Mint 20.1
Kernel: Linux 5.4.0-67-generic
Architecture: x86-64

Distributed validator technology (DVT) solutions such as SSV and Obol offer the possibility to shard an existing key (an account from an hd wallet) into multiple shares (that they then distribute amongst different "operators").
The outcome is similar to a distributed wallet in Dirk, but comes with a lot more custom infrastructure to adopt (SSV token for example).

Having a similar feature for Dirk would leverage Dirk's slashing protection to offer similar benefits to the native distributed wallet, but on "standard" hd wallets.

It could greatly increase Vouch/Dirk's adoption as:

1. we could migrate existing validators to such "sharded" account in Dirk to gain security/protection while remaining the sole "operator", without the whole infrastructure requirements that come with full-fledge solutions such as SSV/Obol
2. unlike a distributed wallet that only Dirk supports so far (ie, Dirk's lock-in), a "standard" wallet can be easily ported to other solutions so there is less reluctance to adopt it (knowing we could easily change if unhappy)

See attestantio/vouch#97 for the discussion that led to this feature request.

Please add trace config to https://github.com/attestantio/dirk/blob/v1.2.0/docs/configuration.md

Exiting a validator from the network that is managed by multiple dirk instances requires an admin ip to be specified in the configuration file of all the dirk instances. Currently this is not specified anywhere in the docs.

How to generate deposit data with multiple dirk instances is also not currently mentioned in the documentation, it might be useful to include it.

Hello!

Currently, if I use s3 wallet store, I can't control a bucket name/location via dirk. We can pass only password to go-eth2-wallet-store-s3 library. This library itself also can't point to a predefined bucket, it determines the name of the bucket via AWS_ACCESS_KEY_ID hash.

I see this problem with that approach:

• We need to add permission s3:CreateBucket with Resource:* to dirk SA (or we need to precalculate hash for AWS_ACCESS_KEY), or change that after bucket was created
• We can't rotate AWS_ACCESS_KEY_ID or use metadata service for getting that because if it will be changed we will create a new empty bucket (so we need manually copy all data from old bucket to a new one)

It would be a good if:

• We already have location field in store structure (which is used for file system type store), and we can use that to point out an endpoint for aws-s3-sdk
• If we could use the location field for that, we would be able to control not only bucket name, but also the whole URL (endpoint in AWS SDK), so we would be able to use any s3-compatible services as s3 store.

Does Dirk work with Teku? Can you show how to use this tool with Teku and Prysm for Medella?

Currently docker hub has a latest tag which tracks master, and a 1.0.2 tag. But it does not have a 1.1.0 tag for the last release.

Main idea - to be able to use Majordomo url for gsm/asm credential files instead of simple local file.
Example:

s3:
  endpoint: <endpoint>
  region: <region>
  bucket: <bucket>
  credential: <majordomo>
  

Google Cloud and AWS both support to load credential via metadata service without any interaction if IAM roles/sa are attached to VM.

Current it's impossible to use because dirk pass only credentialPath, and majordomo strictly use only that too (gsm, asm).
I think it could be a good idea to make the default way getting credentials from metadata service work

Dirk supports asm through majordomo, but as it is currently Dirk doesn't have an explicit case for creating the asm confidant. Vouch does support the asm confidant and this particular snippet of code is missing in Dirk's initMajorDomo.

Dirk's initMajordomo:

Would love to see docs or example setups explaining differences, nuiances and tradeoffs between dirk, web3signer, charon and keyvault!

• https://www.attestant.io/posts/introducing-dirk/
• https://docs.web3signer.consensys.net/HowTo/Store-Keys-Vaults/Use-Hashicorp
• https://docs.obol.tech/docs/int/key-concepts
• https://github.com/bloxapp/key-vault

Many command line tools provide a version flag, usually with -v or --version. Could dirk be updated to support this as well?