unionlabs / union Goto Github PK

We should list which CosmosSDK modules our chain uses; how to call them over RPC/SDK with some examples, and ideally keep this maintained.

Currently, we have the six following public inputs to the circuit:

• The trusted validator set root hash ($F_{r^{2}}$ element)
• The untrusted validator set root hash ($F_{r^{2}}$ element)
• The block header hashed to the field ($F_{r^{2}}$ element)

We could shrink the validator set root hash by truncating the top three bits of the sha256 hash to fit in a single $F_{r}$ element.
This would require upgrading the circuit to truncate the bits, updating its interface, and updating the prover, and the solidity verifier.
We would end up with four public inputs, reducing the evm gas cost by approximately 15k.

We need to generate multiple instances of the ZKP verifier contracts.
For each of them, it would be nice to have few tests with hardcoded block transition and their proofs.
We also need to upload the circuit and it's verifiying/proving keys.

• Unmarshall and verification
• Devnet/Testnet verifier (16 validators)
  • Tests
  • Upload circuit and verifying/proving keys (available through nix run .#download-circuit -- testnet <to_path>
• Mainnet verifier (128 validators)
  • Tests
  • Upload circuit and verifying/proving keys (available through nix run .#download-circuit -- mainnet <to_path>

The eth local devnet is broken since our migration to prysm. The execution layer is no longer producing block, transitively blocking the beacon node from producing blocks as well.

We need to move from the foundry impure build to a nix build for the EVM contracts. The package must consists of the contracts ABI that could be released along the other products. Big advantage here would be reproducible build for the contracts, checkable by anyone willing to verify the deployed bytecode.

Under the Architecture tab, we should have a page dedicated to CometBLS to explain what part of our core tech is.

Probably introduced by https://github.com/unionfi/union/pull/9

uniond does not have description labels set: https://github.com/unionfi/union/pkgs/container/uniond/94094956?tag=v0.5.0-rc1

We should probably add that for discoverability.

I think it'd be good if we structure the root repository such that we have all libraries under a lib directory. forks doesn't make too much sense at the moment, as it doesn't contain git submodules (and thus they're not forks). Ideally, to keep them updated we'd make them git submodules, and add them to the lib directory. serde-utils could be moved there as well then.

depends on #26

The validator section is missing, probably due to a sidebar misconfiguration.

Currently, we have rust/ as the root of the workspace. The top-level Cargo.toml should be moved up to the root of the repo, and the various workspace members can be placed wherever makes sense for them.

It would be great to have a small script that dump a sequence of blocks / validators / signatures from the devnet/testnet and produce a ZKP of each transition so that we can have a complete test suite implemented in EVM (IBC UpdateClient sequence).

We currently build a custom Dockerfile within nix when running the devnet (lodestar image). We can get rid of the imperative way and use the more declarative variant using hercules-ci/arion#169 (comment).

We need to introduce a script to deploys EVM contracts on the devnet (testnet later). The script should compile and list the contracts addresses.

As described, the prysm beacon chain does not support the experimental sync committee light client APIs we need. Hence, we should revert from prysm back to lodestar as it's the only implementation supporting the APIs.

We can generate docs but do not yet host them anywhere. Vercel is a nice option as it offers auth, reviewing and other stuff.

See https://github.com/unionfi/union/actions/runs/5018330673/jobs/8997617944. It's related to the collect-gentx executing when building the devnet-genesis package. The command output the genesis file in stderr (3.8M file...), which may be the reason why nixbuild hang.

Right now we are defining many top-level checks, which is quite noisy. Derivations export a checkPhase, which means that for each derivation, we could verify the integrity of that output without cluttering the global namespace.

The README.md is mainly listing developer-specific stuff. Instead it should describe what Union is; and be a better entryway into other ALL_CAPS files.

Since now all genesis validators are seeds, we should update that in the docs.

The current section is a little bit disorganized and weirdly named. I suggest we rename it to Operators or Infrastructure, and change to contents to divide it between three target audiences:

• Users running on bare metal, ssh-ing into machines (most validators)
• Docker users, geared towards docker-compose (the professional ones)
• NixOS users, basically for just us.

Each should describe how to obtain artifacts, what infrastructure to use, and how to run in testnet, and extend the configuration for mainnet production.

When running the devnet-genesis check and others depending on it - it's possible to generate and use keys that aren't present in the generated genesis file.

Error: failed to get genesis app state from config: account union1rjc32uesdxwp6deen0u2cx2u98u922sxxgap82 balance not in genesis state: map[union12mj7hhjydergxntzrhl08edf8ghnl0xyvzvegw:{Address:union12mj7hhjydergxntzrhl08edf8ghnl0xyvzvegw Coins:100000000000000000000000000stake} union1fyhvpf7zam8cl33r44w0h9yzjnl0uv4zfep3gv:{Address:union1fyhvpf7zam8cl33r44w0h9yzjnl0uv4zfep3gv Coins:100000000000000000000000000stake} union1jk9psyhvgkrt2cumz8eytll2244m2nnz4yt2g2:{Address:union1jk9psyhvgkrt2cumz8eytll2244m2nnz4yt2g2 Coins:10000000000000000000000000stake} union1n4v8mkhz2exfl9zs38aaegwdvjkwcqsrp2vstv:{Address:union1n4v8mkhz2exfl9zs38aaegwdvjkwcqsrp2vstv Coins:100000000000000000000000000stake} union1rm23j4guh6hqwm5pzj3hx7ncwywam777kxmjvf:{Address:union1rm23j4guh6hqwm5pzj3hx7ncwywam777kxmjvf Coins:100000000000000000000000000stake}]

We need to find a way to deterministically generate app-keys for the union devnet.

buf is a service for storing and sharing protobuf definitions. This is convenient if you have a public API or microservices. For our usecase it is simply extra work and not worth the added maintenance.

If a PR alters the docs, we should do a preview deployment to vercel and show the preview URL in a comment. This should be possible by checking if nixbuild.net built the docs, or if it already had it in cache. If built, the docs should be fetched and uploaded.

This can be achieved with something like

run: |
  nix copy --from nixbuild.net $(type -p ./#docs)
  vercel build
  vercel deploy > url.txt
working-directory: docs  
continue-on-error: true    
run: |
  if test -f url.txt; then
    create comment
  fi

With this, we'll get links to previews only if the docs changed. A downside is that the nix store can remove cached docs if they haven't been built in a while. Alternatively we can also only run the preview URL if the docs directory is changed; but that however means that if docs depend on anything outside of the docs directory, we might build broken docs without knowing it.

The current README is generated by ignite (which we do not even use). It should probably explain:

• What it is
• Link to releases
• How to develop on it.

Membership verification verifies if the counterparty light client stores a certain data. There are several data types that we need to verify.
Implementation

Membership verification payload is the following:

 verifyMembershipInnerPayload struct {
    Height           exported.Height `json:"height"`
    DelayTimePeriod  uint64          `json:"delay_time_period"`
    DelayBlockPeriod uint64          `json:"delay_block_period"`
    Proof            []byte          `json:"proof"`
    Path             exported.Path   `json:"path"`
    Value            []byte          `json:"value"`
}

• Proof is just arbitrary bytes. It is provided by the relayer and directly passed to the wasm client.
• Path is the key that the light client expects a certain element to be stored under. (Eg. connection state is stored under the key connections/<conn-id>.)
• Value is the data that we expect the counterparty client to store. This is given to the wasm light client in protobuf encoded format. So it is the wasm light client's job to make sure to convert it to correct encoding before verification.

Data types and their formats in the EVM are the following:

Roughly, the implementation steps should be:

1. Determine the data type by looking at Path.
2. Convert the data to the target format if necessary.
3. Calculate the slot number based on the data types. Assert if this slot number matches the given slot number in the proof.
4. Verify if the data is really stored.

Currently, we're publishing different tags for each architecture we support. Instead, we should use containers multiplatform support, and update the manifest step in CI.

Both as static binary and as a docker image

#8 (comment)

It'd be convenient for NixOS users to have a module to run a node, something like

imports = [ union.nix ]

union = {
   bundle = bundle;
   home = /path/to/home;
   config = { ... } // config.toml override, merged with existing/default
   app = { ... } // app.toml override, merged with existing/default
}

For this; there are a few changes that have to be made:

1. unionvisor init should have a flag to exit with status 0 if a home is already initialized.
2. unionvisor should get the capability to merge toml configurations ({a: 1, b: 2}, {b: 3}) -> {a: 1, b: 3}.
3. Bundles have to be updated with the latest release.

Since we have JSON files for devnet configurations, it would be nice to format them.

EVM tools to generate contracts based on ABI files are very limited. In order to simplify the code generation process, we should expose a dummy contract having the sole purpose of providing the types that must be exposed for generation.

This readme can be linked in the code to also serve as the entrypoint doc comment.

It should describe:

• Unionvisors model.
• What it expects and creates on the machine.
• How to build it.
• How to run it with NixOS if we add the module

Following #18, we no longer rely on prysm for the beacon chain, we can drop all it's package from the repo.

We currently share a single nixbuild ssh key for access, payment, and configuration. Instead, we should regenerate keys, where we securely preserve 1 key as root and have user-specific keys for @cor and @hussein-aitlahcen

Figure out where we can cut gas by optimizing the UpdateClient and ReceivePacket handlers.

For the former, we currently apply:

• smallest possible set for the public input of the ZK circuit
• ClientState using ETH ABI instead of Protobuf
• optimized, evm slot-friendly ConsensusState structure using ETH ABI insteand of Protobuf
• inline merkle root for block header

More optimizations (probably v2):

• move everything in the ZKP circuit (would require non-trivial upgrade of the circuit, not sure it is worth the effort)
• use ZK friendly hashing like MiMC in place of sha256 (require the above to be implemented first)

For packets, the relayer could bulk them and provide a ZKP of aggregated membership proof verification (circuit yet to be done).

Unionpd should have a README.md, describing what it does and how to build the binary.

We currently run nix run .\#gen-proto in order to generate go code based on .proto definitions without the need for Docker images or Makefiles, but due to buf accessing the Buf Schema Registry under the hood, we haven't executed buf generate offline inside of a derivation yet.

This would be a nice addition as we can use it to validate that the .proto and .pb.go sources are in sync as part of nix build .\#uniond.

We need to deploy unionpd somewhere for relayers to generate ZKP in the testnet. I am personally deploying an instance over prover.cryptware.io:443. The instance is behind cloudflare and require TLS to be enabled on the client.

I like how the cosmos-sdk docs are versioned. https://docs.cosmos.network/main If you go to an older version, there's a link at the top that you can go to the latest one. If you go to main, there's a link to the latest release. It appears that they are also using Docusaurus, so we can adopt a similar approach.

I'm open to suggestions for other docs versioning schemes, feel free to make suggestions in the comments

The wasm light client can be invalid because of several factors such as having floating point operations. The validity of the client should be tested in the CI. An easy and legit way to do this is to specify the binary in the genesis and see if it failed. Or we can also just write an integration test in Go code.

We currently do loads of fixup commits in PRs because of spelling errors due to two reasons:

• nix fmt is great, but misses out on spellchecking. nix build ./#checks.{system}.spellcheck is verbose.
• cspell isn't great at checking rust and golang code, and errors on variable names, function names etc. Ideally we would ignore code and only check doc comments.

It'd be nice if tree-fmt supported spellchecking, and more specifically would set regexes per language to ensure we only check what we care about.

As agreed in Matrix we're naming the prover 'Galois'.

Make sure that all references to unionpd are updated, including the packages hosted in ghcr.

We should implement our own ethereum-verifier library that verifies header updates, storage proofs, account state proofs, etc. There are few libraries out there which defines the ethereum consensus spec types. We will just need to follow the implementation that is nicely documented in the official consensus-specs.

These tests were omitted because of some flakiness within nix. We should re-enable them asap.