filecoin-project / eudico Goto Github PK

This is the main entrypoint for the sharding MVP. I'll try to keep here a checklist with TODOs and issues to be implemented for the MVP.

• Shard Actor

  • Add
  • Join
  • #53
  • #59
  • #61
  • #62
  • #85
  • Cross-net messages handling (#85, #88, #93, #94, #118, #119. #92)
  • #80
  • Unit tests

• Eudico

  • #27
  • #33
  • #35
  • #36
  • #25
  • #24
  • #38
  • #26
  • #39
  • #40
  • #31
  • #32
  • #50
  • Cross-shard Filecoin transaction (#85, #88, #93, #94, #118, #119. #92)
  • Atomic execution protocol (#133, #137, #138)

• Testing / Benchmarks

  • #34
  • #9
  • #84

• Post-MVP backlog

  • #37
  • #54
  • #29
  • #79

• Docs

• Housekeeping

  • #78

The first instance of the protocol could simply assume that each participant is given a share of the secret key privately by a central authority and later on be coupled with the DKG.

Every peer needs to listen at least to its shard, the root chain, and its shard parent chain pubsub topic. Additional, to support cross-shard transactions, peers should have a way to proactively be able to join other shard pubsub topic to share and listen to messages.

We need to handle conveniently the pubsub topics a client is listening to when they join/leave a shard.

When leaving a shard we must:

• Remove exchange and hello handlers for the shard if we no longer need them (i.e. we don't want to sync with the chain).

  • eudico/chain/sharding/exchange.go

    Line 55 in ea618e3

    sh.host.SetStreamHandler(protocol.ID(HelloProtoPrefix+sh.netName), sh.handleHelloStream)

  • eudico/chain/sharding/exchange.go

    Line 78 in ea618e3

    sh.host.SetStreamHandler(protocol.ID(BlockSyncProtoPrefix+sh.netName), srv.HandleStream)

• Remove all shard-related processes

  • Stop syncer
  • Stop sync manager
  • Stop mining (if we are still mining)
  • Close pubsub topics
  • etc. (I may be missing things, this will require a walk through all the code).

See: https://eprint.iacr.org/2021/1375.pdf

When a peer joins a shard that has already mined a number of blocks, it doesn't seem to be syncing the previous state correctly. It successfully start listening to new blocks in the shard's pubsub topic but it throws the following error:

2021-10-05T12:26:56.423+0200    ERROR   chain   chain/sync.go:211       Received block with impossibly large height 163

Which means that it didn't synced with the shard.

We currently share the exchange.Client between the root chain and child shards. This may be related:

In order for peers belonging to the same shard to run the same chain and be able to sync their state, their genesis should be the same. Right now, when a peer joins a shard, it deterministically generates its genesis (or so I thought):

Genesis blocks needs to be handled in a way in which all peers can have the same version. We can do this by:

• Figuring out how peers can generate a genesis block deterministically.
• Generating the genesis when a shard is created and storing it in the state of the shardActor so it is accessible for everyone.

(also the stmgr reverse registry)

When we leave a shard, we probably don't need to keep storing the shard's chain in the chainstore. We should figure out a way to (maybe optionally) remove the chain for a shard when we leave it.

Implement it as part of shard.Close()?

• Support native support of filecoin consensus for Eudico, i.e. we can eudico filcns daemon to start a eudico node running filecoin consensus.
• Support for Filecoin Expected Consensus in subnets.

We'll leave out of scope for now:

• Management of power in subnets.
• Adding sectors, mining and bookkeeping in hierarchical consensus.

Use Frost threshold signing scheme where the set of signers is chosen randomly using a random beacon (which will be drand).

We currently accept every block from a valid miner that has followed the consensus algorithm we see in the shard's pubsub topic. We should only accept blocks that have been mined by a miner with mining rights in the shard (i.e. it is in the list of miners with enough stake in the shard). We need to add this check to validate blocks. This requires:

• Getting the state of the shard actor of the shards parent chain.
• Verifying that the block signer is one of the valid miners.

If an Eudico peer has joined a subnet, it should be able to instantiate and re-sync with the subnet after restarting the client. This is not the case in the current implementation. We'll fix this by:

• Persisting state about the subnets a node is syncing with.
• Automatically sync with all the subnets a node was syncing before restarting.
• Additionally, we should include a cmd for nodes to be able to check the list of subnets they are currently syncing with.

We can approach this in different ways:

• When the eudico process is started and the subnet manager is instantiated, we check that state of the different subnets actors and automatically start a new subnet process to sync with the ones we are part.
• We add a sync command for subnets that can be used to explicitly sync with a subnet that we are part of. This prevents from automatically joining every subnet we are a member of when restarting our node.
• We use a config file to specify the subnets to sync with when we start our node (this approach is quite static, it may be useful to support it, but more dynamic approaches as the aforementioned should also be considered).

Done criteria:

• The customers of this testbed work (ConsensusLab) are able to understand how they would use what we're building to accomplish their goals.
• We have all the tasks that we know for this effort listed in #16
• All the tasks in the issue above have an estimate.

Why important:

• Given the goal of Eudico is to enable ConsensusLab to "engineer prototypes for their new consensus designs", we need to make sure this test effort will provide what they want. We don't want to end up in a state where we have delivered something that is different than what we were expecting. This task hopefully removes the done state ambiguity.
• Enables project planners to figure out resourcing and timelines.

Customer:

• ConsensusLab engineer(s) that will use this work
• Project planners (e.g., @BigLep , @jsoares ) who are figuring out resourcing and timelines.

Notes:

I assume this will be something like:

Implement interface X (see Y for an example)
Create a testground test like Z
...

To start with we could consider using S3 and later on adapt to Filecoin/IPFS.
Following discussion with @adlrocha it may be easier to leverage Filecoin infrastructure to store the data (not use the "usual" Filecoin storage which is very hard to retrieve at the moment). Using Filecoin saves some burden over having to run an IPFS node.

Create a script (e.g. bash) that can be used to replicate the new testnet easily.
Update: use docker container. @jleni to start on this next week.

Add subnet integration tests to eudico following: https://github.com/filecoin-project/eudico/tree/eudico/itests

• Lifecycle of a subnet (add, join, mine, leave, kill).
• Cross-msgs (fund, release, cross-msg)
• Sync with a subnet, send transactions.

Currently it's checking worker keys / sig

As described in e7e2dfe#diff-dee04e9e331bf966a02f3dd193fa9dd9a7085ccba29bdae9db4cbaa290deb7ceR333-R346 in order to allow nodes to join shard topics we need to remove pubsub filters or add new filters that check if the topic follows a subnet naming convention.

Allowing listening to any topic may lead to some security issues.

The purpose of this tracking task is to collect estimates for the Eudico testbed that @magik6k is spearheading to help make ConsensusLab self-sufficient in running consensus experiments in 2021.

This table inlines the task name so it's easier for the estimator to know what issue they are giving an estimate to.

This leverages GitHub's task list capability so can easily see the current status of each issue.

• #1
• #2
• #3
• #4
• #5
• #6
• #7
• #8
• #9
• #10
• #11
• #12
• #13
• #14
• #15

Provide an easy way to interact with shards, spawn new shards, join shards, etc. from CLI.

basic PoW consensus scaling difficulty based on avg past block count in tipsets. Should shed light on what bits of consensus code could use deduplicating

Test the protocol but instead of giving each participants their own share of the keys, use the output from the DKG algorithm.

These are the main tasks for the checkpointing project (project B2).

• Integrate Zondax's work and fix outstanding issues
  • #106
  • #102
  • #98
  • #103
  • #105
  • #104
  • #107
  • #120
• Deploy Eudico testnet
  • #42
  • #46
• Deploy Bitcoin regtest
  • Deploy Bitcoin regtest and run a full node (1 node enough)
  • Write reusable scripts that can be used to replicate the system.
• Distributed Key Generation
  • #43
  • #100
  • #108
  • #109
  • #110
  • #184
• Signing
  • #51
  • #52
  • #44
  • #60
  • #47
  • #111
  • #48
  • #112
  • #113
  • #58
  • (Optional) remove the signature aggregator and automate it using Eudico
• #45
• #55
• Push checkpoint transaction (Taproot+OP_RETURN) to Bitcoin blockchain
  • Push checkpoint transaction (Taproot+OP_RETURN) to Bitcoin blockchain with dummy key
  • Push checkpoint transaction (Taproot+OP_RETURN) to Bitcoin blockchain integrated with DKG
• #117
• Verification
  • Get checkpoint transaction from the bitcoin blockchain
  • #114
  • Verify checkpoint against data from Eudico chain
  • #116
• Measurements and Monitoring
  • #56
  • #115
• Bugs
  • #201

Rosario suggested the following implementation: https://github.com/ZenGo-X/multi-party-ecdsa
He also specified the following: That protocol includes the generation of RSA keys for the servers, which are used to do some computation tricks in the distributed computation of the ECDSA signatures. You would not need those.

Another colleague pointed out this implementation: https://github.com/drand/kyber/tree/master/share/dkg that they use in the drand protocol.

Use Frost where the set S of signers is deterministically chosen to be the first t participants (according to their indexes).

Depends on #3

(not sure if this is the best approach yet)

Would be nice to use the existing node constructor to get a libp2p node with pubsub instead of constructing it manually.

There are a few things in the sharding prototype and Eudico's code structure that may be quite "counterintuitive" and I am not proud of. I am deferring it to a near future so we can move fast now, but I wanted to list them here so we revisit them and don't forget about them. Actually, it may be a good exercise to approach this refactor while we write docs/tutorials for Eudico.

• Figure out a better way to aggregate all genesis and consensus-related code in the same place. This goes in this line. But we should also explore what is suggested in #14, coming up with good interfaces to modularize Eudico s much as possible.
• I am really not proud about this one. In order to prevent redundant block for genesis generation in several places, all this code lives in the shard actor package. The reason for this? The shard actor needs to generate a genesis every time it spawns a new shard, and this shard actor needs to be set in genesis also in newly spawned shards. This leads to an import cycle if we take the genesis generation code to some other package outside the shard-actor. This is going to stay like this for now, but it'll need to change.
• Implement the right TipSetExecutor for tspow consensus. There current implementation is a copy-paste of the delegated consensus':

  eudico/chain/consensus/tspow/compute_state.go

  Line 3 in cd8e6c8

  // FIXME: This is the same implementation as the one from delegcns.

• And this is self-explanatory and HORRIBLE:

  • eudico/chain/consensus/tspow/mine.go

    Line 24 in cd8e6c8

    // NOTE: This is super ugly, but I'll use it

  • eudico/chain/consensus/delegcns/mine.go

    Line 22 in cd8e6c8

    // NOTE: This is super ugly, but I'll use it

Some examples:
Rust
Rust (by the authors)
Go
Go (specifically adapted to Taproot's point encoding.)

• Verify what type of Schnorr's signature Taproot implements and whether this is compatible with the spec (especially with respect to the commitment). Update the spec accordingly.
• Verify that taproot signature verifies correctly in the Bitcoin client.

• Deployment of miner actors in shard and onboard new storage from shards.
• Handling the migration of power tables between shards.
• Handling the on boarding/proof of storage in the system (WindowPoSt, etc.) from shards

All the sharding package and other sharding related processes are full of context.TODO(). In order to effectively leave shards and join new ones will have to handle these contexts correctly.

There seems to be an issue for which after spawning a new shard, the node tries to create a new subshard in the child chain with the same name throwing the following error:

2021-10-01T11:04:46.760+0200    ERROR   sharding        sharding/sub.go:195     HandleIncomingMessages failed for shard {"shardID": "bafkqaclumvzxiu3imfzgi", "err": "duplicate validator for topic /fil/msgs/shard/bafkqaclumvzxiu3imfzgi"}
2021-10-01T11:04:46.760+0200    ERROR   sharding        sharding/sub.go:264     Error creating new shard        {"shardID": "bafkqaclumvzxiu3imfzgi", "err": "duplicate validator for topic /fil/msgs/shard/bafkqaclumvzxiu3imfzgi"}
2021-10-01T11:04:46.760+0200    ERROR   events  events/events_called.go:228     chain trigger (@H 2, triggered @ 7) failed: duplicate validator for topic /fil/msgs/shard/bafkqaclumvzxiu3imfzgi

Is this because the changeState routing in sub.go for the child chain detects that there's a new state change in the shard actor of the child chain when it is constructed, or because we are getting some state from the main chain that we shouldn't be getting? Why then can we be getting the ID of the shard in the child shard (which is exactly the same one of the main shard)?

This error is not priority because it won't affect the behavior of the system at this point, but it may be a consequence of a deeper issue so it is worth fixing it now.

We currently support exclusively delegated consensus in shards, so much of the code in https://github.com/filecoin-project/eudico/blob/sharding/chain/sharding/genesis.go is exclusive for delegated consensus.

We should:

• Extract this code so we have implementation for every consensus we will support in shards.
• Fix hard-coded variables and get them from the actor initialization.
• In

  eudico/chain/sharding/mine.go

  Line 14 in 3b88d82

  func (sh *Shard) mineDelegated(ctx context.Context) error {

  there is a lot of duplicate code from DelegatedMinerCmd. Consider extracting this into a package that both can use (I think it make sense adding it to the consensus package).

If possible get an early estimate on the number of nodes the protocol can scale to.
Is there a cryptographic bottleneck or only a communication bottleneck?

When a state change is detected in the shard actor of a shard, we need to check if that state change involves our peer in some way:

• Either because we joined the shard and we are a new miner.
• Or because we staked to join the shard.

We are subscribed to every state change of the shard actor in the shards we are subscribed in, and we need to filter events conveniently.