paritytech / polkadot Goto Github PK

Parachain candidate receipts submitted to the chain contain a list of egress_queue_roots -- these are trie roots of message lists being sent to other parachains by that candidate.

We want to impose some validity checks on the lists:

1. That no parachain is routed to which doesn't exist
2. That the parachain doesn't route to self
3. That the list of egress queue roots is in ascending order by ParaId.
4. That no empty trie roots are included. They should instead be omitted. I'm pretty sure this is available by some constant in the substrate tree somewhere.

For instance, the list of egress queue roots should have length no greater than n_parachains - 1. This is length is prepended under parity-codec, so we can skip decoding any further when this is invalid.

Probably needs alterations in parity-codec to allow for some kind of decoding context.

All genesis block allocations stemming from the Polkadot auction is expressed as an Ethereum address (i.e. a portion of the hash of an ECDSA SECP256k1 public key). This is not compatible with the account-level crypto in Polkadot which is based on SRML's Ed25519 curve. A module should be created in order to allow users to claim their allocations.

It must:

• Provide a storage item which can bee initialised with and hold all users unclaimed allocations (initialisation should be done via the usual JSON chain config).
• Provide a Callable endpoint allowing an account to provide a valid signature from an Ethereum account with a claim, whose payload is the origin AccountId to erase that claim and credit said AccountId with the DOTs of the claim.

In addition to this module, there must be an amendment to the transaction validity logic of the runtime to ensure that calls to this extrinsic need not be funded *providing that the signature which is passed is made by a valid Ethereum account with a current claim.

version 0.3.1-6112021-x86_64-linux-gnu
does not reproduce 100% of the time
Here are logs showing a node hanging. I then (docker) restarted the node and we see the logs keep going all good.

It looks like the same issue that affected parity-rpca few minutes ago.

2019-01-18 09:58:45 Purposefully dropping 39014 ; reason: Bad("Peer sent us a packet with invalid format")
2019-01-18 09:58:45 Banned PeerId(Qmf8bKGnVEoKYcy6zx7Qf2wwLansMU1C4JkxnkeQ4jjZUA)
2019-01-18 09:58:45 Purposefully dropping 39019 ; reason: Bad("Peer sent us a packet with invalid format")
2019-01-18 09:58:45 Banned PeerId(QmWKWG18UWzwStH1fVZ1DVmWNbmzq4qf91465mB55skDnd)
2019-01-18 09:58:46 Purposefully dropping 39021 ; reason: Bad("Peer sent us a packet with invalid format")
2019-01-18 09:58:46 Banned PeerId(QmWepQQVQmXmBBhoHJApyvaeMpQenH7GpJWTU5CWkfoecd)
2019-01-18 09:58:46 connection error: decode error: i/o error: I/O error: Connection reset by peer (os error 104)
2019-01-18 09:58:48 Syncing, target=#191529 (18 peers), best: #33357 (0x4cac…8d46)
2019-01-18 09:58:48 Rejected connection from disabled peer: PeerId(QmWzYU5X1NpFrprD1YZF5Lcj9aE5WF4QEg5FpvQx5XGWG7)
2019-01-18 09:58:49 Rejected connection from disabled peer: PeerId(QmWepQQVQmXmBBhoHJApyvaeMpQenH7GpJWTU5CWkfoecd)
2019-01-18 09:58:51 Rejected connection from disabled peer: PeerId(QmWepQQVQmXmBBhoHJApyvaeMpQenH7GpJWTU5CWkfoecd)
2019-01-18 09:58:53 Syncing, target=#191530 (18 peers), best: #33357 (0x4cac…8d46)
2019-01-18 09:58:54 Rejected connection from disabled peer: PeerId(QmWepQQVQmXmBBhoHJApyvaeMpQenH7GpJWTU5CWkfoecd)
2019-01-18 09:58:56 Purposefully dropping 39112 ; reason: Bad("Peer sent us a packet with invalid format")
2019-01-18 09:58:56 Banned PeerId(QmU1BycVbomYmoDPhjWgHujSoz6UmLjuYokr4VVGKfY6nt)
2019-01-18 09:58:57 Rejected connection from disabled peer: PeerId(QmWepQQVQmXmBBhoHJApyvaeMpQenH7GpJWTU5CWkfoecd)
2019-01-18 09:58:58 Syncing, target=#191531 (18 peers), best: #33357 (0x4cac…8d46)
2019-01-18 09:59:00 Parity ·:· Polkadot
2019-01-18 09:59:00   version 0.3.1-6112021-x86_64-linux-gnu
2019-01-18 09:59:00   by Parity Technologies, 2017, 2018
2019-01-18 09:59:00 Chain specification: Alexander
2019-01-18 09:59:00 Node name:  Crash Burn
2019-01-18 09:59:00 Roles: AUTHORITY
2019-01-18 09:59:02 Best block: #33357
2019-01-18 09:59:02 Local node address is: /ip4/0.0.0.0/tcp/30336/p2p/QmRN5eGCVQNMgqD...q7siRvP6AEZK
2019-01-18 09:59:02 Using authority key 5GgLbUrn3LbuJnZfYA6Ls21k4aYNAcwinaDrehMVB33WkvSA
2019-01-18 09:59:02 Idle (0 peers), best: #33357 (0x4cac…8d46)
2019-01-18 09:59:03 connection error: decode error: i/o error: I/O error: Connection reset by peer (os error 104)
2019-01-18 09:59:03 Purposefully dropping 8 ; reason: Bad("Peer sent us a packet with invalid format")
2019-01-18 09:59:03 Banned PeerId(QmQWZhtBatdb82kaDSQGGzNSFM4F7jTpLTN2v3KUVyKZf7)
2019-01-18 09:59:03 Purposefully dropping 13 ; reason: Bad("Peer sent us a packet with invalid format")
2019-01-18 09:59:03 Banned PeerId(QmWepQQVQmXmBBhoHJApyvaeMpQenH7GpJWTU5CWkfoecd)
2019-01-18 09:59:04 Purposefully dropping 31 ; reason: Bad("Peer sent us a packet with invalid format")
2019-01-18 09:59:04 Banned PeerId(QmXhCuaDFYXXgw6pE7tjLFAx8g4PnhKVSNNbdm89E9JxRn)

Running purge-chain I noticed polkadot appears to default to the share/Substrate/ folders (rather than a polkadot-specific one):

polkadot --dev purge-chain
Are you sure to remove "/home/ben/.local/share/Substrate/chains/dev/db"? (y/n)

Probably related to paritytech/substrate#1465

Version 0.2.15-3720d74-x86_64-linux-gnu
The node crashed after ~24 hours

2018-09-11 07:13:17 Parity ·:· Polkadot
2018-09-11 07:13:17   version 0.2.15-3720d74-x86_64-linux-gnu
2018-09-11 07:13:17   by Parity Technologies, 2017, 2018
2018-09-11 07:13:17 Chain specification: Krumme Lanke
2018-09-11 07:13:17 Node name: Acid Burn
2018-09-11 07:13:17 Roles: AUTHORITY
2018-09-11 07:13:17 Best block: #1323709

2018-09-12 06:16:26 BFT agreement error: Message sender 3d5db678afac4a72b36a25f998732911d854238d0071667c5993f5edc903116f is not a valid authority.
2018-09-12 06:16:27 BFT agreement error: Message sender 3d5db678afac4a72b36a25f998732911d854238d0071667c5993f5edc903116f is not a valid authority.

====================

stack backtrace:
   0:     0x55935fe8db2c - backtrace::backtrace::trace::ha18aa6ab54e5a876
   1:     0x55935fe8cb02 - <backtrace::capture::Backtrace as core::default::Default>::default::h4b1e8887c11facf6
   2:     0x55935fe8cb78 - backtrace::capture::Backtrace::new::h9650079cf787b5a3
   3:     0x55935f74f760 - substrate_cli::panic_hook::panic_hook::h3b42c0eac81fca14
   4:     0x55935f74f478 - core::ops::function::Fn::call::hdad2bc9320d93013
   5:     0x55935ffbe883 - std::panicking::rust_panic_with_hook::he4c3a67f6258a8f9
                        at libstd/panicking.rs:515
   6:     0x55935faa5406 - std::panicking::begin_panic::h43f6d7d030843f98
   7:     0x55935fab07a1 - ring::agreement::agree_ephemeral::h8f671873f8174a10
   8:     0x55935fb44bc4 - <futures::future::chain::Chain<A, B, C>>::poll::h165aff8d34002d8a
   9:     0x55935fb64dcb - <futures::future::chain::Chain<A, B, C>>::poll::h95f05b4b843d58f8
  10:     0x55935fb6f538 - <futures::future::chain::Chain<A, B, C>>::poll::hc5503a98c6150948
  11:     0x55935faa35e8 - <futures::future::and_then::AndThen<A, B, F> as futures::future::Future>::poll::h415fbfb3c2449ef1
  12:     0x55935fbed30b - <futures::future::map::Map<A, F> as futures::future::Future>::poll::headc5df01320a280
  13:     0x55935fbed69b - <futures::future::map::Map<A, F> as futures::future::Future>::poll::hedd70b616c3d397c
  14:     0x55935fc3dc6b - <futures::future::map_err::MapErr<A, F> as futures::future::Future>::poll::hcfeab84f7b6d577e
  15:     0x55935fbe7906 - <futures::future::map::Map<A, F> as futures::future::Future>::poll::h57f0ef49a333020a
  16:     0x55935fc157ba - <libp2p_core::upgrade::apply::UpgradeApplyFuture<C, U, Maf> as futures::future::Future>::poll::h15d03021237b1f85
  17:     0x55935fb6b051 - <futures::future::chain::Chain<A, B, C>>::poll::hb48188e680ae1193
  18:     0x55935faa3668 - <futures::future::and_then::AndThen<A, B, F> as futures::future::Future>::poll::h8a2a227adfbf4679
  19:     0x55935fb5683d - <futures::future::chain::Chain<A, B, C>>::poll::h5457174e39ff311b
  20:     0x55935faa3588 - <futures::future::and_then::AndThen<A, B, F> as futures::future::Future>::poll::h0f73a8cc10531f29
  21:     0x55935fb4519a - <futures::future::chain::Chain<A, B, C>>::poll::h185c4fcf25108a88
  22:     0x55935faa3638 - <futures::future::and_then::AndThen<A, B, F> as futures::future::Future>::poll::h7a2e8626dfaed67e
  23:     0x55935fc55586 - <libp2p_core::connection_reuse::ConnectionReuseDial<T, D, M> as futures::future::Future>::poll::h275e27db89f86091
  24:     0x55935fbe9140 - <futures::future::map::Map<A, F> as futures::future::Future>::poll::h983cf9280cce7688
  25:     0x55935faef203 - <libp2p_transport_timeout::TokioTimerMapErr<InnerFut> as futures::future::Future>::poll::hdc4df65e936842d3
  26:     0x55935fbe7560 - <futures::future::map::Map<A, F> as futures::future::Future>::poll::h57969ae0efa3f797
  27:     0x55935fbf77e8 - <futures::future::map::Map<A, F> as futures::future::Future>::poll::hee8d79230727f90e
  28:     0x55935fc3bf3f - <futures::future::map_err::MapErr<A, F> as futures::future::Future>::poll::h702d76dca6d894d9
  29:     0x55935fb5a593 - <futures::future::chain::Chain<A, B, C>>::poll::h65533c7600372e53
  30:     0x55935faa35b8 - <futures::future::and_then::AndThen<A, B, F> as futures::future::Future>::poll::h2db59248e1acf7d9
  31:     0x55935fb63037 - <futures::future::chain::Chain<A, B, C>>::poll::h8c4485741198a223
  32:     0x55935faa35c8 - <futures::future::and_then::AndThen<A, B, F> as futures::future::Future>::poll::h3e11973aa0c5e8f0
  33:     0x55935fb5dfdf - <futures::future::chain::Chain<A, B, C>>::poll::h7e046121850794aa
  34:     0x55935fbf6758 - <futures::future::then::Then<A, B, F> as futures::future::Future>::poll::h9f97db5d9cb80b3b
  35:     0x55935faa2229 - <libp2p_core::swarm::SwarmEvents<T, F, H> as futures::stream::Stream>::poll::hd798e4e2b8003735
  36:     0x55935fbcf490 - <futures::stream::for_each::ForEach<S, F, U> as futures::future::Future>::poll::hf56650476d71b6a6
  37:     0x55935fbf5dee - <futures::future::select_all::SelectAll<A> as futures::future::Future>::poll::h87e972ae0393706d
  38:     0x55935fb60c2c - <futures::future::chain::Chain<A, B, C>>::poll::h87c8d2c6d4bce5ff
  39:     0x55935fc3c49e - <futures::future::map_err::MapErr<A, F> as futures::future::Future>::poll::h77675111afe2a287
  40:     0x55935fc65381 - futures::task_impl::std::set::h41d1a2e71aefbe57
  41:     0x55935fc5bcb7 - <std::thread::local::LocalKey<T>>::with::h434807c0a17c731f
  42:     0x55935faea286 - <tokio::executor::current_thread::Entered<'a, P>>::block_on::h22a55d6f12a791de
  43:     0x55935fc5c3e9 - <std::thread::local::LocalKey<T>>::with::ha2ab375c862d8531
  44:     0x55935fc5cac9 - <std::thread::local::LocalKey<T>>::with::hede92d2d25fd28a5
  45:     0x55935fc5c739 - <std::thread::local::LocalKey<T>>::with::he3bb956a37fd1d50
  46:     0x55935fc5c110 - <std::thread::local::LocalKey<T>>::with::h9fce941262b48e2d
  47:     0x55935fc2e1fd - tokio::runtime::current_thread::runtime::Runtime::block_on::h725d4e9adacda691
  48:     0x55935fbafcf1 - std::sys_common::backtrace::__rust_begin_short_backtrace::h6aee95d9b4aaa435
  49:     0x55935faa5434 - std::panicking::try::do_call::hf360a7f1f04b7e62
  50:     0x55935ffe1c49 - __rust_maybe_catch_panic
                        at libpanic_unwind/lib.rs:105
  51:     0x55935fbe0fc3 - <F as alloc::boxed::FnBox<A>>::call_box::he287d19513816397
  52:     0x55935ffbbe6a - <alloc::boxed::Box<alloc::boxed::FnBox<A, Output$u3d$R$GT$$u20$$u2b$$u20$$u27$a$GT$$u20$as$u20$core..ops..function..FnOnce$LT$A$GT$$GT$::call_once::h75e539106a648d39
                        at /checkout/src/liballoc/boxed.rs:650
                         - std::sys_common::thread::start_thread::h88a639c99862a9f5
                        at libstd/sys_common/thread.rs:24
  53:     0x55935ffbf3f5 - std::sys::unix::thread::Thread::new::thread_start::h7d7a420a78cfa84d
                        at libstd/sys/unix/thread.rs:90
  54:     0x7f92f530f6b9 - start_thread
  55:     0x7f92f4e2f41c - clone
  56:                0x0 - <unknown>

Thread '<unnamed>' panicked at 'explicit panic', /root/.cargo/git/checkouts/rust-libp2p-98135dbcf5b63918/304e9c7/protocols/secio/src/handshake.rs:465

This is a bug. Please report it at:

    https://github.com/paritytech/polkadot/issues/new

This would be as a substrate-consensus-parachain crate. Rough implementation strategy is to embed a polkadot client into the crate, in a way that can introduce finality and fork-choice based on the relay chain.

Main considerations:

• The approach to chain synchronization may be somewhat different than for freestanding chains.
• The database of the two clients should be unified, perhaps via an alternative Backend implementation

All-in-all, this should be pretty similar to the set required for substrate.

• Post message to another parachain ( #96 )
• Upload balance to an account ( now upwards messages #274 )
• WASM Sandboxing
• Crypto primitives (Ed25519, perhaps blake2 hashing, secp256k1)

Note that there is nothing about reading or interacting with storage. This is because parachains that require storage proofs will import all needed storage as a part of the BlockData which proves validity of the parachain candidate.

Depends on paritytech/substrate#1125

The goal will be for authorities not to vote to finalize (or build upon!) chains where our local piece of data is not stored (#51).

There are some pieces of data which are of arbitrary length returned from the parachain WASM:

• The head-data
• The messages going out to other parachains. Message fees should be based on a two-dimensional system: the number of messages and the total amount of routed data.

If the parachain does not have enough to pay for these fees then no further decoding work should be done.

From https://github.com/paritytech/polkadot#using-docker
The Using Docker link
https://github.com/paritytech/polkadot/blob/master/_includes/doc/docker.adoc
is not found, getting 404 page error.

In the v0.3.0 branch, some feature such as `--ws-external' appear to be missing.

when running command "cargo build", this error occurs. I want to know how to solve it, thanks!

And accept incoming messages as argument.
Validators will need to fetch the extrinsic data that proves the incoming messages before processing.

Bug:

The command line docker run --rm -it chevdor/polkadot:latest polkadot --name "PolkaDocker" crashes before executing.

docker run --rm -it chevdor/polkadot:latest polkadot --name "PolkaDocker"
error: Found argument 'polkadot' which wasn't expected, or isn't valid in this context

USAGE:
    parity [SUBCOMMAND]

Expected:

Have the docker image running.

Workaround:

If polkadot argument is removed then it executes correctly.
docker run --rm -it chevdor/polkadot:latest --name "PolkaDocker"

Env:
Parity version Parity-Ethereum/v2.1.0-nightly-4eab867-20180810/x86_64-linux-gnu/rustc1.28.0

OS: Ubuntu 16.0 LTS
Polkadot Version: parity-polkadot v0.2.16

I attempting to sync with the network, I have come across this error which seems to be spurious:

2018-09-16 11:32:56 Syncing, target=#1407541 (8 peers), best: #720271 (9a4e…16ac)
2018-09-16 11:33:01 Syncing, target=#1407542 (8 peers), best: #720326 (9864…7ac0)

====================

stack backtrace:
   0:     0x55efaa020d1c - backtrace::backtrace::trace::h648cb776e96afd21
   1:     0x55efaa01ffd2 - <backtrace::capture::Backtrace as core::default::Default>::default::h82fbb8ebfa0d4c13
   2:     0x55efaa020048 - backtrace::capture::Backtrace::new::h0e4d4489bcf4d061
   3:     0x55efa9904590 - substrate_cli::panic_hook::panic_hook::he795dc17f95aaf04
   4:     0x55efa99042a8 - core::ops::function::Fn::call::hf5571a2e2ac0abe2
   5:     0x55efaa1607c3 - std::panicking::rust_panic_with_hook::hd04e46cfa0b30fca
                        at libstd/panicking.rs:481
   6:     0x55efaa160369 - std::panicking::continue_panic_fmt::ha2ede74723cdc31c
                        at libstd/panicking.rs:391
   7:     0x55efaa160265 - rust_begin_unwind
                        at libstd/panicking.rs:326
   8:     0x55efaa1b9ecb - core::panicking::panic_fmt::hf3e826a5d8e962e1
                        at libcore/panicking.rs:77
   9:     0x55efa9c73855 - core::result::unwrap_failed::hbefe5b1ea074eb92
  10:     0x55efa9c63988 - <libp2p_core::nodes::swarm::Swarm<TTrans, TMuxer, TUserData>>::handle_node_reached::h9177527572f9e620
  11:     0x55efa9c469a8 - <libp2p_core::nodes::swarm::Swarm<TTrans, TMuxer, TUserData> as futures::stream::Stream>::poll::h3d9e903ce9766a3f
  12:     0x55efa9cd4273 - <substrate_network_libp2p::swarm::Swarm<TUserData> as futures::stream::Stream>::poll::h4cc0344f8b44f3b0
  13:     0x55efa9c87dc9 - <substrate_network_libp2p::service_task::Service as futures::stream::Stream>::poll::h423ee0a9c933e716
  14:     0x55efa9c62d23 - <futures::stream::poll_fn::PollFn<F> as futures::stream::Stream>::poll::ha1c09aea94383516
  15:     0x55efa9cbb698 - <futures::stream::for_each::ForEach<S, F, U> as futures::future::Future>::poll::hffdc90c84bf4ed86
  16:     0x55efa9d38efc - <futures::future::select_all::SelectAll<A> as futures::future::Future>::poll::hbdf8f8d43a8f6e81
  17:     0x55efa9db5dee - <futures::future::chain::Chain<A, B, C>>::poll::hfa5930913e069d9b
  18:     0x55efa9d1d74e - <futures::future::map_err::MapErr<A, F> as futures::future::Future>::poll::hf4a898820ab3af90
  19:     0x55efa9c64ac3 - <futures::future::select::Select<A, B> as futures::future::Future>::poll::h1f2390fb29806220
  20:     0x55efa9d7357b - <futures::future::map::Map<A, F> as futures::future::Future>::poll::h31852609c2bb8da7
  21:     0x55efa9d1b317 - <futures::future::map_err::MapErr<A, F> as futures::future::Future>::poll::h3825263f06eb1961
  22:     0x55efa9d2b4ba - futures::task_impl::std::set::h8cd2b7c93c0a0a43
  23:     0x55efa9c49e0c - <std::thread::local::LocalKey<T>>::with::ha2491f8334b5fab2
  24:     0x55efa9cc7d5c - <tokio::executor::current_thread::Entered<'a, P>>::block_on::h54c534cd82b1681e
  25:     0x55efa9c4a0cf - <std::thread::local::LocalKey<T>>::with::hbfffdc1bca2b229e
  26:     0x55efa9c57772 - tokio::runtime::current_thread::runtime::Runtime::block_on::hdc8b4dcb9173d4d0
  27:     0x55efa9d019ba - std::sys_common::backtrace::__rust_begin_short_backtrace::h062c4ca5bf867efd
  28:     0x55efa9c49bf5 - std::panicking::try::do_call::h01a2373f021281c9
  29:     0x55efaa182ce9 - __rust_maybe_catch_panic
                        at libpanic_unwind/lib.rs:102
  30:     0x55efa9d2a9d0 - <F as alloc::boxed::FnBox<A>>::call_box::h1e81c4e7ae73e534
  31:     0x55efaa17437a - <alloc::boxed::Box<(dyn alloc::boxed::FnBox<A, Output$u3d$R$GT$$u20$$u2b$$u20$$u27$a$RP$$GT$$u20$as$u20$core..ops..function..FnOnce$LT$A$GT$$GT$::call_once::hba950b0efee6dead
                        at liballoc/boxed.rs:656
                         - std::sys_common::thread::start_thread::h851ae18c961caf42
                        at libstd/sys_common/thread.rs:24
  32:     0x55efaa160fe5 - std::sys::unix::thread::Thread::new::thread_start::hd5d0edde91f39da6
                        at libstd/sys/unix/thread.rs:90
  33:     0x7f14da0e56b9 - start_thread
  34:     0x7f14d9c0541c - clone
  35:                0x0 - <unknown>

Thread '<unnamed>' panicked at 'State inconsistency: invalid reach attempt cancel: ()', libcore/result.rs:1009

Worried as I am unsure if this could lead to slashing of my stake if I become a validator

The signatures should be checked in the parachains module of the runtime.
This requires a modification to the statement table for producing full attestations.

I need to check if related to the panic from yesterday.

2018-09-13 16:31:01 Finishing future for proto [100, 111, 116] with 6259 => Ok(())
2018-09-13 16:31:03 Idle (22 peers), best: #1360886 (29e3…1a92)

====================

stack backtrace:
   0:        0x1063695ae - backtrace::backtrace::trace::hf022a0b7b978402b
   1:        0x10636a32c - _$LT$backtrace..capture..Backtrace$u20$as$u20$core..default..Default$GT$::default::h4cbcf54b445d9524
   2:        0x10636a3ad - backtrace::capture::Backtrace::new::hf1084303f4122ad8
   3:        0x1058ccca9 - substrate_cli::panic_hook::panic_hook::hd944ef01822bb5a1
   4:        0x1058cc9bb - core::ops::function::Fn::call::h6de083a2eafa5227
   5:        0x1064b9e58 - std::panicking::rust_panic_with_hook::h24ec12d6e9a2769c
   6:        0x105f776e7 - std::panicking::begin_panic::h002243c25c1a4af0
   7:        0x105fae945 - ring::agreement::agree_ephemeral::h0630a36df3c51acd
   8:        0x10606c78d - _$LT$futures..future..chain..Chain$LT$A$C$$u20$B$C$$u20$C$GT$$GT$::poll::h2ada5404d2d54366
   9:        0x10607b0e4 - _$LT$futures..future..chain..Chain$LT$A$C$$u20$B$C$$u20$C$GT$$GT$::poll::h6b73d4d5aaa76054
  10:        0x1060814c4 - _$LT$futures..future..chain..Chain$LT$A$C$$u20$B$C$$u20$C$GT$$GT$::poll::h82db4695db5b95e8
  11:        0x105f851cd - _$LT$futures..future..and_then..AndThen$LT$A$C$$u20$B$C$$u20$F$GT$$u20$as$u20$futures..future..Future$GT$::poll::h7b5fdfedb1d8560e
  12:        0x105f10f7e - _$LT$futures..future..map..Map$LT$A$C$$u20$F$GT$$u20$as$u20$futures..future..Future$GT$::poll::hae94060a32789d31
  13:        0x105f0e8ee - _$LT$futures..future..map..Map$LT$A$C$$u20$F$GT$$u20$as$u20$futures..future..Future$GT$::poll::h84176090ffbd8d62
  14:        0x105f5c3d1 - _$LT$futures..future..map_err..MapErr$LT$A$C$$u20$F$GT$$u20$as$u20$futures..future..Future$GT$::poll::hee31dd87ff508268
  15:        0x105f0a05b - _$LT$futures..future..map..Map$LT$A$C$$u20$F$GT$$u20$as$u20$futures..future..Future$GT$::poll::h2fe3bbe6b74306d1
  16:        0x1060c565e - _$LT$libp2p_core..upgrade..apply..UpgradeApplyFuture$LT$C$C$$u20$U$C$$u20$Maf$GT$$u20$as$u20$futures..future..Future$GT$::poll::hdfec29cf55a1b275
  17:        0x106088a1f - _$LT$futures..future..chain..Chain$LT$A$C$$u20$B$C$$u20$C$GT$$GT$::poll::h997c10e2d23d2563
  18:        0x105f851ed - _$LT$futures..future..and_then..AndThen$LT$A$C$$u20$B$C$$u20$F$GT$$u20$as$u20$futures..future..Future$GT$::poll::h886f335b29cde49e
  19:        0x10608d3ef - _$LT$futures..future..chain..Chain$LT$A$C$$u20$B$C$$u20$C$GT$$GT$::poll::ha8fbc61e702a06ef
  20:        0x105f8512d - _$LT$futures..future..and_then..AndThen$LT$A$C$$u20$B$C$$u20$F$GT$$u20$as$u20$futures..future..Future$GT$::poll::h3ad9ee0b59b97111
  21:        0x10608fecf - _$LT$futures..future..chain..Chain$LT$A$C$$u20$B$C$$u20$C$GT$$GT$::poll::had7aed5dd6ecca13
  22:        0x105f8508d - _$LT$futures..future..and_then..AndThen$LT$A$C$$u20$B$C$$u20$F$GT$$u20$as$u20$futures..future..Future$GT$::poll::h13dc28ec498825b5
  23:        0x105f71800 - _$LT$libp2p_core..connection_reuse..ConnectionReuseDial$LT$T$C$$u20$D$C$$u20$M$GT$$u20$as$u20$futures..future..Future$GT$::poll::h3e045bcc080d5dee
  24:        0x105f0e623 - _$LT$futures..future..map..Map$LT$A$C$$u20$F$GT$$u20$as$u20$futures..future..Future$GT$::poll::h7fdaa6b547af20c2
  25:        0x105f1895a - _$LT$libp2p_transport_timeout..TokioTimerMapErr$LT$InnerFut$GT$$u20$as$u20$futures..future..Future$GT$::poll::hc942cdce852d4f41
  26:        0x105f0b2d3 - _$LT$futures..future..map..Map$LT$A$C$$u20$F$GT$$u20$as$u20$futures..future..Future$GT$::poll::h4bf14793599240a1
  27:        0x105f13fcd - _$LT$futures..future..map..Map$LT$A$C$$u20$F$GT$$u20$as$u20$futures..future..Future$GT$::poll::h88744fd554cceacd
  28:        0x105f57fdf - _$LT$futures..future..map_err..MapErr$LT$A$C$$u20$F$GT$$u20$as$u20$futures..future..Future$GT$::poll::h4d0fcf201b503071
  29:        0x10609bf49 - _$LT$futures..future..chain..Chain$LT$A$C$$u20$B$C$$u20$C$GT$$GT$::poll::hf067f2cca215c19b
  30:        0x105f8528d - _$LT$futures..future..and_then..AndThen$LT$A$C$$u20$B$C$$u20$F$GT$$u20$as$u20$futures..future..Future$GT$::poll::hb0000bb2685b1a10
  31:        0x10608eee9 - _$LT$futures..future..chain..Chain$LT$A$C$$u20$B$C$$u20$C$GT$$GT$::poll::hac8ce0812941e93d
  32:        0x105f8522d - _$LT$futures..future..and_then..AndThen$LT$A$C$$u20$B$C$$u20$F$GT$$u20$as$u20$futures..future..Future$GT$::poll::h9c0f244b61b97624
  33:        0x10608e079 - _$LT$futures..future..chain..Chain$LT$A$C$$u20$B$C$$u20$C$GT$$GT$::poll::ha90d628aad74199b
  34:        0x105f3814d - _$LT$futures..future..then..Then$LT$A$C$$u20$B$C$$u20$F$GT$$u20$as$u20$futures..future..Future$GT$::poll::hf4271e1503ee1b59
  35:        0x105fa239a - _$LT$libp2p_core..swarm..SwarmEvents$LT$T$C$$u20$F$C$$u20$H$GT$$u20$as$u20$futures..stream..Stream$GT$::poll::hfd3c51becde9683d
  36:        0x105f1a387 - _$LT$futures..stream..for_each..ForEach$LT$S$C$$u20$F$C$$u20$U$GT$$u20$as$u20$futures..future..Future$GT$::poll::he3d650dcafa0c681
  37:        0x105f36dec - _$LT$futures..future..select_all..SelectAll$LT$A$GT$$u20$as$u20$futures..future..Future$GT$::poll::hf1b58aa88a81f2b2
  38:        0x1060788ec - _$LT$futures..future..chain..Chain$LT$A$C$$u20$B$C$$u20$C$GT$$GT$::poll::h5f0ab18383c67aeb
  39:        0x105f59b62 - _$LT$futures..future..map_err..MapErr$LT$A$C$$u20$F$GT$$u20$as$u20$futures..future..Future$GT$::poll::h67e195dd5727fad1
  40:        0x105f8132f - futures::task_impl::std::set::hea022b3c2b539a78
  41:        0x105f3fd5c - _$LT$std..thread..local..LocalKey$LT$T$GT$$GT$::with::hd77de44e9545cb9d
  42:        0x105fef074 - _$LT$tokio..executor..current_thread..Entered$LT$$u27$a$C$$u20$P$GT$$GT$::block_on::h1387abf66f8aab2d
  43:        0x105f3f83e - _$LT$std..thread..local..LocalKey$LT$T$GT$$GT$::with::hb8a3f976145f2b49
  44:        0x105f3f253 - _$LT$std..thread..local..LocalKey$LT$T$GT$$GT$::with::h60960562a31e613a
  45:        0x105f3fb5a - _$LT$std..thread..local..LocalKey$LT$T$GT$$GT$::with::hcc835fe9b42de7e5
  46:        0x105f3f597 - _$LT$std..thread..local..LocalKey$LT$T$GT$$GT$::with::h7019c98b8edf8267
  47:        0x105f7f9e0 - tokio::runtime::current_thread::runtime::Runtime::block_on::h6280c4b84f0ecda5
  48:        0x105f1f684 - std::sys_common::backtrace::__rust_begin_short_backtrace::h1f23d2f10a2ce0cf
  49:        0x105f77727 - std::panicking::try::do_call::hbb7ed9f034b10fa0 (.llvm.780789652720880005)
  50:        0x1064c6e4e - __rust_maybe_catch_panic
  51:        0x105f2e0bb - _$LT$F$u20$as$u20$alloc..boxed..FnBox$LT$A$GT$$GT$::call_box::h26347e0ab72f505e
  52:        0x1064b8b87 - std::sys_common::thread::start_thread::h83e199693f3f6175
  53:        0x1064a32c8 - std::sys::unix::thread::Thread::new::thread_start::h736ad41dfb087ebc
  54:     0x7fff7972f660 - _pthread_body
  55:     0x7fff7972f50c - _pthread_start

Thread '<unnamed>' panicked at 'explicit panic', /Users/will/.cargo/git/checkouts/rust-libp2p-98135dbcf5b63918/304e9c7/protocols/secio/src/handshake.rs:465

This is a bug. Please report it at:

    https://github.com/paritytech/polkadot/issues/new

Parachain candidates that are included in the runtime runtime/src/parachains.rs are passed as AttestedCandidate. The "attestation" is a set of signatures on statements of the block. Currently, we're passing the entire validator public keys in with the signatures, which is a huge overhead:

Since we know the authority keys already, we should instead just pass in an index with the signature instead of the full key. The generating code should be altered correspondingly.

Alternatively, the polkadot-statement-table crate could be changed to use a usize instead of the AuthorityId type here:

This is arguably more elegant but might have harder to follow fallout.

I have a troubles with vendoring v0.3.0 sources.

Reproducing:

[nix-shell:~/devel/polkadot]$ cargo vendor
    Updating git repository `https://github.com/paritytech/substrate`
    Updating git repository `https://github.com/paritytech/yamux`
 Downloading core-foundation-sys v0.5.1
 Downloading stdweb-internal-macros v0.2.2
 Downloading discard v1.0.4
 Downloading core-foundation v0.5.1
 Downloading bitvec v0.8.0
 Downloading vcpkg v0.2.6
 Downloading security-framework v0.2.1
 Downloading once_cell v0.1.6
 Downloading redox_syscall v0.1.42
 Downloading stdweb-derive v0.5.1
 Downloading schannel v0.1.14
 Downloading stdweb-internal-runtime v0.1.2
 Downloading hyper v0.12.14
 Downloading stdweb v0.4.10
 Downloading security-framework-sys v0.2.1
error: failed to sync

Caused by:
  found duplicate version of package `ws v0.7.9` vendored from two sources:

        source 1: https://github.com/tomusdrw/ws-rs#4baef2dc
        source 2: registry https://github.com/rust-lang/crates.io-index

emielvanderhoek/substrate-alexander:v0.3.2 docker image

Docker container crashed with following output to terminal. Restarting the container resulted in immediate crash (upon connecting with alexander peers) with same output, same banned peer and error message. The container was running on a fairly high end laptop with ubuntu 18.04 os. Ironically the same docker image running on two older laptops (one positively ancient), on the same home network, have not crashed but one is stuck at block 200,968 while the other is keeping up at block 201,420 currently. Terminal output on both running nodes shows frequent banned peer messages. My network has been exhibiting higher than usual latency intermittently (thank you Xfinity).

2019-01-23 19:34:14 Banned PeerId(QmXnBNEYRJQqx5HeGfFyvs3UvTraQHEqHKEsVukyvt8YRU) (#44)
2019-01-23 19:34:14 Purposefully dropping 44 ; reason: Bad("Unexpected response packet received from peer")
2019-01-23 19:34:14 Banned PeerId(QmXnBNEYRJQqx5HeGfFyvs3UvTraQHEqHKEsVukyvt8YRU) (#44)
2019-01-23 19:34:14 Purposefully dropping 44 ; reason: Bad("Unexpected response packet received from peer")
2019-01-23 19:34:14 Banned PeerId(QmXnBNEYRJQqx5HeGfFyvs3UvTraQHEqHKEsVukyvt8YRU) (#44)

====================

stack backtrace:
0: 0x56084fb97aad - backtrace::backtrace::trace::h1248f0dc5c279727
1: 0x56084fb96bd2 - <backtrace::capture::Backtrace as core::default::Default>::default::hc9a8e6a2a06dfa11
2: 0x56084f3cf3f8 - substrate_cli::panic_hook::panic_hook::hdbeae637bff12cb8
3: 0x56084f3cf0f8 - core::ops::function::Fn::call::h7c1eb9b6bfa49438
4: 0x56084fc9eaf9 - std::panicking::rust_panic_with_hook::h8cbdfe43764887be
at src/libstd/panicking.rs:495
5: 0x56084fc9e5a1 - std::panicking::continue_panic_fmt::h3d3c5a833c00a5e1
at src/libstd/panicking.rs:398
6: 0x56084fc9e4ee - std::panicking::begin_panic_fmt::h11fdc4cc73917110
at src/libstd/panicking.rs:353
7: 0x56084f125f92 - <substrate_finality_grandpa::Environment<B, E, Block, N, RA> as finality_grandpa::Chain<::Hash, <::Header as sr_primitives::traits::Header>::Number>>::best_chain_containing::hb6426ebf50730bfb
8: 0x56084f1a3043 - <finality_grandpa::voter::VotingRound<H, N, E>>::poll::hff2953c01b4eb618
9: 0x56084f1b2155 - <finality_grandpa::voter::Voter<H, N, E, CommitIn, CommitOut>>::process_best_round::h2d85027424e7ce17
10: 0x56084f1648ff - <finality_grandpa::voter::Voter<H, N, E, CommitIn, CommitOut> as futures::future::Future>::poll::h47d17f9cb7f395de
11: 0x56084f156e54 - <futures::future::select2::Select2<A, B> as futures::future::Future>::poll::hb4fe18efaefbd6ae
12: 0x56084ef57763 - <futures::future::chain::Chain<A, B, C>>::poll::hdd55b4b035c6404b
13: 0x56084ef9eaf6 - <futures::future::loop_fn::LoopFn<A, F> as futures::future::Future>::poll::hdaaca68947fc3932
14: 0x56084f225deb - <futures::future::map_err::MapErr<A, F> as futures::future::Future>::poll::h121f6c2ea4aa2ffc
15: 0x56084f27b55e - <futures::future::select::Select<A, B> as futures::future::Future>::poll::h46a22d0adbf3aef8
16: 0x56084ef51c71 - <futures::future::chain::Chain<A, B, C>>::poll::ha0928f3c38ba4526
17: 0x56084fc683c1 - futures::task_impl::std::set::h4e650e554073e01e
18: 0x56084fc69cc5 - <futures::task_impl::Spawn>::poll_future_notify::he7d23a198d713520
19: 0x56084fc6d170 - std::panicking::try::do_call::h9c843760c3feda6a
20: 0x56084fcaf759 - __rust_maybe_catch_panic
at src/libpanic_unwind/lib.rs:102
21: 0x56084fc6aefb - tokio_threadpool::task::Task::run::h54015314b7848e6d
22: 0x56084fc64955 - tokio_threadpool::worker::Worker::run_task::hdeaf9d053ebc9b85
23: 0x56084fc63ffe - tokio_threadpool::worker::Worker::run::h4cae410655bc13ff
24: 0x56084fc4ae66 - <std::thread::local::LocalKey>::with::he0d6480a2d44d226
25: 0x56084fc4aa27 - <std::thread::local::LocalKey>::with::h085d3aeedcd83eaa
26: 0x56084fc4acb1 - <std::thread::local::LocalKey>::with::ha479ba99e96b54e0
27: 0x56084fc44cb9 - tokio::runtime::builder::Builder::build::{{closure}}::h8dd564ad5c95d76b
28: 0x56084fc6b772 - <std::thread::local::LocalKey>::with::h770463a77b89b22b
29: 0x56084fc6b289 - <std::thread::local::LocalKey>::with::h1dd785a094acf76e
30: 0x56084fc6cd17 - std::sys_common::backtrace::__rust_begin_short_backtrace::h530c2e1673231280
31: 0x56084fc6d12b - std::panicking::try::do_call::h325fb79c9c3b76f7
32: 0x56084fcaf759 - __rust_maybe_catch_panic
at src/libpanic_unwind/lib.rs:102
33: 0x56084fc6ee5f - <F as alloc::boxed::FnBox>::call_box::hc0bbd07e08d7bbc2
34: 0x56084fca333d - <alloc::boxed::Box<(dyn alloc::boxed::FnBox<A, Output=R> + 'a)> as core::ops::function::FnOnce>::call_once::hece536cf07b94f8d
at /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b/src/liballoc/boxed.rs:683
- std::sys_common::thread::start_thread::h9605a7df0f911844
at src/libstd/sys_common/thread.rs:24
- std::sys::unix::thread::Thread::new::thread_start::hca8e72c41fa9d291
at src/libstd/sys/unix/thread.rs:90
35: 0x7fa016a926b9 - start_thread
36: 0x7fa0165b241c - clone
37: 0x0 -

Thread 'tokio-runtime-worker-3' panicked at 'internal error: entered unreachable code: this is a bug. target_hash is in blockchain but wasn't found following all leaves backwards', /root/.cargo/git/checkouts/substrate-7e08433d4c370a21/e920bd2/core/client/src/client.rs:963

This is a bug. Please report it at:

https://github.com/paritytech/polkadot/issues/new

I deployed a Local Two-node Testnet with 2 validators and a adder-collater. The first Alice validator crashed not long after the collater joined. logs are attached.
polkadot_test.txt

Using polkadot 0.3.0-d5bef61-x86_64-linux-gnu, I get the error mentioned in the title.

The collator should merklize the output and the root should go in the candidate receipt.

Since there may be multiple chain heads with Aura or other forkful consensus algorithms.

This could follow a similar strategy to paritytech/substrate#1128, where the GRANDPA authorities are kept synchronized with the regular session keys.

The GrandpaApi needs to be implemented for the runtime as well.

All prior validator sets and random seeds for the last FISHERMAN_PERIOD blocks must be kept, so we can slash after the fact.

Comes after #51 -- this issue is for requesting parachain data and figuring out if it's unavailable locally.

1. Validators only prevote in Grandpa for a (descendant of a) relay chain block if they have all erasure coded pieces (from #51). They only build on blocks if they have just seen it very recently (e.g., last block) or they have all the pieces. We do not want to build on a block that has an unavailable ancestor block. Requires paritytech/substrate#1125
2. The request for missing erasure coded pieces is first sent to the rest of the parachain validators and then the other full nodes of the parachain. If full nodes of the parachain don’t have the parachain block available, a randomly selected validator (called assigned validator) asks all validators for their erasure coded piece of the block. When he receives 𝑘=𝑓+1 pieces, the assigned validator attempts to reconstruct it.
3. If a parachain fisherman publishes a proof that the block is invalid, or furthermore, if an intermediate validator or the validator holding the erasure coded piece refuse to hand over the piece we slash them. Moreover, if the assigned validator publishes that 𝑓+1 pieces cannot be decoded into a blob, then we slash the parachain validators and declare that relay chain block invalid.

Following https://github.com/paritytech/polkadot#install-poc-3-alexander-testnet

After installing Rust, and Clang.
Run this
cargo install --git https://github.com/paritytech/polkadot.git --branch v0.3 polkadot

Got the following errors.

error: failed to compile polkadot v0.3.2 (https://github.com/paritytech/polkadot.git?branch=v0.3#b28ff9e7), intermediate artifacts can be found at /tmp/cargo-installVwwOgK
Caused by:
no matching package named substrate-client found
location searched: https://github.com/paritytech/substrate#c118f72d
required by package polkadot-service v0.3.2 (https://github.com/paritytech/polkadot.git?branch=v0.3#b28ff9e7)
... which is depended on by polkadot-cli v0.3.2 (https://github.com/paritytech/polkadot.git?branch=v0.3#b28ff9e7)
... which is depended on by polkadot v0.3.2 (https://github.com/paritytech/polkadot.git?branch=v0.3#b28ff9e7)

I think this is only collating_for for now.

Polkadot Version
polkadot 0.2.16-9f9474d-x86_64-linux-gnu

Operating System
#75-Ubuntu SMP Fri Aug 10 11:14:32 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Network Connected to
Polkadot PoC-2

Command Used
polkadot --rpc-external --telemetry --ws-external

Comments
Installed from source. Was fully synchronized.

====================

stack backtrace:
0: 0x556b3a656d4c - backtrace::backtrace::trace::hfc1e00859cfb7a3e
1: 0x556b3a656b02 - <backtrace::capture::Backtrace as core::default::Default>::default::h86401582f8734314
2: 0x556b3a656b78 - backtrace::capture::Backtrace::new::h3eabdc08da703b90
3: 0x556b39f375f0 - substrate_cli::panic_hook::panic_hook::h5ed1aa63960cf334
4: 0x556b39f37308 - core::ops::function::Fn::call::h3117b58bd1cf5ee4
5: 0x556b3a789ef3 - std::panicking::rust_panic_with_hook::hba05450791beb933
at libstd/panicking.rs:479
6: 0x556b3a29b166 - std::panicking::begin_panic::hde714489b400635b
7: 0x556b3a40e495 - <libp2p_core::nodes::swarm::Swarm<TTrans, TMuxer, TUserData>>::handle_node_reached::h80b4917608407929
8: 0x556b3a4008b8 - <libp2p_core::nodes::swarm::Swarm<TTrans, TMuxer, TUserData> as futures::stream::Stream>::poll::hf11bb3a3527a7f91
9: 0x556b3a305add - <substrate_network_libp2p::swarm::Swarm as futures::stream::Stream>::poll::h4b9b8f69e55394b2
10: 0x556b3a2b16b2 - <substrate_network_libp2p::service_task::Service as futures::stream::Stream>::poll::ha666d385f2617525
11: 0x556b3a28faa3 - <futures::stream::poll_fn::PollFn as futures::stream::Stream>::poll::h402d23a152f365cd
12: 0x556b3a2eb698 - <futures::stream::for_each::ForEach<S, F, U> as futures::future::Future>::poll::h53826955a49367f6
13: 0x556b3a34c59c - <futures::future::select_all::SelectAll as futures::future::Future>::poll::he343517ce36d6da8
14: 0x556b3a3d5a8e - <futures::future::chain::Chain<A, B, C>>::poll::h40b133ff2956de6d
15: 0x556b3a334520 - <futures::future::map_err::MapErr<A, F> as futures::future::Future>::poll::h2a33886028db2a9d
16: 0x556b3a2a6d23 - <futures::future::select::Select<A, B> as futures::future::Future>::poll::h9a8d2a2b50227a9f
17: 0x556b3a3a8bcb - <futures::future::map::Map<A, F> as futures::future::Future>::poll::h0b93bc3c7284567d
18: 0x556b3a336467 - <futures::future::map_err::MapErr<A, F> as futures::future::Future>::poll::hae0a037ae33c065a
19: 0x556b3a387d3a - futures::task_impl::std::set::hebd512409138d442
20: 0x556b3a29c5dc - <std::thread::local::LocalKey>::with::h8396d1d0ac0c5e7e
21: 0x556b3a2f83fc - <tokio::executor::current_thread::Entered<'a, P>>::block_on::hb5c1f658c3e10725
22: 0x556b3a29c39f - <std::thread::local::LocalKey>::with::h715d39e3ea8ef7c7
23: 0x556b3a29bfe3 - <std::thread::local::LocalKey>::with::h22b8b0a8ec3acd1c
24: 0x556b3a29c924 - <std::thread::local::LocalKey>::with::hf2d224bfca00a9fc
25: 0x556b3a29bb7d - <std::thread::local::LocalKey>::with::h1efc73e04c806417
26: 0x556b3a2a408c - tokio::runtime::current_thread::runtime::Runtime::block_on::h2ed3802424638ac9
27: 0x556b3a329974 - std::sys_common::backtrace::__rust_begin_short_backtrace::h387e1a04313bbc17
28: 0x556b3a29b195 - std::panicking::try::do_call::h28fefbe07268feb4
29: 0x556b3a7acdf9 - __rust_maybe_catch_panic
at libpanic_unwind/lib.rs:105
30: 0x556b3a384020 - <F as alloc::boxed::FnBox>::call_box::hc4b8beb9fe0fc55e
31: 0x556b3a78c05a - <alloc::boxed::Box<(dyn alloc::boxed::FnBox<A, Output$u3d$R$GT$$u20$$u2b$$u20$$u27$a$RP$$GT$$u20$as$u20$core..ops..function..FnOnce$LT$A$GT$$GT$::call_once::h940efee7335c8f36
at /checkout/src/liballoc/boxed.rs:652
- std::sys_common::thread::start_thread::hac18b9ed82bd359d
at libstd/sys_common/thread.rs:24
32: 0x556b3a78a4c5 - std::sys::unix::thread::Thread::new::thread_start::ha15fdd35f58285a3
at libstd/sys/unix/thread.rs:90
33: 0x7f8c245506b9 - start_thread
34: 0x7f8c2407041c - clone
35: 0x0 -

Thread '' panicked at 'State inconsistency ; received unknown ReachAttemptId in NodeReached', /root/.cargo/git/checkouts/rust-libp2p-98135dbcf5b63918/3e53a9d/core/src/nodes/swarm.rs:526

A folder with parachain examples would make it easier for new developers to pick up substrate as well as understand the logic behind deploying a parachain to Polkadot.

The examples should increase in difficulty and complexity. Furthermore, they should be built with Substrate.

• Easy
  • adder by @rphmeier
• Medium
• ...
• Hard
  • ...

Under the new availability scheme, we are considering only a single committee per parachain. Votes from this committee encompass both availability of block data and validity.

Thus the separate Availability statement is not necessary. This should simplify the consensus and router code significantly as well as reducing duty roster size.

In the polkadot-erasure-coding crate (see #56), we split up a bunch of encoded data into byte chunks. Each chunk has the "actual length" prepended to it. This is a bit of a waste, so it should be optimized.

When done, this will need a notification of @drskalman to make sure the specification matches.

Following paritytech/substrate#1397, we might have to update our logging here, too.

The network stalled again at #199,305 so I take the chance to snap a few logs.
Acid Burn did block #199,305.

Here are logs from Acid Burn:

2019-01-19 09:38:54 Prepared block for proposing at 199306 [hash: 0x04c787ecdb95b573040427dbbd6d6e86cfca61a1dee42222434a7d8b4f7cc7d0; parent_hash: 0x256e…a793; extrinsics: [0x1e04…9d5c, 0x6b91…878a, 0x467f…d9ff, 0x3b48…b4e0, 0x0100…fc70, 0xfad1…617e, 0x3f47…fd03]]
2019-01-19 09:38:54 Pre-sealed block for proposal at 199306. Hash now 0x32245c4ad0ca0f280a968e8224369304ff95914c06bc3e55b7bacb593514da52, previously 0x04c787ecdb95b573040427dbbd6d6e86cfca61a1dee42222434a7d8b4f7cc7d0.
2019-01-19 09:38:54 Error with block built on 0x256e41f16b72e1758cca166abcc735d5127f0d91eb4865059ac343aab38ca793: Error(ClientImport("Incorrect base hash"), State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })

and here are logs from another validator (Crash Override):

2019-01-19 09:42:48 Prepared block for proposing at 199306 [hash: 0xbf44ad2123df0076b8ff2ea02ed5518acd7a4be3640d21114bb92cb2f7222759; parent_hash: 0x256e…a793; extrinsics: [0xdce9…f8b3, 0x6b91…878a, 0x467f…d9ff, 0x3b48…b4e0, 0x0100…fc70, 0xfad1…617e, 0x3f47…fd03]]
2019-01-19 09:42:48 Pre-sealed block for proposal at 199306. Hash now 0x01e4c27dc7cef1e25421ca7156afb7a5654d6d809ebb8e0a055b3b8949658892, previously 0xbf44ad2123df0076b8ff2ea02ed5518acd7a4be3640d21114bb92cb2f7222759.
2019-01-19 09:42:48 Error with block built on 0x256e41f16b72e1758cca166abcc735d5127f0d91eb4865059ac343aab38ca793: Error(ClientImport("Incorrect base hash"), State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })

It does not complains but reverts 0 blocks and says best block is #0.

• Polkadot Version
  polkadot 0.2.16-9f9474d-arm-linux-gnu

• Operating System
  Linux tinkerboard 4.4.120-rockchip #2 SMP Thu Mar 8 07:04:46 CET 2018 armv7l armv7l armv7l GNU/Linux

• Network Connected to
  Polkadot PoC-2

• Command Used
  polkadot --telemetry --name node_name

• Comments
  Installed from source. Was not fully synchronized. Also tried restarting the node.

• Output

   0: 0x800c928f - backtrace::backtrace::trace::h984d1af2a55390ce
   1: 0x800c8fcb - <backtrace::capture::Backtrace as core::default::Default>::default::hc60c6616f1fff41f
   2: 0x800c9027 - backtrace::capture::Backtrace::new::he165f021ec91be5b
   3: 0x7fa000db - substrate_cli::panic_hook::panic_hook::hdbc868bd3175a2c2
   4: 0x7f9ffe27 - core::ops::function::Fn::call::h52cdc6cde20bfae6
   5: 0x801e01d3 - std::panicking::rust_panic_with_hook::h22b0068fbbf8b0b4
                at libstd/panicking.rs:479
   6: 0x801dfdef - std::panicking::continue_panic_fmt::h015fb5b89d43591d
                at libstd/panicking.rs:390
   7: 0x801dfcd7 - rust_begin_unwind
                at libstd/panicking.rs:325
   8: 0x8022fa13 - core::panicking::panic_fmt::h2c383e888dce42c0
                at libcore/panicking.rs:77
   9: 0x8022f94b - core::panicking::panic::h14eb4ebea7b4bf80
                at libcore/panicking.rs:52
  10: 0x801f9797 - <core::option::Option<T>>::unwrap::h4d93ce68a2017083
                at /checkout/src/libcore/macros.rs:20
                 - panic_unwind::imp::cleanup::h64cb63747d2a75b9
                at libpanic_unwind/gcc.rs:101
                 - __rust_maybe_catch_panic
                at libpanic_unwind/lib.rs:108
  11: 0x7f886153 - <F as alloc::boxed::FnBox<A>>::call_box::h301ecd6109947f71
  12: 0x801e0947 - <alloc::boxed::Box<(dyn alloc::boxed::FnBox<A, Output$u3d$R$GT$$u20$$u2b$$u20$$u27$a$RP$$GT$$u20$as$u20$core..ops..function..FnOnce$LT$A$GT$$GT$::call_once::hc817d849f98aebf6
                at /checkout/src/liballoc/boxed.rs:652
                 - std::sys_common::thread::start_thread::h834abd292559e1ce
                at libstd/sys_common/thread.rs:24
                 - std::sys::unix::thread::Thread::new::thread_start::h138f645bf45a5122
                at libstd/sys/unix/thread.rs:90

Thread 'ImportQueue' panicked at 'called `Option::unwrap()` on a `None` value', libcore/option.rs:345

For now, we'll have to queue it up and dispatch it based on block-import notifications. The reason for this is that the parachain validators are not determined ahead of time.

In the future, the PVs will probably be known shortly ahead of time, which will optimize this process.

We will need to split up the attestation-gathering logic from the Proposer logic.
The Proposer will still look for a minimum number of attested-to parachains, and we will need something for evaluating blocks and making sure they have enough parachain candidates for the time, etc.

Most of the evaluate logic would be placed in an aura::CustomVerifier that checks to see if the authorship cycle has been instantiated for the parent of a block (doing it if not) and then waiting until enough attestations come in before importing the block.

This might lead to some issues around clock synchrony if we don't evaluate blocks for longer than the block time in Aura, but needs further investigation.

Depends on paritytech/substrate#1124

Currently, we just believe someone if they say they control a specific session key. We will need a cryptographic authentication mechanism for this.

paritytech/substrate#271

After paritytech/substrate#746 is in, we will have finality notification streams and non-instant finality.

Polkawallet project to need query the record of validators&nominators history reward. is very trouble. Requires underlying support.

1. A collator sends a parachain block, its outgoing messages and a light-client proof of correctness of these to parachain validators (a parachain blob) to the parachain validators
2. Once the parachain validators have validated it, they create an erasure coded version with an optimal (𝑛,𝑘) block code of this blob, where 𝑘=𝑓+1.
   They also calculate a Merkle tree for this erasure coded version and add the Merkle root to the things they sign about the block for inclusion on the relay chain.
3. The parachain validators send out these pieces to all validators
4. The parachain block header gets included on the relay chain
5. If a validator has not received an erasure coded piece for every parachain blob that has a header in a relay chain block it request it. The piece they should ask for is different for each validator. Along with the piece, the parachain validator needs to provide the Merkle proof that it comes from the Merkle root on the relay chain.
   --
6. During the execution of GRANDPA protocol, a relay chain validator only prevotes for a (descendant of a) relay chain block if it has received an erasure coded piece for each parachain block header included in that relay chain block. Validators only build on blocks if they have just seen it very recently (e.g., last block) or they have all the pieces. We do not want to build on a block that has an unavailable ancestor block.
7. A validator who has not received an erasure coded piece for a blob header in a relay chain block needs to requests for the missing erasure coded piece. It sends the request first to the parachain validators who also signed off on the validaity of the blob, but were not the parachain validator who was supposed to send the erasure coded piece. Afterwards, the validator requests the erasure coded piece from the other full nodes of the parachain. If full nodes of the parachain are not able to provide the missing pieces, the relay chain validator whose piece is missing does not validate the corresponding relay chain block.

The idea here is that we do not finalize a block until sometime after f+1 honest validators prevote for it. But if that's the case, then 6. should succeed, which means that we only finalize available blocks. If 7. happens fast enough, then we only finalize valid and available blocks. As before, we will need to plan for when we finalize an invalid block.

Backing up erasure-coded pieces
Note that an honest parachain validator can back up the pieces at a (random or preferably trusted) full node of the parachain before it sends them out to relay chain validators. Subsequently, that parachain full node can distribute those pieces to all full nodes of the parachain who can respond to requests from validators that are requesting missing erasure-coded pieces.

Additional context & research is available here.

cc: @montekki

• Polkadot version
  polkadot 0.2.16-52f427bb-x86_64-linux-gnu

• Operating system
  Linux ivo-thinkpad 4.18.9-arch1-1-ARCH #1 SMP PREEMPT Wed Sep 19 21:19:17 UTC 2018 x86_64 GNU/Linux

• Network connected to
  Polkadot PoC-2

• Command used
  polkadot

• Comments
  Installed as per the PoC instructions. It ran for a few times, then I upgraded my system and rebooted, after which it started crashing; the system upgrade could be related, if this is something related to openss

• Output

[ivo@ivo-thinkpad substrate]$ polkadot
2018-10-02 09:15:32 Parity ·:· Polkadot
2018-10-02 09:15:32   version 0.2.16-52f427bb-x86_64-linux-gnu
2018-10-02 09:15:32   by Parity Technologies, 2017, 2018
2018-10-02 09:15:32 Chain specification: Krumme Lanke
2018-10-02 09:15:32 Node name: innocent-experience-8049
2018-10-02 09:15:32 Roles: FULL
2018-10-02 09:15:32 Best block: #1550814
2018-10-02 09:15:32 Local node address is: /ip4/0.0.0.0/tcp/30333/p2p/QmeXqsfjwwVkCgBaUSLBCk2f1jt6j5n4qnUW3D4DBAceL6

====================

stack backtrace:
   0:     0x55b90be20bec - backtrace::backtrace::trace::he78f4c787637d93c
   1:     0x55b90be1fea2 - <backtrace::capture::Backtrace as core::default::Default>::default::h491716ea6cf5e245
   2:     0x55b90be1ff18 - backtrace::capture::Backtrace::new::h731e3be60b0d8d78
   3:     0x55b90b7237c0 - substrate_cli::panic_hook::panic_hook::h1b7f1e370578d968
   4:     0x55b90b7234d8 - core::ops::function::Fn::call::h973abbcda013d9f2
   5:     0x55b90bf5c153 - std::panicking::rust_panic_with_hook::h7d7e0e1ff988fee8
                        at libstd/panicking.rs:481
   6:     0x55b90bf5bcf9 - std::panicking::continue_panic_fmt::he8d778b183a54215
                        at libstd/panicking.rs:391
   7:     0x55b90bf5bbf5 - rust_begin_unwind
                        at libstd/panicking.rs:326
   8:     0x55b90bfb74bb - core::panicking::panic_fmt::h1c54d6121d2e9db5
                        at libcore/panicking.rs:77
   9:     0x55b90bfb73ea - core::panicking::panic::he4a84d173432c46d
                        at libcore/panicking.rs:52
  10:     0x55b90bd73ef0 - openssl::ssl::connector::ctx::hb889dd782396f1fd
  11:     0x55b90bd73f33 - openssl::ssl::connector::SslConnectorBuilder::new::h0e63f7227c29c519
  12:     0x55b90bd72537 - native_tls::TlsConnector::builder::h59a349d339aeeecd
  13:     0x55b90bd5b41a - websocket::client::builder::ClientBuilder::extract_host_ssl_conn::h45a24c3a4efd61a2
  14:     0x55b90bd5b7df - websocket::client::builder::ClientBuilder::wrap_ssl::hc5a3b27f3e782fd0
  15:     0x55b90bd5a5d9 - websocket::client::builder::ClientBuilder::connect::h63855ef77167725c
  16:     0x55b90bd2a443 - substrate_telemetry::init_telemetry::hf4ad6b445ac69c01
  17:     0x55b90b51dbbe - <substrate_service::Service<Components>>::new::hb1f4cb2d49a3b1df
  18:     0x55b90b491d0a - polkadot_service::new_full::h7d2f4a13ff598561
  19:     0x55b90b41ec9d - polkadot_cli::run::h722fd3a014f9bfe7
  20:     0x55b90b3ea3b7 - polkadot::main::hb0d42fa6ff114468
  21:     0x55b90b3f77c2 - std::rt::lang_start::{{closure}}::hecddadd857b21ef7
  22:     0x55b90bf5bb92 - std::rt::lang_start_internal::{{closure}}::hb4408a98b70383f8
                        at libstd/rt.rs:59
                         - std::panicking::try::do_call::h7120cc99658f0544
                        at libstd/panicking.rs:310
  23:     0x55b90bf803c9 - __rust_maybe_catch_panic
                        at libpanic_unwind/lib.rs:102
  24:     0x55b90bf69f65 - std::panicking::try::he4adeff436478156
                        at libstd/panicking.rs:289
                         - std::panic::catch_unwind::h3c194ce486e1b933
                        at libstd/panic.rs:392
                         - std::rt::lang_start_internal::he861fec101d426ef
                        at libstd/rt.rs:58
  25:     0x55b90b3ea4e3 - main
  26:     0x7f7b73fac222 - __libc_start_main
  27:     0x55b90b363459 - _start
  28:                0x0 - <unknown>

Thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', libcore/option.rs:355

This is a bug. Please report it at:

    https://github.com/paritytech/polkadot/issues/new

since the delay might interfere with the Aura slot system, which already is meant to enforce a specific block period.

(probably can be done by setting step_duration = block_period * 2)

Current Blockers in Substrate:

• Skipping ahead in GRANDPA sets and rounds: paritytech/substrate#1069
• Re-enable custom authorship workers in substrate-service: paritytech/substrate#1158

Blockers in Polkadot:

• Parachain candidates are accompanied with full attestation: #39
• GRANDPA integration in runtime: #44
• Update block authorship to support GRANDPA #37
• Allow multiple parachain-attestation sessions to be live at once in network: #40
• Reintroduce Custom CLI: #43
• Disable dynamic-inclusion in authorship
• Run GRANDPA voter on all nodes (cc paritytech/substrate#1271)

Testing:

• Ensure entire test suite passes
• Ensure GRANDPA and regular set changes are working
• Deploy and collate for parachains

Ubuntu 16.04.5 LTS (GNU/Linux 4.4.0-131-generic x86_64)
Polkadot Version: parity-polkadot v0.2.16

49: 0x562bfafcc7f4 - std::panicking::try::do_call::h14ad9761133d2f27
50: 0x562bfb4ce079 - __rust_maybe_catch_panic
at libpanic_unwind/lib.rs:105
51: 0x562bfaf80050 - <F as alloc::boxed::FnBox>::call_box::h2fd6834e137777b8
52: 0x562bfb4ad35a - <alloc::boxed::Box<(dyn alloc::boxed::FnBox<A, Output$u3d$R$GT$$u20$$u2b$$u20$$u27$a$RP$$GT$$u20$as$u20$core..ops..function..FnOnce$LT$A$GT$$GT$::call_once::h940efee7335c8f36
at /checkout/src/liballoc/boxed.rs:652
- std::sys_common::thread::start_thread::hac18b9ed82bd359d
at libstd/sys_common/thread.rs:24
53: 0x562bfb4ab7c5 - std::sys::unix::thread::Thread::new::thread_start::ha15fdd35f58285a3
at libstd/sys/unix/thread.rs:90
54: 0x7f87e8e6c6b9 - start_thread
55: 0x7f87e898c41c - clone
56: 0x0 -

Thread '' panicked at 'explicit panic', /home/polkadot/.cargo/git/checkouts/rust-libp2p-98135dbcf5b63918/304e9c7/protocols/secio/src/handshake.rs:465