One fine day....

Go through, improve readability, add examples and resources...

So it is everything you would like to see the first time you try to deploy tendermint/weave in production.

This package needs solid coverage, should work out of the box. Test up to the abci layer, also app package.

Cover how to write your own module.
Cover how to wire the modules into an app.

See #87

Write a custom ledger app
Allow signing with the device

Sometime in the future... but a man can dream, can't he?

In particular x/auth/controller_test.go and x/coin/controller_test.go are quite ugly.
Sorry about that.

See comments from this PR: #80
@ethanfrey FYI

This builds on #100

Current multisig is N of M threshold contract. Special cases are OR (N = 1) and AND (N = M). However, it does treat all participants of the same weight. What if I want to make a key recovery scheme such as

auth = masterKey OR (2 of [recovery1, recovery2, recovery3]).

Such a scheme could be nice to shard recovery keys over trusted people but just use the master key normally. But it is also impossible with simple threshold algorithm, so we should extend.

One simple (but maybe inefficient) technique is to allow nesting a list of multisig contracts. Thus the above would be

recoveryContract = 2 of [recovery1, recovery2, recovery3]
safeKeyContract = 1 of [masterKey, recoveryContract].

Recovery will have to first have 2 signatures on the tx, explicitly request approval by recoveryContract, then explicitly request approval for safeKeyContract.

There are probably better ways to do that and these should be discussed before beginning this issue, but providing an example should give clarity to possibilities

• format for proofs as protobuf
• Iavl adapter converts to this format
• test proof verification

we want to use a format compatible with tendermint, and ideally cosmos-sdk, to make it easier to support ibc.

Starting from tendermint rpc... let's track down the proof format...

Rpc call returns the abciQuery object (as json)
func ABCIQuery(ctx *rpctypes.Context, path string, data cmn.HexBytes, height int64, prove bool) (*ctypes.ResultABCIQuery, error)

Abci definition

message ResponseQuery {
...
  merkle.Proof proof = 8;
  int64 height = 9;
  string codespace = 10;
}

merkle.Proof format

message ProofOp {
  string type = 1;
  bytes key = 2;
  bytes data = 3;
}

message Proof {
  repeated ProofOp ops = 1 [(gogoproto.nullable)=false];
}

Unclear is what exactly these ProofOp types are and what they mean.
Digging in deeper, we can see the iavl_value_proof defines one operation to represent the entire range proof for one value:

type IAVLValueOp struct {
	// Encoded in ProofOp.Key.
	key []byte
	// To encode in ProofOp.Data.
	Proof *RangeProof `json:"proof"`
}
...
func (op IAVLValueOp) ProofOp() merkle.ProofOp {
	bz := cdc.MustMarshalBinaryLengthPrefixed(op)
	return merkle.ProofOp{
		Type: ProofOpIAVLValue,
		Key:  op.key,
		Data: bz,
	}
}

With the actual range proof, being an iavl-specific internal structure, encoded with go-amino. I cannot see this code actually called, however.

In an attempt to see this in practice, I will look at the canonical abci app, cosmos-sdk.

Look at their BaseApp.Query entry point.

Which will route the query to the data store if the prefix is "/store"

func handleQueryStore(app *BaseApp, path []string, req abci.RequestQuery) (res abci.ResponseQuery) {
	// "/store" prefix for store queries
	queryable, ok := app.cms.(sdk.Queryable)
	if !ok {
		msg := "multistore doesn't support queries"
		return sdk.ErrUnknownRequest(msg).QueryResult()
	}

	req.Path = "/" + strings.Join(path[1:], "/")
	return queryable.Query(req)
}

It seems that rootmulti.Store is the default app.cms implementation, with the actual Query function

Which takes the QueryResult of a substore, and appends a special "multistore proof op":

	store := rs.getStoreByName(storeName)
	queryable, ok := store.(types.Queryable)
	res := queryable.Query(req)
        // ...
	res.Proof.Ops = append(res.Proof.Ops, NewMultiStoreProofOp(
		[]byte(storeName),
		NewMultiStoreProof(commitInfo.StoreInfos),
	).ProofOp())

And finally, we dig into the Query method of the Iavl sub-store, and see the proof formation:

	value, proof, err := tree.GetVersionedWithProof(key, res.Height)
	// ...
	if value != nil {
		// value was found
		res.Value = value
		res.Proof = &merkle.Proof{Ops: []merkle.ProofOp{iavl.NewIAVLValueOp(key, proof).ProofOp()}}
	} else {
		// value wasn't found
		res.Value = nil
		res.Proof = &merkle.Proof{Ops: []merkle.ProofOp{iavl.NewIAVLAbsenceOp(key, proof).ProofOp()}}
	}

So, basically, we just call MutableTree.GetVersionedWithProof and then serialize the returned RangeProof into a custom format.

This may work fine if both client and server are golang and importing the iavl codebase, but I see trouble with handling these not-well-defined proofs in eg. a javascript client.

Final conclusion from this deep-dive in cosmos code:

1. Raw proof structures are available in tendermint/iavl
2. There is a "standard format" to encode them
3. This "standard format" currently stores enormous opaque blobs that cannot be parse in another language
4. We should define a simpler RangeProof -> merkle.ProofOp[] mapping that can be defined in terms of simpler primitives (concat, sha256, etc)
5. The case of one item is collapsed into a subset of range proof. Let us define a simple-to-parse struct for the one-key-present case, then the one-key-missing case, then the larger range proof, then secondary index proofs.... Each one can be it's own issue and should be well defined to be able to validate from non-go code without access to any custom libraries

A generic way to read in the genesis file and pass initial state into the modules

Hello,
I'm trying basecoin (https://github.com/tendermint/basecoin/blob/master/docs/guide/basecoin-basics.md) and weave, it seems basecoin is old. However, basecoin support Multiple outputs, it can be used to send many different types of tokens between many different accounts at once in an atomic transaction.
Can you please tell me how to use weave(mycoind app) to send to multiple different account in one transaction?
Thank you!

Can be TravisCI, CircleCI or Jenkins. Probably https://codecov.io/ for code coverage

• Run all tests on new push
• PRs check test results before merging
• Show test coverage (ideally excluding auto-generated *.pb.go files)

Maybe wait until repo is public?

Make a new batch type that optimizes on write.

It is often that multiple operations may write to the same key (eg. deduct fee from account X, move coin from account X). We want to allow the code to be independent, but do not want to write twice to the underlying merkle tree (which is expensive).

We can add a DedupBatch, that on write will first remove all duplicates, leaving only the last set/delete that effects a given key. It can scan from the last operation, and copy operations into a clean buffer. If the key was not yet seen, add to clean buffer from the end. Then slice the clean buffer to size.

Automatically add them to the keys that we modify in the tx.

We can register code that is called every begin block.
This will get the block header, along with the list of validators that signed it (#76 (comment))

• Allow a way to register a handler on app construction (like normal handler)
• Allow a way to register many of them to be called one after another (like MulitAuth)
• Create a design for cron scheduler handler, an extension that allows executing delayed tasks. (eg. run this at block 50, other tasks at 47, other at 93 and it will make sure the callback is called with the proper data at the proper time)

Before we sign a transaction, we calculate the sign bytes. These are currently the serialized transaction without signatures, followed by the chain Id and then an 8-byte nonce (account sequence).

This is easy to compute, but if we hand the SignBytes into a module to be signed (eg. a ledger app), it is hard to impossible to properly parse. The SignBytes were never meant to hit the wire before so this was not really considered, but Simon made a good point a few places and a proposal here: iov-one/iov-core#56 (comment)

format version | chain ID length | chain ID            | nonce | bz
int32          | int32           | utf8 encoded string | int?  | raw data

I would accept that proposal more or less with minor changes

format version | chain ID length | chain ID            | nonce | bz
4bytes          | int8           | utf8 encoded string | int64 (bigendian)  | raw data

the chain Id should be maximal ~32 bytes, so one byte for encoding the length is sufficient.
the nonce is a fixed 8 byte length, in BigEndian format. ChainId should be a subset of ascii, but we can define it as utf8 to be more permissive in parsing.

The version doesn't need just to be a counter 00000001, 00000002, ... but a unique identifier than can change between different versions. It also is a known magic number to inform how to interpret the following data.

We should use unique prefix(es) that are not simply a counter, to ensure minimal collision with other known formats and encodings. I propose the first version will be the prefix 00CAFE00 (no coffee), somewhat inspired by java class files CAFEBABE.

Updating this calculation will break the signing algorithm used in clients such as weave-js and should be accompanied with a version bump.

A contract must have a list of N addresses, a threshold to assume permission (1 <= threshold <= N) and a threshold to modify members (threshold <= adminThreshold).

This needs a handler to create and update the content of a mutlisig contract.

Each contract has a unique Condition/Address and can control a wallet. Add a middleware package and a field to the tx so the user can explicitly ask to assume authorization of the multisig for the rest of the request. THis will check all current authorizations against the requirements of the contract and either fail or add another Condition to the authorization.

Up to this point, we can use either 0 or 1 multisig in a transaction

The abci interface should support both

References changes in #76

We pass in the app_state to be parsed by each extension. And also handle the chainId internally. However, some apps may want more info, especially the list of initial validators. We should extend the Initializer interface and implementation to add more data there.

Right now, go get github.com/confio/weave fails because of a missing dependency:

$ go get github.com/confio/weave
package github.com/tendermint/go-wire/data: cannot find package "github.com/tendermint/go-wire/data" in any of:
	/home/simon/go1.10.2/src/github.com/tendermint/go-wire/data (from $GOROOT)
	/home/simon/go/src/github.com/tendermint/go-wire/data (from $GOPATH)

It seems like https://github.com/tendermint/go-wire is not a thing anymore

Go over all the packages:

• Ensure there is a nice overview for the package
• Make sure that what is exposed makes sense, rename or make private as needed
• Provide examples to demonstrate expected usage

With TMError, we show 3 levels below where it is called (ErrXX, Wrap, New). We should trim those off the stacktrace from "%+v".

https://godoc.org/github.com/pkg/errors#hdr-Retrieving_the_stack_trace_of_an_error_or_wrapper

We can implement StackTrace() and return Frame[3:]

See how fast things are, as a basis for optimization

In particular:

• Store read/writes, iavl and btree cache
• Crypto
• Marshalling (also memory use, gc)

Total test:

• Stack for mycoin from ABCI interface down

Make sure to add data (eg. 1 million account) into store
Chose useful blocksizes (100?)

This is a minimal version of #32, with a few purposes:

• to make sure the whole stack (incl. tendermint) can properly update validators on a running network
• to provide a tool to easily add/remove validators on testnet (switch nodes from non-validator to validator)
• and as a learning experience writing a weave module

Requirements:

• There should be a new module with a new transaction type. It receives a list of the Validator diff that will be sent on EndBlock and passes it through verbatim. (So basically a client sends the diff that we then pass on to update validator power)
• There should be one key set in the genesis file that is the only key that can authorize this transaction (pubkey/address)
• Unit test the go code behaves as desired
• Integration test that EndBlock message is proper format for tendermint, and validators are dynamically updated.

Steps (WIP):

• Figure out a protocol and implement it via protobuf
• Implement initializer for key
• Implement business logic
• Wrap BL into an extension interface (decorator)
• Unit-test this using goconvey/gomega+ginkgo
• Add a genesis key for an address/pubkey and it's handler
• Integration tests and last touch-ups

Ethan thinks there is a todo there in top level result.go. Let's do it

Blockchain Communication Protocol Daemon.

#91 (comment)

go v1.8+ has a default value for $GOPATH, which should be respected in case the user has no $GOPATH set. Otherwise make deps fails at this point:

cd $GOPATH/src/github.com/tendermint/tendermint && \
		git checkout v0.17.1 && \
		make ensure_deps && make install && \
		git checkout -
/bin/sh: line 0: cd: /src/github.com/tendermint/tendermint: No such file or directory
make[1]: *** [/tendermint] Error 1
make: *** [deps] Error 2

https://beta.golang.org/doc/go1.8#gopath

So we can run the abci app with tendermint, both for dev-mode and later to deploy.

• whilst messing around with init I created a currency "CASH2" (as you do)
• when starting the blockchain, I get a panic with message panic: CASH2: Invalid currency code
• so, either we validate currency names to stop numbers (and other symbols etc?)
• or we do something about the actual panic

Go through, improve readability, add examples and resources...

So it is everything you would like to see the first time you try to write a custom extension on weave.

For full-on HD wallets, and multiple addresses :)

• low priority, but should eventually be the way to go

References changes in #76
References changes in #78

We get both header and who signed in begin block, provide them both to any extension that is interested. This is most useful for BeginBlock handlers, but DeliverTx handlers may care as well.

Idea: monkit, opentracing, prometheus???

Something so we can get metrics from a running system, find real bottlenecks

We want to be able to recover keys with a seed phrase.

This is tied to discussions on the signing process in order to get ledger compatibility

iov-one/iov-core#70

In the end, the simplest is to always perform sha512 on sign bytes before signing anf before verifying signatures. sha512 can be done in software or hardware.

For IOV, we want to create a lookup of name -> wallet and wallet -> name, enforcing unique name on the blockchain.

Doing this well, requires some enhancements to the sdk, that will be generally useful and should be well planned.

1. A way to extend the Wallet in x/coins. We could just add a name field, but many people will want something else. Best is to see how the coins logic can be reused without much extra code. For example, the controller could accept a Holder interface that allows Save and get/set the coin.Set, then other extension could call in for most of the logic. But let's brainstorm this to get a good general mechanism to extend wallet/account

2. We need a unique secondary index (name -> wallet address). Name is stored in each wallet. This can be done not too hard, but it would be a good chance to reflect on making secondary indexes (both unique and not unique) that can provide merkle proofs in a more general way. We can build out some reusable code to be refined with future use cases.

Both should be broken out into individual issues, and this then the integration issue.

And new tendermint deps

There is a problem in cosmos-sdk with constantly changing app_hash: cosmos/cosmos-sdk#520

Investigate if this also affects weave. We have different app logic, but also rely on the same underlying merkle store.

Bech32 is the future: https://en.bitcoin.it/wiki/Bech32
"Readable" addresses with error correction.

We could use this: https://github.com/cosmos/bech32cosmos

11:46:06 jpincas@Jons-Desktop make deps
[INFO]    No mirrors.yaml file exists. Creating new one
[INFO]    https://github.com/tendermint/go-wire being set to https://github.com/ethanfrey/go-wire
[ERROR]    Error writing mirrors.yaml file: open /home/jpincas/.glide/mirrors.yaml: no such file or directory
An Error has occurred
Makefile:48: recipe for target 'deps' failed
make: *** [deps] Error 2

• solved by manually mkdir $HOME/.glide

After developing validators extension for weave and integrating that to mycoind I got some insight into extension development and it's nuances.

We also need to document validators feature for the example app I suppose.

Right now, the body to be signed is something like <raw tx> | <chain ID> | <nonce>, where | means binary concatenation and <symbol> a place holder.

This allows collisions when one body's raw tx ends with the same bytes as some other body's chain ID starts with. E.g.

<raw tx> | <chain ID> | <nonce>
===============================
aabbccdd|eeff|0011
aabbcc|ddeeff|0011

To avoid those kind of collisions, you either need field separators or have fixed field sizes. Separators are usually not practical when working on raw binary data.

Thus I propose the following format instead:

format version | chain ID length | chain ID            | nonce | bz
int32          | int32           | utf8 encoded string | int?  | raw data

From jon:

For sure. I’ve read most of the docs you’ve written now and also gone back over the tendermint docs. I think the place to start will be really teasing apart the abci building stuff from the weavejs stuff. At the moment it can be hard to know exactly which bit you are talking about at any one point. I’ll explain more when we talk.

Ethan:

Yeah, good thought.

Maybe an overview of the pieces. Then explain how to set up a pre-built abci app Then using it to see how it works from the outside.
Like what I have but with a coherent flow...

Afterwards how to build your own abci app. And finally how to build your own client. And connect that to a js frontend
Does that seem like a sensible narrative?

Current output:

I[08-27|14:27:49.336] Info synced                                  module=mycoin height=0 hash=
panic: unexpected end of JSON input

goroutine 51 [running]:
github.com/iov-one/weave/app.(*StoreApp).InitChain(0xc4202240a0, 0x5b8409a8, 0xc4201e8600, 0x11, 0xc4201d83c0, 0xc4201dabe0, 0x1, 0x1, 0x0, 0x0, ...)
    /Users/alex/workspace/go/src/github.com/iov-one/weave/app/store.go:304 +0xbc
github.com/iov-one/weave/vendor/github.com/tendermint/abci/server.(*SocketServer).handleRequest(0xc420224140, 0xc4201d25e0, 0xc4200b2a80)
    /Users/alex/workspace/go/src/github.com/iov-one/weave/vendor/github.com/tendermint/abci/server/socket_server.go:193 +0x76b
github.com/iov-one/weave/vendor/github.com/tendermint/abci/server.(*SocketServer).handleRequests(0xc420224140, 0xc4200b2a20, 0x16aa280, 0xc4200aa048, 0xc4200b2a80)
    /Users/alex/workspace/go/src/github.com/iov-one/weave/vendor/github.com/tendermint/abci/server/socket_server.go:163 +0xe8
created by github.com/iov-one/weave/vendor/github.com/tendermint/abci/server.(*SocketServer).acceptConnectionsRoutine
    /Users/alex/workspace/go/src/github.com/iov-one/weave/vendor/github.com/tendermint/abci/server/socket_server.go:119 +0x269

mycoind can be found in the example directory

We can use a simple staking modules with a fixed set of delegators, who may delegate to different validators. Thus, the private key of the machine is not fixed, but rather the key that can assign it. This allows unwatching for example, and provides a simple module to also control dynamic testnet configurations.

All "bonds" should be indexed by both the validator and the delegator, and this provides an example of N-to-N relations and queries that needs to be modeled with the orm, for the advantage of other use cases as well.

I followed this instruction to install go. However, this resulted in the following error

$ go get github.com/confio/weave
package github.com/confio/weave: cannot download, $GOPATH must not be set to $GOROOT. For more details see: 'go help gopath'

In this case $GOPATH was $HOME/go and $GOROOT was unset. Moving the installation to ~/go1.10.2 caused that go was not in PATH anymore.

I now have

export GOROOT="$HOME/go1.10.2"
export GOPATH="$HOME/go"
export PATH="$PATH:$GOROOT/bin"

in by .bashrc, which seems to work. However, I am not an expert with Go, so I don't want to submit a documentation PR.

Before implementing the signature verification cache, please provide a benchmark that test the current performance. Once we know how much time the current solution takes, we will decide if we want to optimize it. (Update: we have the benchmarks).

Most tx will show up in CheckTx and then in DeliverTx.
Since the verification is relatively expensive operation, we can cache the result to avoid doing the work twice.

Idea:

• cache for height h-1
• cache for height h
• on validate, check if in cache, add result to cache(h)
• on commit, move cache(h) to cache(h-1) and set cache(h) to empty

Simple implementation of LRU

Other approaches are fine, but they:

• should have bounded memory usage (maximum is proportional to max block size)
• in typical cases allow us to avoid verifying the same tx signatures multiple times. This is also important in the case when the tx doesn't fit in block (high load) and is rechecked before proposing to next block. These re-checks shouldn't have to re-verify and thus be very quick.
• doesn't add excess complexity for edge cases. easier to reverify a few transactions than introduce bugs or inefficiencies in the normal flow.

Please run benchmarks before and after this PR to demonstrate differences.

Treat this as an inbox of various and bits and pieces that may or may not be issues. Until I get my head around how everything fits together, some of these may be non-issues.

• mycoind init ... is silent upon success. I can't see any generic 'user message' functionality in the code. Would be good to have confirmation that it has worked. Do any of the commands produce output/confirmation? Is this something that needs to be added?
• I see mycoind init ... can be run again no problem without any protection/confirmation. Could this do with some warning/protection/double confirmation to stop people reinitialising their apps?
• after initialising, genesis.json shows "genesis_time": "0001-01-01T00:00:00Z" which obviously looks odd to me, but perhaps this is like a T0 and all the following timestamps will be relative to this? Still don't know enough about tendermint/weave to know if this is an issue.