iov-one / block-metrics Goto Github PK
View Code? Open in Web Editor NEWStore metrics on tendermint blocks for a chain, with focus on missed blocks, uptime and censorship metrics
License: Other
Store metrics on tendermint blocks for a chain, with focus on missed blocks, uptime and censorship metrics
License: Other
As a user, I want to know the accounts that are associated with a given target.
The relation between all targets and their associated accounts needs to be persisted in the database and kept up-to-date. Additionally, all certificates in an account need to be persisted so that an external call to the bnsapi
is not required to obtain them.
transaction_hash
, block_hash field
should be text
(not bytes) so we can search easily
I updated to go1.11.5 just to run this, and got...
TENDERMINT_RPC="ws://bnsd.hugnet.iov.one" go run cmd/collector/main.go
pkg/metrics/tendermint.go:12:2: cannot find package "github.com/gorilla/websocket" in any of:
/usr/lib/go-1.11/src/github.com/gorilla/websocket (from $GOROOT)
/home/ethan/go/src/github.com/gorilla/websocket (from $GOPATH)
pkg/metrics/pq.go:9:2: cannot find package "github.com/lib/pq" in any of:
/usr/lib/go-1.11/src/github.com/lib/pq (from $GOROOT)
/home/ethan/go/src/github.com/lib/pq (from $GOPATH)
And the cli args are just wrong
Please use the following mappings to display the validator name, rather than id, in the explorer:
61B819A0BCF4E65AF8B6ED3AB287935074B8C7E3 Cosmostation
3B4F5C11663DC10A6D32403F945DED42AE1DD362 StakeWith.Us
8411B44F2FF2CE6A3143121EB5EEC1A23FCF2631 HashQuark
058078082E8ED2431EA61E69657BE27F0D7456FA Node A Team
4A6CDCD260D1527CD1F89ECB5BA3A160FAB3B5F7 Forbole
A811EB8C0E76991BA241278886625CF081EFF874 01node.com
4C74A4E2156493E5FB329BE619C188519629CCE3 Bianjie
28BFE674AF17CFE4736C979CECFFC03A5490DE96 ChainLayer
DF97841F98E18B02F670C62830E13C40FFCE9D1E syncnode
A5F88A83C831E6D84C83EED33870F4015D0FE94A Stake Capital
The validator set will not change for months so feel free to not be clever with the implementation, ie hard-code the values if you wish.
After running the sync on hugnet for a while, the metrics reporter died:
$ TENDERMINT_WS_URI="wss://bns.hugnet.iov.one/websocket" POSTGRES_URI="postgresql://postgres@localhost:5432" go run cmd/collector/main.go
2019/04/01 19:00:37 cannot unmarshal JSONRPC message: read tcp 192.168.1.39:58936->34.76.6.53:443: use of closed network connection
sync: blocks for 11444: query tendermint: -32603: Internal error: failed response
exit status 2
Maybe you need to add the ping-pong websocket keep-alive protocol?
Current design does not include multisig tx in the database. This needs to be fixed.
I just synced with hugnet to find some missed blocks, and got the following:
postgres=# select block_height, proposer_id, participant_ids from blocks where array_length(participant_ids, 1) != 4;
block_height | proposer_id | participant_ids
--------------+-------------+-----------------
5 | 2 | {1,3,4}
10 | 1 | {1,3,4}
12 | 4 | {2,1,4}
69 | 2 | {1,3,4}
11443 | 3 | {1,3,4}
(5 rows)
Note that in 2 of the 5 examples, the proposer was not in the list of participants. Maybe there is some mis-match, like those participants are those who signed the block before or after??
TODO: evaluate state after we finish #1 (and merge #11) to see what is left to do
We need to grab block header info along with which validators signed into a postgresql database. (or other to be determined).
Should allow non-devs (BI?) to write queries to detect downtime (not just total, but also how long at once), also detect censorship attacks (validator A vote never includes when B is proposed), etc....
Connects to tendermint rpc to get this info and runs as a daemon to fill in this database.
Is your feature request related to a problem? Please describe.
This is a refined first step to https://github.com/iov-one/weave/issues/427
Describe the solution you'd like
Let us make a simple infrastructure to collect basic info. This should be a go binary that runs with a few cli flags or environmental variables to configure it. (Ideally just needs two urls). This binary should be in an open-source repo outside of weave. Make a new one, or reuse an existing one.
This collector should connect to (1) a postgresql db and (2) a tendermint node rpc server. On connection to (1), it should check if there is the given schema and run a migration if needed. If there is some data, it should check the most recent block imported and begin after that.
For (2), it should query historical blocks starting with first block not stored in the database, and continue until it has caught up with the blockchain. At that point it should move to subscribe mode and use a block header subscription to update immediately for every block.
Data to store:
Storing this data will resolve this story. The only queries that must be defined (provide sql that can be executed on psql) are:
A key aspect of the validator rewards hinges on knowing when a validator missed its opportunity to propose a block. Therefore, we need to ensure data that can be used to make the determination is recorded.
One possible means to determine a missed proposal opportunity is to observe proposer_priority
. If, for a given validator, proposer_priority
decreased but the validator was not the block proposer then it missed its opportunity to propose. consensus.create_empty_blocks=false
might complicate things.
Please determine the best means to calculate a missed proposal opportunity and ensure that the required data is collected.
Current implementation saves source and the destination of send tx as
{"path":"cash/send","details":"{\"metadata\":{\"schema\":1},\"source\":\"C1888F21C55E9EF00B0220C7CAECBE862C4591F0\",\"destination\":\"FABE94B2D4CF2A7513888F501BB92D7C10B63C74\",\"amount\":{\"fractional\":1,\"ticker\":\"IOV\"}}"}
We need to save addresses as bech32
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.