A Go implementation of the Vega Protocol, a protocol for creating and trading derivatives on a fully decentralised network.
Home Page: https://vega.xyz
License: GNU Affero General Public License v3.0
In GitLab by @cdm on May 23, 2018, 14:46
Can access Tendermint RPC server via a separated/tested client, currently the HTTP call is baked into the Order REST API.
In GitLab by @cdm on Jul 3, 2018, 18:47
At the moment we’re doing this by having tendermint replay the entire blockchain on startup. Hopefully the app could one day start up without having to process every transaction ever — e.g. should to be able to restore known internal state and read store states from a given block height / hash then only play in transactions since then
In GitLab by @edd on Jun 5, 2018, 13:29
On our current constructiveproof testnet (until product#11), if one of the three nodes goes down, no more blocks are made. There are a few theories as to why, but we should have a better understanding of exactly why it is, and address it. If we need four nodes to allow one node to go down, let's add another node. If it's a configuration fix, let's do that.
In GitLab by @edd on Jun 5, 2018, 16:13
As part of product#11, we're moving off constructiveproof and on to some Digital Ocean nodes that @barnabee set up for us. This should give us all the ability to restart servers when they're down and deploy new versions
In GitLab by @asktav on May 31, 2018, 16:45
A small web app needs to be created that serves the custom import path lookup for tools such as go get.
So, if the import path was something like:
import "code.vegaprotocol.io/core/blockchain"Where code.vegaprotocol.io/core mapped to our trading-core repo.
Then the go tool will make an HTTP request to this corresponding URL:
And expect that to return an HTML page with the following <meta> tag inside the <head>:
<meta name="go-import" content="code.vegaprotocol.io/core git https://gitlab.com/vega-protocol/trading-core.git">repo/subpath if the query parameter is not present?In GitLab by @cdm on Jul 5, 2018, 16:41
The VEGA core package will now take on a role of setting up and coordinating the internal engines.
In GitLab by @cdm on May 31, 2018, 15:53
Update stats.vegaprotocol.io to use the following info:
gRPC/REST only, GQL to follow in subsequent ticket(?) -- /statistics
In GitLab by @asktav on May 31, 2018, 10:43
In GitLab by @cdm on Jul 11, 2018, 11:09
Motivation:
Task:
Upgrade to latest tendermint version to resolve the above issues
In GitLab by @edd on Jul 6, 2018, 11:34
Tav guarantees that the serialisation and deserialisation inherent in protobuffs will eventually become a bottleneck and suggests trying some alternatives, both of which can still be generated from the same canonical protobuff definition.
In GitLab by @cdm on May 23, 2018, 14:43
As per title, test on @eddhannay to see if he can change the trading-ui to work with the new REST API without external input
In GitLab by @edd on Jun 21, 2018, 19:02
Our current docker build process for testnet is way more complex than it needs to be. We've got a Vega dockerfile and another separate file, both of which need to be run and then manually deployed to the testnet.
vega-container repoDockerfile from trading-coreIn GitLab by @cdm on May 31, 2018, 15:37
Following a discussion with @eddhannay over basic requirements for the stats dashboard, we need to expose several additional queries via the trading-core service layer / API endpoints (rest to start with). Therefore the new custom WS RPC client needs to support the following calls to endpoints:
Attached is the reference from the stats dashboard that shows loosely how that code converts the raw tendermint rpc response data into 'dashboard stats' that are interesting, we can provide this via our service layer using rest
Reference for each rpc method (https://tendermint.github.io/slate/)
More info on ABCI functions (http://tendermint.readthedocs.io/en/master/abci-spec.html)
In GitLab by @edd on Jul 10, 2018, 17:48
We need an API that can provide a vega client with an overview of their current positions, i.e. details of all their open orders across all markets. Using this we can shows a trader's current positions and their trading history.
In GitLab by @cdm on Jun 11, 2018, 11:48
When the chain is re-played we need to build all orders and trades back from the start including unique deterministic IDs.
All types have an ID field whose value is of the format:
<ID_HASH_VERSION_BYTE> <HASH_BYTES>
Hash(NodeID, “/“, OrderSequenceID)
— NodeID/OrderSequenceID (hashed as above)
NodeID = Generated on VEGA app start up for the particular node
OrderSequenceID = UUID
— Order Id triggered trade, block height, Trade sequence Id for block
Hash(BlockHeight, “/“, OrderID (which triggered the trade), TradeSequenceIDForBlock)
On StartBlock, reset TradeSequenceiDForBlock to 0
In GitLab by @cdm on Jul 21, 2018, 18:07
As a trader
I can cancel my open orders
So that I can respond to changes in the market
In GitLab by @cdm on May 31, 2018, 11:09
IIRC it currently forces the user to set their checkout path / gopath to ~/src/vega/*
This should be flexible to meet developer path requirements.
In GitLab by @cdm on Aug 10, 2018, 15:59
Start thinking about modularisation of VEGA core, specifically settings or configuration for each engine.
In GitLab by @dmgarland on Jul 3, 2018, 16:51
AFAIK with default paths, go expects this repo to be checked out under $GOPATH/src/vega, whereas the clone command would put it under a folder called trading-core. Anyone object to having this repo renamed?
In GitLab by @cdm on Jul 13, 2018, 14:08
The error reporting can be improved by following this guide:
In GitLab by @asktav on May 30, 2018, 18:36
There are two options for the RPC client, using JSON-RPC or WebSockets. We're using WebSockets as it gives us access to the subscribe functionality.
For now, only implement the BroadcastTxCommit method.
In GitLab by @cdm on Aug 10, 2018, 16:01
Historic candles dont appear to be keeping close/open prices correctly, see below for last 50 candles at 3600 seconds (1 hour) size:
"vega": {
"markets": [
{
"candles": [
{
"open": "1",
"close": "1",
"low": "1",
"high": "1",
"date": "2018-07-13T10:32:21Z"
},
{
"open": "1",
"close": "1",
"low": "1",
"high": "1",
"date": "2018-07-13T11:32:21Z"
},
{
"open": "1",
"close": "1",
"low": "1",
"high": "1",
"date": "2018-07-13T12:32:21Z"
},
{
"open": "1",
"close": "1",
"low": "1",
"high": "1",
"date": "2018-07-13T13:32:21Z"
},
{
"open": "1",
"close": "1",
"low": "1",
"high": "1",
"date": "2018-07-13T14:32:21Z"
},
{
"open": "1",
"close": "1",
"low": "1",
"high": "1",
"date": "2018-07-13T15:32:21Z"
},
{
"open": "1",
"close": "1",
"low": "1",
"high": "1",
"date": "2018-07-13T16:32:21Z"
},
{
"open": "1",
"close": "1",
"low": "1",
"high": "1",
"date": "2018-07-13T17:32:21Z"
},
{
"open": "1",
"close": "1",
"low": "1",
"high": "1",
"date": "2018-07-13T18:32:21Z"
},
{
"open": "1",
"close": "1",
"low": "1",
"high": "1",
"date": "2018-07-13T19:32:21Z"
},
{
"open": "1",
"close": "1",
"low": "1",
"high": "1",
"date": "2018-07-13T20:32:21Z"
},
{
"open": "1",
"close": "1",
"low": "1",
"high": "1",
"date": "2018-07-13T21:32:21Z"
},
{
"open": "42",
"close": "42",
"low": "42",
"high": "42",
"date": "2018-07-13T22:32:21Z"
},
{
"open": "42",
"close": "1000",
"low": "42",
"high": "1000",
"date": "2018-07-13T23:32:21Z"
},
{
"open": "1000",
"close": "1000",
"low": "1000",
"high": "1000",
"date": "2018-07-14T00:32:21Z"
},
{
"open": "1000",
"close": "1000",
"low": "1000",
"high": "1000",
"date": "2018-07-14T01:32:21Z"
},
{
"open": "1000",
"close": "1000",
"low": "1000",
"high": "1000",
"date": "2018-07-14T02:32:21Z"
},
{
"open": "1000",
"close": "1000",
"low": "1000",
"high": "1000",
"date": "2018-07-14T03:32:21Z"
},
{
"open": "100",
"close": "100",
"low": "100",
"high": "1000",
"date": "2018-07-14T04:32:21Z"
},
{
"open": "100",
"close": "121",
"low": "100",
"high": "121",
"date": "2018-07-14T05:32:21Z"
},
{
"open": "121",
"close": "121",
"low": "121",
"high": "121",
"date": "2018-07-14T06:32:21Z"
},
{
"open": "121",
"close": "121",
"low": "121",
"high": "121",
"date": "2018-07-14T07:32:21Z"
},
{
"open": "121",
"close": "121",
"low": "121",
"high": "121",
"date": "2018-07-14T08:32:21Z"
},
{
"open": "121",
"close": "121",
"low": "121",
"high": "121",
"date": "2018-07-14T09:32:21Z"
},
{
"open": "121",
"close": "121",
"low": "121",
"high": "121",
"date": "2018-07-14T10:32:21Z"
},
{
"open": "121",
"close": "121",
"low": "121",
"high": "121",
"date": "2018-07-14T11:32:21Z"
},
{
"open": "121",
"close": "120",
"low": "120",
"high": "121",
"date": "2018-07-14T12:32:21Z"
},
{
"open": "0",
"close": "0",
"low": "0",
"high": "0",
"date": "2018-07-14T13:32:21Z"
},
{
"open": "0",
"close": "0",
"low": "0",
"high": "0",
"date": "2018-07-14T14:32:21Z"
},
{
"open": "0",
"close": "0",
"low": "0",
"high": "0",
"date": "2018-07-14T15:32:21Z"
},
{
"open": "0",
"close": "0",
"low": "0",
"high": "0",
"date": "2018-07-14T16:32:21Z"
}
],
"orders": [
{
"id": "V41095-24",
"remaining": "42"
}
]
}
]
}
}
In GitLab by @cdm on Jul 26, 2018, 13:57
It might be possible to overflow the system when calculating partyPositions (MarketPositons) because we use uint64 for size and price everywhere else. This defensive fix will prevent overflow when have 4 trillion 400 billion+ PNL
In GitLab by @cdm on Aug 6, 2018, 16:52
The VEGA.trading web-app requires CORS support from the trading-core to operate as designed (via GraphQL).
Currently (initial impl) does not support CORS
In GitLab by @cdm on Jul 16, 2018, 11:52
Unit tests and integration tests (if possible)
In GitLab by @edd on Jul 6, 2018, 12:04
FIX support will be required as an integration point
The Financial Information eXchange (FIX) Protocol has revolutionised the trading environment, proving fundamental in facilitating many of the electronic trading trends that have emerged over the past decade.
FIX has become the language of the global financial markets used extensively by buy and sell-side firms, trading platforms and even regulators to communicate trade information. This non-proprietary, free and open standard is constantly being developed to support evolving business and regulatory needs, and is used by thousands of firms every day to complete millions of transactions.
https://www.fixtrading.org/what-is-fix/
http://fixwiki.fixtrading.org/index.php/Category:Message
In GitLab by @asktav on May 31, 2018, 10:41
Incorporate the various ABCI-related protobuf types into our repo, with an associated generator script for a Go package.
In GitLab by @cdm on Aug 3, 2018, 09:31
Add filtering to REST/gRPC endpoints using filter support - use common rest params etc to allow easy to use API for developers
In GitLab by @cdm on Aug 10, 2018, 15:45
In GitLab by @cdm on Aug 3, 2018, 12:34
Observe by visiting the following stream:
http://ldn.testnet.vegaprotocol.io:3004/query
Enter the subscription and watch the output
subscription {
candles(market:"BTC/DEC18", interval:5) {
open,
close,
high,
low,
date
}
}
Maks already identified fix for 0 close price, identify and fix all 0 bug also.
In GitLab by @cdm on Aug 3, 2018, 12:06
Currently the logger only logs to console, we should also allow logging to file (new file daily)
Tasks
Default location /tmp/vegalog.txt
Also ability to override on command line flag e.g. -logfile=/blah/projects/testfile.log
In GitLab by @edd on Jun 1, 2018, 09:54
In the CI process we should build a docker image and push it to the Gitlab registry. This will help us set up a testnet (product#11).
In GitLab by @barnabee on Mar 12, 2018, 18:37
Currently the matching-engine maintains Order and Trade state internally. This state is not exposed to other packages. The trades and orders are sent over two broadcast channels and these channels receive orders/trades as they are created. The initial plan is to subscribe to the two channels from a datastore component that can save the order/trade data down to memory or a DB engine e.g. sqlite.
Provide a way for the API-services to read trade/order data without querying the matching engine component directly.
In GitLab by @barnabee on Mar 13, 2018, 18:39
The ME is quick in what are likely to be normal cases but the performance fall-off when submitting large numbers of randomised orders is exponential. There's no reason it ought to be and it's not yet clear if it scales with the size of the book or the cumulative total orders submitted, if the latter that would be a problem.
Potential culprits:
➜ tests (master) ✗ go test -bench=.
goos: darwin
goarch: amd64
pkg: vega/src/tests
BenchmarkMatching100-8 3000 403336 ns/op
BenchmarkMatching1000-8 300 5055301 ns/op
BenchmarkMatching10000-8 10 131372499 ns/op
BenchmarkMatching100000-8 1 13832542777 ns/op
BenchmarkMatching100Allocated-8 3000 415770 ns/op
BenchmarkMatching1000Allocated-8 300 5064084 ns/op
BenchmarkMatching10000Allocated-8 10 133959081 ns/op
BenchmarkMatching100000Allocated-8 1 15091063252 ns/op
BenchmarkMatching100Uniform-8 5000 331250 ns/op
BenchmarkMatching1000Uniform-8 300 4175048 ns/op
BenchmarkMatching10000Uniform-8 10 112162524 ns/op
BenchmarkMatching100000Uniform-8 1 12883249625 ns/op
PASS
ok vega/src/tests 56.036s
(n=100) Elapsed = 624.523µs, average = 6.245µs; matched 34 trades, average 0.34 trades per order.
(n=1000) Elapsed = 6.461691ms, average = 6.461µs; matched 390 trades, average 0.39 trades per order.
(n=10000) Elapsed = 133.170407ms, average = 13.317µs; matched 3938 trades, average 0.3938 trades per order.
(n=100000) Elapsed = 12.658638828s, average = 126.586µs; matched 39465 trades, average 0.39465 trades per order.
(n=1000000) Elapsed = 1h3m18.882776069s, average = 3.798882ms; matched 394627 trades, average 0.394627 trades per order
In GitLab by @cdm on Jul 22, 2018, 18:23
In GitLab by @edd on Jul 6, 2018, 10:24
!15
In GitLab by @barnabee on Mar 12, 2018, 18:38
We need to add an API that can feed data in to #26. It's currently looking like react-stockcharts is the winner. For a basic candlestick chart it expects data that looks a bit like this: https://gitlab.com/vega-protocol/client/blob/add-react-stockcharts/trading-ui/src/components/chart-view/chart-view.tsx#L18-31
{
date: parse(`2018-04-${index}`),
high: randomIntFromInterval(1, 100),
low: randomIntFromInterval(1, 25),
open: randomIntFromInterval(1, 30),
close: randomIntFromInterval(1, 70),
volume: randomIntFromInterval(1, 300)
}We need two things from the API:
The client side API client is this horrible TS file: https://gitlab.com/vega-protocol/client/blob/add-react-stockcharts/trading-ui/src/api/index.ts
In an ideal world, this API would be an API subscription, but that entails a lot of setup on both the server and client side. As a compromise, it should probably be an event stream that, on connection, fires off a bit array of historical data, then individual events as new candles come in. These should trigger redux events, which should replace the current random data generator in https://gitlab.com/vega-protocol/client/blob/add-react-stockcharts/trading-ui/src/components/chart-view/chart-view.tsx
api/index.tsIn:
Out:
An array of
Websockets? Request? Both?
In GitLab by @cdm on Aug 13, 2018, 16:06
In GitLab by @cdm on Aug 3, 2018, 09:24
Add filtering to GraphQL queries, using the new datastore query filters maks has built for the platform.
In GitLab by @cdm on Aug 13, 2018, 12:53
Improve unit test coverage on resolvers (graphql
In GitLab by @asktav on May 30, 2018, 13:48
Blah blah
In GitLab by @cdm on May 31, 2018, 11:29
trading-core has a basic / working CI pipeline which 'does the job' for now, however we can improve on this and expand the ci process to include things like data race and mem sanitiser, improve (fix) the coverage output.
For bonus points, generate and store a docker image in the gitlab registry. Starting point:
In GitLab by @cdm on Jul 9, 2018, 17:18
Our initial datastore implementation does not have both foreign relations passed when storing a Trade. This was an oversight, having discussed with @makspachucki we need to pass order ID's from both aggressive and passive orders to the datastore.
In datastore/types.go:
type Trade struct { msg.Trade OrderId string }
should become:
type Trade struct { msg.Trade PassiveOrderId string AggressiveOrderId string }
Within the VEGA core app we can then update all relationships for new aggressive orders, new trades AND update any remaining on passive orders.
In GitLab by @barnabee on Mar 12, 2018, 18:37
We need an API to drive product#31. The client side implementation of this ticket is already partially implemented, but hasn't yet been hooked up to an API - you can the current implementation by following the instructions here: https://gitlab.com/vega-protocol/client/tree/master/trading-ui/src/stories
The orderbook API needs to take a market as an argument, and return an orderbook type outlined below. This should contain a number of price levels, the number of orders at that price level, and the cumulative volume to that price level. Price levels are defined as part of the product definition based on a price interval - for BTCDEC18 the interval could be 5, making the price levels that a trader can place orders at 5, 10, 15, 20 etc. Each side of the current mark price should contain n price levels, where n is a parameter that defaults to 10.
This API is also likely to drive the Market Depth Chart
message OrderBookDepth {
message DepthAtPriceLevel {
uint64 price
uint64 numOrders
uint64 volumeAtLevel
uint64 cumulativeVolumeAtLevel -- total cumulative remaining size of orders at or better than price, reversed for sell
}
repeated DepthAtPriceLevel buyDepth
repeated DepthAtPriceLevel sellDepth
}
export type OrderBook = {
loading: boolean
loaded: boolean
id: string
bestBid: number
bestOffer: number
midPoint: number
buySide: PriceLevel[]
sellSide: PriceLevel[]
orders: Order[]
}
export type PriceLevel = {
side: Side
price: number
volume: number
orders?: Order[]
total: number
}In GitLab by @cdm on May 23, 2018, 14:28
Currently the vega->main.go adds order/trade channels as follows:
orderSseChan := make(chan msg.Order, sseChannelSize)
tradeSseChan := make(chan msg.Trade, sseChannelSize)
...
config := core.DefaultConfig()
config.Matching.OrderChans = append(config.Matching.OrderChans, orderSseChan)
config.Matching.TradeChans = append(config.Matching.TradeChans, tradeSseChan)
@barnabee suggested we implement a listener approach to subscribing to order channels rather than using append in this way.
In GitLab by @asktav on May 31, 2018, 10:45
We're currently seeing a number of issues with Tendermint where it stops passing along messages after "falling over" at high throughputs. Figure out what's causing the issue and see if we can resolve it, or perhaps work with upstream to resolve it, if it is an issue with Tendermint itself.
In GitLab by @barnabee on Mar 12, 2018, 18:34
In GitLab by @cdm on May 30, 2018, 14:59
Something
In GitLab by @edd on Jun 21, 2018, 15:48
We currently have a situation where a bad transaction submitted to Tendermint will pass CheckTX, but when Tendermint passes it in to DeliverTX it will return a non-OK response code and all subsequent transactions will sit in the backlog forever.
https://gitlab.com/vega-protocol/trading-core/blob/develop/blockchain/blockchain.go#L60
curl 'http://ldn.testnet.vegaprotocol.io:46657/broadcast_tx_async?tx=%221.23452536666973245205||||eyJNYXJrZXQiOiJCVEMvREVDMTgiLCJQYXJ0eSI6IjczNzBmMzRhOWFmOTQ0NjdjYjFlMjVlMGI4NWE1MWFhIiwiU2lkZSI6MCwiUHJpY2 UiOjEsIlNpemUiOjEsIlJlbWFpbmluZyI6MSwiVHlwZSI6MCwiVGltZXN0YW1wcyI6MTUyNTI1NDAzMjc4NH0=%22'
Transaction should be rejected
Transaction will be accepted, and no more blocks will ever be produced
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.