This library bundles different components for lower-level peer-to-peer connection and message exchange:
- Distributed Peer Table (DPT) / Node Discovery
- RLPx Transport Protocol
- Ethereum Wire Protocol (ETH)
- Light Ethereum Subprotocol (LES/2)
The library is based on ethereumjs/node-devp2p as well
as other sub-libraries (node-* named) (all outdated).
To build the dist/ directory, run:
npm run build
You can also use ts-node to run a script without first transpiling to js (you need to npm i --save-dev ts-node first):
node -r ts-node/register [YOUR_SCRIPT_TO_RUN.ts]
All components of this library are implemented as Node EventEmitter objects
and make heavy use of the Node.js network stack.
You can react on events from the network like this:
dpt.on('peer:added', (peer) => {
// Do something...
})
Basic example to connect to some bootstrap nodes and get basic peer info:
Communicate with peers to read new transaction and block information:
Run an example with:
DEBUG=devp2p:* node -r ts-node/register ./examples/peer-communication.ts
Maintain/manage a list of peers, see ./src/dpt/, also includes node discovery (./src/dpt/server.ts)
Create your peer table:
const dpt = new DPT(Buffer.from(PRIVATE_KEY, 'hex'), {
endpoint: {
address: '0.0.0.0',
udpPort: null,
tcpPort: null
}
})
Add some bootstrap nodes (or some custom nodes with dpt.addPeer()):
dpt.bootstrap(bootnode).catch((err) => console.error('Something went wrong!'))
See the following diagram for a high level overview on the library.
Distributed Peer Table. Manages a Kademlia DHT K-bucket (Kbucket) for storing peer information
and a BanList for keeping a list of bad peers. Server implements the node discovery (ping,
pong, findNeighbours).
Creates new DPT object
privateKey- Key for message encoding/signing.options.refreshInterval- Interval in ms for refreshing (callingfindNeighbours) the peer list (default:60s).options.createSocket- A datagram (dgram)createSocketfunction, passed toServer(default:dgram.createSocket.bind(null, 'udp4')).options.timeout- Timeout in ms for serverping, passed toServer(default:10s).options.endpoint- Endpoint information to send with the serverping, passed toServer(default:{ address: '0.0.0.0', udpPort: null, tcpPort: null }).
Uses a peer as new bootstrap peer and calls findNeighbouts.
peer- Peer to be added, format{ address: [ADDRESS], udpPort: [UDPPORT], tcpPort: [TCPPORT] }.
Adds a new peer.
object- Peer to be added, format{ address: [ADDRESS], udpPort: [UDPPORT], tcpPort: [TCPPORT] }.
For other utility functions like getPeer, getPeers see ./src/dpt/dpt.ts.
Events emitted:
| Event | Description |
|---|---|
| peer:added | Peer added to DHT bucket |
| peer:removed | Peer removed from DHT bucket |
| peer:new | New peer added |
| listening | Forwarded from server |
| close | Forwarded from server |
| error | Forwarded from server |
Connect to a peer, organize the communication, see ./src/rlpx/
Instantiate an @ethereumjs/common instance with the network you want to connect to:
const common = new Common({ chain: 'mainnet' })Create your RLPx object, e.g.:
const rlpx = new devp2p.RLPx(PRIVATE_KEY, {
dpt: dpt,
maxPeers: 25,
capabilities: [devp2p.ETH.eth63, devp2p.ETH.eth62],
common: common,
listenPort: null,
})Manages the handshake (ECIES) and the handling of the peer communication (Peer).
Creates new RLPx object
privateKey- Key for message encoding/signing.options.timeout- Peerpingtimeout in ms (default:10s).options.maxPeers- Max number of peer connections (default:10).options.clientId- Client ID string (default example:ethereumjs-devp2p/v2.1.3/darwin-x64/nodejs).options.remoteClientIdFilter- Optional list of client ID filter strings (e.g.['go1.5', 'quorum']).options.capabilities- Upper layer protocol capabilities, e.g.[devp2p.ETH.eth63, devp2p.ETH.eth62].options.listenPort- The listening port for the server ornullfor default.options.dpt-DPTobject for the peers to connect to (default:null, noDPTpeer management).
Manually connect to peer without DPT.
peer- Peer to connect to, format{ id: PEER_ID, address: PEER_ADDRESS, port: PEER_PORT }.
For other connection/utility functions like listen, getPeers see ./src/rlpx/rlpx.ts.
Events emitted:
| Event | Description |
|---|---|
| peer:added | Handshake with peer successful |
| peer:removed | Disconnected from peer |
| peer:error | Error connecting to peer |
| listening | Forwarded from server |
| close | Forwarded from server |
| error | Forwarded from server |
Upper layer protocol for exchanging Ethereum network data like block headers or transactions with a node, see ./src/eth/.
Send the initial status message with sendStatus(), then wait for the corresponding status message
to arrive to start the communication.
eth.once('status', () => {
// Send an initial message
eth.sendMessage()
})
Wait for follow-up messages to arrive, send your responses.
eth.on('message', async (code, payload) => {
if (code === devp2p.ETH.MESSAGE_CODES.NEW_BLOCK_HASHES) {
// Do something with your new block hashes :-)
}
})
See the peer-communication.ts example for a more detailed use case.
Handles the different message types like NEW_BLOCK_HASHES or GET_NODE_DATA (see MESSAGE_CODES) for
a complete list. Currently protocol versions PV62 and PV63 are supported.
Normally not instantiated directly but created as a SubProtocol in the Peer object.
version- The protocol version for communicating, e.g.63.peer-Peerobject to communicate with.send- Wrappedpeer.sendMessage()function where the communication is routed to.
Send initial status message.
status- Status message to send, format{td: TOTAL_DIFFICULTY_BUFFER, bestHash: BEST_HASH_BUFFER, genesisHash: GENESIS_HASH_BUFFER },networkId(respectivelychainId) is taken from theCommoninstance
Send initial status message.
code- The message code, seeMESSAGE_CODESfor available message types.payload- Payload as a list, will be rlp-encoded.
Events emitted:
| Event | Description |
|---|---|
| message | Message received |
| status | Status info received |
Upper layer protocol used by light clients, see ./src/les/.
Send the initial status message with sendStatus(), then wait for the corresponding status message
to arrive to start the communication.
les.once('status', () => {
// Send an initial message
les.sendMessage()
})
Wait for follow-up messages to arrive, send your responses.
les.on('message', async (code, payload) => {
if (code === devp2p.LES.MESSAGE_CODES.BLOCK_HEADERS) {
// Do something with your new block headers :-)
}
})
See the peer-communication-les.ts example for a more detailed use case.
Handles the different message types like BLOCK_HEADERS or GET_PROOFS_V2 (see MESSAGE_CODES) for
a complete list. Currently protocol version LES/2 running in client-mode is supported.
Normally not instantiated directly but created as a SubProtocol in the Peer object.
version- The protocol version for communicating, e.g.2.peer-Peerobject to communicate with.send- Wrappedpeer.sendMessage()function where the communication is routed to.
Send initial status message.
status- Status message to send, format{ headTd: TOTAL_DIFFICULTY_BUFFER, headHash: HEAD_HASH_BUFFER, headNum: HEAD_NUM_BUFFER, genesisHash: GENESIS_HASH_BUFFER },networkId(respectivelychainId) is taken from theCommoninstance
Send initial status message.
code- The message code, seeMESSAGE_CODESfor available message types.reqId- Request ID, will be echoed back on response.payload- Payload as a list, will be rlp-encoded.
Events emitted:
| Event | Description |
|---|---|
| message | Message received |
| status | Status info received |
There are unit tests in the test/ directory which can be run with:
npm run test
This library uses debug debugging utility package.
For the debugging output to show up, set the DEBUG environment variable (e.g. in Linux/Mac OS:
export DEBUG=*,-babel).
Use the DEBUG environment variable to active the logger output you are interested in, e.g.:
DEBUG=devp2p:dpt:*,devp2p:eth node -r ts-node/register [YOUR_SCRIPT_TO_RUN.ts]
For more verbose output on logging (e.g. to output the entire msg payload) use the verbose logger
in addition:
DEBUG=devp2p:dpt:*,devp2p:eth,verbose node -r ts-node/register [YOUR_SCRIPT_TO_RUN.ts]
Exemplary logging output:
Add peer: 52.3.158.184:30303 Geth/v1.7.3-unstable-479aa61f/linux-amd64/go1.9 (eth63) (total: 2)
devp2p:rlpx:peer Received body 52.169.42.101:30303 01c110 +133ms
devp2p:rlpx:peer Message code: 1 - 0 = 1 +0ms
devp2p:rlpx refill connections.. queue size: 0, open slots: 20 +1ms
devp2p:rlpx 52.169.42.101:30303 disconnect, reason: 16 +1ms
Remove peer: 52.169.42.101:30303 (peer disconnect, reason code: 16) (total: 1)
For a complete API reference see the generated documentation.
To update the structure diagram files in the root folder open the devp2p.drawio file in draw.io, make your changes, and open a PR with the updated files. Export svg and png with border width=20 and transparency=false. For png go to "Advanced" and select 300 DPI.
The following is a list of major implementations of the devp2p stack in other languages:
- Python: pydevp2p
- Go: Go Ethereum
- Elixir: Exthereum
- Blog article series on implementing Ethereum protocol stack
MIT
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent
