Giter Club home page Giter Club logo

k-rpc's Introduction

k-rpc

Low-level implementation of the k-rpc protocol used by the BitTorrent DHT.

npm install k-rpc

Read BEP 5 and BEP 44 for more background info.

build status

Usage

var krpc = require('k-rpc')
var rpc = krpc()

var target = new Buffer('aaaabbbbccccddddeeeeffffaaaabbbbccccdddd', 'hex')

// query the BitTorrent DHT to find nodes near the target buffer
rpc.closest(target, {q: 'get_peers', a: {info_hash: target}}, onreply, done)

function onreply (message, node) {
  console.log('visited peer', message, node)
}

function done () {
  console.log('(done)')
}

API

var rpc = krpc([options])

Create a new rpc instance. Options include

{
  // per peer query timeout defaults to 2s
  timeout: 2000,
  // an array of bootstrap nodes. defaults to the BitTorrent bootstrap nodes
  nodes: ['example.com:6881'],
  // how many concurrent queries should be made. defaults to 16
  concurrency: 16,
  // how big should be routing buckets be. defaults to 20.
  k: 20,
  // the local node id. defaults to 20 random bytes
  id: Buffer(...),
  // Length of ID in bytes, defaults to 20 (sha1)
  idLength: 20,
  // optional k-rpc-socket instance
  krpcSocket: krpcSocket(opts)
}

rpc.id

Buffer containing the local node id.

rpc.nodes

Routing table populated by running rpc.populate. This is a k-bucket instance.

rpc.populate(target, query, [callback])

Populate the rpc.nodes routing table with nodes discovered by looking for other peers close to our own local node id using the specified query. The internal routing table will be used for subsequent closest queries to take load of the bootstrap nodes.

// send a find_node query
rpc.populate(rpc.id, {q: 'find_node', a: {id: rpc.id, target: rpc.id}}, function () {
  console.log('internal routing table fully populated')
})

You should call this method as soon as possible to spread out query load in the DHT. Callback is called with (err, numberOfReplies).

rpc.closest(target, query, onreply, [callback])

Find peers close the specified target buffer whilst sending the specified query. onreply will be called with (reply, node) for every reply received and the callback is called with (err, totalNumberOfReplies).

// find peers sharing a torrent info_hash
rpc.closest(infoHash, {q: 'get_peers', a: {id: rpc.id: info_hash: infoHash}}, onreply, function () {
  console.log('no more peers to be found')
})

function onreply (message, node) {
  if (message.r && message.r.values) console.log('received peers')
}

If a closest query is being executed while a population request in being run the closest query will take priority.

You can return false from onreply to stop the query. This is useful if you are only looking for a single peer for example.

function onreply(message, node) {
  console.log('will only fire once')
  return false
}

rpc.query(node, query, callback)

Query a single node. If the node has a token it is set as a.token in the query automatically. Callback is called with (err, reply).

rpc.queryAll(nodes, query, onreply, callback)

Query multiple nodes with the same query. query.a.token will be set as the corresponding nodes token when querying. Callback is called with (err, numberOfReplies) and onreply will be caleld with (reply, node) as the nodes reply.

rpc.destroy()

Destroy the underlying rpc socket.

rpc.on('query', query, node)

Emitted when a query is received.

rpc.response(node, query, response, [nodes], [callback])

Send a response to a node for a specific query. If you pass in an array of nodes {id: nodeId, host: someHost, port: somePort} they will be added to the response.

rpc.error(node, query, error, [callback])

Send an error response for a query.

rpc.on('ping', oldNodes, swapNew)

Emitted when the bucket is full. Try and oldNodes and if one of them fails call swapNew with that node to swap if for a newer one

License

MIT

k-rpc's People

Contributors

feross avatar kr1sten0 avatar mafintosh avatar nazar-pc avatar red54 avatar tehshrike avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

k-rpc's Issues

Possible memory leak in .pending array

@mafintosh @feross the .pending array in this module seems to accumulate an enormous number of stale entries if a mainline DHT is left running connected to the open internet for several days. Below is a heap snapshot from such a service before it crashed due to memory exhaustion (with 95Mb being used by the pending array):

screenshot

My apologies for the image but it does not seem to be possible to export data from a heap snapshot in Chrome. What you can see in this screenshot is a single entry of RPC.pending expanded and you can see that it has an IP address in the host field and that the message is {q: ping}. There are around 70,000 of these .pending elements in the array after the DHT has been running for several days according to the heap snapshot and as far as I can tell they are all "ping" messages.

Is there some reason why the k-rpc instance inside bittorrent-dht should continue keeping those "ping" messages around for a long time? Could it be because it cannot reach the destination host? Perhaps it would be prudent to maintain a timestamp on each entry that goes into the .pending array and expire them after some reasonable time?

I will try to create a simple test case which reliably produces this memory exhaustion but in the mean time perhaps a fix could be to add a timestamp to the entries in the pending array and expire them?

Thanks very much for your time.

Better way to handle No nodes to query error?

I'm using bittorrent-dht to put and get arbitrary data on the DHT.

K-RPC is generating this error despite successful puts & gets. I've tried to suppress this error in my code but it looks like line 127 has it hard coded. Could you simply quietly return?

if (!missing) return cb(new Error('No nodes to query'), 0)
Error: No nodes to query
    at RPC.queryAll (/Users/danraeder/Documents/GitHub/dht-keyvalue/node_modules/k-rpc/index.js:127:27)
    at DHT._put (/Users/danraeder/Documents/GitHub/dht-keyvalue/node_modules/bittorrent-dht/client.js:318:15)
    at DHT.put (/Users/danraeder/Documents/GitHub/dht-keyvalue/node_modules/bittorrent-dht/client.js:283:17)
    at /Users/danraeder/Documents/GitHub/dht-keyvalue/node_modules/bittorrent-dht/client.js:337:12
    at done (/Users/danraeder/Documents/GitHub/dht-keyvalue/node_modules/bittorrent-dht/client.js:677:7)
    at done (/Users/danraeder/Documents/GitHub/dht-keyvalue/node_modules/k-rpc/index.js:248:5)
    at processTicksAndRejections (node:internal/process/task_queues:78:11)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.