Getting this in bthread after updating bcoin:

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: connect ECONNREFUSED
    at errnoException (net.js:904:11)
    at Object.afterConnect [as oncomplete] (net.js:895:19)

I think this lines should be changed:

• https://github.com/indutny/bcoin/blob/master/lib/bcoin/pool.js#L122
• https://github.com/indutny/bcoin/blob/master/lib/bcoin/pool.js#L265

To emit('peerError', ...).

The condition itself is not-fatal, right?

When using bcoin in SPV mode, it fails to sync the chain. e.g. adding:

    network: 'testnet'

in the options in bin/spvnode, the log looks like:

[debug] Requesting headers packet from peer with getheaders (testnet-seed.bitcoin.schildbach.de:18333).
[debug] Height: 0, Hash: 000000000933ea01ad0ee984209779baaec3ced90fa3f408719526f8d77f4943, Stop: undefined
[info] Peer initialized compact blocks (testnet-seed.bitcoin.schildbach.de:18333).
[info] Connected to 144.76.136.19:18333.
[info] Received version (144.76.136.19:18333): version=70014 height=1012703 services=1101 agent=/Satoshi:0.13.0/
[debug] Added time data: samples=2, offset=0 (0 minutes).
[debug] Received verack (144.76.136.19:18333).
[info] Received 1 addrs (hosts=1, peers=2) (144.76.136.19:18333).
[debug] Refilling peers (2/8).
[info] Peer initialized compact blocks (144.76.136.19:18333).
[debug] Received 2000 headers from peer (testnet-seed.bitcoin.schildbach.de:18333).
[debug] Requesting 2000/2000 blocks from peer with getdata (testnet-seed.bitcoin.schildbach.de:18333).
[debug] Requesting headers packet from peer with getheaders (testnet-seed.bitcoin.schildbach.de:18333).
[debug] Height: -1, Hash: 0000000005bdbddb59a3cd33b69db94fa67669c41d9d32751512b5d7b68c71cf, Stop: undefined
[debug] Received 2000 headers from peer (testnet-seed.bitcoin.schildbach.de:18333).
[debug] Requesting headers packet from peer with getheaders (testnet-seed.bitcoin.schildbach.de:18333).
[debug] Height: -1, Hash: 00000000185b36fa6e406626a722793bea80531515e0b2a99ff05b73738901f1, Stop: undefined

And it continues at Height: -1 forever. Eventually it comes all the way around and re-downloads headers it previously downloaded. It appears it's failing to connect the received block headers?

one of the things that makes me wary of using this library is that it hasn't been tested in the wild. What is the main reason not to use something like bitcoinjs-lib or bitcore for the bitcoin primitives: tx, block, chain, etc? I can think of one - bitcoinjs-lib uses ecurve vs elliptic (and soon vs secp256k1-node i hope. bitcore uses elliptic i think), but a fork of bitcoinjs-lib or bitcore with the EC implementation swapped out would still feel more reliable.

Using LMDB would solve a lot of issues (primarily the LRU caches littered all over the wallet code). The wallet requires much faster reads than the chain, LMDB offers that, as well as hot backups (something that's currently hacked into the leveldb backend and only works because iterators snapshot the tree).

I've rewritten the node.js lmdb module completely (rvagg/archived-lmdb#14) and it seems stable enough. It doesn't expose envs/dbs/txns since it's a mirror of the leveldb API. Faster reads and hot backups are nice though.

Further rewriting: transactions (the non-bitcoin kind) would solve a lot of problems. A future module that properly exposes all LMDB features would be ideal (one for node.js exists, but the API is synchronous). There were a few times I tried to hack in a key "state" by wrapping a leveldb batch because I needed txns so bad.

I have a regtest privatekey (not extended, like 'cTKcQBz...') I want to import for tx signing. So I made a bcoin.keyring.fromSecret('privkey'), then I sign with tx.sign(keyring), but isSigned() returns false. What I'm doing wrong?

I am trying to use this library to set up a SPV node, but I have hard time due to a lack of documentation.
Is anybody taking care of that? I would like to help doing that, but I have no clue what to write.

Code:

Source: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray/of

Proposed solution:
use new Int16Array([1]).buffer instead. Should yield same result

Just wondering if BIP65 or BIP113 affect this.

Hello, wondering if anyone is using this in a wallet right now. I actually built a system off of coined/termcoin but that seems to be having an issue and I am not sure if its a bug in coined/termcoin, the way I am using it, or this library, or something that chained with the new version of bitcoin core. If people are using it for lightweight wallets today I would be glad to hear that, because this seems about 3 times simpler to integrate than bitcore in terms of BIP37 support.

Basically termcoin is unable to find a block sometimes and that causes it to crash.. I have an issue open with him also. Just sort of hoping to hear from someone that I am not the only one using this before I have to redo this using bitcore or something else because the other options seem more complicated to use as a lightweight wallet.

There will be another upcoming wallet migration (4 to 5). Hopefully it's the last one.

Bcoin's wallet db supports multiple wallets. This is tricky when it comes to indexing. The way I ended up doing it is to have every wallet "scoped" within its own environment for tx indexing. There is no global tx database with secondary indexing for wallet ids. If a tx maps to multiple wallets, it gets written to multiple txdb's.

Technically a waste of space, but it makes sense when you consider: code simplicity, the fact that wallets should be isolated from each other, and fast iteration over coins/txs/etc without having to do extra lookups to obtain the actual data.

That being said, there still needs to be 4 things that are globally exposed:

1. The chain sync state
2. Maps of block txs to wallet ids
3. Maps of outpoints to wallet ids
4. Maps of address hashes to wallet ids

Since the actual data is indexed under the wallet id, all we need are the ids (uint32s).

Chain Sync State

The chain sync state needs to be global to maintain atomicity with the chain. Blocks are connected globally -- writes of the sync state are not atomic with any wallet's txdb. This is okay because the whole point of the sync state is that it's allowed to fall out of sync during a crash in order to re-sync properly.

The real purpose of the sync state is to recover from a crash during a reorg. If we crash while blocks are being disconnected, a rescan isn't going to help that. The sync state allows us to rollback history until we find a block on the main chain and rescan from there.

Maps

Block Maps

Block maps are necessary for reorgs. We need to know which blocks and which transactions pertain to which wallets in order to roll back history.

Outpoint Maps

When a transaction comes in, we need to know if any of the outpoints its redeeming are ours. If one of them is, we need to figure out which wallets it pertains to (due to multisig, watch only, etc, it may pertain to multiple wallets).

Address Maps

Likewise, when a transaction comes in, we need to know if any of the output addresses map to wallets.

Unlike the chain sync state, maps are written atomically to the database. When a tx is inserted, the flow is something like:

for each output
  is output ours?
    retrieve global outpoint map (by txid/index)
    insert wallet-id into outpoint map
    save outpoint map

is transaction confirmed?
  retrieve global block map (by height)
  insert wallet-id into block map for txid
  save block map

This kind of sucks since it requires an extra read. If we didn't care about atomicity, we could do this as one big write from the outside.

The other thing that's rough is these maps might get big. The outpoint map will be 1 or 2 wallet ids in the average case, but it could be up to 30 in the worst case (assuming 2 15-of-15 multisig wallets are sending to each other -- not common).

The trickier thing is probably the block map. Depending on how many transactions bcoin gets in a block, this can be a pain to deserialize and reserialize constantly.

Again, if we didn't care about atomicity, these maps could be one big write and no reads.

Also note that a tx insertion must invoke a write lock for the whole wallet. A tx insertion may derive a new address. That address derivation needs to be in the same atomic write as the tx insertion.

There is another solution: to create one giant atomic write batch for all wallets, used during tx insertion. This involves adding a global write lock to the wallet database. This is also a pain and hurts performance (despite the fact that databases generally have an internal write lock and can't write multiple things at the same time), and my benchmarks confirm it. What this means: if a transaction is coming in, and a wallet unrelated to said tx wants to derive a new address -- they can't do it until the lock is released.

If anyone has some brighter ideas, I'd love to hear them.

It'd be extremely convenient to have a single folder with self-contained demos of various features (as raw HTML is fine!).

I noticed my transactions with multiple inputs (signed with different public keys) are getting rejected by peers:

Usally this is the error message:

Transaction rejected {"message":"tx","ccode":64,"reason":"non-canonical"}

Out of maybe 10-20 tests, I saw this from one peer:

Transaction rejected {"message":"tx","ccode":16,"reason":"mandatory-script-verify-flag-failed"}

This is just from using the standard tx.out tx.input and wallet.sign
methods.

After reading up on non-canonical DER signatures, apparently modern bitcoin DER
signatures can't be zero padded. @indutny, I figured you would know much more
about this than me. As far as I know, elliptic's DER signatures are implemented
correctly.

Here's a truncated/formatted example of one of the transactions that was rejected:

{ cost: '0.0901546',
  totalIn: '0.10577674',
  inputs:
   [ { value: '0.07079674',
       script:
        [ '3046022100d8c3c7cdaa6c63d9b60df7a099cf0357336e83d9e5141840a52c73c25c247c2c022100b6342503b23d604fe7e3ad55131f3b4001cb8731a585b96f399470c9e819989801',
          '025a45c74334ebaa81764929916b9e7720aadfcc1102b80bbc1e9c21c50bbd1786' ] },
     { value: '0.00008',
       script:
        [ '30440220715115caad5af46528ae2455ab2dd44a768144e14dff1d82da6092a24d79b3d30220325fcb84e0dded3f97b00c17772d6ea307ba0967091d3f8fd9694f5ae77382fa01',
          '025a45c74334ebaa81764929916b9e7720aadfcc1102b80bbc1e9c21c50bbd1786' ] },
     { value: '0.0099',
       script:
        [ '3046022100fc5b6024e539413e75d6d95ce4290ff13bf15e60639eb1881b04e8a3738ce9af0221009d744ee3829ac2fb0dab9258506668935923072faf7f7bb900e9a896f86bb42801',
          '025a45c74334ebaa81764929916b9e7720aadfcc1102b80bbc1e9c21c50bbd1786' ] },
     { value: '0.005',
       script:
        [ '30450220353aeea2021af2517a1f4040a0ec170bf496f2745df86029b8ed6f1f494976a8022100d14e6fb65a709c5925c9855b5b8b43465ca3d136b7236b3987a338881d2bfa5e01',
          '025a45c74334ebaa81764929916b9e7720aadfcc1102b80bbc1e9c21c50bbd1786' ] },
     { value: '0.02',
       script:
        [ '3046022100a482d747aea18a169296838015d998a49c938d80425b7913133ece9aef9f496e022100bf833a80f380c86bf579b84227369e077062599a24901d06ea78bc7cac3fe45f01',
          '025a45c74334ebaa81764929916b9e7720aadfcc1102b80bbc1e9c21c50bbd1786' ] } ],
  totalOut: '0.10567674',
  outputs:
   [ { value: '0.09',
       script: '["dup","hash160",[88,160,127,93,24,227,18,223,47,110,237,249,41,251,136,47,173,14,40,68],"eqverify","checksig"]' },
     { value: '0.01567674',
       script: '["dup","hash160",[219,80,8,122,56,94,134,123,84,115,69,225,80,246,117,3,196,160,182,166],"eqverify","checksig"]' } ] }

Obviously, I was sending 0.09 BTC to an address, and the change of 0.01567674 got returned to myself.

My second thought was, perhaps the transaction framer is incorrectly handling multiple inputs and producing a corrupt tx packet (missing a byte or something), but I checked the code and nothing seemed untoward. Here's the framed and reparsed transaction: https://gist.github.com/chjj/454aa88cf723c58ccb4b

Unfortunately, in that gist, I did not log the tx fully populated with unpent previous outputs, but I promise there were enough unspent outputs there. The code to produce the log in this post:

console.log({
  cost: utils.toBTC(amount.add(new bn(self.dust)).add(new bn(self.fee))),
  totalIn: utils.toBTC(tx.funds('in')),
  inputs: tx.inputs.map(function(input) {
    return {
      value: utils.toBTC(input.out.tx.outputs[input.out.index].value),
      script: [
        utils.toHex(input.script[0]),
        utils.toHex(input.script[1])
      ]
    };
  }),
  totalOut: utils.toBTC(tx.funds('out')),
  outputs: tx.outputs.map(function(output) {
    return {
      value: utils.toBTC(output.value),
      script: JSON.stringify(output.script)
    };
  })
});

I will continue to investigate this, but was wondering if you had any ideas, @indutny.

I'm running into a problem where the wallet's balance is not always calculated correctly. This happens very frequently. It seems to have to do with the order transactions are passed into wallet.tx.add(). Still investigating.

guys, thanks for making this!

i tried to run the example from the readme, with a testnet address I know to have confirmed transactions, and testnet dns seeds. Other than connection-related log messages, I never managed to get a single 'tx' event. Can you please create an example that's tested to work? Let me know if you need any more info on my setup

Hi,
please change pool.js
45 this.originalSeeds = (options.seeds || network.seeds).map(utils.parseHost);
50 this.size = options.size || 32;
51 this.parallel = options.parallel || 2000;
52 this.redundancy = options.redundancy || 1;

54 this._createConnection = options.createConnection;
55 this._createSocket = options.createSocket;

75 this.backoff = {
76 delta: options.backoffDelta || 500,
77 max: options.backoffMax || 5000
78 };

to
45 this.originalSeeds = (**this.**options.seeds || network.seeds).map(utils.parseHost)
50 this.size = **this.**options.size || 32;
51 this.parallel = **this.**options.parallel || 2000;
52 this.redundancy = **this.**options.redundancy || 1;

54 this._createConnection = this.options.createConnection || 0;
55 this._createSocket = this.options.createSocket || 0;

75 this.backoff = {
76 delta: **this.**options.backoffDelta || 500,
77 max: **this.**options.backoffMax || 5000
78 };

in current version, when options==undefined, library is not

so new wallets can skip the bulk of initial sync time

I frequently get "bcoin.js:53953 Uncaught AssertionError: "undefined" == true" error:

1475061664 Status: tip=00000000e7c58ced1193b9c3af1d8525c019641254ba91fb42ad177e100d29e7 ts=2009-11-24T02:41:47Z height=27907 progress=11.48% blocks=1480 orphans=1 active=0 queue=1 target=486604799 peers=8 pending=19 highest=431920 jobs=20
1475061664 [Error] undefined
1475061664 WARNING: Reorganizing chain.

Is there any support for running this node within a React Native (RN) supported Javascript runtime, on device? A lot of RN developers and businesses would be interested in accepting Bitcoin, and using bcoin alongside RN might help enable that.

Received transaction afd185329738b3ee5969d32fe9297c45abdb97728589f307afec82bc507a3911 from 5.103.136.90.
The test script runs until it inevitably crashes on unhandled peer errors:

events.js:72
throw er; // Unhandled 'error' event
^
Error: Timed out: verack
at _req.entry.ontimeout as _onTimeout
at Timer.listOnTimeout as ontimeout

Instead of crashing on some problem at a peer, would it be possible to use another peer in bcoin?

When creating a wallet, add address events are emitted during the key derivation, but callbacks for them are not registered until the wallet is added to the pool. When I run the example SPV app, it's impossible to get the derived keys to be watched by the pool.
I'm solving it now by passing the pool to Wallet constructor and calling watchWallet() on it as soon as possible.

I have the following code example:

var bcoin = require('bcoin').set('main');

// Create a blockchain and store it in leveldb.
// `db` also accepts `rocksdb` and `lmdb`.
var prefix = process.env.HOME + '/my-bcoin-environment';
var chain = new bcoin.chain({ db: 'leveldb', location: prefix + '/chain' });

var mempool = new bcoin.mempool({ chain: chain });

// Create a network pool of peers with a limit of 8 peers.
var pool = new bcoin.pool({ chain: chain, mempool: mempool, maxPeers: 8 });

// Open the pool (implicitly opens mempool and chain).
pool.open(function(err) {
  if (err)
    throw err;

  // Connect, start retrieving and relaying txs
  pool.connect();

  // Start the blockchain sync.
  pool.startSync();

  // Watch the action
  chain.on('block', function(block) {
    console.log('Connected block to blockchain:');
    console.log(block);
  });

  mempool.on('tx', function(tx) {
    chain.db.getTX(tx.hash, function(err, txinfo) {
      console.log(err, txinfo);
    });
  });

  pool.on('tx', function(tx) {
    console.log('Saw transaction:');
    console.log(tx.rhash);
  });
});

For some reason, I'm getting undefined undefined, what is the proper way of getting transaction info from the local db?

I'm running the example app modified to work with latest changes in master.

var pool = bcoin.pool();
var wallet = new bcoin.wallet({
    priv: privkey
});

pool.addWallet(wallet);

wallet.on('balance', function(balance) {
    var btc = bcoin.utils.toBTC(balance);
    console.log('Your wallet balance has been updated: %s', btc);
});

pool.once('full', finish);
process.once('SIGINT', finish);

pool.startSync();

var once = false;
function finish() {
    if (once)
        return;
    once = true;
    console.log('Balance: %s', wallet.balance());
    console.log('Finished syncing');
}

I expect it to update the balance based on confirmed transactions in the block chain. The transactions are picked up from the chain but it doesn't look like tx-pool does anything with them. I'm logging all events and can see pool block event with the relevant block, tx and watched events with the transaction itself.

I grepped for addTx(), and it turns out Wallet.addTx() is used only in miner.js and wallet-test.js. Any hint on how to make the transaction pool work?

Running version 0.15.0 with node v4.6.0 (EDIT: appears to be resolved on v6.7.0), copy pasted the P2P example from here: https://github.com/bcoin-org/bcoin#connecting-to-the-p2p-network

Getting this error message:

user@laptop ~/dev/btc $ node index.js 
/home/user/dev/btc/index.js:1
(function (exports, require, module, __filename, __dirname) { var bcoin = require('bcoin').set('main');
                                                                                           ^

TypeError: require(...).set is not a function
    at Object.<anonymous> (/home/orw/dev/btc/index.js:1:92)
    at Module._compile (module.js:409:26)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Function.Module.runMain (module.js:441:10)
    at startup (node.js:139:18)
    at node.js:974:3

From the README

Bitcoin deals with really big numbers on a regular basis. Javascript Numbers lose precision after 53 bits. It is absolutely necessary to use big numbers when dealing with satoshi values.

I'm sure there's something I'm missing, but it would seem 53 bits is more than enough to accurately represent all possible values?

> Math.pow(2,53)/(21e6 * 1e8)
4.289142502257615

Hi,
I'm trying to change default CnainDB file path, but bcoin.chaindb(chain, options) is undefined.

When i run this code

'use strict';

var bcoin = require('bcoin');

const testnet = true;

const MAIN_WALLET_PATH = process.env.HOME + '/workspace/db/main-wallet.json';
const TEST_WALLET_PATH = process.env.HOME + '/workspace/db/test-wallet.json';

var chain;
if (testnet)
  bcoin.chaindb(chain, {file: 'process.env.HOME' + '/workspace/db/main-chain'});
else
  bcoin.chaindb(chain, {file: 'process.env.HOME' + '/workspace/db/test-chain'});

then i got error:

TypeError: bcoin.chaindb is not a function

maybe you need to transfer the code

708 /**
709  * ChainDB
710 */
711
712 var BLOCK_SIZE = 112;
713 // var BLOCK_SIZE = 116;
714
715 function ChainDB(chain, options) {
716  if (!(this instanceof ChainDB))
717    return new ChainDB(chain);
***
***
***

from file chain.js to new file chaindb.js ?

when try to use the library with browserify
var pool = bcoin.pool({
size: 8,
type: 'spv',
network: 'testnet'
});

ERROR

Error: Assertion failed
throw new Error(msg || 'Assertion failed');

When syncing up, after a merkleblock reponse is received from a peer, tx messages follow with interesting transactions. The block is saved in lastBlock var, and there's logic in place to accumulate incoming txs internally and emit a single merkleblock event at the end.

  } else if (cmd === 'tx') {
    payload = bcoin.tx(payload, this.lastBlock);
    if (this.lastBlock) {
      if (payload.block) {
        this.lastBlock.txs.push(payload);
        return;
      } else {
        this._emitMerkle(this.lastBlock);
      }
    }
  }

Unfortunately, a transaction can be added to the lastBlock merkleblock object only when it has a block set on itself. But it will be set on the transaction in the constructor only if lastBlock contains the transaction.

if (block && this.ts === 0) {
    if (block.type === 'merkleblock') {
      if (block.hasTX(this.hash('hex')))
        this.setBlock(block, index);
    } else {
      this.setBlock(block, index);
    }
  }

Eventually, a watched event is not triggered in Pool._handleTX() after the block has been added to the chain.

I have forked bcoin for Uro. so far - update.js produces a valid preload and the addresses being generated are valid too. You can see the changes in the uro branch or github.com/uro-/bcoin

What I have noticed is frequent timeout even when connecting to localhost - I am running the sample code on the front page.

I am getting alot of these messages - any ideas what I might be doing wrong (anything to do with me running Node on WIndows 8.1 over Wi-Fi)?:

events.js:72
throw er; // Unhandled 'error' event
^
Error: Timed out: verack
at _req.entry.ontimeout as _onTimeout
at Timer.listOnTimeout as ontimeout

events.js:72
throw er; // Unhandled 'error' event
^
Error: Timed out: verack
at _req.entry.ontimeout as _onTimeout
at Timer.listOnTimeout as ontimeout

events.js:72
throw er; // Unhandled 'error' event
^
Error: read ECONNRESET
at errnoException (net.js:904:11)
at TCP.onread (net.js:558:19)

I also get the "verack" error sometimes.

The current mempool tx selection for mining only sorts txs in dependency order and adds them to a block template while minding the sigop and weight limits. Prioritizing is necessary if we ever want to have a real mining backend.

To do this, we should keep track of cumulative descendant fees, size, etc. on mempool entries for deciding which dependency chain to include in a block.

Hi,

I noticed that bcoin is pretty much a collection of different modules and made a search on npm. Apparently @czzarr has split up bcoin into separate modules on npm. I'm assuming for adding support for different kind of blockchains/networks.

 $ npm search bcoin
NAME            DESCRIPTION                                              AUTHOR         DATE       VERSION KEYWORDS                                  
bcoin           Bitcoin bike-shed                                        =indutny =chjj 2014-06-17 0.14.2  bitcoin bcoin                             
bcoin-block     Bitcoin block object                                     =czzarr        2014-11-05 1.0.0   bcoin bitcoin block                       
bcoin-protocol  Bitcoin protocol, parser and framer                      =czzarr        2014-10-30 1.2.0   bcoin bitcoin parser framer               
bcoin-script    Bitcoin Script encoder, decoder and runner               =czzarr        2014-10-28 1.0.0   bcoin bitcoin script                      
bcoin-tx        Bitcoin transaction object                               =czzarr        2014-11-04 1.2.0   bcoin bitcoin transaction tx              
bcoin-utils     Utility functions for bcoin                              =czzarr        2014-11-04 1.1.0   bcoin bitcoin utils base58 varint         

What do you guys think about putting all these together into bcoin? It would be a shame if there are bug fixes made on separate modules and not being able to share the work.

/cc @chjj @ryanxcharles

A few points to allow for an HD lru cache when unencrypted:

• An HD lru cache is possible if we're careful. Only use one if we're not encrypted. Clear and zero when we encrypt. This is a big one for me: an HD lru cache increases perf when deriving addresses by a lot.
• A WalletKey contains a reference to the HD private key and/or public key Buffer.
• A sign lock is needed. Keys are not to be zeroed elsewhere if we are in the process of signing. The other possibility is to always copy the privateKey buffer when instantiating a WalletKey, lock may still be necessary to prevent other race conditions.
• Public keys need to be copied to a new buffer. We want to keep public keys in memory since they're essential in order for the walletkey's address-like features to work.

Fetching the current version from the master branch and running npm build on node v4.6.0 fails with 2 errors, 1 related to gyp and 1 related to leveldb (looks like you're trying to fetch a certain branch that doesn't exist anymore):

user@laptop ~/dev/btc/node_modules/bcoin $ npm install
npm ERR! git rev-list -n1 leveldb-1.19: fatal: ambiguous argument 'leveldb-1.19': unknown revision or path not in the working tree.
npm ERR! git rev-list -n1 leveldb-1.19: Use '--' to separate paths from revisions, like this:
npm ERR! git rev-list -n1 leveldb-1.19: 'git <command> [<revision>...] -- [<file>...]'
npm ERR! git rev-list -n1 leveldb-1.19: 
npm WARN optional dep failed, continuing leveldown@git://github.com/Level/leveldown.git#leveldb-1.19
/
> [email protected] install /home/user/dev/btc/node_modules/bcoin/node_modules/bcoin-native
> node-gyp rebuild

gyp WARN download NVM_NODEJS_ORG_MIRROR is deprecated and will be removed in node-gyp v4, please use NODEJS_ORG_MIRROR
gyp WARN download NVM_NODEJS_ORG_MIRROR is deprecated and will be removed in node-gyp v4, please use NODEJS_ORG_MIRROR
gyp WARN download NVM_NODEJS_ORG_MIRROR is deprecated and will be removed in node-gyp v4, please use NODEJS_ORG_MIRROR
make: Entering directory `/home/user/dev/btc/node_modules/bcoin/node_modules/bcoin-native/build'
  CC(target) Release/obj.target/bcoin-native/src/poly1305-donna/poly1305-donna.o
  CC(target) Release/obj.target/bcoin-native/src/chacha20-simple/chacha20_simple.o
  CC(target) Release/obj.target/bcoin-native/src/scrypt/insecure_memzero.o
  CC(target) Release/obj.target/bcoin-native/src/scrypt/sha256.o
../src/scrypt/sha256.c:97:40: error: ‘restrict’ undeclared here (not in a function)
 SHA256_Transform(uint32_t state[static restrict 8],
                                        ^
../src/scrypt/sha256.c:97:49: error: expected ‘]’ before numeric constant
 SHA256_Transform(uint32_t state[static restrict 8],
                                                 ^
../src/scrypt/sha256.c:98:5: error: expected ‘;’, ‘,’ or ‘)’ before ‘const’
     const uint8_t block[static restrict 64],
     ^
../src/scrypt/sha256.c:162:52: error: ‘restrict’ undeclared here (not in a function)
 SHA256_Pad(SHA256_CTX * ctx, uint32_t tmp32[static restrict 72])
                                                    ^
../src/scrypt/sha256.c:162:61: error: expected ‘]’ before numeric constant
 SHA256_Pad(SHA256_CTX * ctx, uint32_t tmp32[static restrict 72])
                                                             ^
../src/scrypt/sha256.c:216:27: error: ‘restrict’ undeclared here (not in a function)
     uint32_t tmp32[static restrict 72])
                           ^
../src/scrypt/sha256.c:216:36: error: expected ‘]’ before numeric constant
     uint32_t tmp32[static restrict 72])
                                    ^
../src/scrypt/sha256.c: In function ‘libcperciva_SHA256_Update’:
../src/scrypt/sha256.c:261:2: warning: implicit declaration of function ‘_SHA256_Update’ [-Wimplicit-function-declaration]
  _SHA256_Update(ctx, in, len, tmp32);
  ^
../src/scrypt/sha256.c: At top level:
../src/scrypt/sha256.c:274:27: error: ‘restrict’ undeclared here (not in a function)
     uint32_t tmp32[static restrict 72])
                           ^
../src/scrypt/sha256.c:274:36: error: expected ‘]’ before numeric constant
     uint32_t tmp32[static restrict 72])
                                    ^
../src/scrypt/sha256.c: In function ‘libcperciva_SHA256_Final’:
../src/scrypt/sha256.c:291:2: warning: implicit declaration of function ‘_SHA256_Final’ [-Wimplicit-function-declaration]
  _SHA256_Final(digest, ctx, tmp32);
  ^
../src/scrypt/sha256.c: At top level:
../src/scrypt/sha256.c:326:27: error: ‘restrict’ undeclared here (not in a function)
     uint32_t tmp32[static restrict 72], uint8_t pad[static restrict 64],
                           ^
../src/scrypt/sha256.c:326:36: error: expected ‘]’ before numeric constant
     uint32_t tmp32[static restrict 72], uint8_t pad[static restrict 64],
                                    ^
../src/scrypt/sha256.c:326:41: error: expected ‘;’, ‘,’ or ‘)’ before ‘uint8_t’
     uint32_t tmp32[static restrict 72], uint8_t pad[static restrict 64],
                                         ^
../src/scrypt/sha256.c: In function ‘libcperciva_HMAC_SHA256_Init’:
../src/scrypt/sha256.c:365:2: warning: implicit declaration of function ‘_HMAC_SHA256_Init’ [-Wimplicit-function-declaration]
  _HMAC_SHA256_Init(ctx, _K, Klen, tmp32, pad, khash);
  ^
../src/scrypt/sha256.c: At top level:
../src/scrypt/sha256.c:379:27: error: ‘restrict’ undeclared here (not in a function)
     uint32_t tmp32[static restrict 72])
                           ^
../src/scrypt/sha256.c:379:36: error: expected ‘]’ before numeric constant
     uint32_t tmp32[static restrict 72])
                                    ^
../src/scrypt/sha256.c: In function ‘libcperciva_HMAC_SHA256_Update’:
../src/scrypt/sha256.c:393:2: warning: implicit declaration of function ‘_HMAC_SHA256_Update’ [-Wimplicit-function-declaration]
  _HMAC_SHA256_Update(ctx, in, len, tmp32);
  ^
../src/scrypt/sha256.c: At top level:
../src/scrypt/sha256.c:406:27: error: ‘restrict’ undeclared here (not in a function)
     uint32_t tmp32[static restrict 72], uint8_t ihash[static restrict 32])
                           ^
../src/scrypt/sha256.c:406:36: error: expected ‘]’ before numeric constant
     uint32_t tmp32[static restrict 72], uint8_t ihash[static restrict 32])
                                    ^
../src/scrypt/sha256.c:406:41: error: expected ‘;’, ‘,’ or ‘)’ before ‘uint8_t’
     uint32_t tmp32[static restrict 72], uint8_t ihash[static restrict 32])
                                         ^
../src/scrypt/sha256.c: In function ‘libcperciva_HMAC_SHA256_Final’:
../src/scrypt/sha256.c:427:2: warning: implicit declaration of function ‘_HMAC_SHA256_Final’ [-Wimplicit-function-declaration]
  _HMAC_SHA256_Final(digest, ctx, tmp32, ihash);
  ^
make: *** [Release/obj.target/bcoin-native/src/scrypt/sha256.o] Error 1
make: Leaving directory `/home/user/dev/btc/node_modules/bcoin/node_modules/bcoin-native/build'
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:276:23)
gyp ERR! stack     at emitTwo (events.js:87:13)
gyp ERR! stack     at ChildProcess.emit (events.js:172:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)
gyp ERR! System Linux 4.4.0-24-generic
gyp ERR! command "/usr/bin/nodejs" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/user/dev/btc/node_modules/bcoin/node_modules/bcoin-native
gyp ERR! node -v v4.6.0
gyp ERR! node-gyp -v v3.4.0
gyp ERR! not ok 
npm WARN optional dep failed, continuing [email protected]

I was looking at the code and I noticed that in scriptVector method at mtx.js when it looks for the script public key for the P2PK comparison case it tries with prev.get(1):

MTX.prototype.scriptVector = function scriptVector(prev, vector, ring) {
  var i, n;

  // P2PK
  if (prev.isPubkey()) {
    if (!utils.equal(prev.get(1), ring.publicKey)) //<--- HERE
      return false;

    vector.set(0, opcodes.OP_0);

    return true;
  }

(...)

}

My question is: souldn't it be like the signVector method where it gets the public key with prev.get(0)?

If I change prev.get(1) to prev.get(0) at scriptVector now a coinbase tx imported with fromRaw(str, enc) verifies correctly. Why does it happen? I'm getting my cb tx hex data from generate command in regtest mode.

Say you have 0104 in your buffer (or [1, 4] in bcoin style). It will be encoded and pushed onto the stack as [4].
The problem is that the decode function will always decode it as [0x54].

Am I missing something or is it really a bug? If it's a bug, can we fix it by pushing n instead of [n] for OP_1-OP_16 ?

@chjj

I would like to avoid try/catch but I see that there are several errors that are not handled, and I don't know what to do exactly.

After few seconds the client in the README file (the example) is crashing with one of these errors:

events.js:141
      throw er; // Unhandled 'error' event
      ^
Error: Timed out: verack
  at [object Object].entry.ontimeout [as _onTimeout] (/Users/koalalorenzo/Projects/test/node_modules/bcoin/lib/bcoin/peer.js:246:12)
  at Timer.listOnTimeout (timers.js:92:15)

events.js:141
      throw er; // Unhandled 'error' event
      ^
Error: socket hangup
  at Peer.error [as _error] (/Users/koalalorenzo/Projects/test/node_modules/bcoin/lib/bcoin/peer.js:231:48)
  at Socket.<anonymous> (/Users/koalalorenzo/Projects/test/node_modules/bcoin/lib/bcoin/peer.js:80:10)
  at Socket.g (events.js:260:16)
  at emitOne (events.js:77:13)
  at Socket.emit (events.js:169:7)
  at TCP._onclose (net.js:469:12)

Any suggestion? I have tried to handle the event 'error' on socket but nothing is happening.

SPV Sync completes ok but it's a bit slow and experiences a lot of timeouts, ban thresholds and ECONNREFUSED errors. I didn't try a full sync.

Doubling the timeouts in peer.js and network.js helps and I don't know if ECONNREFUSED is related to the Node issue here: nodejs/node-v0.x-archive#5488

Test code is here:

var pool = new bcoin.pool({ size: 32, debug: true, type: 'spv', fullNode: false, multiplePeers: true, network: 'main', });
pool.startSync();

Why?

• better type system (https://flowtype.org/, https://djcordhose.github.io/flow-vs-typescript/2016_hhjs.html)
• perfect support in https://nuclide.io/docs/languages/flow/#features
• defacto standard for modern JavaScript
• https://github.com/babel/babili

What do you think?

utils.nextTick uses setImmediate in Node environments and then process.nextTick in browser builds.

setImmediate and process.nextTick do not behave the same. In browserified builds, process.nextTick schedules recursive callbacks synchronously and bcoin blocks the application thread on expensive tasks.

It seems it would be more appropriate to use setImmediate or setTimeout, whichever of the two is available.

As I user of copay I would love to use it with bcoin instead of bitcore-node + bitcore-wallet-service. Before I start writing any code I'd like to hear your opinion, have you thought about this before? Do you know about any project implementing a multiplatform graphical wallet that uses bcoin as the backend?

When running the client in the SPV mode:

chain.js:1138
    if (block.getCoinbaseHeight() > self.bestHeight)
              ^

TypeError: block.getCoinbaseHeight is not a function

https://github.com/indutny/bcoin/blob/master/lib/bcoin/chain.js#L1137

block variable contains a MerkleBlock.

The generators branch is a rewrite of bcoin using promises and generators: https://github.com/bcoin-org/bcoin/tree/generators

Unfortunately, promises and generators are slow and use a lot of memory when compared to callbacks (initial sync: 1ms block processing time vs 400 usec block processing time, 160-200mb heap size vs 75mb heap size, high watermark of 110mb heapused vs 50mb heapused).

Once async/await becomes ubiquitous, I see bcoin switching to that, but only if it is well optimized.

If anyone can optimize the generators branch, I'd be willing to switch now.

I've been experiencing a servere lack of tx packets and transactions in general using bcoin.

According to the bitcoind source, tx packets should get sent after a merkleblock (basically, any merkleblock with tx hash in it should be accompanied by the tx packet afterwards). This doesn't seem to work with bcoin. If I were to just do:

pool.on('block', function(block) {
  if (block.hash.length) {
    console.log('Found block/%s', block.hash('hex'));
    console.log('Expecting tx/%s any second now.', block.hash[0]);
  }
});

pool.on('tx', function(tx) {
  console.log('Found tx/%s', tx.hash('hex'));
});

I do not get any tx packets, ever.

I also know peer._handleInv automatically calls getdata on the transactions advertised by the peer (as an aside, it's surprising how few tx's are advertised by peers), but even then, I rarely get transactions without doing something like:

pool.on('block', function(block) {
  if (block.tx.length) {
    block.tx.forEach(function(hash) {
      console.log('Getting transaction: %s', hash);
      pool.getTX(hash, function(tx, range) {});
    });
  }
});

pool.on('tx', function(tx) {
  console.log('Found tx/%s', tx.hash('hex'));
});

To make things even more confusing, the bitcoin spec specifies that everything after the block nonce is a varint called txn_count with an array of full transactions afterwards (not just hashes, and not separate packets?).

Unless I'm reading that wrong, and I must be, becuase the official bitcoin client seems to do different and just sends the hashes like bcoin expects (Currently, bcoin parses merkleblock transaction hashes after the txn_count. No extra tx packets seem to come in afterward either). But it does send full transactions as separate tx packets.

It seems bitcore parses full transactions on a merkleblock instead of just grabbing tx hashes. Not sure how that's possible.

It could be that it's rare that peers have transactions in their mempool to send, but using a test script with bitcore I get a seemingly endless supply of tx events.

Am I missing something here? Out of these 3 models which is correct?

what do you think about using secp256k1-node? It uses native/elliptic depending on the environment

• Ledger: https://github.com/LedgerHQ/ledger-node-js-api
• Trezor: https://github.com/trezor/trezor.js/ and https://github.com/trezor/trezor-link

Thoughts:

Create a kind of "hardware mode" for wallets, similar to watch only. Account creation must always grab an xpubkey from the device and pass it in. Change watchOnly byte to mode (0 = normal, 1 = watch only, 2 = hardware). Add a hwtype byte (0 = ledger, 1 = trezor, etc).

The ledger and trezor bindings are not modules I would want to use. The trezor module uses flow type and es7 modules, and probably other nonsense that requires compilation. Ledger's binding is lacking examples or even a readme. Making sane bindings that expose only the features we need would be ideal.

Ledger communication is specified here: https://ledgerhq.github.io/btchip-doc/bitcoin-technical.html

Hello,
I am trying to develop a simple SPV Explorer and I am facing some issues when I am using the method watch.

For example if I run: coffee main.coffee --debug --transaction {{hash}} (as example) I am not able to see in the output that specific transaction. The code behind is simple calling the Explorer.call_transaction with the following code:

  call_transaction: (tx_hash, callback) ->
    # Use a callback to get from peers a specific transaction
    console.log "Watching TX: #{tx_hash}" if @_debug

    @daemon.node.on 'tx', (_tx, _peer) =>
      hash = bcoin.utils.revHex(_tx.hash('hex')) 

      if hash is tx_hash and !~@_txs_received.indexOf(_tx)
        # Running the callback if it's the first time we see this tx
        console.log "TX Watched #{tx_hash}" if @_debug
        @_txs_received.push _tx
        callback _tx, _peer if callback

    @daemon.node.watch tx_hash

According to what was written in the comment by @chjj in #44, it should work... but instead, even if I watch a transaction that I know is going to be broadcasted to the node, I don't see it in the log output.

What am I doing wrong? can somebody help me?
Thanks

I haven't been able to execute any transaction's scripts yet due to unknown opcodes (script.execute returns false). Just putting this up here on the todo list.

I've never done this on a project of mine, but I just force pushed master to undo an accidental merge of a totally unused wallet branch. Master was in a bad state and was nearly impossible to resolve any conflicts with the soon-to-be merged wallet branch. The bottom line is it was pissing me off, so I fixed it.

Note that the forced commits are signed with my gpg key.

Synchonization of full node @ http://browser.bcoin.io/?network=main&prune=true is stuck on ~24% with:

1471194308 Rejecting block 000000000003ad1e0186eca4049f5fc7c8ff2e409e268da5bfdd995d388c3426 (seed.bitcoin.sipa.be:8333): ccode=invalid reason=bad-prevblk.
1471194308 Sending reject packet to peer (seed.bitcoin.sipa.be:8333).
1471194308 Handled orphan 00000000000af2fe8f68c507bde8399fd777f1778d2b08be4a78ca9edefce229.