Currently after shutdown node will continue to run and log after the process appears to be shutdown, and in some cases ^C will not cleanly shutdown and the process needs to be closed manually.

Replace leveldb.

sudo node index.js

/bitcore-node/node_modules/levelup/lib/util.js:59
throw new LevelUPError(
^
LevelUPError: Installed version of LevelDOWN (1.0.2) does not match required version (~0.10.0)
at getLevelDOWN (/bitcore-node/node_modules/levelup/lib/util.js:59:11)
at LevelUP.open (/bitcore-node/node_modules/levelup/lib/levelup.js:113:37)
at new LevelUP (/bitcore-node/node_modules/levelup/lib/levelup.js:86:8)
at Function.BitcoreNode.create (/bitcore-node/lib/node.js:52:5)
at Object. (/bitcore-node/index.js:15:26)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Function.Module.runMain (module.js:497:10)
at startup (node.js:119:16)
at node.js:902:3

Include a command line utility for interacting with a bitcore-node with the following abilities:

Step 1: Install

npm install -g bitcore-node

Install bitcore-node using the global option to add bitcore-node to your path.

Step 2: Configure

bitcore-node create mynode "My Node"
cd mynode
bitcore-node add wallet-service
bitcore-node add insight-api

This will create a new directory and add services to your node via configuration settings similar to:

mynode
├── bitcore-node.json
├── data
│  ├── bitcoin.conf
│  ├── bitcore-node.db
│  │   ├── 163997.ldb
│  │   └── ...
│
├── node_modules
│   ├── bitcore
│   ├── bitcore-node
│   ├── bitcore-wallet-service
│   └── insight-api
└── package.json

With a bitcore-node.json file similar to:

{
  "name": "My Node",
    "modules": [
      "bitcore-wallet-service"
      "insight-api"
   ],
   "datadir": "./data",
   "network": "livenet"
}

And a package.json in the format:

{
  "dependencies": {
    "bitcore": "^0.12.15",
    "bitcore-node": "^1.0.0",
    "bitcore-wallet-service": "^3.0.0",
    "bitcore-insight-api": "^4.0.0"
  }
}

Step 3: Start

$ bitcore-node start

The bitcore-node start command will then dynamically create a node based on the configuration options in bitcore-node.json, load all of the necessary node modules, and configure the node with the enabled services.

calling getTransaction inside daemon.on('tx', function(txid) will always fail.

this is because the said transaction is yet to enter the mempool. unsure if this is working as intended or a bug.

It should increase block handling speeds by about 5 times.

The daemon ready event is sometimes being called too soon during regtesting, as the RPC interface may not be ready, as well as the wallet.

During syncing, after a while, I am getting:

[Error: Bitcoin Core JSON-RPC: host=127.0.0.1 port=8332: Request Error: connect EADDRNOTAVAIL]
network monitor disconnected

and syncing is being stopped, indeed. How can I convince bitcore-node to restore the connection without shutting down and starting it again?

After several attempts with syncing, there have been repeated process out of memory errors, even with 16GiB. And has been reported in other issues such as #72

00000000d1cf0efbccc63c16cf07e68aa8cbc939ec3c267b408fbb5a50734340 208123 vel 0.2 b/s 46.67219823961429% synced
00000000ad640842188e7a6d858495af779a050aa90b90863c67d98aefa5ac63 208124 vel 0.2 b/s 46.67242249257162% synced
00000000a5a0921483964a48e09266c3755074879f3933faccc3a7ae6387f0dd 208125 vel 0.2 b/s 46.67264674552896% synced
00000000156bd9c2dae657324d47f8bdab490838973fdb6d8ea6fc386dc4cdab 208126 vel 0.2 b/s 46.672870998486296% synced
00000000cdf70b4c804c0a9bcf1189647c909051c86c4dc64cace9e7403a9c52 208127 vel 0.2 b/s 46.67309525144363% synced
FATAL ERROR: CALL_AND_RETRY_2 Allocation failed - process out of memory
Aborted (core dumped)

000000000000040922c830bae82aac73edfc670b65016f87f852181e909ad1fc 212784 vel 0 b/s 60.0093066542578% synced
Error: { [Error: write EPIPE] code: 'EPIPE', errno: 'EPIPE', syscall: 'write' }
network monitor disconnected

/bitcore-node/lib/networkmonitor.js:110
throw reason;
^
Error: write EPIPE
at errnoException (net.js:901:11)
at Object.afterWrite (net.js:718:19)
npm ERR! weird error 8
npm WARN This failure might be due to the use of legacy binary "node"
npm WARN For further explanations, please read
/usr/share/doc/nodejs/README.Debian

Restarting the server, bitcoind, and running npm start again yields:

restarting Tip is 000000000000040922c830bae82aac73edfc670b65016f87f852181e909ad1fc
No blockchain yet
No blockchain yet
No blockchain yet
...
All headers fetched, total = 212785
...

--just appears to sit there. Any logs we can check?

Querying for a list of blocks, or a list of transactions in a block does not provide the ids, or hashes, of the entities returned. Maybe I'm missing something, but this makes it awfully difficult to be usable...

/v1/blocks returns the following, which contains no block hashes, or txids:

[
  {
    "header": {
      "version": 1,
      "prevHash": "0000000000000000000000000000000000000000000000000000000000000000",
      "merkleRoot": "4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b",
      "time": 1231006505,
      "bits": 486604799,
      "nonce": 2083236893
    },
    "transactions": [
      {
        "version": 1,
        "inputs": [
          {
            "prevTxId": "0000000000000000000000000000000000000000000000000000000000000000",
            "outputIndex": 4294967295,
            "sequenceNumber": 4294967295,
            "script": "04ffff001d0104455468652054696d65732030332f4a616e2f32303039204368616e63656c6c6f72206f6e206272696e6b206f66207365636f6e64206261696c6f757420666f722062616e6b73"
          }
        ],
        "outputs": [
          {
            "satoshis": 5000000000,
            "script": "4104678afdb0fe5548271967f1a67130b7105cd6a828e03909a67962e0ea1f61deb649f6bc3f4cef38c4f35504e51ec112de5c384df7ba0b8d578a4c702b6bf11d5fac"
          }
        ],
        "nLockTime": 0
      }
    ]

Setup repo following Bitcore standards

Hi,
Right now, the sync process is quiet slow. Reason for this is Node being a single threaded technology, is unable to do utilize the processing power available on machine(in which bitcore-node is running) to the extent it should be utilizing.

Right now, its become difficult to run sync for days(especially for start-ups) & going farward, after an year or two, it will be impossible to do it.

Need to increase the sync speed is obvious(hope you are of the same perspective).

We can achieve this by using node cluster in the following manner :-

The entire bitcore-node should be split into 2 separate node processes.

a. First process reads the data from bitcoin-qt and, passes the result to second process. This will be a single threaded process. No use of cluster needed.

This thread can also be used to store the data to bitcore-node DB and go for next block.

b. Second node process which uses node-cluster and, does all the processing intensive work. This process should by default use "Number of processors available -2".

This passes the entire block along with transactions to the first process which stores the block to bitcore-node DB and, goes to fetch the next block.

By doing this, we won't have the problem of database lock as, single thread will be using the level-db operations(1st process) and, processor intensive work would be delegated to another node-process using cluster. This definitely will increase the sync process.

Hope you find the suggestion helpful.

How does this work with Litecoin or Dogecoin? What needs to be changed?

This doc (https://docs.google.com/document/d/1rXQdfr8VDBSzheEn0KynnCKCMB0oAsMunkjdG4aY-18/edit#) says that the endpoint:

/v1/transaction/:hash/outputs/:index

should return an output_full with attached:

transaction_id
index
satoshis
script
spent
spending_transactions
type: ‘p2sh’, ‘p2pkh’, etc.

Unfortunately I only see
satoshis
script

namely an output_summary.

Is it because I'm not fully synchronized? I don't think this is the issue because I'm asking for the outputs of a really old transaction and I'm halfway the full synchronization.

Thanks

Copying this issue over from bitcoind.js:

i tried setting it up on my Mac but had some issues, and I don't want to make this a forum for debugging my build. Could you provide a Dockerfile? Thanks!

bitpay/bitcoind.js#59

Tip is 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
Connection to bitcoind failed
network monitor disconnected

Error: connect ECONNREFUSED
at errnoException (net.js:901:11)
at Object.afterConnect as oncomplete

bitcoind works perfect

E.g. Storage and mailbox plugins

testusers-MacBook-Pro:bitcore-node testuser$ lldb -- node example/node.js 
(lldb) target create "node"
Current executable set to 'node' (x86_64).
(lldb) settings set -- target.run-args  "example/node.js"
(lldb) run
Process 63964 launched: '/Users/testuser/.nvm/versions/node/v0.12.7/bin/node' (x86_64)
Process 63964 stopped
* thread #7: tid = 0x1ad2a, 0x00007fff8c53d286 libsystem_kernel.dylib`__pthread_kill + 10, name = 'bitcoin-msghand', stop reason = signal SIGABRT
    frame #0: 0x00007fff8c53d286 libsystem_kernel.dylib`__pthread_kill + 10
libsystem_kernel.dylib`__pthread_kill:
->  0x7fff8c53d286 <+10>: jae    0x7fff8c53d290            ; <+20>
    0x7fff8c53d288 <+12>: movq   %rax, %rdi
    0x7fff8c53d28b <+15>: jmp    0x7fff8c538c53            ; cerror_nocancel
    0x7fff8c53d290 <+20>: retq   

It's likely an issue with process_messages bindings.

• installation
• configuration
  etc

So the block API results use Bitcore Block toJSON formatting.

When making a request to the /addresses or /addresses/utxos endpoint - results are cached for a considerable amount of time, making them all but useless for a real-time application. The ?noCache flag from Insight is no longer present, has this been changed, or is there a way to disable caching of these endpoints in general?

A collection of other bitcoin related developers and I have been iterating on a common-blockchain API standard in terms of available end points for block explorers.
It is a standard to provide complete informational representations of various objects on the blockchain, without compromise, and is easily testable.

The aim of this is to provide an API that could be easily hot-swapped if a service goes down or is overloaded.

Naturally this API is open to change, and we're very open to feedback as to the end points that are currently provided. It'd be great if you would be open to iterating on this with us such that 'swapping' out block explorers for web services is dead-simple in the future.

Below is a list of common-blockchain repositories, many of them are simple wrappers, but they each have several pain points due to missing information or missing end points, not provided by the block explorers themselves.

https://github.com/weilu/cb-blockr
https://github.com/dcousens/cb-node - a pure API implementation, the ideal case
https://github.com/dcousens/cb-chain
https://github.com/dcousens/cb-helloblock
https://github.com/jprichardson/cb-insight

and make changes to handle more than 1 connection (consensus)

Hi,
I have setup a bitcoin node, let it fully sync, install bitcore-node and try to sync it as well.
After running for a week it stalls at block height 265458 then it gives following error in error log.

https://gist.github.com/yangit/caab3297873ff17b791c

I did that twice and had same error at same block height.
Any ideas?

Current bitcore-node requires a running instance of Bitcoin Core (bitcoind). Remove this dependency and include it inside bitcore-node codebase by using https://github.com/bitpay/bitcoind.js/

Include a means to be able to select and add services and extend functionality without the necessity for modules to be a part of bitcore-node repository.

clean vm, fresh install...

npm start

[email protected] start /data/bitcore-node
node index.js

module.js:340
throw err;
^
Error: Cannot find module 'eventemitter2'
at Function.Module._resolveFilename (module.js:338:15)
at Function.Module._load (module.js:280:25)
at Module.require (module.js:364:17)
at require (module.js:380:17)
at Object. (/data/bitcore-node/lib/node.js:4:20)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.require (module.js:364:17)
npm ERR! weird error 8
npm WARN This failure might be due to the use of legacy binary "node"
npm WARN For further explanations, please read
/usr/share/doc/nodejs/README.Debian

npm ERR! not ok code 0

Integrate with https://github.com/bitpay/chainlib-bitcoin that is using https://github.com/bitpay/node-libbitcoinconsensus

Copying over from the bitcore issue:

bitcore should

• have a pull tester robot, for each new commit, just like Bitcoin Core
• pass all the tests that Bitcoin Core has in the pulltester

bitpay/bitcore#255

Hello,

I'm running bitcore-node on a quite small system and have noticed that it tends to grow quite a bit. My current workaround (after the latest cached headers addition) is to restart the process from time to time..

At start right now it goes to ~16% of system memory, and after 20 minutes or so it has grown to 20-22% of total system memory. I think this is a result of caching quite a bit of data, and i think it would be a good thing to clean the cache from time to time.
Performance-wise i cannot really see any difference in blocks/s before and after the restart so i think this would be safe.

Look into the possibility of using (or others):

• https://www.npmjs.com/package/node-pre-gyp
• https://gitian.org/

All config parameters seem to be read from config/default.yml. Nothing that is read by config.js is injected.

See below. Logging was added.

console.log(bitcoindConf);

$ node config.js
{ protocol: 'http',
  user: 'bituser',
  pass: 'testpass',
  host: '127.0.0.1',
  port: '18333',
  p2pPort: '18332',
  p2pHost: '127.0.0.1',
  dataDir: '/var/lib/bitcoindtestnet3',
  disableAgent: true }

$ node index.js
{ BitcoreNode: { NetworkMonitor: { network: 'livenet', host: 'localhost', port: 8333 } } }
Connection to bitcoind failed

Hi, I have installed bitcore-node successfully on Ubuntu 14.04 server using instructions from README.md

But when I start with "npm start" i got:

> [email protected] start /home/btc/apps/bitcore-node
> node bin/start.js

/home/btc/apps/bitcore-node/node_modules/bindings/bindings.js:83
        throw e
              ^
Error: /home/btc/apps/bitcore-node/build/Release/bitcoind.node: undefined symbol: bitcoinconsensus_verify_script
    at Error (native)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at bindings (/home/btc/apps/bitcore-node/node_modules/bindings/bindings.js:76:44)
    at Object.<anonymous> (/home/btc/apps/bitcore-node/lib/daemon.js:4:35)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)

npm ERR! Linux 3.16.0-45-generic
npm ERR! argv "/usr/bin/node" "/usr/bin/npm" "start"
npm ERR! node v0.12.7
npm ERR! npm  v2.11.3
npm ERR! code ELIFECYCLE
npm ERR! [email protected] start: `node bin/start.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] start script 'node bin/start.js'.
npm ERR! This is most likely a problem with the bitcore-node package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node bin/start.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls bitcore-node
npm ERR! There is likely additional logging output above.

Fresh install, default.yaml configured but getting:

/bitcore-node/node_modules/config/lib/config.js:177
throw new Error('Configuration property "' + property + '" is not defined'
^
Error: Configuration property "BitcoreHTTP.BitcoreNode" is not defined
at Config.get (/bitcore-node/node_modules/config/lib/config.js:177:11)
at Object. (/bitcore-node/index.js:13:24)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Function.Module.runMain (module.js:497:10)
at startup (node.js:119:16)
at node.js:902:3

Attempting to sync, it get's to about 60% and crashes. Attempting to restart eventually yields:

All headers fetched, total = 230926
starting network monitor
No blocks yet
network monitor disconnected
network monitor disconnected
Unhandled rejection TypeError: Cannot read property 'length' of undefined
at Output.toBufferWriter (/data/bitcore-node/node_modules/bitcore/lib/transaction/output.js:147:31)
at /data/bitcore-node/node_modules/bitcore/lib/transaction/transaction.js:272:12
at Function.forEach (/data/bitcore-node/node_modules/bitcore/node_modules/lodash/dist/lodash.js:3297:15)
at Transaction.toBufferWriter (/data/bitcore-node/node_modules/bitcore/lib/transaction/transaction.js:271:5)
at Transaction.toBuffer (/data/bitcore-node/node_modules/bitcore/lib/transaction/transaction.js:261:15)
at Transaction.getHash (/data/bitcore-node/node_modules/bitcore/lib/transaction/transaction.js:130:33)
at Transaction.hashProperty.get (/data/bitcore-node/node_modules/bitcore/lib/transaction/transaction.js:105:34)
at Object..extend.getBlockForTransaction (/data/bitcore-node/lib/services/transaction.js:73:45)
at TransactionService._confirmTransaction (/data/bitcore-node/lib/services/transaction.js:177:16)
at /data/bitcore-node/lib/services/block.js:298:40
at Array.map (native)
at /data/bitcore-node/lib/services/block.js:297:45
From previous event:
at BlockService.confirm (/data/bitcore-node/lib/services/block.js:285:21)
at /data/bitcore-node/lib/node.js:118:36
at Array.map (native)
at Promise.each.then.then.deleteHeight (/data/bitcore-node/lib/node.js:117:56)
From previous event:
at /data/bitcore-node/lib/node.js:116:8
at /data/bitcore-node/lib/eventbus.js:26:14
at Array.map (native)
at processEvent (/data/bitcore-node/lib/eventbus.js:25:52)
at /data/bitcore-node/lib/eventbus.js:50:14
at process._tickCallback (node.js:448:13)
From previous event:
at EventBus.process (/data/bitcore-node/lib/eventbus.js:48:6)
at Peer. (/data/bitcore-node/lib/networkmonitor.js:59:14)
at Peer.emit (events.js:95:17)
at Peer._readMessage (/data/bitcore-node/node_modules/bitcore-p2p/lib/peer.js:221:10)
at Socket. (/data/bitcore-node/node_modules/bitcore-p2p/lib/peer.js:167:10)
at Socket.emit (events.js:95:17)
at Socket. (stream_readable.js:765:14)
at Socket.emit (events.js:92:17)
at emitReadable (_stream_readable.js:427:10)
at emitReadable (_stream_readable.js:423:5)
at readableAddChunk (_stream_readable.js:166:9)
at Socket.Readable.push (_stream_readable.js:128:10)
at TCP.onread (net.js:529:21)

The directory /utils and the file sync.js (referenced in the README file) are missing.
As far as I understand this is a new version of insight-api and I was wondering if I could you that version to delete my db and build it over.

Thanks