znort987 / blockparser Goto Github PK

I removed blockparser completely, reinstalled the git and still same issue. All requirements are installed as well not sure what to do I've had this working before.

Hi,

I want to know how we may use the blockparser to scan other Alt-SHA-256-coins. I mean what part of the code we have to tweak.

Thank.

Make is failing on:

AWS Ubuntu 10.04
gcc version 4.9.1 (Ubuntu 4.9.1-16ubuntu6)

with message:

lnk -- parser
lto-wrapper: g++ returned 1 exit status
/usr/bin/ld: lto-wrapper failed
collect2: error: ld returned 1 exit status
Makefile:307: recipe for target 'parser' failed
make: *** [parser] Error 1

researche says:
The current implementation of LTO makes no attempt to generate
bytecode that is portable between different types of hosts. The
bytecode files are versioned and there is a strict version check,
so bytecode files generated in one version of GCC will not work
with an older/newer version of GCC.

What version of GCC did you generate the the source file?
Thanks

Hi - any thoughts on why i'm receiving this message or what i should look into to resolve?

ubuntu@xx:~/blockparser$ ./parser show

info: starting command "dumpTX"
warning: no TX hashes specified, using the infamous 10K pizza TX
Segmentation fault (core dumped)

Is the issue memory? I am running this on a mico EC2 instance and looking at your documentation the ram is probably way too low.

I installed the blockparser but I cannot run any of the commands. I get this error message:

user@ubuntu:$ cd blockparser
user@ubuntu:/blockparser$ ./parser all

info: mem at start = 0.000 Gigs
info: starting command "allBalances"
fatal: problem accessing directory /home/user/.bitcoin
Aborted (core dumped)

I presume it doesnt find the blockchain files. I saved "blk00000.dat" and so on in this directory:
/home/user/user/Blockchain/blocks

I copied this blockchain from my windows 10 OS into the VMware Ubuntu.

I dont know why the filepath shows "user/user", I access the file actually by clicking only one such folder.
The blockparser readme says "You'll need a block chain somewhere on your hard drive." (but not where exactly).

I also tried to install the blockchain as recommended in the blockparser readme, but came only as far as this:

user@ubuntu:$ git clone https://github.com/bitcoin/bitcoin
user@ubuntu:$ cd bitcoin
user@ubuntu:~/bitcoin$ ./Makefile
bash: ./Makefile: No such file or directory

When I open the /bitcoin folder I can see the file Makefile.am . I dont know why it doesnt work.

I have a vultr instance: Ubuntu 14.10 x64, 8 CPU (some kind of xeon 3.2-3.4ghz), 16GB RAM, 300GB SSD and 10GB swap. I've build blockparser and run: ./parser balance > all.txt
When the blockchain was without index (txindex=0 in bitcoin.conf), it worked very slow.
Now I set txindex=1 and run ./bitcoind -reindex. ~230k blocks have been reindexed so far and I've run parser again - it took 830 secs to parse 50 MegaAddr (there are ~88kk) - it's quite faster but the speed is falling.

Do I need to build full index? If yes, pls note that in your main manual.

UPD. Looks like I need more ram - 16gb is completely used.

when i make in the
"Distributor ID: Ubuntu
Description: Ubuntu 14.04.2 LTS
Release: 14.04
Codename: trusty"
with this error make: *** No targets specified and no makefile found. Stop.

• -please help ~~ thanku

I was trying to modify blockparser to work with an X11 coin (Uro) and when I try to run a command I get "warning: at depth 0 in chain, failed to locate parent block " (many of them, followed by a segmentation fault). I noticed that somebody else on a different X11 coin (Truck) was also having these problems.

Does blockparser not work with X11 blockchains?

i've dumped all addresses but after the address number 500'ish all addreses are XXXXXXX..

did i miss something ?

i used ./parser allBalances >allBalances.txt

I get stuck at -- /usr/bin/ld: cannot find -lcrypto during the make

What is going on, and how can I get past this?

I tried to do gcc -lcrypto
and the result was file not found. I then tried to install random crypto libraries, but no luck and I may have to uninstall them, because now I do gcc -lcrypto and the return is:
In function _start': /build/buildd/glibc-2.21/csu/../sysdeps/i386/start.S:111: undefined reference tomain'

Hi there,

I've found the same issue described at the bottom of this past one

#46

I had too many problems with the required dependencies in my physical machine, hence I decided to start your installation in a fresh install, where I haven't found any problem with dependencies at all.

I was trying your installation instructions when I got stuck at the same point (symbolic link to my host already made)

user@vm:~/blockparser$ ./parser transactions 06f1b66fa14429389cbffa656966993eab656f37

info: mem at start = 0.000 Gigs
info: starting command "transactions"
info: loading block chain from directory: /home/user/.bitcoin
info: block chain size = 0.000 Gigs
info: initializing hash tables
info: estimated number of blocks = 0.00K
info: estimated number of transactions = 0.00M
info: done initializing hash tables - mem = 0.000 Gigs
info: pass 1 -- walk all blocks and build headers ...
warning: found no blocks - giving up                                                       
user@vm:~/blockparser$ cat /etc/issue
Ubuntu 16.04 LTS \n \l

user@vm:~/blockparser$ 

BTW, I tipped you a few days ago from 1Eva

Cheers.

I cannot run blockparser due to this Error:

user@ubuntu:~/blockparser$ ./parser all
info: mem at start = 0.000 Gigs
info: starting command "allBalances"
info: loading block chain from directory: /home/user/.bitcoin
info: block chain size = 72.788 Gigs
info: initializing hash tables
Segmentation fault (core dumped)

Does anyone have a solution?

sudo apt-get install sparsehash needs to be added, otherwise make falls with
coudn't find google/dense_hash_map

Hi,

Thanks for the great work, i hope you can assist. After perl make i get the following:

util.cpp: In function ‘bool addrToHash160(uint8_t*, const uint8_t*, bool, bool)’:
util.cpp:606:24: error: ‘BN_CTX_init’ was not declared in this scope
BN_CTX_init(ctx);
^
util.cpp: In function ‘void hash160ToAddr(uint8_t*, const uint8_t*, bool, uint8_t)’:
util.cpp:729:24: error: ‘BN_CTX_init’ was not declared in this scope
BN_CTX_init(ctx);
^
c++ -- .r... -- option.cpp
c++ -- .r... -- rmd160.cpp
c++ -- .r... -- sha256.cpp
c++ -- .r... -- opcodes.cpp
c++ -- .r... -- callback.cpp
c++ -- .r... -- t/sha256_t.cpp
At global scope:
cc1plus: warning: unrecognized command line option ‘-Wno-unused-private-field’
c++ -- .r... -- t/secp256k1_t.cpp
/tmp/GmlKr7pZ:387: recipe for target '.objs/d5cd7ce3e58ff7a8ebc2786c817cb6dcd1a22603.o' failed
make: *** [.objs/d5cd7ce3e58ff7a8ebc2786c817cb6dcd1a22603.o] Error 1
c++ -- .r... -- t/ripemd-160_t.cpp
c++ -- .r... -- t/base58_t.cpp
c++ -- .r... -- tst.cpp
c++ -- .r... -- test.cpp
make: Target 'all' not remade because of errors.

I would be very great full if you can assist.

Hi, calculating allBalances takes a lot of memory and time. Is there any way of caching previous calculations up to some block, then start just from there next times?.
It would be nice to have an option to --cache and --use-cache or something similar.

Hi,

The normal -DBITCOIN compilation is OK.

But, I'm not able to compile blockparser for alt-coins (i.e. Peercoin or Litecoin). I have the following errors when I build the utility after commenting -DBITCOIN and uncommenting -DPEERCOIN or -DLITECOIN from the Makefile:

~/blockparser> make
c++ -- callback.cpp
c++ -- cb/allBalances.cpp
c++ -- cb/closure.cpp
c++ -- cb/dumpTX.cpp
c++ -- cb/help.cpp
c++ -- cb/pristine.cpp
c++ -- cb/rewards.cpp
c++ -- cb/simpleStats.cpp
c++ -- cb/sql.cpp
c++ -- cb/taint.cpp
c++ -- cb/transactions.cpp
cc -- h9/blake.c
cc -- h9/bmw.c
cc -- h9/cubehash.c
cc -- h9/echo.c
cc -- h9/groestl.c
cc -- h9/jh.c
cc -- h9/keccak.c
cc -- h9/luffa.c
cc -- h9/shavite.c
In file included from cb/dumpTX.cpp:4:0:
./util.h:289:5: error: expected identifier before ‘)’ token
In file included from cb/taint.cpp:25:0:
./util.h:289:5: error: expected identifier before ‘)’ token
In file included from cb/allBalances.cpp:4:0:
./util.h:289:5: error: expected identifier before ‘)’ token
In file included from cb/sql.cpp:4:0:
./util.h:289:5: error: expected identifier before ‘)’ token
In file included from cb/pristine.cpp:4:0:
./util.h:289:5: error: expected identifier before ‘)’ token
In file included from cb/transactions.cpp:5:0:
./util.h:289:5: error: expected identifier before ‘)’ token
In file included from cb/simpleStats.cpp:4:0:
./util.h:289:5: error: expected identifier before ‘)’ token
In file included from callback.cpp:3:0:
./util.h:289:5: error: expected identifier before ‘)’ token
In file included from cb/rewards.cpp:4:0:
./util.h:289:5: error: expected identifier before ‘)’ token
cc -- h9/simd.c
cb/dumpTX.cpp: In static member function ‘static void DumpTX::showScriptInfo(const uint8_t_, uint64_t)’:
cb/dumpTX.cpp:254:46: error: too few arguments to function ‘void hash160ToAddr(uint8_t_, const uint8_t_, int)’
In file included from cb/dumpTX.cpp:4:0:
./util.h:262:10: note: declared here
cb/allBalances.cpp: In member function ‘virtual void AllBalances::wrapup()’:
cb/allBalances.cpp:331:48: error: too few arguments to function ‘void hash160ToAddr(uint8_t_, const uint8_t_, int)’
In file included from cb/allBalances.cpp:4:0:
./util.h:262:10: note: declared here
cb/sql.cpp: In member function ‘virtual void SQLDump::endOutput(const uint8_t_, uint64_t, const uint8_t_, uint64_t, const uint8_t_, uint64_t)’:
cb/sql.cpp:248:64: error: too few arguments to function ‘void hash160ToAddr(uint8_t_, const uint8_t_, int)’
In file included from cb/sql.cpp:4:0:
./util.h:262:10: note: declared here
make: *** [.objs/pristine.o] Erreur 1
make: *** Attente des tâches non terminées....
make: *** [.objs/simpleStats.o] Erreur 1
make: *** [.objs/rewards.o] Erreur 1
make: *** [.objs/callback.o] Erreur 1
make: *** [.objs/dumpTX.o] Erreur 1
make: *** [.objs/transactions.o] Erreur 1
make: *** [.objs/sql.o] Erreur 1
make: *** [.objs/allBalances.o] Erreur 1
make: *** [.objs/taint.o] Erreur 1
In file included from cb/closure.cpp:4:0:
./util.h:289:5: error: expected identifier before ‘)’ token
cb/closure.cpp: In member function ‘virtual void Closure::wrapup()’:
cb/closure.cpp:139:39: error: too few arguments to function ‘void hash160ToAddr(uint8_t_, const uint8_t_, int)’
In file included from cb/closure.cpp:4:0:
./util.h:262:10: note: declared here
make: *** [.objs/closure.o] Erreur 1

Kind regards,
Thi.

When I do a ./make on my Ubuntu 14.04 it fails with the following error:

$ ./make
c++ -- .r... -- parser.cpp
c++ -- .r... -- cb/taint.cpp
c++ -- .r... -- cb/sql.cpp
c++ -- .r... -- cb/dumpTX.cpp
c++ -- .r... -- cb/closure.cpp
c++ -- .r... -- cb/headers.cpp
c++ -- .r... -- cb/rawdump.cpp
c++ -- .r... -- cb/rewards.cpp
c++ -- .r... -- cb/pristine.cpp
In file included from cb/dumpTX.cpp:4:0:
./util.h:215:45: fatal error: google/dense_hash_map: File or directory not found.
             #include <google/dense_hash_map>

And a lot more errors regarding dense hash map.

I tried to get its source from https://github.com/sparsehash/sparsehash and include the missing files, but at the end it still misses a sparseconfig.h I can't find.

Any hints?

If blockchain storage on btrfs filesystem - very low perfomance (allBalance work 10 hour - 55%). After - io load 100% (44-60 mb/sec), cpu load - 0 %. Btrfs not support swap-files, blockparse use mmap, and that similar technologies. Is not bug blockparser, but it is necessary to describe it in the readme. Sorry for bad english.

i am keep getting this error "problem accessing directory /home/insight/.bitcoin"

How change path for parser from Bitcoin to Dogecoin? Like this fatal: problem accessing directory /home/r/.bitcoin to home/r/.dogecoin
Ubuntu 17.04

It would be awesome to have Peercoin support, do you think it's doable?

The "." is not at its right place and has a factor of 100.

i.e.:

18725.81870000 336f7b5cc27b04f46845e6b350a6921756cb352e PDH9AeruMUGh2JzYYTpaNtjLAcfGV5LEto

Should be 1872581.870000 PPC and not 18725.81870000 PPC

https://bkchain.org/ppc/address/PDH9AeruMUGh2JzYYTpaNtjLAcfGV5LEto

Hi, I just installed your soft with great interested but even if everything went smoothly during installation I wasn't able to run the "simple" tests you put in the README.
I'm on Ubuntu 13.10 if it's of any help.

Thanks in advance.

At least on my machine, it takes on the order of a thousand times as long to run simpleStats as the readme says it should (the number appears to be from 2012, so not surprising)

Why this app uses more than 13GB of ram? any reason? why not using the hard drive?

Hello all,

Any help in order to compile blockparser for Litecoin? No sure what to modify...

Thanks in advance.

However it works without -DWANT_DENSE.

(gdb) where
#0  0x000000000042b105 in google::dense_hashtable<std::pair<unsigned char const* const, Chunk*>, unsigned char const*, Hash256Hasher, google::dense_hash_map<unsigned char const*, Chunk*, Hash256Hasher, Hash256Equal, google::libc_allocator_with_realloc<std::pair<unsigned char const* const, Chunk*> > >::SelectKey, google::dense_hash_map<unsigned char const*, Chunk*, Hash256Hasher, Hash256Equal, google::libc_allocator_with_realloc<std::pair<unsigned char const* const, Chunk*> > >::SetKey, Hash256Equal, google::libc_allocator_with_realloc<std::pair<unsigned char const* const, Chunk*> > >::copy_from(google::dense_hashtable<std::pair<unsigned char const* const, Chunk*>, unsigned char const*, Hash256Hasher, google::dense_hash_map<unsigned char const*, Chunk*, Hash256Hasher, Hash256Equal, google::libc_allocator_with_realloc<std::pair<unsigned char const* const, Chunk*> > >::SelectKey, google::dense_hash_map<unsigned char const*, Chunk*, Hash256Hasher, Hash256Equal, google::libc_allocator_with_realloc<std::pair<unsigned char const* const, Chunk*> > >::SetKey, Hash256Equal, google::libc_allocator_with_realloc<std::pair<unsigned char const* const, Chunk*> > > const&, unsigned long) [clone .local.85] (this=0x7fff5eb61810, ht=..., min_buckets_wanted=<optimized out>) at /usr/include/c++/4.6/bits/stl_construct.h:76
#1  0x000000000041d020 in __base_ctor (min_buckets_wanted=<optimized out>, ht=..., this=0x7fff5eb61810) at /usr/include/google/sparsehash/densehashtable.h:708
#2  google::dense_hashtable<std::pair<unsigned char const* const, Chunk*>, unsigned char const*, Hash256Hasher, google::dense_hash_map<unsigned char const*, Chunk*, Hash256Hasher, Hash256Equal, google::libc_allocator_with_realloc<std::pair<unsigned char const* const, Chunk*> > >::SelectKey, google::dense_hash_map<unsigned char const*, Chunk*, Hash256Hasher, Hash256Equal, google::libc_allocator_with_realloc<std::pair<unsigned char const* const, Chunk*> > >::SetKey, Hash256Equal, google::libc_allocator_with_realloc<std::pair<unsigned char const* const, Chunk*> > >::resize_delta(unsigned long) [clone .local.81] (this=0x645160, delta=<optimized out>) at /usr/include/google/sparsehash/densehashtable.h:599
#3  0x0000000000405cc2 in initHashtables () at /usr/include/google/sparsehash/densehashtable.h:649
#4  main (argc=<optimized out>, argv=<optimized out>) at parser.cpp:839

vagrant@precise64:~$ apt-show-versions libssl-dev build-essential g++-4.4 libboost-all-dev libsparsehash-dev git-core perl
build-essential/precise-updates uptodate 11.5ubuntu2.1
g++-4.4/precise uptodate 4.4.7-1ubuntu2
git-core/precise uptodate 1:1.7.9.5-1
libboost-all-dev/precise uptodate 1.48.0.2
libsparsehash-dev/precise uptodate 1.10-1build1
libssl-dev/precise-security uptodate 1.0.1-4ubuntu5.20
perl/precise-security uptodate 5.14.2-6ubuntu2.4

vagrant@precise64:~/blockparser$ git rev-parse HEAD
3294d0bf16a109382dbbf08d521e4a6e28bf8954

Hello @znort987, I ran the blockparser sql command (which ran successfully) and currently importing the data into mysql via the .bash script. Everything seems to be working fine so far. Great job!

However, I have a question about the SQL data structure. I noticed it dumps all the data into blocks, transactions, inputs and outputs tables in the db. I see that a block contains many transactions and the transactions contain many inputs and outputs. I also see that the input is related to an output (via it's outputID) column and I see that that the output contains a destAddress and value (which I assume is the wallet address and btc value being received by that address?). However I can't find where the sending address is specified. Is there something I'm missing or is this information not available? Please let me know if you can help clairfy. Thanks!

@ubuntu:~/blockparser$ make
lnk -- parser
/tmp/cciT6l1M.ltrans8.ltrans.o: In function Rewards::endOutput(unsigned char const*, unsigned long, unsigned char const*, unsigned long, unsigned char const*, unsigned long)': cciT6l1M.ltrans8.o:(.text+0xba2): undefined reference toEC_KEY_new_by_curve_name'
cciT6l1M.ltrans8.o:(.text+0xbc4): undefined reference to o2i_ECPublicKey' cciT6l1M.ltrans8.o:(.text+0xbd8): undefined reference toEC_KEY_set_conv_form'
cciT6l1M.ltrans8.o:(.text+0xbe7): undefined reference to i2o_ECPublicKey' cciT6l1M.ltrans8.o:(.text+0xbf4): undefined reference toEC_KEY_free'
cciT6l1M.ltrans8.o:(.text+0xc48): undefined reference to EC_KEY_free' /tmp/cciT6l1M.ltrans12.ltrans.o: In functionSQLDump::startBlock(Block const_, unsigned long)':
cciT6l1M.ltrans12.o:(.text+0x8a4): undefined reference to SHA256_Init' cciT6l1M.ltrans12.o:(.text+0x8b6): undefined reference toSHA256_Update'
cciT6l1M.ltrans12.o:(.text+0x8c3): undefined reference to SHA256_Final' cciT6l1M.ltrans12.o:(.text+0x8cd): undefined reference toSHA256_Init'
cciT6l1M.ltrans12.o:(.text+0x8df): undefined reference to SHA256_Update' cciT6l1M.ltrans12.o:(.text+0x8ec): undefined reference toSHA256_Final'
/tmp/cciT6l1M.ltrans22.ltrans.o: In function addrToHash160(unsigned char_, unsigned char const_, bool, bool) [clone .constprop.114]': cciT6l1M.ltrans22.o:(.text+0x14d): undefined reference to BN_set_word'
cciT6l1M.ltrans22.o:(.text+0x1d5): undefined reference toBN_mul_word' cciT6l1M.ltrans22.o:(.text+0x1e4): undefined reference to BN_add_word'
cciT6l1M.ltrans22.o:(.text+0x1fe): undefined reference toBN_bn2mpi' cciT6l1M.ltrans22.o:(.text+0x21c): undefined reference to BN_bn2mpi'
cciT6l1M.ltrans22.o:(.text+0x521): undefined reference toBN_CTX_new' cciT6l1M.ltrans22.o:(.text+0x530): undefined reference to BN_CTX_init'
cciT6l1M.ltrans22.o:(.text+0x535): undefined reference toBN_new' /tmp/cciT6l1M.ltrans22.ltrans.o: In function hash160ToAddr(unsigned char_, unsigned char const_, unsigned char) [clone .constprop.100]':
cciT6l1M.ltrans22.o:(.text+0xef6): undefined reference to BN_mpi2bn' cciT6l1M.ltrans22.o:(.text+0xf23): undefined reference toBN_copy'
cciT6l1M.ltrans22.o:(.text+0xf2f): undefined reference to BN_get_word' cciT6l1M.ltrans22.o:(.text+0xf6e): undefined reference toBN_div'
cciT6l1M.ltrans22.o:(.text+0x1084): undefined reference to BN_CTX_new' cciT6l1M.ltrans22.o:(.text+0x1093): undefined reference toBN_CTX_init'
cciT6l1M.ltrans22.o:(.text+0x1098): undefined reference to BN_new' cciT6l1M.ltrans22.o:(.text+0x10a4): undefined reference toBN_new'
cciT6l1M.ltrans22.o:(.text+0x10b0): undefined reference to BN_new' cciT6l1M.ltrans22.o:(.text+0x10bc): undefined reference toBN_new'
cciT6l1M.ltrans22.o:(.text+0x10d4): undefined reference to BN_set_word' /tmp/cciT6l1M.ltrans23.ltrans.o: In functionbuildBlockHeaders() [clone .lto_priv.219]':
cciT6l1M.ltrans23.o:(.text+0xfd8): undefined reference to SHA256_Init' cciT6l1M.ltrans23.o:(.text+0xfee): undefined reference toSHA256_Update'
cciT6l1M.ltrans23.o:(.text+0xffe): undefined reference to SHA256_Final' cciT6l1M.ltrans23.o:(.text+0x100b): undefined reference toSHA256_Init'
cciT6l1M.ltrans23.o:(.text+0x1020): undefined reference to SHA256_Update' cciT6l1M.ltrans23.o:(.text+0x1030): undefined reference toSHA256_Final'
/tmp/cciT6l1M.ltrans30.ltrans.o: In function sha256(unsigned char_, unsigned char const_, unsigned long)': cciT6l1M.ltrans30.o:(.text+0x15): undefined reference to SHA256_Init'
cciT6l1M.ltrans30.o:(.text+0x23): undefined reference toSHA256_Update' cciT6l1M.ltrans30.o:(.text+0x2e): undefined reference to SHA256_Final'
/tmp/cciT6l1M.ltrans30.ltrans.o: In functionsolveOutputScript(unsigned char_, unsigned char const_, unsigned long, unsigned char_)': cciT6l1M.ltrans30.o:(.text+0x113): undefined reference to SHA256_Init'
cciT6l1M.ltrans30.o:(.text+0x126): undefined reference toSHA256_Update' cciT6l1M.ltrans30.o:(.text+0x133): undefined reference to SHA256_Final'
cciT6l1M.ltrans30.o:(.text+0x13d): undefined reference toRIPEMD160_Init' cciT6l1M.ltrans30.o:(.text+0x14f): undefined reference to RIPEMD160_Update'
cciT6l1M.ltrans30.o:(.text+0x15c): undefined reference toRIPEMD160_Final' cciT6l1M.ltrans30.o:(.text+0x1d2): undefined reference to RIPEMD160_Init'
cciT6l1M.ltrans30.o:(.text+0x1e4): undefined reference toRIPEMD160_Update' cciT6l1M.ltrans30.o:(.text+0x1f1): undefined reference to RIPEMD160_Final'
collect2: error: ld returned 1 exit status
Makefile:310: recipe for target 'parser' failed
make: *** [parser] Error 1

I did everything as described in the readme, but it stops at this point:

user@ubuntu:$ cd blockparser
user@ubuntu:/blockparser$ make
make: *** No targets specified and no makefile found. Stop.

What is wrong?

I get these warning messages (snipped) on every fresh install with satoshi bitcoind last version (0.7.2) while performing any operation (simpleStats, SQL dump...):

warning: at depth 3864 in chain, failed to locate parent block 000000000000040167c69f7a835d124214316b72d9a9fc2ec2413a238b6ab6da
warning: at depth 1109 in chain, failed to locate parent block 000000000000040167c69f7a835d124214316b72d9a9fc2ec2413a238b6ab6da
warning: at depth 7599 in chain, failed to locate parent block 000000000000040167c69f7a835d124214316b72d9a9fc2ec2413a238b6ab6da
warning: at depth 13145 in chain, failed to locate parent block 000000000000040167c69f7a835d124214316b72d9a9fc2ec2413a238b6ab6da
warning: at depth 20124 in chain, failed to locate parent block 000000000000040167c69f7a835d124214316b72d9a9fc2ec2413a238b6ab6da
warning: at depth 3220 in chain, failed to locate parent block 000000000000040167c69f7a835d124214316b72d9a9fc2ec2413a238b6ab6da
warning: at depth 11125 in chain, failed to locate parent block 000000000000040167c69f7a835d124214316b72d9a9fc2ec2413a238b6ab6da
warning: at depth 9522 in chain, failed to locate parent block 000000000000023b3bc9f555a761bd400dcf41bce3116eaf488ae6904be0ce2c
warning: at depth 8124 in chain, failed to locate parent block 000000000000040167c69f7a835d124214316b72d9a9fc2ec2413a238b6ab6da
warning: at depth 3490 in chain, failed to locate parent block 000000000000040167c69f7a835d124214316b72d9a9fc2ec2413a238b6ab6da
warning: at depth 4340 in chain, failed to locate parent block 000000000000023b3bc9f555a761bd400dcf41bce3116eaf488ae6904be0ce2c
warning: at depth 10194 in chain, failed to locate parent block 000000000000023b3bc9f555a761bd400dcf41bce3116eaf488ae6904be0ce2c
warning: at depth 166 in chain, failed to locate parent block 000000000000023b3bc9f555a761bd400dcf41bce3116eaf488ae6904be0ce2c
warning: at depth 1742 in chain, failed to locate parent block 000000000000023b3bc9f555a761bd400dcf41bce3116eaf488ae6904be0ce2c
warning: at depth 20410 in chain, failed to locate parent block 000000000000040167c69f7a835d124214316b72d9a9fc2ec2413a238b6ab6da
warning: at depth 3488 in chain, failed to locate parent block 000000000000040167c69f7a835d124214316b72d9a9fc2ec2413a238b6ab6da
warning: at depth 5937 in chain, failed to locate parent block 000000000000023b3bc9f555a761bd400dcf41bce3116eaf488ae6904be0ce2c
warning: at depth 566 in chain, failed to locate parent block 000000000000023b3bc9f555a761bd400dcf41bce3116eaf488ae6904be0ce2c
warning: at depth 5972 in chain, failed to locate parent block 000000000000023b3bc9f555a761bd400dcf41bce3116eaf488ae6904be0ce2c
warning: at depth 5899 in chain, failed to locate parent block 000000000000023b3bc9f555a761bd400dcf41bce3116eaf488ae6904be0ce2c
warning: at depth 11022 in chain, failed to locate parent block 000000000000023b3bc9f555a761bd400dcf41bce3116eaf488ae6904be0ce2c
warning: at depth 10152 in chain, failed to locate parent block 000000000000040167c69f7a835d124214316b72d9a9fc2ec2413a238b6ab6da

and for this reason, stops at block 188527 (since 000000000000040167c69f7a835d124214316b72d9a9fc2ec2413a238b6ab6da is block 188528).

Any advice?
Thanks.

Nice work. Your code is super clean, simple and fast.

I'd like to ask you some ideas about how to implement real time parser of new transactions in parser.cpp
For example a callback to read all tx with a determined OP_RETURN pattern and then keep looping showing new transactions with that pattern.

What do you think is the best way to implement it? I don't need specific details just some hints from your side about what would be the optimal way.

I can't make on Ubuntu 14.04. I get a lot of errors. Maybe use a different version Ubuntu? Pls help

On an ubuntu install I did with an encrypted home directory, the program won't even compile. Perhaps this limitation is worthy of mention in README.md

strace ./parser closure returns

http://pastebin.com/yEfexr7G

Someone use this with bitcoin-qt 0.10 ?
Whith my blockchain archive, the blockparser crash.

The O_DIRECT flag to the open() call in parser.cpp in line 403 gives me an EINVAL and the binary segfaults. From my ubuntu 12.04 open(2):

   O_DIRECT support was added under Linux in kernel version 2.4.10.  Older Linux kernels simply ignore this flag.  Some file systems may not implement the flag and open() will fail with EINVAL if it is used.

My kernel version:
Linux euler 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 16:19:23 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

Just removing the flag from the open call seems to be enough. Maybe there's some define to check to see whether it's supported or not.

Any plans for multisig support?

info: 426834 blocks, 170.954 MegaAddrs ,
info: 426837 blocks, 170.963 MegaAddrs ,
info: 426840 blocks, 170.971 MegaAddrs ,
info: 426843 blocks, 170.977 MegaAddrs ,
info: 426847 blocks, 170.988 MegaAddrs ,
info: 426851 blocks, 170.998 MegaAddrs ,
Killed

it got killed twice, with 32GB Ram and 64GB SSD Swap.
at block 426851 seems the last it can handle, any idea?

I want to use this program with Cryptoescudo / CESC, what is the gExpectedMagic for CryptoEscudo? Where can i find it?

Hello,
I'm trying to dump the blockchain into sql with ./parser sql since 2 weeks. It always ends with a segmentation fault at the same block. Of course, I re-downloaded the whole blockchan a couple of times and reindexed it. Has anyone the same problem? Is there anything special with that block?
I tried it with latest blockparser version as well as with the old one within the Bitiodine project. None works.

Here the last lines of blocks.txt:

380585|0000000000000000062A51F306EB30550685811A32D71EC0C5CFFBCA2594E83D|1445837119
380586|00000000000000000702868C4DA5FDB21A4EB7F56D5F6B4713F45B5926D69FA1|1445837516
380587|00000000000000000F421239C817E0BCB0C2EE858A7A4DD7D3BC51129261738F|1445837808
380588|0000000000000000091A480924495D0C1E8F8E61B6F75B7C099D7F24B48

I want to dump the blockchain, so i run:

parser sqldump

It starts to dump and seems to work fine, until block 225500 is reached. It then ends with a segmentation fault:

block= 224000 nbOutputs=31901131
block= 224500 nbOutputs=32325594
block= 225000 nbOutputs=32761533
block= 225500 nbOutputs=33208004
Segmentation Fault

This error is reproducable for me, i downloaded the sources three days ago, so they should be up to date.

Hello,

when compiling the LiteCoin version, the following error pops up

scrypt/pbkdf2.cpp:33:18: error: variable or field ‘HMAC_SHA256_Init’ declared void
HMAC_SHA256_Init(HMAC_SHA256_CTX * ctx, const void * _K, size_t Klen)
^
scrypt/pbkdf2.cpp:33:18: error: ‘HMAC_SHA256_CTX’ was not declared in this scope
scrypt/pbkdf2.cpp:33:36: error: ‘ctx’ was not declared in this scope
HMAC_SHA256_Init(HMAC_SHA256_CTX * ctx, const void * _K, size_t Klen)

This is on a Gentoo Linux.

Any ideas?
Thank you
J

Hi!
Blockparser is all setup for me, using an AWS instance with Ubuntu 14.04 with a fresh install! However, to fully get the potential, I would like to visualize all the data extracted. Not sure which tool, or if there is something more suitable than others.

If I'd have to choose between R and Python, I'd go for something made with Python.

(I know this gets a little out of the scope of your utility, but you may know much better than me about a tool for this)

removed

I ran blockparser allBalances for this output:

info: done

info: found 597329 addresses with non zero balance
info: found 3152742 addresses in total
info: shown:3152742 addresses

info: pass 4 -- done.
info: all done in 112.46 seconds
info: mem at end = 0.000 Gigs

So it says to have found 3 million addresses. But on bitocoinrichlist.com it says there are 144,520,819 addresses:

http://www.bitcoinrichlist.com/charts/bitcoin-distribution-by-address?atblock=410000

I saved allBalances to all.txt attached and it seems it only got as far as 2012:
"State of the ledger at block 169011 (minted : Wed Feb 29 06:02:41 2012)"

https://www.dropbox.com/s/u36cghwtwgmlw5u/all.txt?dl=0

How do I get the missing blocks?

Hi,
I know it takes a long time to get allbalance... but is it possible that it takes so much time:

12779 2.1 94.7 30095976 23387992 pts/5 D+ Aug14 58:23 /home/username/bt -parser allBalances -w 1000000

It is running since 2 days and on block 395000, am i doing something wrong?

Intel 6 Core, 24GB ram, Ubuntu 14.04

Also i would ask, is there a way to pause it and continue later? Or do i start from the beginning?

Thanks

using the allbalances command with bitcoin-core 0.13 blokchain always prints garbage output:

5000000000 �\0$wŠ™O¯��>
\ g‰°?ɪΠ1 1234915431 0 0
5000000000 =\0&rѸ�ã–b2�ˆºŒ¨/ÔåÊ 1 1232758347 0 0
5000000000 >\0.Ró`hnO��U™�p+ÚDyñ 1 1236920555 0 0
5000000000 H\0œx²��Û£àÀº“.¡åJŽš( 1 1232773213 0 0
5000000000 �\0®îW®åea´Ü¿Ï]ƒ÷óK8Û 1 1235274673 0 0
5000000000 €\0äÿú€ñA�à)£O�G-(°Ù­ 1 1233148853 0 0
5000000000 „\0¸¿¥¹Š€˜#�e‹½–ƒJê.� 1 1235914652 0 0

Have you done any testing on performance in regards to using __builtin_expect? Does it give any performance increase at all?

Hi,

would be nice if someone could implement an argument for a different blockchain location then ~/.bitcoin.
Most people I know have the blockchain on a different hdd because ssds are quite expensive.

Thanks in advance.

Tried on both vagrant's hashicorp/precise64 and ubuntu/trusty64. Compile crashes on different files in both cases, but usually it looks like:

c++ -- util.cpp
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.6/README.Bugs> for instructions.
make: *** [.objs/pristine.o] Error 4
make: *** Waiting for unfinished jobs....

Host system is MacOSX 10.10.1