cculianu / fulcrum Goto Github PK
View Code? Open in Web Editor NEWA fast & nimble SPV Server for BCH, BTC, and LTC
License: Other
fulcrum's People
Contributors
Stargazers
Watchers
Forkers
dagurval echterago vitaeteam francobenner bchplease indexchain hegjon infertux christroutner blockparty-sh nirvananimbusa mave95 sklayer dgenr8 jcramer baby636 givelotus ejose19 actorforth mainnet-pat savyp runcitadel aarondewes 22388o catsduck quad jimtendo crypto-currency-projects maren7 pseudoramdom bitister shmutalov davidecoltro fsmv sep21-2009 baro77 matthew1838 openoms melroy89 tradewind886 dorucioclea modenero nyusternie fujicoin dorveille fort-nakamoto colder-is-better mariusk koad shitcoinssuck sampleblood bamahodl foundation-devices h10086733 kdmukai jkister kitzen-io trickydastickylate flipstarter-causes-cash fyookball cyberkayt souvikhaldar cerveeza77 miz1choobi guardianoftherealmfulcrum's Issues
Release binary for Raspberry Pi
After writing https://www.sparrowwallet.com/docs/server-performance.html I was advised to look at Fulcrum as an alternative to ElectrumX. As I noted in the article, profiling ElectrumX's indexing behaviour shows that its single threaded approach does not scale on limited hardware. I am unsure how Fulcrum tackles this, but I'd like to try it.
The Raspberry Pi is probably the most common hardware used to run BTC nodes given the number of prebuilt node packages targeting it (Umbrel, Nodl, MyNode, Ronin Dojo etc). Currently these platforms all contain Electrs, which does not scale well with "deep" wallets.
As such I think it would be interesting to see if Fulcrum can be the "goldilocks" private Electrum server implementation - fast enough indexing, and with an inclusive index able to provide reasonable performance for deep wallets. Compiling Fulcrum for this platform seems daunting however: https://wiki.qt.io/RaspberryPi2EGLFS - has this been attempted? Is this a reasonable proposition? If so I would love to see a release binary targeting this architecture.
Cannot connect to Fulcrum via Tor
Hello,
I have successfully synced Fulcrum on raspberry pi 4 and trying to set up Tor hidden service to use for remote connections. Fulcrum works connecting on my local network specifing device IP and Port. However I am unable to connect via Tor, I don't know
how working config looks like, since it is very different from electrum config file I was using before.
I have set up following in my torrc config:
Torrc
HiddenServiceDir /var/lib/tor/hidden_service_fulcrum/
HiddenServiceVersion 3
HiddenServicePort 50002 127.0.0.1:50002and then printed my hostname using: 'sudo cat /var/lib/tor/hidden_service_fulcrum/hostname'
xyz.onion
My config file for fulcrum is set up like this:
Fulcrum config
datadir = /path/to/db/
bitcoind = 127.0.0.1:8332
rpcuser = username
rpcpassword = password
cert = /path/
key = /path/
peering = false
ssl = 0.0.0.0:50002
tor_hostname=xyz.onion (copied from hostname)
tor_tcp_port = 50001
tor_ssl_port = 50002- ports allowed for UFW
NGINX
upstream fulcrum {
server 127.0.0.1:50001;
}
server {
listen 50002 ssl;
proxy_pass fulcrum;
}Trying to connect with my wallet for both:
xyz.onion:50001
-//- :50002Wallet is confirmed to work with Tor since I used it for a long time with electrum. Am I missing something in my config files?
QObject and std::unique_ptr do the same thing
I noticed in your code that you created a private member that is a unique pointer which holds a QObject inherting object. In the following case a QThreadPool.
Line 121 in 65bc0fe
When a QObject is deleted all the children of it are also deleted. So you don't need the unique_ptr, all you need to do is pass in 'this' to the constructor of the QThreadPool. Then when your class is deleted it will delete the pool member anyway.
Raspi 4 binary
I'm attempting to run the 'Fulcrum-1.6.0-arm64-linux.tar.gz' binary on a raspi 4 and getting:
pi@raspberrypi:~/Fulcrum $ ./Fulcrum
bash: ./Fulcrum: cannot execute binary file: Exec format error
Should this work on a raspi 4?
Error on batched request
Fulcrum 1.6.0 ((Release 0bf5ddf, I recompiled to latest master branch) is giving an error on being presented with a batched request. Here are the relevant log lines from Sparrow:
Send: [{"jsonrpc":"2.0","method":"blockchain.estimatefee","params":[1],"id":1},{"jsonrpc":"2.0","method":"blockchain.estimatefee","params":[2],"id":2},{"jsonrpc":"2.0","method":"blockchain.estimatefee","params":[3],"id":3},{"jsonrpc":"2.0","method":"blockchain.estimatefee","params":[4],"id":4},{"jsonrpc":"2.0","method":"blockchain.estimatefee","params":[5],"id":5},{"jsonrpc":"2.0","method":"blockchain.estimatefee","params":[10],"id":10},{"jsonrpc":"2.0","method":"blockchain.estimatefee","params":[25],"id":25},{"jsonrpc":"2.0","method":"blockchain.estimatefee","params":[50],"id":50}]
Recv: [{"error":{"code":-32600,"message":"Invalid request"},"id":null,"jsonrpc":"2.0"}]
When Sparrow sends these requests non-batched, all works well:
Send: {"jsonrpc":"2.0","method":"blockchain.estimatefee","params":[1],"id":4}
Recv: {"id":4,"jsonrpc":"2.0","result":5.074e-05}
Send: {"jsonrpc":"2.0","method":"blockchain.estimatefee","params":[2],"id":5}
Recv: {"id":5,"jsonrpc":"2.0","result":5.074e-05}
Send: {"jsonrpc":"2.0","method":"blockchain.estimatefee","params":[3],"id":6}
Recv: {"id":6,"jsonrpc":"2.0","result":5.074e-05}
Send: {"jsonrpc":"2.0","method":"blockchain.estimatefee","params":[4],"id":7}
Recv: {"id":7,"jsonrpc":"2.0","result":3.004e-05}
Send: {"jsonrpc":"2.0","method":"blockchain.estimatefee","params":[5],"id":8}
Recv: {"id":8,"jsonrpc":"2.0","result":3.004e-05}
Send: {"jsonrpc":"2.0","method":"blockchain.estimatefee","params":[10],"id":9}
Recv: {"id":9,"jsonrpc":"2.0","result":1.381e-05}
Send: {"jsonrpc":"2.0","method":"blockchain.estimatefee","params":[25],"id":10}
Recv: {"id":10,"jsonrpc":"2.0","result":1.019e-05}
The batched request does work in ElectrumX and Electrs.
Support running behind a load balancer
There are several headers that need to be supported for banning not to ban the load balancer. Specifically X-Forward-IP. Fixing this would allow someone to run a bunch of copies of this under one IP/domain.
Build fails on fedora-32-armhfp
gcc -c -pipe -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -march=armv7-a -mfpu=vfpv3-d16 -mtune=generic-armv7-a -mabi=aapcs-linux -mfloat-abi=hard -DNDEBUG -fomit-frame-pointer -O2 -w -D_REENTRANT -fPIC -DQT_DEPRECATED_WARNINGS -DUSE_QT_IN_BITCOIN -DHAVE_ENDIAN_H -DHAVE_DECL_HTOBE16 -DHAVE_DECL_HTOLE16 -DHAVE_DECL_BE16TOH -DHAVE_DECL_LE16TOH -DHAVE_DECL_HTOBE32 -DHAVE_DECL_HTOLE32 -DHAVE_DECL_BE32TOH -DHAVE_DECL_LE32TOH -DHAVE_DECL_HTOBE64 -DHAVE_DECL_HTOLE64 -DHAVE_DECL_BE64TOH -DHAVE_DECL_LE64TOH -DHAVE_JEMALLOC_HEADERS -DQT_NO_DEBUG -DQT_NETWORK_LIB -DQT_CORE_LIB -I. -Isrc -isystem /usr/include/qt5 -isystem /usr/include/qt5/QtNetwork -isystem /usr/include/qt5/QtCore -I. -I/usr/lib/qt5/mkspecs/linux-g++ -o secp256k1.o src/bitcoin/secp256k1/secp256k1.c
In file included from src/bitcoin/secp256k1/secp256k1.c:18:
src/bitcoin/secp256k1/util.h:110:37: error: '__int128' is not supported on this target
110 | SECP256K1_GNUC_EXT typedef unsigned __int128 uint128_t;
| ^~~~~~~~
See https://download.copr.fedorainfracloud.org/results/jonny/BitcoinCash/fedora-32-armhfp/01706384-fulcrum/builder-live.log.gz for details
Support for EC certificates
Hi, thank you for this promising project! I'm currently migrating my old ElectrumX server but I hit a roadblock as Fulcrum only accepts RSA certificates. Here is the error I'm getting:
Unable to read private key from foobar.key. Please make sure the file is readable and contains a single RSA private key in PEM format.
$ file foobar.key
foobar.key: PEM EC private keyIt seems QSsl::KeyAlgorithm::Rsa is hardcoded here. I'm not familiar with C++ but would there be a way to automatically detect the algorithm or perhaps fall back until we find the right one? Alternatively, we could add a new config option like keyalgorithm=ec? FWIW ElectrumX seemed to handle this automatically as I didn't configure the algorithm explicitly. Thanks.
Fulcrum database corruption
Hi,
Since Fulcrum is becoming more popular among bitcoin community due to its great features, I would like to ask a question, once database is complete, is fulcrum still prone to database corruption? If it is, how can I ensure safe reboot of device without corrupting the database?
Are there any easy set of commands for raspberry pi users, who do not have much experience in Linux? Would be great to add for RaspiBolt guides etc. Also for me as a linux noob who managed to corrupt it in several ways...
Thank you very much for response
Add option to use `.cookie` file
It would be great if Fulcrum could read from the cookie file instead of using rpc credentials, as it saves the user from managing another pair of credentials by relying on directory permissions.
how to get height to use with get_merkle?
I have a transaction hash, which uniquely identifies a transaction. I can make a call to get the transaction hex so I can parse the transaction contents.
In some usecases, such as with listunspent and get_history I get the block height as part of the call, but what am I expected to do if I start with a transaction, and need to learn if it is included in a block, and if so, get the merkle proof?
( could call get with verbose flag on, but that only gives me confirmations - not the actual blockheight)
Fulcrum process error starting [VirtualBox vboxsf fsync]
First run. Ubuntu 20.04.3 LTS.
Here it is the output:
:~/Fulcrum-1.6.0-x86_64-linux$ ./Fulcrum fulcrum-config.conf
[2022-02-01 20:45:53.335] Enabled JSON parser: simdjson
[2022-02-01 20:45:53.336] simdjson implementations:
[2022-02-01 20:45:53.336] haswell: Intel/AMD AVX2 [not supported]
[2022-02-01 20:45:53.336] westmere: Intel/AMD SSE4.2 [supported]
[2022-02-01 20:45:53.336] fallback: Generic fallback implementation [supported]
[2022-02-01 20:45:53.336] active implementation: westmere
[2022-02-01 20:45:53.338] jemalloc: version 5.2.1-0-gea6b3e9
[2022-02-01 20:45:53.338] Qt: version 5.15.2
[2022-02-01 20:45:53.338] rocksdb: version 6.14.6-ed43161
[2022-02-01 20:45:53.338] simdjson: version 0.6.0
[2022-02-01 20:45:53.338] ssl: OpenSSL 1.1.1f 31 Mar 2020
[2022-02-01 20:45:53.338] zmq: libzmq version: 4.3.3, cppzmq version: 4.7.1
[2022-02-01 20:45:53.338] Fulcrum 1.6.0 (Release 5e95c0f) - mar. feb. 1, 2022 20:45:53.338 CET - starting up ...
[2022-02-01 20:45:53.339] Max open files: 1048576 (increased from default: 1024)
[2022-02-01 20:45:53.339] Loading database ...
[2022-02-01 20:45:53.356] FATAL: Caught exception: Error opening meta database: IO error: While fsync: a directory: Invalid argument (path: /media/minodobitcoin/Fulcrum/meta)
[2022-02-01 20:45:53.356] Stopping Controller ...
[2022-02-01 20:45:53.356] Closing storage ...
[2022-02-01 20:45:53.356] Shutdown complete
The meta directory is created fine with the following files:
CURRENT IDENTITY LOCK LOG MANIFEST-000001
and this is the content of the LOG file:
2022/02/01-20:45:53.341824 7fee5a546340 Git sha rocksdb_build_git_sha:ed4316166f67ec892603014634840d29f460f611
2022/02/01-20:45:53.341829 7fee5a546340 Compile date Jan 25 2022
2022/02/01-20:45:53.341880 7fee5a546340 DB SUMMARY
2022/02/01-20:45:53.341882 7fee5a546340 DB Session ID: WGTZJQPJ1PCBOCL5U50X
2022/02/01-20:45:53.342360 7fee5a546340 SST files in /media/minodobitcoin/Fulcrum/meta dir, Total Num: 0, files:
2022/02/01-20:45:53.342364 7fee5a546340 Write Ahead Log file in /media/minodobitcoin/Fulcrum/meta:
2022/02/01-20:45:53.342367 7fee5a546340 Options.error_if_exists: 0
2022/02/01-20:45:53.342369 7fee5a546340 Options.create_if_missing: 1
2022/02/01-20:45:53.342371 7fee5a546340 Options.paranoid_checks: 1
2022/02/01-20:45:53.342373 7fee5a546340 Options.env: 0x5595d9f3a6a0
2022/02/01-20:45:53.342375 7fee5a546340 Options.fs: Posix File System
2022/02/01-20:45:53.342377 7fee5a546340 Options.info_log: 0x7fee5a10cf60
2022/02/01-20:45:53.342379 7fee5a546340 Options.max_file_opening_threads: 16
2022/02/01-20:45:53.342380 7fee5a546340 Options.statistics: (nil)
2022/02/01-20:45:53.342382 7fee5a546340 Options.use_fsync: 0
2022/02/01-20:45:53.342384 7fee5a546340 Options.max_log_file_size: 0
2022/02/01-20:45:53.342386 7fee5a546340 Options.max_manifest_file_size: 1073741824
2022/02/01-20:45:53.342388 7fee5a546340 Options.log_file_time_to_roll: 0
2022/02/01-20:45:53.342390 7fee5a546340 Options.keep_log_file_num: 5
2022/02/01-20:45:53.342392 7fee5a546340 Options.recycle_log_file_num: 0
2022/02/01-20:45:53.342393 7fee5a546340 Options.allow_fallocate: 1
2022/02/01-20:45:53.342395 7fee5a546340 Options.allow_mmap_reads: 0
2022/02/01-20:45:53.342397 7fee5a546340 Options.allow_mmap_writes: 0
2022/02/01-20:45:53.342399 7fee5a546340 Options.use_direct_reads: 0
2022/02/01-20:45:53.342400 7fee5a546340 Options.use_direct_io_for_flush_and_compaction: 0
2022/02/01-20:45:53.342402 7fee5a546340 Options.create_missing_column_families: 0
2022/02/01-20:45:53.342404 7fee5a546340 Options.db_log_dir:
2022/02/01-20:45:53.342406 7fee5a546340 Options.wal_dir: /media/minodobitcoin/Fulcrum/meta
2022/02/01-20:45:53.342408 7fee5a546340 Options.table_cache_numshardbits: 6
2022/02/01-20:45:53.342409 7fee5a546340 Options.WAL_ttl_seconds: 0
2022/02/01-20:45:53.342411 7fee5a546340 Options.WAL_size_limit_MB: 0
2022/02/01-20:45:53.342413 7fee5a546340 Options.max_write_batch_group_size_bytes: 1048576
2022/02/01-20:45:53.342415 7fee5a546340 Options.manifest_preallocation_size: 4194304
2022/02/01-20:45:53.342417 7fee5a546340 Options.is_fd_close_on_exec: 1
2022/02/01-20:45:53.342418 7fee5a546340 Options.advise_random_on_open: 1
2022/02/01-20:45:53.342420 7fee5a546340 Options.db_write_buffer_size: 0
2022/02/01-20:45:53.342422 7fee5a546340 Options.write_buffer_manager: 0x7fee5a11e750
2022/02/01-20:45:53.342424 7fee5a546340 Options.access_hint_on_compaction_start: 1
2022/02/01-20:45:53.342425 7fee5a546340 Options.new_table_reader_for_compaction_inputs: 0
2022/02/01-20:45:53.342427 7fee5a546340 Options.random_access_max_buffer_size: 1048576
2022/02/01-20:45:53.342429 7fee5a546340 Options.use_adaptive_mutex: 0
2022/02/01-20:45:53.342431 7fee5a546340 Options.rate_limiter: (nil)
2022/02/01-20:45:53.342433 7fee5a546340 Options.sst_file_manager.rate_bytes_per_sec: 0
2022/02/01-20:45:53.342435 7fee5a546340 Options.wal_recovery_mode: 2
2022/02/01-20:45:53.342436 7fee5a546340 Options.enable_thread_tracking: 0
2022/02/01-20:45:53.342438 7fee5a546340 Options.enable_pipelined_write: 0
2022/02/01-20:45:53.342440 7fee5a546340 Options.unordered_write: 0
2022/02/01-20:45:53.342442 7fee5a546340 Options.allow_concurrent_memtable_write: 1
2022/02/01-20:45:53.342443 7fee5a546340 Options.enable_write_thread_adaptive_yield: 1
2022/02/01-20:45:53.342445 7fee5a546340 Options.write_thread_max_yield_usec: 100
2022/02/01-20:45:53.342447 7fee5a546340 Options.write_thread_slow_yield_usec: 3
2022/02/01-20:45:53.342449 7fee5a546340 Options.row_cache: None
2022/02/01-20:45:53.342450 7fee5a546340 Options.wal_filter: None
2022/02/01-20:45:53.342452 7fee5a546340 Options.avoid_flush_during_recovery: 0
2022/02/01-20:45:53.342454 7fee5a546340 Options.allow_ingest_behind: 0
2022/02/01-20:45:53.342456 7fee5a546340 Options.preserve_deletes: 0
2022/02/01-20:45:53.342458 7fee5a546340 Options.two_write_queues: 0
2022/02/01-20:45:53.342459 7fee5a546340 Options.manual_wal_flush: 0
2022/02/01-20:45:53.342461 7fee5a546340 Options.atomic_flush: 0
2022/02/01-20:45:53.342463 7fee5a546340 Options.avoid_unnecessary_blocking_io: 0
2022/02/01-20:45:53.342465 7fee5a546340 Options.persist_stats_to_disk: 0
2022/02/01-20:45:53.342467 7fee5a546340 Options.write_dbid_to_manifest: 0
2022/02/01-20:45:53.342468 7fee5a546340 Options.log_readahead_size: 0
2022/02/01-20:45:53.342470 7fee5a546340 Options.file_checksum_gen_factory: Unknown
2022/02/01-20:45:53.342472 7fee5a546340 Options.best_efforts_recovery: 0
2022/02/01-20:45:53.342474 7fee5a546340 Options.max_bgerror_resume_count: 2147483647
2022/02/01-20:45:53.342475 7fee5a546340 Options.bgerror_resume_retry_interval: 1000000
2022/02/01-20:45:53.342477 7fee5a546340 Options.allow_data_in_errors: 0
2022/02/01-20:45:53.342479 7fee5a546340 Options.max_background_jobs: 1
2022/02/01-20:45:53.342481 7fee5a546340 Options.max_background_compactions: -1
2022/02/01-20:45:53.342483 7fee5a546340 Options.max_subcompactions: 1
2022/02/01-20:45:53.342484 7fee5a546340 Options.avoid_flush_during_shutdown: 0
2022/02/01-20:45:53.342486 7fee5a546340 Options.writable_file_max_buffer_size: 1048576
2022/02/01-20:45:53.342488 7fee5a546340 Options.delayed_write_rate : 16777216
2022/02/01-20:45:53.342490 7fee5a546340 Options.max_total_wal_size: 0
2022/02/01-20:45:53.342492 7fee5a546340 Options.delete_obsolete_files_period_micros: 21600000000
2022/02/01-20:45:53.342493 7fee5a546340 Options.stats_dump_period_sec: 600
2022/02/01-20:45:53.342495 7fee5a546340 Options.stats_persist_period_sec: 600
2022/02/01-20:45:53.342497 7fee5a546340 Options.stats_history_buffer_size: 1048576
2022/02/01-20:45:53.342499 7fee5a546340 Options.max_open_files: 40
2022/02/01-20:45:53.342501 7fee5a546340 Options.bytes_per_sync: 0
2022/02/01-20:45:53.342502 7fee5a546340 Options.wal_bytes_per_sync: 0
2022/02/01-20:45:53.342504 7fee5a546340 Options.strict_bytes_per_sync: 0
2022/02/01-20:45:53.342506 7fee5a546340 Options.compaction_readahead_size: 0
2022/02/01-20:45:53.342508 7fee5a546340 Options.max_background_flushes: -1
2022/02/01-20:45:53.342509 7fee5a546340 Compression algorithms supported:
2022/02/01-20:45:53.342511 7fee5a546340 kZSTDNotFinalCompression supported: 0
2022/02/01-20:45:53.342513 7fee5a546340 kZSTD supported: 0
2022/02/01-20:45:53.342515 7fee5a546340 kXpressCompression supported: 0
2022/02/01-20:45:53.342517 7fee5a546340 kLZ4HCCompression supported: 0
2022/02/01-20:45:53.342829 7fee5a546340 kLZ4Compression supported: 0
2022/02/01-20:45:53.342834 7fee5a546340 kBZip2Compression supported: 1
2022/02/01-20:45:53.342836 7fee5a546340 kZlibCompression supported: 1
2022/02/01-20:45:53.342838 7fee5a546340 kSnappyCompression supported: 0
2022/02/01-20:45:53.342841 7fee5a546340 Fast CRC32 supported: Not supported on x86
2022/02/01-20:45:53.344249 7fee5a546340 [/db_impl/db_impl_open.cc:290] Creating manifest 1
2022/02/01-20:45:53.346493 7fee5a546340 [WARN] [/db_impl/db_impl_open.cc:1805] Persisting Option File error: OK
2022/02/01-20:45:53.346603 7fee5a546340 [/db_impl/db_impl.cc:447] Shutdown: canceling all background work
2022/02/01-20:45:53.346692 7fee5a546340 [/db_impl/db_impl.cc:623] Shutdown complete
How to run Fulcrum with bchd, using HTTPS-only JSON-RPC
@cculianu Indeed! I'm running bchd with TLS enabled for the gRPC API and that enables TLS for the RPC API as well. Fulcrum expect the RPC API to be plain HTTP so I was using nginx proxy_pass https://127.0.0.1:8334/ to strip the TLS encryption. However nginx adds a new header automatically - Connection: keep-alive - that you cannot hide even with proxy_hide_header. So I tried patching Fulcrum to ignore it, which mitigated the issue. However nginx would still send a Connection: close header after a while. At which point I said fuck nginx then tried with socat which worked perfectly!
So if anyone is running bchd with TLS enabled and wants to point Fulcrum to it, you can use socat TCP-LISTEN:1337,fork,reuseaddr ssl:127.0.0.1:8334,verify=0 then set bitcoind = 127.0.0.1:1337 in fulcrum.conf.
Originally posted by @infertux in #28 (comment)
Update
As of this commit: ce4b969, you can now just give Fulcrum the --bitcoind-tls option on the CLI or specify bitcoind-tls = true in the conf file to use Fulcrum with bchd HTTPS.
Allow setting custom zmqpubhashblock port
Since it's possible to run ZMQ channels on different ports than default (and some setups like Samourai Dojo use it like this), it would be great if it would be possible to set zmqpubhashblock value either via conf file or CLI (or both ideally) so Fulcrum can leverage this feature even if not running on default port.
I am not skilled in C++, otherwise I would be happy to make a PR.
JSON-RPC batching?
Seems that JSON-RPC batching (as described in spec) is not supported:
$ echo '{"id": 1, "method": "blockchain.scripthash.get_balance", "params": ["716decbe1660861c3d93906cb1d98ee68b154fd4d23aed9783859c1271b52a9c"]}' | timeout 2 openssl s_client -quiet -connect electrumx.no-ip.eu:50002 2>/dev/null
{"id":1,"jsonrpc":"2.0","result":{"confirmed":51432,"unconfirmed":0}}
$ echo '[{"id": 1, "method": "blockchain.scripthash.get_balance", "params": ["716decbe1660861c3d93906cb1d98ee68b154fd4d23aed9783859c1271b52a9c"]}]' | timeout 2 openssl s_client -quiet -connect electrumx.no-ip.eu:50002 2>/dev/null
{"error":{"code":-31999,"message":"Json Error: expected object"},"id":null,"jsonrpc":"2.0"}version tested:
$ echo '{"id": 1, "method": "server.version", "params": []}' | timeout 2 openssl s_client -quiet -connect electrumx.no-ip.eu:50002 2>/dev/null
{"id":1,"jsonrpc":"2.0","result":["Fulcrum 1.4.0","1.4"]}
FATAL: Caught exception: File size is not a multiple of recordSize
Hi,
looks like my VM got rebootet and afterwards Fulcrum doesn't start anymore with the error FATAL: Caught exception: File size is not a multiple of recordSize. Do I have to delete the db folder and sync again from scratch or is there anything else I can try to fix this?
Jun 15 07:17:00 server Fulcrum[82589]: Loaded SSL certificate: Internet Widgits Pty Ltd expires: Tue March 12 2026 12:42:59
Jun 15 07:17:00 server Fulcrum[82589]: Loaded key type: private algorithm: RSA
Jun 15 07:17:00 server Fulcrum[82589]: Enabled JSON parser: simdjson
Jun 15 07:17:00 server Fulcrum[82589]: simdjson implementations:
Jun 15 07:17:00 server Fulcrum[82589]: haswell: Intel/AMD AVX2 [supported]
Jun 15 07:17:00 server Fulcrum[82589]: westmere: Intel/AMD SSE4.2 [supported]
Jun 15 07:17:00 server Fulcrum[82589]: fallback: Generic fallback implementation [supported]
Jun 15 07:17:00 server Fulcrum[82589]: active implementation: haswell
Jun 15 07:17:00 server Fulcrum[82589]: jemalloc: version 5.2.1-0-gea6b3e9
Jun 15 07:17:00 server Fulcrum[82589]: Qt: version 5.15.2
Jun 15 07:17:00 server Fulcrum[82589]: rocksdb: version 6.14.6-ed43161
Jun 15 07:17:00 server Fulcrum[82589]: simdjson: version 0.6.0
Jun 15 07:17:00 server Fulcrum[82589]: ssl: OpenSSL 1.1.1f 31 Mar 2020
Jun 15 07:17:00 server Fulcrum[82589]: zmq: libzmq version: 4.3.3, cppzmq version: 4.7.1
Jun 15 07:17:00 server Fulcrum[82589]: Fulcrum 1.7.0 (Release 4ee413a) - Wed Jun 15, 2022 07:17:00.367 UTC - starting up ...
Jun 15 07:17:00 server Fulcrum[82589]: Max open files: 1048576 (increased from default: 1024)
Jun 15 07:17:00 server Fulcrum[82589]: Loading database ...
Jun 15 07:17:05 server Fulcrum[82589]: DB memory: 512.00 MiB
Jun 15 07:17:05 server Fulcrum[82589]: Coin: BTC
Jun 15 07:17:05 server Fulcrum[82589]: Chain: main
Jun 15 07:17:05 server Fulcrum[82589]: FATAL: Caught exception: File size is not a multiple of recordSize
Jun 15 07:17:05 server Fulcrum[82589]: Stopping Controller ...
Jun 15 07:17:05 server Fulcrum[82589]: Closing storage ...
Jun 15 07:17:05 server Fulcrum[82589]: Shutdown complete
Allow basic RPC handling during initial sync
Hey Calin,
Unless I am doing something wrong, I am noticing that Fulcrum rejects all requests while it is doing its initial block sync. When I try to connect, I get connect errror: connect ECONNREFUSED 172.18.0.3:50001.
I think it would be nice from a user experience / developer experience stand point to accept basic RPC calls during initial sync so that a web UI that relies on Fulcrum can communicate sync progress to the user. Basically, just what block Fulcrum is currently synced up to.
I'm not sure if it's helpful, but Electrs implemented a similar feature a few months ago. Here is the PR for it: romanz/electrs#543
If there is a way of doing this already, please let me know!
listunspent does not conform to expected behavior
From Rosco Kalis:
Hey Calin, I'm using electrum-cash to send some transactions, using a 2-of-3 cluster (all running Fulcrum 1.2.8). Before running the tx, this is the listunspent result:
[
{
height: 648038,
tx_hash: '5fd1f016ffad963a31d7fc11a86c2949090ed8169994e761a2fad597e9b8f8a7',
tx_pos: 0,
value: 10000
},
{
height: 648038,
tx_hash: '5fd1f016ffad963a31d7fc11a86c2949090ed8169994e761a2fad597e9b8f8a7',
tx_pos: 1,
value: 264829
},
]
Then, after sending a tx that sends one output of 10k, and one output of the remaining balance back to the same address, this is the listunspent result:
[
{
height: 648038,
tx_hash: '5fd1f016ffad963a31d7fc11a86c2949090ed8169994e761a2fad597e9b8f8a7',
tx_pos: 0,
value: 10000
},
{
height: 648038,
tx_hash: '5fd1f016ffad963a31d7fc11a86c2949090ed8169994e761a2fad597e9b8f8a7',
tx_pos: 1,
value: 264829
},
]
[
{
height: 648038,
tx_hash: '5fd1f016ffad963a31d7fc11a86c2949090ed8169994e761a2fad597e9b8f8a7',
tx_pos: 0,
value: 10000
},
{
height: 648038,
tx_hash: '5fd1f016ffad963a31d7fc11a86c2949090ed8169994e761a2fad597e9b8f8a7',
tx_pos: 1,
value: 264829
},
{
height: 0,
tx_hash: '5e4bacac56527b3438e8eb4f68c54f83eaa39ac64ef6b57e98d814feed72f2d7',
tx_pos: 0,
value: 10000
},
{
height: 0,
tx_hash: '5e4bacac56527b3438e8eb4f68c54f83eaa39ac64ef6b57e98d814feed72f2d7',
tx_pos: 1,
value: 264336
}
]
So this includes the old unconfirmed spent outputs and the new unconfirmed unspent outputs. But the electrum docs at https://electrum-cash-protocol.readthedocs.io/en/latest/protocol-methods.html say the following:
A list of unspent outputs in blockchain order. This function takes the mempool into account. Mempool transactions paying to the address are included at the end of the list in an undefined order. Any output that is spent in the mempool does not appear.
So given that documentation I would expect the old outputs to disappear from the result, ad only return the new ones. Any idea what's up here?
Fulcrum crashes with sigsegv every few minutes
Recently my Fulcrum server started to crash regularly (several times per hour). It only happens for BTC, so either something in the chain, or a deliberate DOS attack against BTC servers. The logs show nothing unsual. The most recent crash caused the database to go corrupt, so I guess now I'm back to resync for the next day.
Apr 11 07:18:38 Fulcrum[3873906]: [2021-04-11 07:18:38.845] <TcpSrv 0.0.0.0:50099> New TCP Client.56026 5.121.xxx.xxx:18059,
17 clients total
Apr 11 07:18:38 Fulcrum[3873906]: [2021-04-11 07:18:38.849] <TcpSrv 0.0.0.0:50099> New TCP Client.56027 27.71.xxx.xxx:38319,
18 clients total
Apr 11 07:18:38 systemd[1]: fulcrum-btc.service: Main process exited, code=killed, status=11/SEGV
Apr 11 07:18:38 systemd[1]: fulcrum-btc.service: Failed with result 'signal'.
Apr 11 07:18:39 systemd[1]: fulcrum-btc.service: Scheduled restart job, restart counter is at 89.
Apr 11 07:18:39 systemd[1]: Stopped Fulcrum Bitcoin Daemon.
Apr 11 07:18:39 systemd[1]: Started Fulcrum Bitcoin Daemon.
...
Apr 11 07:19:22 Fulcrum[3878612]: [2021-04-11 07:19:22.471] <WsSrv 0.0.0.0:50003> WebSocket handshake failed for 116.68.xxx.xxx:44368, reason: Connection lost
Apr 11 07:19:22 Fulcrum[3878612]: [2021-04-11 07:19:22.471] <WsSrv 0.0.0.0:50003> WebSocket handshake failed for 116.68.xxx.xxx:44368, reason: Bad HTTP: {"jsonrpc":"2.0","id":0,"method":"serve…arams":[]}
Apr 11 07:19:22 fulcrum-btc.service: Main process exited, code=killed, status=11/SEGV
Apr 11 07:19:22 fulcrum-btc.service: Failed with result 'signal'.
Apr 11 07:19:22 systemd[1]: fulcrum-btc.service: Scheduled restart job, restart counter is at 90.
Apr 11 07:19:22 systemd[1]: Stopped Fulcrum Bitcoin Daemon.
Apr 11 07:19:22 systemd[1]: Started Fulcrum Bitcoin Daemon.
...
Apr 11 19:56:06 systemd[1]: fulcrum-btc.service: Main process exited, code=killed, status=11/SEGV
Apr 11 19:56:06 systemd[1]: fulcrum-btc.service: Failed with result 'signal'.
Apr 11 19:56:07 systemd[1]: fulcrum-btc.service: Scheduled restart job, restart counter is at 109.
Apr 11 19:56:07 systemd[1]: Stopped Fulcrum Bitcoin Daemon.
Apr 11 19:56:07 systemd[1]: Started Fulcrum Bitcoin Daemon.
...
Apr 11 19:56:07 Fulcrum[4047419]: [2021-04-11 19:56:07.490] DB memory: 420.00 MiB
Apr 11 19:56:07 Fulcrum[4047419]: [2021-04-11 19:56:07.490] Coin: BTC
Apr 11 19:56:07 Fulcrum[4047419]: [2021-04-11 19:56:07.490] Chain: main
Apr 11 19:56:07 Fulcrum[4047419]: [2021-04-11 19:56:07.511] FATAL: Caught exception: It appears that Fulcrum was forcefully killed in the middle of committng a block to the db. We cannot figure out where exactly in the update process Fulcrum was killed, so we cannot undo the inconsistent state caused by the unexpected shutdown. Sorry!
Apr 11 19:56:07 Fulcrum[4047419]: The database has been corrupted. Please delete the datadir and resynch to bitcoind.
Apr 11 19:56:07 Fulcrum[4047419]: [2021-04-11 19:56:07.512] Stopping Controller ...
Apr 11 19:56:07 Fulcrum[4047419]: [2021-04-11 19:56:07.512] Closing storage ...
Apr 11 19:56:07 Fulcrum[4047419]: [2021-04-11 19:56:07.555] Shutdown complete
Apr 11 19:56:07 systemd[1]: fulcrum-btc.service: Main process exited, code=exited, status=1/FAILURE
Unify admin port in example config files
fulcrum-example-config.conf suggest admin port 9000 while in fulcrum-quick-config.conf is set to 8000
Documentation of FulcrumAdmin uses port 8000
So I suggest using also port 8000 in the config file fulcrum-example-config.conf
Updated logo
I made this logo for you.
Version 1
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 124 124">
<defs/>
<defs>
<linearGradient id="a" x1="50%" x2="50%" y1="0%" y2="100%">
<stop offset="0%" stop-color="#2C3463"/>
<stop offset="100%" stop-color="#1F2547"/>
</linearGradient>
<linearGradient id="b" x1="50%" x2="50%" y1="0%" y2="100%">
<stop offset="0%" stop-color="#323B71"/>
<stop offset="100%" stop-color="#252C55"/>
</linearGradient>
</defs>
<g fill="none" fill-rule="evenodd">
<rect width="124" height="124" fill="url(#a)" rx="20"/>
<rect width="124" height="124" fill="url(#b)" rx="20"/>
<path fill="#E9432A" d="M62 95L51.579 74.158h20.842z"/>
<path fill="#FFF" d="M50.709 72.421L40.288 51.579h20.843zM39.421 49.842L29 29h20.842zM62 49.842L51.579 29h20.842zM84.579 49.842L74.158 29H95zM73.291 72.421L62.869 51.579h20.843z"/>
</g>
</svg>
Version 2
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 0 124 124">
<defs/>
<defs>
<linearGradient id="b" x1="50%" x2="50%" y1="0%" y2="100%">
<stop offset="0%" stop-color="#2C3463"/>
<stop offset="100%" stop-color="#1F2547"/>
</linearGradient>
<linearGradient id="c" x1="50%" x2="50%" y1="0%" y2="100%">
<stop offset="0%" stop-color="#323B71"/>
<stop offset="100%" stop-color="#252C55"/>
</linearGradient>
<linearGradient id="e" x1="50%" x2="50%" y1="0%" y2="100%">
<stop offset="0%" stop-color="#FFF" stop-opacity=".252"/>
<stop offset="100%" stop-color="#FFF" stop-opacity=".087"/>
</linearGradient>
<rect id="a" width="124" height="124" x="0" y="0" rx="20"/>
</defs>
<g fill="none" fill-rule="evenodd">
<mask id="d" fill="#fff">
<use xlink:href="#a"/>
</mask>
<use fill="url(#b)" xlink:href="#a"/>
<rect width="124" height="124" fill="url(#c)" mask="url(#d)" rx="20"/>
<rect width="124" height="124" y="-5" fill="url(#e)" mask="url(#d)" rx="20"/>
<g mask="url(#d)">
<path fill="#E9432A" d="M62 95L51.579 74.158h20.842z"/>
<path fill="#FFF" d="M50.709 72.421L40.288 51.579h20.843zM39.421 49.842L29 29h20.842zM62 49.842L51.579 29h20.842zM84.579 49.842L74.158 29H95zM73.291 72.421L62.869 51.579h20.843z"/>
</g>
</g>
</svg>
Limits / Pagination for get_history
As discussed in BitcoinUnlimited/ElectrsCash#91 the get_history is an expensive call that would be useful to apply limits to.
I propose adding an optional from_height so that wallets can keep track of which blockheight they are fully up to date on and not request old or duplicate information.
Question about using Electrum Server
This is my first attempt at using an Electrum Server with a Bitcoin core node. My question is if I previously used my wallets with the node should I go and delete the wallet.dat files from the node? From what I understand connecting my wallet (Sparrow) to the Fulcrum server will prevent anyone from seeing what wallets I'm using. But will having those old wallet.dat files on the node remove privacy?
Dojo cannot connect to fulcrum
Dojo cannot connect to local indexer which is a fulcrum server that I installed before. Someone can help¿? Is something that I have to change in fulcrum config file to let dojo in?
zmq: unknown endpoint protocol ipc
I'm using ipc for rawblock and rawtx zmq notifications with bitcoind. It looks like Fulcrum doesn't recognize it:
[2022-04-11 16:24:58.373] <BitcoinDMgr> getzmqnotifications: unknown endpoint protocol ipc:///tmp/bitcoind.block.raw for type rawblock
[2022-04-11 16:24:58.374] <BitcoinDMgr> getzmqnotifications: unknown endpoint protocol ipc:///tmp/bitcoind.tx.raw for type rawtx
[2022-04-11 16:24:58.374] <BitcoinDMgr> getzmqnotifications: query to bitcoind returned a result in an unexpected format
Will Fulcrum be getting ipc support for zmq notifications? Or do I need to switch to tcp?
Do not fallback to TLS v1.1 or TLS v1.0 [SSL & WSS]
IETF has already deprecated all SSL protocols, TLS 1.0, and TLS 1.1
Fulcrum still supports TLS v1.1 and TLS v1.0 - all modern browsers are supporting TLS v1.2+
Changing the following line in websocket.cpp & Servers.cpp from:
socket->setProtocol(QSsl::SslProtocol::AnyProtocol);
to:
socket->setProtocol(QSsl::SslProtocol::TlsV1_2OrLater);
should work (untested)
TLS supported versions test:
Try https://bch.loping.net:50002 or https://bch.loping.net:50004 in https://www.cdn77.com/tls-test
Initial synch performance
When building the initial index using Fulcrum on a RPi (connected to a 1Tb SSD), my expectation was that the process would be entirely CPU bound parsing blocks. This is the case for single-threaded ElectrumX, but my hope was that Fulcrum, using a non-interpreted language and multiple threads, would be able to significantly increase the speed of this process. ElectrumX takes almost 2 months to build the initial index on such limited hardware, and frequently fails with memory allocation errors.
This expectation has been borne out to a great extent - 70% indexing after 40 hours is much better progress. That said, processing has now dropped (at times) to 26 blocks/min around height 513000, presumably as the result of larger blocks. At this rate it will take another 4-8 days to finish the index. This can be compared to some extent against Electrs, which builds its (admittedly more limited) index within 24 hours on the same hardware.
I have been trying to determine the bottleneck - it appears that Fulcrum is neither CPU or IO bound, generally running at a quarter CPU available and modest IO. This leads me to believe that it is bitcoind's RPC interface that is the cause, specifically bitcoind's global locks that are noted in the Fulcrum configuration file for the bitcoind_clients parameter. Note that in the case of my setup, I have bitcoind running on another (similarly low power) machine on the local network. I've also tried running Fulcrum on a Linux desktop machine against the same bitcoind node, and have noted even lower CPU usage, which seems to support this.
I believe the reason that Electrs is able to achieve such fast processing is it's use of P2P interface to download blocks (see implementation here). No locks are obtained, and the IP Fulcrum is running on can be whitelisted in bitcoind so there are no restrictions in total bandwidth. This effectively mimics the IBD for bitcoind, but against a single peer. Of course, it's possible to use the P2P interface for more than the initial indexing as well.
My hope is that if initial indexing on Fulcrum can be brought to a comparable speed to Electrs, it can replace Electrs on popular prebuilt node packages which generally target the RPi4. Electrs faces significant challenges loading deep wallets due to the data architecture design, which I have described here: https://www.sparrowwallet.com/docs/server-performance.html. Barring a significant architectural change to that project, I believe Fulcrum could be a better alternative as it can perform the address-to-txid+height lookup internally. This may be the case already, but having a similar indexing speed would make the comparison even clearer.
When using very large fast-sync value: Out of memory during initial sync and now needs a full resync
What is the correct way to calculate how much memory be dedicated to fast-sync?
The database is now likely in an inconsistent state. To recover, you will need to delete the datadir and do a full resynch. Sorry!
Yikes!
I have a machine with 16 GB RAM running ubuntu jammy and I had selected
fast-sync = 5000
in my fulcrum.conf.
This is the state of the memory and swap after the crash.
$ free
total used free shared buff/cache available
Mem: 16260064 693200 9750516 18700 5816348 15169084
Swap: 4194300 368368 3825932
Logs follow.
May 16 11:46:44 chromebox Fulcrum[40476]: [2022-05-16 11:46:44.715] <Controller> Processed height: 572000, 77.7%, 6.78 blocks/sec, 17676.0 txs/sec, 53343.4 addrs/sec
May 16 11:48:49 chromebox Fulcrum[40476]: [2022-05-16 11:48:49.725] <Controller> Processed height: 573000, 77.8%, 8.00 blocks/sec, 19979.3 txs/sec, 61808.0 addrs/sec
May 16 11:51:01 chromebox Fulcrum[40476]: [2022-05-16 11:51:01.443] <Controller> Processed height: 574000, 77.9%, 7.59 blocks/sec, 18816.8 txs/sec, 56846.0 addrs/sec
May 16 11:53:07 chromebox Fulcrum[40476]: [2022-05-16 11:53:07.508] <Controller> Processed height: 575000, 78.1%, 7.93 blocks/sec, 20139.2 txs/sec, 58876.8 addrs/sec
May 16 11:55:37 chromebox Fulcrum[40476]: [2022-05-16 11:55:37.656] <Controller> Processed height: 576000, 78.2%, 6.66 blocks/sec, 16321.4 txs/sec, 55649.1 addrs/sec
May 16 11:56:24 chromebox Fulcrum[40476]: [2022-05-16 11:56:24.194] <Controller> Storage UTXO Cache: Flushing to DB ...
May 16 11:58:41 chromebox Fulcrum[40476]: [2022-05-16 11:58:41.197] <BitcoinD.1> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 11:58:41 chromebox Fulcrum[40476]: [2022-05-16 11:58:41.198] <Task.DL 548688 -> 736614> 658887: FAIL: bitcoind connection lost
May 16 11:59:31 chromebox Fulcrum[40476]: [2022-05-16 11:59:31.175] <Controller> Task errored: Task.DL 548688 -> 736614, error: bitcoind connection lost
May 16 11:59:32 chromebox Fulcrum[40476]: [2022-05-16 11:59:32.217] <Controller> Failed to synch blocks and/or mempool
May 16 11:59:32 chromebox Fulcrum[40476]: [2022-05-16 11:59:32.217] <Controller> Initial sync ended, flushing and deleting UTXO Cache ...
May 16 11:59:32 chromebox Fulcrum[40476]: [2022-05-16 11:59:32.217] <Controller> Storage UTXO Cache: Flushing to DB ...
May 16 12:01:22 chromebox Fulcrum[40476]: [2022-05-16 12:01:22.677] <Controller> Block height 736618, downloading new blocks ...
May 16 12:01:22 chromebox Fulcrum[40476]: [2022-05-16 12:01:22.700] <Controller> fast-sync: Enabled; UTXO cache size set to 5000000000 bytes (available physical RAM: 12915953664 bytes)
May 16 12:02:20 chromebox Fulcrum[40476]: [2022-05-16 12:02:20.004] <Controller> Processed height: 577000, 78.3%, 1.24 blocks/sec, 3428.3 txs/sec, 11279.9 addrs/sec
May 16 12:05:50 chromebox Fulcrum[40476]: [2022-05-16 12:05:50.774] <Controller> Processed height: 578000, 78.5%, 4.74 blocks/sec, 11356.6 txs/sec, 37047.3 addrs/sec
May 16 12:09:04 chromebox Fulcrum[40476]: [2022-05-16 12:09:04.563] <Controller> Processed height: 579000, 78.6%, 5.16 blocks/sec, 12189.2 txs/sec, 43140.5 addrs/sec
May 16 12:11:50 chromebox Fulcrum[40476]: [2022-05-16 12:11:50.697] <Controller> Processed height: 580000, 78.7%, 6.02 blocks/sec, 13946.7 txs/sec, 48521.0 addrs/sec
May 16 12:14:46 chromebox Fulcrum[40476]: [2022-05-16 12:14:46.175] <Controller> Processed height: 581000, 78.9%, 5.70 blocks/sec, 13131.9 txs/sec, 49405.7 addrs/sec
May 16 12:18:16 chromebox Fulcrum[40476]: [2022-05-16 12:18:16.052] <Controller> Processed height: 582000, 79.0%, 4.76 blocks/sec, 11332.8 txs/sec, 41584.5 addrs/sec
May 16 12:22:01 chromebox Fulcrum[40476]: [2022-05-16 12:22:01.031] <Controller> Processed height: 583000, 79.1%, 4.44 blocks/sec, 10382.9 txs/sec, 38208.3 addrs/sec
May 16 12:25:09 chromebox Fulcrum[40476]: [2022-05-16 12:25:09.660] <Controller> Processed height: 584000, 79.3%, 5.30 blocks/sec, 11878.4 txs/sec, 41079.0 addrs/sec
May 16 12:28:05 chromebox Fulcrum[40476]: [2022-05-16 12:28:05.615] <Controller> Processed height: 585000, 79.4%, 5.68 blocks/sec, 12563.3 txs/sec, 42346.6 addrs/sec
May 16 12:31:59 chromebox Fulcrum[40476]: [2022-05-16 12:31:59.938] <Controller> Processed height: 586000, 79.6%, 4.27 blocks/sec, 9796.1 txs/sec, 33338.8 addrs/sec
May 16 12:35:04 chromebox Fulcrum[40476]: [2022-05-16 12:35:04.353] <Controller> Processed height: 587000, 79.7%, 5.42 blocks/sec, 11442.7 txs/sec, 39589.0 addrs/sec
May 16 12:37:52 chromebox Fulcrum[40476]: [2022-05-16 12:37:52.686] <Controller> Processed height: 588000, 79.8%, 5.94 blocks/sec, 11601.0 txs/sec, 38190.6 addrs/sec
May 16 12:40:01 chromebox Fulcrum[40476]: [2022-05-16 12:39:58.134] <BitcoinD.1> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:40:01 chromebox Fulcrum[40476]: [2022-05-16 12:39:58.098] <BitcoinD.3> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:40:01 chromebox Fulcrum[40476]: [2022-05-16 12:39:58.134] <BitcoinD.2> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:41:40 chromebox Fulcrum[40476]: [2022-05-16 12:41:37.715] <BitcoinD.1> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:42:20 chromebox Fulcrum[40476]: [2022-05-16 12:41:37.316] <BitcoinD.2> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:42:20 chromebox Fulcrum[40476]: [2022-05-16 12:41:38.674] <Task.DL 576931 -> 736618> 698589: FAIL: bitcoind connection lost
May 16 12:42:20 chromebox Fulcrum[40476]: [2022-05-16 12:41:42.397] <Task.DL 548687 -> 736614> 698595: FAIL: bitcoind connection lost
May 16 12:42:20 chromebox Fulcrum[40476]: [2022-05-16 12:41:45.722] <BitcoinD.3> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:42:20 chromebox Fulcrum[40476]: [2022-05-16 12:41:48.338] <Task.DL 576931 -> 736618> 698591: FAIL: bitcoind connection lost
May 16 12:42:20 chromebox Fulcrum[40476]: [2022-05-16 12:41:58.281] <Task.DL 576931 -> 736618> 698585: FAIL: bitcoind connection lost
May 16 12:42:20 chromebox Fulcrum[40476]: [2022-05-16 12:41:55.140] <Task.DL 548687 -> 736614> 698576: FAIL: bitcoind connection lost
May 16 12:42:20 chromebox Fulcrum[40476]: [2022-05-16 12:42:04.464] <Task.DL 576931 -> 736618> 698582: FAIL: bitcoind connection lost
May 16 12:42:20 chromebox Fulcrum[40476]: [2022-05-16 12:42:08.555] <Task.DL 548687 -> 736614> 698583: FAIL: bitcoind request timed out
May 16 12:42:20 chromebox Fulcrum[40476]: [2022-05-16 12:42:12.581] <BitcoinD.2> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:42:20 chromebox Fulcrum[40476]: [2022-05-16 12:42:12.581] <BitcoinD.1> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:42:20 chromebox Fulcrum[40476]: [2022-05-16 12:42:14.066] <Task.DL 548687 -> 736614> 698593: FAIL: bitcoind request timed out
May 16 12:42:20 chromebox Fulcrum[40476]: [2022-05-16 12:42:17.703] <BitcoinD.3> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:42:20 chromebox Fulcrum[40476]: [2022-05-16 12:42:17.706] <BitcoinD.2> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:42:20 chromebox Fulcrum[40476]: [2022-05-16 12:42:17.706] <BitcoinD.1> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:42:37 chromebox Fulcrum[40476]: [2022-05-16 12:42:19.784] <Task.DL 576930 -> 736618> 698586: FAIL: bitcoind connection lost
May 16 12:42:37 chromebox Fulcrum[40476]: [2022-05-16 12:42:23.256] <BitcoinD.1> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:42:37 chromebox Fulcrum[40476]: [2022-05-16 12:42:23.256] <BitcoinD.2> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:42:37 chromebox Fulcrum[40476]: [2022-05-16 12:42:23.681] <Task.DL 576930 -> 736618> 698573: FAIL: bitcoind connection lost
May 16 12:42:37 chromebox Fulcrum[40476]: [2022-05-16 12:42:26.601] <BitcoinD.3> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:42:37 chromebox Fulcrum[40476]: [2022-05-16 12:42:34.264] <Task.DL 576930 -> 736618> 698590: FAIL: bitcoind request timed out
May 16 12:42:37 chromebox Fulcrum[40476]: [2022-05-16 12:42:35.097] <BitcoinD.2> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:42:37 chromebox Fulcrum[40476]: [2022-05-16 12:42:35.098] <BitcoinD.3> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:42:37 chromebox Fulcrum[40476]: [2022-05-16 12:42:35.097] <BitcoinD.1> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:42:37 chromebox Fulcrum[40476]: [2022-05-16 12:42:35.093] <Task.DL 576932 -> 736618> 698592: FAIL: bitcoind connection lost
May 16 12:42:37 chromebox Fulcrum[40476]: [2022-05-16 12:42:35.093] <Task.DL 548689 -> 736614> 698584: FAIL: bitcoind connection lost
May 16 12:42:37 chromebox Fulcrum[40476]: [2022-05-16 12:42:35.093] <Task.DL 576930 -> 736618> 698607: FAIL: Unable to find a good BitcoinD connection
May 16 12:42:37 chromebox Fulcrum[40476]: [2022-05-16 12:42:35.103] <Task.DL 576932 -> 736618> 698588: FAIL: bitcoind connection lost
May 16 12:42:37 chromebox Fulcrum[40476]: [2022-05-16 12:42:35.103] <Task.DL 576932 -> 736618> 698594: FAIL: bitcoind connection lost
May 16 12:42:37 chromebox Fulcrum[40476]: [2022-05-16 12:42:35.104] <Task.DL 576932 -> 736618> 698614: FAIL: Unable to find a good BitcoinD connection
May 16 12:42:37 chromebox Fulcrum[40476]: [2022-05-16 12:42:35.103] <Task.DL 548689 -> 736614> 698579: FAIL: bitcoind connection lost
May 16 12:42:37 chromebox Fulcrum[40476]: [2022-05-16 12:42:35.105] <Task.DL 548689 -> 736614> 698587: FAIL: bitcoind request timed out
May 16 12:42:37 chromebox Fulcrum[40476]: [2022-05-16 12:42:35.105] <Task.DL 548689 -> 736614> 698615: FAIL: Unable to find a good BitcoinD connection
May 16 12:44:48 chromebox Fulcrum[40476]: [2022-05-16 12:44:42.136] <BitcoinD.2> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:45:15 chromebox Fulcrum[40476]: [2022-05-16 12:44:42.179] <BitcoinD.3> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:45:15 chromebox Fulcrum[40476]: [2022-05-16 12:45:08.799] <BitcoinD.3> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:45:15 chromebox Fulcrum[40476]: [2022-05-16 12:45:10.064] <BitcoinD.1> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:45:15 chromebox Fulcrum[40476]: [2022-05-16 12:45:10.200] <BitcoinD.2> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:46:33 chromebox Fulcrum[40476]: [2022-05-16 12:45:26.071] <BitcoinD.2> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:46:33 chromebox Fulcrum[40476]: [2022-05-16 12:45:25.650] <BitcoinD.3> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:46:33 chromebox Fulcrum[40476]: [2022-05-16 12:45:25.570] <BitcoinD.1> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:46:33 chromebox Fulcrum[40476]: [2022-05-16 12:45:44.234] <BitcoinD.2> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:46:33 chromebox Fulcrum[40476]: [2022-05-16 12:45:44.234] <BitcoinD.3> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:46:33 chromebox Fulcrum[40476]: [2022-05-16 12:45:44.234] <BitcoinD.1> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:46:33 chromebox Fulcrum[40476]: [2022-05-16 12:46:00.392] <BitcoinD.1> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:46:33 chromebox Fulcrum[40476]: [2022-05-16 12:46:00.392] <BitcoinD.2> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:46:33 chromebox Fulcrum[40476]: [2022-05-16 12:46:00.392] <BitcoinD.3> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:46:33 chromebox Fulcrum[40476]: [2022-05-16 12:46:21.347] <BitcoinD.1> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:46:33 chromebox Fulcrum[40476]: [2022-05-16 12:46:21.347] <BitcoinD.3> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:46:33 chromebox Fulcrum[40476]: [2022-05-16 12:46:21.347] <BitcoinD.2> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:47:51 chromebox Fulcrum[40476]: [2022-05-16 12:46:44.477] <BitcoinD.3> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:47:51 chromebox Fulcrum[40476]: [2022-05-16 12:46:44.668] <BitcoinD.1> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:47:51 chromebox Fulcrum[40476]: [2022-05-16 12:46:44.557] <BitcoinD.2> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:47:51 chromebox Fulcrum[40476]: [2022-05-16 12:47:16.298] <BitcoinD.2> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:47:51 chromebox Fulcrum[40476]: [2022-05-16 12:47:16.127] <BitcoinD.1> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:47:51 chromebox Fulcrum[40476]: [2022-05-16 12:47:16.298] <BitcoinD.3> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:47:51 chromebox Fulcrum[40476]: [2022-05-16 12:47:36.388] <BitcoinD.2> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:47:51 chromebox Fulcrum[40476]: [2022-05-16 12:47:35.955] <BitcoinD.3> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:47:51 chromebox Fulcrum[40476]: [2022-05-16 12:47:35.956] <BitcoinD.1> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:47:51 chromebox Fulcrum[40476]: [2022-05-16 12:47:50.523] <BitcoinD.3> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:47:51 chromebox Fulcrum[40476]: [2022-05-16 12:47:50.523] <BitcoinD.1> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:47:51 chromebox Fulcrum[40476]: [2022-05-16 12:47:50.523] <BitcoinD.2> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:49:49 chromebox Fulcrum[40476]: [2022-05-16 12:49:49.067] <Controller> Task errored: Task.DL 576931 -> 736618, error: bitcoind connection lost
May 16 12:49:49 chromebox Fulcrum[40476]: [2022-05-16 12:49:49.087] <Controller> Failed to synch blocks and/or mempool
May 16 12:49:49 chromebox Fulcrum[40476]: [2022-05-16 12:49:49.088] <Controller> Initial sync ended, flushing and deleting UTXO Cache ...
May 16 12:49:49 chromebox Fulcrum[40476]: [2022-05-16 12:49:49.088] <Controller> Storage UTXO Cache: Flushing to DB ...
May 16 12:50:29 chromebox Fulcrum[40476]: [2022-05-16 12:50:23.075] <BitcoinD.2> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:50:56 chromebox Fulcrum[40476]: [2022-05-16 12:50:23.159] <BitcoinD.1> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:50:56 chromebox Fulcrum[40476]: [2022-05-16 12:50:23.247] <BitcoinD.3> Lost connection to bitcoind, will retry every 5 seconds ...
May 16 12:50:55 chromebox systemd[1]: fulcrum.service: A process of this unit has been killed by the OOM killer.
May 16 12:50:57 chromebox systemd[1]: fulcrum.service: Main process exited, code=killed, status=9/KILL
May 16 12:50:57 chromebox systemd[1]: fulcrum.service: Failed with result 'oom-kill'.
May 16 12:50:57 chromebox systemd[1]: fulcrum.service: Consumed 13h 53min 54.250s CPU time.
^C
$ ps -ef |grep fulcr
cloudge+ 79958 42821 0 15:17 pts/0 00:00:00 grep --color=auto fulcr
$ free
total used free shared buff/cache available
Mem: 16260064 693200 9750516 18700 5816348 15169084
Swap: 4194300 368368 3825932
$ sudo systemctl start fulcrum
$ journalctl -fu fulcrum
May 16 15:18:14 chromebox Fulcrum[80083]: [2022-05-16 15:18:14.937] active implementation: haswell
May 16 15:18:14 chromebox Fulcrum[80083]: [2022-05-16 15:18:14.939] jemalloc: version 5.2.1-0-gea6b3e9
May 16 15:18:14 chromebox Fulcrum[80083]: [2022-05-16 15:18:14.939] Qt: version 5.15.2
May 16 15:18:14 chromebox Fulcrum[80083]: [2022-05-16 15:18:14.940] rocksdb: version 6.14.6-ed43161
May 16 15:18:14 chromebox Fulcrum[80083]: [2022-05-16 15:18:14.940] simdjson: version 0.6.0
May 16 15:18:14 chromebox Fulcrum[80083]: [2022-05-16 15:18:14.940] ssl: OpenSSL 3.0.2 15 Mar 2022
May 16 15:18:14 chromebox Fulcrum[80083]: [2022-05-16 15:18:14.940] zmq: libzmq version: 4.3.3, cppzmq version: 4.7.1
May 16 15:18:14 chromebox Fulcrum[80083]: [2022-05-16 15:18:14.940] Fulcrum 1.6.0 (Release 5e95c0f) - Mon May 16, 2022 15:18:14.940 UTC - starting up ...
May 16 15:18:14 chromebox Fulcrum[80083]: [2022-05-16 15:18:14.940] Max open files: 32767 (increased from default: 20000)
May 16 15:18:14 chromebox Fulcrum[80083]: [2022-05-16 15:18:14.942] Loading database ...
May 16 15:18:23 chromebox Fulcrum[80083]: [2022-05-16 15:18:23.931] DB memory: 512.00 MiB
May 16 15:18:23 chromebox Fulcrum[80083]: [2022-05-16 15:18:23.931] Coin: BTC
May 16 15:18:23 chromebox Fulcrum[80083]: [2022-05-16 15:18:23.931] Chain: main
May 16 15:18:23 chromebox Fulcrum[80083]: [2022-05-16 15:18:23.946] Verifying headers ...
May 16 15:18:26 chromebox Fulcrum[80083]: [2022-05-16 15:18:26.993] Initializing header merkle cache ...
May 16 15:18:28 chromebox Fulcrum[80083]: [2022-05-16 15:18:28.550] Checking tx counts ...
May 16 15:18:33 chromebox Fulcrum[80083]: [2022-05-16 15:18:33.040] 442052400 total transactions
May 16 15:18:33 chromebox Fulcrum[80083]: [2022-05-16 15:18:33.313] UTXO set: 59652808 utxos, 5010.836 MB
May 16 15:18:33 chromebox Fulcrum[80083]: [2022-05-16 15:18:33.313] BitcoinDMgr: starting 3 bitcoin RPC clients ...
May 16 15:18:33 chromebox Fulcrum[80083]: [2022-05-16 15:18:33.313] BitcoinDMgr: started ok
May 16 15:18:33 chromebox Fulcrum[80083]: [2022-05-16 15:18:33.423] <Controller> Block height 736635, downloading new blocks ...
May 16 15:18:33 chromebox Fulcrum[80083]: [2022-05-16 15:18:33.423] <Controller> fast-sync: Enabled; UTXO cache size set to 5000000000 bytes (available physical RAM: 15456243712 bytes)
May 16 15:18:33 chromebox Fulcrum[80083]: [2022-05-16 15:18:33.919] <Controller> FATAL: Failed to spend: 65f9ff4d8ac3d62e2bae53c14671402473310e48cabb3e676455aa365b906d8d:1 (spending txid: ae8d143159b0f278e93138b6f08059ee1cbca27915d2b74441cdf8c644465cfb)
May 16 15:18:33 chromebox Fulcrum[80083]: The database is now likely in an inconsistent state. To recover, you will need to delete the datadir and do a full resynch. Sorry!
Can't use hostname in tor_proxy
It's not possible to define tor_proxy with a hostname instead of an IP. This is especially needed for Docker, when you define a tor service and don't know it's IP.
Deprecated/removed functions are available outside negotiated version scope.
When I connect to 1.2.5 and 1.2.6 fulcum servers and negotiate protocol version 1.4.1, the blockchain.address.subscribe method should not be accessible and I expect to get:
message: 'unknown method "blockchain.address.subscribe"'
.. as the blockchain.address.* methods were removed in 1.3 and only re-introduced in 1.4.3.
Instead the methods seems to function as if it ignored the version negotiation entirely.
New release?
It seems that Fulcrum has some nice unreleased features that are only available from master (fast-sync option, etc.).
Maybe it's time to publish new release?
bchd support
From issue #27, we are creating a new issue since this arose:
@infertux wrote:
Alright the EC certificate works fine now but I'm running into another issue. Have you tried to point Fulcrum at bchd instead of bitcoind? It looks like bchd is more strict about checking HTTP headers and refusing the request sent by Fulcrum because it's missing the Host header:
<BitcoinD.1> 400 (header): HTTP/1.1 400 Bad Request: missing required Host header
Indeed if I MITM with netcat I cannot see any Host header:
$ nc -vlp 8334
listening on [any] 8334 ...
connect to [127.0.0.1] from localhost [127.0.0.1] 42256
POST / HTTP/1.1
Content-Type: application/json-rpc
Authorization: Basic blah==
Content-Length: 38
{"id":5,"method":"ping","params":[]}Sending the request with curl works just fine (notice the extra headers):
$ curl http://127.0.0.1:8334/ -vi -H "Authorization: Basic blah==" --data '{"id":5,"method":"ping","params":[]}'
> POST / HTTP/1.1
> Host: 127.0.0.1:8334
> User-Agent: curl/7.64.0
> Accept: */*
> Authorization: Basic blah==
> Content-Length: 36
> Content-Type: application/x-www-form-urlencoded
[...]
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Connection: close
Connection: close
< Content-Type: application/json
Content-Type: application/json
<
{"jsonrpc":"1.0","result":null,"error":null,"id":5}Originally posted by @infertux in #27 (comment)
FIXME: dsp addTx returned false for dspHash
Just saw some weird logs. Nothing appears to be broken.
<SynchMempool> addNewTxs: dsp addTx returned false for dspHash: 0077b5e7727615c07cd11a9f4ea2dc9189d8f1f775325dd6d27a2beb0a594c96, txid: 9a3cbbd258ab7006b0c095d105337497acdafc1fd572cf8a3a2143bf0275ae34. FIXME!
<SynchMempool> addNewTxs: dsp addTx returned false for dspHash: 97c5d6897c2cab2fefeb9d81d3edfc951f8f492bba960b25ec338d0e3e82846c, txid: 9a3cbbd258ab7006b0c095d105337497acdafc1fd572cf8a3a2143bf0275ae34. FIXME!
<SynchMempool> addNewTxs: dsp addTx returned false for dspHash: 4d4e481d37c37688d167e434226e09a47d68bb94db8fbafef1dfeba367493a3b, txid: 9a3cbbd258ab7006b0c095d105337497acdafc1fd572cf8a3a2143bf0275ae34. FIXME!
Fulcrum 1.5.2 (Release f340a65)
get_balance negative unconfirmed
Sometimes occours that blockchain.address.get_balance method returns a negative integer triggering an Exception followed by a disconnection on electrum side.
File "/usr/local/lib/python3.10/site-packages/electrum/interface.py", line 1003, in get_balance_for_scripthash
assert_non_negative_integer(res['unconfirmed'])
File "/usr/local/lib/python3.10/site-packages/electrum/interface.py", line 94, in assert_non_negative_integer
raise RequestCorrupted(f'{val!r} should be a non-negative integer')
electrum.interface.RequestCorrupted: -1091317 should be a non-negative integer
Maybe returning zero here when there's a negative integer can be a fix?
References:
Sending JSON-RPC 'error' before idle-disconnect causes some Electron Cash clients to raise a Python Exception
In this code here:
Line 1810 in 0aadd53
In the above linked code, we send a spurious error 'notification-like' message to clients right before dropping them on idle-disconnect. This causes existing Electron Cash instances to crash if they later parse the message and see no method field in the message.
Related issue in Electron Cash: Electron-Cash/Electron-Cash#1774
This is a bug in Fulcrum, however. Really we should just be dropping the clients with no spurious error message being sent to them.
FATAL: Cannot seek to write header
I'm trying to set up a new Fulcrum instance but it keeps crashing around block height 426k. I tried to remove /var/lib/fulcrum/datadir/ then sync again but that didn't help.
Fulcrum[24411]: [2020-09-10 15:49:39.361] <Controller> Processed height: 425000, 65.2%, 4.10 blocks/sec, 6282.7 txs/sec, 22033.2 addrs/sec
Fulcrum[24411]: [2020-09-10 15:53:20.802] <Controller> Processed height: 426000, 65.3%, 4.51 blocks/sec, 7058.6 txs/sec, 24676.1 addrs/sec
Fulcrum[24411]: [2020-09-10 15:54:54.765] <Controller> FATAL: Cannot seek to write header for /var/lib/fulcrum/datadir/txnum2txhash
Fulcrum[24411]: [2020-09-10 15:54:54.766] (Debug) void App::cleanup()
Fulcrum[24411]: [2020-09-10 15:54:54.767] Stopping Stats HTTP Servers ...
Any ideas?
WebSocket support for Firefox requests
Turns out Firefox starts WebSockets with an undefined HTTP version, which breaks the connection to Fulcrum:
GET / undefined
Host: bch.loping.net:50004
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:67.0) Gecko/20100101 Firefox/67.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Sec-WebSocket-Version: 13
Origin: http://localhost:3000
Sec-WebSocket-Extensions: permessage-deflate
Sec-WebSocket-Key: K716Hio2iYwR9C1xVpronw==
Connection: keep-alive, Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
See https://github.com/cculianu/Fulcrum/blob/master/src/WebSocket.cpp#L517 and https://gitlab.com/GeneralProtocols/electrum-cash/-/merge_requests/16
FATAL: Caught exception: It appears that Fulcrum was forcefully killed in the middle of committing a block...
Great SPV server! It is fast and just what I needed.
I've encountered a problem when restarting Fulcrum:
FATAL: Caught exception: It appears that Fulcrum was forcefully killed in the middle of committng a block to the db. We cannot figure out where exactly in the update process Fulcrum was killed, so we cannot undo the inconsistent state caused by the unexpected shutdown. Sorry!
I restarted the server (via systemd) and also created an image snapshot for the server and then this error appears.
Having used Electrum Cash for a while, I never had any problems with this before (for years) even thought I would restart/terminate instances abruptly.
Is there anyway to prevent this problem from happening or gracefully recover? Thank you!
Build fails on fedora-rawhide-x86_64
g++ -c -pipe -std=c++1z -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -DNDEBUG -fomit-frame-pointer -O2 -std=gnu++1z -w -D_REENTRANT -fPIC -DQT_DEPRECATED_WARNINGS -DUSE_QT_IN_BITCOIN -DHAVE_ENDIAN_H -DHAVE_DECL_HTOBE16 -DHAVE_DECL_HTOLE16 -DHAVE_DECL_BE16TOH -DHAVE_DECL_LE16TOH -DHAVE_DECL_HTOBE32 -DHAVE_DECL_HTOLE32 -DHAVE_DECL_BE32TOH -DHAVE_DECL_LE32TOH -DHAVE_DECL_HTOBE64 -DHAVE_DECL_HTOLE64 -DHAVE_DECL_BE64TOH -DHAVE_DECL_LE64TOH -DHAVE_JEMALLOC_HEADERS -DQT_NO_DEBUG -DQT_NETWORK_LIB -DQT_CORE_LIB -I. -Isrc -I/usr/include/qt5 -I/usr/include/qt5/QtNetwork -I/usr/include/qt5/QtCore -I. -I/usr/lib64/qt5/mkspecs/linux-g++ -o RecordFile.o src/RecordFile.cpp
In file included from src/RecordFile.cpp:19:
src/RecordFile.h:87:14: error: 'unique_lock' in namespace 'std' does not name a template type
87 | std::unique_lock<std::shared_mutex> lock;
| ^~~~~~~~~~~
src/RecordFile.h:32:1: note: 'std::unique_lock' is defined in header '<mutex>'; did you forget to '#include <mutex>'?
31 | #include <memory>
+++ |+#include <mutex>
32 |
src/RecordFile.cpp: In constructor 'RecordFile::BatchAppendContext::BatchAppendContext(RecordFile&)':
src/RecordFile.cpp:190:16: error: class 'RecordFile::BatchAppendContext' does not have any field named 'lock'
190 | : rf(rf_), lock(rf.rwlock)
| ^~~~
Feature request: `blockchain.transaction.subscribe` and `blockchain.transaction.unsubscribe`
Currently, in order to get notified when a transaction is confirmed, you have to subscribe to all changes to the recipient address, then look for changes to the specific transaction itself. This is problematic, because when done, if you unsubscribe from the address you might unintentionally unsubscribe from address changes you previously subscribed to elsewhere.
A proper solution is to add transaction subscribe/unsubscribe methods similar to the address subscriptions, that return the status of the transaction (some hash of the transaction metadata). This would be safe(r) to subscribe and unsubscribe to, and also easier to understand and would keep code cleaner in both design and legibility.
Fail to build on Fedora 35 rawhide
Fedora 35 is under development, so it might be a bug in the setup, or it can be an early warning that will happen in a newer toolchain. Builds on Fedora 34 and Fedora 33.
g++ -c -pipe -std=c++1z -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -DNDEBUG -fomit-frame-pointer -O2 -std=gnu++1z -w -D_REENTRANT -fPIC -DQT_DEPRECATED_WARNINGS -DUSE_QT_IN_BITCOIN -DHAVE_ENDIAN_H -DHAVE_DECL_HTOBE16 -DHAVE_DECL_HTOLE16 -DHAVE_DECL_BE16TOH -DHAVE_DECL_LE16TOH -DHAVE_DECL_HTOBE32 -DHAVE_DECL_HTOLE32 -DHAVE_DECL_BE32TOH -DHAVE_DECL_LE32TOH -DHAVE_DECL_HTOBE64 -DHAVE_DECL_HTOLE64 -DHAVE_DECL_BE64TOH -DHAVE_DECL_LE64TOH -DHAVE_DECL___BUILTIN_CLZL -DHAVE_DECL___BUILTIN_CLZLL -DGIT_COMMIT=\"\" -DENABLE_ZMQ -DHAVE_JEMALLOC_HEADERS -DQT_NO_DEBUG -DQT_NETWORK_LIB -DQT_CORE_LIB -I. -Isrc -I/usr/include/qt5 -I/usr/include/qt5/QtNetwork -I/usr/include/qt5/QtCore -I. -I/usr/lib64/qt5/mkspecs/linux-g++ -o Util.o src/Util.cpp
/bin/sh: line 1: git: command not found
In file included from /usr/include/pthread.h:33,
from /usr/include/c++/11/x86_64-redhat-linux/bits/gthr-default.h:35,
from /usr/include/c++/11/x86_64-redhat-linux/bits/gthr.h:148,
from /usr/include/c++/11/ext/atomicity.h:35,
from /usr/include/c++/11/bits/basic_string.h:39,
from /usr/include/c++/11/string:55,
from /usr/include/qt5/QtCore/qbytearray.h:52,
from /usr/include/qt5/QtCore/QByteArray:1,
from src/ByteView.h:21,
from src/Util.h:21,
from src/Mixins.h:21,
from src/App.h:21,
from src/Util.cpp:19:
src/Util.cpp:66:48: error: call to non-'constexpr' function 'long int __sysconf(int)'
66 | static constexpr unsigned PLATFORM_STACK_MIN = PTHREAD_STACK_MIN;
| ^~~~~~~~~~~~~~~~~
/usr/include/bits/pthread_stack_min-dynamic.h:24:17: note: 'long int __sysconf(int)' declared here
24 | extern long int __sysconf (int __name) __THROW;
| ^~~~~~~~~
make: *** [Makefile:1896: Util.o] Error 1
gcc-c++: 11.2.1
qt5-qtbase-devel: 5.15.2
glibc-devel: 2.34
Full build log: https://download.copr.fedorainfracloud.org/results/jonny/BitcoinCash/fedora-rawhide-x86_64/02404254-fulcrum/build.log.gz
blockchain.transaction.relayable / status?
I had a talk with shammah about how he uses electrum for stamp, and we identified a feature that might be good to have: checking if a transaction is relayable or not.
For the case not, it could be due to it being a double-spend, or not matching standardness, or too low fee etc.
The call would take a hex-encoded raw transaction, or an array of such, and then return information about the transactions validity / relayability. It would be particulary useful in places that works with off-line transactions where part of the check for validity is that it's unspent and valid according to the network itself.
Input: Array of raw transactions, and optional broadcast flag.
Output: Boolean for total outcome, array of txid:bool, and optional broadcast txid.
It might be possible to include this functionality as part of #32 (both as a standalone blockchain.transaction.status and as a subscription), rather than making a separate feature.
bitcoind connection lost during syncing
Below is a comment I received.
"I keep getting bitcoind connection lost during syncing. Is this normal, and should I continue to let this run? It will now only process a height once every hour or so. Has anyone else had this issue, and if so what do you do in this situation? This is my 5th attempt at installing Fulcrum. Any help would be greatly appreciated."
May 30 06:55:00 sovereign1 Fulcrum[1686]: [2022-05-30 06:55:00.771] <Controller> Task errored: Task.DL 361020 -> 738534, error: bitcoind connection lost
May 30 06:55:00 sovereign1 Fulcrum[1686]: [2022-05-30 06:55:00.772] <Controller> Failed to synch blocks and/or mempool
May 30 06:55:00 sovereign1 Fulcrum[1686]: [2022-05-30 06:55:00.772] <Controller> Initial sync ended, flushing and deleting UTXO Cache ...
May 30 06:55:00 sovereign1 Fulcrum[1686]: [2022-05-30 06:55:00.772] <Controller> Storage UTXO Cache: Flushing to DB ...
May 30 06:55:03 sovereign1 Fulcrum[1686]: [2022-05-30 06:55:03.926] <Controller> Block height 738537, downloading new blocks ...
May 30 06:55:03 sovereign1 Fulcrum[1686]: [2022-05-30 06:55:03.926] <Controller> fast-sync: Enabled; UTXO cache size set to 7000000000 bytes (available physical RAM: 12080082944 bytes)
May 30 07:22:31 sovereign1 Fulcrum[1686]: [2022-05-30 07:22:31.534] <BitcoinD.3> Lost connection to bitcoind, will retry every 5 seconds ...
May 30 07:22:32 sovereign1 Fulcrum[1686]: [2022-05-30 07:22:31.534] <Task.DL 361303 -> 738537> 730515: FAIL: bitcoind connection lost
May 30 07:22:32 sovereign1 Fulcrum[1686]: [2022-05-30 07:22:31.534] <Task.DL 361303 -> 738537> 730518: FAIL: bitcoind connection lost
May 30 07:22:32 sovereign1 Fulcrum[1686]: [2022-05-30 07:22:31.534] <BitcoinD.2> Lost connection to bitcoind, will retry every 5 seconds ...
May 30 07:22:32 sovereign1 Fulcrum[1686]: [2022-05-30 07:22:31.535] <Task.DL 361303 -> 738537> 730517: FAIL: bitcoind connection lost
May 30 07:22:36 sovereign1 Fulcrum[1686]: [2022-05-30 07:22:36.534] <BitcoinD.1> Lost connection to bitcoind, will retry every 5 seconds ...
May 30 07:22:36 sovereign1 Fulcrum[1686]: [2022-05-30 07:22:36.534] <Task.DL 361303 -> 738537> 730519: FAIL: bitcoind connection lost
Running tcp service on regtest
Hey Calin,
I am struggling to get the tcp service to run on regtest.
I have fulcrum connected to my bitcoind node and it is processing the regtest data (fulcrum_1 | [2022-04-22 19:40:58.811] <Controller> Processed 132 new blocks with 132 txs (132 inputs, 263 outputs, 132 addresses), verified ok.)
This is the output from fulcrum:
fulcrum git:(master) ✗ docker-compose -f docker-compose.regtest.yml up fulcrum
fulcrum_bitcoind_1 is up-to-date
Recreating fulcrum_fulcrum_1 ... done
Attaching to fulcrum_fulcrum_1
fulcrum_1 | Generating a RSA private key
fulcrum_1 | .........................................+++++
fulcrum_1 | .............................................................................+++++
fulcrum_1 | writing new private key to '/data/fulcrum.key'
fulcrum_1 | -----
fulcrum_1 | [2022-04-22 19:40:56.796] 'datadir' specified both via the CLI and the configuration file. The CLI arg will take precedence.
fulcrum_1 | [2022-04-22 19:40:56.797] Enabled JSON parser: simdjson
fulcrum_1 | [2022-04-22 19:40:56.797] simdjson implementations:
fulcrum_1 | [2022-04-22 19:40:56.797] fallback: Generic fallback implementation [supported]
fulcrum_1 | [2022-04-22 19:40:56.797] active implementation: fallback
fulcrum_1 | [2022-04-22 19:40:56.799] jemalloc: version 5.2.1-0-gea6b3e9
fulcrum_1 | [2022-04-22 19:40:56.799] Qt: version 5.15.2
fulcrum_1 | [2022-04-22 19:40:56.799] rocksdb: version 6.14.6-ed43161
fulcrum_1 | [2022-04-22 19:40:56.799] simdjson: version 0.6.0
fulcrum_1 | [2022-04-22 19:40:56.799] ssl: OpenSSL 1.1.1n 15 Mar 2022
fulcrum_1 | [2022-04-22 19:40:56.799] zmq: libzmq version: 4.3.4, cppzmq version: 4.7.1
fulcrum_1 | [2022-04-22 19:40:56.799] Fulcrum 1.6.0 (Release 7468990) - Fri Apr 22, 2022 19:40:56.799 UTC - starting up ...
fulcrum_1 | [2022-04-22 19:40:56.799] Max open files: 1048576
fulcrum_1 | [2022-04-22 19:40:56.799] Loading database ...
fulcrum_1 | [2022-04-22 19:40:57.241] DB memory: 512.00 MiB
fulcrum_1 | [2022-04-22 19:40:57.244] Verifying headers ...
fulcrum_1 | [2022-04-22 19:40:57.247] BitcoinDMgr: starting 3 bitcoin RPC clients ...
fulcrum_1 | [2022-04-22 19:40:57.247] BitcoinDMgr: started ok
fulcrum_1 | [2022-04-22 19:40:57.247] Stats HTTP: starting 1 server ...
fulcrum_1 | [2022-04-22 19:40:57.248] Starting listener service for HttpSrv 127.0.0.1:8080 ...
fulcrum_1 | [2022-04-22 19:40:57.248] Service started, listening for connections on 127.0.0.1:8080
fulcrum_1 | [2022-04-22 19:40:57.253] <BitcoinDMgr> Coin: BTC
fulcrum_1 | [2022-04-22 19:40:57.352] <Controller> Chain: regtest
fulcrum_1 | [2022-04-22 19:40:57.352] <Controller> Block height 131, downloading new blocks ...
fulcrum_1 | [2022-04-22 19:40:57.352] <Controller> fast-sync: Not enabled
fulcrum_1 | [2022-04-22 19:40:58.811] <Controller> Processed 132 new blocks with 132 txs (132 inputs, 263 outputs, 132 addresses), verified ok.
fulcrum_1 | [2022-04-22 19:40:58.813] <Controller> Initializing header merkle cache ...
fulcrum_1 | [2022-04-22 19:40:58.815] <Controller> Block height 131, up-to-date
fulcrum_1 | [2022-04-22 19:40:58.815] SrvMgr: starting PeerMgr ...
fulcrum_1 | [2022-04-22 19:40:58.816] FATAL: PeerMgr cannot be started for the given chain "regtest"
fulcrum_1 | [2022-04-22 19:40:58.816] Stopping Stats HTTP Servers ...
fulcrum_1 | [2022-04-22 19:40:58.816] Stopping Controller ...
fulcrum_1 | [2022-04-22 19:40:58.831] Stopping ZMQ notifier ...
fulcrum_1 | [2022-04-22 19:40:58.832] Stopping SrvMgr ...
fulcrum_1 | [2022-04-22 19:40:58.832] Stopping BitcoinDMgr ...
fulcrum_1 | [2022-04-22 19:40:58.833] Closing storage ...
fulcrum_1 | [2022-04-22 19:40:59.058] Shutdown complete
I also verified that bitcoind had completed it's initial block download and it looks like it has. This is the output from running getblockchaininfo.
{
"chain": "regtest",
"blocks": 151,
"headers": 151,
"bestblockhash": "74966bfa888c63de43e179f1e425d58a4dc148a0960e35556ccf073a239429c1",
"difficulty": 4.656542373906925e-10,
"mediantime": 1650656642,
"verificationprogress": 1,
"initialblockdownload": false,
"chainwork": "0000000000000000000000000000000000000000000000000000000000000130",
"size_on_disk": 45450,
"pruned": false,
"softforks": {
"bip34": {
"type": "buried",
"active": false,
"height": 500
},
"bip66": {
"type": "buried",
"active": false,
"height": 1251
},
"bip65": {
"type": "buried",
"active": false,
"height": 1351
},
"csv": {
"type": "buried",
"active": false,
"height": 432
},
"segwit": {
"type": "buried",
"active": true,
"height": 0
},
"testdummy": {
"type": "bip9",
"bip9": {
"status": "started",
"bit": 28,
"start_time": 0,
"timeout": 9223372036854775807,
"since": 144,
"statistics": {
"period": 144,
"threshold": 108,
"elapsed": 8,
"count": 8,
"possible": true
},
"min_activation_height": 0
},
"active": false
},
"taproot": {
"type": "bip9",
"bip9": {
"status": "active",
"start_time": -1,
"timeout": 9223372036854775807,
"since": 0,
"min_activation_height": 0
},
"height": 0,
"active": true
}
},
"warnings": ""
}
My fulcrum.conf file looks like this:
datadir = /data/regtest
bitcoind = bitcoind:18443
tcp = 0.0.0.0:50001
I noticed in #91 that there is an output about the TcpSrv starting but I am not seeing that in my output. Any idea why the tcp server is not starting?
Is it be possible to give txnum2txhash the same structure on disk as txhash2txnum?
Me again :-)
Would it be possible to store the flat file txnum2txhash as smaller files like it is done in the directory txhash2txnum? The reason for this question is that I backup my whole system every month via rsync which only copies the files that have changed. This is usually a fraction of the total number of files on my system and results in a huge speedup.
For the bitcoin blockchain I perform multiple rsyncs until nothing is copied anymore and then "repair" the blockchain running the software on a different computer. In this way I'm able to keep the bitcoin software running while making a backup and keep all my connections.
From what I've seen, a similar trick is possible for Fulcrum but it's the big 21 GB txnum2txhash that makes it hard/impossible. But even if it's necessary to stop Fulcrum, do a final rsync and then restart Fulcrum, the downtime would be much smaller.
More in general, I think for backup purposes it's always a good idea to keep files on disk relatively small.
Implementing blockchain.outpoint.subscribe method
Hi,
I can't start without saying that your work on Fulcrum is impressive and deserves much more attention. My experience with it is so much better than ElectrumX both during sync and once synced ! It is a really impressive work !
I would like to know how hard it would be to implement blockchain.outpoint.subscribe method from Electrum 1.5 in Fulcrum. The main application would be to show the spending transaction of an outpoint in a block explorer. You can look at my implementation at janoside/btc-rpc-explorer#356.
Implementing blockchain.outpoint.subscribe is straightforward in ElectRS because it builds a transaction inputs' index as explained here. Such implementation in ElectRS is available at romanz/electrs#454. If I understand well which indexes Fulcrum build, there is no such index so it won't be that easy. I don't wish to take the path of building a new index of spending transactions because building the indexes is a sufficiently painful process as it is already. That's said I would gladly try to sync and use a version of Fulcrum with such index if you implement it in the future.
However, Fulcrum builds already a lot more indexes than ElectRS and I think it is possible to have a quite efficient solution without building a new one. My idea is to use the scripthash of the outpoint.
- Using
scripthash_historyindex, Fulcrum will get all thetxnumof the scripthash of the outpoint. - Using
txhash2txnum, it can also get thetxnumof the funding transaction. Now we use the fact that in BTC blockchain, transactions in the same block are ordered in topological order, so we know that thetxnumof the spending transaction MUST be greater than thetxnumof the funding transaction. I guess you want to keep Fulcrum fully compatible with BCH, I don't follow BCH protocol change but I know it used CTOR for a while, so in this case the check must be relaxed a bit: the spending transaction has atxnumhigher than the firsttxnumof the block including the funding transaction (it is a rather small performance loss in the BTC case to keep everything compatible). - Fulcrum load every transactions returned by the
scripthash_historyindex that have a greatertxnumthan the funding transaction and check if they spend the outpoint of the query. If it finds the spending transaction, returns it.
This is clearly suboptimal solution. It can also be a terrible DoS vector if we query the spending transaction of an outpoint with an highly used scripthash that only spend it after many other transactions involving this scripthash. I think if the outpoint is not spent we can use the scripthash_unspent index to detect it and avoid the almost exhaustive search among all transactions involving a scripthash. That said, I would like to try and see if it is more efficient than ElectRS which must always load full blocks to answer each query of an outpoint, it can be terribly slow if we want to look at all the spending transactions of the 4000 outpoints of a transaction like d22c53dd5a79fb6cc7a10ed6531761ae43ce1ce2ecd1ebbc5746ba82c71e6d70
I would like to have your opinion on this idea, do you think it is worth to try ? Do you have a better idea ?
Thanks again for the amazing work, this is an truely amazing software !
Phoenix lightning wallet won't connect to my Fulcrum Electrum server
First of all: Fulcrum is an amazing Electrum server! Very fast!
Just wanted to bring this issue to your attention: ACINQ/phoenix#188.
Consider naming TARGET with only lowercase letters for Linux/BSD
I would like to package Fulcrum for Fedora Linux and would stay as close to upstream and Fedora guidelines as possible.
Installing binaries in the $PATH starting with capital letters make it un-userfriendly for users who starts programs from commandline.
Nothing in the GUI, labels, description, marketing needs to be changed. OpenSCAD solves it like this openscad.pro:
macx {
TARGET = OpenSCAD
}
else {
TARGET = openscad$${SUFFIX}
}
On my machine there is only 4 non-X executables out of 3660 that contains capital letters.
$ ls /usr/bin | wc -l
3660
$ ls -ld /usr/bin/*[[:upper:]]*
-rwxr-xr-x. 1 root root 17008 May 15 14:30 /usr/bin/checkXML5
lrwxrwxrwx. 1 root root 5 Jan 29 2020 /usr/bin/Mail -> mailx
lrwxrwxrwx. 1 root root 26 Jul 8 23:18 /usr/bin/mvnDebug -> /etc/alternatives/mvnDebug
-rwxr-xr-x. 1 root root 30920 Aug 13 13:36 /usr/bin/oLschema2ldif
lrwxrwxrwx. 1 root root 4 Mar 30 2020 /usr/bin/X -> Xorg
-rwxr-xr-x. 1 root root 277 Mar 30 2020 /usr/bin/Xorg
-rwxr-xr-x. 1 root root 2111968 Mar 30 2020 /usr/bin/Xvfb
-rwxr-xr-x. 1 root root 2410584 Mar 30 2020 /usr/bin/Xwayland
Missing 1.7.0 ARM build?
Hey there. In your recent 1.7.0 release notes, you state that there is an ARM build available, yet no file is provided in the assets section. I was just wondering if that is by mistake if there's some issue.
Having an ARM build ready for 1.6.0 was very nice feature.
python client cannot gracefully close transport (waits forever), it needs to abort
An asyncio python client using asyncio.BaseTransport.close() never manages to close a transport (with a Fulcrum server);
it must abort it using asyncio.WriteTransport.abort.
We have already talked about this on IRC some time ago, I just wanted to log the issue so it can easily be referenced.
See IRC log:
[2020-12-16 17:38:56 +0100] <ghost43> NilacTheGrim: oh btw, you might be interested in https://github.com/spesmilo/electrum/commit/e83f0dd3fc4e19c8845fc21f344400f1b0a7eebf - see commit msg. we have narrowed the issue down to fulcrum servers
[2020-12-16 17:40:59 +0100] <ghost43> although I appreciate that the interesting and actionable details from your PoV are missing :P I can only tell you that closing the session with a fulcrum server is slow but not with other servers, on a good network connection
[2020-12-16 21:07:33 +0100] <NilacTheGrim> ghost43, regarding that Fulcrum issue -- hmm. Not sure why that would be. What exactly is going on there on the socket level?
[2020-12-16 21:07:56 +0100] <NilacTheGrim> what is interface.close()? just a regular socket close? why would that take so long?
[2020-12-16 21:08:19 +0100] <NilacTheGrim> i did actually notice this gUI hang thing
[2020-12-16 21:17:43 +0100] <NilacTheGrim> fulcrum just doesn't receive the close packet until like 10 second delay. i literally have it here on my lan. i tried socat and other tools and it works fine. something is funny in the python lib .. weird that it doesn't happen on E-X though.
[2020-12-16 21:25:27 +0100] <NilacTheGrim> ghost43, i'm investigating but so far this looks like some sort of bug/out of spec behavior in aiorpcx
[2020-12-16 21:29:09 +0100] <ghost43> NilacTheGrim: when I looked, it seemed aiorpcx is not doing anything weird -- it is using stdlib asyncio stuff
[2020-12-16 21:29:39 +0100] <NilacTheGrim> i'm looking now
[2020-12-16 21:30:53 +0100] <NilacTheGrim> note that Fulcrum works with a plethora of different client implementations (including the old socket stuff Electron Cash still uses), plus javascript clients, explorers. i have never heard of issues closing the connection. I also cm doing it now here .. using C library to close the socket. nothing is weird at all. yet when I use Electrum latest with aiorpcx it does indeed hang. node that close() can take a timeout parameter so you could also close
[2020-12-16 21:30:53 +0100] <NilacTheGrim> without having to spawn a new thread group or whatever.
[2020-12-16 21:33:13 +0100] <ghost43> NilacTheGrim: https://github.com/kyuupichan/aiorpcX/blob/a5f9875f65b63384c33e8a13aa6ab66e8c5bc852/aiorpcx/rawsocket.py#L123
[2020-12-16 21:33:19 +0100] <ghost43> https://docs.python.org/3/library/asyncio-protocol.html#asyncio.BaseTransport.close
[2020-12-16 21:33:22 +0100] <ghost43> https://docs.python.org/3/library/asyncio-protocol.html#asyncio.WriteTransport.abort
[2020-12-16 21:34:35 +0100] <ghost43> self._closed_event gets set when asyncio calls connection_lost
[2020-12-16 21:34:56 +0100] <ghost43> ISTM the whatever the cause is, it is in asyncio itself
[2020-12-16 21:35:04 +0100] <ghost43> -the
[2020-12-16 21:37:52 +0100] <NilacTheGrim> i am not familiar with aiorpcx but it seems to do a bunch of work internally when closing gracefully like cancelling futures and whatnot. stil this should not be fulcrum-specific. so weird man.
[2020-12-16 21:38:12 +0100] <NilacTheGrim> the thing is so whatever weird stuff is happening -- if it indeed is some out-of-spec issue -- electrumx uses the same lib so it has the same weirdness, lol.
[2020-12-16 21:38:24 +0100] <NilacTheGrim> let me try against another implementation.. electrs works ok .. right?
[2020-12-16 21:39:31 +0100] <NilacTheGrim> yeah it does. ok well troubleshooting...
[2020-12-16 21:45:01 +0100] <NilacTheGrim> ghost43: maybe related: https://bugs.python.org/issue39758
[2020-12-16 21:47:35 +0100] <NilacTheGrim> this appears tobe a bug in python itself when running against microsoft tcp stacks sometimes not always. lol. guess what? my server is on a microsoft tcp stack. still troubleshooting. doesn't happen in python 3.6
[2020-12-16 22:08:19 +0100] <NilacTheGrim> yeah this is definitely a known bug in python asyncio -- and only happens against microsoft tcp stacks!
[2020-12-16 22:08:35 +0100] <NilacTheGrim> (my server happens to be on ms) -- see the linked issue to python.org
[2020-12-16 22:11:53 +0100] <NilacTheGrim> ghost43, yep can confirm -- trying against a non-MS TCP stack Fulcrum server, runs ok. Trying against MS-TCP stack Fulcrum hangs. (this is on Electrum with the commit prior to the fix) -- I bet if you were to run ElectrumX on Windows it would have the same issue.
[2020-12-16 22:13:38 +0100] <NilacTheGrim> (ok also happens in python 3.6 -- definitely a bug in python and/or MS TCP stack interaction -- still unfixed)
[2020-12-16 22:32:52 +0100] <ghost43> NilacTheGrim: good find, that issue looks related. what do you mean by ms tcp stack? do you mean running the server on Windows? because I have tested against multiple fulcrum servers and all exhibited it, e.g. btc.electroncash.dk:60002:s, fulcrum.keff.org:50002:s, fulcrum.grey.pw:51002:s
[2020-12-16 22:33:01 +0100] <ghost43> I doubt those all run on windows
[2020-12-16 22:33:39 +0100] <NilacTheGrim> those run linux..
[2020-12-16 22:33:44 +0100] <NilacTheGrim> let me try it now..
[2020-12-16 22:34:42 +0100] <NilacTheGrim> i didn't realize there's more than 2 Fuclrum servers..
[2020-12-16 22:34:52 +0100] <NilacTheGrim> electroncash.dk is down for me.. hmm.
[2020-12-16 22:37:34 +0100] <NilacTheGrim> the only other one that is up for me is fulcrum.grey.pw:51002.. it's happening on that one. the other 2 are down.
[2020-12-16 22:38:12 +0100] <ghost43> weird, I have just tested connecting to all three and they are up
[2020-12-16 22:39:22 +0100] <NilacTheGrim> the electroncash.dk one is definitely Linux -- i know the admin he's on Linux -- it's happening now for you there too? I'm on commit hash 19f806ddf47634fe0c821244a62b41c71c4138d3 but cannot connect there at all
[2020-12-16 22:40:07 +0100] <ghost43> I have just tested whether I can connect to them now; but a few days ago when I committed that I have tested and reproduced the issue on all three
[2020-12-16 22:42:26 +0100] <NilacTheGrim> so maybe it's not just windows stacks but some other interaction. i was able to get the hang with that test script in the python.org issue...
[2020-12-16 22:51:03 +0100] <NilacTheGrim> man this socket issue is driving me crazy -- glad you have a workaround. it's worrying.
[2020-12-16 22:51:09 +0100] <NilacTheGrim> i will try and isolate it further...
[2020-12-16 22:51:32 +0100] <NilacTheGrim> it definitely is a thing with python SSL tho -- weird that other implementations like electrs are unaffected. will dig further...
[2020-12-16 22:52:35 +0100] <NilacTheGrim> there's a big huge explanation here too of what they think may be happening with some stacks: https://github.com/python-trio/trio/blob/31e2ae866ad549f1927d45ce073d4f0ea9f12419/trio/_ssl.py#L779-L829
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.