bitcoin-tools / nodebuilder Goto Github PK

Run a secure Bitcoin Core node with ease.

License: MIT No Attribution

Shell 98.49% Dockerfile 1.51%

bitcoin bitcoin-api bitcoin-blockchain bitcoin-blocks bitcoin-core bitcoin-core-rpc bitcoin-development bitcoin-docker bitcoin-explorer bitcoin-full-node

nodebuilder's People

Contributors

Stargazers

Watchers

Forkers

22388o

nodebuilder's Issues

Add optional Tor support

Detect which operating system is running

Fix indents in CI

Sending Ctrl+C / SIGINT shouldn't close the child process bitcoin-qt

Replace Google's DNS address with Quad9's

Updated hardcoded archival node variable

Increase the hard-coded value for archival_node_required_disk_in_gib (line 256).

The sweet spot is about 50 to 100 GB more than the size of blocks/ + chainstate/.

Back in June 2023, that sum was 522. Now, it's 586.

user1@comp1:~$ du -hd1 .bitcoin/
576G	.bitcoin/blocks
9.9G	.bitcoin/chainstate
586G	.bitcoin/

Bring it back into the sweet spot.

On macOS, add core-utils dependency

Add optional I2P Support

bug: err command not found

Seems to be a regressoin.

This happens when running into an error while checking for internet:

user1@comp1:~/Documents/GitHub/nodebuilder$ ./nodebuilder 
./nodebuilder: line 77: err: command not found
\E[6n
user1@comp1:~/Documents/GitHub/nodebuilder$ ^C
user1@comp1:~/Documents/GitHub/nodebuilder$ ^C
user1@comp1:~/Documents/GitHub/nodebuilder$ shellcheck nodebuilder 
user1@comp1:~/Documents/GitHub/nodebuilder$ bash -x nodebuilder 
+ set -o errexit
+ set -o nounset
+ set -o pipefail
+ trap handle_exit EXIT
+ trap 'handle_error ${LINENO:-}' ERR
+ target_bitcoin_version=26.0
++ uname -m
+ target_bitcoin_architecture=x86_64
+ case "$(uname -s)" in
++ uname -s
+ target_bitcoin_operating_system=linux-gnu
++ mktemp -d
+ temp_directory=/tmp/tmp.5redc0oxE7
+ bitcoin_core_data_dir=/home/user1/.bitcoin
+ user_data_dir=/home/user1/.local/share
+ shortcut_image_source=https://github.com/bitcoin-tools/nodebuilder/raw/master/data/bitcoin.png
+ shortcut_image_file=/home/user1/.local/share/images/bitcoin.png
+ timeout --preserve-status 0.1 nc -z 1.1.1.1 53
+ timeout --preserve-status 0.1 nc -z 8.8.8.8 53
+ timeout --preserve-status 0.5 nc -z bitcoincore.org 443
+ err 'ERROR: Check for active internet failed.'
nodebuilder: line 77: err: command not found
++ handle_error 77
++ ensure_cursor_is_on_column_one
++ printf %s '\E[6n'
\E[6n++ read -rsdR cursor_position
+++ handle_exit
+++ '[' -n /tmp/tmp.5redc0oxE7 ']'
+++ rm -rf -- /tmp/tmp.5redc0oxE7/

Add support for Fedora

Use functions to improve log management

https://linuxconcept.com/bash-script-log-management/

Update the CI section of the test doc

Replace echo -e and echo -n with printf

Inconsistent Behavior Across Different Systems

Another issue you might encounter is inconsistent behavior of the echo command across different systems. This is because the behavior of echo can vary depending on the system and the shell. For instance, some versions of echo support the ‘-e’ option, while others don’t.

If you’re writing a script that will be run on different systems, it might be better to use printf instead of echo, as printf has more consistent behavior.

Taken from https://ioflood.com/blog/bash-echo-newline/

The Linux user running Bitcoin Core should not have sudo permissions

./nodebuilder: line 37: temp_directory: unbound variable

bug: printf isn't printing line breaks with \n

This is the output to the user.

Syncing the block headers (second pass).\nHeaders synced:     747559\nChain sync size:    0 MiB\nDisk free space:    18 GiB\n\nSynchronizing can take weeks on a slow connection.\n

If reboot is required after dist-upgrade, auto-start on boot

Comply with the XDG Base Directory Specification

Check if binary sha256sum is on PATH

Something like this:

if command -v sha256sum >/dev/null; then
   sha256sum_cmd="sha256sum"
elif command -v shasum >/dev/null; then
   sha256sum_cmd="shasum -a 256"
else
   >&2 echo "Either 'sha256sum' or 'shasum' must on PATH."
   >&2 echo "Exiting..."
   exit 1
fi

Taken from this GitHub commit

refactor: check if current_bitcoin_version string is empty before comparing to expected versoin

On macOS, use proper data directory

Use ~/Library/Application Support/Bitcoin instead of ~/.bitcoin.

https://github.com/bitcoin/bitcoin/blob/master/doc/bitcoin-conf.md#default-configuration-file-locations

Speed up the IBD by increasing dbcache

When RAM is sufficient, set Bitcoin Core to run with increased dbcache.

No more than 25% of the total RAM on the system.

On low-RAM systems, set Bitcoin Core to run with reduced memory

Reduce dbcache.
Reduce maxmempool.
Reduce maxconnections.
Set MALLOC_ARENA_MAX=1.

https://github.com/bitcoin/bitcoin/blob/master/doc/reduce-memory.md

Add macOS support details to README

Remove unnecessary lines from Dockerfile

bug: cannot create directory, permission denied

...
  Extracting Bitcoin Core... ok.
  Running the unit tests... ok.
  Installing Bitcoin Core... mkdir: cannot create directory '/usr/local/share/man/man1': Permission denied
user1@comp1: $ mkdir -p /usr/local/share/man/man1
mkdir: cannot create directory '/usr/local/share/man/man1/': Permission denied
user1@comp1: $ sudo mkdir -p /usr/local/share/man/man1
user1@comp1: $

Dockerfile should clone with depth=1

Only require one DNS internet check to pass, not both

Add macOS support

Add Docker support details to README

CI is failing due to restart required

From CI job https://github.com/bitcoin-tools/nodebuilder/actions/runs/7501247007/job/20421542028:

+ echo -n 'Performing a system upgrade... '
+ sudo NEEDRESTART_MODE=a apt-get -qq dist-upgrade --assume-yes
+ '[' -f /var/run/reboot-required ']'
++ cat /var/run/reboot-required.pkgs
+ echo -en '\nREBOOT REQUIRED to upgrade the following:\nlibc6\n\nPRESS ANY KEY to reboot or press Ctrl+C to exit... '
Performing a system upgrade... 
REBOOT REQUIRED to upgrade the following:
libc6

+ read -rsn1
+ echo Rebooting.
+ sudo reboot
PRESS ANY KEY to reboot or press Ctrl+C to exit... Rebooting.
+ exit 0
+ handle_exit
+ '[' -n /tmp/tmp.m1PIsurix2 ']'
+ rm -rf -- /tmp/tmp.m[1](https://github.com/bitcoin-tools/nodebuilder/actions/runs/7501247007/job/20421542028?pr=3#step:10:1)PIsurix[2](https://github.com/bitcoin-tools/nodebuilder/actions/runs/7501247007/job/20421542028?pr=3#step:10:2)/

On macOS, clean up this error with `/proc/1/cgroup`

. ok.
  Removing temporary files... ok.
Creating application shortcuts... grep: /proc/1/cgroup: No such file or directory
/bin/bash: line 266: [: : integer expression expected
ok.
Starting Bitcoin Core... grep: /proc/1/cgroup: No such file or directory
/bin/bash: line 383: [: : integer expression expected
ok.

and

Disk free space:    58 GiB
grep: /proc/1/cgroup: No such file or directory
/bin/bash: line 464: [: : integer expression expected
Synchronizing can take weeks on a slow connection.

Add .gitignore

Add dependency `bc`

PySpelling CI check is broken

Add netcat as a dependency

Reorder ci jobs - spellcheck and shfmt should run in first stage

Update copyright date

On macOS, brew installation should be unattended

Add option for compiling Bitcoin from source

Distinguish between being run in a terminal vs run inside another script

This will be helpful as the project evolves to allow other projects to use the nodebuilder repo as a submodule.

Such as "install Bitcoin and perform the IBD, then afterwards run this other code that uses Bitcoin RPC calls."

Here's the contents of test_script_for_childparent.sh:

#!/bin/bash

echo "state\$PPID is $(ps -o stat= -p $PPID)"
echo "state\$\$ is $(ps -o stat= -p $$)"

Here's what happens when you run it:

user1@comp1:~/Desktop$ ./test_script_for_childparent.sh 
state$PPID is Ss
state$$ is S+
user1@comp1:~/Desktop$ ./test_script_for_childparent.sh &
[1] 10955
user1@comp1:~/Desktop$ state$PPID is Ss+
state$$ is S
[1]+  Done                    ./test_script_for_childparent.sh

Here's the contents of test_script_launcher.sh:

#!/bin/bash

/bin/bash -c "./test_script_for_childparent.sh"

Here's what happens when you run it:

user1@comp1:~/Desktop$ ./test_script_launcher.sh 
state$PPID is S+
state$$ is S+
user1@comp1:~/Desktop$ ./test_script_launcher.sh &
[1] 10967
user1@comp1:~/Desktop$ state$PPID is S
state$$ is S

[1]+  Done                    ./test_script_launcher.sh

CI warning Node.js 16 actions are deprecated

Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.

Remove unnecessary docker install dependencies

Increase internet timeouts by 10x

During check for internet, increase the timeouts 10x from 0.1 -> 1 and 0.5 -> 5.

user1@comp1:~$ timeout --preserve-status 10 nc -z 1.1.1.1 53
user1@comp1:~$ echo $?
0
user1@comp1:~$ timeout --preserve-status 0.1 nc -z 1.1.1.1 53
user1@comp1:~$ echo $?
143
user1@comp1:~$ timeout --preserve-status 0.1 nc -z 1.1.1.1 53 && echo ok
user1@comp1:~$ timeout --preserve-status 1 nc -z 1.1.1.1 53 && echo ok
ok

Add OpenDNS to internet check

On macOS, check if brew is installed before re-installing

Update/upgrade brew instead

Add support for Docker

Don't output to the user with echo

Don't print the message "Starting Bitcoin Core... Bitcoin Core starting"

+ printf %s 'Starting Bitcoin Core... '
+ '[' true = true ']'
+ bitcoind --daemonwait
Starting Bitcoin Core... Bitcoin Core starting
+ printf '%s\n' ok.
ok.