digitalbitbox / bitbox-base Goto Github PK

make

I tried to register qemu on the host, but it does not seems to have worked

docker run --rm --privileged multiarch/qemu-user-static:register

As discussed in #53, we are currently describing the configuration of services like bitcoin.conf for bitcoind in quite a lot of detail in our docs/:

• https://base.shiftcrypto.ch/applications/bitcoin-core.html

There's many other such config files, systemd services which currently duplicates the configs and adds commentary within docs/.

We also define the actual configs like bitcoin.conf, at time of writing inside the customize-armbian-rockpro64.sh script:

• https://github.com/digitalbitbox/bitbox-base/blob/master/armbian/base/build/customize-armbian-rockpro64.sh#L313

We should consider work to:

• refactor out configs from customize-armbian-rockpro64.sh
• update the standalone config files to include as much of the comments and context as makes sense from the docs
• create some templated .md files with placeholder values for the actual config contents
• add a preprocessor script that reads the templated .md files as well as the config files and renders the concrete .md files
• add some warning to not edit the autogenerated .md files
• add CI support for regenerating the .md files, or at least giving an error if the generated files are not up-to-date

From the following: https://github.com/digitalbitbox/bitbox-base/blob/d2d75f06f7fcde655209bbb11a01faafa3fb26c1/armbian/base/build.conf#L26-L28

it would seem like this is setting the root password. However, it appears to be doing more than that. What it actually does is locking the user base if the root password wasn't set to a custom string in https://github.com/digitalbitbox/bitbox-base/blob/d2d75f06f7fcde655209bbb11a01faafa3fb26c1/armbian/base/customize-armbian-rockpro64.sh#L219-L223

When building the image, I set BASE_ADD_SSH_KEYS=true in armbian/base/build-local.conf and update armbian/base/config/ssh/authorized_keys to contain my ssh key.
I didn't assign any password to BASE_LOGINPW because all I want is to ssh with my key as base user, and this essentially locked me out.

So, it forces me to do the "unsafe" option, setting BASE_LOGINPW to a non-empty string, even though I won't use it.

Additionally, a message during the build is also confusing:

CONFIGURATION:
    USER / PASSWORD:    base / some-password
...

which would appear as if it was the base user password but in reality what it actually does later is:

+ BASE_LOGINPW_FINAL=some-password
+ echo root:some-password
+ chpasswd

I would suggest being more explicit and rename BASE_LOGINPW to something like BASE_ROOT_PASSWORD and don't conflate that option with the base user at all.

This issue tracks the progess in implementing the BitBox Base Supervisor (bbbsupervisor), written in Go. The code and docuumentation is located in bitbox-base/tools/bbbsupervisor.

Features
System control

• fan control:
  • implemented as bbbfancontrol
• monitor disk space
  • action: warn user through app
  • very tight logrotation and read-only rootfs already implemented

Bitcoin Core

• detect initial block download: query Prometheus for bitcoind_ibd=1
  • actions:
    • touch or delete /data/systemd-triggers/bitcoind-fully-synced
    • set dbcache to 300MB or 2000GB with e.g. bbb-config set bitcoin_dbcache 300
    • when IBD finished, start lightningd and electrs
• detect no space left: monitor journal for No space left on device
  • action: warn user through app

Electrs

• detect auth failure to bitcoind:
  • monitor journal for WARN - reconnecting to bitcoind: no reply from daemon
  • actions:
    • recreate bitcoind cookie env file /mnt/ssd/bitcoin/.bitcoin/.cookie.env
    • restart electrs.service

Currently, we do support building the Go binaries for BitBox Base from any directory using Docker. If the user invokes the default top-level make target (or make docker-build-go, to build only the Go binaries), everything should Just Work:

• https://github.com/digitalbitbox/bitbox-base/blob/master/Makefile#L16

However, if the user wants to build directly on their system, our build scripts do enforce some requirements on their environment:

• https://github.com/digitalbitbox/bitbox-base/blob/master/scripts/check-go-env.sh#L10

Specifically, they need to check out this repo under $GOPATH/src/, since that's a requirement as long as we use vendor/ setup to manage 3rd party dependencies. See #42 and the following page for more context:

• https://github.com/golang/go/wiki/GOPATH#vendor-directories-ignored-outside-of-gopath

It should be possible to remove this restriction if we switched to Go modules:

• https://github.com/golang/go/wiki/Modules#modules

This issue can be closed once a developer with a modern Go toolchain installed can build all Go binaries in this repo without any assumption about where the repo is located in their filesystem.

Edit: Included more context. Original description follows:

I tried make -C tools/bbbsupervisor native and was told to move the repo to $GOPATH/src/....

As an idea, you could apply a trick like this so you can compile it right there. Since bitbox-base is not primarily a Go repo, such an approach might be justified.)

For some time now, the build script for Armbian, run in Docker privileged mode, fails at the end when mounting the image on the loopback device. The build script does not fail, it just produces an invalid image.

Strangely, running the build process a second time (always using caching CLEAN_LEVEL=oldcache) runs without errors and produces the correct image.

Opened issue in armbian/build#1472

In armbian/mender-convert.sh you call
git clone -b master …. mender-convert
During the conversion of the image, the build fails with

[mender-convert-extract] Sorry but the provided option is not supported: from-raw-disk-Image

I see that the master branch of mender-convert has changed two days ago and the command line Parameters used by bitbox-base no longer work.
If you pull 1.2.2 instead of master, then it works again.

In our current build process, we use VirtualBox VMs managed by Vagrant to build the Armbian-based OS images for the Base:

• https://base.shiftcrypto.ch/os/armbian-build.html

This has some downsides:

• all developers or automated systems need Vagrant and VirtualBox installed, with specific versions; in comparison, Docker is becoming a standard dependency to manage dependencies of various tech stacks for many developers
• we already use Docker in the Base project for building Go programs, so using it for Armbian builds would reduce the set of dependencies
• having several virtualization technologies in the same repo has already led to unfortunate interactions / extra complexity, so just relying on Docker would be preferable

This work was earlier discussed in a PR comment thread.

The official Armbian docs for building with Docker are available at:

• https://docs.armbian.com/Developer-Guide_Building-with-Docker/

This issue can be closed once support for building Armbian OS images with Docker has been merged to master, or if we decide not to pursue this.

At time of writing, we have travis-ci.org integration for this repo, and as of today via #88 we also have experimental circleci.com integration.

As described in #39 (comment), Travis has issues with slow builds preventing us from switching to using it for the Armbian OS image builds. We are exploring whether CircleCI is better here.

There's starting to be quite a few different options / ideas about CI, so this issue attempts to gather them into one place so we can make a decision for what system to use that will serve our needs best:

1. Shift Gitlab
2. CircleCI
3. Travis
4. Semaphore2

To expand on the Shift Gitlab alternative, we have a self-hosted Enterprise Edition Gitlab instance which we are evaluating currently. Once we figure out internally what the long-term future is for that tool (which we could use for source code management, CI/CD, deploy pipelines etc) and how we'd work with external contributors like for the BitBox Base project, it might be preferable to use it for CI needs as a default choice.

The needs for the BitBox Base project that I can see are:

1. Run QA checks (tests, linters, etc)
2. Do builds of individual applications, like the middleware
3. Build the main Armbian image
4. (Possibly) Upload build results or otherwise deploy them

Here's what I see from the serial output while trying to boot bin/img-mender/0.1.0/BitBoxBase-v0.1.0-RockPro64.img created from bin/img-armbian/BitBoxBase_Armbian_RockPro64.img with make -C armbian mender-artefacts, all built from commit d2d75f0 aka latest tag 0.1.0.

vcc-phy-regulator@vcc_phy: ; enabling (ret: -38)
vdd-log@vdd_log: ; enabling (ret: -38)
MMC:   sdhci@fe330000: 0, dwmmc@fe320000: 1
Card did not respond to voltage select!
mmc_init: -95, time 19
*** Warning - No block device, using default environment

In:    serial@ff1a0000
Out:   serial@ff1a0000
Err:   serial@ff1a0000
Model: Pine64 RockPro64
normal boot
Net:   eth0: ethernet@fe300000
Hit any key to stop autoboot:  0
Card did not respond to voltage select!
mmc_init: -95, time 21
switch to partitions #0, OK
mmc1 is current device
Scanning mmc 1:1...
Found U-Boot script /boot.scr
reading /boot.scr
3136 bytes read in 3 ms (1020.5 KiB/s)
## Executing script at 00500000
Saving Environment to MMC...
Card did not respond to voltage select!
mmc_init: -95, time 20
!No block device
Boot script loaded from mmc 0:1
Card did not respond to voltage select!
mmc_init: -95, time 20
** Bad device mmc 0 **
Card did not respond to voltage select!
mmc_init: -95, time 21
"Synchronous Abort" handler, esr 0x96000010
ELR:     f7f6dd5c
LR:      f7f1b160
x0 : 0000000000000000 x1 : 0000000000000002
x2 : 0000000000000010 x3 : 0000000000000000
x4 : 00000000f7f81df0 x5 : 0000000000000002
x6 : 0000000000000008 x7 : 0000000000000044
x8 : 0000000000000001 x9 : 0000000000000008
x10: 00000000f5f0043f x11: 0000000000000006
x12: 000000000001869f x13: 00000000ffffffff
x14: 00000000f5f0297c x15: 00000000ffffffff
x16: 000000001e534356 x17: 00000000104080aa
x18: 00000000f5f0ee08 x19: 0000000000000000
x20: 00000000f5f34170 x21: 00000000f5f34100
x22: 00000000f7fb1000 x23: 00000000f5f34120
x24: 0000000000000000 x25: 00000000f5f35290
x26: 0000000000000000 x27: 0000000000000000
x28: 0000000000000000 x29: 00000000f5f009a0

Resetting CPU ...

and then it repeats all over again in an endless loop.

However, booting the same image pre Mender's convertion (bin/img-armbian/BitBoxBase_Armbian_RockPro64.img) works fine:

In:    serial@ff1a0000
Out:   serial@ff1a0000
Err:   serial@ff1a0000
Model: Pine64 RockPro64
Net:   eth0: ethernet@fe300000
Hit any key to stop autoboot:  0
Card did not respond to voltage select!
mmc_init: -95, time 20
switch to partitions #0, OK
mmc1 is current device
Scanning mmc 1:1...
Found U-Boot script /boot/boot.scr
2949 bytes read in 30 ms (95.7 KiB/s)
## Executing script at 00500000
Boot script loaded from mmc 1
167 bytes read in 23 ms (6.8 KiB/s)
7803492 bytes read in 873 ms (8.5 MiB/s)
20987912 bytes read in 2276 ms (8.8 MiB/s)
96273 bytes read in 203 ms (462.9 KiB/s)
** File not found /boot/dtb/rockchip/overlay/rockchip-fixup.scr **
## Loading init Ramdisk from Legacy Image at 04000000 ...
   Image Name:   uInitrd
   Image Type:   AArch64 Linux RAMDisk Image (gzip compressed)
   Data Size:    7803428 Bytes = 7.4 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 01f00000
   Booting using the fdt blob at 0x1f00000
   Loading Ramdisk to f5790000, end f5f01224 ... OK
   reserving fdt memory region: addr=1f00000 size=7d000
   Loading Device Tree to 00000000f5710000, end 00000000f578ffff ... OK

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 4.4.198-rockchip64 (root@builder) (gcc version 8.3.0 (GNU Toolchain for the A-profile Architecture 8.3-2019.03 (arm-rel-8.36)) ) #3 SMP Tue Nov 19 00:05:14 CET 2019
[    0.000000] Boot CPU: AArch64 Processor [410fd034]
[    0.000000] earlycon: Early serial console at MMIO32 0xff1a0000 (options '')
[    0.000000] bootconsole [uart0] enabled

My SD card is just a regular Kingston 16Gb class 10.

The hardware is practically identical to https://base.shiftcrypto.ch/hardware/components.html except I'm using a consumer version of PCIe M.2 SSD adapter, and my USB UART doesn't have the issue described in https://base.shiftcrypto.ch/hardware/hsm/hsm-uart.html. But I don't think it all matters given the base armbian image boots.

At the time of the images build, https://github.com/armbian/build was at commit 35e69432988106d7bc0e6895c36010f5426ee5ef and https://github.com/mendersoftware/mender-convert at 56f4123864100cc81d87d3b6f66274d23d2177ba, which BTW another point - maybe these need to be pinned to a specific commit instead of always "git clone ..." from whatever their master/HEAD is, or use submodules or something else.

Again, it's not possible to send tokens with MyEtherWallet

https://www.reddit.com/r/MyEtherWallet/comments/c3m7ye/digital_bitbox_doesnt_work_with_myetherwallet/ery3j8r?utm_source=share&utm_medium=web2x

It feels kind of weird that building the image by default includes "random" keys in https://github.com/digitalbitbox/bitbox-base/blob/d2d75f06f7fcde655209bbb11a01faafa3fb26c1/armbian/base/config/ssh/authorized_keys#L1-L4

Maybe a better option would be to add armbian/base/config/ssh/authorized_keys to .gitignore and document better how to add a key in the config? Say, somewhere around https://github.com/digitalbitbox/bitbox-base/blob/d2d75f06f7fcde655209bbb11a01faafa3fb26c1/armbian/base/build.conf#L23-L24

More generally, I'm wondering why A/B partition update was chosen over say NixOS or Guix which would provide properties equivalent to those listed as advantages in atomic disk image updates.

Some reasoning is published at https://base.shiftcrypto.ch/update/update-method.html but it seems to focus only on atomic disk image vs imperative scripts/packages. So, maybe that page could be expanded with more details?

Thanks.

In a similar setup to bbb tor --verify-config warns that Directory /var/lib/tor/hidden_service_bitcoind/ cannot be read: Permission denied

I'm not 100% sure and might miss something here.

Do we create the dirs for the hidden services somewhere?
i.e.

mkdir /var/lib/tor/hidden_service/ 
chown debian-tor:debian-tor /var/lib/tor/hidden_service/ 
chmod 0700 /var/lib/tor/hidden_service/

(hidden_service in the example code would be hidden_serice_bitcoind and so on)

Is there a reason why you use the tag sunxi-4.20 beside it being the latest release?
On the main armbian repo, I want to try master, but I am not sure if I would not break something else down the road.

I have a issue where Base keeps trying to reconnect with a my other node (LND 0.8.1) with already established channel. Base can hold connection with other peers and even holds other channel (LN-BIG).

c-lightning logs:

Dec 29 10:27:36 base lightningd[2175]: 2019-12-29T10:27:36.414Z DEBUG lightningd(2175): lnd.pubkey chan #3: Got depth change 1723->1724 for txid
Dec 29 10:27:36 base lightningd[2175]: 2019-12-29T10:27:36.414Z DEBUG lightningd(2175): lnd.pubkey chan #3: Funding tx txid depth 1724 of 3
Dec 29 10:27:36 base lightningd[2175]: 2019-12-29T10:27:36.415Z DEBUG lightningd(2175): lnd.pubkey chan #3: Funding tx txid confirmed, but peer disconnected
Dec 29 10:29:28 base lightningd[2175]: 2019-12-29T10:29:28.021Z DEBUG lightningd(2175): lnd.pubkey chan #3: Will try reconnect in 300 seconds
Dec 29 10:29:28 base lightningd[2175]: 2019-12-29T10:29:28.021Z DEBUG lightning_connectd(2202): Failed connected out for lnd.onion5: Connection establishment: Operation now in progress. lnd.pubkey:9735: Connection establishment: Operation now in progress.
Dec 29 10:34:40 base lightningd[2175]: 2019-12-29T10:34:40.642Z DEBUG lightningd(2175): lnd.pubkey chan #3: Will try reconnect in 300 seconds
Dec 29 10:34:40 base lightningd[2175]: 2019-12-29T10:34:40.642Z DEBUG lightning_connectd(2202): Failed connected out for lnd.onion5: Connection establishment: Operation now in progress. lnd.pubkey:9735: Connection establishment: Operation now in progress.
Dec 29 10:39:50 base lightningd[2175]: 2019-12-29T10:39:50.705Z DEBUG lightningd(2175): lnd.pubkey chan #3: Will try reconnect in 300 seconds
Dec 29 10:39:50 base lightningd[2175]: 2019-12-29T10:39:50.705Z DEBUG lightning_connectd(2202): Failed connected out for lnd.onion5: Connection establishment: Operation now in progress. lnd.pubkey:9735: Connection establishment: Operation now in progress.
Dec 29 10:42:37 base lightningd[2175]: 2019-12-29T10:42:37.806Z DEBUG lightningd(2175): lnd.pubkey chan #3: Got depth change 1724->1725 for txid
Dec 29 10:42:37 base lightningd[2175]: 2019-12-29T10:42:37.806Z DEBUG lightningd(2175): lnd.pubkey chan #3: Funding tx txid depth 1725 of 3
Dec 29 10:42:37 base lightningd[2175]: 2019-12-29T10:42:37.807Z DEBUG lightningd(2175): lnd.pubkey chan #3: Funding tx txid confirmed, but peer disconnected
Dec 29 10:44:38 base lightningd[2175]: 2019-12-29T10:44:38.113Z DEBUG lightningd(2175): lnd.pubkey chan #3: Got depth change 1725->1726 for txid
Dec 29 10:44:38 base lightningd[2175]: 2019-12-29T10:44:38.113Z DEBUG lightningd(2175): lnd.pubkey chan #3: Funding tx txid depth 1726 of 3
Dec 29 10:44:38 base lightningd[2175]: 2019-12-29T10:44:38.114Z DEBUG lightningd(2175): lnd.pubkey chan #3: Funding tx txid confirmed, but peer disconnected
Dec 29 10:45:04 base lightningd[2175]: 2019-12-29T10:45:04.179Z DEBUG lightningd(2175): lnd.pubkey chan #3: Will try reconnect in 300 seconds
Dec 29 10:45:04 base lightningd[2175]: 2019-12-29T10:45:04.179Z DEBUG lightning_connectd(2202): Failed connected out for lnd.onion5: Connection establishment: Operation now in progress. lnd.pubkey:9735: Connection establishment: Operation now in progress.
Dec 29 10:50:17 base lightningd[2175]: 2019-12-29T10:50:17.800Z DEBUG lightningd(2175): lnd.pubkey chan #3: Will try reconnect in 300 seconds
Dec 29 10:50:17 base lightningd[2175]: 2019-12-29T10:50:17.800Z DEBUG lightning_connectd(2202): Failed connected out for lnd.onion5: Connection establishment: Operation now in progress. lnd.pubkey:9735: Connection establishment: Operation now in progress.
Dec 29 10:52:09 base lightningd[2175]: 2019-12-29T10:52:09.010Z DEBUG lightningd(2175): lnd.pubkey chan #3: Got depth change 1726->1727 for txid
Dec 29 10:52:09 base lightningd[2175]: 2019-12-29T10:52:09.010Z DEBUG lightningd(2175): lnd.pubkey chan #3: Funding tx txid depth 1727 of 3
Dec 29 10:52:09 base lightningd[2175]: 2019-12-29T10:52:09.011Z DEBUG lightningd(2175): lnd.pubkey chan #3: Funding tx txid confirmed, but peer disconnected
Dec 29 10:53:09 base lightningd[2175]: 2019-12-29T10:53:09.182Z DEBUG lightningd(2175): lnd.pubkey chan #3: Got depth change 1727->1728 for txid
Dec 29 10:53:09 base lightningd[2175]: 2019-12-29T10:53:09.183Z DEBUG lightningd(2175): lnd.pubkey chan #3: Funding tx txid depth 1728 of 3
Dec 29 10:53:09 base lightningd[2175]: 2019-12-29T10:53:09.184Z DEBUG lightningd(2175): lnd.pubkey chan #3: Funding tx txid confirmed, but peer disconnected
Dec 29 10:54:09 base lightningd[2175]: 2019-12-29T10:54:09.448Z DEBUG lightningd(2175): lnd.pubkey chan #3: Got depth change 1728->1729 for txid
Dec 29 10:54:09 base lightningd[2175]: 2019-12-29T10:54:09.449Z DEBUG lightningd(2175): lnd.pubkey chan #3: Funding tx txid depth 1729 of 3
Dec 29 10:54:09 base lightningd[2175]: 2019-12-29T10:54:09.450Z DEBUG lightningd(2175): lnd.pubkey chan #3: Funding tx txid confirmed, but peer disconnected
Dec 29 10:55:31 base lightningd[2175]: 2019-12-29T10:55:31.039Z DEBUG lightningd(2175): lnd.pubkey chan #3: Will try reconnect in 300 seconds
Dec 29 10:55:31 base lightningd[2175]: 2019-12-29T10:55:31.039Z DEBUG lightning_connectd(2202): Failed connected out for lnd.onion5: Connection establishment: Operation now in progress. lnd.pubkey:9735: Connection establishment: Operation now in progress.
Dec 29 11:00:41 base lightningd[2175]: 2019-12-29T11:00:41.322Z DEBUG lightningd(2175): lnd.pubkey chan #3: Will try reconnect in 300 seconds
Dec 29 11:00:41 base lightningd[2175]: 2019-12-29T11:00:41.322Z DEBUG lightning_connectd(2202): Failed connected out for lnd.onion5: Connection establishment: Operation now in progress. lnd.pubkey:9735: Connection establishment: Operation now in progress.

LND logs:

Dec 29 10:42:59 nodl lnd[2600]: 2019-12-29 10:42:59.721 [WRN] SRVR: Already have 1 persistent connection requests for [email protected]:9735, connecting anyway.
Dec 29 10:43:02 nodl lnd[2600]: 2019-12-29 10:43:02.984 [ERR] SRVR: Unable to connect to [email protected]:9735: socks connect tcp 127.0.0.1:9050-> [email protected]:9735: unknown error connection refused
Dec 29 10:52:49 nodl lnd[2600]: 2019-12-29 10:52:49.255 [WRN] SRVR: Already have 1 persistent connection requests for [email protected]:9735, connecting anyway.
Dec 29 10:52:51 nodl lnd[2600]: 2019-12-29 10:52:51.655 [ERR] SRVR: Unable to connect to [email protected]:9735: socks connect tcp 127.0.0.1:9050-> [email protected]:9735: unknown error connection refused
Dec 29 10:55:52 nodl lnd[2600]: 2019-12-29 10:55:52.702 [WRN] SRVR: Already have 1 persistent connection requests for [email protected]:9735, connecting anyway.
Dec 29 10:55:53 nodl lnd[2600]: 2019-12-29 10:55:53.119 [ERR] SRVR: Unable to connect to [email protected]:9735: socks connect tcp 127.0.0.1:9050-> [email protected]:9735: unknown error connection refused

I replaced any sensitive informations, hope it won't make it unreadable :)

It seems it's connection specific bug/issue for these two peers.

As mentioned in #62:

It would be nice, if the dockerfile would include regtest setups of c-lightning and bitcoind to check if the middleware is still capable of communicating / routing traffic with them.

We have a travis-ci.sh script which Travis runs on every commit. At time of writing, the make docker-build-go target is invoked by Travis, but we can easily add more make targets to add regression tests, as described above.

We might also want to check that all links in docs/ resolve, as suggested here.

This issue can be closed once integration tests have been added to CI, or if we decide not to pursue this work.

To test and run the services that we use on the base on local machines, it would be useful to have these services running in docker containers with their respective ports exposed. This should also include some easy methods to mine blocks, create transactions and open lightning channels.