Description

After introducing the read-only root file system, the timezone can no longer be set with timedatectl set-timezone.

How to Reproduce

Steps to reproduce the behavior:

1. Log in by ssh
2. Set a new timezone, e.g.: timedatectl set-timezone UTC
3. Reboot remote
4. Check timezone with date: back to CEST

Expected behavior

Timezone can be set and is persisted during reboots.

Your Environment

• Version used: latest master
• Running on:
  • YIO Remote
  • Standalone Raspberry Pi

Additional context

The /etc file system overlay is only an in-memory overlay which is not persisted during reboots.

Same task as YIO-Remote/remote-software#349

Expected Behavior or Design

The YIO Remote software is licensed under GPL v3 or later (see license information in each GitHub repository). This should not just be reflected in the LICENSE.md file but also in every source file.

Current Behavior or Design

Almost no source file has a license information.

Possible Solution

Add the same file header to each and every source file.
The new wifi control source files in feature/322-WiFi_rewrite branch have a file header defined. I suggest using this header as a template. See below in detailed description.

Detailed Description and Additional Information

• How to use GNU licenses for your own software (GNU docs)
• Using the SPDX License List for Tagging and Linking

Suggested license template for new files

I.e. entirely written by one author:

/******************************************************************************
 *
 * Copyright (C) <YEAR(S)> <AUTHOR> <EMAIL>
 *
 * This file is part of the YIO-Remote software project.
 *
 * YIO-Remote software is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 *
 * YIO-Remote software is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with YIO-Remote software. If not, see <https://www.gnu.org/licenses/>.
 *
 * SPDX-License-Identifier: GPL-3.0-or-later
 *****************************************************************************/

Suggested license template for derived files

I.e. using code from other GPL v3 compatible sources:

/******************************************************************************
 *
 * Copyright (C) <YEAR(S)> <AUTHOR> <EMAIL>
 *
 * Third party work used:
 *
 * <PROJECT_DESCRIPTION>
 * Copyright (C) <YEAR(S)> <ORIGINAL_AUTHOR> <ORIGINAL_EMAIL>
 * Licensed under <LICENSE>.
 *
 *
 * This file is part of the YIO-Remote software project.
 *
 * YIO-Remote software is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 *
 * YIO-Remote software is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with YIO-Remote software. If not, see <https://www.gnu.org/licenses/>.
 *
 * SPDX-License-Identifier: GPL-3.0-or-later
 *****************************************************************************/

Example wifi_wpasupplicant.h

/******************************************************************************
 *
 * Copyright (C) 2019 Markus Zehnder <[email protected]>
 *
 * Third party work used:
 *
 * DigitalRooster - QT/QML internet radio, podcast player and alarmclock.
 * Copyright (C) 2018 Thomas Ruschival <[email protected]>
 * Licensed under GPL 3.0 or later.
 *
 * wpaCute - A graphical wpa_supplicant front end.
 * Copyright (C) 2018 [email protected]
 * Licensed under BSD license.
 *
 *
 * This file is part of the YIO-Remote software project.
 *
 * YIO-Remote software is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 *
 * YIO-Remote software is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with YIO-Remote software. If not, see <https://www.gnu.org/licenses/>.
 *
 * SPDX-License-Identifier: GPL-3.0-or-later
 *****************************************************************************/

Expected Behavior or Design

After a system update the system needs to be supervised if the update was successful:

• Is the remote is able to boot the updated system?
• Does the new app start successfully?

If the update doesn't work, the system needs to be automatically revert to the previous state.

Possible Solution

• U-Boot boot counter
• Successful boot flag in U-Boot environment, set by the remote app or a watchdog

Detailed Description and Additional Information

Boot count example: https://afterhourscoding.wordpress.com/2020/07/26/integrating-swupdate-with-u-boot/

Expected Behavior or Design

Minimal proof of concept setup with U-Boot and SWUpdate:

• Two identical root partitions
• U-Boot chooses one partition based on an environment variable

Possible Solution

For the system update feature the following tools are required in remote-os and must be added and configured in Buildroot:

• U-Boot
• SWUpdate
• Updated partition layout
  • Enhanced SD image build script to create two partitions.

Expected Behavior or Design

System update images may only be applied to matching devices.

Possible Solution

Use SWUpdate's hardware-compatibility feature.

Detailed Description and Additional Information

Besides the remote software image there are currently two developer images: rpi0w and rpi3.
These images are not compatible with the remote device. Therefore we need to prevent updating a device with an invalid image.

Expected Behavior or Design

Faster boot and load time.

Current Behavior or Design

Currently the boot and load time takes quite a bit.

Possible Solution

I believe with optimising systemd services and their order we can improve the boot time.

Detailed Description and Additional Information

https://www.youtube.com/watch?v=4Fjfqz6FxC8

Description

The Docker build image is out of date and doesn't work anymore with the current YIO GitHub repositories.

How to Reproduce

Steps to reproduce the behavior:

1. Follow documentation
2. Error

Expected behavior

Works as described.

Provide a step by step development setup guide to build a SD card image.

The documentation should be written for non Linux geeks so that non-Linux developers can spin up a fresh Linux VM and create a successful build.

Expected Behavior or Design

For simple system updates there should be a possibility to upload a system image to the remote and trigger the update.

Possible Solution

As proof of concept use the Mongoose daemon mode from SWUpdate.

Mongoose is a daemon mode of SWUpdate that provides a web server, web interface and web application.

Detailed Description and Additional Information

https://sbabic.github.io/swupdate/mongoose.html

The /boot partition is named "NO NAME" which is not very meaningful and hard to identify.
Especially as a developer when dealing with multiple SD cards in different projects.

TODO: set a descriptive name. For example "YIO BOOT"

Expected Behavior or Design

The Docker build image is always available from a public Docker registry.

Current Behavior or Design

Docker build image is no longer available from Google cloud registry.

Possible Solution

Move image to Docker hub.
Setup automated builds as for the GitHub cross compile action Docker image.

Expected Behavior or Design

We need a flexible way to define environment specific configuration values, e.g. log directory, remote-os version, directories where remote-software is installed etc.

I expect these settings to be defined as environment variables. This allows a high degree of flexibility and supports different environments. Environment variables can easily be read from any script and programming language. For testing they can also be easily overridden.

This allows the software update script to retrieve all necessary information without hardcoded paths etc.
The remote-software can easily determine if it is running on the YIO remote hardware and the version of remote-os.

Current Behavior or Design

Hard coded values in scripts and remote software.

Possible Solution

• Prefix YIO related environment variables with YIO_
• Define all ENV variables in /etc/profile.d/yio.sh
  This script will automatically be sourced from /etc/profile at startup.

Initially the following variables should be defined:

• YIO_HOME: installation directory of remote-software
• YIO_OS_VERSION: remote-os version number
• YIO_LOG_DIR: logging directory

Further variables could be defined for the configuration directory / file(s), the WLAN interface name, individual log file names etc.
What needs to be considered though is the YIO remote hardware configuration file and which values take precedence.

Detailed Description and Additional Information

• https://serverfault.com/questions/375252/what-is-the-best-place-to-setup-system-wide-environment-variables-on-linux
• https://wiki.debian.org/EnvironmentVariables

Is there a particular reason that the boot partition is mounted as /mnt/boot?
Besides the www copy script, are there any accesses from the remote app?

I would like to change it to the default /boot mount point.

Reasons:

• Using other development hardware requires additional software and installation scripts, e.g. Pimoroni's HyperPixel which changes the /boot/config.txt file.
• It's the standard mount point :-)

Expected Behavior or Design

The web-configurator can be updated from a distribution archive to allow auto-updates from new releases on GitHub.

Current Behavior or Design

The web-configurator must be updated manually, or the user must wait until a new remote-os SD card image is made available.

Possible Solution

Same concept as with the remote-software update feature. See #35.

Detailed Description and Additional Information

The web-configurator distribution archive of a GitHub release contains a version.txt file holding the release version. This will be be used to check against new releases on GitHub.
Initially, there will be no compatibility check, if a new release is compatible with the installed remote-software version. This might be added in a future release.

Description

After introducing the read-only root file system, ssh keys can no longer be copied to the device with ssh-copy-id root@yio-remote.

How to Reproduce

Steps to reproduce the behavior:

1. Generate your local SSH keys with ssh-keygen
2. Copy your public key to the remote: ssh-copy-id root@yio-remote
3. Enter root password at prompt
4. See error: mkdir: can't create directory '.ssh': Read-only file system

Expected behavior

SSH keys can be copied to remote and are persisted in the data partition.

Your Environment

• Version used: latest master
• Running on:
  • YIO Remote
  • Standalone Raspberry Pi

Additional context

Error reason: there's no overlay for /root or bind-mount for /root/.ssh

Provide a step by step development setup guide to build a SD card image on macOS.

Expected Behavior or Design

Use an official cross-compilation toolchain for shorter rebuilds.

Current Behavior or Design

Using an internal cross-compilation toolchain with gcc 9, which doesn't provide any benefits to the external Arm toolchain, just longer build times.

Possible Solution

Setting BR2_TOOLCHAIN_EXTERNAL=y

Detailed Description and Additional Information

https://buildroot.org/downloads/manual/manual.html#_cross_compilation_toolchain

Migrate to the current kernel. The 4.19 kernel is planned to be an LTS kernel (long term support) afaik.
This ensures support of newer devices, e.g. using a Pi 4 for development which isn't supported with 4.14.

Main goals:

• Use current Buildroot LTS release 2020.02.x
• Update RPI kernel version to the defined 4.19.x version in the RPi Zero W Buildroot defconfig

Open issues:

• wpa_supplicant control socket is no longer created. Probably because of systemd configuration or the following change: buildroot/buildroot@c27708e
  --> works with provided wpa_supplicant.conf in /boot. Must be an initial configuration issue.
• Bluetooth serial console doesn't work anymore added in #56
  --> Bt pairing doesn't work anymore due to security changes, looks like we need a pairing agent now
• Update build toochain: releases built with Qt 5.12.4 don't run on the new Qt 5.12.8!
  • Test if app cross-compiled with Qt 5.12.8 runs on the current remote-os with Qt 5.12.4
    --> doesn't work, even though Qt promises backward & forward binary compatibility in patch releases 👎
  • Update GitHub actions with Qt 5.12.8

Expected behavior
Use case: Initial boot (fresh SD card image) with wpa-supplicant configuration file provided in boot/wpa_supplicant.conf

1. Sets WiFi configuration in initial boot without AP
2. Remote application starts using the configured network
3. Configuration web server doesn't run

Current behavior

• WiFi configuration is taken from provided configuration file
• Remote app detects WiFi setup mode: displays configuration screen to open a web browser and navigate to yio.remote
• Only option on the screen is shutdown

Detailed description
Configuration mode is triggered by control file: /wifisetup. This file may not be created if a wifi configuration file is provided by the user.

See related issues:

• #12: WiFi cfg copy from /boot
• #13: Improve WiFi setup scripts

Possible implementation
Check for provided configuration file and don't create /wifisetup control file.

Let's try to start thinking about it how to do it.

For inspiration:
https://mkrak.org/2018/01/10/updating-embedded-linux-devices-part1/

The build process, mainly the Docker based build image, works, but includes many hacks.
For a future release management and to be able to create reproducible builds the build process needs to be improved.

Goals:

• Docker build image:
  • Check if it's possible to bind mount source repositories
  • Enhance build log file: include output from info and other relevant information
  • Buildroot specific build commands (e.g. make source, make menuconfig, savedefconfig, etc)
  • Buildroot: option to specify defconfig (e.g. for development image)
• Use separated output directories for build artefacts:
  • Architecture (arm, Linux x64, macOS, Windows)
  • Debug & release per architecture
• Include metadata (timestamp, version, Git commit, etc.)
• Files to copy from remote-software in the build process:
  • remote
  • plugins
  • config.json
  • translations.json
  • fonts
  • icons
  • images
• Remove binary files in source repositories:
  • remote-os: remote app, plugins, config.json, translations.json, fonts, icons, images
  • remote-software: plugins
• remote-os build: option to use binary release of remote-software & integration plugins
• Look into using a main Makefile or Qt project

In my developer image journald constantly uses around 20% CPU

63     1 root     S    77932  20%   0% /usr/lib/systemd/systemd-journald

Assumptions:

• journald needs to configured for the slow file system
• some processes produce too many log entries.
  Journals 10min after an initial boot:

# ls -lah /var/log/journal/33de407d550a438b9dd25f23caf18d76/
total 61M
drwxr-sr-x    2 root     systemd-    4.0K Oct 28 20:33 .
drwxr-sr-x    4 root     systemd-    4.0K Feb 14  2019 ..
-rw-r-----    1 root     systemd-    6.1M Oct 28 20:48 system.journal
-rw-r-----    1 root     systemd-    6.1M Oct 28 20:33 system@1196ca2117c446808dc8df67a32609c4-0000000000000001-000581d7e395712f.journal
-rw-r-----    1 root     systemd-    6.1M Oct 28 20:39 user-1000.journal
-rw-r-----    1 root     systemd-    6.1M Oct 28 20:33 user-1000@3b7f731e37034dd2b4b1430f977baa13-0000000000000119-000581d7e4138088.journal
-rw-r-----    1 root     systemd-    6.1M Oct 28 20:34 user-1002.journal
-rw-r-----    1 root     systemd-    6.1M Feb 14  2019 user-1002@efc231c617cb4ab795d19718bae1a967-000000000000013b-000581d7e41cd781.journal
-rw-r-----    1 root     systemd-    6.1M Oct 28 20:34 user-1007.journal
-rw-r-----    1 root     root        6.1M Oct 28 20:33 user-1007@c75fdcc1c578477382842e8bb5badd25-00000000000000d8-000581d7e3b93a37.journal
-rw-r-----    1 root     systemd-    6.1M Oct 28 20:34 user-1008.journal
-rw-r-----    1 root     systemd-    6.1M Feb 14  2019 user-1008@f08417c667f649a3a61ff6834d6e7f8a-0000000000000115-000581d7e41091f0.journal

TODO:

• check if this only happens in the developer image or also in the regular image
• check if this is new with the 4.14.98 kernel
• verify configuration

Ressources:

• https://www.freedesktop.org/software/systemd/man/journald.conf.html
• Google for systemd-journald high cpu usage raspberry pi: About 341’000 results!

Expected Behavior
A wpa-supplicant configuration file in the boot partition named wpa_supplicant.conf will override the WiFi configuration.

• Initial boot: no access point shall be created and the remote uses the provided configuration to setup the network
• Reboot with or without existing WiFi setup: existing configuration shall be overwritten

Current Behavior
All tests were performed with a valid boot/wpa_supplicant.conf file.
Use cases:

1. Initial Boot: corrupt configuration
   • The wpa_supplicant.conf is indeed copied to /etc/wpa_supplicant/wpa_supplicant-wlan0.conf but the WiFi setup gets corrupted:
   • Static IP address is set from the access point configuration
   • Device doesn't connect to the network
   • No access point is started
2. Reboot with a working WiFi network: OK
3. Wifi reset: corrupt configuration until device is rebooted
   • Device looses network connection and no access point is started
4. Wifi reset & reboot: OK

Steps to Reproduce
Non-deterministic, about 9 out of 10 times it works.

Expected Behavior or Design

When using a SD card which is larger than the remote-os (> 1 GB), the root partition should auto-expand to use available space. Otherwise the root partition remains at ~ 500 MB.

Current Behavior or Design

Partition remain the same as defined in the SD card image.

Possible Solution

At first boot auto-expand the root partition. Maybe add a marker file / configuration option in the /boot partition to enable / disable auto-expansion.

Detailed Description and Additional Information

For a possible solution to imitate how Raspian expands the file system: https://www.raspberrypi.org/forums/viewtopic.php?t=174434#p1113971

Migrate YIO specific files from cloned buildroot repo and create a buildable project.

Main goals:

• Use buildroot as Git submodule
• Use the same buildroot release
• Use the same RPI kernel sources
  • Set remote Git tar release URL as source instead of manually downloaded local file
• Build the same binary SD image as with the old project
• Create initial master commit to use dev branch for further migrations

Description

building the latest remote-os with SKIP_BUILD_IMAGE leads to a make error at the end:

WARN: not building SD card image: disabled with SKIP_BUILD_IMAGE
make[1]: Leaving directory '/home/yio/projects/yio/remote-os/buildroot'
cp -f /home/yio/projects/yio/remote-os/buildroot/output/images/yio-* /home/yio/projects/yio/remote-os/release/
cp: cannot stat '/home/yio/projects/yio/remote-os/buildroot/output/images/yio-': No such file or directory
make: ** [Makefile:37: remote] Error 1

It looks like it tries to copy the yio images but they cannot be found because they are skipped in the build.

How to Reproduce

Steps to reproduce the behavior:

1. checkout latest remote-os
2. make SKIP_BUILD_IMAGE=y
3. wait
4. do not profit because it exits with an error :(

Expected behavior

finish build without errors

Your Environment

• Operating System and version if not running on YIO Remote: Ubuntu 20.04 Focal

During Initial Setup the Pi is not able to scan or connect to (European) WiFi that is using CH12,13 or 14. First the WiFi Region needs to be set.

Expected behavior
Remote application starts as fast as possible (within seconds) on a RPi zero, no matter if the original hardware is used or on a development setup with missing sensors or input devices.

Current behavior
On my development setup which doesn't have a touch input device driver the application takes over 5 minutes to start. After the initial start it can be closed and restarted within seconds.

Detailed description
The problem is that the random number generator doesn't get enough entropy.
journald revealed it:

-- Reboot --
Oct 24 19:38:09 YIO-Remote ntpd[110]: receive: Unexpected origin timestamp 0xe00fbd81.eb03fb44 does not match aorg 0000000000.00000000 from [email protected] xmt 0xe15c6001.f8ba28fa
3f:1f:dc:38:20:f7:bc:e5:7d:5c:97:b6:96:47:e4:85:01:c1:ce:55 from 172.16.16.197:62317
Oct 24 19:38:30 YIO-Remote systemd[1]: systemd-hostnamed.service: Succeeded.
Oct 24 19:43:49 YIO-Remote ntpd[110]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized
Oct 24 19:44:09 YIO-Remote kernel: random: crng init done
Oct 24 19:44:09 YIO-Remote kernel: random: 7 urandom warning(s) missed due to ratelimiting
Oct 24 19:44:09 YIO-Remote remote[97]: QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
Oct 24 19:44:10 YIO-Remote remote[97]: evdevtouch: Cannot open input device /dev/input/event0 (No such file or directory)

Device booted up: 19:38:09
crng init done: 19:44:09
Remote app finally started: 19:44:09

Possible implementation

Install rng-toolstools:
https://unix.stackexchange.com/questions/522271/rpi-buildroot-random-crng-init-done-not-enough-entropy-how-to-configure

Expected Behavior or Design

Use the current Buildroot LTS version to keep up to date with patches, security fixes and newer Qt version.

Current Behavior or Design

Buildroot 2020.02 is currently used, which is no longer supported with new LTS release.

Possible Solution

a) Full major upgrade from 2020.02 to 2021.02. This means migrating quite a few components and breaking changes:

• Upgrade to Buildroot 2021.02
• Migrate to Qt 5.15
• Migrate Linux kernel to 5.4
• Migrate RPi DTS

b) Two step migration:

1. Update Buildroot to 2021.02 and keep RPi 4.19.x kernel
2. Update RPi Kernel to 5.4.x

Detailed Description and Additional Information

This might be a bumpy migration, especially migrating the Linux kernel and DTS definition.

Option b) looks like the better migration path. This would give us an updated Qt 5.15 and all system packages would be up-to-date and supported until April 2022.

Is your feature request related to a problem? Please describe.
Currently the remote mounts the /boot and / partitions as read-write and writes logs to /var/log. Over time this will cause the Micro SD card to wear out.

Describe the solution you'd like
Ideally the remote would re-mount the partitions as read-only, and have the logs kept in tmpfs (i.e. volatile storage). Optionally these could also be redirected to an external syslog server.

Describe alternatives you've considered
Usage of endurance micro SD cards will extend the life of the card obviously, however for the majority of the time since r/w access & logs aren't needed this seems like a wasteful use of endurance.

Expected Behavior or Design

Remote app plugins can be updated from a distribution archive to allow auto-updates from new releases on GitHub.

Current Behavior or Design

The plugins must be updated manually, or the user must wait until a new remote-os SD card image is made available.

Possible Solution

Same concept as with the remote-software update feature. See #35.

Detailed Description and Additional Information

The plugin distribution archive is in a similar format as the app archive described in #35.

The Qt plugin provides metadata in json format. One attribute is the current release version. This will be be used to check against new releases on GitHub.
Initially, there will be no compatibility check, if a new release is compatible with the installed remote-software version. This might be added in a future release.

For developing low level functions for the YIO-Remote it's best to run the application on the same environment as the final release. Issues pop up much earlier and it's easier to debug the cause.

Goals:

• Base the image on the develop branch
  • Whenever possible only add additional tools
  • Libraries must remain in the same version, otherwise the develop branch must be upgraded as well
• Add required tools for Qt remote interactions:
  • rsync
  • sftp
  • gdb server for remote debugging
• Add wpa-supplicant configuration file template in boot partition
• Add config.txt & cmdline.txt templates for HDMI output in boot partition
• Add support for additional hardware
  • Device overlays and drivers for Pimorony's HyperPixel 3.5 screen
  • Device overlays and drivers for Pimorony's HyperPixel 4.0 screen
• Detailed logging
• Optional: add USB support to Kernel
  • Optional: USB gadget drivers for serial console and network

Expected behavior
Use cases:

• Initial boot or after a WiFi reset:

  1. wpa-supplicant configuration doesn't exist, remote is not connected to network.
  2. AP is started
  3. After user sets WiFi configuration in web site the wpa-supplicant configuration is created, AP terminated and remote connects to network.

• wpa-supplicant configuration file provided in boot/wpa_supplicant.cfg

  1. Sets WiFi configuration in initial boot without AP
  2. Overrides current wpa-supplicant configuration if already connected to network

Current behavior

• Mostly it works. I would say at least 9 out of 10 times.
• WiFi configuration can get corrupted if something fails during startup configuration
• Sometimes the remote screen shows an error after connecting to the network:
  "Not connected .... Please try again", even though the device was able to connect to the WiFi network.

Detailed description
See related issue #12: WiFi cfg copy from /boot

Possible implementation
Improving the setup process:

• add error handling and logging to be able to find out where it fails
• additional consistency checks
• increase timeout for WiFi connection detection during setup process. Probably remote-software needs to be changed.

/dev/watchdog

The web configuration redirect issues in #8 has been fixed in master and dev.
However, the web server configuration is inconsistend and doesn't work if WiFi has been setup by the configuration file in /boot.

@nklerk and @martonborzak How and when shall the web-configurator be accessible?

Current behavior:

• After WiFi setup: lighttp configuration is switch to configuration mode
  • lighttp configuration setup is active (/etc/lighttpd/lighttpd-config.conf)
  • web-configurator is accessible
  • after reboot lighttp is switched off
• WiFi configuration provided by /boot/wpa_supplicant.conf in initial boot
  • default lighttp configuration active (/etc/lighttpd/lighttpd.conf)
  • web-configurator NOT accessible. 403

Suggested change:

• Same lighttpd configuration after access point mode or wifi-copy
• Default lighttpd configuration: web-configurator

Open questions:

• leave lighttpd running after access point setup?
• start lighttpd after device reboot? See #10

• add partition to hold the original image for recovery
• add partition to hold the downloaded image for whole system update

Current state:

• Reboot: webserver only runs if WiFi setup hasn't been completed yet
• After WiFi setup: webserver keeps running until remote is rebooted

Question:

@nklerk Will the webserver be used for the web-configurator or something else?

Proposed change:

• If required: change lighttpd service state to enabled
• Not required: shutdown lighttpd after WiFi setup

rootfs overlay/urs/lib/fonts/ can be deleted as fonts were moved to /usr/bin/yio-remote/fonts/

The remote-os build involves a few setup steps and has many dependencies. Furthermore Linux is required, which makes it harder to use for non-Linux developers.

Goals:

• "One command" build which creates a ready to use SD card image as output
• Easy to switch between Git branches: build master, build dev, etc.
• Option to pull latest Git sources
• Parameters for common buildroot tasks: clean, defconfig, etc.
• Manual mode: shell access to container
• Volumes for output, Git sources, etc.
• Works on:
  • Linux
  • macOS Mojave (10.14)
  • Windows 10

Related issues:

• #4 Document Linux development setup
• #5 Document macOS development setup

Possible implementation:

Build the SD image of the current dev branch of the remote-os Git repository:

docker run yio-remote/build remote-os dev --pull

Request:

GET / HTTP/1.1
Host: 10.2.1.217
Connection: keep-alive
DNT: 1
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9,nl;q=0.8

Response:

HTTP/1.1 302 Found
Location: http://yio.remote/index.php
Content-Length: 0
Date: Sat, 28 Sep 2019 17:53:01 GMT
Server: lighttpd/1.4.53

Expected Behavior or Design

The remote-app can be updated from an update archive to allow auto-updates from new releases.

Current Behavior or Design

A proof of concept script.

Possible Solution

• remote-software app is responsible for:
  • Checking for available updates.
    Either manually in the settings menu, or automatically if auto-update is enabled
  • User interaction & feedback
  • Downloading latest update archive.
• Update script in remote-os is response for:
  • Archive verification und unpacking
  • Calling optional pre-install script within archive
  • Stopping app
  • Replacing current app with new app
  • Old app is saved as backup.
    An old backup will be removed.
  • Calling optional post-install script within archive
  • Replacing the update script if the update archive contains a newer version
  • Starting app or rebooting remote

Detailed Description and Additional Information

• App install location is read from $YIO_HOME
• Update is logged in a dedicated log file. Directory is taken from $YIO_LOG_DIR_UPDATE

For a seamless update process the current directory layout must be changed:

• fully separate the remote-software project, i.e. no other resources may be included or merged together by the remote-os build.
  • move web-configurator outside the app
  • move plugins outside the app
• create a dedicated script directory
• use a common Linux location(s) to store all YIO resources

Target directory structure

   |-opt
   |---yio
   |-----app
   |-----app-plugins
   |-----media
   |-------splash
   |-----scripts
   |-----web-configurator

Update archive specification

.
├── app.tar.gz
├── build.info
├── hooks
│   ├── post-install.sh
│   └── pre-install.sh
├── md5sums
└── version.txt

• Tar or zip archive containing:
  • app.tar.gz: Data archive containing the complete /opt/yio/app folder
  • Meta information:
    • md5sums: Mandatory md5sum file of all files in the archive
    • version.txt: Mandatory text file containing the version number on the first line
    • build.info: Optional build information file
  • Optional installation hook scripts:
    • hooks/pre-install.sh: Optional pre-installation action
    • hooks/post-intall.sh: Optional post-installation action

Expected Behavior or Design

If there's a wpa_supplicant.conf file in /boot then this configuration needs to be applied to the WiFi configuration. If the configuration hasn't changed, then the copy & WiFi restart is not required.

Current Behavior or Design

At the moment the file is processed at each boot, even if the destination configuration file is the same. This involves at least a 15 second boot delay which is in most cases not necessary.

Possible Solution

Compare the /boot/wpa_supplicant.conf file if it needs to be processed, i.e. if it is the same file as currently present in the system. Either by a diff operation or by checksum comparison.

Detailed Description and Additional Information

WiFi copy script: /opt/yio/scripts/wifi-copy-config.sh
(Old location: /usr/bin/yio-remote/)

Start updating toolchain, RPi kernel and used libraries. Instead of directly using the 4.19 kernel, the latest 4.14.x version is used to gain insights of possible issues.

Main goals:

• Use latest RPI kernel in 4.14.x release branch
  This should ensure compatibility with buildroot configuration and used libraries
• Latest compatible buildroot version which successfully compiles RPI 4.14 kernel
  The current release doesn't build wpa_supplicant anymore!
  Error: undefined references to 'tls_prf_sha256'
• Document used libraries in buildroot
• Check library updates

Is your feature request related to a problem? Please describe.
A console is essential to analyze any issues on the remote. Especially if a wifi connection cannot be established, there should be another way to connect to the remote.

Describe the solution you'd like
There should be an easy way to activate a bluetooth serial console. Either through a marker file on the boot partition or a configuration entry.

Describe alternatives you've considered
Since there's no more UART or USB available, there's only Bluetooth left.

Additional context
How to manually activate a Bluetooth serial console:

killall bluetoothd
/usr/libexec/bluetooth/bluetoothd -C
sdptool add SP
hciconfig hci0 up
hciconfig hci0 piscan
rfcomm watch hci0 1 getty rfcomm0 115200 vt100

• Compatibility mode -C is required to use the deprecated sdptool.
  Unfortunately I have no idea how to setup a serial console with BlueZ 5, besides everything apparently needs to be done through dbus. If anyone knows how this is done, please let me know.
• rfcomm is a CPU hog once a console is connected. On the RPi 0 it uses ~ 50% CPU!
  Solution: patch rfcomm.c in bluez/tools and increase delay from 200ns to 200ms: https://www.raspberrypi.org/forums/viewtopic.php?p=955425#p1072405

Expected Behavior or Design

SWUpdate images are automatically being created during build.

Possible Solution

Enhance build script :-)

Detailed Description and Additional Information

I am unable to get the correct time to be displayed.

it would be nice to be able to set it manually until it can pickup the time from servers.

Do we need to support WPS?
Personally I don't like WPS, have never and will never use it, nor do I have the hardware to test it.
I just want to ask your opinion if it's an expected feature for 'regular non-techie home users'.

My opinion is: won't fix & we better spend time on more important missing features.
But if it's something that many people expect we can put it in the low priority backlog...

More information on how to implement it with wpa_supplicant: wpa_supplicant and Wi-Fi Protected Setup (WPS)

At the moment "batteryFuelGauge.capacity setting" will be overwritten when remote-software is updated. I might add a similar handling as with config.json, e.g. read it from /boot if available, otherwise use the default location.