Giter Club home page Giter Club logo

sfos-upgrade's Introduction

sfos-upgrade

Scripts for fail-safe and semi-automated upgrading of SailfishOS at the command line with logging


Upgrading SailfishOS at the GUI (via Settings→SailfishOS updates) provides very little information about its progress / process / success, beyond reading /var/log/systemupdate.log after an upgrade. This can make troubleshooting issues hard.
Furthermore the GUI offers no control which SailfishOS version to upgrade to.

In contrast to that, Jolla's guide how to upgrade SailfishOS at the command line offers full control, while lacking any logs or safety checks.
Furthermore it is tedious and error prone to issue multiple, critical commands manually at the command line.

sfos-upgrade performs the steps to upgrade SailfishOS at the command line in an automated manner, while providing extensive safety measures plus full log output at the screen and in a log file.

Safety measures:

  • Check free space on the root filesystem.
  • Check BTRFS allocation, if the root filesystem uses BTRFS.
  • Check battery state (since v1.0).
  • Check if upgrading to a correct and available SailfishOS version.
  • Check if omitting (i.e. "jumping over") a stop release (since v0.3).
  • Automatically unapply all Patches, if Patchmanager 2 is installed.
  • Stop systemd services for cron, btrfs-balance-checker etc. (since v2.2).
  • Terminate running processes, which may disturb upgrading SailfishOS (since v2.7).
  • Disable all OpenRepos' repositories, when upgrading from a SailfishOS version below 1.0.4 (since v0.4).
  • Emit a warning when downgrading.
  • Prevent downgrades, which would likely break the SailfishOS installation (since v3).

Usage (as root user):

  • sfos-upgrade [<version>]
    With a version number provided as parameter, it sets SSU to this version and in release mode before upgrading to this SailfishOS version. This is the regular use case.
    Without a version number, it retrieves the one set for SSU to perform slightly relaxed checks, but does not alter SSU's settings for upgrading. Hence the version to upgrade to and SSU's "release mode" have to be set (by e.g. ssu re <version>) before executing sfos-upgrade without a parameter.

  • sfos-upgrade --verify
    Performs a "samegrade" operation, i.e. checks if the recent versions of all available RPMs are installed and updates or installs them accordingly.
    This option was introduced with v3.7.0.

  • sfos-upgrade -?|--help
    Emits a brief usage description.

When an upgrade succeeded, reboot, and do not miss to run post_sfos-upgrade (as root) then!
When sfos-upgrade failed, run post_sfos-upgrade (as root) before rebooting and then run sfos-upgrade again.
Not running post_sfos-upgrade (in either case) will result in an huge upgrade log file (containing many duplicate entries), plus may result (as any SailfishOS upgrade at the command line without tidying efforts afterwards) in RPMs failing to install (with "unmet dependency" / "Fatal error: nothing provides X needed by Y" errors) and annoying notifications from the store-client that an upgrade to the installed version is available.

Logs are originally written to /var/log/systemupdate_*.log-dupes.txt and tidied by tidy_log-dupes (which is called by post_sfos-upgrade) to /var/log/systemupdate_*.log.txt.

Notes:

  • Built RPMs are available in the release section and for easy installation on SailfishOS at OpenRepos.
  • All operations comprise the RPMs from all enabled repositories, because that is version --dup's implicit behaviour (as with pkcon upgrade-system and zypper dist-upgrade / zypper dup too, all utilising libzypp).
  • When upgrading from a long outdated SailfishOS version (e.g. after a "factory reset"), sfos-upgrade eases and speeds up the process of upgrading to a recent SailfishOS release via consecutively installing all "stop releases" on the way.
    Simply run sfos-upgrade <intended version>, reboot, and repeat: it will guide you through all stop releases.
    When upgrading to SailfishOS releases < 4.1.0, you may omit running post_sfos-upgrade between consecutive SailfishOS upgrades (but do reboot each time!). But you shall run it after having upgraded to any SailfishOS release ≥ 4.1.0 or the ultimately intended version.
  • To ensure that a SailfishOS installation is complete and up to date, use sfos-upgrade --verify; this will "samegrade" to the installed version, which is as close as one can get to the version --verify lost since SailfishOS 2.2.1 (which only checked for missing or not up-to-date RPMs, while "samegrading" will also install them) without zypper. With it (per pkcon install zypper), a zypper refresh && zypper verify --dry-run comes closer to what version --verify did (only checking). While zypper can also be used for up-/down-/same-grading, that is rather a "last resort"-measure than the primary recommendation, hence use sfos-upgrade for that.
  • sfos-upgrade supports all public SailfishOS releases and should work fine for upgrading from / to any release (it accepts only version numbers of at least 1.0.0.0 at the command line, but omits this check when called without a parameter after utilising ssu to pre-set a version to upgrade to).
  • sfos-upgrade is simply a frontend for ssu re and version --dup, performing a multitude of checks before initiating the upgrade proper, while post_sfos-upgrade carries out the "Final clean up" steps from Jolla's manual upgrade guide plus an also necessary pkcon refresh, which some seem to omit when upgrading manually at the command line (often running into aforementioned issues later, then).

sfos-upgrade's People

Contributors

direc85 avatar okxa avatar olf0 avatar pcfe avatar sebix avatar self-perfection avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

sfos-upgrade's Issues

[Bug] upgrade_version: unbound variable

SailfishOS VERSION (Settings → About product → Build): 3.4.0.24

HARDWARE (Settings → About product → Manufacturer & Product name): F(x)tec Pro¹ (qx1000, not the Pro¹X which is qx1050)

sfos-upgrade VERSION ( rpm -qi sfos-upgrade ): 3.9.13

BUG DESCRIPTION

After a fresh reflash of SFOS on a F(x)tec Pro¹ (version 3.4.0.24, which is the latest working image), I tried to upgrade to the latest SFOS version. I downloaded sfos-upgrade 3.9.13 rpm from openrepos (via a browser search).

I tried running sfos-upgrade without arguments, with any supported version as argument, with "--verify", but got this error message :

/usr/bin/sfos-upgrade: line 179: upgrade_version: unbound variable
/usr/bin/sfos-upgrade: line 179: installed_version: unbound variable
Comparing versions failed when checking, if downloading lists of SailfishOS releases is necessary.

STEPS TO REPRODUCE

  • flash a new 3.4.0.24 image on F(x)tec Pro¹
  • install latest sfos-upgrade release (3.9.13)
  • run sfos-upgrade

ADDITIONAL INFORMATION

[root@Pro1 defaultuser]# ssu re
Device release is currently: 3.4.0.24
[root@Pro1 defaultuser]# sfos-upgrade 
/usr/bin/sfos-upgrade: line 179: upgrade_version: unbound variable
/usr/bin/sfos-upgrade: line 179: installed_version: unbound variable
[root@Pro1 defaultuser]# sfos-upgrade --verify
/usr/bin/sfos-upgrade: line 179: upgrade_version: unbound variable
/usr/bin/sfos-upgrade: line 179: installed_version: unbound variable
Comparing versions failed when checking, if downloading lists of SailfishOS releases is necessary.
[root@Pro1 defaultuser]# sfos-upgrade 4.4.0.72
/usr/bin/sfos-upgrade: line 179: upgrade_version: unbound variable
/usr/bin/sfos-upgrade: line 179: installed_version: unbound variable
Comparing versions failed when checking, if downloading lists of SailfishOS releases is necessary.
[root@Pro1 defaultuser]# sfos-upgrade 4.0.1.45
/usr/bin/sfos-upgrade: line 179: upgrade_version: unbound variable
/usr/bin/sfos-upgrade: line 179: installed_version: unbound variable
Comparing versions failed when checking, if downloading lists of SailfishOS releases is necessary.
[root@Pro1 defaultuser]# pkcon get-details sfos-upgrade
Resolving                                                                                                                                                                                                               [                                                                                                           ] (0%)  More than one package matches:
1. sfos-upgrade-3.9.13-release6.noarch [installed]
2. sfos-upgrade-3.9.13-1.2.1.jolla.noarch [sailfishos-chum]

Then I installed version 3.9.12 (also from openrepos website), and it worked :

[root@Pro1 defaultuser]# pkcon get-details sfos-upgrade
Resolving                                                                                                                                                                                                               [                                                                                                           ] (0%)  More than one package matches:
1. sfos-upgrade-3.9.12-release5.noarch [installed]
2. sfos-upgrade-3.9.13-1.2.1.jolla.noarch [sailfishos-chum]

#
# at this point, I had made some tests, including a `ssu re 4.4.0.72` and forgot to reset SSU release to 3.4.0.24, which explains why at first it tries to upgrade to 4.4.0.72
#

[root@Pro1 defaultuser]# sfos-upgrade 
Notice: Upgrading from 3.4.0.24 to 4.4.0.72 would omit installing 4.0.1.48 as a stop release!
Warning: Doing so will likely break this SailfishOS installation, unfortunately often only subtly so the issues caused might be observed way later!
Do you really want to continue? (y/N)  
Aborting upon user request.
[root@Pro1 defaultuser]# sfos-upgrade 4.0.1.48
Notice: The installed version 3.4.0.24 is smaller than the one currently set for SSU (4.4.0.72).
A possible reason for this is, that the 'sailfish-osupdateservice osupdate-check' invoked by osupdate-check.service (which is regularly triggered by osupdate-check.timer) does more than just checking (observed with SailfishOS 3.2.1 and earlier): E.g., setting ssu to the recent release version or next stop release.
Never mind, the version for SSU will be set correctly again, later on.

Notice: Mind that sfos-upgrade is best run on a freshly rebooted device.

Notice: Do you want to upgrade SailfishOS from 3.4.0.24 to 4.0.1.48? (y/N) y

Notice: For troubleshooting issues with the upgrade proper, please consult https://jolla.zendesk.com/hc/en-us/articles/360005795474

- Stopping osupdate-check.timer

- Setting SSU to SailfishOS release:
Changing release from 4.4.0.72 to 4.0.1.48
Your device is now in release mode!

- Fetching and installing the SailfishOS upgrade from 3.4.0.24 to 4.0.1.48 (this may take a while):
REFRESHING CACHE AND DOWNLOADING PACKAGES
Finished transaction (status=1, runtime=805987ms)
UPGRADING SYSTEM
Finished transaction (status=1, runtime=273105ms)
FINISHING

So it's probably a regression between version 3.9.12 and 3.9.13.
This bug is maybe related to #64

[Suggestion] Consider removing SSU's caches before / after upgrading

DESCRIPTION

Consider performing
rm -r /var/cache/ssu/
ssu ur

But when? Likely both, before and after updating (on SFOS < 4.4.0), because users may start on any affected release and end on one, but never use sfos-upgrade before or after that, again.

ADDITIONAL INFORMATION

Properly support community adaptions

To properly support community adaptions of SailfishOS is probably a longer endeavor, as there currently are many unknowns (for me, at least):

  • As I do not own or have access to a device with a community adaption, I never had a chance to try sfos-upgrade on one. Hence the current, "official" state of support for community adaptions is something between "unknown" and "unsupported".
  • OTOH, @r0kk3rz' side note (the last paragraph of this comment) sounds as if sfos-upgrade does already basically work on devices with community adaptions of SailfishOS.

As sfos-upgrade up to now always uses version --dup to perform the upgrade proper, which checks if rnd-dist-upgrade is installed (usually in /usr/bin/) and uses either that or pkcon upgrade, respectively complains (since SFOS3.2.1?), I now wonder why the "zypper dance" / zypper upgrade is the primary suggestion from SailfishOS porters for upgrading installations of community adaptions.

Hence I compiled a list of simple questions, trying to pose them in a way that they can be answered with "Yes" or "No".

For my basic understanding:

  1. Does the GUI page Settings -> Sailfish OS updates exist on "community devices"?
  2. If so, does it work?
  3. Is the Jolla Store app available?
  4. If so, does it work?
    (Granted that one created an Jolla Store account, is logged in to it and only for apps free of charge.)

To understand the technical differences:

  1. Is the version shell script preinstalled (usually in /usr/bin/)?
  2. Is the rnd-dist-upgrade binary installed (usually in /usr/bin/), when "developer mode" is enabled in the SailfishOS' Settings (which I assume every user of a "community device" has)?
  3. Do you understand, why zypper upgrade is preferred over pkcon upgrade by "community porters", in contrast to Jolla's general preference of pkcon over zypper (also specifically for the upgrade case)?
    Both, pkcon and zypper use libzypp as backend (on SailfishOS) and hence should yield the same package transactions and result. Maybe zypper's more elaborate output and configurability is the only reason, why porters prefer it.

[Bug] script exits silently when current release isn't detected.

SailfishOS VERSION (Settings → About product → Build): 4.4.0.64

HARDWARE (Settings → About product → Manufacturer & Product name): Xperia 10iii

sfos-upgrade VERSION ( rpm -qi sfos-upgrade ): 3.9.9-release3

BUG DESCRIPTION

Update to 4.4.0.72 not detected

STEPS TO REPRODUCE

run

  sfos-upgrade 4.4.0.72

observe that nothing happens at all. No error msg, no updating, just nothing.

ADDITIONAL INFORMATION

From doing things like sh -x it tuens out the release is not yet listed in the docs.s.o wiki/site. So the parsing done by the script fails.

Snippet if the last few lines of sh -x sfos-update 4.4.0.72:

1.0.0.3
1.0.0.1
0.99.6.8
0.99.6.3
0.99.5.11
0.99.5.8
0.99.5.6'
++ curl -sSkL https://github.com/sailfishos/docs.sailfishos.org/raw/master/Releases/README.md
++ cut -s -d '|' -f 2,5
++ tr -d ' '
+ sailfishdocs_sfos_list=
++ echo ''
++ cut -s -d '|' -f 1
++ grep '^[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*$'
+ sailfishdocs_sfos_releases=

I realize that is not the fault of the script.

But it should output something in this case, not silently exit.

<Please consider which other pieces of information may be relevant: Denote if this is not always reproducible, if this is a regression (then name to which older version), attach relevant data such as log files or the systemd journal, provide text output via copy & paste or output-redirecton (no screenshots, please!) etc.>

Create SRPMs / support build systems

See the first try to adapt to obs with subsequent discussion and a separate discussion, the first one spawned / experimental commit a31be78.

SRPM is now provided, see this comment for details.
But I still don't know / understand, if this (an SRPM) resolves the original issue (integration into build systems, e.g. obs).

P.S.: Building SRPMs works now, per commit 41d29a4.
An SRPM can be downloaded e.g. per curl -sSLO https://github.com/Olf0/%{name}/releases/download/%{version}-%{release}/%{name}-%{version}-%{release}.src.rpm, while the original TAR.GZ source archive can be downloaded e.g. per curl -sSLo %{name}-%{version}-%{release}.tar.gz https://github.com/Olf0/%{name}/archive/%{version}-%{release}.tar.gz (that is equivalent to curl -sSLo %{source} %{source1}, when SOURCE1 was still set, i.e. before aforementioned commit 41d29a4).

btrfs_allocation is not grepped correctly on Jolla 1 v 1.0.8.19.

On Jolla 1 version 1.0.8.19 the default (on line 364) in sfos-upgrade

btrfs_allocation="$(btrfs filesystem df / | grep '^Data, ' | cut -s -f 2 -d ':' | tr ',' '\n' | tr -d ' ' | rev | grep '^Bi*G[0-9][0-9]\.[0-9][0-9]*=' | sed 's/^Bi*G//g' | tr -d '.' | rev)"

gives incorrect results.

But if used (grep '^Data' instead of grep '^Data, '):

btrfs_allocation="$(btrfs filesystem df / | grep '^Data' | cut -s -f 2 -d ':' | tr ',' '\n' | tr -d ' ' | rev | grep '^Bi*G[0-9][0-9]\.[0-9][0-9]*=' | sed 's/^Bi*G//g' | tr -d '.' | rev)"

It gives correct results

[root@Jolla nemo]# btrfs filesystem df / | grep '^Data' | cut -s -f 2 -d ':' | tr ',' '\n' | tr -d ' ' | rev | grep '^Bi*G[0-9][0-9]\.[0-9][0-9]*=' | sed 's/^Bi*G//g' | tr -d '.' | rev
total=964
used=287

I don't have other devices to test the btrfs command with, so I dont know how these differ in different versions, and what is a universal fix.

Support for themepacksupport

Hi,

Would you mind adding the pre-upgrade script for themepacksupport?

Its path is /usr/share/harbour-themepacksupport/ocr.sh

(It restores default themes/display options in one click. I have a systemd service if the upgrade is triggered via UI, but I'm pretty sure it doesn't work if the upgrade is triggered via cli.).

Br,

Fra

Failed to extract the current "stop releases"

Hey,
Is this warning dangerous for the upgrade operation? What could I do about it?

# sfos-upgrade 3.4.0.24
[D] unknown:0 - Got http_proxy from environment, will not talk to connman
Warning: Failed to extract the current "stop releases" from https://jolla.zendesk.com/hc/en-us/articles/201836347#4.1
Hence using an internal, potentially outdated list of stop releases instead.
Do you really want to continue? (Y/N)

Also, why those information is needed?
Any help would be appreciated.

Add flag to skip stop releases

For ported devices stop releases aren't always necessary or possible depending on the state of the adaptation repositories for that device.

it would be nice to tell sfos-update to skip the stop release and go straight to the version its been told to, with appropriate warnings of course

unstable parsing of battery state

Hi,

trying to use the script from sfos-upgrade-3.5-5.noarch.rpm on a Gemini PDA under SFOS 3.1.0.11 fails with the script exiting silently (exit code 127).

bash -x tracing reveals that:

  1. line 412 battery_path="/sys/class/power_supply/*battery*" is not evaluated in the loop following it, when there is only one file, /sys/class/power_supply/battery. I worked around this by removing the asterisks. Not sure why this happens though, AFAICS it shouldn't
  2. after this in Line 415, the sourcing fails. This is because when the charger is not connected, the uevent file has:
POWER_SUPPLY_STATUS=Not charging

the space causing an incorrect sourcing.
I worked around this by rewriting that section to:

if [ -e "${battery_uevents}/uevent" ]
then
  sed 's/STATUS=\(.*\)/STATUS="\1"/' "${battery_uevents}/uevent" > /tmp/battuevent$$
  source /tmp/battuevent$$
  battery_info="sourced"
  break
fi

which is a dirty, dirty hack but works for me.

[Suggestion] Save zypper package cache before installing

DESCRIPTION

Having gone through a bumpy upgrade process from a SFOS 1.x release to current on a Jolla Tablet, the idea arose that, as the upgrade process basically does

  1. download packages to temporary cache directory
  2. install everything from that cache to system
  3. delete the cache

... and repeats that process every time a particular update step is performed, why not skip step 1. by having the option to supply a prepopulated cache directory.

ADDITIONAL INFORMATION

Consider the case of a J1 or Tablet which has been reset to factory.

One would now go through the step releases to update to a certain target release. It's likely one has performed such an update previously, and it's likely the packages needed to do the upgrade have not changed since the last time.

So in case a (large enough) SD card is available, that could be used to persist the cache dir for every (stop) release, and supply it for repeated from-factory-reset upgrade tasks.

Comparing version strings issue

sfos-upgrade fails to work on Inoi R7.

[root@Sailfish nemo]#  sfos-upgrade 3.1.0.11
Notice: The installed version 3.0.0.11+omp0.1.3.10 is smaller than the one curr
ently set for SSU (3.1.0.11).
A possible reason for this is, that the \'sailfish-osupdateservice osupdate-che
ck\' invoked by osupdate-check.service (which is regularly triggered by osupdat
e-check.timer) does more than just checking (observed with SailfishOS 3.0.2 and
earlier): E.g., setting ssu to the recent release version or next stop release
.
Never mind, the version for SSU will be set correctly again, later on.

/usr/bin/sfos-upgrade: line 74: [: 11+omp0: integer expression expected
/usr/bin/sfos-upgrade: line 78: [: 11+omp0: integer expression expected
Warning from function compare_versions: Version strings "3.0.0.8" and "3.0.0.11
+omp0.1.3.10" are not comparable!
[root@Sailfish nemo]#

Upgrading from 3.0.2.8 to 3.0.3.10 on an Aigo tablet fails

Original report at OpenRepos by @dalas_revo:
"Unfortunately upgrading from 3.0.2.8 to 3.0.3.10 on the Aigo (Jolla Tablet) fails for me with sfos-upgrade, complaining about a missing /sys/class/power_supply/battery/uevent."

I might introduce an override for this after some more analysis:

  • Sounds, as if that did not happen during earlier upgrades of SailfishOS with sfos-upgrade on your Aigo tablet, or was upgrading from 3.0.2.8 to 3.0.3.10 the first use of sfos-upgrade on your Aigo?
  • What is the output of the following commands (as root) on this device?
    1. mount | fgrep sysfs
    2. ls -l /sys/class/power_supply
    3. ls -l /sys/class/power_supply/battery
    4. cat /sys/class/power_supply/battery/uevent

Checking the battery charge is important, thus I want to avoid implementing an override, if a proper solution can be introduced.

[Bug] final_sfos_releases: unbound variable

SailfishOS VERSION (Settings → About product → Build): 4.3.0.15

HARDWARE (Settings → About product → Manufacturer & Product name): Sony Xperia XA2 (4113)

sfos-upgrade VERSION ( rpm -qi sfos-upgrade ): 3.9.4-release3

BUG DESCRIPTION

/usr/bin/sfos-upgrade: line 177: final_sfos_releases: unbound variable
/usr/bin/sfos-upgrade: line 196: final_sfos_releases: unbound variable
seems the newly added feature pulling releases from sailfishos does not work.

STEPS TO REPRODUCE

start from 4.3.0.12
run sfos-update 4.4.0.64 (can't remember, which version it was run on)
stop release 4.3.0.15 get's detected
install, reboot, run post_sfos-upgrade
this refreshes repositories
at some point the new version 3.9.4.-release3 of sfos-upgrade get's installed
the remaining step sfos-update 4.4.0.64 is not successful because of error message for line 177 and 196

ADDITIONAL INFORMATION

at the time of reporting, /sailfishos/docs.sailfishos.org/master/Releases/README.md does not contain 4.4.0.64 yet, while coderus.openrepos.net/whitesoft/sailversion does

Syntax error: forgotten end-of-loop at around line 415

Within the current 2.7-1 release I found an error that prevented me from executing the script in the first run:
Within the for-loop starting from line 406 (current master branch) you forgot to end the (outer) if clause at around line 415 (between the emit_newline and the end-of-for-loop line).
Little hint: As I found out last week, bash also has an integrated syntax check with the option '-n', I used frequently since then, but maybe you know it already.
Nonetheless, great script and great work in general.

Best regards,
Frank

Unnecessary Stop Release enforced? [Bug] Unbound variable!

SailfishOS VERSION : 2.0.0.10

HARDWARE : Jolla Tablet

sfos-upgrade VERSION: 3.9.17

QUESTION

Internal list of stop releases: Is sfos-upgrade more zealous than the Jolla list?

STEPS TO REPRODUCE

Running sfos-upgrade on 2.0.0.10, wanting to update to 2.2.0.29 will output:

Notice: Upgrading from 2.0.0.10 to 2.2.0.29 would omit installing 2.0.5.6 as a stop release!

Now, the d.s.o page on Releases sais this step, 2.0.0.10 -> 2.2.0.29 should be possible.

I don't mind sfos-update being more careful than the list prescribes, but I was curious whether there was maybe some outdated internal list in the tool, or an omission on Jolla's list.

ADDITIONAL INFORMATION

By the way, sfos-upgrade 3.9.17 is capable of performing its duty on SFOS 1.1.9.30 which was a bit surprising to me but kudos to you for making it so very portable.

And more by the way, actually sfos-upgrade does give an error when running it on such old releases, namely:

[root@Jolla nemo]# sfos-upgrade 2.2.0.29
/usr/bin/sfos-upgrade: line 205: sailfishdocs_sfos_releases: unbound variable
Warning: "2.2.0.29" does not seem to be a publicly released SailfishOS version!
Do you really want to continue? (y/N) y

I haven't investigated deeply but I assume it's simply because ancient ssl certificates on such old releases will make the curl call(s) fail. Harmless in my book so I'm ignoring that.

Feature request: test the free space in /opt/alien

Use case:

  • for users of microG (like me), enabling the older mapsv1 API used by some older apps requires patching system.img
  • as far as I've heard using the original Google Play Services requires patching system.img

So if a user keeps both a backup of system.img and the patched one, the /opt partition doesn't have enough room left for an updated system.img and the upgrade fails.

Checking that the partition usage is <60% should do the trick

sfos-upgrade 3.8.3: line 182: recent_stop_releases: unbound variable

I haven't pondered it deeply but I suspect there is a typo in line 182 where $recent_stop_releases should be $recent_sfos_releases (because that is defined in line 171).

Ack!
Thank you, I wanted to get it out quickly, and was just tested per bash -n, which does not catch this kind of error.

That suspicion is deepened by the fact that renaming that variable makes everything work normally.

Exactly.

[Bug] Free space amount doubled in warning

SailfishOS VERSION: Sailfish OS 4.4.0.72 (Vanha Rauma)

HARDWARE: Gemini PDA 0.0.5.1

sfos-upgrade VERSION: 3.9.11

BUG DESCRIPTION

The "not enough free space" warning gives twice the amount that df reports.

defaultuser@GeminiPDA:~ $ df -h /; df -k /; devel-su sfos-upgrade
Filesystem                Size      Used Available Use% Mounted on
rootfs                    2.3G      1.9G    430.0M  82% /
Filesystem           1K-blocks      Used Available Use% Mounted on
rootfs                 2454256   1971912    440368  82% /
Password:
Notice: Less than 1 GiB (880736 KiB) free space on the root filesystem.
Please consider to clean up or to enlarge the root filesystem, see:
https://gitlab.com/Olf0/sailfishX#33-increasing-the-root-lvm-volume-size

Not at all sure where the 880736 KiB come from, because executing the command for $free_space in sfos-upgrade gives:
(relevant line in script)

defaultuser@GeminiPDA:~ $ df -k / | sed -n '2p' | rev | grep '^/ ' | tr -s ' ' | cut -s -f 3 -d ' ' | rev
440368
defaultuser@GeminiPDA:~ $ echo $(( 440368 * 2 ))
880736

STEPS TO REPRODUCE

df -k / && devel-su sfos-upgrade

ADDITIONAL INFORMATION

It's not a busybox vs bash thing:

defaultuser@GeminiPDA:~ $ echo $BASH_VERSION
5.0.18(1)-release

defaultuser@GeminiPDA:~ $ busybox bash -c "df -k / | sed -n '2p' | rev | grep '^/ ' | tr -s ' ' | cut -s -f 3 -d ' ' | rev"
440368
defaultuser@GeminiPDA:~ $ /bin/bash -c "df -k / | sed -n '2p' | rev | grep '^/ ' | tr -s ' ' | cut -s -f 3 -d ' ' | rev"
440368

[Bug] Erroneous allocation calculation with btrfs

SailfishOS VERSION (Settings → About product → Build): 4.5.0.21

HARDWARE (Settings → About product → Manufacturer & Product name): Pine64 PinePhone Pro

sfos-upgrade VERSION ( rpm -qi sfos-upgrade ): 3.11.1

BUG DESCRIPTION

When relying on btrfs (as opposed to btrfs-balancer), the way allocation is computed is mistaken.

this code segment gets as an inputs the Data chunks allocated, and how much file data is written into them. The ratio computed is the occupancy of the chunks themselves, this doesn't give any information about the unallocated space on the device.

Instead btrsf filesystem show should be used (or btrfs filesystem usage on more recent versions).

STEPS TO REPRODUCE

  • Format and install the root partition as btrfs
  • For maintenance, install package btrfs-utils (instead of btrfs-balancer like on the original Jolla 1).
  • Run balance (e.g.: balance start -v -dusage=80 -musage=80 /)
  • Run sfos-upgrade (e.g.: sfos-upgrade 4.5.0.24)
  • sfos-upgrade will mistakenly complain:

    Aborting: Less than 2 GiB unallocated data space (.6 GiB) on the root filesystem (BTRFS)!
    Please balance the btrfs root filesystem before retrying.

ADDITIONAL INFORMATION

Here are my numbers (after balancing):

### Just for reference, it's on a 8GiB partition:
$ df -h /
Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk2p2  7.8G  3.3G  4.4G  44% /

### Allocated chunks on the device(s) on /:
### on device 1, we have allocated 4.20GiB in various chunks, the total space usable on that device is 7.75GiB
$ btrfs filesystem show /
Label: 'root'  uuid: 7d24fbb1-73c3-42e2-bae1-c02321491dd5
        Total devices 1 FS bytes used 3.26GiB
        devid    1 size 7.75GiB used 4.20GiB path /dev/mmcblk2p2

Btrfs v3.16

### Content of the chunks:
### here's exactly how the allocated 4.20GiB chunks are split between Data, System and Metadata, and how much is written into them
$ btrfs filesystem df /
Data, single: total=3.91GiB, used=3.15GiB
System, single: total=32.00MiB, used=16.00KiB
Metadata, single: total=256.00MiB, used=114.08MiB
unknown, single: total=9.64MiB, used=0.00

Currently, sfos-upgrade computes using this line:

Data, single: total=3.91GiB, used=3.15GiB

and this gets the information that .8 GiB are unwritten on the allocated chunks.
(This is expected: after balancing the chunks will probably be very compact).

What it should do is use all per fdevice line(s):

   devid    1 size 7.75GiB used 4.20GiB path /dev/mmcblk2p2

Total number of space on device available is 7.75, 4.20 has been allocated to chunks, 3.55 is unallocated and can still further be allocated to Data or Metadata chunks.

A possible piece of bash code that does this:

btrfs_allocation="$(btrfs filesystem show / )"
btrfs_total=0
btrfs_used=0
while read ld i ls t lu u lp p; do
   # look for lines like:
   #       devid    1 size 7.75GiB used 4.20GiB path /dev/mmcblk2p2
   if [[ "$ld" != 'devid' || "$ls" != 'size' || "$lp" != 'path'  ]]; then
      continue
   fi 
   # parse numbers with regex and tally them
   [[ "$t" =~ ([0-9]*)\.([0-9]*)GiB ]] && (( btrfs_total += "${BASH_REMATCH[1]}${BASH_REMATCH[2]}" ))
   [[ "$u" =~ ([0-9]*)\.([0-9]*)GiB ]] && (( btrfs_used += "${BASH_REMATCH[1]}${BASH_REMATCH[2]}" ))
done <<< "$btrfs_allocation"
btrfs_unallocated="$((btrfs_total-btrfs_used))"
echo $btrfs_unallocated

I've tested it in GNU/bash (I don't know if it works in busybox bash. Does it support regex? Otherwise the tr+sed+grep+etc. route would be needed) .

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.