hirak99 / yabsnap Goto Github PK

I've tried reinstalling multiple times but I keep getting this error whenever I try to install or remove any packages. I've only had it happen on my laptop and not on my desktop despite both running the latest version of yabsnap:

Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/usr/share/yabsnap/code/main.py", line 173, in
main()
File "/usr/share/yabsnap/code/main.py", line 167, in main
_config_operation(
File "/usr/share/yabsnap/code/main.py", line 114, in _config_operation
snapper.create(comment)
File "/usr/share/yabsnap/code/snap_operator.py", line 107, in create
self._create_and_maintain_n_backups(
File "/usr/share/yabsnap/code/snap_operator.py", line 89, in _create_and_maintain_n_backups
previous_snaps = [
File "/usr/share/yabsnap/code/snap_operator.py", line 89, in
previous_snaps = [
File "/usr/share/yabsnap/code/snap_operator.py", line 37, in _get_old_backups
yield snap_holder.Snapshot(pathname)
File "/usr/share/yabsnap/code/snap_holder.py", line 75, in init
self.metadata = _Metadata.load_file(self._metadata_fname)
File "/usr/share/yabsnap/code/snap_holder.py", line 65, in load_file
return cls(**json.load(f))
File "/usr/lib/python3.10/json/init.py", line 293, in load
return loads(fp.read(),
File "/usr/lib/python3.10/json/init.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

When executing pacman commands I get the following error:

(1/1) Triggering yabsnap pre-installation snapshots...
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/share/yabsnap/code/main.py", line 189, in <module>
    main()
  File "/usr/share/yabsnap/code/main.py", line 180, in main
    _config_operation(
  File "/usr/share/yabsnap/code/main.py", line 120, in _config_operation
    snapper.on_pacman()
  File "/usr/share/yabsnap/code/snap_operator.py", line 149, in on_pacman
    self._create_and_maintain_n_backups(
  File "/usr/share/yabsnap/code/snap_operator.py", line 122, in _create_and_maintain_n_backups
    for expired in _all_but_last_k(previous_snaps, count - 1):
  File "/usr/share/yabsnap/code/snap_operator.py", line 62, in _all_but_last_k
    raise ValueError(f"k = {k} < 0")
ValueError: k = -1 < 0
error: command failed to execute correctly
error: failed to commit transaction (failed to run transaction hooks)

This is due to changes introduced with 12bc3b3 in the function _create_and_maintain_n_backups.
When keep_preinstall is zero and one is substracted of count in the method _create_and_maintain_n_backups the function _all_but_last_k raises an exception.

My proposed fix is:

--- a/src/code/snap_operator.py
+++ b/src/code/snap_operator.py
@@ -122,1 +122,1 @@
-        for expired in _all_but_last_k(previous_snaps, count - 1):
+        for expired in _all_but_last_k(previous_snaps, max(count - 1, 0)):

Cheers,
JMH.

There should be a command to clear snapshots in bulk.

It should have options to specify which tags, and what algorithm should be applied to the clean up.

Forked from #11

Currently, configs are read from /etc/configs/yabsnap, we should provide an argument to use a different location or files.

Can't understand how yabsnap work with snapshots with trigger I, witch are creates by pacman pre hook..

Config is:

$ grep -v "^#" /etc/yabsnap/configs/root.conf
[DEFAULT]

source = /

dest_prefix = /.snapshots/@root-

trigger_interval = 1 hour

min_keep_secs = 1800

keep_user = 1

keep_preinstall = 1

preinstall_interval = 5 minutes

keep_hourly = 5
keep_daily = 1
keep_weekly = 1
keep_monthly = 1
keep_yearly = 1

keep_preinstall is 1

But list of snapshots are:

$ yabsnap list
Config: /etc/yabsnap/configs/root.conf (source=/)
Snaps at: /.snapshots/@root-...
  20231024121406   I   (6 days 23h ago)      pacman --color=always --sync --needed p7zip
  20231024123022  S    (6 days 23h ago)      
  20231025092712   I   (6 days 2h ago)       pacman --color=always --sync --sysupgrade
  20231025200741   I   (5 days 15h ago)      /usr/bin/pacman -U /home/nebulosa/Documents/yabsnap/yabsnap-2.0.7-2-any.pkg.tar.zst
  20231026162630   I   (4 days 19h ago)      pacman --color=always --sync --sysupgrade
  20231026230426   I   (4 days 12h ago)      pacman --color=always --sync --sysupgrade
  20231027120119   I   (3 days 23h ago)      pacman --color=always --sync --sysupgrade
  20231028112115   I   (3 days ago)          pacman --color=always --sync --sysupgrade
  20231029142211   I   (1 day 21h ago)       pacman --color=always --sync --sysupgrade
  20231030092423   I   (1 day 2h ago)        pacman --color=always --sync --sysupgrade
  20231030120211   I   (23h 32m ago)         pacman --color=always --sync --sysupgrade
  20231030193026  S    (16h 4m ago)          
  20231030233958   I   (11h 54m ago)         pacman --color=always --sync --sysupgrade
  20231031092224  S    (2h 12m ago)          
  20231031094105   I   (1h 53m ago)          pacman --color=always --upgrade /home/nebulosa/.cache/pikaur/pkg/yabsnap-2.0.11-1-any.pkg.tar.zst
  20231031103029  S    (1h 4m ago)           
  20231031113006  S    (4m 41s ago) 

What did I miss?

As of AUR package version 1.05.3-1, yabsnap has its systemd unit files in /etc/systemd/system/. However, units provided by installed packages should live in ‘/usr/lib/systemd/system/’:

https://wiki.archlinux.org/title/Systemd#Writing_unit_files

Hello ,

I am currently using yabsnap for managing snapshots on two volumes, namely @root and @data. The current functionality of yabsnap takes snapshots of both volumes whenever pacman performs any actions. However, I would like to propose the addition of a feature that allows users to toggle the behavior of yabsnap, specifically deciding whether snapshots should be taken for the @data volume when pacman performs action. This toggle option would provide users with more control over the snapshot process, ensuring snapshots are only captured for the @root volume when pacman performs actions.

Thanks

Harden the CLI by validating a mount to be btrfs, before operations and on yabsnap list and refuse operating if the mount is not btrfs.

Would it be possible to add a way to run a script after creating/removing a snapshot?

I have a cron job to run a fish script to populate a refind config with options to boot into snapshots.
It's this if you are curious: create-refind-fallbacks.txt
But it would be far better if it just ran after yabsnap made a modification to the root snapshots.
This could also allow me to have a script show a notification when it ran.

Some kind of implementation could look like this in the config:

post_transaction_script = /root/scripts/create-refind-fallbacks.fish

Or is there another way without having to implement it in yabsnap?

Thank you for your time and effort!

sudo yabsnap create
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/usr/share/yabsnap/code/main.py", line 189, in
main()
File "/usr/share/yabsnap/code/main.py", line 152, in main
if configs.is_schedule_enabled():
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/yabsnap/code/configs.py", line 157, in is_schedule_enabled
for config in iterate_configs(None):
File "/usr/share/yabsnap/code/configs.py", line 138, in iterate_configs
config = Config.from_configfile(fname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/yabsnap/code/configs.py", line 98, in from_configfile
setattr(result, key, int(value))
^^^^^^^^^^
ValueError: invalid literal for int() with base 10: '1jjjjjjjjjjjjjjjjjjjjjj'

When you build/install yabsnap package it launch run_tests.sh and shows this warning:

/home/nebulosa/yabsnap/src/yabsnap-2.0.7/src/code/configs.py:177: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/nebulosa/yabsnap/src/yabsnap-2.0.7/src/code/example_config.conf' mode='r' encoding='UTF-8'>
  for line in open(_example_config_fname()):
ResourceWarning: Enable tracemalloc to get the object allocation traceback

Created: /tmp/yabsnap_config_test_rdlhy2bi
............
----------------------------------------------------------------------
Ran 12 tests in 0.015s

OK

A little reseach shows me that you need edit file config.py like that:

   with open(_example_config_fname()) as ecn:
        for line in ecn:
            line = line.strip()
            if source and line.startswith("source ="):
                line = f"source = {source}"
            elif line.startswith("dest_prefix ="):
                line = f"dest_prefix = /.snapshots/@{name}-"
            lines.append(line)

and warning is gone.

Am I right? If so, please edit strings or I can open Pull Request for convenience

First, thanks a lot for this tool! This is the best one I have found for managing automated snapshot.

Although it seems focused on system snapshots (and rollback), I would find it very useful to be able to manage snapshots for non-root users, i.e. created and owned by a user other than root.

What I have tried so far:

1. Create a configuration at $XDG_CONFIG_HOME/yabsnap/home.conf with
   • source set to the user home (e.g. source = /home/the_user)
   • destination set to a directory (and subvolume) owned by the user (e.g. dest_prefix = /home/the_user/.snapshots/@home-)
2. Run (as the_user) yabsnap --source $XDG_CONFIG_HOME/yabsnap/home.conf create.

The next step would be to create a user systemd unit and timer to automate the snapshots.

My current issue is at step 2: yabsnap exits successfully, with no output to neither stdout nor stderr, and no snapshot is created at the configured destination.

Am I doing something wrond? Is this use case not supported?
Thanks in advance for any advice!

Made a bash script as "proof of work" and now using with fzf preview or just itself. Main reason is to get more info about installing, upgrading, deleting packages - which are they.

Idea is simple: parse strings from /var/log/pacman.log after all transactions with packages are completed and updating comment in *-meta.json file. Should work as post installation hook i guess.

Is it possible to add this feature in main python code? I'm not so good in Python, so I made a bash example how it might look like.. :)

The second question is: is it possible make a comment for snapshots with trigger S, e.g. Hourly shapshot, Weekly shapshot, etc?

I deleted Rollback folder inside /.snapshots
and rebooted, bootloader cant find my root partition now.
BTW,what is the rollback folder.

I would really like to see snapshot comments made by yabsnap also show up in grub-btrfs boot menu. 'Type' doesn't really matter but would be a nice bonus. Not seeing the pacman command is the one thing keeping me from ditching snapper

There should be a way to specify TTL, either in config or via commandline (commandline takes precedence).

Any snaps beyond TTL should be considered as expired, and cleaned up on next scheduled run.

Snaps having TTL should show it on yabsnap list.

I can see that the install-to-dest.sh script is grepping for Arch Linux in the file /etc/issue. I think that's a very fragile way of doing it since the system administrator may modify that file at any time (see issue(5) for details).

Instead, make yabsnap depend on the lsb-release package and make sure (1) that /etc/lsb-release exists, and (2) that $DISTRIB_ID equals Arch. Perhaps something along these lines:

ME="${0##*/}"
ETC_LSB_RELEASE='/etc/lsb-release'

test -r "$ETC_LSB_RELEASE" || {
  printf '%s: cannot read %s\n' "$ME" "$ETC_LSB_RELEASE" >&2
  exit 1
}

. "$ETC_LSB_RELEASE"

test "$DISTRIB_ID" = 'Arch' || {
  printf '%s: Not an Arch based distro, not proceeding.\n' "$ME" >&2
  exit 1
}

In 262bceb you add the apache2 license header to the files.

This includes a copyright claim # Copyright 2022 Google LLC.
Is this correct?

Also, in Contributing.md you set the requirement of signing Googles CLA to contribute.

Is this project owned by you personally, or by Google LLC?

I was having an issue installing the latest version of yabsnap. I would keep hitting this error here:

install: cannot stat 'artifacts/pacman/05-yabsnap-pacman-pre.hook': No such file or directory

I found that the install-to-dest.sh script has the wrong path for the hook. It says 05 instead of 01. https://github.com/hirak99/yabsnap/blob/9dd7753165dcbe84446f89a07ff426383e2b1f18/scripts/install-to-dest.sh#L59C43-L59C43. In my case I changed the file listed in the artifacts folder from 01 to 05.

Turns out there was some prior work to update the PKGBUILD here, #18 (comment).

I think the script should be updated to reference the correct path, but if the hook's priority has changed according to the linked comment above, maybe there should be some extra logic to remove the existing hook as well?

Triyng install yabsnap with pikaur and getting this error:

ERROR: ld.so: object 'libfakeroot.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.

Full log is here: https://0x0.st/HJkP.txt

And the second point is - in README file in Quick Start /etc/configs/yabsnap/CONFIGNAME.conf but right path is: /etc/yabsnap/configs/CONFIGNAME.conf Please fix the path.

Log:
$ sudo yabsnap create-config config
[sudo] password for nebulosa:
Config directory does not exist. Use 'create-config' command to create a config.

Created: /etc/yabsnap/configs/config.conf
Please edit the file to set 'source = ' field.

First of all, thanks for this project.
Secondly, there is a mistake in the instruction. Here:
https://github.com/hirak99/yabsnap/blob/master/README.md
In section "Recommended Subvolume Layout". If you follow the instruction step by step, the OS will not boot because it will not be able to mount "@.snapshots"
You probably need to change the last line of instructions or an entire section.
I tested this in Debian 12 (stable), but I think that the problem will also occur in other distros.

The following exits successfully with no error -

yabsnap --verbose --source this_source_does_not_exist create

Need to add a descriptive error when the source is not found.

Forked from #11

When you write yabsnap list it shows all information about snapshopts, but when you want delete some of them it will be not so easy, because name of snapshots is 14-digit timestamp, that really hard to enter, only copy and past.

For convinience you can use the fzf in that scenario (e.g. bash alias snapctl):
ls -d /.snapshots/*/ | sed 's/\/.snapshots\///g;s/\/$//g' | fzf -m --reverse --preview 'echo -n $(basename {1})": "; cat /.snapshots/{1}-meta.json | echo $(tr -d "\"}{")' --preview-window right:70%:wrap | xargs -I snap sudo yabsnap delete /.snapshots/snap'

It works as expected and already could be mentined in README, but really wants use yabsnap with fzf like that:
yabsnap list -q | fzf ... --preview yabsnap list {1} | xargs ... yabnsap delete ... , where:

yabsnap list -q it lists names of snapshots,
yabsnap list PATH|TIMESTAMP [--oneline] shows info only for one snapshot (--oneline - comma-separared info in one line)

Any thoughts?

I think I should report this issue on github because the PKGBUILD seems to be no problem.
I can't built with this error:

+ chmod -R g-w,a-w src/ /home/djw/.cache/paru/clone/yabsnap/pkg/yabsnap/usr/share/yabsnap
++ which selinuxenabled
+ mkdir -p /home/djw/.cache/paru/clone/yabsnap/pkg/yabsnap/usr/bin
+ ln -sf /usr/share/yabsnap/yabsnap.sh /home/djw/.cache/paru/clone/yabsnap/pkg/yabsnap/usr/bin/yabsnap
+ chmod 755 /home/djw/.cache/paru/clone/yabsnap/pkg/yabsnap/usr/bin/yabsnap
chmod: 无法对悬空符号链接 '/home/djw/.cache/paru/clone/yabsnap/pkg/yabsnap/usr/bin/yabsnap' 进行操作==> 错误： 在 package() 中发生一个错误。    正在放弃...
错误: 未能构建 'yabsnap-2.0.2-1': 

The ln -sf /usr/share/yabsnap/yabsnap.sh $PKGDIR/usr/bin/yabsnap shold be ln -sf $PKGDIR/usr/share/yabsnap/yabsnap.sh $PKGDIR/usr/bin/yabsnap or removing /? There seems to be a missing point here and yabsnap-git in AUR has the same error.

Error, warnings, etc. should be colorized when appropriate so that they are harder to miss.

The pacman snapshot gets taken a little too late and includes the /var/lib/pacman/db.lck. It's a bit annoying to remove it every time you reset a pacman snapshot

I have updated PKGBUILD using this article.

New PKGBUILD and the install file.

Why? Because, it's not quite right to build package through bash script - it cause errors in future like #17 and other mistakes with permission of files (e.g. .service files should be read only not executable)

I hope you find this usefil and will accept my edits