Bazarr Ver. 0.8.4.3 about iocage-plugin-index HOT 18 CLOSED

@dimitrow7 , @tprelog after long debugging and testing (at least I learned a bit about how the frontend talks through the websocket and about the middelwared service/lib). I think I managed to find the cause of why this weird reinstallation happens. Seems like it was a bug in iocage, or more correct a bug in the python distutils lib that iocage uses. Opened up a PR with a fix which seem to solve the issue (I've tested to reinstall the plugins using UI while changing the iocage scripts and reloading the middelwared manually on my setup)

from iocage-plugin-index.

Good job @fulder.

from iocage-plugin-index.

The version is controlled by upstream FreeBSD package repos. Somebody can modify the plugin to use "latest" instead of "quarterly" though, which is updated more frequently.

https://github.com/ix-plugin-hub/iocage-plugin-index/blob/master/bazarr.json#L16

from iocage-plugin-index.

This is for the required plugins like python, lib, git, etc. Bazarr is being pulled from github repositories

from iocage-plugin-index.

Related to:

• fulder/iocage-plugin-bazarr#1
• morpheus65535/bazarr#930

and this really is an issue in the Bazarr plugin itself. I've just merged a fix for it to master branch and am testing if it works better. If everything looks OK I will merge the plugin to the 11.3-RELEASE branch as well.

@kmoore134 The bazarr doesn't seem to be a part of FreeBSD repos (that's why I've installed it in a more manuall manner) So there shouldn't be any need of updating the bazarr.json in here although I do concur it would probably be better to use latest. Could maybe open a PR for that + removal of no longer neede packages after the plugin update.

from iocage-plugin-index.

I've merged the fix to the 11.3-RELEASE branch now but not quite sure how to make a proper update of the plugin. Is the only fix to reinstall it from scratch?

This also raises a question I have been thinking about regarding the update process; what about plugins where auto updating is not an option or not as easy as using git (the official plexmediaplayer plugin is a good example where updates for FreeBSD are not automated and where you have to download a new package manually)

Wouldn't it be nice to have some sort of versioning for the plugin and e.g an optional post_update.sh file which could be run after the jail updates (this would also make it possible to commit different post_update.sh files on release branches in order to make the migration unique between different kind of versions)

from iocage-plugin-index.

what about plugins where auto updating is not an option

I'm using a bash script for this in one of my plugins. I made a simple menu that launches when you enter the jail console so its easy to run the scripts.

Wouldn't it be nice to have some sort of versioning for the plugin

I would 100% like to also see this as well

an optional post_update.sh file which could be run after the jail updates

We already have this. And there is apre_update as well. I'm using both of these files

Edit: Both pre_update.sh and post_update.sh are updated with every time a user updates the plugin. But the post_install.sh is not.

from iocage-plugin-index.

@tprelog thanks for the fast reply and really good tips, been playing around with both the pre_update.sh and post_update.sh the past few days and think I managed to make the plugin update in a more proper manner.

The weird thing I saw during my testing was that my (rc.d) service didn't quite restart correctly after the update. I guess it has to do with this function: https://github.com/iocage/iocage/blob/cdfc119363cd63664298942de2551476d26b3569/iocage_lib/ioc_plugin.py#L1035, if I understand it correctly it actually stops/start the services both during pre_update.sh and post_update.sh execution. Maybe there are some issues with how I start/stop the service in the plugin but that didn't really work for me.

Right now I've just disabled and stopped it in my own pre_update.sh and then enable and start it up as the last thing in my post_update.sh. This made the update a bit more stable from what I have seen during my testing.

In order to make it a bit more versioned (more for my sake right now) I've added shell script files for migrations between different kind of tags in my repo (see post_update.sh and migrations). This will probably be a bit more structured and make me remember what was needed in different migrations between different versions.

Nice idea with the menu while execing into a jail! Will think about if I should do something like that as well, but I guess using the new update scripts you could achieve an upgrade for plugins which lack automatic update features?

EDIT: I realized I missed KEYWORD: shutdown in my rc.d script. After adding it the restart of my service worked better but I still find it a bit weird that the service is started before package remove and install. If the service uses some of the installed libs very often it could cause some weird behavior during the update.

from iocage-plugin-index.

The version is controlled by upstream FreeBSD package repos. Somebody can modify the plugin to use "latest" instead of "quarterly" though, which is updated more frequently.

https://github.com/ix-plugin-hub/iocage-plugin-index/blob/master/bazarr.json#L16

So there shouldn't be any need of updating the bazarr.json in here although I do concur it would probably be better to use latest. Could maybe open a PR for that + removal of no longer neede packages after the plugin update.

@kmoore134 I've opened #66 for the the bazarr.json changes

from iocage-plugin-index.

Maybe there are some issues with how I start/stop the service in the plugin but that didn't really work for me.

My plugins don't stop and start the main service when they update either. I think for this to work you would need to configure a settings.json See Example

Nice idea with the menu while execing into a jail! Will think about if I should do something like that as well, but I guess using the new update scripts you could achieve an upgrade for plugins which lack automatic update features?

Yes there is no need for the extra menu. You could certainly handle this with the pre/post update scripts. Another option ( actually more detail on how I do this ) is to add the update function to the service rc.d/script. In other words

service $NAME upgrade

The menu basically just calls the service commands

from iocage-plugin-index.

Its not working. Fresh install on FreeNAS Plugin - not create bazzar in rc.d

from iocage-plugin-index.

@dimitrow7 thank you for more testing, did you check inside /usr/local/etc/rc.d/ ? I did test to install it using both UI, using iocage CLI to a fresh jail and also tried the update several times without any issues. Does the admin UI at <IP>:6767 work for you?

My test install output:

> fetch https://raw.githubusercontent.com/ix-plugin-hub/iocage-plugin-index/11.3-RELEASE/bazarr.json

bazarr.json                                            570  B 8308 kBps    00s


> cat bazarr.json 
{
  "name": "bazarr",
  "plugin_schema": "2",
  "release": "11.3-RELEASE",
  "artifact": "https://github.com/fulder/iocage-plugin-bazarr",
  "pkgs": [
    "ca_root_nss",
    "python37",
    "py37-pip",
    "py37-libxml2",
    "py37-sqlite3",
    "libxslt",
    "git",
    "unrar"
  ],
  "packagesite": "http://pkg.FreeBSD.org/FreeBSD:11:amd64/quarterly",
  "fingerprints": {
    "iocage-plugins": [
      {
        "function": "sha256",
        "fingerprint": "b0170035af3acc5f3f3ae1859dc717101b4e6c1d0a794ad554928ca0cbb2f438"
      }
    ]
  },
  "official": "false"
}


> iocage fetch -P bazarr.json --name bazarr2 dhcp=on
Plugin: bazarr
  Official Plugin: false
  Using RELEASE: 11.3-RELEASE
  Using Branch: 11.3-RELEASE
  Post-install Artifact: https://github.com/fulder/iocage-plugin-bazarr
  These pkgs will be installed:
    - ca_root_nss
    - python37
    - py37-pip
    - py37-libxml2
    - py37-sqlite3
    - libxslt
    - git
    - unrar

Testing Host DNS response to pkg.FreeBSD.org
Testing bazarr2's SRV response to pkg.FreeBSD.org
Testing bazarr2's DNSSEC response to pkg.FreeBSD.org

Installing plugin packages:
  - ca_root_nss... 
  - python37... 
  - py37-pip... 
  - py37-libxml2... 
  - py37-sqlite3... 
  - libxslt... 
  - git... 
  - unrar... 

Fetching artifact... 
Cloning git repository

Running post_install.sh
Clone bazarr repo
Clone bazarr repo
Cloning into '/usr/local/bazarr'...
Clone bazarr repo
Updating files: 100% (3498/3498), done.
Installing bazarr requirements
Updating files: 100% (3498/3498), done.
Collecting lxml>=4.3.0 (from -r /usr/local/bazarr/requirements.txt (line 1))
Updating files: 100% (3498/3498), done.
  Downloading https://files.pythonhosted.org/packages/39/2b/0a66d5436f237aff76b91e68b4d8c041d145ad0a2cdeefe2c42f76ba2857/lxml-4.5.0.tar.gz (4.5MB)
Updating files: 100% (3498/3498), done.
Installing collected packages: lxml
Updating files: 100% (3498/3498), done.
  Running setup.py install for lxml: started
Updating files: 100% (3498/3498), done.
    Running setup.py install for lxml: finished with status 'done'
Updating files: 100% (3498/3498), done.
Successfully installed lxml-4.5.0
Updating files: 100% (3498/3498), done.
Creating new user with name: bazarr
Updating files: 100% (3498/3498), done.
Start bazarr service
Updating files: 100% (3498/3498), done.
bazarr_enable:  -> YES
Updating files: 100% (3498/3498), done.
Starting bazarr.
Updating files: 100% (3498/3498), done.
Post install completed!
Updating files: 100% (3498/3498), done.

Admin Portal:
http://192.168.1.144:6767/

>  iocage exec bazarr2 ls -l /usr/local/etc/rc.d
total 38
-rwxr-xr-x  1 root  wheel  1042 Apr 16 18:31 bazarr
-r-xr-xr-x  1 root  wheel   689 Apr 16 03:13 git_daemon
-r-xr-xr-x  1 root  wheel  1021 Apr  9 03:23 svnserve
-r-xr-xr-x  1 root  wheel  1239 Apr  2 06:26 tcsd
-r-xr-xr-x  1 root  wheel   628 Apr  2 06:06 tpmd

from iocage-plugin-index.

@tprelog

My plugins don't stop and start the main service when they update either. I think for this to work you would need to configure a settings.json See Example

Isn't these only for setting updates? And the service restart there is probably just run at setting changes. At FreeNAS docs:

Command to run when restarting the plugin service after changing settings.

Have you tried to add the KEYWORD: shutdown to your service file rc.d file? It made my plugins stop and start better. The python code run during the iocage updates executes a rc.shutdown command and reading the rc.shutdown docs (See the "Operation of rc.shutdown" steps) you can see that it will fetch all the files with this keyword and execute run_rc_script() with faststop.

Yes there is no need for the extra menu. You could certainly handle this with the pre/post update scripts. Another option ( actually more detail on how I do this ) is to add the update function to the service rc.d/script. In other words

service $NAME upgrade

The menu basically just calls the service commands

Good point 👍 I have actually done this for other services (not yet in any plugin) as well :)

from iocage-plugin-index.

Its not working. Fresh install on FreeNAS Plugin - not create bazzar in rc.d

@dimitrow7 @kmoore134 @fulder

I've also seen this before with my plugins ... Sometimes the overlay folder is not applied. I think it's a seperate bug but I don't know if it's FreeNAS or iocage.

Please try the following the to recreate this issue and confirm.

1. Install a community plugin ( make a note of the name you choose )
   -- Confirm the plugin has installed correctly and started.

2. Stop the plugin

3. Uninstall the plugin

4. Install the plugin again
   -- Important: to trigger the bug, you must use the exact same name as before

When the plugin is installed for the second time, it will not have any files included from the overlay.

At this point you can press UPDATE and the missing files will be downloaded and correctly applied

I have also found that rebooting the entire FreeNAS between steps 3 and 4 is the only way I can correctly reinstall the plugin using the exact same name

from iocage-plugin-index.

@tprelog I have done these steps a lot while testing my own plugins and never really had much troubles with the overlay not being applied like you describe. (I do have a vague memory of rc.d folder not being created, my post_install.sh in bazarr actually does a mkdir -p command for that folder, but as this was one of the first plugins I developed it could be due to some minor bug during the development process too)

I just run a test both using iocage CLI as well as the UI and followed your steps. Both time my plugin did start up corretly (with the overlay folder created) the second time. I tried to create a simple test plugin with only an overlay folder and again it worked just fine to install it twice.

What version of FreeNAS are you running?

I have seen a weird behaviour some times while installing the plugin with the same name where the new created jail got a <OLD_NAME>_1 name automatically but still one with <OLD_NAME> was create and in a corrupt state. But that could be a total different issue.

This issue has started to grow a lot with other discussion and found issues (mostly my own fault 🤚). Could we maybe open up new one for the one you're descirbing @tprelog ?

@dimitrow7 did you test installing a fresh installation only once or several times? Did you maybe use the same name of the jail so it could maybe be the same problem @tprelog is having? Do you have the output logs for the failed installation? I have tried many times with the bazarr plugin after the last updates and haven't been able to reproduce it.

from iocage-plugin-index.

@fulder Fresh installation with same name - fault, whit new name - no issues.

from iocage-plugin-index.

@dimitrow7 sounds like exactly the bug @tprelog described. I managed to reproduce it today but only while using the UI. Not sure what iocage commands it executes and how it is different from using the new iocage CLI. But just running a iocage destroy <NAME> seems to solve the problem as well.

It could be some bug when it comes to iocage (again probably depends on what the UI really does in the background). I found this code: https://github.com/iocage/iocage/blob/cdfc119363cd63664298942de2551476d26b3569/iocage_lib/ioc_plugin.py#L1196 if the copy does fail (could be for several reasons) it is just ignored (without logging anything either).

from iocage-plugin-index.

A new release of FreeNAS has been released yesterday: https://www.ixsystems.com/blog/library/freenas-11-3-u3/ including this fix. I have verfied that the overlay works as intended while installing a plugin with the same name twice.

If you want/can, could you verify if it looks fine for you as well and we could probably close this issue now =)

from iocage-plugin-index.