At the moment, if delete_unmanaged is set for a resource type, the deletion for that resource happens immediately after creation/in-place updates are performed.

This is not a workable solution when other resources depend on that resource within the remote instance, because they haven't had the chance to have their configuration updated to remove the dependency.

Add a delete_remote manager method (and implementing config method) that gets called after update_remote to handle this case, and move all existing resource deletion logic within the plugins into that method.

Requirements:

• Actual individual modules should be visible in the logging module, not just buildarr.main or buildarr.plugins.<plugin>
  • This likely entails each file importing/creating its own logger, along the lines of:

    from logging import getLogger
    
    logger = getLogger(__name__)

• Use the same logger object for global and plugin/instance-specific logging
• Allow modifying the state of all existing loggers so that the active plugin/instance context is reflected in the logging output

If it's possible to do this only with the logging module that would be preferred, but external libraries will be considered if it's very difficult without them.

Looks like it's possible only with logging: https://stackoverflow.com/a/533077

Checklist:

1. Set milestone to v0.1.2
2. Update the tool.poetry.version field in pyproject.toml
3. Get the raw changelog using the following command:

   $ git log --oneline --decorate v0.1.1..HEAD

4. Add release notes to docs/release-notes.md
5. Create pull request: #28
6. Merge pull request
7. Check that the CI pipeline passed on main: https://github.com/buildarr/buildarr/actions/runs/4218103154
8. Tag the new release
9. Check that the release was automatically published to PyPI: https://github.com/buildarr/buildarr/actions/runs/4218112224
10. Push an update to http://github.com/buildarr/buildarr-docker to release to Docker Hub
11. Push an update to https://buildarr.github.io
12. Close issue and release milestone, and create the next one

At the moment plugins are getting this value directly from the Buildarr configuration structure.

An oversight when adding timeouts to API calls means that both the Sonarr and Prowlarr plugin try to refer to this structure when using their own ad-hoc commands, where it doesn't exist, meaning that fetching instance-specific configurations is not working as expected.

The global state attribute will be set to a sane default if configuration is not loaded (30 seconds).

The new Jellyseerr plugin contains a workaround for this issue. That workaround can be removed with the introduction of this attribute.

Buildarr already natively runs on Windows as a standalone application, but a few users have requested the capability of registering and managing Buildarr as a Windows service, to run alongside *Arr application services that run on Windows.

Take care to preserve the abiity to install plugins and define configuration in a place that is relatively easy for the user to access.

• Do research on how to package Python applications into an installer
• Do research on how to register a Python daemon as a Windows service
• Implement Buildarr as a Windows service
• Build WIndows installer for Buildarr in a CI pipeline

This plugin should be vendored into Buildarr itself (like the current Sonarr plugin), and do everything a normal plugin does, but not be loaded under normal circumstances.

This will be useful for creating unit tests for all features of Buildarr, including the plugin API, without having to specify an external plugin for testing purposes.

• Module: buildarr.plugins.dummy
• Use the plugin entry point interface to allow it to be loaded by Buildarr
• Only load the plugin if the BUILDARR_TESTING environment variable is undefined or set to False
• Implement all the required and optional parts a plugin:
  • Configuration
    • ConfigPlugin.hostname
    • ConfigPlugin.port
    • ConfigPlugin.protocol
    • ConfigPlugin.instances
    • ConfigPlugin.uses_trash_metadata
    • ConfigPlugin.get_trash_metadata
    • ConfigBase.from_remote
    • ConfigBase.update_remote
  • Manager
  • Secrets
• Add a basic WSGI app/HTTP server for logging and responding to requests from Buildarr
• Format the code in a very simple and easy-to-understand way to allow it to be one of the references a new developer can use for how a plugin should be implemented

This is to handle a variety of potential cases:

• Make defining the name of something in the configuration case-insensitive
• Values are enumerated (e.g. select field types), but cannot be statically defined as Enum classes.

Requirements:

• Remember the case of the original value, and return it when queried
  • e.g. Set iteration, key iteration, conversion back to regular dict and set/list classes
• Case-insensitive key lookups (dict_obj["key"]) and existence checks ("key" in set_dict_obj)
• Case-insensitive equality checks using a == b
  • Dictionary keys and set values must evaluate as equal if the keys are the same in terms of value, but not the same in terms of case

At the moment it is a dynamically generated property using the @property decorator.

For consistency with other plugin-overridden functions, change this to a regular function.

This is a backwards-incompatible change.

This would be a good way to clean up handling rendering of instance links and substituting resource IDs for names before update_remote is run.

1. Move the TRaSH-Guides metadata folder into global state, rather than passing it as a parameter
2. Add the following manager method, which will call into corresponding Config methods by default. The point of passing both the local and remote instance configs at the same time is to allow one set of requests to render both objects at once.

   class Manager(Generic[Config, Secrets]):
       def render_local_remote(self, secrets: Secrets, local_instance_config: Config, remote_instance_config: Config) -> Tuple[Config, Config]:
           return type(local_instance_config).render_local_remote(local_instance_config, remote_instance_config)

   • Or should this be a per-Config object render method? the corresponding requests would have to run twice, but it would be much simpler to implement.

Add a --secrets-file argument to buildarr run and buildarr daemon, which allows the user to override where the secrets file is read from and saved.

Non-absolute paths will be resolved relative to the current working directory of the Buildarr process, which is the expected behaviour when passing relative paths in to a command from the command line.

Add an equals remote map parameter, similar to decoder and encoder, where the remote map can supply the function used to compare local and remote values.

https://beta.ruff.rs/docs

It's still in beta technically, but it's not a pre-release on PyPI, and is already quite popular, with it being used in major projects such as FastAPI, and, ironically enough, isort itself.

The major benefits of this change include:

• It supports almost everything that isort and Flake8 individually do, in addition to automatic fixing of certain errors (such as unused imports).
• Due to being written in Rust, Ruff is much faster (anywhere from 10 to 100 times faster), allowing pre-commit hooks and the linting CI pipeline to run a lot faster than they currently do.
• Ruff supports all minor versions of Python 3.7 to 3.11, so we don't need to cater to Flake8 only supporting specific Python 3.8 minor versions.

The following bugs occur in the current version of Buildarr:

• BaseIntEnum has no specific handlers in the default decoder and formatting functions
• When BaseIntEnum objects are exported using the JSON/YAML encoder, the BaseIntEnum encoder does not get used as BaseIntEnum is a subclass of int, so the default behaviour is used:

  jellyseerr:
    ...
    settings:
      ...
      users:
        default_permissions:
        - 1048576  # Should be 'MANAGE-ISSUES'
        - 32  # Should be 'REQUEST'

Remove the following features which have been deprecated and scheduled for removal in Buildarr v0.5.0:

• The BaseIntEnum configuration/secrets attribute type - Deprecated due to #91
• The render_trash_metadata run stage - Replaced with the render stage as part of #92

This is so Buildarr can track the version of the loaded plugin, and output it to the user.

Must be done before the Sonarr plugin gets forked.

At the moment, with the exception of includes, all relative paths defined in the Buildarr configuration for local paths are treated as relative to the current working directory of the Buildarr process.

Buildarr needs to introduce a way to resolve these paths relative to the file actually being read.

Since this could entail introducing a new configuration attribute type, it should be done before the Sonarr plugin is split into its own repository.

Due to how existence checks and deletions for resources have to mostly be implemented on the plugin side, there isn't much Buildarr can do to make this easier.

A helper function for logging the exact configuration attributes that will be removed from the hose can be added, however.

def log_delete_local_attrs(self, tree: str, deleted: bool = True) -> None:
    ...

Say we have a name-keyed resource dictionary with the tmdb_id and year resources defined on them. This will improve verbosity of the logs of their deletion from:

2023-03-29 20:40:41,129 buildarr:1 buildarr.config.base [INFO] <example> (default) example.settings.resource.definitions['name']: (...) -> (deleted)

To:

2023-03-29 20:40:41,129 buildarr:1 buildarr.config.base [INFO] <example> (default) example.settings.resource.definitions['name'].tmdb_id: 123456 -> (deleted)
2023-03-29 20:40:41,129 buildarr:1 buildarr.config.base [INFO] <example> (default) example.settings.resource.definitions['name'].year: 2023 -> (deleted)

For unmanaged resources that won't be deleted, this will be changed from:

2023-03-29 20:40:41,129 buildarr:1 buildarr.config.base [DEBUG] <example> (default) example.settings.resource.definitions['name']: (...) (unmanaged)

To:

2023-03-29 20:40:41,129 buildarr:1 buildarr.config.base [DEBUG] <example> (default) example.settings.resource.definitions['name'].tmdb_id: 123456 (unmanaged)
2023-03-29 20:40:41,129 buildarr:1 buildarr.config.base [DEBUG] <example> (default) example.settings.resource.definitions['name'].year: 2023 (unmanaged)

If the default instance is the only one defined, it should work.

$ poetry run buildarr run ../temp/buildarr-jellyseerr.yml
2023-04-10 23:40:55,439 buildarr:60104 buildarr.cli.run [INFO] Buildarr version 0.4.1 (log level: INFO)
2023-04-10 23:40:55,440 buildarr:60104 buildarr.cli.run [INFO] Loading configuration file 'C:\Users\karam\dev\github.com\buildarr\temp\buildarr-jellyseerr.yml'
2023-04-10 23:40:55,458 buildarr:60104 buildarr.cli.run [INFO] Finished loading configuration file
2023-04-10 23:40:55,463 buildarr:60104 buildarr.cli.run [INFO] Loaded plugins: jellyseerr (0.1.0), sonarr (0.4.1)
2023-04-10 23:40:55,463 buildarr:60104 buildarr.cli.run [INFO] Loading instance configurations
Traceback (most recent call last):
  File "c:\program files\python38\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\program files\python38\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\karam\dev\github.com\buildarr\buildarr-jellyseerr\.venv\Scripts\buildarr.exe\__main__.py", line 7, in <module>
  File "C:\Users\karam\dev\github.com\buildarr\buildarr-jellyseerr\.venv\lib\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\karam\dev\github.com\buildarr\buildarr-jellyseerr\.venv\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "C:\Users\karam\dev\github.com\buildarr\buildarr-jellyseerr\.venv\lib\site-packages\click\core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\karam\dev\github.com\buildarr\buildarr-jellyseerr\.venv\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\karam\dev\github.com\buildarr\buildarr-jellyseerr\.venv\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\karam\dev\github.com\buildarr\buildarr-jellyseerr\.venv\lib\site-packages\buildarr\cli\run.py", line 136, in run
    _run(secrets_file_path, use_plugins)
  File "C:\Users\karam\dev\github.com\buildarr\buildarr-jellyseerr\.venv\lib\site-packages\buildarr\cli\run.py", line 173, in _run
    load_instance_configs(use_plugins)
  File "C:\Users\karam\dev\github.com\buildarr\buildarr-jellyseerr\.venv\lib\site-packages\buildarr\config\load.py", line 182, in load_instance_configs
    instance_config = plugin_manager.get_instance_config(
  File "C:\Users\karam\dev\github.com\buildarr\buildarr-jellyseerr\.venv\lib\site-packages\buildarr\manager\__init__.py", line 78, in get_instance_config
    return config.get_instance_config(instance_name)
  File "C:\Users\karam\dev\github.com\buildarr\buildarr-jellyseerr\.venv\lib\site-packages\buildarr\config\models.py", line 167, in get_instance_config
    return instance_config_class(**self.dict(exclude={"instances"}, exclude_unset=True))
  File "pydantic\main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for JellyseerrInstanceConfig
settings -> services -> sonarr -> definitions -> Sonarr (HD) -> instance_name
  Target instance 'default' not defined in plugin 'sonarr' configuration (type=value_error)

Create a new test-config command that reads in a Buildarr configuration file as normal, and test the following things (report the results to the log):

• Global configuration parsing
• Instance confguration parsing, for each instance
• Instance-to-instance dependency resolution

Checklist:

1. Set milestone to v0.2.0
2. Get the raw changelog using the following command:

   $ git log --oneline --decorate v0.1.2..HEAD

3. Update the tool.poetry.version field in pyproject.toml
4. Add release notes to docs/release-notes.md
5. Create pull request: #42
6. Merge pull request
7. Check that the CI pipeline passed on main: https://github.com/buildarr/buildarr/actions/runs/4250914480
8. Tag the new release
9. Check that the release was automatically published to PyPI:
   • Release pipeline succeeded: https://github.com/buildarr/buildarr/actions/runs/4250935085
   • Release available on PyPI: https://pypi.org/project/buildarr/0.2.0
10. Push an update to http://github.com/buildarr/buildarr-docker to release to Docker Hub
11. Check that the release was automatically published to Docker Hub: callum027/buildarr:0.2.0
12. Push an update to https://buildarr.github.io
13. Close release milestone
14. Create new release milestone
15. Close this issue

Maintaining the integrity of dry runs is a lot more work than anticipated, and is increasingly incompatible with new plugin API features being incorported into Buildarr (instance initialisations in particular).

From Buildarr v0.5.0, remove the command-line option for carrying out dry runs (buildarr run --dry-run) from the Buildarr CLI.

The internal global state attribute (state.dry_run) will be retained, but made a read-only property that always returns False. This keeps open the possibility of re-establishing dry runs if an easier-to-maintain method of implementing it is discovered.

Checklist:

1. Set milestone to v0.4.1
2. Get the raw changelog using the following command:

   $ git log --oneline --decorate v0.4.0..HEAD

3. Update the tool.poetry.version field in pyproject.toml
4. Add release notes to docs/release-notes.md
5. Create pull request: #89
6. Merge pull request
7. Check that the CI pipeline passed on main: https://github.com/buildarr/buildarr/actions/runs/4647959434
8. Tag the new release
9. Check that the release was automatically published to PyPI: https://github.com/buildarr/buildarr/actions/runs/4647969149
10. Push an update to http://github.com/buildarr/buildarr-docker to release to Docker Hub
11. Push an update to https://buildarr.github.io
12. Close release milestone
13. Create new release milestone
14. Close this issue

The cause of the problem appears to be that the change is happening in the host OS rather than the container, and inotify is silently dropping the filesystem events, as documented in this issue.

Switch Buildarr to use PollingObserver to make it poll for changed files, instead. While the polling is running Buildarr isn't doing anything in particular, and the polling isn't very resource-intensive, so this shouldn't be a problem.

As Buildarr has been built out, some source files have grown quite large (e.g. buildarr/config/__init__.py). Some things are in places they shouldn't be (e.g. general purpose Pydantic type definitions under config only).

The way runtime state is currently stored has also resulted in problems such as unable to properly type hint many structures, and having to pass too many values around as parameters.

With these factors in mind, refactor Buildarr to improve future maintainability and unlock the ability for new features to be more easily added.

This can be done for Poetry projects using the poetry-publish action from GitHub Marketplace.

• Create and add the PyPI API token to the repository secrets
• Create a pull request for adding the action to the pipeline
  • #11
• Merge the pull request adding the action to the pipeline
• Check that the action automatically publishes the next release to PyPI

After investigation, it looks like it is likely a bug in Pydantic where annotated SecretStr fields cannot be idempotently passed a SecretStr object when instantiating a model.

Reported here: pydantic/pydantic#5088

Work around by converting Password and SonarrApiKey to subclasses of SecretStr.

buildarr.yml:

---

sonarr:
  hostname: "localhost"
  port: 8989
  protocol: "http"
  api_key: "1a2b3c4d5e1a2b3c4d5e1a2b3c4d5e1a"

Output:

2023-02-22 08:23:51,411 buildarr:76436 buildarr.main [INFO] Buildarr version 0.1.2 (log level: INFO)
2023-02-22 08:23:51,411 buildarr:76436 buildarr.main [INFO] Loading configuration file 'C:\Users\karam\dev\github.com\buildarr\temp\buildarr.yml'
2023-02-22 08:23:51,447 buildarr:76436 buildarr.main [INFO] Finished loading configuration file
2023-02-22 08:23:51,469 buildarr:76436 buildarr.main [INFO] Plugins loaded: sonarr
2023-02-22 08:23:51,472 buildarr:76436 buildarr.main [INFO] Using plugins: sonarr
2023-02-22 08:23:51,474 buildarr:76436 buildarr.main [INFO] Loading secrets file from 'secrets.json'
2023-02-22 08:23:51,475 buildarr:76436 buildarr.main [INFO] Secrets file does not exist, will create new file
2023-02-22 08:23:51,475 buildarr:76436 buildarr.plugins.sonarr default [INFO] Checking secrets
2023-02-22 08:23:51,476 buildarr:76436 buildarr.plugins.sonarr default [INFO] Connection test failed using cached secrets (or not cached), fetching secrets
SecretStr
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\karam\dev\github.com\buildarr\buildarr\.venv\lib\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\karam\dev\github.com\buildarr\buildarr\.venv\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "C:\Users\karam\dev\github.com\buildarr\buildarr\.venv\lib\site-packages\click\core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\karam\dev\github.com\buildarr\buildarr\.venv\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\karam\dev\github.com\buildarr\buildarr\.venv\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\karam\dev\github.com\buildarr\buildarr\buildarr\cli\run.py", line 91, in run
    _run(config, use_plugins)
  File "C:\Users\karam\dev\github.com\buildarr\buildarr\buildarr\cli\run.py", line 184, in _run
    instance_secrets = plugins[plugin_name].secrets.get(instance_config)
  File "C:\Users\karam\dev\github.com\buildarr\buildarr\buildarr\plugins\sonarr\secrets.py", line 78, in get
    return cls(
  File "pydantic\main.py", line 342, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for SonarrSecrets
api_key
  str type expected (type=type_error.str)

Some plugins manage instances that require some kind of initialisation or setup process before it can be managed by Buildarr using the normal process.

Add optional is_initialized and initialize functions to the run process, that run after rendering the instance configuration, but before fetching secrets and fetching the remote config. Plugins can override these functions and carry out any initialisation tasks they need to.

Plugins for applications that don't need initialisation can leave them unimplemented, and Buildarr will skip the initialisation stage for that plugin.

Use cases:

• A Buildarr-managed Sonarr instance using another Buildarr-managed Sonarr instance as an import list provider, without having to define the API key.
• Connecting Prowlarr to Sonarr/Radarr instances without having to provide duplicate details to connect them

Requirements:

• Plugins should be able to implement adding information such as API keys and other instance-specific configuration attributes to a resource, with the user only having to define the name to the resource being referenced.
  • Plugins should also be able to use the information from Buildarr to connect to the instance, and make requests to resolve things such as resource IDs for names.
• Buildarr should automatically order operations such that any instance in any plugin that is referenced by another, should be processed first, in a dependency chain.
  • If two instances reference each other (cyclic dependency), raise an error.

Internally we make calls to importlib_metadata.version in multiple places to get the Buildarr version.

This should only be done in one place, and then that variable should be imported wherever it is needed.

Why shouldn't we use it?

• It's not an "official" Sonarr API function
• It requires using both a regular expression extraction and the json5 library to parse the JavaScript object response (which is much slower than the built-in json library for compliant JSON)

This endpoint should only really be used to fetch the API key automatically from an instance that doesn't require authentication, since it isn't available anywhere else.

Here are the following places where we could avoid using it:

• buildarr/plugins/sonarr/config/__init__.py:330-333 - can be changed to use the /api/v3/system/status endpoint instead
• buildarr/plugins/dummy/config/__init__.py:192-195 - create a new API function in the Dummy server to use

The type of the attribute is Set[NabCategory], with NabCategory being an implemented BaseIntEnum. #91 is unrelated.

Buildarr appears to be passing the raw integers into the NabCategory constructor, which means there is probably an issue with the decoder's field type handler.

2023-04-13 21:59:03,348 buildarr:1 buildarr_sonarr.api [DEBUG] <sonarr> (sonarr-anime) GET http://sonarr-anime:8989/api/v3/config/indexer
2023-04-13 21:59:03,349 buildarr:1 urllib3.connectionpool [DEBUG] <sonarr> (sonarr-anime) Starting new HTTP connection (1): sonarr-anime:8989
send: b'GET /api/v3/config/indexer HTTP/1.1\r\nHost: sonarr-anime:8989\r\nUser-Agent: python-requests/2.28.2\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\nX-Api-Key: <snip>\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: X-Application-Version: 3.0.10.1567
header: Vary: Accept
header: Cache-Control: no-cache, no-store, must-revalidate, max-age=0
header: Pragma: no-cache
header: Expires: 0
header: Access-Control-Allow-Origin: *
header: Content-Encoding: gzip
header: Content-Type: application/json; charset=utf-8
header: Server: Mono-HTTPAPI/1.0
header: Date: Thu, 13 Apr 2023 09:59:03 GMT
header: Transfer-Encoding: chunked
header: Keep-Alive: timeout=15,max=100
2023-04-13 21:59:03,352 buildarr:1 urllib3.connectionpool [DEBUG] <sonarr> (sonarr-anime) http://sonarr-anime:8989 "GET /api/v3/config/indexer HTTP/1.1" 200 None
2023-04-13 21:59:03,352 buildarr:1 buildarr_sonarr.api [DEBUG] <sonarr> (sonarr-anime) GET http://sonarr-anime:8989/api/v3/config/indexer -> status_code=200 res={'minimumAge': 0, 'retention': 0, 'maximumSize': 0, 'rssSyncInterval': 15, 'id': 1}
2023-04-13 21:59:03,352 buildarr:1 buildarr_sonarr.api [DEBUG] <sonarr> (sonarr-anime) GET http://sonarr-anime:8989/api/v3/indexer
2023-04-13 21:59:03,353 buildarr:1 urllib3.connectionpool [DEBUG] <sonarr> (sonarr-anime) Starting new HTTP connection (1): sonarr-anime:8989
send: b'GET /api/v3/indexer HTTP/1.1\r\nHost: sonarr-anime:8989\r\nUser-Agent: python-requests/2.28.2\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\nX-Api-Key: <snip>\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: X-Application-Version: 3.0.10.1567
header: Vary: Accept
header: Cache-Control: no-cache, no-store, must-revalidate, max-age=0
header: Pragma: no-cache
header: Expires: 0
header: Access-Control-Allow-Origin: *
header: Content-Encoding: gzip
header: Content-Type: application/json; charset=utf-8
header: Server: Mono-HTTPAPI/1.0
header: Date: Thu, 13 Apr 2023 09:59:03 GMT
header: Transfer-Encoding: chunked
header: Keep-Alive: timeout=15,max=100
2023-04-13 21:59:03,360 buildarr:1 urllib3.connectionpool [DEBUG] <sonarr> (sonarr-anime) http://sonarr-anime:8989 "GET /api/v3/indexer HTTP/1.1" 200 None
2023-04-13 21:59:03,360 buildarr:1 buildarr_sonarr.api [DEBUG] <sonarr> (sonarr-anime) GET http://sonarr-anime:8989/api/v3/indexer -> status_code=200 res=[{'enableRss': True, 'enableAutomaticSearch': True, 'enableInteractiveSearch': True, 'supportsRss': True, 'supportsSearch': True, 'protocol': 'torrent', 'priority': 25, 'downloadClientId': 0, 'name': 'Nyaa.si (Prowlarr)', 'fields': [{'order': 0, 'name': 'baseUrl', 'label': 'URL', 'value': 'http://prowlarr:9696/2/', 'type': 'textbox', 'advanced': False}, {'order': 1, 'name': 'apiPath', 'label': 'API Path', 'helpText': 'Path to the api, usually /api', 'value': '/api', 'type': 'textbox', 'advanced': True}, {'order': 2, 'name': 'apiKey', 'label': 'API Key', 'value': '<snip>', 'type': 'textbox', 'advanced': False}, {'order': 3, 'name': 'categories', 'label': 'Categories', 'helpText': 'Drop down list, leave blank to disable standard/daily shows', 'value': [5070], 'type': 'select', 'advanced': False, 'selectOptionsProviderAction': 'newznabCategories'}, {'order': 4, 'name': 'animeCategories', 'label': 'Anime Categories', 'helpText': 'Drop down list, leave blank to disable anime', 'value': [5070], 'type': 'select', 'advanced': False, 'selectOptionsProviderAction': 'newznabCategories'}, {'order': 5, 'name': 'animeStandardFormatSearch', 'label': 'Anime Standard Format Search', 'helpText': 'Also search for anime using the standard numbering', 'value': False, 'type': 'checkbox', 'advanced': False}, {'order': 6, 'name': 'additionalParameters', 'label': 'Additional Parameters', 'helpText': 'Additional Newznab parameters', 'type': 'textbox', 'advanced': True}, {'order': 7, 'name': 'minimumSeeders', 'label': 'Minimum Seeders', 'helpText': 'Minimum number of seeders required.', 'value': 5, 'type': 'number', 'advanced': True}, {'order': 8, 'name': 'seedCriteria.seedRatio', 'label': 'Seed Ratio', 'helpText': "The ratio a torrent should reach before stopping, empty is download client's default", 'value': 3.0, 'type': 'textbox', 'advanced': True}, {'order': 9, 'name': 'seedCriteria.seedTime', 'label': 'Seed Time', 'unit': 'minutes', 'helpText': "The time a torrent should be seeded before stopping, empty is download client's default", 'type': 'number', 'advanced': True}, {'order': 10, 'name': 'seedCriteria.seasonPackSeedTime', 'label': 'Season-Pack Seed Time', 'unit': 'minutes', 'helpText': "The time a torrent should be seeded before stopping, empty is download client's default", 'type': 'number', 'advanced': True}], 'implementationName': 'Torznab', 'implementation': 'Torznab', 'configContract': 'TorznabSettings', 'infoLink': 'https://wiki.servarr.com/sonarr/supported#torznab', 'tags': [], 'id': 1}]
Traceback (most recent call last):
  File "/usr/local/bin/buildarr", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/buildarr/cli/run.py", line 146, in run
    _run(secrets_file_path, use_plugins)
  File "/src/buildarr/cli/run.py", line 305, in _run
    remote_instance_config = manager.from_remote(instance_config, instance_secrets)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/buildarr/manager/__init__.py", line 165, in from_remote
    return instance_config.from_remote(secrets)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/buildarr_sonarr/config/__init__.py", line 336, in from_remote
    settings=SonarrSettingsConfig.from_remote(secrets),
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/buildarr/config/base.py", line 84, in from_remote
    fields[field_name] = field.type_.from_remote(secrets)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/buildarr_sonarr/config/indexers.py", line 1069, in from_remote
    definitions={
                ^
  File "/usr/local/lib/python3.11/site-packages/buildarr_sonarr/config/indexers.py", line 1070, in <dictcomp>
    indexer["name"]: INDEXER_TYPE_MAP[indexer["implementationName"]]._from_remote(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/buildarr_sonarr/config/indexers.py", line 191, in _from_remote
    **cls.get_local_attrs(
      ^^^^^^^^^^^^^^^^^^^^
  File "/src/buildarr/config/base.py", line 242, in get_local_attrs
    local_attrs[attr_name] = decoder(remote_attr)
                             ^^^^^^^^^^^^^^^^^^^^
  File "/src/buildarr/config/base.py", line 194, in <lambda>
    lambda v: cls._decode_attr(attr_name, v),
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/buildarr/config/base.py", line 711, in _decode_attr
    return cls._decode_attr_(cls.__fields__[attr_name].type_, value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/buildarr/config/base.py", line 732, in _decode_attr_
    return attr_type(value)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/enum.py", line 717, in __call__
    return cls.__new__(cls, value)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/enum.py", line 1133, in __new__
    raise ve_exc
ValueError: [5070] is not a valid NabCategory

Most (if not all) of the root validators present in the codebase can be converted to attribute-specific validators.

There are a couple of advantages to doing this:

• When there are validation errors in multiple attributes, they will all be output as an aggregated error message.
• Pydantic will show the configuration tree for the attribute that triggered the validation error, making it easier for the user to understand what configuration attribute is causing it.

Disadvantages:

• If attributes require referencing another attribute to validate it, the parsing order matters (only successfully parsed values up to that point are available in values).
  • Most of the time this shouldn't be a concern because attributes depending on other attributes tend to naturally be defined after them, but in some cases we might need to rearrange attribute definitions to ensure this works properly.

At the moment Buildarr relies on the parent folder having secure permissions. secrets.json is created using whatever default umask is currently set for the user.

Ensure secrets.json has permissions of 660 (user and group write, others none) when created by setting umask to 711 before writing to the file. If it already exists, ensure correct permissions are set on it in that case as well.

Due to the defaults set by Poetry when adding packages, the Buildarr main package's version restrictions for dependencies are too strict.

Buildarr is not only an application, but a library for plugins as well, so to make sure Buildarr plays as nicely as possible with other libraries, it should only enforce restrictions on package versions when it absolutely needs to.

Add unit tests to the Buildarr package and repository.

• Use the pytest library
• Add code coverage testing using pytest-cov, and use the Code Coverage Summary action to render the results in GitHub
• Add JUnit test reporting, and use the JUnit Report Action to render the results in GitHub
• Use pytest-mock for more integration of mocking/monkey patching into the test suite
• Run unit tests for all supported Python versions

Checklist:

1. Set milestone to v0.5.0
2. Get the raw changelog using the following command:

   $ git log --oneline --decorate v0.4.2..HEAD

3. Update the tool.poetry.version field in pyproject.toml
4. Add release notes to docs/release-notes.md
5. Create pull request: #119
6. Merge pull request
7. Check that the CI pipeline passed on main: https://github.com/buildarr/buildarr/actions/runs/4709977807
8. Tag the new release
9. Check that the release was automatically published to PyPI: https://github.com/buildarr/buildarr/actions/runs/4709985282
10. Push an update to http://github.com/buildarr/buildarr-docker to release to Docker Hub
11. Push an update to https://buildarr.github.io
12. Close release milestone
13. Create new release milestone
14. Close this issue

It seems like the trigger conditions need some adjustment to make it actually run, but we also need to make sure the pre-commit workflow passes first before it publishes the release.

Checklist:

1. Set milestone to v0.3.0
2. Create branch
3. Get the raw changelog using the following command:

   $ git log --oneline --decorate v0.2.0..HEAD

4. Update the tool.poetry.version field in pyproject.toml
5. Add release notes to docs/release-notes.md
6. Create pull request: #52
7. Merge pull request
8. Check that the CI pipeline passed on main: https://github.com/buildarr/buildarr/actions/runs/4422927721
9. Tag the new release
10. Check that the release was automatically published to PyPI: https://github.com/buildarr/buildarr/actions/runs/4422946088
11. Push an update to http://github.com/buildarr/buildarr-docker to release to Docker Hub
12. Push an update to https://buildarr.github.io
13. Close release milestone
14. Create new release milestone
15. Close this issue

Checklist:

1. Set milestone to v0.4.0
2. Get the raw changelog using the following command:

   $ git log --oneline --decorate v0.3.0..HEAD

3. Update the tool.poetry.version field in pyproject.toml
4. Add release notes to docs/release-notes.md
5. Create pull request: #80
6. Merge pull request
7. Check that the CI pipeline passed on main: https://github.com/buildarr/buildarr/actions/runs/4574761780
8. Tag the new release
9. Check that the release was automatically published to PyPI: https://github.com/buildarr/buildarr/actions/runs/4574917196
10. Push an update to http://github.com/buildarr/buildarr-docker to release to Docker Hub
11. Push an update to https://buildarr.github.io
12. Close release milestone
13. Create new release milestone
14. Close this issue

This is to reflect the changes made to the plugin architecture in #12.

• Move API functions to buildarr.plugins.sonarr.api
• Remove SonarrSecrets from get_initialize_js, and take in host URL and API key as normal
• Make SonarrConfig and SonarrSecrets use the generic interface, and replace base class type definitions with specific ones
• Split SonarrConfig into SonarrInstanceConfig (instance-specific) and SonarrConfig (global)
• Remove unnecessary shebangs from plugin.py and cli.py
• Remove all extra newlines left where the file descriptor used to be
• Define the initialize.js response parsing regex as a pre-compiled re.Pattern (INITIALIZE_JS_RES_PATTERN)

An error in the implementation of the new instance secrets fetching code means that Buildarr fetches instance secrets that are not in secrets.json, but it does not write the successfully retrieved secrets to the variable that gets added to the secrets data structure.

This results in Buildarr never being able to run unless secrets.json is hand-written to contain the cached secrets.

At the moment, Buildarr implictly accepts the return value of SecretsPlugin.get as the correct secrets for the instance, and any cached secrets from secrets.json are used as-is.

Usually this is no problem, but Buildarr should actively check that these secrets are usable.

Add a SecretsPlugin.test() -> bool function to the specification and all vendored plugins, where the the plugin performs a connection test to the instance using the secrets, and returning True or False depending on the result.

Integrate use of this function into the Buildarr run.

Checklist:

1. Set milestone to v0.4.2
2. Get the raw changelog using the following command:

   $ git log --oneline --decorate v0.4.1..HEAD

3. Update the tool.poetry.version field in pyproject.toml
4. Add release notes to docs/release-notes.md
5. Create pull request: #109
6. Merge pull request
7. Check that the CI pipeline passed on main: https://github.com/buildarr/buildarr/actions/runs/4698640321
8. Tag the new release
9. Check that the release was automatically published to PyPI: https://github.com/buildarr/buildarr/actions/runs/4698663427
10. Push an update to http://github.com/buildarr/buildarr-docker to release to Docker Hub
11. Push an update to https://buildarr.github.io
12. Close release milestone
13. Create new release milestone
14. Close this issue

In-place assignments to models take place in a few locations, mainly for populating dynamically assigned attributes during the rendering stage, or resolving attributes set from an instance link.

Validation of in-place assignments in Pydantic models are disabled by default, but we want them enabled in this case to ensure the model is always in a correct state. So up until this point, any configuration models that have a rendering stage or use instance links are expected to enable validate_assignments in their class config.

We should set validate_assignment to True globally, so that this is no longer necessary, and because it is generally better practice to do so.

This might have implications for existing models (incorrectly constrained models might raise validation errors), so do this on a backwards-incompatible release, to ensure plugins are compatible with the change ahead of time.

• Sonarr V4 - Development being tracked in this issue: buildarr/buildarr-sonarr#7
• Radarr - Now released as buildarr-radarr: https://github.com/buildarr/buildarr-radarr
• Prowlarr - Now released as buildarr-prowlarr: https://github.com/buildarr/buildarr-prowlarr
• Bazarr
• Unmanic
• Tdarr
• Lidarr
• Readarr
• Mylar
• Jellyseerr - Now released as buildarr-jellyseerr: https://github.com/buildarr/buildarr-jellyseerr

This allows additional constraints to be added to Password by plugins via Annotated.

In some resources, it is expected that a URL to a local server without a fully-qualified domain name (FQDN) can be passed in to an attribute (e.g. API URL).

The current validation type that Buildarr uses for these, HttpUrl, is designed to be used with public websites and is too strict, i.e. it will reject URLs from localhost or some other non-qualified hostname.

All of these references must be changed to the less strict AnyHttpUrl.

https://docs.pydantic.dev/usage/types/#urls