minimization / content-resolver Goto Github PK

Hello,

we'd like to keep an eye on @core and @core dependencies. Looking at https://github.com/minimization/content-resolver/blob/master/config_specs/workload.yaml, defining workloads using groups is not supported.

Please add support for defining workloads using groups, not just packages.

Thank you.

The Views section has many items, currently in a random sort, which makes it hard to use. Let's make it sorted alphabetically.

There's a lot of white space in the HTML output, get rid of that.

This should help some of the larger lists load faster.

See python3-toml being in the ELN repo:
https://odcs.fedoraproject.org/composes/production/latest-Fedora-ELN/compose/AppStream/x86_64/os/Packages/python3-toml-0.10.2-11.eln125.noarch.rpm

but the app does not think it exist in eln:
https://tiny.distro.builders/view-rpm--view-eln--python3-toml.html

In the backend, the script first installs all bases and use cases into directories and then queries them to get the data it needs.

That's quite inefficient and takes a long time (but I knew how to do that!).

So we need to find a way to get the same data without the installation step. I'm sure it's possible and it might be even quite easy.

Content Resolver can recommend maintainers of shared dependencies in views.

However, it only takes the workload definitions and the dependency information into account. It doesn't allow maintainers to arbitrarily volunteer to own something (like a Python maintainer offering to own some Python-related packages, without necessarily pulling them in themselves).

I want to add a functionality to allow maintainers to explicitly maintain a package. This needs to then influence the dependency recommendations, too.

My goal is to make Content Resolver easier to contribute by others. And because it's a fairly complex thing, people need to have confidence that their changes didn't break something. Especially now when one of the things I want this project to focus on is performance improvements (which will involve concurency and other fun stuff).

So let's set up a CI and a set of tests for the core functionality, to make sure our changes don't break Content Resolver.

The initial setup (this issue) is about:

• Setting up a CI for this repo using GitHub Actions (needs to run in a Fedora container)
• A basic test that just runs Content Resolver with the test configs on every PR

Some workloads (like KDE Plasma Desktop) have numerous weak dependencies that lead to a fully functional experience. These weak dependencies are declared as weak dependencies as a compromise, not because the intended desktop experience is to go without them.

It would be great if this intention can be codified in the workloads being tracked by the content resolver, so that we can have weak dependencies included (obviously denoted specially in the graph and charts) so that we can be sure we've captured everything.

The Use cases by releases view is empty now!

It needs to list all the use cases, and for each to offer reports comparing the use case on all Fedora releases. Because there might be multiple bases each use case is install on, it needs to offer potentially multiple reports per use case — each for one base.

So the items on the page linked above would look like this:

Apache HTTP Server

compared between all releases.
Fedora Container Base | Empty base

And the reports something like this:

As in the Use cases on bases reports, the required packages could be indicated in the table as well.

When comparing images, it's easier to read the evolution of image changes if the columns move left to right, i.e. F30 --> F31 --> Rawhide.

https://tiny.distro.builders/view--view-c9s.html

How can I browse the content set?

Build dependencies are currently resolved in an external service (dep-tracker because every SRPM (source package) needs to be rebuilt for all architectures. That's necessary because dependencies can vary across different architectures. And unlike with RPMs (binary packages), SRPMs are not distributed per-architecture, as their primary use case is to distribute the sources rather than provide dependency information.

Because the build dependencies are resolved by an external service , the results might be lagging even several hours. There's also currently no distinction between direct build dependencies and dependencies of build dependencies in on the package details page.

So I want to figure out how to optimise this situation.

Currently there's just a single level of unwanted packages.

Implement multiple levels of unwanted packages:

• unwanted in the runtime set
• unwanted completely (runtime + build)

And also have a distinction between

• unwanted in the set
• unwanted only by a specific maintainer (they don't want to own the package, but are OK if someone else does)

This must not over-complicate the user experience. So I need to come up with a simple and elegant solution.

Recently, I have removed dependency on rubygem-minitest-reporters from rubygem-zeitwerk and from rubygem-nokogiri. As a result, rubygem-minitest-reporters should soon disappear from content resolver. That is fine.

However, it would be useful to see the history. Even if it was just historical snapshots of the content. That would allow to compare the evolution of content set.

As seen in https://tiny.distro.builders/workload--sst_cs_infra_services-subversion-eln--fedora-empty-base--repository-fedora-eln--aarch64.html

 Problem 1: conflicting requests
  - nothing provides python(abi) = 3.9 needed by python3-subversion-1.14.0-6.module_eln+13718+74d85072.aarch64
 Problem 2: package subversion-perl-1.14.0-6.module_eln+13718+74d85072.aarch64 requires perl(:MODULE_COMPAT_5.32.1), but none of the providers can be installed
  - package subversion-perl-1.14.0-6.module_eln+13718+74d85072.aarch64 requires libperl.so.5.32()(64bit), but none of the providers can be installed
  - conflicting requests
  - package perl-libs-4:5.32.1-477.module_eln+13903+73720ee2.aarch64 is filtered out by modular filtering

Whenever I display page like https://tiny.distro.builders/workload-overview--httpd-no-weak-deps--repository-fedora-rawhide.html on my Firefox under XFCE, the canvas with the graph gradually grows vertically, keeping CPU quite busy.

Looking at https://www.chartjs.org/docs/latest/general/responsive.html#important-note, the div around should be with position: relative. I am able to stop the growth when I put say style="position: relative; height: 80vh" (in Firefox web developer editor) to the div class="card-body". However, I have no idea what value of height I should pick for a potential pull request.

The Important note also suggest that another way of fixing the problem is disabling the resize altogether, with `maintainAspectRation: true'. But I have no idea what that change would break and how important the resizing is for Content Resolver's operation.

I don't know what causes that issue to manifest itself on my Firefoxes and how to prevent it on my side, while I understand it does not happen for others. However, since the Chart.js documentation says "this method requires the container to be relatively positioned", I believe Content Resolver should go with that recommendation/requirement anyway.

Pushing package lists and the overall size as a plain text into git will give us a nice view into the history — we'll be able to see the package and size difference between any points in time.

And thanks to Pagure or Github, we'll get a free web UI for that!

Since all the results are already pushed to the reports repo, all that needs to happen is to output those lists.

The format could be as simple as:

< Use case name >
on < Base name >
42 MB
---
pkg1
pkg2
pkg3

Several teams would appreciate being notified they were next in line (or soon to be) for maintaining a package whose dependency another team (so far the maintainer) had dropped. This would ideally be achieved via some automated means based on CR data.

For example: team A is the 3rd in the line of succession to maintain a package. Team Z is the current maintainer and has dropped the dependency. Accordingly, team A is now the 2nd in the line of succession for maintaining the package. Team Y is the likely new maintainer as soon as team Z negotiates it (which probably was already underway).

As such, a notification should be sent to team A alerting it of the new status quo, including the possibility of team Y also looking forward to dropping its dependency and thus request team A to own the package. Team A would then be advised to prepare for that possible state, which could encourage a possible elimination of the dependency as well.

I am interested in the following use-case: after the packages of a workload are installed, remove a list of packages which I don't want to be present. Such packages could be brought as dependencies.

This list could be provided as part of the workload configuration.

For minimisation one of the main tasks is identifying where my team has dependencies on packages which other teams have marked "unwanted".

I can't see an easy way to do this today, am I missing something?

I have to iterate through each package in https://tiny.distro.builders/view-unwanted--view-eln.html and check whether one of my team's packages is listed.

Ideally I want a view like https://tiny.distro.builders/view-rpm--view-eln--libdb.html but doing the inverse - per maintainer / team:

• a list of packages in my workloads which runtime-require packages marked unwanted
• same for weak deps
• same for build deps

Already partially addressed by the JSON payloads that can be retrieved in some views.
It wold be also useful to dynamically retrieve the recommendations that Content Resolver issues, based on the multikple SSTs that require a package at different levels of dependency

I really love the listing where the UI tells me the size of a binary RPM: would it make sense to break this down even further and list all the files an RPM install + size of the files?

Hello, could you please help me understand why is this package in AppStream?

https://tiny.distro.builders/view-rpm--view-eln--python3-babel.html

It is runtime-required by 3 packages - 2 are in buildroot and one is in CRB.

We maintain this package as a CRB-only dependency of Sphinx (also CRB-only). We do not wish to maintain it in AppStream, but we do not know who should own it there. Based on the visible data, this should also be in CRB-only.

Thanks.

When packages are listed explicitly, for example

  packages:
  - httpd

in https://github.com/minimization/content-resolver-input/blob/master/configs/httpd-no-weak-deps.yaml, the package is then listed as (required) in the output, for example at https://tiny.distro.builders/workload--httpd-no-weak-deps--fedora-container-base--repository-fedora-rawhide--x86_64.html.

However, when packages are listed via groups like

  groups:
  - core

in https://github.com/minimization/content-resolver-input/blob/master/configs/sst_security_readiness-core-weak.yaml, the packages installed because they are explicitly listed in the groups are not distinguished from their dependencies, they are all listed as (dependency), see https://tiny.distro.builders/workload--sst_security_readiness-core-weak--fedora-empty-base--repository-fedora-eln--x86_64.html.

The support for groups was added via #18 but it seems like the presentation of results should be amended as well, to make it clear in the output which packages are listed part of the groups (potentially also distinguishing mandatory and default ones) and which packages are dependencies.

Could I please get the dependency graph in a machine-readable form, preferably a single json file?

Clicking in the web app to explore a complex dep tree ultimately leads me to loops and dead ends. I basically get lost. I need the data, so I can visualize my own subgraph of what I need to see.

What needs to be in the data:

• components/source packages
• binary packages
• binary packages mapped to source packages (edges from binary packages to source packages)
• requires (edges from binary packages to binary packages)
• buildrequires (edges from source packages to binary packages)
• binary packages and/or components mapped to workloads
• information about unwanted-ness for each binary package and/or component

Thanks.

There doesn't appear to be any way to select an "alternative" in a workload. e.g. iptables could be iptables-nft or iptables-legacy. How do I select an alternative?

[root@vmhost-fedora-test1 ~]# alternatives --list
[..]
iptables                auto    /usr/sbin/iptables-legacy
ebtables                auto    /usr/sbin/ebtables-legacy
arptables               auto    /usr/sbin/arptables-nft

When comparing images, it's easier to read the evolution of image changes if the columns move left to right, i.e. F30 --> F31 --> Rawhide.