Since some builds lag behind, opam logs should show the 8 letter sha hash of the opam-repo revision. This is included in the Dockerfile output but not the log error output.

Helps until we have #3

I have a package (in a PR: Opam publish camlp5 7.13 #17132) that's failing on a couple of platforms (camlp5 7.13) and am trying to use the opam ci infrastructure to investigate. I can get the logs via the web UI, but thought I'd try to learn how to use opam-ci. There seems to be a problem (or maybe this isn't what opam-ci is designed for). Neither the status nor logs commands show anything:

chet@digg:~/Hack/Camlp5/src/camlp5$ opam ci status -m [email protected] --filter=all 

Try 'opam-ci logs <pkg>' to see details of build failures for each package.

chet@digg:~/Hack/Camlp5/src/camlp5$ opam ci status camlp5

Try 'opam-ci logs <pkg>' to see details of build failures for each package.

chet@digg:~/Hack/Camlp5/src/camlp5$ opam ci logs camlp5
camlp5: No failures found with these constraints

Not sure what I'm doing wrong .....

Add a obi-gitlab command that generates the CI pipeline for a GitLab-based build

The OCaml version strings for the latest OCaml compiler version, the latest stable version and so on can be retrieved using the ocaml-version rather than being hardcoded in opam-ci itself.

We should port the hardcoded uses to library calls from ocaml-version, and add any utility function to that upstream library if necessary.

Right now the version number is hardcoded into src/opam-ci/main.ml and ought to be generated from the git-version or release version.

# File "src/obi-gitlab/gitlab.ml", line 239, characters 16-24:
# Error: This pattern matches values of type [? `Aarch32 ]
#        but a pattern was expected which matches values of type OV.arch
#        The second variant type does not allow tag(s) `Aarch32

I'm trying to install this by pinning it locally and get the error above.

Right now, we use the default Buildkite queue for every job in the infrastructure. A big bulk build will use up all the slots for an architecture, resulting in everything else being queued up.

To solve this, we need the following queues:

• bulk: for bulk builds
• infra: for infrastructure containers
• default: misc stuff

Unfortunately, there is no way to prioritise queues, so we need to overcommit them in the agents. This might mean that if everything is maxxed out, we are pushing heavy load averages. However, this will be slow, but not a fatal error.

There is some discussion on buildkite/feedback#147 about improving this situation upstream in Buildkite, but for now overcommit should be fine for us.

Right now every container uses the opam-repository master branch, which has the opam1 metadata. This allows (e.g.) opam-ci to use the opam-1 git revs to track PR tests, but increases the time it takes to do updates in the containers due to the opam1->2 conversion.

At some point, we need to coordinate with the datakit-ci and switch the containers to use the 2.0 branch natively, and test the conversion scripts separately.

The current views only list packages that fail. It would be useful to list packages that are marked as uninstallable against the latest version of OCaml, and do not have a newer version that does compile on the new compiler.

This would let us (e.g.):

• list packages that have not yet been ported to safe-string (due to 4.06 constraints)
• list packages that have not yet been ported to 4.07 due to ppx ast changes

We do need to filter out depfails, to keep the list manageable. In the spirit of ensuring that no bug in opam-ci should be unfixable, we do also need a tag to mark a package as "validly uninstallable", since the upstream may want to simply deprecate a package and not let it compile with latest versions of the compiler.

requested by @dra27

The current dashboard display requires an ANSI and Unicode tty due to the use of colours. This is not optimal for several reasons:

• it makes it difficult to use with screen readers
• file redirection is messy
• it also fails to work on Windows due to Consolas not supporting the right format (via @dra27)

We should fallback to a more verbose rendering format in these situations instead of blasting escape codes to the screen unconditionally.

There are some fields in the Obi metadata that are usually empty; for example the logs field. We can simply omit these from the sexp output rather than just redundantly output empty fields.

If thing that I often do when trying to debug a failing build is to check the version of the installed packages. With travis, this means scrolling a bit to find opam actions.

With obi it's not possible to scroll as the logs are trimed a bit; instead it would be nice to have a way to list either the packages which were planned to be installed and/or the list of packages successfully installed without the error.

While opam-ci status can filter by maintainers and tags, it cannot simply filter based on a package name. This can be added via positional parameters; e.g.:

opam-ci status mirage mirage-xen

I'm trying to install this tool on my default ocaml switch (5.1.0) but dependency resolution is timing out. My guess is that this is because it depends on jbuilder and not dune. Is this tool still supported?

For now kit-ty-kate/opam-check-all uses the obi library to the list of maintainers per packages but nothing more. To be able to use the obi fully in opam-check-all here what I would need:

• Full logs: opam-check-all needs access to the full log of the opam build. It is "common" to see errors being displayed outside the "opam error square" either because it was too long, because there is not enough informations or because one of the dependencies failed (and is not retrievable outside of this particular combination, e.g. some particular mix of depopts or depexts makes it fail in a way it wouldn't otherwise). Access to the latest logs through a link seems doable (for 3 versions of OCaml, it takes about 2Go of disk space)
• Obi.Index.params consistency: opam-check-all needs some kind of hint to know which versions of OCaml are available (are they hardcoded ?), whether x86_64 builds have been successfully done (can I be assured they are always available ?) and which distributions have been built. It might be good to have some kind of header in the index file to have this information available right away.

Concerning the distributions issue: I guess I'd have to make a ranking at some point. opam-check-all uses Debian Unstable as it seems to be the distribution with the most external package available (e.g. latest versions of LLVM, latest version of SDL used by tsdl, and so on…).
However if anybody have some hints about which distributions would be the most suited to display result by default (not Debian Stable in my opinion), it would be much appreciated.

The index doesn't currently store the opam-repository metadata, but it does have the revision of opam-repo that a given run was built against. Some runs (like ppc64le) naturally lag behind the faster amd64 builds, so different results come from different revisions of opam-repo.

This is problematic when reporting results, since we want to indicate to the user that a given branch may be lagging behind. We could extend the type pkgs (currently just a pkg list) to also include more information about the opam-repository revision hashes. Then the client could determine which the latest revision is, and report the mismatch in opam-ci logs without having to have a full checkout of opam-repository.

The lints branch of obi-logs contains opam-lint that is regularly run the main opam repository.

There is information in there relevant to package maintainers, such as broken distfiles and/or constraint inconsistencies. We should hook this into the main Obi index by parsing the output.

Fixing this is blocked on ocaml/opam#3342 in upstream opam, since the existing opam lint output is difficult to parse directly. (cc @AltGr as fyi)

Currently this is just obi logs

The obi index currently just prompts users to upgrade if it fails to parse the sexpression file in https://github.com/ocaml/obi-logs/tree/index.

We could allow for smoother upgrades by having the client have an epoch variable that is incremented on an incompatible index format change, and for the opam-ci client to checkout index-<epoch> to get the right branch. If a branch with a higher epoch exists, then the opam-ci client could warn the user to run opam update -u to get the latest version.

Alternatively, we could also commit explicitly versioned files instead of index.sxp and just use the same branch. This increases the data transfer required for the client though.

obi-buildkite index will delete unused metadata files in the builds branch, but the scripts in .buildkite do not currently push this GC back to the upstream obi-logs repository. This is necessary to ensure that the builds branch stays a reasonable size.

The opam-ci logs command currently outputs a Dockerfile which has to be manually cut-and-pasted and built. Instead, this functionality should be shifted into an opam-ci docker subcommand which can:

# generate an interactive container
opam-ci docker run xen-evtchn.1.0.3 --compiler=4.06 --distro=debian-9
# build an image called `local-container`
opam-ci docker build -t local-container xen-evtchn.1.0.3 --compiler=4.06 --distro=debian-9
# just output the Dockerfile to stdout
opam-ci docker file xen-evtchn.1.0.3 --compiler=4.06 --distro=debian-9 

There are some packages that are orphaned and could use a new maintainer. Add an `opam-ci status --filter=orphaned' to list these packages, so that willing community maintainers can fork them and/or refresh them to the latest versions of the compiler.

Detecting orphaned maintainer strings requires a few heuristics, as sometimes the field is blank and other times set to the opam-repository issues url.

See ocaml/opam-repository#12182 for one such example package (syslog)

I have this message:

> $ jbuilder exec -- opam-ci status -m [email protected]
opam-ci: We have encountered a total failure to parse the upstream
         metadata.
         Please run `opam update -u` to get the latest version of opam-ci
         that is compatible with the latest log format.
         If that does not work, you can try pinning to the development
         version of opam-ci via `opam pin add opam-ci --dev`.
         Please report this issue with at https://github.com/ocaml/obi/issues
         with this backtrace:
         (Failure "unable to find version string in index.sxp")

                  Raised at file "src/opam-ci/repos.ml", line 128, characters
         15-75

But both options do not work (my opam is up-to-date and I am running the dev version). I think the message should explain how to reset the state manually as well.