juliaregistries / registrator.jl Goto Github PK
View Code? Open in Web Editor NEWJulia package registration bot
Home Page: https://julialang.org/packages
License: Other
registrator.jl's People
Stargazers
Watchers
Forkers
fredrikekre ericforgy tanmaykm nhdaly testregistries adamslc invenia transgirlcodes iamed2 tpoisot dlfivefifty chakravala staticfloat nicoleepp mmolignano vdayanand simonbyrne visr nkottary davidanthoff sstadick hakkelt bauglir andreasnoack juliatagbot ianbutterworth giordano racinmat arnaudh tiagopereira bozenkhaa standardgalactic ylkcy marwahaha playfloor logankilpatrick juliacomputing jlperla quinnj eford thiago-simoes mortenpi davelesterregistrator.jl's Issues
Error registering KristofferC/OhMyREPL.jl#162
Repository: KristofferC/OhMyREPL.jl
Issue/PR: 162
Command: [register()]
Stacktrace:
version 0.5.0 already exists
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] check_new_version(::Array{VersionNumber,1}, ::VersionNumber) at ./version.jl:225
[3] #register#6(::String, ::Bool, ::Function, ::String, ::String) at /home/nkottary/Registrator.jl/src/Registrator.jl:172
[4] #register at /home/nkottary/Registrator.jl/src/Server.jl:0 [inlined]
[5] handle_register(::Main.Registrator.RegServer.RegParams) at /home/nkottary/Registrator.jl/src/Server.jl:465
[6] handle_register_events(::Main.Registrator.RegServer.RegParams) at /home/nkottary/Registrator.jl/src/Server.jl:452
[7] tester() at /home/nkottary/Registrator.jl/src/Server.jl:481
[8] (::getfield(Main.Registrator.RegServer, Symbol("##4#7")))() at /home/nkottary/Registrator.jl/src/Server.jl:358
[9] recover(::getfield(Main.Registrator.RegServer, Symbol("##4#7"))) at /home/nkottary/Registrator.jl/src/Server.jl:347
[10] (::getfield(Main.Registrator.RegServer, Symbol("##3#6")))() at ./task.jl:259
JuliaRegistrar should remove the branch when the PR is merged
Allow additional registries for verification of package names/uuids in register
Dependencies are verified here only in the registry you're using to register. It should be possible to pass extra registries for verification for packages with cross-registry dependencies.
Server experiences GIT errors on Heroku
@nkottary On Heroku, I've used heroku container:push and heroku container:release to get a Registrator instance running and I'm testing it with BioJulia/FASTX.jl#2.
During the handling of a registration even however there seems to be an issue with cloning or finding where the repositories cloned are. I'm no Heroku expert but I'm going to try and investigate why this is happening, but wanted to list this here to see if anyone has any thoughts on this?
2019-04-09T10:27:50.193120+00:00 heroku[web.1]: State changed from crashed to starting
2019-04-09T10:28:00.079100+00:00 heroku[web.1]: Starting process with command `/bin/sh -c /bin/bash\ run.sh`
2019-04-09T10:28:27.286139+00:00 app[web.1]: ┌ Info: Starting server...
2019-04-09T10:28:27.286173+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:936
2019-04-09T10:28:28.320327+00:00 heroku[web.1]: State changed from starting to up
2019-04-09T10:28:28.148755+00:00 app[web.1]: ┌ Info: GitHub webhook starting...
2019-04-09T10:28:28.148778+00:00 app[web.1]: │ trigger = r"@bioreg (.*?)$"
2019-04-09T10:28:28.148781+00:00 app[web.1]: │ http_ip = 0.0.0.0
2019-04-09T10:28:28.148783+00:00 app[web.1]: │ http_port = 54778
2019-04-09T10:28:28.148785+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:923
2019-04-09T10:30:28.924599+00:00 app[web.1]: ┌ Debug: Received event for BioJulia/FASTX.jl, phrase: RegexMatch("@bioreg `register()`", 1="`register()`")
2019-04-09T10:30:28.924614+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:588
2019-04-09T10:30:28.947150+00:00 app[web.1]: ┌ Debug: Checking if comment is by collaborator
2019-04-09T10:30:28.947155+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:381
2019-04-09T10:30:36.704321+00:00 app[web.1]: ┌ Debug: Comment is by collaborator
2019-04-09T10:30:36.704337+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:68
2019-04-09T10:30:36.716420+00:00 app[web.1]: ┌ Debug: Comment is on an issue
2019-04-09T10:30:36.716423+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:81
2019-04-09T10:30:36.753396+00:00 app[web.1]: ┌ Debug: Will use branch
2019-04-09T10:30:36.753400+00:00 app[web.1]: │ brn = master
2019-04-09T10:30:36.753403+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:83
2019-04-09T10:30:36.753435+00:00 app[web.1]: ┌ Debug: Comment is on a pull request
2019-04-09T10:30:36.753438+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:90
2019-04-09T10:30:36.753506+00:00 app[web.1]: ┌ Debug: Event pre-check validity: true
2019-04-09T10:30:36.753508+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:125
2019-04-09T10:30:36.761458+00:00 app[web.1]: ┌ Info: Creating registration pull request for BioJulia/FASTX.jl branch: `master`
2019-04-09T10:30:36.761461+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:629
2019-04-09T10:30:37.681350+00:00 app[web.1]: ┌ Info: Processing register event
2019-04-09T10:30:37.681367+00:00 app[web.1]: │ reponame = BioJulia/FASTX.jl
2019-04-09T10:30:37.681369+00:00 app[web.1]: │ target_registry_name = bio
2019-04-09T10:30:37.681370+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:794
2019-04-09T10:30:37.828419+00:00 heroku[router]: at=info method=POST path="/" host=calm-brushlands-59162.herokuapp.com request_id=3dc1fc59-3f48-4a99-a0e8-5ef4ae21f89a fwd="192.30.252.45" dyno=web.1 connect=1ms service=13055ms status=200 bytes=52 protocol=https
2019-04-09T10:30:38.452664+00:00 app[web.1]: ┌ Debug: Getting gitcommit object for sha
2019-04-09T10:30:38.452679+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:476
2019-04-09T10:30:38.614210+00:00 app[web.1]: ┌ Debug: Getting tree object for sha
2019-04-09T10:30:38.614214+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:478
2019-04-09T10:30:38.735229+00:00 app[web.1]: ┌ Debug: Project file found
2019-04-09T10:30:38.735245+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:484
2019-04-09T10:30:38.735252+00:00 app[web.1]: ┌ Debug: Getting projectfile blob
2019-04-09T10:30:38.735254+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:486
2019-04-09T10:30:39.020974+00:00 app[web.1]: ┌ Debug: Decoding base64 projectfile contents
2019-04-09T10:30:39.020987+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:494
2019-04-09T10:30:39.231773+00:00 app[web.1]: ┌ Debug: Checking project file validity
2019-04-09T10:30:39.231777+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:497
2019-04-09T10:30:39.359284+00:00 app[web.1]: ┌ Debug: Checking whether Project.toml is non-empty and parseable
2019-04-09T10:30:39.359307+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:414
2019-04-09T10:30:39.507952+00:00 app[web.1]: ┌ Debug: Checking whether Project.toml contains name, uuid and version
2019-04-09T10:30:39.507956+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:436
2019-04-09T10:30:39.527710+00:00 app[web.1]: ┌ Debug: Event validity: true
2019-04-09T10:30:39.527718+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:315
2019-04-09T10:30:41.939738+00:00 app[web.1]: ┌ Debug: get info from package registry
2019-04-09T10:30:41.939753+00:00 app[web.1]: └ @ Registrator /home/registrator/.julia/packages/Registrator/MSkwb/src/register.jl:168
2019-04-09T10:30:41.939972+00:00 app[web.1]: ┌ Debug: get up-to-date clone of registry
2019-04-09T10:30:41.939975+00:00 app[web.1]: └ @ Registrator /home/registrator/.julia/packages/Registrator/MSkwb/src/register.jl:174
2019-04-09T10:30:42.316497+00:00 app[web.1]: ┌ Debug: get up-to-date clones of registry dependencies
2019-04-09T10:30:42.316512+00:00 app[web.1]: └ @ Registrator /home/registrator/.julia/packages/Registrator/MSkwb/src/register.jl:179
2019-04-09T10:30:49.758285+00:00 app[web.1]: ┌ Debug: branch registry repo
2019-04-09T10:30:49.758305+00:00 app[web.1]: └ @ Registrator /home/registrator/.julia/packages/Registrator/MSkwb/src/register.jl:188
2019-04-09T10:30:49.834584+00:00 app[web.1]: ┌ Debug: cleaning up possibly inconsistent registry
2019-04-09T10:30:49.834588+00:00 app[web.1]: │ registry_path = /home/registrator/registries/ccbd2cc2-2954-11e9-1ccf-f3e7900901ca
2019-04-09T10:30:49.834590+00:00 app[web.1]: │ err = nothing
2019-04-09T10:30:49.834591+00:00 app[web.1]: └ @ Registrator /home/registrator/.julia/packages/Registrator/MSkwb/src/register.jl:365
2019-04-09T10:30:50.228875+00:00 app[web.1]: ┌ Info: Unexpected error: IOError: could not spawn `git -C /home/registrator/registries/ccbd2cc2-2954-11e9-1ccf-f3e7900901ca -c user.name=BioJuliaRegistrar -c [email protected] checkout -qf master`: no such file or directory (ENOENT)
2019-04-09T10:30:50.228893+00:00 app[web.1]: │ Stacktrace:
2019-04-09T10:30:50.228895+00:00 app[web.1]: │ [1] _spawn_primitive(::String, ::Cmd, ::Array{Any,1}) at ./process.jl:400
2019-04-09T10:30:50.228898+00:00 app[web.1]: │ [2] #505 at ./process.jl:413 [inlined]
2019-04-09T10:30:50.228900+00:00 app[web.1]: │ [3] setup_stdios(::getfield(Base, Symbol("##505#506")){Cmd}, ::Array{Any,1}) at ./process.jl:497
2019-04-09T10:30:50.228902+00:00 app[web.1]: │ [4] _spawn at ./process.jl:412 [inlined]
2019-04-09T10:30:50.228903+00:00 app[web.1]: │ [5] #run#515(::Bool, ::Function, ::Cmd) at ./process.jl:725
2019-04-09T10:30:50.228905+00:00 app[web.1]: │ [6] run at ./process.jl:724 [inlined]
2019-04-09T10:30:50.228908+00:00 app[web.1]: │ [7] #register#7(::String, ::Array{String,1}, ::Bool, ::Dict{String,String}, ::Function, ::String, ::Pkg.Types.Project, ::String) at /home/registrator/.julia/packages/Registrator/MSkwb/src/register.jl:190
2019-04-09T10:30:50.228909+00:00 app[web.1]: │ [8] (::getfield(Registrator, Symbol("#kw##register")))(::NamedTuple{(:registry, :registry_deps, :push, :gitconfig),Tuple{String,Array{String,1},Bool,Dict{String,String}}}, ::typeof(Registrator.register), ::String, ::Pkg.Types.Project, ::String) at ./none:0
2019-04-09T10:30:50.228913+00:00 app[web.1]: │ [9] handle_register(::Registrator.RegServer.RequestParams{Registrator.RegServer.IssueTrigger}, ::Dict{String,Any}) at /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:831
2019-04-09T10:30:50.228914+00:00 app[web.1]: │ [10] handle_events(::Registrator.RegServer.RequestParams{Registrator.RegServer.IssueTrigger}) at /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:796
2019-04-09T10:30:50.228916+00:00 app[web.1]: │ [11] (::getfield(Registrator.RegServer, Symbol("#do_action#10")))() at /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:908
2019-04-09T10:30:50.228917+00:00 app[web.1]: │ [12] #recover#9(::Int64, ::Int64, ::Int64, ::Function, ::String, ::getfield(Registrator.RegServer, Symbol("#keep_running#12")), ::getfield(Registrator.RegServer, Symbol("#do_action#10")), ::getfield(Registrator.RegServer, Symbol("#handle_exception#11"))) at /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:890
2019-04-09T10:30:50.228919+00:00 app[web.1]: │ [13] recover at /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:888 [inlined]
2019-04-09T10:30:50.228920+00:00 app[web.1]: │ [14] request_processor at /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:911 [inlined]
2019-04-09T10:30:50.228921+00:00 app[web.1]: │ [15] (::getfield(Registrator.RegServer, Symbol("##16#18")))() at ./task.jl:259
2019-04-09T10:30:50.228923+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:799
2019-04-09T10:30:50.252435+00:00 app[web.1]: ┌ Debug: An unexpected error occured during registration.
2019-04-09T10:30:50.252439+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:580
2019-04-09T10:30:50.252446+00:00 app[web.1]: ┌ Debug: Posting comment to PR/issue
2019-04-09T10:30:50.252447+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:518
2019-04-09T10:30:50.977531+00:00 app[web.1]: ┌ Info: Done processing register event
2019-04-09T10:30:50.977546+00:00 app[web.1]: │ reponame = BioJulia/FASTX.jl
2019-04-09T10:30:50.977553+00:00 app[web.1]: │ target_registry_name = bio
2019-04-09T10:30:50.977563+00:00 app[web.1]: └ @ Registrator.RegServer /home/registrator/.julia/packages/Registrator/MSkwb/src/Server.jl:802
2019-04-09T10:30:50.979095+00:00 heroku[router]: at=info method=POST path="/" host=calm-brushlands-59162.herokuapp.com request_id=ba1ae3b0-4a59-4411-be90-943ca8181763 fwd="192.30.252.34" dyno=web.1 connect=1ms service=29ms status=204 bytes=50 protocol=https
Running `register` for a new package does not update Registry.toml
I did:
julia> using Pkg, Registrator
julia> tree_hash = String(readchomp(`git rev-parse master^\{tree\}`))
"6e0456db2c9c814ba80d711988a374da5f8a6115"
julia> url = "https://gitlab.invenia.ca/eric.davies/BlankPackage.jl.git"
"https://gitlab.invenia.ca/eric.davies/BlankPackage.jl.git"
julia> inv_reg_url = "https://gitlab.invenia.ca/eric.davies/PackageRegistry"
"https://gitlab.invenia.ca/eric.davies/PackageRegistry"
julia> Registrator.register(url, project, tree_hash; registry=inv_reg_url, push=true)
remote:
remote: To create a merge request for register/BlankPackage/v0.1.0, visit:
remote: https://gitlab.invenia.ca/eric.davies/PackageRegistry/merge_requests/new?merge_request%5Bsource_branch%5D=register%2FBlankPackage%2Fv0.1.0
remote:
Branch 'register/BlankPackage/v0.1.0' set up to track remote branch 'register/BlankPackage/v0.1.0' from 'origin'.
Registrator.RegBranch("BlankPackage", v"0.1.0", "register/BlankPackage/v0.1.0", nothing)It generated the new package directory properly but did not change Registry.toml at all, so there is no entry for that package.
Register Registrator?
Registrator does not appear to be registered in General, despite having tagged versions.
authentication
How do we know which PRs we should run through the whole registration CI process? We don't want to allow arbitrary people to make a PR and automatically kick off CI since it's fairly easy to see how that allows people to run arbitrary code on our registration CI machines (bitcoin mining, anyone?). If the person opening the PR is a committer, that's fine, we can just kick off right away. If the person opening the PR is not a committer, however, we should probably require an approval of the PR from someone who is a committer to the repo. There is a GitHub interface for PR approvals, so we should probably use that on GitHub at least. On other platforms, we should use whatever comparable approval API there is or at the very minimum, just look for a comment of the form "approve" from someone with commit bit.
Thoughts?
[FR] Allow tagging by adding a comment to a commit
This will be convenient. You make a commit locally, with title whatever you want and with comment body "@JuliaRegistrator register()". And then when you push, the registrator does its job without you having to open your mozilla firefox or chrome or etc.
Automate merging/tagging/releasing
- A
Generalregistry admin comments on the registration PR with@JuliaRegistrator approved()if they think everything on the PR looks good. - The corresponding PR (if exists) on the package repo is then merged and a release is created.
- The
Generalregistry PR is merged.
A package should not be allowed to change its UUID
This will break all packages that depend on it: KristofferC/OhMyREPL.jl#156.
experience from trying out the process
Thanks for all the great work. I tried registering a package, here are some suggestions:
-
README should clarify that the user should install the relevant app, currently RegistratorTest app
-
It would be great if commands did not have to be quoted with `` (like bors), if that is necessary for some reason, please clarify in the README.
-
It is unclear if (a) issue edits or (b) issue comments trigger the bot. Experimentally, comments seem to work.
-
I suspect that tagging a new version should be triggered the same as registering, please clariy this. In general, please add some detailed suggestions about version/release management, as various ideas were discussed and it may not be clear which one ended up being implemented.
Allow registering with a tag, like with a branch
You can provide a branch to register, but it would be nice to be able to provide a tag, especially considering the Registrator app doesn't tag.
there should be a configuration file
Instead of having to change an included source: https://github.com/JuliaComputing/Registrator.jl/blob/ef3e4b11c03fd24796a16dd24e1badb24524d1e7/src/Server.jl#L13
That will let the same packaged setup to be run with different configurations.
Error registering KristofferC/OhMyREPL.jl#153
Repository: KristofferC/OhMyREPL.jl
Issue/PR: 153
Command: [register()]
Stacktrace:
version 0.5.1 skips over 0.5.0
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] check_new_version(::Array{VersionNumber,1}, ::VersionNumber) at ./version.jl:228
[3] #register#6(::String, ::Bool, ::Function, ::String, ::String) at /home/nkottary/Registrator.jl/src/Registrator.jl:172
[4] #register at /home/nkottary/Registrator.jl/src/Server.jl:0 [inlined]
[5] handle_register(::Main.Registrator.RegServer.RegParams) at /home/nkottary/Registrator.jl/src/Server.jl:465
[6] handle_register_events(::Main.Registrator.RegServer.RegParams) at /home/nkottary/Registrator.jl/src/Server.jl:452
[7] tester() at /home/nkottary/Registrator.jl/src/Server.jl:481
[8] (::getfield(Main.Registrator.RegServer, Symbol("##4#7")))() at /home/nkottary/Registrator.jl/src/Server.jl:358
[9] recover(::getfield(Main.Registrator.RegServer, Symbol("##4#7"))) at /home/nkottary/Registrator.jl/src/Server.jl:347
[10] (::getfield(Main.Registrator.RegServer, Symbol("##3#6")))() at ./task.jl:259
Error registering KristofferC/OhMyREPL.jl#155
Repository: KristofferC/OhMyREPL.jl
Issue/PR: 155
Command: [register()]
Stacktrace:
ArgumentError: Malformed UUID string: "5fbsdf4-9ced-5910-84b2-373655c76a03"
Stacktrace:
[1] Base.UUID(::String) at ./uuid.jl:39
[2] Pkg.Types.Project(::Dict{String,Any}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Pkg/src/Types.jl:437
[3] #read_project#41(::Nothing, ::Function, ::Base.GenericIOBuffer{Array{UInt8,1}}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Pkg/src/Types.jl:492
[4] read_project at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Pkg/src/Types.jl:480 [inlined]
[5] is_pfile_valid(::String) at /home/nkottary/Registrator.jl/src/Server.jl:123
[6] Main.Registrator.RegServer.ProcessedParams(::Main.Registrator.RegServer.RegParams) at /home/nkottary/Registrator.jl/src/Server.jl:188
[7] handle_register(::Main.Registrator.RegServer.RegParams) at /home/nkottary/Registrator.jl/src/Server.jl:462
[8] handle_register_events(::Main.Registrator.RegServer.RegParams) at /home/nkottary/Registrator.jl/src/Server.jl:452
[9] tester() at /home/nkottary/Registrator.jl/src/Server.jl:481
[10] (::getfield(Main.Registrator.RegServer, Symbol("##4#7")))() at /home/nkottary/Registrator.jl/src/Server.jl:358
[11] recover(::getfield(Main.Registrator.RegServer, Symbol("##4#7"))) at /home/nkottary/Registrator.jl/src/Server.jl:347
[12] (::getfield(Main.Registrator.RegServer, Symbol("##3#6")))() at ./task.jl:259
Add package validation
Originating issue: JuliaLang/Pkg.jl#1094 (case sensitive file names breaks MacOS/Windows builds)
It would be great to have a set of validations that guide package developers and help finding problems early. With simple checks (and modest effort?) the quality of life of developers and users could be improved.
The validations I'm referring to are in vein of Rs check.
See http://r-pkgs.had.co.nz/check.html for inspiration:
- Checking for hidden files and directories.
- Checking for portable file names.
- Checking for sufficient/correct file permissions.
- Checking installed package size.
- ...
Having this validation on CI as well as available for package authors would be great. Conceptionally this validations might run together with or prior to the packages test.
pre-release version is allowed?
I tried to register 0.2.2-DEV JuliaFolds/Transducers.jl@9d5ff06 and it actually worked nkottary/General#39
Is it expected? Can Pkg handle it?
package registration process
See discussion in JuliaLang/Pkg.jl#849.
Verification of version specifiers
KristofferC/OhMyREPL.jl#162 adds a nonsense version specifier but the PR is created anyway. Will likely break Pkg if that happens.
Error registering KristofferC/OhMyREPL.jl#157
Repository: KristofferC/OhMyREPL.jl
Issue/PR: 157
Command: [register()]
Stacktrace:
version 0.5.0 already exists
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] check_new_version(::Array{VersionNumber,1}, ::VersionNumber) at ./version.jl:225
[3] #register#6(::String, ::Bool, ::Function, ::String, ::String) at /home/nkottary/Registrator.jl/src/Registrator.jl:172
[4] #register at /home/nkottary/Registrator.jl/src/Server.jl:0 [inlined]
[5] handle_register(::Main.Registrator.RegServer.RegParams) at /home/nkottary/Registrator.jl/src/Server.jl:465
[6] handle_register_events(::Main.Registrator.RegServer.RegParams) at /home/nkottary/Registrator.jl/src/Server.jl:452
[7] tester() at /home/nkottary/Registrator.jl/src/Server.jl:481
[8] (::getfield(Main.Registrator.RegServer, Symbol("##4#7")))() at /home/nkottary/Registrator.jl/src/Server.jl:358
[9] recover(::getfield(Main.Registrator.RegServer, Symbol("##4#7"))) at /home/nkottary/Registrator.jl/src/Server.jl:347
[10] (::getfield(Main.Registrator.RegServer, Symbol("##3#6")))() at ./task.jl:259
Testing Registrator server locally results in 400 errors
I've cloned the package with the intention of testing the server locally, but I'm running into issues.
First I cd to the image/scripts folder and made a config file:
[server]
http_port = 8080
http_ip = "127.0.0.1"
stop_file = "/tmp/stopregistrator"
log_level = "debug"
[github]
user = "BenJWard"
email = "[email protected]"
name = "Ben J. Ward"
token = "<< a token I generated for my GH user >>"
secret = "<< the secret for the GH app I created >>"
app_id = 27978
priv_pem = "biojulia-registrar.2019-04-07.private-key.pem"
[targets.bio]
repo = "https://github.com/BioJulia/BioJuliaRegistry"
base_branch = "master"
[registrator]
trigger = "bioreg"
report_issue = false
issue_repo = "JuliaComputing/Registrator.jl"
reply_comment = true
set_status = true
enc_key = "<< a random string of numbers (len 16), I made >>"
disable_pull_request_trigger = false
disable_approval_process = false
[slack]
alert = false
token = "xxxx"
channel = "xxxx"Next I started up the server locally:
N82106:scripts bward$ julia run.jl conf.toml > >(tee -a stdout.log) 2> >(tee -a stderr.log >&2)
┌ Warning: Package Registrator does not have Logging in its dependencies:
│ - If you have Registrator checked out for development and have
│ added Logging as a dependency but haven't updated your primary
│ environment's manifest file, try `Pkg.resolve()`.
│ - Otherwise you may need to report an issue with Registrator
└ Loading Logging into Registrator from project dependency, future warnings for Registrator are suppressed.
┌ Info: Starting server...
└ @ Registrator.RegServer /Users/bward/.julia/packages/Registrator/JvPmZ/src/Server.jl:920
┌ Info: GitHub webhook starting...
│ trigger = r"@bioreg (.*?)$"
│ http_ip = 127.0.0.1
│ http_port = 8080
└ @ Registrator.RegServer /Users/bward/.julia/packages/Registrator/JvPmZ/src/Server.jl:907Then, to expose my localhost to a url for the github app to use as a webhook, I used ngrok:
ngrok by @inconshreveable (Ctrl+C to quit)
Session Status online
Account Ben J. Ward (Plan: Free)
Version 2.3.25
Region United States (us)
Web Interface http://127.0.0.1:4040
Forwarding http://9f2f77f4.ngrok.io -> http://localhost:8080
Forwarding https://9f2f77f4.ngrok.io -> http://localhost:8080
Connections ttl opn rt1 rt5 p50 p90
1 0 0.00 0.00 0.12 0.12
HTTP Requests
-------------
Next I go to the github app and enter https://9f2f77f4.ngrok.io in the webhook url field and save. (Note I have not filled out a webhook secret, a setup URL, or a User authorization callback URL). Next, I install the app on BioJulia/FASTX.jl, open a new issue (BioJulia/FASTX.jl#2), make a comment to activate the bot and... nothing. All I get is a notice in my ngrok window saying:
HTTP Requests
-------------
POST / 400 Bad Request
The GitHub app has permissions:
Read-Only: Repository Contents
Read + Write: Issues, Pull Requests, Commit Statuses
And it subscribes to Issue Comment and Commit Comment events.
The personal access token I use for the server has been given the public_repo scope (I wasnt sure if it needs others).
@nkottary Is there something I'm doing that is obviously wrong? Is there more I need to do before I can get this running locally? Once I get it working locally I can try and get it running on heroku. I'm happy to add you to BioJulia so as you can see the GitHub App settings and so on that I've created.
Include "new" in the PR title for packages that register for the first time
check new version numbers
JuliaComputing@43b083e removed the call to check_new_version. Previously this made sure that the new version was a sensible next version number after the existing ones. I think that we should at least warn if a version number skips the known ones and we should definitely not allow re-registering existing version numbers.
version specifier should be "expanded"
We shouldn't put version specifiers in semver form directly into the registry. Instead they should be expanded like:
julia> Pkg.Types.semver_spec("1.0, ^2.0")
VersionSpec("[1.0.0-1, 2.0.0-2]")
and that version span should be used.
`register` method should cleanup/rollback edits on error
Changes made by register to registry on the disk should be rolled back on error. Leftover changes/new files cause subsequent ops to get confused.
It may be simpler to just discard the folder and clone afresh.
Server does not auto-login with GIT?
Following fixing #71, my Heroku installation of Registrator now gets further in the process, however, it halts as at some point GIT asks for a username and password:
2019-04-09T14:38:08.021638+00:00 app[web.1]: ┌ Debug: commit changes
2019-04-09T14:38:08.021644+00:00 app[web.1]: └ @ Registrator /home/registrator/.julia/packages/Registrator/DcyDi/src/register.jl:357
2019-04-09T14:38:08.031292+00:00 app[web.1]: ┌ Debug: push -f branch to remote
2019-04-09T14:38:08.031295+00:00 app[web.1]: └ @ Registrator /home/registrator/.julia/packages/Registrator/DcyDi/src/register.jl:370
2019-04-09T14:38:08.293122+00:00 app[web.1]: fatal: could not read Username for 'https://github.com': No such device or address
Is there something I have to do to get it to log in with the GitHub user automatically?
From reading around I can think of a few ways of doing it, one is to use ssh instead of http.
Another is to try and use the API token in the config to push to git push https://[email protected]/foo/bar instead of git -C /home/registrator/registries/ccbd2cc2-2954-11e9-1ccf-f3e7900901ca -c user.name=BioJuliaRegistrar -c [email protected] push -q -f -u origin register/FASTX/v0.1.0
@nkottary What do you think is the best way to get around this? I'm happy to submit a PR with any needed fixes.
Test issue
Test body
Verification of dependencies
KristofferC/OhMyREPL.jl#154 adds an (intentionally) faulty dependency that doesn't exist in the registry.
The PR to the registry seems to go through fine though [edit: the PR got updated so this link is no longer valid]: https://github.com/nkottary/General/pull/5/files#diff-74aa886f65423fb0e2de436bd6efc5baR26. This will likely break the package manager.
Namespacing?
To avoid name clashes, it would be great if registries could allow something like namespacing (similar to npm scopes), i.e. https://docs.npmjs.com/about-scopes.
I understand I can create my own registry and ask users to add my registry to their paths (or however that would work), but scoping seems ultimately doable and good UX.
This would be great for organizations. For example, I've recently been working on JuliaFinance. I think it makes sense for JuliaFinance to have it's own registry, but it would be nice to let users access this registery via the General registry like this:
pkg> add @JuliaFinance/PortfolioModels"@JuliaFinance" is registered in General registry and redirects to the JuliaFinance registry where it finds PortfolioModels.jl.
JuliaQuant also has a PortfolioModels.jl, but that looks quite different than what I would imagine for a package called PortfolioModels.jl so rather than wrestle over package names, scoping seems like a good solution.
Knowing this community, I'm almost certain this has already been discussed (I've brought it up on Slack I think), so any links would be appreciated, but I thought having an issue here could make sense too.
Edit: I created a new registry just to try to get a feel for how it could work.
It seems reasonable from a UX perspective to ask users to do a
pkg> registry add https://github.com/JuliaFinance/JuliaFinance.gitSo then the scoping is just telling Pkg which local registry to use, e.g.
pkg> add @JuliaFinance/Countriesjust means "Use the local JuliaFinance registry to add Countries.jl."
Seems like an easy revision to Pkg.add 🤔 Maybe I'll give it a shot.
Tests for Registrator.jl itself.
This package has as of right now 2 tests.
Both fairly auxillary.
That is kind of terrifying. Considering this is software that basically 100% of julia developers/contributors to the package ecosystem will be depending on.
(Related: for this kind of of thing, one really wants to do some from of mocking, so that the network components etc can be stubbed out so that everything can tested in parts.)
Warning about initial version
Registrator is telling me Warning: version 0.8.0 is invalid initial version (try 0.0.1, 0.1, 1.0) on a package where the previous version is 0.7.1. It doesn’t seem to block anything.
How to allow dependencies to be found in multiple registries?
I've seen from this PR, https://github.com/JuliaComputing/Registrator.jl/pull/56, that multiple registries can be used for dependency resolution. Namely that the register function accepts a registry_deps::Vector{String} argument, however the handle_register function in the Server, currently does not call register with a registry_deps argument:
rbrn = register(pp.cloneurl, Pkg.Types.read_project(copy(IOBuffer(pp.projectfile_contents))), pp.tree_sha;
registry=target_registry["repo"],
push=true,
gitconfig=Dict("user.name"=>config["github"]["user"], "user.email"=>config["github"]["email"]))So when I tested the server locally (for a BioJulia registry) with a package that has deps in General I saw:
Debug: cleaning up possibly inconsistent registry
│ registry_path = /Users/bward/repos/regist/image/scripts/registries/ccbd2cc2-2954-11e9-1ccf-f3e7900901ca
│ err = Error in `[deps]`: Package 'BioSymbols' with UUID: 3c28c6f8-a34d-59c4-9654-267d177fcfa9 not found in registry or stdlib
If I understand the code, it should be simple enough to allow a list of registries in the conf.toml, and let the server use them with the register method. Does that sound sensible? I can try this and PR it if it sounds reasonable.
Registrator doesn't work on Julia 1.0.3
The register code references Pkg.Types.Project which doesn't exist for the release of Pkg included in Julia 1.0.3
julia> using Pkg, Registrator
[ Info: Precompiling Registrator [4418983a-e44d-11e8-3aec-9789530b3b3e]
ERROR: LoadError: LoadError: UndefVarError: Project not defined
Stacktrace:
[1] getproperty(::Module, ::Symbol) at ./sysimg.jl:13
[2] top-level scope at none:0
[3] include at ./boot.jl:317 [inlined]
[4] include_relative(::Module, ::String) at ./loading.jl:1044
[5] include at ./sysimg.jl:29 [inlined]
[6] include(::String) at /Users/rory/.playground/share/testpkg/depot/packages/Registrator/FivKN/src/Registrator.jl:1
[7] top-level scope at none:0
[8] include at ./boot.jl:317 [inlined]
[9] include_relative(::Module, ::String) at ./loading.jl:1044
[10] include(::Module, ::String) at ./sysimg.jl:29
[11] top-level scope at none:2
[12] eval at ./boot.jl:319 [inlined]
[13] eval(::Expr) at ./client.jl:393
[14] top-level scope at ./none:3
in expression starting at /Users/rory/.playground/share/testpkg/depot/packages/Registrator/FivKN/src/register.jl:157
in expression starting at /Users/rory/.playground/share/testpkg/depot/packages/Registrator/FivKN/src/Registrator.jl:13
ERROR: Failed to precompile Registrator [4418983a-e44d-11e8-3aec-9789530b3b3e] to /Users/rory/.playground/share/testpkg/depot/compiled/v1.0/Registrator/Obz5J.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1203
[3] _require(::Base.PkgId) at ./loading.jl:960
[4] require(::Base.PkgId) at ./loading.jl:858
[5] require(::Module, ::Symbol) at ./loading.jl:853
julia version compatiblity should probably have a lower bound
KristofferC/OhMyREPL.jl#158 adds support for julia 0.1 but that is unmaintained. Should probably enforce julia 1.0+ (or 0.7+).
name bikeshed
@ararslan expressed minor disappointment that we didn't call this "Julia Registrar". It's not too late! In fact, if we're going to change the name, we should do it as early as possible. So let's do some bikeshedding. Some options:
- Keep both JuliaRegistrator as the org name and Registrator.jl as the package name.
- Change to JuliaRegistrary but keep Registrator.jl
- Change both to JuliaRegistrar and Registrar.jl
I omitted changing the package but not the org since that doesn't make sense to me. I'm happy to make a PR to change the names if we decide on something (it's kind of busy work).
Sorting should be according to "version" and not string.
See e.g. the sorting in https://github.com/nkottary/General/pull/5/files#diff-c258257db4ab0edbe4ee02a9dc78bb2a.
A concern for scientific projects
(pasting this from Slack)
The "old way", whereby tagging a release on GitHub meant also tagging a release in METADATA is very powerful and I hate to see it go. But for the sake of change and improvement old ways have to be forgotten. There is something from the old way that should not be forgotten though: that there is an automatic link between package release and GitHub release.
The main reason: scientific repositories that use e.g. ZENODO to have a DOI for their code base. ZENODO nicely integrates with GitHub and stores the data in a way that is directly tied with the GitHub release.
I fear that with the new system of "making a pull request" a bad thing may happen: simply forgetting to tag a GitHub release for every "Package release". Especially in the case of many subsequent bugfix releases I hope you can see this as a realistic scenario.
Make clear that the tag should be created after PR is merged
The message here: caseykneale/ChemometricsTools.jl#5 (comment) should IMO be updated to make clear that the tag should not be created until the registry PR is merged.
Alternate trigger methods + Git service integration
More or less pasted from Slack:
I had some thoughts on Registrator, namely the trigger method and git service (i.e. GitHub, GitLab, etc.) integration:
I personally don't like the current issue/PR/commitcomment release trigger. It feels clunky, as if we're misusing a feature. Some ideas:
- A push to a branch whose name parses to a version, i.e.
v1.2.3, or more explicitly,release-v1.2.3. One nice thing about this is you can keep the branch around if you want, or delete it so there are no artifacts of the release process other than the actual release. - A web interface. Doing it this way would make it much easier to decouple the bot from GitHub Apps, in fact I don't think any prior repo setup would be required if you could give authorization on the spot. Any Git service with OAuth could be supported in this manner. Thinking about this more, my opinion is that this is the best option.
I also did a bit of work trying to get GitLab.jl working on 1.0, and I came to the realization that the efforts to keep it in line with GitHub.jl will basically always fail (as they have already). It's not really surprising that GitHub.jl gets more attention, but I think we can help fix this by designing a package that abstracts the underlying service provider (GH, GL, etc.) away from the functionality (issues, PRs, etc.) since that functionality is mostly consistent between providers. Then adding a new provider would basically just be setting a bunch of URLs (very idealistic I admit, I haven't looked into the details of this very closely).
Feedback on these ideas is most welcome.
Test Plan
- Open an issue on an unregistered package repository with
register()in its contents or as comment.
-
a. Registrator should comment with an error message if:
- i. Package name does not have '.jl'
- ii. Package master branch has no
Project.toml - iii.
Project.tomlis empty/invalid
-
b. If there are no errors, Registrator should:
- i. Create a pull request on the registry containing the appropriate changes
- ii. PR body should contain Package name, version, creator and reviewer
- iii. Reply with a link to the PR on the issue
- iv. If PR already exists do nothing (only push to branch)
- Create a pull request that modifies the Project.toml in a registered package and have
register()as content or comment.
-
a. Registrator should comment with an error message if:
- i. Package name does not have '.jl'
- ii. PR does not edit
Project.toml - iii.
Project.tomlis empty/invalid
-
b. If there are no errors, Registrator should:
- i. Create a pull request on the registry containing the appropriate changes
- ii. PR body should contain Package name, version, creator and reviewer
- iii. Reply with a link to the PR on the issue
- iv. If PR already exists do nothing (only push to branch)
- For an issue comment of the form
register(branch)
-
a. Registrator should comment with an error message if:
- i. Package name does not have '.jl'
- ii. Branch
branchdoes not exist - iii.
Project.tomlis not found inbranch - iii.
Project.tomlonbranchis empty/invalid
-
b. If there are no errors, Registrator should:
- i. Create a pull request on the registry containing the appropriate changes
- ii. PR body should contain Package name, version, creator and reviewer
- iii. Reply with a link to the PR on the issue
- iv. If PR already exists do nothing (only push to branch)
- For things like API errors Registrator should:
- Raise an issue containing the stacktrace and event details
- reply saying an unexpected error occured and link to the issue
register() seems like it could be accidentally invoked
A bit of bike shedding but register() seems like it can occur in normal code people paste. Perhaps we should have something more name spaced to invoke Registrator?
Bug in compat test with comma-separated versions
Dependencies not found?
So I tried to register my package again today, and it seems like the julia registrator can't find the dependancies which I'm um downloading from the registry?
caseykneale/ChemometricsTools.jl#5
Error while trying to register: Error in [deps]: Package 'CSV' with UUID: 336ed68f-0bac-5ca0-87d4-7b16caf5d00b not found in registry or stdlib
Any advice appreciated. I'd love to help here, but I have no idea whats going on.
Nicer error messages for common errors.
When we hit an error that is "expected", i.e. is nothing wrong with Registrator itself we should probably try to give a nice error message and not dump an unrelated stacktrace, cf:
Observations from testing
Hi, thanks for making this. I tried it out with FastClosures and got several very helpful error messages (c42f/FastClosures.jl#12) before coming up with the following:
https://github.com/nkottary/General/pull/33/files
However I wondered whether this should also be a failure or a big warning: it changes the git tree hash of an existing version. (Obviously this will be required to fix things in some limited cases, but it seems dangerous to allow by default.)
Creating a new version, I then generated the following:
https://github.com/nkottary/General/pull/34/files
which looks better.
Now if this wasn't a test, I'd have liked to cancel issue 33 but I can't because I don't have permissions on nkottary/General. It might be nice if there was a JuliaRegistrator command to cancel a registration for cleanup purposes.
I also tried to clarify the docs a bit: JuliaRegistrator/juliaregistrator.github.io#1
Cheers!
Error registering nkottary/Example.jl:4
Repository: nkottary/Example.jl
Issue/PR: 4
Command: register()
Stacktrace:
UndefVarError: package_repo not defined
Stacktrace:
[1] (::getfield(Main.Registrator, Symbol("##2#3")){String,String})(::String) at /home/nkottary/Registrator.jl/src/Registrator.jl:84
[2] mktempdir(::getfield(Main.Registrator, Symbol("##2#3")){String,String}, ::String) at ./file.jl:581
[3] get_project at /home/nkottary/Registrator.jl/src/Registrator.jl:52 [inlined]
[4] #register#6(::String, ::Bool, ::Function, ::String, ::String) at /home/nkottary/Registrator.jl/src/Registrator.jl:121
[5] #register at /home/nkottary/Registrator.jl/src/Server.jl:0 [inlined]
[6] handle_register(::Main.Registrator.RegServer.RegParams) at /home/nkottary/Registrator.jl/src/Server.jl:377
[7] handle_register_events(::Main.Registrator.RegServer.RegParams) at /home/nkottary/Registrator.jl/src/Server.jl:364
[8] tester() at /home/nkottary/Registrator.jl/src/Server.jl:393
[9] (::getfield(Main.Registrator.RegServer, Symbol("##4#7")))() at /home/nkottary/Registrator.jl/src/Server.jl:278
[10] recover(::getfield(Main.Registrator.RegServer, Symbol("##4#7"))) at /home/nkottary/Registrator.jl/src/Server.jl:267
[11] (::getfield(Main.Registrator.RegServer, Symbol("##3#6")))() at ./task.jl:259
Keep key order in Package.toml
Would be nice to keep the order, name, uuid, repo instead of name, repo, uuid, see e.g. https://github.com/nkottary/General/pull/35/files
Document about versioning?
Readme does not say anything about how to specify what version should be tagged.
(Minor, major, patch).
I am guessing you use Project.toml to specify?
How can I use this for a non General registry?
We have our own registry for a bunch of lab internal stuff. Can I use Registrator for that use case? How would I configure it so that it works on our own registry, and not the General public one?
Is it general ready to be used?
allow package to be registered with multiple registries?
Either by having a registrator setup able to target more than one registries.
And/or by allowing multiple registrators to be configured for a repo.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.