flathub-infra / flat-manager-hooks Goto Github PK

View Code? Open in Web Editor NEW

2.0 2.0 4.0 201 KB

Hooks for integrating Flathub's backend with flat-manager

License: Apache License 2.0

Rust 96.20% C 0.07% Shell 3.73%

flat-manager-hooks's People

Contributors

Stargazers

Watchers

Forkers

jameswestman razzeee aleixpol ximion

flat-manager-hooks's Issues

413 Payload Too Large On "Committing Builds"

My Flathub build gave me this error in the "Committing Builds" stage:

| Error: HTTP status client error (413 Payload Too Large) for url (https://hub.flathub.org/api/v1/job/106271/check/review)
| 
| Stack backtrace:
|    0: flathub_hooks::utils::retry
|    1: flathub_hooks::config::Config::mark_still_pending
|    2: flathub_hooks::review::do_review
|    3: flathub_hooks::main
|    4: std::sys_common::backtrace::__rust_begin_short_backtrace
|    5: std::rt::lang_start::{{closure}}
|    6: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
|              at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/ops/function.rs:284:13
|       std::panicking::try::do_call
|              at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:502:40
|       std::panicking::try
|              at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:466:19
|       std::panic::catch_unwind
|              at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panic.rs:142:14
|       std::rt::lang_start_internal::{{closure}}
|              at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/rt.rs:148:48
|       std::panicking::try::do_call
|              at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:502:40
|       std::panicking::try
|              at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:466:19
|       std::panic::catch_unwind
|              at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panic.rs:142:14
|       std::rt::lang_start_internal
|              at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/rt.rs:148:20
|    7: main
|    8: __libc_start_call_main
|              at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
|    9: __libc_start_main_impl
|              at ./csu/../csu/libc-start.c:392:3
|   10: _start
| 
| Check status updated to Failed("Check command cd \"/tank/repo/builds/65964\" && FLAT_MANAGER_BUILD_ID=\"65964\" FLAT_MANAGER_JOB_ID=\"106271\" \"/usr/local/bin/flathub-hooks\" \"review\" \"-c\" \"/etc/flathub-hooks.json\" exited unsuccessfully: exit status: 1, stderr: [2023-11-24T09:17:25Z INFO  flathub_hooks::review] Refs present in build: [\"appstream2/aarch64\", \"runtime/com.thebrokenrail.MCPIReborn.Debug/x86_64/stable\", \"app/com.thebrokenrail.MCPIReborn/x86_64/stable\", \"appstream/aarch64\", \"runtime/com.thebrokenrail.MCPIReborn.Sources/x86_64/stable\", \"screenshots/x86_64\", \"appstream2/x86_64\", \"app/com.thebrokenrail.MCPIReborn/aarch64/stable\", \"appstream/x86_64\", \"runtime/com.thebrokenrail.MCPIReborn.Debug/aarch64/stable\", \"screenshots/aarch64\"]\n[2023-11-24T09:17:28Z INFO  flathub_hooks::review] Submitting appdata for review: ReviewRequest { build_id: 65964, job_id: 106271 }\n[2023-11-24T09:17:29Z INFO  flathub_hooks::utils] HTTP status client error (413 Payload Too Large) for url (https://hub.flathub.org/api/v1/job/106271/check/review)\n[2023-11-24T09:17:29Z INFO  flathub_hooks::utils] Retrying (1/5) in 1 seconds...\n[2023-11-24T09:17:30Z INFO  flathub_hooks::utils] HTTP status client error (413 Payload Too Large) for url (https://hub.flathub.org/api/v1/job/106271/check/review)\n[2023-11-24T09:17:30Z INFO  flathub_hooks::utils] Retrying (2/5) in 2 seconds...\n[2023-11-24T09:17:32Z INFO  flathub_hooks::utils] HTTP status client error (413 Payload Too Large) for url (https://hub.flathub.org/api/v1/job/106271/check/review)\n[2023-11-24T09:17:32Z INFO  flathub_hooks::utils] Retrying (3/5) in 4 seconds...\n[2023-11-24T09:17:36Z INFO  flathub_hooks::utils] HTTP status server error (502 Bad Gateway) for url (https://hub.flathub.org/api/v1/job/106271/check/review)\n[2023-11-24T09:17:36Z INFO  flathub_hooks::utils] Retrying (4/5) in 8 seconds...\n[2023-11-24T09:17:44Z INFO  flathub_hooks::utils] HTTP status client error (413 Payload Too Large) for url (https://hub.flathub.org/api/v1/job/106271/check/review)\n[2023-11-24T09:17:44Z INFO  flathub_hooks::utils] Retrying (5/5) in 16 seconds...\n[2023-11-24T09:18:00Z INFO  flathub_hooks::utils] HTTP status client error (413 Payload Too Large) for url (https://hub.flathub.org/api/v1/job/106271/check/review)\nError: HTTP status client error (413 Payload Too Large) for url (https://hub.flathub.org/api/v1/job/106271/check/review)\n\nStack backtrace:\n   0: flathub_hooks::utils::retry\n   1: flathub_hooks::config::Config::mark_still_pending\n   2: flathub_hooks::review::do_review\n   3: flathub_hooks::main\n   4: std::sys_common::backtrace::__rust_begin_short_backtrace\n   5: std::rt::lang_start::{{closure}}\n   6: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once\n             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/ops/function.rs:284:13\n      std::panicking::try::do_call\n             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:502:40\n      std::panicking::try\n             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:466:19\n      std::panic::catch_unwind\n             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panic.rs:142:14\n      std::rt::lang_start_internal::{{closure}}\n             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/rt.rs:148:48\n      std::panicking::try::do_call\n             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:502:40\n      std::panicking::try\n             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:466:19\n      std::panic::catch_unwind\n             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panic.rs:142:14\n      std::rt::lang_start_internal\n             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/rt.rs:148:20\n   7: main\n   8: __libc_start_call_main\n             at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16\n   9: __libc_start_main_impl\n             at ./csu/../csu/libc-start.c:392:3\n  10: _start\n")
\ Job completed successfully
\ Check flathub-hooks has failed
Job failed: {'check_name': 'flathub-hooks', 'build_id': 65964, 'job_id': 106271, 'status': 3, 'status_reason': 'Check command cd "/tank/repo/builds/65964" && FLAT_MANAGER_BUILD_ID="65964" FLAT_MANAGER_JOB_ID="106271" "/usr/local/bin/flathub-hooks" "review" "-c" "/etc/flathub-hooks.json" exited unsuccessfully: exit status: 1, stderr: [2023-11-24T09:17:25Z INFO  flathub_hooks::review] Refs present in build: ["appstream2/aarch64", "runtime/com.thebrokenrail.MCPIReborn.Debug/x86_64/stable", "app/com.thebrokenrail.MCPIReborn/x86_64/stable", "appstream/aarch64", "runtime/com.thebrokenrail.MCPIReborn.Sources/x86_64/stable", "screenshots/x86_64", "appstream2/x86_64", "app/com.thebrokenrail.MCPIReborn/aarch64/stable", "appstream/x86_64", "runtime/com.thebrokenrail.MCPIReborn.Debug/aarch64/stable", "screenshots/aarch64"]\n[2023-11-24T09:17:28Z INFO  flathub_hooks::review] Submitting appdata for review: ReviewRequest { build_id: 65964, job_id: 106271 }\n[2023-11-24T09:17:29Z INFO  flathub_hooks::utils] HTTP status client error (413 Payload Too Large) for url (https://hub.flathub.org/api/v1/job/106271/check/review)\n[2023-11-24T09:17:29Z INFO  flathub_hooks::utils] Retrying (1/5) in 1 seconds...\n[2023-11-24T09:17:30Z INFO  flathub_hooks::utils] HTTP status client error (413 Payload Too Large) for url (https://hub.flathub.org/api/v1/job/106271/check/review)\n[2023-11-24T09:17:30Z INFO  flathub_hooks::utils] Retrying (2/5) in 2 seconds...\n[2023-11-24T09:17:32Z INFO  flathub_hooks::utils] HTTP status client error (413 Payload Too Large) for url (https://hub.flathub.org/api/v1/job/106271/check/review)\n[2023-11-24T09:17:32Z INFO  flathub_hooks::utils] Retrying (3/5) in 4 seconds...\n[2023-11-24T09:17:36Z INFO  flathub_hooks::utils] HTTP status server error (502 Bad Gateway) for url (https://hub.flathub.org/api/v1/job/106271/check/review)\n[2023-11-24T09:17:36Z INFO  flathub_hooks::utils] Retrying (4/5) in 8 seconds...\n[2023-11-24T09:17:44Z INFO  flathub_hooks::utils] HTTP status client error (413 Payload Too Large) for url (https://hub.flathub.org/api/v1/job/106271/check/review)\n[2023-11-24T09:17:44Z INFO  flathub_hooks::utils] Retrying (5/5) in 16 seconds...\n[2023-11-24T09:18:00Z INFO  flathub_hooks::utils] HTTP status client error (413 Payload Too Large) for url (https://hub.flathub.org/api/v1/job/106271/check/review)\nError: HTTP status client error (413 Payload Too Large) for url (https://hub.flathub.org/api/v1/job/106271/check/review)\n\nStack backtrace:\n   0: flathub_hooks::utils::retry\n   1: flathub_hooks::config::Config::mark_still_pending\n   2: flathub_hooks::review::do_review\n   3: flathub_hooks::main\n   4: std::sys_common::backtrace::__rust_begin_short_backtrace\n   5: std::rt::lang_start::{{closure}}\n   6: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once\n             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/ops/function.rs:284:13\n      std::panicking::try::do_call\n             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:502:40\n      std::panicking::try\n             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:466:19\n      std::panic::catch_unwind\n             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panic.rs:142:14\n      std::rt::lang_start_internal::{{closure}}\n             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/rt.rs:148:48\n      std::panicking::try::do_call\n             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:502:40\n      std::panicking::try\n             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:466:19\n      std::panic::catch_unwind\n             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panic.rs:142:14\n      std::rt::lang_start_internal\n             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/rt.rs:148:20\n   7: main\n   8: __libc_start_call_main\n             at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16\n   9: __libc_start_main_impl\n             at ./csu/../csu/libc-start.c:392:3\n  10: _start\n', 'results': None}
program finished with exit code 1
elapsedTime=53.075667

Fail the publish hook if there is more than one appstream file

Cleanup metadata code

The metadata element doesn't exist anymore and shouldn't be used, we're using custom now instead

#101 (comment)

Builds stuck in the Validating state

We're reliably hitting this block of code: https://github.com/flatpak/flat-manager/blob/b07fe8652357ce16d9625f72f9acceaca94d860a/src/jobs/check_job.rs#L200

I ended up commenting it out, which is probably not the proper fix.

flat-manager fails to encode nested JSON

I think that since the hooks were changed to use flatpak-builder-lint for doing the build validation, /build/:id/check/:check/job endpoints became completely broken.

~/d/f/flat-manager master• [1] https -x -A bearer -a $flattoken hub.flathub.org/api/v1/build/57602/check/flathub-hooks/job Content-Type:application/json
HTTP/1.1 400 Bad Request
Connection: keep-alive
Content-Length: 68
Content-Type: text/plain; charset=utf-8
Date: Sun, 22 Oct 2023 21:05:19 GMT
Server: nginx/1.18.0 (Ubuntu)

Json deserialize error: EOF while parsing a value at line 1 column 0

f-b-l returns errors, if any, as JSON. I suspect the nested JSON confuses flat-manager, as it somehow works for /extended, but is double escaped:

~/d/f/flat-manager master• ❱ https -x -A bearer -a $flattoken hub.flathub.org/api/v1/build/57602/extended Content-Type:application/json
HTTP/1.1 200 OK
Connection: keep-alive
Content-Encoding: gzip
Content-Type: application/json
Date: Sun, 22 Oct 2023 21:06:50 GMT
Server: nginx/1.18.0 (Ubuntu)
Transfer-Encoding: chunked
Vary: Accept-Encoding, Accept-Encoding

{
    "build": {
        "app_id": null,
        "build_log_url": "https://buildbot.flathub.org/#/builders/6/builds/74969",
        "commit_job_id": 91268,
        "created": "2023-10-19T20:50:14.517054",
        "extra_ids": [],
        "id": 57602,
        "public_download": true,
        "publish_job_id": 91380,
        "published_state": 2,
        "repo": "stable",
        "repo_state": 5
    },
    "build_refs": [
        {
            "build_id": 57602,
            "build_log_url": null,
            "commit": "3cd7156a5638dda05157d7de5c4ebe749790a2bfb47089d195fa60f3f0017e61",
            "id": 377192,
            "ref_name": "screenshots/x86_64"
        },
        {
            "build_id": 57602,
            "build_log_url": null,
            "commit": "17f3f94f7747234dc49bc4138090627af2ef500c4c6cdc35fd66202df60211c2",
            "id": 377193,
            "ref_name": "app/com.github.micahflee.torbrowser-launcher/x86_64/stable"
        },
        {
            "build_id": 57602,
            "build_log_url": null,
            "commit": "384d195f4048cbd40248fe196a0fed2b37c9267a3515e018521af6d3f5f1fc44",
            "id": 377194,
            "ref_name": "runtime/com.github.micahflee.torbrowser_launcher.Debug/x86_64/stable"
        },
        {
            "build_id": 57602,
            "build_log_url": null,
            "commit": "5445147bad21846b4a357cdc1efb9d7a1efa2126051a99cf44ca9bc1de40439d",
            "id": 377195,
            "ref_name": "runtime/com.github.micahflee.torbrowser_launcher.Sources/x86_64/stable"
        }
    ],
    "checks": [
        {
            "build_id": 57602,
            "check_name": "flathub-hooks",
            "job_id": 91269,
            "results": "{\"diagnostics\":[{\"refstring\":\"app/com.github.micahflee.torbrowser-launcher/x86_64/stable\",\"is_warning\":false,\"category\":\"flatpak_builder_lint\",\"data\":{\"stdout\":{\"errors\":[\"appstream-missing-developer-name\",\"appstream-failed-validation\"],\"warnings\":[\"finish-args-contains-both-x11-and-wayland\",\"finish-args-contains-both-x11-and-fallback\"]},\"stderr\":\"\"}}]}",
            "status": 2,
            "status_reason": "One or more validations failed."
        }
    ]
}

502 status code on /job/92524/check/review

Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:     Error: HTTP status server error (502 Bad Gateway) for url (https://hub.flathub.org/api/v1/job/92524/check/review)
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:     
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:     Stack backtrace:
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:        0: flathub_hooks::utils::retry
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:        1: flathub_hooks::config::Config::mark_still_pending
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:        2: flathub_hooks::review::do_review
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:        3: flathub_hooks::cmd_review::ReviewArgs::run
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:        4: flathub_hooks::main
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:        5: std::sys_common::backtrace::__rust_begin_short_backtrace
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:        6: std::rt::lang_start::{{closure}}
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:        7: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:                  at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/ops/function.rs:284:13
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:           std::panicking::try::do_call
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:                  at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:502:40
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:           std::panicking::try
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:                  at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:466:19
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:           std::panic::catch_unwind
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:                  at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panic.rs:142:14
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:           std::rt::lang_start_internal::{{closure}}
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:                  at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/rt.rs:148:48
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:           std::panicking::try::do_call
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:                  at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:502:40
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:           std::panicking::try
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:                  at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:466:19
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:           std::panic::catch_unwind
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:                  at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panic.rs:142:14
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:           std::rt::lang_start_internal
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:                  at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/rt.rs:148:20
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:        8: main
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:        9: __libc_start_call_main
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:                  at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:       10: __libc_start_main_impl
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:                  at ./csu/../csu/libc-start.c:392:3
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]:       11: _start
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]: [2023-10-22T10:08:55Z INFO  flatmanager::jobs::check_job] #92524: Check status updated to Failed("Check command cd \"/tank/repo/builds/58237\" && FLAT_MANAGER_BUILD_ID=\"58237\" FLAT>
Oct 22 10:08:55 master.flathub.org flat-manager[3832269]: [2023-10-22T10:08:55Z INFO  flatmanager::jobs::job_executor] #92524: Job succeeded

This happened as part of the build 58237, blocking it from being published. Not sure what else to include here to make it more useful.

Do not submit test builds to moderation

Somewhat related to flathub/flathub#4630

Assuming the used branches are correct, we should be only moderating changes to the stable builds.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.