Giter Club home page Giter Club logo

taskcluster's Introduction


Taskcluster
Taskcluster

Inspecting a task on Taskcluster UI

The task execution framework that supports Mozilla's continuous integration and release processes.

Taskcluster Status License Chat netlify pre-commit


Usage

This repository is used to develop, build, and release the Taskcluster services.

Table of Contents

Team Mentions

Do you need to reach a specific subset of the team? Use the team handles to mention us with GitHub's @mention feature.

Team Name Use To...
@taskcluster/Core ping members of the Taskcluster team at Mozilla
@taskcluster/services-reviewers ping reviewers for changes to platform services and libraries
@taskcluster/frontend-reviewers ping people who can review changes to frontend (and related) code in the services monorepo
@taskcluster/security-folks ping people who do security things

Contributors

Thanks goes to these wonderful people (emoji key):

James Lal
James Lal

πŸ’» πŸ‘‹
Selena Deckelmann
Selena Deckelmann

πŸ’» πŸ‘‹
Dustin J. Mitchell
Dustin J. Mitchell

πŸ’» πŸ‘‹
Wander Lairson Costa
Wander Lairson Costa

πŸ’» πŸ‘‹
Greg Arndt
Greg Arndt

πŸ’» πŸ‘‹
Pete Moore
Pete Moore

πŸ’» πŸ”§
Hassan Ali
Hassan Ali

πŸ’» πŸ‘‹
John Whitlock
John Whitlock

πŸ’» πŸ‘‹
Brian Stack
Brian Stack

πŸ’» πŸ‘‹
John Ford
John Ford

πŸ’» πŸ‘‹
Eli Perelman
Eli Perelman

πŸ’» πŸ‘‹
Jonas Finnemann Jensen
Jonas Finnemann Jensen

πŸ’» πŸ‘‹
owlishDeveloper
owlishDeveloper

πŸ’» πŸ‘‹
Miles Crabill
Miles Crabill

πŸ’» πŸ‘‹
Chris Cooper
Chris Cooper

πŸ’» πŸ‘‹
Mathieu Leplatre
Mathieu Leplatre

πŸ’» πŸ‘‹
Rob Thijssen
Rob Thijssen

πŸ’»
Anup
Anup

πŸ’»
Hammad Akhtar
Hammad Akhtar

πŸ’»
Chinmay Kousik
Chinmay Kousik

πŸ’»
Anthony Miyaguchi
Anthony Miyaguchi

πŸ’»
Ana Rute Mendes
Ana Rute Mendes

πŸ’»
Andrea Del Rio
Andrea Del Rio

πŸ’»
kristelteng
kristelteng

πŸ’»
Elena Solomon
Elena Solomon

πŸ’»
Xavier L.
Xavier L.

πŸ’»
Yann Landry
Yann Landry

πŸ’»
Ayub
Ayub

πŸ’»
lteigrob
lteigrob

πŸ’»
Bastien Abadie
Bastien Abadie

πŸ’»
Amjad Mashaal
Amjad Mashaal

πŸ’»
Tom Prince
Tom Prince

πŸ’»
Samantha Yu
Samantha Yu

πŸ’»
Auni Ahsan
Auni Ahsan

πŸ’»
alex
alex

πŸ’»
Alisha Aneja
Alisha Aneja

πŸ’»
Prachi Manchanda
Prachi Manchanda

πŸ’»
Simon Fraser
Simon Fraser

πŸ’»
Yashvardhan Didwania
Yashvardhan Didwania

πŸ’»
Cynthia Pereira
Cynthia Pereira

πŸ’»
Hashini Galappaththi
Hashini Galappaththi

πŸ’»
Fienny Angelina
Fienny Angelina

πŸ’»
Kanika Saini
Kanika Saini

πŸ’»
Biboswan Roy
Biboswan Roy

πŸ’»
sudipt dabral
sudipt dabral

πŸ’»
Ojaswin
Ojaswin

πŸ’»
ΠœΠ°Ρ‚Ρ€Π΅ΡˆΠΊΠ°
ΠœΠ°Ρ‚Ρ€Π΅ΡˆΠΊΠ°

πŸ’»
Alok Kumar
Alok Kumar

πŸ’»
Arshad Kazmi
Arshad Kazmi

πŸ’»
Jason Yang
Jason Yang

πŸ’»
Shubham Gupta
Shubham Gupta

πŸ’»
Arun Kumar Mohan
Arun Kumar Mohan

πŸ’»
Brian Pitts
Brian Pitts

πŸ’»
E. Dunham
E. Dunham

πŸ’»
Shubham Chinda
Shubham Chinda

πŸ’»
Patrick Kang
Patrick Kang

πŸ’»
Rishabh Budhiraja
Rishabh Budhiraja

πŸ’»
ededals
ededals

πŸ’»
Ajin Kabeer
Ajin Kabeer

πŸ’»
Catherine Chepkurui
Catherine Chepkurui

πŸ’»
Jo
Jo

πŸ’»
vishakha
vishakha

πŸ’» πŸ“–
Noor Fatima
Noor Fatima

πŸ’»
Michael
Michael

πŸ’»
Mariana Zangrossi
Mariana Zangrossi

πŸ’»
ANURADHAJHA99
ANURADHAJHA99

πŸ’»
Edil
Edil

πŸ’»
Olympia
Olympia

πŸ’» πŸ“–
Michael Ozoemena
Michael Ozoemena

πŸ’»
lailahgrant
lailahgrant

πŸ’»
km-js
km-js

πŸ’»
Carolina Machado
Carolina Machado

πŸ’»
reenesa
reenesa

πŸ’»
Kelli Blalock
Kelli Blalock

πŸ’»
naima shaikh
naima shaikh

πŸ’»
Jiwoon Kim
Jiwoon Kim

πŸ’»
Michael Umanah
Michael Umanah

πŸ’»
Fahd Jamal A.
Fahd Jamal A.

πŸ“–
shilpi verma
shilpi verma

πŸ’»
somchi
somchi

πŸ’»
Anastasia
Anastasia

πŸ’»
Lubna
Lubna

πŸ’»
Soundharya AM
Soundharya AM

πŸ’»
Mustafa Jebara
Mustafa Jebara

πŸ’»
Aryaman Puri
Aryaman Puri

πŸ’»
Simon Sapin
Simon Sapin

πŸ’»
thoran
thoran

πŸ’»
Manish Giri
Manish Giri

πŸ’»
Tiger Oakes
Tiger Oakes

πŸ’»
Ricky Taylor
Ricky Taylor

πŸ’»
Alex Lopez
Alex Lopez

πŸ’»
Michelle
Michelle

πŸ› πŸš‡
Mrs. Velena
Mrs. Velena

πŸ’»
Ahmed A.
Ahmed A.

πŸ’»
Matt Boris
Matt Boris

πŸ’» πŸ”§
Yaraslau Kurmyza
Yaraslau Kurmyza

πŸ’» πŸ”§
Bastien Orivel
Bastien Orivel

πŸ’»
HamdiAmine
HamdiAmine

πŸ’»

This project follows the all-contributors specification. Contributions of any kind are welcome!

taskcluster's People

Contributors

acmiyaguchi avatar allcontributors[bot] avatar alopezz avatar arshadkazmi42 avatar ccooper avatar ckousik avatar dependabot[bot] avatar djmitche avatar edil-it-them avatar eijebong avatar eliperelman avatar escapewindow avatar gregarndt avatar helfi92 avatar imbstack avatar jhford avatar jonasfj avatar jwhitlock avatar lightsofapollo avatar lotas avatar matt-boris avatar moosej avatar owlishdeveloper avatar petemoore avatar renovate-bot avatar renovate[bot] avatar ricky26 avatar t0xiccode avatar taskcluster-bot avatar walac avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

taskcluster's Issues

[UI] always allow collapsing the drawer

Could you add an action to collapse the drawer or leave the accordion button when the screen is wide (then I can choose to collapse it)?

As a user, I would like to hide the navigation when I am looking at content. This would provide more space to view content and prevent horizontal scrolling.

Here is a screenshot showing the limited content space (half of the table is displayed, and I must scroll horizontally to see the other half):
image

New UI feels slower

Loading a large task group feels slower than in the old UI, despite being actually faster, because it only displays the list once it has everything while the old UI would start displaying things before everything is downloaded.

Displaying one task, though, is slower.

Triggering builds through tagging

I'd like to trigger TaskCluster to run in response to pushing a tag to my repository on GitHub. The GitHub API docs on the "push" event read, "Any Git push to a Repository, including editing tags or branches." This leads me to believe I can react to tags in my .taskcluster.yml file with something like

$if: 'event.ref == "refs/heads/master" || event.ref[:13] == "refs/tags/tc-"'

I can verify that the webhook event was sent from GitHub, but I don't know how to determine if this triggered any work on TaskCluster. Under these circumstances, I would expect that the tagged commit would be updated with a "build status" indicator icon in the repository's history listing.

Is this expected to trigger activity on TaskCluster? If so, is there some other way to view the tasks? Thanks!

Residual images need to be deleted in the case of failure

I noticed that if the build process fails (in particular, it seems to be a concern for a process where we install things on an image) the residual image is left on the disk. It lacks tags, and looks like just a copy of a base image. I think these are best be deleted in case of failure

(Example - building tools service. If the installation of nginx on a node image fails, we are left with an untagged copy of the original node image)

[UI] Hooks: Disable "save" button unless some values have changed

Summary:
When I go back to an old hook tab that I had open, I see that the "save" button isn't disabled, which makes me wonder: do I have unsaved changes? Did I change the hook before I left the tab some hours ago? If we disabled the hook unless there's unsaved changes, it could improve usability.

To reproduce:

  1. Go to a hook

[cllient-shell] Errors when cancelling large graphs: `dial tcp: lookup queue.taskcluster.net: too many open files`

Steps to reproduce

  1. Land a commit in-tree.
  2. Realize it needs a follow up, and land it.
  3. ./taskcluster group cancel GROUP_ID_OF_WRONG_COMMIT

Results

More than half of task aren't cancelled. tc-cli spews hundreds of lines like:

2017/06/12 17:26:20 Error: Post https://queue.taskcluster.net/v1/task/M69eLACyQCSMA6z_SELITA/cancel: dial tcp: lookup queue.taskcluster.net: too many open files

I guess this is a limitation on the OS-side, where you can't have more than a certain number of ports open at the same time. I don't know the internals of the client too much, though.

Entity too large can cause us to silently error

getsentry/raven-node#153 affects us. I just saw it in tc-gh


Feb 16 13:33:55 taskcluster-github app/web.1:   * data.builds[997] should have required property 'eventId' 
Feb 16 13:33:55 taskcluster-github app/web.1:     at ServerResponse.res.reply (/app/node_modules/taskcluster-lib-api/lib/api.js:144:21) 
Feb 16 13:33:55 taskcluster-github app/web.1:     at Object._callee2$ (/app/lib/api.js:368:51) 
Feb 16 13:33:55 taskcluster-github app/web.1:     at tryCatch (/app/node_modules/regenerator-runtime/runtime.js:62:40) 
Feb 16 13:33:55 taskcluster-github app/web.1:     at GeneratorFunctionPrototype.invoke [as _invoke] (/app/node_modules/regenerator-runtime/runtime.js:336:22) 
Feb 16 13:33:55 taskcluster-github app/web.1:     at GeneratorFunctionPrototype.prototype.(anonymous function) [as next] (/app/node_modules/regenerator-runtime/runtime.js:95:21) 
Feb 16 13:33:55 taskcluster-github app/web.1:     at step (/app/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30) 
Feb 16 13:33:55 taskcluster-github app/web.1:     at /app/node_modules/babel-runtime/helpers/asyncToGenerator.js:28:20 
Feb 16 13:33:55 taskcluster-github app/web.1:     at process._tickDomainCallback (internal/process/next_tick.js:129:7) 
Feb 16 13:33:55 taskcluster-github app/web.1: Failed to log error to Sentry: Error: HTTP Error (413): undefined 

The exception was indeed quite a long list of validation errors. We should probably handle this case intelligently.

Allow clearer lists of commands in a payload

Due to payload->command being passed directly to taskcluster, our .taskcluster.yml files are ending up of the form:

   command:
     - "/bin/bash"
     - "-c"
     - "nix-env -iA nixpkgs.gnumake nixpkgs.curl nixpkgs.cacert && export SSL_CERT_FILE=$HOME/.nix-profile/etc/ssl/certs/ca-bundle.crt && mkdir /src && cd /src && curl -L https://github.com/mozilla-releng/services/archive/$GITHUB_HEAD_SHA.tar.gz -o $GITHUB_HEAD_SHA.tar.gz && tar zxf $GITHUB_HEAD_SHA.tar.gz && cd services-$GITHUB_HEAD_SHA && ./.taskcluster.sh"

See https://github.com/mozilla-releng/services/blob/4925b72d47be422b226a066a59d00818e6d26f3c/.taskcluster.yml#L39 for the first of 40-some examples for that project.

Wrapping this up in a script reduces visibility when checking the .yml for what's to be run, and reduces flexibility for having a slightly different setup for each component.

Other CI service .yml files allow either a before_install/dependencies style check to allow the same set of commands to be run as setup for each task, or the command list is an implicit &&. For example, in .gitlab-ci.yml the following:

script:
    - nix-env -iA nixpkgs.gnumake nixpkgs.curl nixpkgs.cacert
    - export SSL_CERT_FILE=$HOME/.nix-profile/etc/ssl/certs/ca-bundle.crt
    - ./.taskcluster.sh

Would be turned into a set -e bash script, effectively, for execution, with each list entry being a line in the file.

I think we desperately need to avoid the massive && structures, and so would like opinions on which direction to take.

Greenkeeper doesn't work with new tc-github integration

Previously greenkeeper landings would fail to generate taskcluster runs for a branch, but generate taskcluster runs for PRs (this may have been a bug).

Now it won't generate taskcluster runs at all.

It says to add greenkeeper[bot] as a collaborator, however, greenkeeper is an integration and doesn't have a bot account.

Can we allow integrations as committers?

For example, see mozilla/example-addon-repo#59

[taskcluster-web]: Show "recently-created" task definitions

As I rotate between tickets I'm working on, I need to re-create some very similar builds. For example, last week, I created some tasks that would trigger a build on my custom worker_type. However, it's been a couple work days, and I've changed contexts. I can't quite remember exactly how my task definition looked, so I need to re-create it. It would potentially be neat if I could use recently-created task definitions.

Note: I'm not sure how you could handle de-duping: I wouldn't want to see every task definition I've created, because they'll all be vaguely different with different timestamps. For me, I've probably spun up over 50 tasks in the last week, but I've only done maybe 3 "types" of tasks.

Allow setting the GitHub status context in .taskcluster.yml

For wpt we have a situation where on some pushes we run multiple TaskCluster taskgroups with different settings (this is controlled by pushes to different branches). Unfortunately the way that the status API works in GitHub it will hide all but the latest run with a specific context name, making it hard fo figure out where to find all the results. This could be solved by giving the ability to override the context name for a specific set of tasks from the .taskcluster.yml file.

The main issue is presumably that this wants to be per repository, not per task, so one ends up duplicating some of the task logic to set the right context for each event.

[client-shell] Create a `taskcluster trigger-hook` command

$ taskcluster trigger-hook <hookGroupId>/<hookId>
KGt8egfvRaqxczIRgOScaw

What makes this interesting is, I would like an easy way to add parameters to the payload. For example:

$ taskcluster trigger-hook project-releng/cron-task-integration-mozilla-inbound \
    --parameter force-cron-job=nightly \
    --parameter head-rev=ea3272999e36

Currently these parameters are ignored on the server, but that's OK -- we'll get that fixed separately.

[taskcluster-web] Missing view titles

STR:

  1. Navigate to http://localhost:5080/tasks/groups/cbxWJaKyTKagmTMjEb17FQ

Actual: There is no view title.
Expected: "Task Group" shows in the app bar.

screen shot 2019-02-11 at 9 53 42 am

  1. Navigate to http://localhost:5080/tasks/groups

Actual: There is no view title.
Expected: "Task Groups" show in the app bar.

  1. Navigate to http://localhost:5080/tasks/cbxWJaKyTKagmTMjEb17FQ

Actual: There is no view title.
Expected: "Task" shows in the app bar.

  1. Navigate to http://localhost:5080/tasks

Actual: There is no view title.
Expected: "View Tasks" shows in the app bar.


In order to add a title to the app bar, one has to provide the title prop to the Dashboard component of the view.

[UI] Allow opening hooks in new tab

Summary:
I want to view both the staging and nightly hooks for Fenix. However, when I ctrl-click, it doesn't open the link in a new tab - instead, it brings my current tab to the hook's page, and making me have to re-open and re-search for Fenix manually to open the other hook.

I would love to be able to

  • Right click, open in new tab
  • ctrl-click to open in new tab
    on items in the hooks list

To reproduce:

  1. Go to hooks
  2. Search for fenix
  3. ctrl-click on fenix-nightly to (try to) open the link in a new tab

[UI] Loading circle is momentarily not centred

Describe the bug
When the page is loading, the loading circle (or spinner) is not centred until the side and top bar appear.

To Reproduce
Steps to reproduce the behavior:

  1. Access a task, such as this

Expected behavior
Either:

  • loading circle first appear in the dead centre of the browser window, then relocate when the side and top bar loads.
    or
  • side and top bar are prioritized to be drawn on screen first, thereby not requiring the loading circle to relocate.

Screenshots
Attached.
screen shot 2019-02-14 at 11 47 14
screen shot 2019-02-14 at 11 47 11

Platform (please complete the following information):

  • OS: macOS
  • Browser: Firefox Nightly
  • Version: 67.0a1

Additional context
I understand this is a very superficial issue, though I feel it affects the first time perception of users.

Release events don't work?

I started trying to make use of release events today, and as far as I can tell, events are never fired for them. For example, I was watching the "exchange/taskcluster-github/v1/release" exchange with the Pulse Inspector while I created both tags and full fledged releases on https://github.com/testbhearsum/balrog/releases, but no events showed up. I also have two Tasks that should be created when a Release event is fired that have routes attached to them, and neither one of them showed up in the Index Browser.

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Location: package.json
Error type: The renovate configuration file contains some invalid settings
Message: Configuration option 'packageRules[0].node' should be a json object, Invalid configuration option: engine-strict, Invalid configuration option: eslintIgnore, Invalid configuration option: heroku-run-build-script, Invalid configuration option: metatests, Invalid configuration option: name, Invalid configuration option: packageRules[0].yarn, Invalid configuration option: packageRules[1].@azure/arm-compute, Invalid configuration option: packageRules[1].@azure/arm-network, Invalid configuration option: packageRules[1].@azure/ms-rest-azure-js, Invalid configuration option: packageRules[1].@azure/ms-rest-js, Invalid configuration option: packageRules[1].@azure/ms-rest-nodeauth, Invalid configuration option: packageRules[1].@octokit/auth-app, Invalid configuration option: packageRules[1].@octokit/core, Invalid configuration option: packageRules[1].@octokit/plugin-retry, Invalid configuration option: packageRules[1].@octokit/rest, Invalid configuration option: packageRules[1].@sentry/node, Invalid configuration option: packageRules[1].@slack/web-api, Invalid configuration option: packageRules[1].ajv, Invalid configuration option: packageRules[1].ajv-formats, Invalid configuration option: packageRules[1].amqplib, Invalid configuration option: packageRules[1].apollo-server-express, Invalid configuration option: packageRules[1].aws-sdk, Invalid configuration option: packageRules[1].body-parser, Invalid configuration option: packageRules[1].body-parser-graphql, Invalid configuration option: packageRules[1].chalk, Invalid configuration option: packageRules[1].compression, Invalid configuration option: packageRules[1].content-security-policy, Invalid configuration option: packageRules[1].content-type, Invalid configuration option: packageRules[1].cors, Invalid configuration option: packageRules[1].cron-parser, Invalid configuration option: packageRules[1].dataloader, Invalid configuration option: packageRules[1].debug, Invalid configuration option: packageRules[1].deepmerge, Invalid configuration option: packageRules[1].ejs, Invalid configuration option: packageRules[1].email-templates, Invalid configuration option: packageRules[1].express, Invalid configuration option: packageRules[1].express-session, Invalid configuration option: packageRules[1].express-sslify, Invalid configuration option: packageRules[1].fast-azure-storage, Invalid configuration option: packageRules[1].fast-json-stable-stringify, Invalid configuration option: packageRules[1].generate-password, Invalid configuration option: packageRules[1].get-caller-file, Invalid configuration option: packageRules[1].googleapis, Invalid configuration option: packageRules[1].got, Invalid configuration option: packageRules[1].graphql, Invalid configuration option: packageRules[1].graphql-depth-limit, Invalid configuration option: packageRules[1].graphql-playground-middleware-express, Invalid configuration option: packageRules[1].graphql-scalars, Invalid configuration option: packageRules[1].graphql-type-json, Invalid configuration option: packageRules[1].graphql-validation-complexity, Invalid configuration option: packageRules[1].hashids, Invalid configuration option: packageRules[1].hawk, Invalid configuration option: packageRules[1].hsts, Invalid configuration option: packageRules[1].immutable, Invalid configuration option: packageRules[1].iterall, Invalid configuration option: packageRules[1].js-yaml, Invalid configuration option: packageRules[1].json-e, Invalid configuration option: packageRules[1].json-parameterization, Invalid configuration option: packageRules[1].jsonwebtoken, Invalid configuration option: packageRules[1].jwks-rsa, Invalid configuration option: packageRules[1].lodash, Invalid configuration option: packageRules[1].loglevel, Invalid configuration option: packageRules[1].marked, Invalid configuration option: packageRules[1].matrix-js-sdk, Invalid configuration option: packageRules[1].memorystore, Invalid configuration option: packageRules[1].mkdirp, Invalid configuration option: packageRules[1].netmask, Invalid configuration option: packageRules[1].newrelic, Invalid configuration option: packageRules[1].node-fetch, Invalid configuration option: packageRules[1].node-forge, Invalid configuration option: packageRules[1].nodemailer, Invalid configuration option: packageRules[1].oauth2orize, Invalid configuration option: packageRules[1].p-queue, Invalid configuration option: packageRules[1].p-synchronize, Invalid configuration option: packageRules[1].parseurl, Invalid configuration option: packageRules[1].passport, Invalid configuration option: packageRules[1].passport-auth0, Invalid configuration option: packageRules[1].passport-github, Invalid configuration option: packageRules[1].pg, Invalid configuration option: packageRules[1].pg-connection-string, Invalid configuration option: packageRules[1].qs, Invalid configuration option: packageRules[1].quick-lru, Invalid configuration option: packageRules[1].regex-escape, Invalid configuration option: packageRules[1].request-ip, Invalid configuration option: packageRules[1].rimraf, Invalid configuration option: packageRules[1].sanitize-html, Invalid configuration option: packageRules[1].sentry-api, Invalid configuration option: packageRules[1].sift, Invalid configuration option: packageRules[1].slugid, Invalid configuration option: packageRules[1].subscriptions-transport-ws, Invalid configuration option: packageRules[1].superagent, Invalid configuration option: packageRules[1].taskcluster-client, Invalid configuration option: packageRules[1].taskcluster-db, Invalid configuration option: packageRules[1].taskcluster-lib-api, Invalid configuration option: packageRules[1].taskcluster-lib-app, Invalid configuration option: packageRules[1].taskcluster-lib-azqueue, Invalid configuration option: packageRules[1].taskcluster-lib-config, Invalid configuration option: packageRules[1].taskcluster-lib-iterate, Invalid configuration option: packageRules[1].taskcluster-lib-loader, Invalid configuration option: packageRules[1].taskcluster-lib-monitor, Invalid configuration option: packageRules[1].taskcluster-lib-postgres, Invalid configuration option: packageRules[1].taskcluster-lib-pulse, Invalid configuration option: packageRules[1].taskcluster-lib-references, Invalid configuration option: packageRules[1].taskcluster-lib-scopes, Invalid configuration option: packageRules[1].taskcluster-lib-urls, Invalid configuration option: packageRules[1].taskcluster-lib-validate, Invalid configuration option: packageRules[1].thirty-two, Invalid configuration option: packageRules[1].topo-sort, Invalid configuration option: packageRules[1].type-is, Invalid configuration option: packageRules[1].uuid, Invalid configuration option: packageRules[1].walk, Invalid configuration option: packageRules[2].acorn-loose, Invalid configuration option: packageRules[2].acorn-walk, Invalid configuration option: packageRules[2].apollo-cache-inmemory, Invalid configuration option: packageRules[2].apollo-client, Invalid configuration option: packageRules[2].apollo-link-http, Invalid configuration option: packageRules[2].apollo-link-ws, Invalid configuration option: packageRules[2].app-root-dir, Invalid configuration option: packageRules[2].assume, Invalid configuration option: packageRules[2].builtin-modules, Invalid configuration option: packageRules[2].c8, Invalid configuration option: packageRules[2].commander, Invalid configuration option: packageRules[2].console-taskgraph, Invalid configuration option: packageRules[2].cronstrue, Invalid configuration option: packageRules[2].cross-env, Invalid configuration option: packageRules[2].dockerode, Invalid configuration option: packageRules[2].error-stack-parser, Invalid configuration option: packageRules[2].eslint, Invalid configuration option: packageRules[2].github-slugger, Invalid configuration option: packageRules[2].glob, Invalid configuration option: packageRules[2].graphql-tag, Invalid configuration option: packageRules[2].inquirer, Invalid configuration option: packageRules[2].is-uuid, Invalid configuration option: packageRules[2].json-stable-stringify, Invalid configuration option: packageRules[2].markdown-table, Invalid configuration option: packageRules[2].md-directory, Invalid configuration option: packageRules[2].mocha, Invalid configuration option: packageRules[2].mock-aws-s3, Invalid configuration option: packageRules[2].mock-fs, Invalid configuration option: packageRules[2].mockdate, Invalid configuration option: packageRules[2].moment, Invalid configuration option: packageRules[2].mz, Invalid configuration option: packageRules[2].nock, Invalid configuration option: packageRules[2].open-editor, Invalid configuration option: packageRules[2].qlobber, Invalid configuration option: packageRules[2].semver, Invalid configuration option: packageRules[2].sinon, Invalid configuration option: packageRules[2].snake-case, Invalid configuration option: packageRules[2].taskcluster-lib-testing, Invalid configuration option: packageRules[2].temporary, Invalid configuration option: packageRules[2].ws, Invalid configuration option: packageRules[2].yarn-minify, Invalid configuration option: packageRules[2].zen-observable, Invalid configuration option: packageRules[2].zurvan, Invalid configuration option: private, Invalid configuration option: renovate, Invalid configuration option: resolutions, Invalid configuration option: scripts, Invalid configuration option: version, Invalid configuration option: workspaces, The "node" object can only be configured at the top level of a config but was found inside "packageRules[0]"

[client-shell] Display error.message on failure

$ ./taskcluster api auth awsS3Credentials read-write downloads.taskcluster.net 'taskcluster-cli/*'
Error: Request failed: Non-2xx StatusCode: 400 received in 1 attempts

so I got a 400.. but what's the error message? 😿

The error responses are described here.

[UI] Prefetch link on mouse hover

We should write a component that wraps the Link component of react-router-dom which uses the onMouseOver event to prefetch the link in order to increase perceived performance.

debugging issues with the loader stinks

Example: We have a cycle, we know that, but the way that we're checking for cycles makes it impossible to determine where the cycle is without manually tracing it. We also can't run the graphviz generator target to see where the cycle is because we do the cycle check first.

Two things:

  1. this error message ought to say what the cycle is
  2. we should be able to generate a graphviz dot file that shows the cycle
/Users/jhford/taskcluster/cloud-mirror/node_modules/topo-sort/tsort.js:88
    throw new Error('At least 1 circular dependency in nodes: \n\n' + circular.join('\n') + '\n\nGraph cannot be sorted!');
          ^
Error: At least 1 circular dependency in nodes:

cfg
profile
redis
sqs
monitor
process
validator
api
registeredCacheManagers
server
queueUrl
queue
cacheManagers

Graph cannot be sorted!
    at [object Object].TopoSort.sort (/Users/jhford/taskcluster/cloud-mirror/node_modules/topo-sort/tsort.js:88:11)
    at Object.loader (/Users/jhford/taskcluster/cloud-mirror/node_modules/taskcluster-lib-loader/src/loader.js:161:26)
    at Object.<anonymous> (/Users/jhford/taskcluster/cloud-mirror/src/main.js:61:8)
    at Module._compile (module.js:413:34)
    at Object.Module._extensions..js (module.js:422:10)
    at Module.load (module.js:357:32)
    at Function.Module._load (module.js:314:12)
    at Function.Module.runMain (module.js:447:10)
    at startup (node.js:148:18)
    at node.js:405:3

[taskcluster-lib-scopes] Remove usage of Node built-in 'assert' from published library

Hi!

Treeherder uses taskcluster-lib-scopes in the browser, for the taskcluster actions feature - eg:
https://github.com/mozilla/treeherder/blob/23036375f9faec4c0f343e51024eb7feb2195c45/ui/models/taskcluster.js#L64-L68

In the upcoming webpack 5, webpack no longer automatically polyfills node built-ins in imported libraries in an effort to reduce unintended bundle size bloat.

Trying webpack 5 out against Treeherder I get several errors like:

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need these module and configure a polyfill for it.

If you want to include a polyfill, you need to install 'assert'.
If you don't want to include a polyfill, you can use an empty module like this:
        resolve.alias: { "assert": false }
 @ ./node_modules/taskcluster-lib-scopes/src/index.js 7:2-26
 @ ./ui/models/taskcluster.js 4:0-61 59:11-30
 @ ./ui/models/push.js 11:0-45 94:11-27 97:13-29 110:11-27 113:13-29 157:13-29 160:15-31
 @ ./ui/logviewer/App.jsx 10:0-39 150:8-17
 @ ./ui/logviewer/index.jsx 12:0-24 13:27-30

Whilst we could configure a polyfill, I think it would be best for taskcluster-lib-scopes to stop using assert in non-test code, so it's browser-compatible out of the box.

Many thanks :-)

Builder needs a retry

While working on a project where I had to use builder frequently, I noticed that sometimes the creation of an image fails for some relatively force majeure reasons (like connectivity problems during npm install etc). When restarted with no changes made, the builder is usually able to complete the build. So I thought that it would be great if the builder could restart the failed build on its own, at least a couple of times, so that I don't have to watch it too closely.

Also, it would be great if it could happily skip things after the third failed retry, so that the maximum possible amount of images could be built without human intervention.

[UI] Remove redundant sign-in chooser

Summary:
Right now, there's only one way to Sign in: via credentials. Since, in my workflow, I've already made it explicit that I've wanted to sign in by clicking the button, and I've only got one sign in method, I think that I should go straight to the "Sign in with credentials" form

2019-01-16-144911_460x223_scrot

[client-shell] `taskcluster pulse-listen` command

Read about pulse here:
https://wiki.mozilla.org/Auto-tools/Projects/Pulse

It would be cool to have a command to inspect pulse exchanges...

taskcluster pulse-listen [--format yaml|json] <exchange> <routingKeyPattern>

where 'exchange' is a pulse exchange... Then it'll print messages to terminal as JSON or YAML, like:

exchange: ...
routingKeys:
  - ... # primary routing key
  - ... # all the CC'ed routing keys
message: ...
---
exchange: ...
routingKeys:
  - ... # primary routing key
  - ... # all the CC'ed routing keys
message: ...
---

Or if --format json is specified it could print a JSON blob per line, so that it can be fed into jq.

We can store the username/password in the config system for taskcluster-cli... These should also default to PULSE_USERNAME and PULSE_PASSWORD env vars, if not config values are entered...
Notice: this should primarily be based on the stuff in config/...
Maybe also support options --username or --password to overwrite these...


Finally, we do have exchange meta-data... similar to the API references...
See: https://docs.taskcluster.net/manual/integrations/tools/references
Perhaps we can use those to auto-generate commands that will support listening to pulse exchanges...
I see this as just being a way that people don't have to write the entire exchange name in taskcluster pulse listen <exchange> <routingkey> command...

Maybe, it's like taskcluster api queue-events task-created --task-id ... and then from the reference doc for queueEvents it knows that task-created -> taskCreated and taskId -> task-id is a component of the default routingkey. Hence, it can construct the <exchange> <routingkey> pair and listen...
This might be a later addition... Initial stuff is to just get

[UI] Documentation redirects

The legacy documentation site maintains a list of redirects from old pages to new pages. This list will need to be updated to any changes that may happen from #54, and also may need to be captured in react-router's redirects.

test websocktunnel client with disappearing requests...

Let's try to stream a response out (like a livelog) using whclient, while a livelog viewer downloads half the request before aborting the request in various dirty ways.

I fear I've seen requests getting stuck, we should test for this.

[taskcluster-web] Can't trigger hook

Describe the bug
The menu that shows all the hook actions doesn't pop up

To Reproduce

  1. Go to a hook
  2. Mouse over the bottom right menu
  3. Option to trigger hook doesn't appear

Expected behavior
I should be able to click a button to trigger a hook

Screenshots
out

Platform (please complete the following information):

  • OS: Ubuntu 18.04
  • Firefox 64

[client-shell] Add -v / --verbose

This option should interface with the standard log library. If the option is not given, the logs should be directed to a null output. If the option is given, direct that output to stderr.

Allow "approval" of a push by collaborators

From taskcluster/taskcluster-rfcs#95

The idea would be that someone on a whitelist (configurable in the defeault-branch .taskcluster.yml) can write something specific in a comment like @taskcluster: please run and the push would be re-run as if it had been allowed to begin with.

The downside is that this requires either caching the Github PR webhook payload, or re-constructing it from the comment webhook payload. With JSON-e, we will be exposing payloads to the users directly, so re-constructing is not a great option as it would be constantly out of date.

Make an example with a decision task

Notable problem is:
https://github.com/taskcluster/taskcluster-github/blob/master/src/intree.js#L16-L29

when the decision task from .taskcluster.yml is created, it's not able to declare a role in:
task.scopes such that it can actually use all the scopes it has...

Or we need an example of how to do this...
Maybe it's a parameter substitution...


In other words tc-gh is setting authorizedScopes, so the task created can use those scopes... but only if it declares them in task.scopes, which is hard to do with the limited template support.

[UI] Hooks: behaviour when group (but not items) matches search isn't ideal

Summary:
Searching for all hooks in a group doesn't quite work, since the search that matches the hook's name probably doesn't match the items, too

To reproduce:

  1. Go to hooks
  2. Search for "project-gecko"
  3. Expand the "project-gecko" group
  • There's no hooks in "project-gecko" πŸ€”

Preferred behaviour:
We could show all items in a group if the group name matches? πŸ€”

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.