The example provided in https://docs.taskcluster.net/docs/reference/integrations/taskcluster-github/docs/taskcluster-yml-v1 doesn't list the required field created. See mozilla-mobile/fenix#64 (comment)

I found a snippet of older code from the templating engine that was used for specifying after how long artifacts would expire:

artifacts:
  'public':
    type: 'directory'
    path: '/home/worker/artifacts/public'
    expires: '{{#from_now }}7 days{{/from_now}}'

however, this fails with:

Rejecting Schema: http://schemas.taskcluster.net/docker-worker/v1/payload.json#\nErrors:\n * data.artifacts['public'].expires should match format \"date-time\""

Apparently some sort of similar functionality needs to be added to the github parser to allow this type of expiry to be specified (for now, I'm dropping the expires option).

Compare these two tasks, generated from the same PR:

1. https://queue.taskcluster.net/v1/task/ERiMtxoYSri-M0YWhMzBSA
2. https://queue.taskcluster.net/v1/task/ICyhFSvWTFWYJfmTi15pKA

The tokens in the first task have been properly substituted, such that we have a valid task. The second task still contains the literal text {{ event.base.user.login }}.

The missing substitutions seem to be intermittent and happened about one time in six for me (at a guess).

I noticed that {{ event.base.user.login }} is a token which is replaced for a pull request, but not replaced for a push, so I wonder if there is a race condition, such that when a commit is pushed to a branch where there is an active pull request, sometimes the push event might win the race against the pull request event, and then the task is generated as if it was for a regular push rather than for a pull request. This is based on looking at the data in

CC @djmitche @imbstack @owlishDeveloper

The URL that taskcluster-github uses for task-graph-inspector links (f.e. the status icon link and the "Details" link in a pull request) contains no trailing slash, f.e.:

https://tools.taskcluster.net/task-graph-inspector/#Rjr993ErSe6LRpv3O-rS5A

But when you follow that link, the URL gets changed to a version that contains a trailing slash, f.e.:

https://tools.taskcluster.net/task-graph-inspector/#Rjr993ErSe6LRpv3O-rS5A/

Unfortunately, the change happens in a way that adds a history item to the browser's history for the tab. So if you then press the browser "Back" button, you go "back" to the same page, but without the trailing slash on the URL. And you have to press "Back" again to get back to the page you were on before.

If it's necessary to add the slash, it should happen in a way that doesn't add an item to the browser's history.

Which creates one of these and silently fails on github.

https://sentry.io/taskcluster/taskcluster-github/issues/200755273/

ResourceNotFoundError: The specified resource does not exist.
RequestId:32fd1c27-0002-000d-21fe-8155cf000000
Time:2017-02-08T11:27:56.4469691Z
  File "/app/node_modules/fast-azure-storage/lib/table.js", line 482, in null.<anonymous>
    var err         = new Error(message);
  File "/app/node_modules/promise/lib/core.js", line 37, in tryCallOne
    return fn(a);
  File "/app/node_modules/promise/lib/core.js", line 123, in null.<anonymous>
    var ret = tryCallOne(cb, self._65);
  File "/app/node_modules/asap/raw.js", line 50, in flush
    queue[currentIndex].call();
  File "internal/process/next_tick.js", line 67, in _combinedTickCallback
...
(1 additional frame(s) were not displayed)

ResourceNotFoundError: The specified resource does not exist.
RequestId:32fd1c27-0002-000d-21fe-8155cf000000
Time:2017-02-08T11:27:56.4469691Z

When a patch is made through the github UI, it ends up getting set to the owner of the repo. In the docs we say that it is whoever pushed.

https://tools.taskcluster.net/task-group-inspector/#/dMpINi51Rn6fI1TrMD1fzg/dMpINi51Rn6fI1TrMD1fzg?_k=n17428

https://sentry.io/taskcluster/taskcluster-github/issues/208036664/

TypeError: Cannot read property 'name' of null
  File "/app/lib/api.js", line 46, in getPullRequestDetails
    'event.head.repo.name': eventData.pull_request.head.repo.name,
  File "/app/lib/api.js", line 223, in Object._callee$
    msg.details = getPullRequestDetails(body);
  File "/app/lib/api.js", line 303, in Object.<anonymous>
    return _ref.apply(this, arguments);
...
(11 additional frame(s) were not displayed)

TypeError: Cannot read property 'name' of null

https://sentry.io/taskcluster/taskcluster-github/issues/222528905/

YAMLException: duplicated mapping key at line 315, column 7:
          mounts:
          ^
  File "/app/lib/handlers.js", line 241, in Handlers._callee8$
    c = yaml.safeLoad(repoconf);
...
(17 additional frame(s) were not displayed)

YAMLException: duplicated mapping key at line 315, column 7:
          mounts:
          ^

See https://tools.taskcluster.net/task-graph-inspector/#CxZTDmkKTzSxPjrWumSbJA/I1JH2PSIT5if58ZCi1vkTQ/0

It's related to moby/moby#783

I hit this issue on my local machine yesterday using docker 1.8.2 and aufs. Switching to overlayfs or btrfs fixed the issue. Interestingly, I cannot reproduce it, when build the image locally.

See https://docs.taskcluster.net/reference/core/github/api-docs

The docs need writing, and we need to define request/response schemas, otherwise nobody can integrate with the service.

This is something I've only noticed in the past week or so - sometimes it takes upwards of 5 minutes for Taskcluster to notice a change to a pull request. For example, on mozilla-releng/balrog#211, 9f9980a was added at 11:35am eastern, but the Tasks were not created until 11:40.

This isn't the end of the world, but it's unfortunate that nothing about the PR is updated for that long.

GitHub is changing URLs associated with GitHub apps (nee integrations). Details and timetable in the blog post

It is now deprecated in favor of just doing it with the queue directly.

https://sentry.io/taskcluster/tc-github-staging/issues/222628365/

EntityAlreadyExistsError: The specified entity already exists.
RequestId:de8a0085-0002-00bc-2190-8c71f4000000
Time:2017-02-21T22:22:00.6435068Z
  File "/app/node_modules/fast-azure-storage/lib/table.js", line 482, in null.<anonymous>
    var err         = new Error(message);
  File "/app/node_modules/promise/lib/core.js", line 37, in tryCallOne
    return fn(a);
  File "/app/node_modules/promise/lib/core.js", line 123, in null.<anonymous>
    var ret = tryCallOne(cb, self._65);
  File "/app/node_modules/asap/raw.js", line 50, in flush
    queue[currentIndex].call();
  File "internal/process/next_tick.js", line 67, in _combinedTickCallback
...
(1 additional frame(s) were not displayed)

EntityAlreadyExistsError: The specified entity already exists.
RequestId:de8a0085-0002-00bc-2190-8c71f4000000
Time:2017-02-21T22:22:00.6435068Z

Tasks should show up in treeherder if the

extra:
  treeherder:
    ....

Currently, we'll get a warning in papertrail, like:

 Tue, 17 Jan 2017 18:09:46 GMT taskcluster-github error (reported to sentry) while calling job handler: YAMLException: bad indentation of a sequence entry at line 83, column 39: 

Instead we should probably post a message to PR / commit to say that the .taskcluster.yml is malformed, and mark it as a failure. Currently, you just don't get a message e.g. in the PR so you don't know if the format was bad, or if the service is down.

CC: @owlishDeveloper @imbstack @djmitche

#203 (comment)

In the endpoint that receives the webhook from github, we remove certain characters from certain fields. I'm not sure why this is yet (I'm guessing maybe pulse wants the routing key to have no '.'?) but I don't see where we're converting back inside the worker in order to call out to the github api. I don't think (also not confirmed) that github understands science%mozilla%org to mean science.mozilla.org. I think this might cause us to not be able to run repositories with '.' in them, but not confirmed quite yet.

Sometimes we end up with infrastructure failures as part of PR checks. These are usually fixable by rerunning the Task, but doing so doesn't change the status on Github. For example, mozilla-releng/balrog#515 had one failing Task (https://tools.taskcluster.net/groups/RxbsRtsYTmu_PeOQK58DHQ/tasks/RxbsRtsYTmu_PeOQK58DHQ/runs/0), which I reran. The Task (and Group) are green now, but the status check is still showing a failure. (For posterity, here's a screenshot of both: https://hearsum.ca/~bhearsum/2ad3488b.png)

This is particularly inconvenient when Taskcluster is a required status check -- patches cannot be merged without changing something on the branch to get an entirely new commit + CI.

Getting this message here.

CC @owlishDeveloper @djmitche @imbstack

See https://tools.taskcluster.net/task-graph-inspector/#CxZTDmkKTzSxPjrWumSbJA/I1JH2PSIT5if58ZCi1vkTQ/0

There are actually 2 issues here:

1. $GITHUB_HEAD_BRANCH is not set, but checkout-pull-request requires it.
2. it fails to checkout but exits 0, what makes the build green.

Currently when the greenkeeper bot pushes a branch and makes a PR on any of our repositories that are managed that way, tc-github gives a 403 claiming Bad Signature. I have a hunch that this is because tc-github doesn't know that we've allowed greenkeeper to be doing this sort of thing. I need to investigate further, but presumably there's a way we can figure out that the org has given permissions to the bot and therefore allow it to run builds on pushes/PRs.

(This is @jhford) We should be posting to the status api that github exposes on conclusion of the task graph with overall status.

Over in mozilla/positron#121, I've been improving Positron's .taskcluster.yml file via a series of incremental changes. And until recently, each time I've pushed a change, TaskCluster has used the new .taskcluster.yml file to configure a task graph run.

But the most recent change, mozilla/positron@46799e7, triggered a task graph run that appears to be using the original, unmodified .taskcluster.yml rather than the one on the branch.

Strangely, the most recent change also triggered two runs:

• https://tools.taskcluster.net/task-graph-inspector/#N-Pj3JQgRQeSu0Nd-xF6PA is referenced by the status "dot" next to the revision ID and the "Details" link.
• https://tools.taskcluster.net/task-graph-inspector/#NFbkLaNiS_CVE01iwpdQrA is referenced by this TaskClusterRobot comment on the revision.

Note: I'm using a branch in the upstream repository (https://github.com/mozilla/positron) rather than one in my fork (https://github.com/mykmelez/positron) because of this "note on testing" on TaskCluster GitHub:

TaskCluster GitHub will not use .taskcluster.yml files from pull requests. To test .taskcluster.yml changes create a new branch on the master/base repository and open pull requests against it or make pushes to it (depending on the events you have enabled).

Based on that note, I would expect TaskCluster to do what it has been doing until the most recent change: use the revised .taskcluster.yml file in the branch, since the branch is on the master/base repository.

A PR with https://github.com/mozilla/example-addon-repo/blob/master/.taskcluster.yml#L110 in it was rejected due to a lack of scopes: mozilla/example-addon-repo#37

I'm going to investigate if this was broken by an update to tc-gh or if tc-gh just isn't allowed to assign things to that workertype or if the repo itself needs extra scopes.

@Standard8 did this used to work for you or is it a new task?

A PR was created by greenkeeper, on a branch called greenkeeper/update-all. The resulting task was:

https://tools.taskcluster.net/task-group-inspector/#/C6HDrZviTcusOfVhqcneFw/KT9QPaemTZqMVevN8Y2-6A?_k=ds29s1

As you can see from the task details, the GITHUB_HEAD_BRANCH variable was set to greenkeeper rather than greenkeeper/update-all.

It would be unbelievably awesome if we could have status badges for including in README.md files, a bit like other CI providers do. Maybe we can integrate with http://shields.io/ ?

Not sure how widespread this is at the moment.

#149 has statuses reported for 5cfa55d and 7c55062. However, the task groups linked in them are identical. We can see that different tasks were triggered for each revision latter, earlier.

I have a ticket open with github support now over the case of the disappearing status links on branch commit pages. We have a sorta-workaround now:

11:56:22 <gerard-majax> how am I supposed to get link to taskcluster in the mean time?
11:56:29 <bstack> I'll open a bug on github to reference
11:56:33 <bstack> which repo?
11:56:35 <bstack> and branch?
11:56:42 <gerard-majax> deepspeech
11:57:03 <gerard-majax> https://github.com/mozilla/tensorflow/tree/v1.0.0-warpctc
11:59:17 <bstack> So here is every build we've done for tensorflow: https://github.taskcluster.net/v1/builds?organization=mozilla&repository=tensorflow and https://github.taskcluster.net/v1/repository/mozilla/tensorflow/v1.0.0-warpctc/latest is a link to the latest build on that branch
11:59:47 <bstack> https://github.taskcluster.net/v1/repository/mozilla/tensorflow/v1.0.0-warpctc/badge.svg will get you a badge for your readme as well if you'd like :p
12:00:05 <&garndt> hah I like that badge
12:00:13 <bstack> url hacking should let you use those links for any repo you like until we get word back from github

Badges such as this one are now just showing up as empty black images with the correct border. Not sure what changed unfortunately.

This might be looking at the builds table and... not sure what happens next.

Not sure if we should figure out a way to handle this with retries or not. Leaving this for now.

https://sentry.io/taskcluster/taskcluster-github/issues/204412157/

Error: {"message": "No server is currently available to service your request. Sorry about that. Please try resubmitting your request and contact us if the problem persists."}

Error: {"message": "No server is currently available to service your request. Sorry about that. Please try resubmitting your request and contact us if the problem persists."}