https://help.github.com/articles/about-codeowners/

The main limitation on GitHub's implementation is that it ignores approvals by people without write access.

This plugin could use the CODEOWNERS file, and respect it.

What are your thoughts on adding the option to @Mention a list of github users?
Maybe something similar to:

comments:
  mention-users: [ user1, user2 ]
  message: 'FYI...'

Context

A team may decide that every PR should have at least an assignee. By default zero number of assignees is acceptable.

Details

Support Advanced settings:

  assignee:
    min: 1
    max: 1
    message: 'Custom message...'

and simple:

assignee: 1 # min number of assignees.

Context

Mergeable will fail if the linting of the source code fails making sure that every code follows the linting rules

Solution

To be discussed in the discussion

When mergeable is invoked it should create a pending status on the PR while it is validating so that the user knows that something is happening.

Preferably we do an async call to create the status at the beginning of the handler function before we do anything else:
https://github.com/jusx/mergeable/blob/40cef874823b14a1cb637e43a2643969f27cc0e9/lib/handler.js#L17

Highly consider abstracting out the block starting at https://github.com/jusx/mergeable/blob/40cef874823b14a1cb637e43a2643969f27cc0e9/lib/handler.js#L45 to https://github.com/jusx/mergeable/blob/40cef874823b14a1cb637e43a2643969f27cc0e9/lib/handler.js#L51

The pending status description should say: "Checking to see if this is mergeable..."

When mergeable comments on an issue because a validation fails it adds all the information why. That's helpful. But when the issue is fixed, e.g. by adding a label and a description, the comment stays. That can be quite confusing and it also adds some noise to the issue.

It would be great if mergeable would remove its own comment if it doesn't apply anymore.

This would also prevent multiple mergeable comments in a row, when an issue is edited and mergeable is triggered by each part of the edit.

Example: Add an issue without description and label, mergeable comments, add a label, mergeable comments again, add a description, no further comment, but the two comments which are there now, don't apply anymore.

Story

As a user
I would want to know all the checks Mergeable ran including the status of each run
so that I can be assured that all the test I've wanted to ran is done

Acceptance Criteria

In both success and failure status for PR,
Show the tests ran in description in the following format :
Status Test Description

where all the failed tests are shown top of the list.

optional ***: we could add details on the test parameter that is passed into each test

Support for the advanced format configuration of Approvals for minimum reviewers:

  approvals:
    min: 5
      message: 'Custom message...'

If you want this feature, please react with a 👍

Context

Allow exception to certain rules, for example, if title contains (fix), don't validate milestone check.

Solution

To be discussed in comments

Some projects require that every source file starts with a piece of text related to license and copyright.

The CI could block a PR if some new files are introduced without these lines.

Title contains export does not pass mergeable because it contains substring exp.

When a push is made to a repos, GH will send a check_suite.requested event. In this event, the check status is set to queued

According to the docs:

The requested action triggers when new code is pushed to the app's repository.

For example, this event is received when a push is made to master and also when a push is made to the branch of a PR.

We need to handle this especially so when this event is received and the payload has an array of pull_request that is not empty. This array contains all PRs that has the same head_branch and head_sha as the push (according to the docs). Not sure how it can ever be more than one element though.

The problem

This may cause intermittent "hanging" checks in a PR — when a user pushes to the branch of the PR, two events are received by Mergeable: pull_request.synchronize and check_suite.requested — IF the check_suite.requested is received after mergeable finishes processing the pull_request.synchronize event and sent a status completed check, that check is potentially now in queue status.

There is a race-condition.

Potential Solution

What we need to do when check_suite.requested event is received:

• Add this event to special cases function and get the pull_request. For now only get the first element and assume there's ever only one element in the pull_request array.
• Add check_suite.requested to the checks action.
• If there are config errors call Checks.run
• else processWorkflow.

Update 2019-02-07

Did a little bit more testing with various scenarios:

1. As long as checks are created and updated with complete or cancelled status, everything looks normal from a user perspective and there is no need to take action on the check_suite.requested event for that PR. It works as expected.

2. If there are no checks manually created and completed for a PR where the check_suite.requested event is triggered then the UI will not show anything in the conversations tab, however it will show a queued status for the app in the checks tab. It does not prevent merging though.

Currently, mergeable will always create a check and finish it with completed status for every event it is asked to listen to in the configuration -- regardless of whether the check_suitre.requested is received. This is the expected behavior.

There isn't any race conditions that I can determine since we do 1. The check_suite.requested event has a totally different check run id as compared to the one we receive when creating a check_suite.

Is it possible to install this app on GitHub Enterprise?

begins_with does not seem to be validating properly.

To Reproduce

1. Create a recipe like this:

version: 2
mergeable:
  - when: pull_request.*, pull_request_review.*
    validate:
      - do: title
        begins_with:
          match: 'feat', 'bug'
          message: 'Type is required....'

2. Set the title of the PR to be "Something that does not match"

Actual

The PR is still mergeable. Result is pass.

Expected:

PR should not be mergeable. Result should be fail.

It'd be nice if we could remove the requirement for approvals for certain PRs. In particular, I'd be interesting in making PRs created by dependabot to not require approvals. This could be done based on a label, e.g.

mergeable:
  pull_requests:
    approvals:
      min: 5
      except:
         label: dependencies

If this is something that would be accepted, I can work on a PR to implement it.

EDIT:

Actually, I see there's an undocumented and/or operator that could be used to implement this. Then all that'd be needed is adding a 'label' rule to processFilters.

mergeable:
  pull_requests:
    approvals:
      or:
        - min: 5
        - label: '^dependencies$'

This is primarily an edge case:

To Reproduce:

• By default the minimum approvals is 1.
• Approve a PR.
• Then change the review status to Comment, which removes your approval status.

Actual:

Mergeable is true.

Expected:

Mergeable should be false since the number of approval count is actually zero.

Context

When the mergeable YML does not have expected indentation or the format is wrong js-yaml throws an exception and mergeable does not stop running.

Proposal

• Mergeable should catch these exceptions and notify.
• Potential Notifications:
  • Set a status on PR with bad YML configuration found. Ideally the details should link to a mergeable
  • Send a checks with explanation about YML problem.
  • Comment on the PR or issue when the problem is found.

Looking at the full config in readme, it looks like there is no way to request for no_empty for a PR's milestone like we can do for the description.

I was able to replicate the behaviour using regex, using the following config:

mergeable:
  pull_requests:
    milestone:
      must_include:
        regex: '([^\s]*)'
        message: 'This PR cannot be merge unless it belongs to a milestone.'

Should it be added though?

I have added check for milestone in PRs in my config.
When I create a PR, if I forget to add milestone, the mergeable will mark it as failed.
If I add the PR in a milestone, the mergeable check remains red.
I have to re-trigger a check by changing the title, the description or the labels.

By the way, re-run via Checks tab does not seem to be running.

The re-run check capability is not working.

Once you click on it, it'll say that it's pending and it'll stay like that for ever until an event occurs (e.g. a label is added, etc). Only then is the mergeable check re-run.

Is it possible to have a configuration to check the labels in the following way:
(A | B). If B is present then a label with prefix XYZ must also be present. I think the answer is no but just wanted to check. Thanks.

Recently, Mergeable started reporting the error "Resource not accessible by integration". Not sure what's causing this. Any ideas?

Context

• Currently there's too much information to display in the UI when a PR fails validation.

Solution

• Details should display in Checks.

It would be nice to have mergeable (automagically) request a review from all people in the reviewers list.

Context

Some PRs may have a TODO list (list of checkboxes). Typically an Engineer may have this as a checklist for all that needs to be done while PR is in WIP.

Solution

Should be mergeable if not all the checkboxes are checked.

Simple configuration:

todo: true

Advanced:

todo: 
   enabled: true
   message: 'a custom message'

The example regex provided in readme such as ^\\(feat\\)|^\\(doc\\)|^\\(fix\\) should only have one \ instead of \\

As a user I would want all the errors to be formatted in a way that is easy for me to read and understand the exact details so that I know exactly which area to fix.

@jusx do you have UI preference?

Make it easy to configure mergeable such that the title follows a certain format. For example, in previous projects we've required certain prefixes like (feat), (doc), (fix) in the title. This should be easily configured without complex regex like this #12 (comment)

Instead we should be able to do something like this:

title: 
   excludes: 'wip'
   includes: '^\(feat\)|^\(doc\)|^\(fix\)' 

Probably will still want to be backward compatible with:

title: 'wip'

We are receiving a “This PR is stable comment” in all Issues today! I guess started since the latest commits today! It is spamming us and there are around 200 issues open which are receiving comment. Please take a look

If you want this feature, please react with a 👍

We want to be able to label an issue or pull request when an event occurs. There are two scenarios where this applies:

1. When an event occurs under certain conditions (validation) -- example: Label based on the inclusion of certain keywords in the description of the issue / PR.
2. When an event occurs -- example: Auto-label when an issue is opened (i.e. with the label "triage")

Implementing a Label Action class will automatically make scenario 1 possible:

version: 2
mergeable:
  - when: issues.opened
    validate:
      - do: description
        must_include:
          regex: 'Depends on'
    pass:
      - do: label
        name: Dependencies

For scenario 2 there isn't a good way to write the recipe. Ideally if we don't want to spcify any validations we should be able to execute any actions whenever an event occurs:

when: issues.opened
   execute:
      - do: label
         name: 'triage'

Without the execute block we may be able to write it as:

when: issues.opened
   validate:
   pass:
      - do: label
         name: 'triage'

The above should work (I have not verified) but semantically clunky....

Question: Are there other use cases that fits our current workflow architecture?

History

Below is the original description for this issue written based on the older version of mergeable

Context

When new issues are created, auto-label them with a specified label i.e triage this will allow repo managers to easily identify which issues need to be triaged

Solution

issues:
  opened:
    add-label:
      label: 'triage' // create label if not found
      only-when-empty: true | false // only add label if no other label exists

When trying to validate pull request titles, mergeable fails with the following:

I've tried using the example from the readme:

title:
  must_include:
    regex: `^\\(feat\\)|^\\(doc\\)|^\\(fix\\)`
      message: `Title must have prefixes for the following: (feat), (doc), (fix)`

and the following:

  must_include:
    regex: `#\d{7,}'

Context

Having a lot open PR is a pain to deal with, especially when there are merge conflicts, one way to deal with it to limit the number PR an author can open, this will force the authors to get the help needed to merge their older PR before creating new ones

Solution

To Be discussed in the comments

Context

In large repos (usually a monolith) with many contributors the number of PRs can be long. Some PRs become stale and eventually is no longer up to date.

Solution

Prompt the author when a PR stays open for n number of days.

Configuration:

workflow:
   stale:
      pull_request: 
         days: 20 # number of days.
         message: 'some message' # custom message. 

I've installed Mergeable in my fresh private repository which also has two Jenkins builds attached using Webhooks. Here is my work in progress .github/mergeable.yml:

# https://github.com/jusx/mergeable
mergeable:
    pull_requests:
        # Minimum of 5 approvals is needed.
        approvals: 1

        stale:
            # number of days for a PR to be considered stale. A comment is posted when it is stale.
            days: 20

            # Optional property. When not specified the default is used. The default message is:
            # There haven't been much activity here. This is stale. Is it still relevant? This is a friendly reminder to please resolve it. :-)
            message: 'This is PR is stale. Please follow up!'

        title:
            must_exclude:
                regex: 'wip'
                message: 'This PR is work in progress.'

            begins_with:
                match: '(FEAT\-)|(BF\-)'
                message: 'PR title should start with FEAT- or BF-'

        description:
            no_empty:
                enabled: true
                message: 'PR requires a description'

After creating my first PR, 3 checks show up - two Jenkins checks (as expected) and Mergeable check. The Mergeable check has the "Mergeable Tests running" state. Jenkins build finish successfully but Mergeable stays in "Tests running" state. I've made some errors in the PR which should be caught by Mergeable, just to test if it's working but nothing happens.

It has been like this for the last 3 hours. Will it finish eventually?

The message for non-matching milestone in a PR is not displayed.

Validate that a user or list of users have reviewed the pull request in order for it to be mergeable.

Reviewer list should be configurable in the YAML with:

  approvals:
    required:
      reviewers: [ user1, user2 ]   # list of gh usernames
      message: 'Custom message...'

or

  approvals:
    required: [ user1, user2 ] 

Label should support must_include and must_exclude similar to title in validation and in the YML.

Our use case is the following. We'd like to have our current milestone set in the mergeable.yml so that only PRs targeting the current milestone are able to be merged. Once we release this version, we'd like to change the milestone in mergeable.yml and have all PRs that were previously targeting this version re-run, so that the check passes as they now target the current milestone.

I believe this goes beyond this use case though. If you change your configuration on mergeable.yml, all PRs which mergeable reported something should re-run in order to reflect the up-to-date state with this new configuration.

As mergeable grows in visibility and contributors, having a linter would help keep code contributions as clean as possible.

ESLint has become the go-to, but others are available as well.

When my Mergeable config is invalid, primarily due to an easy YAML mistake, Mergeable check does not show up on my PRs at all, I have no indication that my config is wrong. Instead, it looks like Mergeable has died. :-(

Context

There are instances where there are dependent files. For example when a project is using yarn but yarn.lock is not necessarily automatically generated but is required to be committed for deployment to various env. to work.

Solution

Validate that modified files includes all configured dependent file list.

Advanced configuration:

dependent:
   files:
      - 'package.json'
      - 'yarn.lock'
   message: 'custom message' # this is optional.

Context

A standard have been specified for consistencies for Issues.

Solution

Validate Issues for title, labels, milestones, projects, assignee.

Create a message upon issue.open event.

The PR should not be mergeable when there is no description on the PR. Suggested description:

"Description on the pull request should not be empty."

My PR's are required to be prefixed with a certain tag. Is it possible to add the option to say phrase prefix must be present?

• Validate that a project is assigned as per configuration. Simple configuration project: 'project name'
• If PR is not associated with a project then validate that the description specifies an issue with the correct project being associated. Validation should look for the auto closing notation as specified here: https://help.github.com/articles/closing-issues-using-keywords/ and follow the issue(s) to validate.

Update README with examples of Complex logic checks as per the PR #75

Here’s a failure I have encountered today. Notice how the summary says “Invalid configuration”, presumably the feature from #156, but the contents show a server error. The configuration has not been changed.

This is, of course, a minor cosmetic bug, the app is great!

Support the following configuration for the description validator:

  description:
    no_empty:
      enabled: false
      message: 'Custom message...'
    must_include:
      regex: 'feat'
      message: 'Custom message...'
    must_exclude:
      regex: 'DO NOT MERGE'
      message:

When the configuration specifies a milestone requirement it currently validates that the milestone on the PR has the correct association.

If PR is not associated with a milestone then validate that the description specifies an issue with the correct milestone being associated. Validation should look for the auto closing notation as specified here: https://help.github.com/articles/closing-issues-using-keywords/ and follow the issue(s) to validate.

Reproduce

• add user1 to the approvals required list in configuration
• as user1 create a PR.

Actual

PR is not mergeable.

Expected

PR should be mergeable since the author of a PR cannot review his/her own PR and hence should be ignored as a required reviewer.