Currently, Shepherd will keep going even if steps for an individual repo fail. We should have a --bail option that aborts the entire process as soon as something fails.

Add some sort of introductory guidance to TypeScript for those wanting to make a contribution but may not be familiar with TypeScript.

When my pr message generation fails it'd be helpful if stdout/stderr from the command was printed out. Currently I only get this output:

shepherd pr --repos NerdWallet/review-client migrations/2018-08-dependency-updates

[NerdWallet/review-client] 1/1
$ node $SHEPHERD_MIGRATION_DIR/pr-message.js
Step "node $SHEPHERD_MIGRATION_DIR/pr-message.js" exited with 1
✖ Failed to generate PR message

shepherd merge <migration> [--repos org1/repo1,org2/repo2] [--ignore-checks] [--ignore-approvals]

Merge all mergeable PRs for a migration. This would help when managing large migrations, e.g. in the hundreds of repos. I think the safest version of this command defaults to requiring 1 PR approval, 100% approval by reviewers (or at least no changes-requested reviews), and all status checks to be successful.

Arguments:

• --ignore-checks: Don't require status checks to exist/be successful
• --ignore-approvals: Don't require at least one approval, or 100% approval

Introduction

As an engineer, I want to be able to quickly and easily tell which version of shepherd is installed from the command line.

Proposed Change

Add version command to cli.

Introduction

We use RenovateBot but DependaBot is still enabled for shepherd repo triggering notifications within Slack channel and upgrade PRs already covered by Renovate.

Acceptance Criteria

• DependaBot is no longer enabled on Shepherd repo
• We no longer see PRs being opened by DependaBot
• We no longer receive Slack notifications from DependaBot

In some orgs, PRs are required to have at least one reviewer. It would be a great feature to allow a migration to automatically add review requests during PR creation. CODEOWNERS is a great way to do this automatically in Github, but not all repos will have this configured.
The mechanism for getting the list of reviewers to add could be through a hook pr_reviewers which would operate much like the pr_message hook except that it would gather a comma-separated list of reviewers.

Something like:

shepherd apply-local <migration> <repo path>

Where migration is the name of the migration to run and repo path is a local filesystem path to the repo. The "migration name" can ultimately become pretty rich and support specifying an npm package as a migration (which could get installed automatically) or a git remote (which could get cloned automatically), but we can start with supporting just a simple local filesystem path for the migration too. This should enable a workflow like this:

hub clone nerdwallet/shepherd-migrations
npx @nerdwallet/shepherd apply-local shepherd-migrations/react-16 ~/projects/my-cool-app

Some considerations:

• Should not rely on using the GitHub API (thus apply-local instead of apply).
• $SHEPHERD_DATA_DIR might need to get to set to a temp directory or something.
• It'd be nice to print what would normally have been the PR message to stdout after running the migration.
• There might be a better name than apply-local?

/cc @nwalters512

Introduction

This issue picks up where issue #103 leaves off. In particular, this issue addresses bringing jest and jest related dependencies up-to-date.

Proposed Change

• Remove jest-plugin-fs dependency in favor of using local artifacts
  • Idea here is to use a few publicly accessible github repos
• Test each shepherd cli command

ROI

• Resolves remaining 206 vulnerabilities
• Unit tests more aligned with user actions

Currently there's two types of auth needed to use shepherd:

1. GitHub API auth. Currently, api auth can come from either a GITHUB_TOKEN environment variable or a ~/.netrc file. See code:

   shepherd/src/adapters/github.ts

   Lines 36 to 49 in 40f4272

   	if (process.env.GITHUB_TOKEN) {
   	this.octokit.authenticate({
   	type: 'oauth',
   	token: process.env.GITHUB_TOKEN,
   	});
   	} else {
   	const netrcAuth = netrc();
   	// TODO: we could probably fail gracefully if there's no GITHUB_TOKEN
   	// and also no .netrc credentials
   	this.octokit.authenticate({
   	type: 'basic',
   	username: netrcAuth['api.github.com'].login,
   	password: netrcAuth['api.github.com'].password,
   	});

2. Git remote repository access. Currently, shepherd just expects command like git clone to work in the current environment (e.g., your ssh key is set up).

This information should be documented and the tutorial should mention how to set things up if you haven't already.

We could also better handle the failure cases when auth is not available and print helpful error message.

One thing I've found difficult when writing/attempting migrations is figuring out what I have, what state they're in, and what my next steps are.

There are four commands I'd like to add to help with this (btw naming suggestions welcome; these make sense to me but I haven't run them by anybody else)

shepherd migrations

List all migrations I have in my local cache (default: ~/.shepherd)

shepherd migrations

2019-09-15-add-foo
2019-09-16-remove-bars

shepherd migration-state

Tell me how much progress I've made in each migration

shepherd migration-state

(This is a rough sketch; I'd want to use something like cli-table to render this)

migration   | checked out | applied    | committed          | pushed     | pr opened | pr mergeable
------------|-------------|------------|--------------------|------------|-----------|-------------
add-foo     |  150/200    | 130/150    |  130/130           |            |           |       
            |   (75%)     |  (86%)     |  (100%)            |            |           |       
            |             |            |  *OLD*: 1 hour ago |            |           |       
------------|-------------|------------|--------------------|------------|-----------|-------------
remove-bar  | 200/200     | 130/200    | 130/130            | 130/130    | 130/130   |  120/130
            |  (100%)     |  (65%)     |  (100%)            | (100%)     | (100%)    |   (92%)

• Lists all migrations in my local shepherd dir
• Prints out a summary of how many repos made it through each stage, total numbers and percentage
• Flags old stages. If I apply-ed the migration add-foo after committing it, let me know I need to re-do that

This reqires doing more state tracking, probably in a JSON file in the shepherd migration directory. Every time I run a shepherd command I should update that JSON file as I pass through each repo, and then shepherd migration-state does aggregation across all the migrations.

shepherd migration-state <migration>

Tell me how much progress I've made in a given migration. Again, rough sketch

repo        | checked out | applied    | committed  | pushed     | pr opened | pr mergeable
------------|-------------|------------|------------|------------|-----------|-------------
lib-foo     |      ✓      |     ✓      |  ✓         |     ✓      |     ✓     |       ✗
app-baz     |      ✓      |     ✓      |  ✓         |     ?      |           |       
lib-boop    |      ✓      |     ✗      |            |            |           |       
app-bar     |      ✗      |            |            |            |           |  
------------|-------------|------------|------------|------------|-----------|-------------
summaries   |     75%     |    66%     |     100%   |     50%    |    100%   |     100%

• List every repo I've gone through for a given migration, and the progress we've made on it
• Mark each one that explicitly failed a step with an X
• Mark each one that we just haven't run yet (see: app-bar in example) with a ?
• Probably flag old commands the same as the version that isn't migration-specific, so if we needed to e.g. re-run apply + commit for app-baz it would be flagged as old
• Summarize by percentage at bottom

Narrative

As a Shepherd user, I want quick and easy access to the version of Shepherd installed via cli.

Acceptance Criteria

• When I run shepherd --version or shepherd -v, I see the semver for Shepherd.

Hi!
It seems shepherd is using the auth method Github is in process of deprecating: https://developer.github.com/changes/2020-02-10-deprecating-auth-through-query-param (I get an email from Github about using it when running shepherd).
Is there some dependency you need to bump, or is this in shepherd itself?

Per comment from @parshap on #2:

I feel we should persist state using json instead of yaml. Yaml is most useful when optimizing for human editing.

I ran shepherd pr against a repo that already had a PR opened and merged and received this 422 error as a result:

[NerdWallet/user-review-client] 35/38
$ node $SHEPHERD_MIGRATION_DIR/pr-message.js
Step "node $SHEPHERD_MIGRATION_DIR/pr-message.js" exited with 0
✔ Generated PR message
{ HttpError: {"message":"Validation Failed","errors":[{"resource":"PullRequest","field":"head","code":"invalid"}],"documentation_url":"https://developer.github.com/v3/pulls/#create-a-pull-request"}
    at response.text.then.message (/Users/ppourkhomami/projects/nerdwallet/shepherd2/node_modules/@octokit/rest/lib/request/request.js:72:19)
    at process._tickCallback (internal/process/next_tick.js:109:7)
  name: 'HttpError',
  code: 422,
  status: undefined,
  headers:
   { 'access-control-allow-origin': '*',
     'access-control-expose-headers': 'ETag, Link, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval',
     connection: 'close',
     'content-length': '184',
     'content-security-policy': 'default-src \'none\'',
     'content-type': 'application/json; charset=utf-8',
     date: 'Tue, 28 Aug 2018 00:44:16 GMT',
     'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
     server: 'GitHub.com',
     status: '422 Unprocessable Entity',
     'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
     'x-accepted-oauth-scopes': '',
     'x-content-type-options': 'nosniff',
     'x-frame-options': 'deny',
     'x-github-media-type': 'github.v3; format=json',
     'x-github-request-id': 'F406:64C6:2210746:2B40DAB:5B849AE0',
     'x-oauth-scopes': 'repo',
     'x-ratelimit-limit': '5000',
     'x-ratelimit-remaining': '4932',
     'x-ratelimit-reset': '1535419709',
     'x-runtime-rack': '0.068436',
     'x-xss-protection': '1; mode=block' } }
✖ Failed to create pull request

Erroring is the right thing to do, but a nicer error message would be useful.

Introduction

Deprecate Node 8 and add support for Node 14.

When application starts, we reset the branch; when it fails, we reset any work that was done on it. However, we don't remember that application failed, so we end up force-pushing the removal of all existing commits to the remote. This has the unintended side effect of closing the PR. We should either maintain a list of repo statuses or merge apply and commit.

Problem

When first trying to run shepherd, I'm getting an error related to login() that I suspect is due to lack of Github auth credentials.

This is version 1.2.0 (current latest) on node 10.15.0

shepherd checkout temp

[before-after-hook]: "Hook()" repurposing warning, use "Hook.Collection()". Read more: https://git.io/upgrade-before-after-hook-to-1.4
TypeError: Cannot read property 'login' of undefined
    at new GithubAdapter (/Users/aturek/try-shepherd/node_modules/@nerdwallet/shepherd/lib/adapters/github.js:55:59)
    at Object.adapterForName (/Users/aturek/try-shepherd/node_modules/@nerdwallet/shepherd/lib/adapters/index.js:10:20)
    at Object.<anonymous> (/Users/aturek/try-shepherd/node_modules/@nerdwallet/shepherd/lib/cli.js:61:36)
    at Generator.next (<anonymous>)
    at fulfilled (/Users/aturek/try-shepherd/node_modules/@nerdwallet/shepherd/lib/cli.js:5:58)
    

Repro steps

1. yarn add @nerdwallet/shepherd

2. mkdir temp

3. write this into temp/shepherd.yml (sample YAML file from the blog post)

   id: 2018.07.16-eslintrc-yml
   title: Rename all .eslintrc files to .eslintrc.yml
   adapter:
     type: github
     search_query: org:NerdWallet path:/ filename:.eslintrc
   hooks:
     should_migrate:
       - ls .eslintrc
       - git log -1 --format=%cd | grep 2018 --silent
     apply: mv .eslintrc .eslintrc.yml
     pr_message: echo "Hey! This PR renames `.eslintrc` to `.eslintrc.yml`" 

4. shepherd checkout temp

When fetching a very long list of matching repos from GitHub, a consumer will run into a rate limiting error during shepherd checkout.

dbarnes:shepherd-migrations dbarnes$ shepherd checkout migrations/2018-09-cdn-url-updates/
{ HttpError: {
  "documentation_url": "https://developer.github.com/v3/#abuse-rate-limits",
  "message": "You have triggered an abuse detection mechanism. Please wait a few minutes before you try again."
}

    at response.text.then.message (/srv/nerdwallet/shepherd/node_modules/@octokit/rest/lib/request/request.js:72:19)
    at process._tickCallback (internal/process/next_tick.js:103:7)
  name: 'HttpError',
  code: 403,
  status: undefined,
  headers:
   { 'access-control-allow-origin': '*',
     'access-control-expose-headers': 'ETag, Link, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval',
     connection: 'close',
     'content-encoding': 'gzip',
     'content-security-policy': 'default-src \'none\'; base-uri \'self\'; block-all-mixed-content; connect-src \'self\' uploads.github.com status.github.com collector.githubapp.com api.github.com www.google-analytics.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com; font-src assets-cdn.github.com; form-action \'self\' github.com gist.github.com; frame-ancestors \'none\'; frame-src render.githubusercontent.com; img-src \'self\' data: assets-cdn.github.com media.githubusercontent.com camo.githubusercontent.com identicons.github.com collector.githubapp.com avatars0.githubusercontent.com avatars1.githubusercontent.com avatars2.githubusercontent.com avatars3.githubusercontent.com github-cloud.s3.amazonaws.com; manifest-src \'self\'; media-src \'none\'; script-src assets-cdn.github.com; style-src \'unsafe-inline\' assets-cdn.github.com',
     'content-type': 'application/json; charset=utf-8',
     date: 'Mon, 24 Sep 2018 21:30:00 GMT',
     'expect-ct': 'max-age=2592000, report-uri="https://api.github.com/_private/browser/errors"',
     'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
     'retry-after': '60',
     server: 'GitHub.com',
     status: '403 Forbidden',
     'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
     'transfer-encoding': 'chunked',
     'x-content-type-options': 'nosniff',
     'x-frame-options': 'deny',
     'x-github-media-type': 'github.v3; format=json',
     'x-github-request-id': '5A8C:0D02:256FFC7:2EBD74C:5BA95758',
     'x-runtime-rack': '0.007640',
     'x-xss-protection': '1; mode=block' } }

Proposal:
Try/catch when fetching and obey the retry-after header returned from GitHub before continuing to paginate through the list of results.
https://developer.github.com/v3/guides/best-practices-for-integrators/

When a user uses an ssh key for git commands and their key expires, shepherd silently fails when attempting to check out repositories for a migration during shepherd checkout by not forwarding the password prompt to the user and staying at the "checking out {repo_name}" message indefinitely.

I'd like to be able to run some lightweight filters that can't be encoded in the GitHub search API before checking out the repo to do heavier weight repos.

For example, my search results are matching repos that has node_modules checked in and the query is matching files inside of node_modules.

For example, a url like this: riphunter07/ATproject/ATproject/node_modules.old/react-native/jest/setup.js.

I could encode NOT node_modules in my query, but I think that would exclude modules with:

const foo = require('./node_modules/foo/foo.js');

Github lets you do searches with file names but I don't think that lets you do negative file name searches:

Libraries/Renderer/shims/ReactNative fork:false language:JavaScript filename: NOT native_modules NOT flow-typed

Being able to do this early filtering (on paths, or even other things that GitHub returns as part of the API search results) would let me avoid having to clone hundreds of repos that are using React Native to realize my search is finding code in react-native core in each of these projects.

A common pattern that's emerging is that migrations have dependencies, such as from npm. Those currently have to be manually installed. In the future, where Shepherd will be able to fetch migration specs remotely, the users of a script shouldn't have to care about where it's being stored on disk in order to go install its dependencies. We should provide a hook so that migration specs can specify work to be done the first time the migration is run (i.e. on shepherd checkout xxxx when the working directory for that migration does not exist.

We could also add Indy-style understanding of different files, so we'd automatically run npm install if a migration dir contains a package.json.

cc @parshap thoughts on this?

There's some common logic commonly performed in migrations that would benefit from a set of shared utility helpers. This issue should document/discuss potential use cases.

• Check the recency of the repo (e.g., commit in the past 6 months)
• Check if a dependency (at a given semver range) exists in package.json

Feel free to update the list above.

Sometimes you have an exact list of repos you want to apply a migration to, it'd be nice if you could provide a list of repos in shepherd.yml instead of using a GitHub search.

I can currently search for PRs opened via Shepherd with a GitHub search string like this: org:nerdwallet is:pr shepherd eslint, but that's somewhat fuzzy and it'd be good to be able to do a more precise search.

My suggestion is to support a search pattern like this: org:nerdwallet label:shepherd-migration shepherd_migration_2018_06_25_eslint_config_rename.

• Apply a shepherd-migration label to all PRs
• Generate a unique string that gets put in the body text of the PR

The unique string can be generated off of the "id" of the migration. This string should contain only underscores and not hyphens as GitHub search breaks up hyphenated words into individual words and does allow an exact match (regex for valid strings should be something like /[a-zA-Z0-9_]+/). I would do a string search and replace for anything that doesn't match that regex and replace it with an underscore.

The use case for this is to be able to use the GitHub UI to get a quick idea of what state a certain migration is in and help move PRs that are still open along.

Dependabot encountered the following error when parsing your .dependabot/config.yml:

The property '#/update_configs/0/' contains additional properties ["prefix"] outside of the schema when none are allowed
The property '#/update_configs/0/commit_message' of type null did not match the following type: object

Please update the config file to conform with Dependabot's specification using our docs and online validator.

Opening PRs sometimes fails due to missing base branch:

[nerdwallet/interstitial-client] 3/3
$ node $SHEPHERD_MIGRATION_DIR/pr-message.js
Step "node $SHEPHERD_MIGRATION_DIR/pr-message.js" exited with 0
✔ Generated PR message
{ HttpError: Empty value for parameter 'base': undefined
    at values.forEach (/Users/ppourkhomami/projects/nerdwallet/shepherd2/node_modules/@octokit/rest/lib/plugins/endpoint-methods/validate.js:80:15)
    at Array.forEach (<anonymous>)
    at Object.keys.forEach.parameterName (/Users/ppourkhomami/projects/nerdwallet/shepherd2/node_modules/@octokit/rest/lib/plugins/endpoint-methods/validate.js:54:12)
    at Array.forEach (<anonymous>)
    at validate (/Users/ppourkhomami/projects/nerdwallet/shepherd2/node_modules/@octokit/rest/lib/plugins/endpoint-methods/validate.js:32:31)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:160:7)
  name: 'HttpError',
  code: 400,
  status: 'Bad Request',
  headers: undefined }
✖ Failed to create pull request

It looks like defaultBranch is undefined here:

Shepherd is awesome!

It could be nice to get a command that can blow away any dirty state.json that has been created in $SHEPHERD_DATA_DIR while developing.

perhaps shepherd clean or something along those lines.

I wanted to open a discussion on the possibility of making the Shepherd "branding" during PR creation optional via env var / CLI option. The commit title in particular breaks things looking for semantic commit messages.

It would be nice to have an option to print the PR URL from the CLI after creation as some people usually copy paste the PR links to Slack to get them reviewed by someone else. This would make the process a lot easier.

Currently, the Shepherd workflow involves running the checkout, apply, commit, push, and pr commands. The individual commands are useful when iterating on implementing a migration. However, when running an existing migration it might be convenient to do everything in one command. This would enable one command usage via npx like npx shepherd run path/to/migration

Some options/features that may be useful:

• The command should probably feel "idempotent" and "recoverable". Meaning if you run it once and run into an error half way through, you end up in a reasonable, expected state. If you fix the issue and run the command, your process should complete. You shouldn't end up with duplicate PRs, necessary commits/pushes, or anything like that.
• --skip-pr, --skip-checkout, etc in order to skip certain parts of the workflow
• --dry-run: This should probably apply to all other commands too, but instead of actually doing operations print a summary of what would happen.

Several repos in our org do not rely on master being the source of truth...
Is there a way to make a custom branch name the default?
Is it possible to make the branch name to be dynamically decided before checkout based on the repo name or based on a shell command?

We would like to use Shepherd to standardize master as the main branch. But we need a mechanism like described above to help us get there step by step.

NerdWallet's internal CI process includes a check for an updated changelog and will fail if it wasn't updated. This automated check ensures that we're documenting all changes given that it can be easy to forget to update the changelog.

From @parshap comments on #2:

Nit: I don't like that we're not maintaining 1-to-1 mapping of output from the underlying command due the .trim() and newline addition from logger.info(). Maybe a logger.raw() method that doesn't do that?
Also, you should call .setEncoding('utf8') on the streams before reading data otherwise you could technically get a split surrogate pair.

Introduction

Shepherd has a number of advisory exceptions when installing off of v1.4.1 ranging from low to high.

(544 low, 14 moderate, 85 high)

Proposed Changes

Update dependencies to reduce vulnerabilites

Strategy

Phase I

• Packages
  • simple-git
  • ora
  • @types/ora
  • @types/joi -> @types/hapi__joi
  • joi -> @hapi/joi
• CI changes
  • drop support for Node 6; support Node 8, 10, and 12

Phase II

• chalk
• child-process-promise (already on latest)
• js-yaml
• lodash
• log-symbols

Phase III

• Packages
  • commander
  • fs-extra
  • fs-extra-promise
  • netrc
• Audit
  • run npm audit fix

Notes

• Breaking out remaining deps into a separate issue as those will require large changes.

Screenshots

Narrative

As a maintainer of Shepherd, I want a quick and easy way to keep dependencies up-to-date.

Proposed Change

The proposed change is to integrate renovate, renovate approve, semantic commits, and semantic-releases.

With the right combination of settings across these four Github apps, CI/CD can take care of updating minor and patch dependencies (relative to the semantic commit message), approving and merging of the corresponding PRs, publishing a new tag version, and finally updating release notes (if needed) and changelog.

ROI

• near real time vulnerability and security patches
• reduced overhead for maintainers

DependaBot vs RenovateBot

Nice to have

• WIP app

Notes

• Shepherd currently uses dependabot (see https://dependabot.com)
• Renovate Approve (see https://github.com/apps/renovate-approve)
• Renovate (see https://github.com/marketplace/renovate)
• Semantic Release (see https://github.com/semantic-release/github)
• Semantic Pull Requests (see https://github.com/zeke/semantic-pull-requests)

Currently, we rely on either a .netrc configuration or a GITHUB_TOKEN environment variable for GitHub API authentication. We could take some pages out of the hub command and allow people to provide their username/password on the CLI, generate a token for them, and persist it for future use.

Relevant: https://hub.github.com/hub.1.html

We have two options here:

• Assume migrations are idempotent (hard)
• Before applying, remove all commits from branch and start over, and then force-push new commits to branch

The second option is easier and more robust, which is what I'm inclined to go with. However, it introduces the problem of potentially overwriting manual human work on the branch. To get around that, we should pull any changes from the remote and then check if any are not prefixed with [shepherd], which we'll assume means they were made by a human. If we find any, we can either skip that repo entirely or leave a message on the PR. We should also document exactly how this works.

This is particularly relevant to #14 - we have to make a lot of requests to fetch data about all repos. The GraphQL API has much more favorable rate limiting for the types of requests we're doing: https://developer.github.com/v4/guides/resource-limitations/. We can use https://www.apollographql.com/client/ to very easily do these queries.

Migration configuration specified in shepherd.yml should be overridable by command line options. This helps make migrations more reusable. For example, you could specify --adapter.search_query "org:nerdwallet filename:package.json react" to run a migration but with a different GitHub search query.

Notes:

• Some parts of shepherd.yml probably make less sense to be configurable by the command line, but for sake of flexibility and simplicity of implementation, we should probably just allow all keys to be overridable.
• Before/after "hooks" would probably be useful to run commands before or after the commands specified by the migration (e.g., --hooks.should_migrate.before "$(git show -s --format=%ct HEAD) <= $(date -d 'now - 100 days' +%s))".

In addition to local paths to a shepherd.yml migration file (e.g., shepherd apply ./path/to/migration), we could support migrations defined by npm packages (e.g., shepherd apply @nerdwallet/react-16-migration). Like npx, Shepherd would then fetch and install that packages on-demand as needed. This is useful for sharing migrations with the community. Shepherd could then be used without installing anything: npx shepherd apply @nerdwallet/react-16-migration.

Notes:

• You should be able to specify a specific version of a package like you can in npx (e.g., @nerdwallet/[email protected]).
• We should look at how npx installs packages to a temporary location and when it determines to update the package. Ideally we could call out to npx to do this.
• It might be useful to support migrations located in sub-directories (such as @nerdwallet/shepherd-migrations/react-16).

If you're search query contains multiple matches for a given repo, that repo attempts to be checked out multiple times.

[NerdWallet/credit-cards-client] 27/278
✔ Checked out repo
> Running should_migrate steps
$ grep -rl -m 1 '.' -e "cdn.nerdwallet.com" --exclude-dir=node_modules | head -1
./client/components/pre-auth-modal/pre-auth-modal.less
Step "grep -rl -m 1 '.' -e "cdn.nerdwallet.com" --exclude-dir=node_modules | head -1" exited with 0
$ git log -1 --format=%cd | grep 2018
Sat Sep 22 04:46:19 2018 +0000
Step "git log -1 --format=%cd | grep 2018" exited with 0
✔ Completed all should_migrate steps successfully
> Running post_checkout steps
✔ Completed all post_checkout steps successfully

[NerdWallet/credit-cards-client] 28/278
✔ Checked out repo
> Running should_migrate steps
$ grep -rl -m 1 '.' -e "cdn.nerdwallet.com" --exclude-dir=node_modules | head -1
./client/components/pre-auth-modal/pre-auth-modal.less
Step "grep -rl -m 1 '.' -e "cdn.nerdwallet.com" --exclude-dir=node_modules | head -1" exited with 0
$ git log -1 --format=%cd | grep 2018
Sat Sep 22 04:46:19 2018 +0000
Step "git log -1 --format=%cd | grep 2018" exited with 0
✔ Completed all should_migrate steps successfully
> Running post_checkout steps
✔ Completed all post_checkout steps successfully

Hi,

It seems that a certain type of rate-limit returned by github isn't handled with the looks like you've been rate-limited, trying again in 60s interface. Maybe github returns different kinds of rate-limit errors?

ballen:2019-02-jenkins-node-10-libraries ballen$ shepherd checkout .
{ HttpError: {
  "documentation_url": "https://developer.github.com/v3/#abuse-rate-limits",
  "message": "You have triggered an abuse detection mechanism. Please wait a few minutes before you try again."
}

    at response.text.then.message (/srv/nerdwallet/lib/node_modules/@nerdwallet/shepherd/node_modules/@octokit/rest/lib/request/request.js:72:19)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  name: 'HttpError',
  code: 403,
  status: undefined,
  headers:
   { 'access-control-allow-origin': '*',
     'access-control-expose-headers':
      'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type',
     connection: 'close',
     'content-encoding': 'gzip',
     'content-security-policy':
      'default-src \'none\'; base-uri \'self\'; block-all-mixed-content; connect-src \'self\' uploads.github.com www.githubstatus.com collector.githubapp.com api.github.com www.google-analytics.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com; font-src github.githubassets.com; form-action \'self\' github.com gist.github.com; frame-ancestors \'none\'; frame-src render.githubusercontent.com; img-src \'self\' data: github.githubassets.com media.githubusercontent.com camo.githubusercontent.com identicons.github.com collector.githubapp.com avatars0.githubusercontent.com avatars1.githubusercontent.com avatars2.githubusercontent.com avatars3.githubusercontent.com github-cloud.s3.amazonaws.com; manifest-src \'self\'; media-src \'none\'; script-src github.githubassets.com; style-src \'unsafe-inline\' github.githubassets.com',
     'content-type': 'application/json; charset=utf-8',
     date: 'Thu, 21 Feb 2019 00:45:25 GMT',
     'expect-ct':
      'max-age=2592000, report-uri="https://api.github.com/_private/browser/errors"',
     'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
     'retry-after': '60',
     server: 'GitHub.com',
     status: '403 Forbidden',
     'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
     'transfer-encoding': 'chunked',
     'x-content-type-options': 'nosniff',
     'x-frame-options': 'deny',
     'x-github-media-type': 'github.v3; format=json',
     'x-github-request-id': '497C:9603:E50DB0:115AB09:5C6DF4A5',
     'x-xss-protection': '1; mode=block' } }

From a suggestion @franjohn21 left on #14 re: the pr-status command:

suggest rerunning the checkout command so that the repo is removed from the list of repos for this migration

Introduction

In 2019 a decision was made to deprecate TSLint in favor of ESLint (see [1]).

To that end, Shepherd should target migrating from TSLint to ESLint in 2020.

Acceptance Criteria

• Deprecate TSLint in favor of ESLint
• Update README to account for this change

Notes

• Several migration strategies have been published online. For example, [2].

References

1. https://medium.com/palantir/tslint-in-2019-1a144c2317a9
2. https://medium.com/@KevinBGreene/migrating-from-tslint-to-eslint-6382f8bd3b6

the shepherd apply command seems to fail when running with the --repos flag

Context:

shepherd apply . --repos Nerdwallet/article-client

[Nerdwallet/article-client] 1/1
✔ Successfully reset repo
⠦ Resetting branchDEPRECATED (@octokit/rest): `pullRequests.getAll()` is deprecated, use `pullRequests.list()`
Error: fatal: ambiguous argument 'origin/undefined': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'

    at /usr/local/lib/node_modules/@nerdwallet/shepherd/node_modules/simple-git/promise.js:21:26
    at Git.<anonymous> (/usr/local/lib/node_modules/@nerdwallet/shepherd/node_modules/simple-git/src/git.js:458:18)
    at Function.Git.fail (/usr/local/lib/node_modules/@nerdwallet/shepherd/node_modules/simple-git/src/git.js:1478:18)
    at fail (/usr/local/lib/node_modules/@nerdwallet/shepherd/node_modules/simple-git/src/git.js:1436:20)
    at /usr/local/lib/node_modules/@nerdwallet/shepherd/node_modules/simple-git/src/git.js:1445:16
    at process._tickCallback (internal/process/next_tick.js:109:7)
✖ Failed to reset branch; not applying migration

From @parshap comments on #2:

I think this should be (optionally) configurable separately from the .shepherd data directory too. Also consider calling it config.yaml.
--data-directory that defaults to ~/.shepherd
--config-path that defaults to ~/.shepherd/config.yaml

Changes

• update octokit

Notes

• Github deprecation timeline: https://developer.github.com/changes/2020-02-10-deprecating-auth-through-query-param/
  • TL;DR brownouts will start March 17, 2021 and full deprecation targeted for May 5, 2021

Context

Hi! I'm on the React Native team at Facebook. We are working on a big reimplementation of the core framework which will result in a few different breaking changes in the API. Unfortunately, not all of these changes can be migrated with codemods. If they could, then shepherd would be an absolutely perfect tool for us to use to help migrate the community.

Instead, for some of these breaking changes, we are planning on writing guides and instructions for how to make these changes. Even those these migrations can't be automated, we'd like to help the community migrate as easily as possible.

We think Shepherd has a lot of the pieces to help us with this!

What we want to accomplish

What we would love to do is file issues on the repos that match the search criteria, linking to the lines of code that needs to change. To accomplish this we will still need to do the GitHub search, and check out the repos to find the specific callsites that will need to change (likely using an AST traversal).

A lot of the basics of what we want to do seem the same as what shepherd does today. The main difference is that instead of running a migration and opening a PR, we need to run a script to generate the issue text (the script would calculate the links to those lines of code) and post the issue.

I'm curious if you have any thoughts for us on if you think shepherd would be a reasonable tool for this, or if you'd recommend just building something separate?

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

• chore(deps): update amannn/action-semantic-pull-request action to v5.5.2
• chore(deps): update actions/cache action to v4
• chore(deps): update actions/setup-node action to v4
• chore(deps): lock file maintenance

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• fix(deps): update dependency joi to v17.13.1
• chore(deps): update dependency eslint to v9
• fix(deps): update octokit monorepo (major) (@octokit/core, @octokit/plugin-rest-endpoint-methods, @octokit/plugin-retry, @octokit/plugin-throttling, @octokit/types)
• Click on this checkbox to rebase all open PRs at once

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• fix(deps): update dependency chalk to v5
• fix(deps): update dependency commander to v12
• fix(deps): update dependency log-symbols to v6
• fix(deps): update dependency ora to v8

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository