python / cherry-picker Goto Github PK
View Code? Open in Web Editor NEWππβ Utility script for backporting/cherry-picking CPython changes from master into one of the maintenance branches.
License: Apache License 2.0
ππβ Utility script for backporting/cherry-picking CPython changes from master into one of the maintenance branches.
License: Apache License 2.0
For example, this backport could have been automated (the only conflict was on the AC hashes):
perhaps even a custom merge driver could be written for this?
cherry_picker
needs an option to specify the git remote
name for the CPython repo.
Currently it is hardcoded to "upstream" or "origin", and can't be used if a meaningful name like "python" is used
E.g.
~/repos/cpython$ git remote -v
python [email protected]:python/cpython.git (fetch)
python [email protected]:python/cpython.git (push)
mine [email protected]:markshannon/cpython.git (fetch)
mine [email protected]:markshannon/cpython.git (push)
Thanks to #262 (@webknjaz) I can publish to PyPI from Travis CI, all within GitHub web interface, and without going to the command line.
But I found myself still doing several manual steps:
__init__.py
)Can this be made even simpler? Perhaps I was doing unnecessary steps?
I think that creating the release branch, dropping ".devX", and creating tag can be fully automated somehow.
In #48, I attempted to cut a new release, but when I followed the instructions, nothing happened. As I was working to figure it out, I found other problems (like the travis-ci badge is a 404 and the release branch didn't run in CI).
Something gone wrong, so I decided to recreate the backport. But the second attempt failed. As well as all following. Here is the result of the last failure.:
$ cherry_picker --continue
π π β
Failed to push to origin βΉ
To github.com:serhiy-storchaka/cpython.git
! [rejected] backport-8f60298-3.11 -> backport-8f60298-3.11 (stale info)
error: Π½Π΅ Π²Π΄Π°Π»ΠΎΡΡ Π½Π°Π΄ΡΡΠ»Π°ΡΠΈ Π΄Π΅ΡΠΊΡ ΠΏΠΎΡΠΈΠ»Π°Π½Π½Ρ Π² Β«github.com:serhiy-storchaka/cpython.gitΒ»
branch backport-8f60298-3.11 has been deleted.
Backport PR:
...
I tried to remove the branch in my GitHub clone, but it did not help until I pulled from it into my local clone.
It seems, that when something gone wrong, cherry_picker
pushed the broken branch into my remote repository on GitHub and removed it from the local repository. In the following attempts it failed to push into the remote repository (either because it already existed, or because it was removed, but it did not know about this), but removed the branch from the local repository anyway.
I expect from it to use more forced method to push the branch into the remote repository (if it is already not used), and if the operation fails due to stale info, try to pull from the remote repository and repeat.
And of course it should not just remove the branch from the local repository. Instead it should keep it until you fix the error and continue with cherry_picker --continue
, or give up and cancel it with cherry_picker --abort
.
As @warsaw discovered the hard way on python/cpython#101024 , if pushing the branch to the selected (or default) remote fails for any reason (e.g. attempting to push to upstream
instead of origin
, as happened here), the branch is deleted anyway instead instead of issuing a clear error message and existing with it intact. This results in potentially loosing a large amount of hard, tedious work manually resolving backport conflicts (unless the user is a Git expert who knows how to recover it via git reflog
), and is a very frustrating and unfriendly user experience.
% cherry_picker --continue
git switchπ π β
Failed to push to origin βΉ
remote: error: GH006: Protected branch update failed for refs/heads/backport-49cae39-3.10.
remote: error: You're not authorized to push to this branch. Visit https://docs.github.com/articles/about-protected-branches/ for more information.
To github.com:python/cpython.git
! [remote rejected] backport-49cae39-3.10 -> backport-49cae39-3.10 (protected branch hook declined)
error: failed to push some refs to 'github.com:python/cpython.git'
branch backport-49cae39-3.10 has been deleted.
Backport PR:
[3.10] gh-101021: Document binary parameters as bytes (GH-101024).
(cherry picked from commit 49cae39ef020eaf242607bb2d2d193760b9855a6)
Co-authored-by: Bob Kline <[email protected]>
If pushing fails, the PUSHING_TO_REMOTE_FAILED state is set https://github.com/python/cherry-picker/blob/main/cherry_picker/cherry_picker.py#L407, but then push_to_remote just returns and the branch is deleted regardless of the state https://github.com/python/cherry-picker/blob/main/cherry_picker/cherry_picker.py#L521 . As far as I can tell, setting the PUSHING_TO_REMOTE_FAILED state has no effect and cherry-picker just continues and terminates normally regardless.
I'm not sure the best way to fix this within Cherry_Picker's error handling and UX design, but the most obvious solution is to just have it raise e.g. RuntimeError
and exit instead. There may be other situations where this happens as well, so it might be worth investigating any other known failure codepaths further.
As a sidenote I also discovered after much trial and error that you need to pass --no-auto-pr
and --pr-remote upstream
every time you call cherry picker --continue
to get it to work, instead of it being stored in .gitconfig
. This is very unintiuitve, and could also potentially lead to this error as well.
Also, calling --dry-run --continue
in the middle of a cherry pick to see what it would do next completely borks things, and requires wiping the config and starting over to recover.
I think with https://help.github.com/articles/checking-out-pull-requests-locally/ and https://gist.github.com/piscisaureus/3342247#gistcomment-1566295 cherry-picker could do better and even backport rebase merges and normal merge commits.
instead of Travis CI.
If, for whatever reason, the backport-...
branch already exists, cherry-picker ends up in a bad state:
Ξ» python -m cherry_picker e5ee6a87a3b6fbf1a7e3856248885811f04d1f89 3.4
π π β
Now backporting 'e5ee6a87a3b6fbf1a7e3856248885811f04d1f89' into '3.4'
Error checking out the branch backport-e5ee6a8-3.4.
fatal: A branch named 'backport-e5ee6a8-3.4' already exists.
Ξ» python -m cherry_picker e5ee6a87a3b6fbf1a7e3856248885811f04d1f89 3.4
π π β
Usage: __main__.py [OPTIONS] [COMMIT_SHA1] [BRANCHES]...
Try '__main__.py -h' for help.
Error: Run state cherry-picker.state=BACKPORT_LOOP_START in Git config is not known.
Perhaps it has been set by a newer version of cherry-picker. Try upgrading.
Valid states are: BACKPORT_PAUSED, UNSET. If this looks suspicious, raise an issue at https://github.com/python/core-workflow/issues/new.
As the last resort you can reset the runtime state stored in Git config using the following command: `git config --local --remove-section cherry-picker`
running cherry_picker 2.2.0 for a 3.12 backport, I get subprocess.CalledProcessError: Command '['git', 'log', '--format=%H', '3.12..']' returned non-zero exit status 128.
and nothing else.
I'm left to manually recreate that command line and run it myself to try and figure out what git's damage is. Running git I see fatal: ambiguous argument '3.12..': unknown revision or path not in the working tree.
which is better at least, but still not wholly informative (of course it isn't, git is [censored])...
(the real source of my error is likely what shows up when I try this:)
$ git checkout 3.12
error: pathspec '3.12' did not match any file(s) known to git
hint: '3.12' matched more than one remote tracking branch.
...
When git fails, we should at least let git speak for itself (regardless of utility) and not swallow the git error output.
The option to set the PR remote does work with cherry_picker --continue
, but currently needs to be specified explicitly, rather than being remembered from the original CLI invocation.
I know that this is a tool primarily for CPython but I found myself in a use case where I could benefit from cherry-picker's help but the branch name I want to backport to isn't version-like. To be specific, I want to use it with a branch named stable
because it allows me to replace ReadTheDocs's stable version without going through the release process:
https://docs.readthedocs.io/en/stable/versions.html#versioned-documentation
I can't seem to find a good reason for this limitation, the only thing the version in the branch appears to be used for is sorting - it wouldn't be too hard to primarily sort by version but fallback to lexicographic order if the branch doesn't contain a version. If the concern is that user passes the program something that is a commit rather than a branch name then we could enforce that it is in a different way (though you already do this indirectly when creating a backport branch using upstream/MAINTENANCE_BRANCH_NAME
as a base which doesn't work with commits).
I'd be happy to implement this if accepted by the way.
Usually backports are created directly from the main branch. But if there were significant changes in the previous version, it is simpler to create backports to older versions from this version. So you can get titles like "[3.11] [3.12] Fix something..."
This is redundant, only the leftmost prefix matters. It increases the length of the PR title (limited by GitHub UI) and the first line in the commit message (limited by the width of the screen or whatever). The way of creating backports is irrelevant, it is just what was more convenient for the author.
Cherry picker calls the webbrowser module from open_pr to open a web browser. I never want this. My terminal windows are most often not capable of opening a useful web browser and as such may resort to opening dumb terminal browsers like links or lynx that I refuse to tough github with. Just print the URL (as it already does) and leave it at that.
Some people may want a web browser. making it an option (I don't care what the default is) is fine by me.
Every time when I run cherry_picker
I get the following error:
π π β
Usage: cherry_picker [OPTIONS] [COMMIT_SHA1] [BRANCHES]...
Try "cherry_picker -h" for help.
Error: Run state cherry-picker.state=CHECKING_OUT_DEFAULT_BRANCH in Git config is not known.
Perhaps it has been set by a newer version of cherry-picker. Try upgrading.
Valid states are: BACKPORT_PAUSED, UNSET. If this looks suspicious, raise an issue at https://github.com/python/core-workflow/issues/new.
As the last resort you can reset the runtime state stored in Git config using the following command: `git config --local --remove-section cherry-picker`
cherry_picker
has been updated, it is 1.3.2.
Every time I need to run git config --local --remove-section cherry-picker
to make cherry_picker
working. This becomes annoying. Can it be fixed permanently?
At this point(where we have --pr-remote
, --[no-]push
, --upstream-remote
(#35), and --no-auto-pr
(#36) flags, it seems worth considering adding some way to configure user preferences on a global and perhaps local level.
The most obvious solution seems to be using git config
since it supports both global and local user configuration already, it is used by cherry-picker already, and therefore should be relatively straight-forward.
Small note here: I think it would be better to choose a different section than cherry-picker
as currently cherry-picker suggest to use git config --local --remove-section cherry-picker
as last resort for resetting cherry-picker's state.
Of course, there could instead be a single file at some configuration path (~/.config/cherry_picker/preferences.toml
?) for setting both global and local level preferences, i.e.:
[global]
upstream_remote = "the_real_deal"
pr_remote = "mine"
no_auto_pr = true
# I'm not entirely sold on the idea of using paths, but it would be the simplest to implement for sure
[local."/home/jack/CPython"]
upstream_remote = "python"
but I don't know if there are any upsides to this over just using git config
This is happening for me everytime I do manual backports:
(.venv) ~/Desktop/cpython2 main β
Β» cherry_picker a1c4923d65a3e4cea917745e7f6bc2e377cde5c5 3.12
π π β
Now backporting 'a1c4923d65a3e4cea917745e7f6bc2e377cde5c5' into '3.12'
Error cherry-pick a1c4923d65a3e4cea917745e7f6bc2e377cde5c5.
Auto-merging Lib/test/test_cmd_line.py
CONFLICT (content): Merge conflict in Lib/test/test_cmd_line.py
error: could not apply a1c4923d65a... gh-116858: Add `@cpython_only` to several tests in `test_cmd_line` (#116859)
hint: After resolving the conflicts, mark them with
hint: "git add/rm <pathspec>", then run
hint: "git cherry-pick --continue".
hint: You can instead skip this commit with "git cherry-pick --skip".
hint: To abort and get back to the state before "git cherry-pick",
hint: run "git cherry-pick --abort".
Recorded preimage for 'Lib/test/test_cmd_line.py'
Failed to cherry-pick a1c4923d65a3e4cea917745e7f6bc2e377cde5c5 into 3.12 βΉ
... Stopping here.
To continue and resolve the conflict:
$ cherry_picker --status # to find out which files need attention
# Fix the conflict
$ cherry_picker --status # should now say 'all conflict fixed'
$ cherry_picker --continue
To abort the cherry-pick and cleanup:
$ cherry_picker --abort
(.venv) ~/Desktop/cpython2 backport-a1c4923-3.12 β Β§ 255 β οΈ
Β» git add --all
(.venv) ~/Desktop/cpython2 backport-a1c4923-3.12 β β
Β» cherry_picker --continue --pr-remote=fork
π π β
Traceback (most recent call last):
File "/Users/sobolev/Desktop/cpython2/.venv/bin/cherry_picker", line 8, in <module>
sys.exit(cherry_pick_cli())
^^^^^^^^^^^^^^^^^
File "/Users/sobolev/Desktop/cpython2/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/sobolev/Desktop/cpython2/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/Users/sobolev/Desktop/cpython2/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/sobolev/Desktop/cpython2/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/sobolev/Desktop/cpython2/.venv/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/sobolev/Desktop/cpython2/.venv/lib/python3.11/site-packages/cherry_picker/cherry_picker.py", line 802, in cherry_pick_cli
cherry_picker.continue_cherry_pick()
File "/Users/sobolev/Desktop/cpython2/.venv/lib/python3.11/site-packages/cherry_picker/cherry_picker.py", line 591, in continue_cherry_pick
commits = get_commits_from_backport_branch(base)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/sobolev/Desktop/cpython2/.venv/lib/python3.11/site-packages/cherry_picker/cherry_picker.py", line 902, in get_commits_from_backport_branch
output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/sobolev/.pyenv/versions/3.11.5/lib/python3.11/subprocess.py", line 466, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/sobolev/.pyenv/versions/3.11.5/lib/python3.11/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'log', '--format=%H', '3.12..']' returned non-zero exit status 128.
fork
is my own fork: https://github.com/sobolevn/cpython/ which I use to send PRs to the main repo.
Things I've tried:
.git/config
filecherry-picker
I mostly use 3.11.5
as my venv tooling.
Looking through existing issues - no matches :(
$ python -m cherry_picker e6d1aa1ac65b6908fdea2c70ec3aa8c4f1dffcb5 3.10 3.9
Now backporting 'e6d1aa1ac65b6908fdea2c70ec3aa8c4f1dffcb5' into '3.10'
Switched to a new branch 'backport-e6d1aa1-3.10'
Branch 'backport-e6d1aa1-3.10' set up to track remote branch '3.10' from 'upstream'.
Error cherry-pick e6d1aa1ac65b6908fdea2c70ec3aa8c4f1dffcb5.
Auto-merging Lib/test/test_contextlib_async.py
CONFLICT (content): Merge conflict in Lib/test/test_contextlib_async.py
...
[resolve conflict]
$ python -m cherry_picker --continue
π π β
remote:
remote: Create a pull request for 'backport-e6d1aa1-3.10' on GitHub by visiting:
remote: https://github.com/belm0/cpython/pull/new/backport-e6d1aa1-3.10
remote:
To github.com:belm0/cpython.git
* [new branch] backport-e6d1aa1-3.10 -> backport-e6d1aa1-3.10
Backport PR URL:
https://github.com/python/cpython/compare/3.10...belm0:backport-e6d1aa1-3.10?expand=1
Switched to branch 'main'
Deleted branch backport-e6d1aa1-3.10 (was 91814316ed).
branch backport-e6d1aa1-3.10 has been deleted.
(cherry_picker exited with success, but backport to branch 3.9 was not processed)
So that in fs the path to the package would be cherry-picker/cherry_picker
and not cherry_picker/cherry_picker
which is less confusing + it's a good practice to have a repo name not being python importable which prevents some other confusions when you're in the wrong dir.
Bandit 1.7.5 adds B113:request_without_timeout and the following issue was found in this workflow run:
>> Issue: [B113:request_without_timeout] Requests call without timeout
Severity: Medium Confidence: Low
CWE: CWE-400 (https://cwe.mitre.org/data/definitions/400.html)
More Info: https://bandit.readthedocs.io/en/1.7.5/plugins/b113_request_without_timeout.html
Location: ./cherry_picker/cherry_picker.py:443:19
442 url = CREATE_PR_URL_TEMPLATE.format(config=self.config)
443 response = requests.post(url, headers=request_headers, json=data)
444 if response.status_code == requests.codes.created:
installed package cherry-picker 2.0.0, Python 3.9.7
These apps are now globally available
- cherry_picker
done! β¨ π β¨
% cherry_picker 8e8f7522171ef82f2f5049940f815e00e38c6f42 3.9
Traceback (most recent call last):
File "/Users/user12345/.local/bin/cherry_picker", line 5, in <module>
from cherry_picker import cherry_pick_cli
ImportError: cannot import name 'cherry_pick_cli' from 'cherry_picker'
As in the title:
Ξ» python -m cherry_picker --no-push b8fcac989f82e678695bb3024881c01d86bc00b0 3.4
π π β
Now backporting 'b8fcac989f82e678695bb3024881c01d86bc00b0' into '3.4'
Branch 'backport-b8fcac9-3.4' set up to track remote branch '3.4' from 'upstream'.
Switched to a new branch 'backport-b8fcac9-3.4'
[backport-b8fcac9-3.4 b1877be9f] Create sss (#30)
Date: Mon Aug 2 18:47:51 2021 +0200
1 file changed, 1 insertion(+)
create mode 100644 sss
Finished cherry-pick b8fcac989f82e678695bb3024881c01d86bc00b0 into backport-b8fcac9-3.4 π
--no-push option used.
... Stopping here.
To continue and push the changes:
$ cherry_picker --continue
To abort the cherry-pick and cleanup:
$ cherry_picker --abort
Ξ» python -m cherry_picker --continue
π π β
remote:
remote: Create a pull request for 'backport-b8fcac9-3.4' on GitHub by visiting:
remote: https://github.com/jack1142/Red-DiscordBot/pull/new/backport-b8fcac9-3.4
remote:
To https://github.com/jack1142/Red-DiscordBot
* [new branch] backport-b8fcac9-3.4 -> backport-b8fcac9-3.4
Backport PR URL:
https://github.com/Cog-Creators/Red-DiscordBot/compare/3.4...jack1142:backport-b8fcac9-3.4?expand=1
Your branch is up to date with 'origin/V3/develop'.
Switched to branch 'V3/develop'
Deleted branch backport-b8fcac9-3.4 (was b01d61ef4).
branch backport-b8fcac9-3.4 has been deleted.
Backport PR:
[3.4] Create sss (#30).
(cherry picked from commit b8fcac989f82e678695bb3024881c01d86bc00b0)
Co-authored-by: jack1142 <[email protected]>
This also results in broken formatting on the PR creation page:
#26 (and maybe others) have been fixed for some time, but continue to cause toil because the fix isn't visible to users. Can we get a release cut?
I see from the readme there's some release automation. As a core committer, can I cut a release?
If after resolving conflict, I use cherry_picker --no-push --continue
, cherry_picker
will push to remote (and open webbrowser/make PR through API) regardless.
Maybe I'm doing something wrong, but when I have used cherry_picker --continue
I inevitably lose the title and description I would have ended up with had there not been any merge conflict. Not sure if there's any way to prevent that or to output what the message would have been so I can copy-and-paste it into the eventual PR?
toml
was last released on 02.11.2020: https://pypi.org/project/toml/#history
There are multiple known bugs: https://github.com/uiri/toml/issues
Maintainer is not supporting their project: uiri/toml#361
Multiple big projects like mypy
, black
, typeshed
have already switched.
The transition seems to be rather simple: basically swap import toml
with import tomli
and use a proper encoding
when calling loads
.
Related: https://www.python.org/dev/peps/pep-0680/
If this is something you think should be done, I can send a PR.
Currently cherry_picker simply replaces any #
with GH-
in commit messages. It can produce errors like in python/cpython#114879:
* fix missing 'GH-' format support
When trying to backport this PR, miss-islington failed and advised running cherry_picker
from the CLI. When running
cherry_picker 067597522a9002f3b8aff7f46033f10acb2381e4 3.11
It failed with this traceback:
~/projects/cpython/3.11 [3.11|β] $ Doc/venv/bin/cherry_picker 067597522a9002f3b8aff7f46033f10acb2381e4 3.11
π π β
Now backporting '067597522a9002f3b8aff7f46033f10acb2381e4' into '3.11'
[backport-0675975-3.11 5beb16b77f] gh-92897: Ensure `venv --copies` respects source build property of the creating interpreter (GH-92899)
Author: Jeremy Kloth <[email protected]>
Date: Tue Jul 5 10:08:20 2022 -0500
5 files changed, 76 insertions(+), 40 deletions(-)
Backport PR URL:
https://github.com/python/cpython/compare/3.11...vsajip:backport-0675975-3.11?expand=1
Traceback (most recent call last):
File "/disk2/vinay/projects/cpython/3.11/Doc/venv/bin/cherry_picker", line 8, in <module>
sys.exit(cherry_pick_cli())
^^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.11/Doc/venv/lib/python3.12/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.11/Doc/venv/lib/python3.12/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.11/Doc/venv/lib/python3.12/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.11/Doc/venv/lib/python3.12/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.11/Doc/venv/lib/python3.12/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.11/Doc/venv/lib/python3.12/site-packages/cherry_picker/cherry_picker.py", line 645, in cherry_pick_cli
cherry_picker.backport()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.11/Doc/venv/lib/python3.12/site-packages/cherry_picker/cherry_picker.py", line 400, in backport
self.cleanup_branch(cherry_pick_branch)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.11/Doc/venv/lib/python3.12/site-packages/cherry_picker/cherry_picker.py", line 361, in cleanup_branch
self.checkout_default_branch()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.11/Doc/venv/lib/python3.12/site-packages/cherry_picker/cherry_picker.py", line 226, in checkout_default_branch
self.run_cmd(cmd)
^^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.11/Doc/venv/lib/python3.12/site-packages/cherry_picker/cherry_picker.py", line 181, in run_cmd
output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vinay/.local/lib/python3.12/subprocess.py", line 454, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vinay/.local/lib/python3.12/subprocess.py", line 558, in run
raise CalledProcessError(retcode, process.args,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
subprocess.CalledProcessError: Command '('git', 'checkout', 'main')' returned non-zero exit status 128.
Now I was in the 3.11 branch so I'm not sure it should be doing a git checkout main
. But anyway, the PR was created (in that it opened the "Create pull request" page in the browser, which I clicked to confirm - the PR is here. But if I now try to do the same thing for 3.10, I get
~/projects/cpython/3.10 [3.10|β] $ Doc/venv/bin/cherry_picker 067597522a9002f3b8aff7f46033f10acb2381e4 3.10
π π β
Usage: cherry_picker [OPTIONS] [COMMIT_SHA1] [BRANCHES]...
Try 'cherry_picker -h' for help.
Error: Run state cherry-picker.state=CHECKING_OUT_DEFAULT_BRANCH in Git config is not known.
Perhaps it has been set by a newer version of cherry-picker. Try upgrading.
Valid states are: BACKPORT_PAUSED, UNSET. If this looks suspicious, raise an issue at https://github.com/python/cherry-picker/issues/new.
As the last resort you can reset the runtime state stored in Git config using the following command: `git config --local --remove-section cherry-picker`
I did git config --local --remove-section cherry-picker
and tried again, but now I get
~/projects/cpython/3.10 [3.10|β] $ git config --local --remove-section cherry-picker
~/projects/cpython/3.10 [3.10|β] $ Doc/venv/bin/cherry_picker 067597522a9002f3b8aff7f46033f10acb2381e4 3.10
π π β
Traceback (most recent call last):
File "/disk2/vinay/projects/cpython/3.10/Doc/venv/bin/cherry_picker", line 8, in <module>
sys.exit(cherry_pick_cli())
^^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.10/Doc/venv/lib/python3.12/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.10/Doc/venv/lib/python3.12/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.10/Doc/venv/lib/python3.12/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.10/Doc/venv/lib/python3.12/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.10/Doc/venv/lib/python3.12/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.10/Doc/venv/lib/python3.12/site-packages/cherry_picker/cherry_picker.py", line 645, in cherry_pick_cli
cherry_picker.backport()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.10/Doc/venv/lib/python3.12/site-packages/cherry_picker/cherry_picker.py", line 375, in backport
self.fetch_upstream()
^^^^^^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.10/Doc/venv/lib/python3.12/site-packages/cherry_picker/cherry_picker.py", line 173, in fetch_upstream
self.run_cmd(cmd)
^^^^^^^^^^^^^^^^^
File "/disk2/vinay/projects/cpython/3.10/Doc/venv/lib/python3.12/site-packages/cherry_picker/cherry_picker.py", line 181, in run_cmd
output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vinay/.local/lib/python3.12/subprocess.py", line 454, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vinay/.local/lib/python3.12/subprocess.py", line 558, in run
raise CalledProcessError(retcode, process.args,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
subprocess.CalledProcessError: Command '['git', 'fetch', 'upstream', '--no-tags']' returned non-zero exit status 1.
Hi there,
On https://github.com/python/cherry-picker#demo The installation demo video is broken: 404 Not Found. Maybe we can update/delete the link.
After running cherry_picker (2.0, latest release), I get this output ending with a traceback:
$ cherry_picker --continue
π π β
remote:
remote: Create a pull request for 'backport-23acadc-3.9' on GitHub by visiting:
remote: https://github.com/python/cpython/pull/new/backport-23acadc-3.9
remote:
To https://github.com/python/cpython
* [new branch] backport-23acadc-3.9 -> backport-23acadc-3.9
Backport PR URL:
https://github.com/python/cpython/compare/3.9...python:backport-23acadc-3.9?expand=1
Traceback (most recent call last):
File "/Users/jaraco/.local/bin/cherry_picker", line 8, in <module>
sys.exit(cherry_pick_cli())
File "/Users/jaraco/.local/pipx/venvs/cherry_picker/lib/python3.9/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/Users/jaraco/.local/pipx/venvs/cherry_picker/lib/python3.9/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/Users/jaraco/.local/pipx/venvs/cherry_picker/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/jaraco/.local/pipx/venvs/cherry_picker/lib/python3.9/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/Users/jaraco/.local/pipx/venvs/cherry_picker/lib/python3.9/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Users/jaraco/.local/pipx/venvs/cherry_picker/lib/python3.9/site-packages/cherry_picker/cherry_picker.py", line 615, in cherry_pick_cli
cherry_picker.continue_cherry_pick()
File "/Users/jaraco/.local/pipx/venvs/cherry_picker/lib/python3.9/site-packages/cherry_picker/cherry_picker.py", line 473, in continue_cherry_pick
self.cleanup_branch(cherry_pick_branch)
File "/Users/jaraco/.local/pipx/venvs/cherry_picker/lib/python3.9/site-packages/cherry_picker/cherry_picker.py", line 349, in cleanup_branch
self.checkout_default_branch()
File "/Users/jaraco/.local/pipx/venvs/cherry_picker/lib/python3.9/site-packages/cherry_picker/cherry_picker.py", line 221, in checkout_default_branch
self.run_cmd(cmd)
File "/Users/jaraco/.local/pipx/venvs/cherry_picker/lib/python3.9/site-packages/cherry_picker/cherry_picker.py", line 180, in run_cmd
output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 424, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '('git', 'checkout', 'master')' returned non-zero exit status 1.
I suspect the issue is that master
is no longer a branch in the repo and it should be main
.
I ran "py -3 -m pip install cherry_picker" and all went well.
In my 3.7 worktree, I ran "py -3 -m cherry_picker f3e8209 3.6" and with one more click, it produced python/cpython#2321. Great.
The only problem, which I mentioned before on another issue, is that on Windows, checking out 3.6 to prepare the PR irreversibly touches most .c files, so that a subsequent rebuild recompiles most C files. This time I made sure that cherry_picker is the culprit by running a rebuild immediately before and after cherry_picker. Starting in my 3.6 worktree would be worse because cherry_picker would checkout master and cause the same rebuild issue, plus leave the wrong branch checked out.
Request and suggestion. Run git status and capture the first line of the response: "On branch xyz". If is 'master', '3.6, '3.5', or '2.7', store it and execute f'git checkout {xyz}' instead of hard-coding master as the return checkout. Or just store xyz whatever it is and check it back out, so cherry_picker restores the status quo whatever it was.
I get the following error when tried to backport python/cpython#31938.
$ cherry_picker dbbe4d2d0075fa0e95b069fb4780d79aae3514c7 3.9
π π β
Now backporting 'dbbe4d2d0075fa0e95b069fb4780d79aae3514c7' into '3.9'
Error cherry-pick dbbe4d2d0075fa0e95b069fb4780d79aae3514c7.
The previous cherry-pick is now empty, possibly due to conflict resolution.
If you wish to commit it anyway, use:
git commit --allow-empty
Otherwise, please use 'git cherry-pick --skip'
On branch backport-dbbe4d2-3.9
Your branch is up to date with 'upstream/3.9'.
You are currently cherry-picking commit dbbe4d2d00.
(all conflicts fixed: run "git cherry-pick --continue")
(use "git cherry-pick --skip" to skip this patch)
(use "git cherry-pick --abort" to cancel the cherry-pick operation)
Untracked files:
(use "git add <file>..." to include in what will be committed)
1.diff
bpo-42218.py
test.c
test.err
test.out
test.tests
test2.c
test_multiprocessing_fork.tests
test_multiprocessing_fork2.tests
nothing added to commit but untracked files present (use "git add" to track)
Failed to cherry-pick dbbe4d2d0075fa0e95b069fb4780d79aae3514c7 into 3.9 βΉ
... Stopping here.
To continue and resolve the conflict:
$ cherry_picker --status # to find out which files need attention
# Fix the conflict
$ cherry_picker --status # should now say 'all conflict fixed'
$ cherry_picker --continue
To abort the cherry-pick and cleanup:
$ cherry_picker --abort
I have just upgraded cherry_picker
from a very old version, so it may be related. I still did not try to use cherry_picker
with other PRs.
python/cpython/gpshead (main)$ cherry_picker a6c1c04d4d2339f0094422974ae3f26f8c7c8565 3.12 --upstream-remote=upstream --pr-remote=origin
π π β
You're not inside a cpython repo right now! π
python/cpython/gpshead (main)$ cherry_picker --version
cherry_picker, version 2.2.0
It appears that some validation logic was added that doesn't take actual git clone configurations into account.
That logic appears to have been there "forever" (5 years per Blame) but there was a recent change a month-ish ago to the b471da1 that added a get_state_and_verify()
call to check_repo()
. Related?
The other issue aside from the bug is that the error message is inaccurate and thus unhelpful - It does not provide full details on exactly what it thinks is amiss in the state is detected vs exactly what state expects so I can't tell anything about the nature of the bug or if there's something simple I could do given I am within a git clone of cpython with many remote repos and branches that I've been using for years.
Getting overly creative with pdb, the error message that was suppressed was apparently:
ValueError: Run state cherry-picker.state=BACKPORT_LOOP_START in Git config is not known.
Perhaps it has been set by a newer version of cherry-picker. Try upgrading.
Valid states are: BACKPORT_PAUSED, UNSET. If this looks suspicious, raise an issue at https://github.com/python/cherry-picker/issues/new.
As the last resort you can reset the runtime state stored in Git config using the following command: `git config --local --remove-section cherry-picker`
The readme for cherry-picker is getting long. Perhaps we can have documentation hosted in readthedocs or something similar.
Hi there, thanks for making (and maintaining) this cool project!
I would want to ask the maintainers of this project to consider reviewing at least some of the open PRs and cutting a new release. About a year ago I wanted to help out with cleaning out the issue tracker so I've made a bunch of PRs (some of which have already been merged but not released: #58, #61, #40) with fixes and enhancements. There has been no new release since November, and I feel like there's already a decent chunk of unreleased changes (other than my own PRs, there's also #63 adding support for --mirror
) that could be made available to everyone using cherry-picker.
Here's the list of not-yet merged PRs that pass tests and aren't a draft:
I personally think that the single most important PR here is #46 because the lack of attribution just plain sucks when it's really the only thing that as a contributor to an open source project you can expect.
#35 fixes an issue of a CPython's core developer so that would probably be good to have but there are some open questions in the PR for it which may not make it possible to review at the current point in time.
Other than that, I think that #43 probably makes the most important fixes as it makes cherry_picker easier to use when you have to resolve conflicts which is probably the only time when CPython's core developers have to actually use cherry_picker manually so making that process better is IMO helpful.
Personally, I have a stake in #70 but I don't imagine it being useful to a lot of people, and definitely not useful for CPython workflow so that's just on my wishlist π
I hope this summary is helpful.
I'm not trying to demand anything here, it is a voluntarily-maintained project. All I want is for you to consider my ask, nothing more :)
Simply setting PR number under self.pr_number
after successfully creating PR would be enough:
cherry-picker/cherry_picker/cherry_picker.py
Line 334 in 6b302ad
In addition to allowing to refer to the PR after the backport, it also allows checking if the PR was actually created successfully which right now is not possible either (unless you catch stdout but that's quite bothersome in comparison to simple API modification in this project).
After 2021-05-03, CPython's default branch will become main
.
Since cherry-picker was designed for CPython initially, it would be great to use the main
branch when backporting.
Others can configure the branch name using the default_branch
config option, documented in https://github.com/python/cherry-picker#options
This happened to me on the freshly upgraded openSUSE/Tumbleweed:
tumbleweed-pkg~/r/cpython (3.9)$ python3 -mpip install -U cherry_picker@git+https://github.com/python/cherry-picker.git
Defaulting to user installation because normal site-packages is not writeable
Collecting cherry_picker@ git+https://github.com/python/cherry-picker.git
Cloning https://github.com/python/cherry-picker.git to /tmp/pip-install-jzzbsfql/cherry-picker_cfe6f82ef02c43ad8cc54ccd332ba7e4
Running command git clone --filter=blob:none --quiet https://github.com/python/cherry-picker.git /tmp/pip-install-jzzbsfql/cherry-picker_cfe6f82ef02c43ad8cc54ccd332ba7e4
Resolved https://github.com/python/cherry-picker.git to commit 3e48d193d8ae19001883a25ccb825c39a7f36522
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: click>=6.0 in /usr/lib/python3.11/site-packages (from cherry_picker@ git+https://github.com/python/cherry-picker.git) (8.1.3)
Requirement already satisfied: gidgethub in /home/matej/.local/lib/python3.11/site-packages (from cherry_picker@ git+https://github.com/python/cherry-picker.git) (5.3.0)
Requirement already satisfied: requests in /usr/lib/python3.11/site-packages (from cherry_picker@ git+https://github.com/python/cherry-picker.git) (2.31.0)
Requirement already satisfied: uritemplate>=3.0.1 in /usr/lib/python3.11/site-packages (from gidgethub->cherry_picker@ git+https://github.com/python/cherry-picker.git) (4.1.1)
Requirement already satisfied: PyJWT[crypto]>=2.4.0 in /usr/lib/python3.11/site-packages (from gidgethub->cherry_picker@ git+https://github.com/python/cherry-picker.git) (2.7.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3.11/site-packages (from requests->cherry_picker@ git+https://github.com/python/cherry-picker.git) (2023.5.7)
Requirement already satisfied: charset_normalizer<4,>=2 in /usr/lib/python3.11/site-packages (from requests->cherry_picker@ git+https://github.com/python/cherry-picker.git) (3.1.0)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.11/site-packages (from requests->cherry_picker@ git+https://github.com/python/cherry-picker.git) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/lib/python3.11/site-packages (from requests->cherry_picker@ git+https://github.com/python/cherry-picker.git) (2.0.3)
Requirement already satisfied: cryptography>=3.4.0 in /usr/lib64/python3.11/site-packages (from PyJWT[crypto]>=2.4.0->gidgethub->cherry_picker@ git+https://github.com/python/cherry-picker.git) (41.0.1)
Requirement already satisfied: cffi>=1.12 in /usr/lib64/python3.11/site-packages (from cryptography>=3.4.0->PyJWT[crypto]>=2.4.0->gidgethub->cherry_picker@ git+https://github.com/python/cherry-picker.git) (1.15.1)
Requirement already satisfied: pycparser in /usr/lib/python3.11/site-packages (from cffi>=1.12->cryptography>=3.4.0->PyJWT[crypto]>=2.4.0->gidgethub->cherry_picker@ git+https://github.com/python/cherry-picker.git) (2.21)
tumbleweed-pkg~/r/cpython (3.9)$ cherry_picker --dry-run 18dfbd035775c15533d13a98e56b1d2bf5c65f00 3.9 3.10 3.11
π π β
Dry run requested, listing expected command sequence
dry-run: git remote get-url upstream
dry-run: git fetch upstream --no-tags
Now backporting '18dfbd035775c15533d13a98e56b1d2bf5c65f00' into '3.11'
dry-run: git remote get-url upstream
dry-run: git checkout -b backport-18dfbd0-3.11 upstream/3.11
dry-run: git cherry-pick -x 18dfbd035775c15533d13a98e56b1d2bf5c65f00
dry-run: git show -s --format=%B 18dfbd035775c15533d13a98e56b1d2bf5c65f00
Traceback (most recent call last):
File "/home/matej/.bin/cherry_picker", line 8, in <module>
sys.exit(cherry_pick_cli())
^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/matej/.local/lib/python3.11/site-packages/cherry_picker/cherry_picker.py", line 645, in cherry_pick_cli
cherry_picker.backport()
File "/home/matej/.local/lib/python3.11/site-packages/cherry_picker/cherry_picker.py", line 387, in backport
commit_message = self.amend_commit_message(cherry_pick_branch)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/matej/.local/lib/python3.11/site-packages/cherry_picker/cherry_picker.py", line 269, in amend_commit_message
updated_commit_message = f"""{commit_prefix}{self.get_commit_message(self.commit_sha1)}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/matej/.local/lib/python3.11/site-packages/cherry_picker/cherry_picker.py", line 211, in get_commit_message
message = self.run_cmd(cmd).strip()
^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'strip'
I just installed cherry_picker 2.0.0 in a conda environment and I get the following error when trying to run it:
$ cherry_picker --help
Traceback (most recent call last):
File "/home/antoine/miniconda3/envs/cherry_picker/bin/cherry_picker", line 5, in <module>
from cherry_picker import cherry_pick_cli
ImportError: cannot import name 'cherry_pick_cli' from 'cherry_picker' (/home/antoine/miniconda3/envs/cherry_picker/lib/python3.9/site-packages/cherry_picker/__init__.py)
Installed packages:
$ pip list
Package Version
------------- -------------------
certifi 2020.12.5
cffi 1.14.5
chardet 4.0.0
cherry-picker 2.0.0
click 8.0.0
cryptography 3.4.7
gidgethub 5.0.1
idna 2.10
pip 21.0.1
pycparser 2.20
PyJWT 2.1.0
requests 2.25.1
setuptools 52.0.0.post20210125
toml 0.10.2
uritemplate 3.0.1
urllib3 1.26.4
wheel 0.36.2
https://github.com/python/cherry-picker/actions/runs/6481613542/job/17599733638
I ended up publishing using flit publish
manually.
As people have learned I am not always the most observant person in the world. My initial attempts to use cherry-picker
were complicated by the fact that I paid attention to messages from git cherry-pick
, not cherry-picker
's own messages. It would be nice if the message sources were better distinguished, either by dimming the messages from git cherry-pick
or by coloring the messages from cherry-picker
.
If you think something like this would be useful, I could work on a PR. It seems that any desired changes would be straightforward, as current console output already uses click.echo()
.
I have my remote origin set to https://github.com/AA-Turner/cpython/
.
PS> git remote -v
origin https://github.com/AA-Turner/cpython/ (fetch)
origin https://github.com/AA-Turner/cpython/ (push)
upstream https://github.com/python/cpython/ (fetch)
upstream https://github.com/python/cpython/ (push)
cherry_picker
runs git config --get remote.origin.url
, which returns the above url, and then runs username = result.replace(":", "/").split("/")[-2]
. This returns cpython
, which means that I have to manually edit the URL on every backport.
cherry_picker
should instead trim any trailing slash.
A
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.