followtheprocess / pytoil Goto Github PK

How would this feature be useful?

When using pytoil show either local or remote, there is no distinction between repo's that are natively
"yours" and ones which are forks of other peoples repos.

Sometimes it's useful to make this distinction.

Describe the solution you'd like

For there to be a pytoil show forks command or similar (perhaps pytoil show remote --forks or pytoil show local --forks. Or perhaps just to highlight the forks a different colour or something.

Describe alternatives you've considered

No real alternatives

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

If you use pytoil to create a new project from a cookiecutter and that cookiecutter modifies the name of the project (e.g. project name to project slug), pytoil will open the wrong thing in VSCode on completion.

I got this to happen when creating a project called async-pytoil with pytoil new async-pytoil --cookie <poetry pypackage>. Because the cookiecutter would have changed async-pytoil to async_pytoil VSCode opened up a random file instead.

The rest of the process was fine, just the call to VSCode.

This might be tricky to fix properly as the replaced project name is only available after the cookiecutter has finished rendering.

Expected Behavior

Should have opened up the newly created async_pytoil folder not a random file called async-pytoil

Steps To Reproduce

pytoil new <something with hyphen> --cookie <template that replaces hyphens with underscores>

Environment

- OS: MacOS
- Python: 3.10
- pytoil: 0.15.1

Anything else?

No response

Is there an existing issue for this?

• I have searched the existing issues

How would this feature be useful?

Currently pytoil only has native support for opening projects in VSCode, however with click you can call click.edit which looks at the value of $EDITOR, meaning a user could use pycharm, vim, sublime, whatever they like and it should still all work nicely!

Describe the solution you'd like

Have a config key editor which is the same idea as token in that if it's populated that editor will be used, if not pytoil will look for $EDITOR, and finally fail telling the user to specify an editor in one of the above ways.

Describe alternatives you've considered

Either manually implementing click.edit which seems like a waste of time when it's right there for free, or just only supporting VSCode.

Anything else?

No response

How would this feature be useful?

Often something I find myself doing is going through all my local projects and doing git pull to pull down any changes I may have made on GitHub or on another machine.

It would be good if pytoil could automate this workflow!

Describe the solution you'd like

Some command like pytoil sync that took a list of repos as an argument and had a pytoil sync --all option to do all of them which looped through every project I had locally and did the following:

• Check if I'm on the default branch (if not, skip this project completely to be safe)
• Check if there are uncommitted changes (again, if not, skip completely)
• If both of the above are handled, do git pull to synchronise changes from remotes.

Describe alternatives you've considered

Doing it manually, or a separate script. I'd rather have it as part of pytoil

Describe the bug

When checking out your fork from GitHub, you get the following Pydantic validation error:

Traceback (most recent call last):
  File "/Users/tomfleet/.local/bin/pytoil", line 8, in <module>
    sys.exit(app())
  File "/Users/tomfleet/.local/pipx/venvs/pytoil/lib/python3.9/site-packages/typer/main.py", line 214, in __call__
    return get_command(self)(*args, **kwargs)
  File "/Users/tomfleet/.local/pipx/venvs/pytoil/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/tomfleet/.local/pipx/venvs/pytoil/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/tomfleet/.local/pipx/venvs/pytoil/lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/tomfleet/.local/pipx/venvs/pytoil/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/tomfleet/.local/pipx/venvs/pytoil/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/tomfleet/.local/pipx/venvs/pytoil/lib/python3.9/site-packages/typer/main.py", line 497, in wrapper
    return callback(**use_params)  # type: ignore
  File "/Users/tomfleet/.local/pipx/venvs/pytoil/lib/python3.9/site-packages/pytoil/cli/checkout.py", line 130, in checkout
    elif repo.exists_remote(api=api):
  File "/Users/tomfleet/.local/pipx/venvs/pytoil/lib/python3.9/site-packages/pytoil/repo/repo.py", line 78, in exists_remote
    _ = api.get_repo(self.name)
  File "/Users/tomfleet/.local/pipx/venvs/pytoil/lib/python3.9/site-packages/pytoil/api/api.py", line 137, in get_repo
    return Repository(**response)  # type: ignore
  File "pydantic/main.py", line 406, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 2 validation errors for Repository
parent -> permissions
  field required (type=value_error.missing)
source -> permissions
  field required (type=value_error.missing)

To Reproduce

Checkout a fork you own from GitHub.

Expected behavior

Should clone, then open the project in VSCode

Screenshots

If applicable, add screenshots to help explain your problem.

Environment

pytoil --version

Additional context

This is due to a field on the Pydantic model Repository being marked as required when it is optional on GitHub's API

The solution you would like

When we checkout a remote repo with the --venv flag telling pytoil to create the appropriate virtual environment, it should look for certain requirements files (e.g. requirements.txt, pyproject.toml, setup.cfg) and have logic to determine whether it is setuptools, poetry, flit etc etc. and automatically install the project and it's requirements.

Process flow would be something like:

# pseudocode

if "requirements_dev.txt":
    # prefer dev because it will have everything
    "pip install -r requirements_dev.txt"
elif "requirements.txt":
    "pip install -r requirements.txt"
elif "pyproject_toml":
    if setuptools:
        "pip install -e .[dev]"
    elif poetry:
        "poetry install"
    elif flit:
        "flit install --deps develop --symlink --python .venv/bin/python"
    else:
        error()
elif conda:
    # only way of detecting conda is to look for environment.yml
    "conda env create --file environment.yml"
else:
    error()

Obviously this makes a few assumptions but will work in majority of cases, if user wants specific behaviour just don't use --venv and set up environment manually.

When using pytoil new --starter python --venv [venv|conda] pytoil should export a populated requirements.txt or environment.yml file into the created project which would include all common_packages specified in the config as well as any additional packages passed on the command line.

The solution you would like

On invoking the mentioned command, and after the starter template and virtual environment has been set up, pytoil should export the requirements. This will be different depending on the environment:

• venv: Just write the list of packages to requirements.txt in append mode
• conda: Run conda env export --from-history --name <envname>

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

If you checkout a owner/repo pattern of something that doesn't exist, you get the following:

Expected Behavior

Prior to asking the user whether to fork or clone, it should verify that the target repo actually exists on GitHub, if not report to the user it could be a typo etc.

Steps To Reproduce

No response

Environment

- OS: MacOS
- Python: 3.10.3
- pytoil: 0.29.0

Anything else?

No response

Is there an existing issue for this?

• I have searched the existing issues

How would this feature be useful?

Not essential but go has go bug which helps you file an issue about the language. I like the idea, might help tease out some bugs from people (if any) who are using it.

Describe the solution you'd like

A pytoil bug command that just opens up pytoils issues. Super simple!

Describe alternatives you've considered

No response

Anything else?

No response

The solution you would like

Currently pytoil checkout can only clone repos you own. However, a common workflow to work on an open source project is to fork someone else's repo, then clone your fork.

Pytoil checkout could also do this!

One possible solution:

• If the project argument to checkout matches the form of user/repo (regex or just simple split on '/') then pytoil forks the passed repo to the user's account (POST request to the fork repo endpoint) then clones the user's fork.

We might have to sleep for a second or two to allow the fork to go through as this is done asynchronously on GitHub's end.

Once the fork is registered to the user the rest of Pytoil's functionality will work as expected.

Is there an existing issue for this?

• I have searched the existing issues

How would this feature be useful?

pytoil currently only pays attention to repos owned by the authenticated user, it would be useful if it could also support repos where the user is not an owner but a member of an organisation.

Describe the solution you'd like

Something along the lines of:

pytoil show remote

- myrepo1
- myrepo2
- myorg/repo1
- myorg/repo2

As far as I can see this will involve a few things:

• Changing the GraphQL query to also return member repos
• Change the behaviour of pytoil checkout:
  • pytoil checkout myrepo would behave as normal
  • pytoil checkout someone/repo --fork would fork someone elses repo then checkout
  • pytoil checkout org/repo would checkout your orgs repo

Describe alternatives you've considered

N/A

Anything else?

No response

Is there an existing issue for this?

• I have searched the existing issues

How would this feature be useful?

Sometimes if you have lots of projects locally, want to clean it all up but keep certain projects, it can be a pain to list these out to pytoil remove, and you might have local changes which prevent you from doing pytoil remove --all and then pytoil pull <ones to keep>.

A command that can be used to remove all but one or more projects would be great.

Describe the solution you'd like

Either a separate command (something like pytoil keep?) or something like a --all-but option on pytoil remove (I have a slight preference for the latter)

Usage would be like this

pytoil remove --all-but <keep1> <keep2> <keep3>

Then everything under projects_dir other than those specified get's deleted.

Describe alternatives you've considered

Just doing it manually or the --all then pull thing, neither of which are ideal.

Anything else?

No response

Edit: This might have to be a separate command after all as click does not support variadic arguments to options i.e. --all-but <any number of projects> is not allowed

Is your feature request related to a problem

When using pytoil pull, each repo is cloned synchronously. This is fine for a low number of repos but if a user requests a lot of repos or uses pytoil pull --all this takes an unnecessary amount of time as each pull waits for the previous one to complete.

The solution you would like

For a number of repos higher than a certain amount, we could pull them in parallel. I'm not sure what the best approach here is. It's an IO problem really so something like threading or asyncio but I'm not sure how asyncio would integrate with the rest of (synchronous) pytoil.

I think one possible approach might be:

• Calculate the number of repos to pull as normal
• If number of repos is high enough so that concurrency makes sense (otherwise just do it normally):
  • Set up a task queue with the clone urls
  • Set up a worker pool of some fixed size with the Git.clone() method as target
  • Set it loose on the task queue
• Ensure everything is done (concurrent.futures maybe?) and then message out to the user as normal

This issue is here more of a reminder to myself, this is probably quite a challenging one to implement properly, but if anyone seeing this wants to have a go then by all means send a PR 👍🏻

Describe the bug

When starting a new project using pytoil new <project> --venv venv, you get the following error:

To Reproduce

Run the following command:

pytoil new test --venv venv

Expected behavior

Pytoil should create a new blank project with a standard python virtual environment

Screenshots

If applicable, add screenshots to help explain your problem.

See above

Environment

pytoil --version

python version: 3.9.6 (default, Jul  3 2021, 11:29:30) [Clang 12.0.5 (clang-1205.0.22.11)]
pytoil version: v0.6.3
platform: Darwin
OS: macOS-11.4-x86_64-i386-64bit
arch: x86_64

Additional context

Add any other context about the problem here.

Describe the bug

When calling pytoil checkout user/repo it tries to fork every time rather than checking if you have already forked that particular repo.

This is an oversight on my end when writing the fork functionality.

To Reproduce

# Where user/repo is a fork you have already forked
pytoil checkout user/repo

Expected behavior

Should warn that the fork already exists and exit.

Screenshots

If applicable, add screenshots to help explain your problem.

Environment

pytoil --version

Additional context

Add any other context about the problem here.

How would this feature be useful?

Once you fork a project, probably one of the most common things to do is clone your fork, the gh cli natively supports this for example.

Describe the solution you'd like

on pytoil checkout someoneelse/repo we first create a fork of that repo under the user, forking happens asynchronously so there's no guarantee the fork exists during runtime.

It would be good though if pytoil could detect when the fork was completed and then do the following:

• Clone the users fork (normal checkout functionality here)
• Set the remote upstream to the original repo

One possible implementation would be a short sleep hidden behind a loading spinner to allow GitHub to reach
a synchronous state and the fork to become available.

The gh CLI appears to do this by prompting the user if they want to fork, by which time the fork has completed.

Describe alternatives you've considered

Just doing it manually.

How would this feature be useful?

Would be pretty cool to do for example pytoil search myproject and it would essentially grep through local and remote
project lists and show you where that project is.

Or better yet, do something with it i.e. checkout

Describe the solution you'd like

See above

Describe alternatives you've considered

Describe the bug

When asking for a starter template, the info log output shows the repr of the starter object which is quite ugly

To Reproduce

Run the following command:

pytoil new test --starter python

Expected behavior

Should surface a nicer looking message e.g. instead of:

ℹ Creating 'test' from starter: <Starter.python: 'python'>.

Should be:

ℹ Creating 'test' from starter: 'python'

Screenshots

See above

Environment

pytoil --version

python version: 3.9.6 (default, Jul  3 2021, 11:29:30) [Clang 12.0.5 (clang-1205.0.22.11)]
pytoil version: v0.7.0
platform: Darwin
OS: macOS-11.4-x86_64-i386-64bit
arch: x86_64

Additional context

Add any other context about the problem here.

Is there an existing issue for this?

• I have searched the existing issues

How would this feature be useful?

The new async version of pytoil is pretty rapid, with lots of async, GraphQL API, concurrency etc. but it still makes an API call every time something like pytoil show remote is used.

This seems somewhat wasteful?

Describe the solution you'd like

Pytoil keeps some sort of simple cache of the user's remote projects, this could be as simple as a single file somewhere. A toml example below:

[cache]
last_refresh = "<datetime>"

[[repos]]
name = "<reponame>"
size = 14
created = "<datetime>"
modified = "<datetime>"

[[repos]]
name = "<another>"
size = 12
created = "<datetime>"
modified = "<datetime>"

This way the cache could provide the full output to pytoil show remote without hitting the GitHub API.

Set a sensible default timeout that if we exceed, pytoil will fetch the latest info and repopulate the cache.

Probably should add a flag to all subcommands that use the API like --refresh which will refresh the cache manually.

Describe alternatives you've considered

No response

Anything else?

No response

Is there an existing issue for this?

• I have searched the existing issues

How would this feature be useful?

Mamba is a much faster C++ implementation of the conda package manager, a lot of people use it over conda. For the commands we run in pytoil it's a straight swap of the command name so should be fairly easy to configure.

Describe the solution you'd like

A config key conda_bin or similar that can accept conda or mamba, then that program will be used in all conda operations

Describe alternatives you've considered

Currently no way of doing this

Anything else?

No response

Is there an existing issue for this?

• I have searched the existing issues

How would this feature be useful?

Currently pytoil checkout someoneelse/repo just automatically forks the repo to the user's GitHub and then clones the fork, which is a sensible default and is probably what you want 90% of the time.

However, sometimes you just want to clone the other person's repo (e.g. to checkout a PR if you're a collaborator), so pytoil should provide some mechanism for doing this.

Describe the solution you'd like

Two approaches (I can't decide which I prefer):

1. Make the default to ask the user interactively whether they want to fork and clone, or just clone the original. This interactivity can be overridden with a --fork or a --clone flag
2. Keep the default behaviour as is and provide a --no-fork or --clone flag which simply clones the original

Describe alternatives you've considered

No alternative to this functionality currently (other than manually clone of course)

Anything else?

No response

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

The code blocks are formatted with the shell syntax highlighting but this caused weird highlighting on words like local etc.

Try something like console or just plain text if not.

Not really a bug, just thought I'd track it here with everything else

Expected Behavior

No response

Steps To Reproduce

No response

Environment

- OS:
- Python:
- pytoil:

Anything else?

No response

How would this feature be useful?

Currently pytoil looks for the code binary on $PATH to know how to open VSCode. However this breaks if the only VSCode build you have is the VSCode Insiders.

Describe the solution you'd like

Also support looking for the code-insiders binary, potentially with a config option to select which one to prefer.

Describe alternatives you've considered

None, currently having VSCode Insiders only breaks pytoils VSCode functionality

The solution you would like

pytoil --version should be the one stop shop to get all the environment/installation info for pytoil. This will be very handy for debugging on other installations.

Stuff like:

• python version
• OS
• Arch
• pytoil version
• versions of dependencies

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Pytoil currently uses the "updatedAt" graphql field to populate the same field on repo info, show remote etc. This shows when the git object was last updated.

However, a more accurate field to choose would be "pushedAt" as this will bring back the timestamp at which the main branch was changed, which is probably what the user wants when looking for "updated at".

Currently pytoil shows out of date change timestamp info.

Expected Behavior

If you push to main (or merge a PR in), repo info should show the timestamp of the push not when the underlying git object was updated (which can be several hours out of date)

Steps To Reproduce

Change a repo and do pytoil info on it, you won't see the changed timestamp.

Environment

- OS:MacOs
- Python: 3.10.2
- pytoil: 0.26.0

Anything else?

No response