tdsmith / homebrew-pypi-poet Goto Github PK

Generates Homebrew resource stanzas for Python packages.

License: MIT License

Python 100.00%

homebrew-pypi-poet's Introduction

homebrew-pypi-poet

Invoked like poet foo for some package foo which is presently installed in sys.path, determines which packages foo and its dependents depend on, downloads them from pypi and computes their checksums, and spits out Homebrew resource stanzas.

poet -f foo will give you a complete Homebrew formula.

poet -s foo will write a resource stanza for a single package foo, which does not need to be installed, without considering its dependencies.

poet will use the versions of the packages that you presently have installed. If a package it wants to reference is not installed, the latest version on pypi will be downloaded and checksummed and its dependencies will not be considered.

The easiest way to use poet is to create a virtualenv, use pip or setuptools to install the target package and its dependencies in the virtualenv, and then pip install homebrew-pypi-poet and run poet inside the virtualenv.

Usage is like:

usage: poet [-h] [-V]
            [--single package [package ...] | --formula package |
             --resources package]
            [--also package]

Generate Homebrew resource stanzas for pypi packages and their dependencies.

optional arguments:
  -h, --help            show this help message and exit
  --single package [package ...], -s package [package ...]
                        Generate a resource stanza for one or more packages,
                        without considering dependencies.
  --formula package, -f package
                        Generate a complete formula for a pypi package with
                        its recursive pypi dependencies as resources.
  --also package, -a package
                        Specify an additional package that should be added to
                        the resource list with its recursive dependencies. May
                        not be used with --single. May be specified more than
                        once.
  --resources package, -r package
                        Generate resource stanzas for a package and its
                        recursive dependencies (default).
  -V, --version         show program's version number and exit

License

homebrew-pypi-poet is offered under the MIT license.

Contributors

homebrew-pypi-poet is maintained by Tim D. Smith. Robson Peixoto, Alessio Bogon, Julien Maupetit, and Zhiming Wang are thanked for their helpful contributions!

homebrew-pypi-poet's People

Stargazers

Watchers

homebrew-pypi-poet's Issues

formula class is bogus for hyphenated names

iMac-TMP:~ joe$ brew instal -dvs mongo-orchestration
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/mongo-orchestration.rb
Error: /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/mongo-orchestration.rb:1: syntax error, unexpected '-', expecting '<' or ';' or '\n'
class Mongo-orchestration < Formula
            ^
Please report this bug:

Add option to "refresh" formulas

Given an already existing formula:

Update any existing pypi references to the versions in the current virtualenv
Figure out what to do about any "extra" pypi references in the formula; often optional packages get written into the formula and it's important to maintain them, but sometimes packages drop dependencies
Mostly preserve the order of the resources in the original formula

Project unmaintained?

Hello, what is the current maintenance status of this project?

failure fetching info for vobject

Might have to do with the unusual version (https://pypi.python.org/pypi/vobject/0.8.1c)?

All I get is

  resource "vobject" do
    url ""
     ""
  end

Deprecate project

Homebrew will now take care of getting the necessary resource blocks if you supply it the initial download URL for a package, so this project isn't really that important.

problems creating formula for gprof2dot

I think the source of the issue is that poet is trying to fetch information for urllib2 which appears to be part of the standard python library. Give it a try yourself to take a look at the debugging output:

$ poet gprof2dot
Traceback (most recent call last):
  File "/usr/local/bin/poet", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/site-packages/poet/poet.py", line 203, in main
    print(resources_for(package))
  File "/usr/local/lib/python2.7/site-packages/poet/poet.py", line 157, in resources_for
    nodes = make_graph(package)
  File "/usr/local/lib/python2.7/site-packages/poet/poet.py", line 131, in make_graph
    package_data = research_package(package, dependencies[package]['version'])
  File "/usr/local/lib/python2.7/site-packages/poet/poet.py", line 95, in research_package
    format(name, version or ''))
  File "/usr/local/homebrew/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/homebrew/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/usr/local/homebrew/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/homebrew/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 475, in error
    return self._call_chain(*args)
  File "/usr/local/homebrew/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/local/homebrew/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 558, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 404: Not Found

gprof2dot is available through pip/PyPi

Example of poet autodeployment with Travis CI

This is an FYI for the poet maintainers, not a bug. I thought you might be interested in how I set up automated deployment to my homebrew tap. Perhaps it's useful to someone else as an example.

I use TravisCI to run build tests and deploy to PyPI if the build is tagged and passes. After deployment, the macOS build will run poet (twice) to generate the necessary information for a homebrew formula, and then automatically update the homebrew formula on Github.

Access to the homebrew tap on Github is through a Github token for a Github account with push access that I created for doing the updates. (In theory one should be able to avoid the second account using Github's ssh deploy keys, but that route does not work for me since the version of ssh available on Travis CI's macOS images is crippled somehow.)

The relevant files are:
https://github.com/jbarlow83/OCRmyPDF/blob/master/.travis.yml
https://github.com/jbarlow83/OCRmyPDF/blob/master/.travis/osx_brew.sh
https://github.com/jbarlow83/OCRmyPDF/blob/master/.travis/autobrew.py

I find this works much better for me than manually updating the formula because I do a few releases per month.

It also illustrates a possible shortcoming in poet's feature set: the inability to specify dependencies on non-Python system libraries that are required to build. (Or maybe this is a conscious decision.)

Fail on pip install

$ mktmpenv
New python executable in tmp-46e53108f671f48b/bin/python
Installing setuptools, pip...done.
Requirement already up-to-date: pip in /Users/robinho/.virtualenvs/tmp-46e53108f671f48b/lib/python2.7/site-packages
Requirement already up-to-date: setuptools in /Users/robinho/.virtualenvs/tmp-46e53108f671f48b/lib/python2.7/site-packages
This is a temporary environment. It will be deleted when you run 'deactivate'.
$ pip install homebrew-pypi-poet
Collecting homebrew-pypi-poet
  Could not find any downloads that satisfy the requirement homebrew-pypi-poet
  No distributions at all found for homebrew-pypi-poet
$ pip search homebrew-pypi-poet
homebrew-pypi-poet        - Writes Homebrew stanzas for pypi packages

Add option to exclude package

My organisation has homebrew formulas for internal tools written in python that are not published to pypi so fail the dependency resolution phase, when trying to get a url.

Error generating a formula for FoBiS.py

Hi,
I’m trying to generate a formula for FoBiS.py and am encountering the following error:

$ poet -f FoBiS.py
Traceback (most recent call last):
  File "/usr/local/bin/poet", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/site-packages/poet/poet.py", line 184, in main
    print(formula_for(args.formula))
  File "/usr/local/lib/python2.7/site-packages/poet/poet.py", line 141, in formula_for
    root = nodes[package]
KeyError: ‘FoBiS.py’

Do you know if there is a work around for this? I think it may be a bug.

Recognize when packages are installed with extras

i.e. magic-wormhole depends on twisted[tls] but poet just sees the twisted dependency and so doesn't pull in cryptography and friends

use the pypi.io API to get sha256 hashes

It's not "stable" but it "works"; the url blocks look like:

{
    "comment_text": "",
    "digests": {
        "md5": "1954a2eca9a07b1cb16c5ef3bb0a98ef",
        "sha256": "58ea1acc7c71a3f24d7b2c58c84d766f88a7bdfd60f14250046ad95d29992db4"
    },
    "downloads": 0,
    "filename": "homebrew-pypi-poet-0.6.0.tar.gz",
    "has_sig": false,
    "md5_digest": "1954a2eca9a07b1cb16c5ef3bb0a98ef",
    "packagetype": "sdist",
    "python_version": "source",
    "size": 4550,
    "upload_time": "2016-04-14T07:23:18",
    "url": "https://pypi.io/packages/2e/8a/ef7264a374e60b64e78936b64e558a233dfdf6feda3afe18284344dba9d8/homebrew-pypi-poet-0.6.0.tar.gz"
}

<dstufft>   As a reminder: "once Warehouse is ready to go live pypi.io is going to be the new PyPI URL and pypi.python.org will redirect to it, however pypi.io is currently pre-production grade so we don't have any alerting on if it goes down and if it's broken I won't stay up all night to fix it etc"

Support extra index URLs

Adding support for extra index URLs would greatly help in generating Formulae for packages published to internal/self-hosted Python Package Indexes.

Consider using virtualenv_install_with_resources for --formula

Code is easy to fix — just need to update FORMULA_TEMPLATE in poet.py, and virtualenv_install_with_resources is usually enough. However, as far as I can tell we do lose some flexibility like the ability to pass additional options to setup.py. Do you think it's worthwhile?

Packages in private pip repositories are not supported

I have a private package hosted in our company's artifactory. I have a extra-index-url set in my pip config but homebrew-pypi-poet doesn't respect extra-index-url. It just has a hardcoded package resolution to https://pypi.io/pypi/{}/json in https://github.com/tdsmith/homebrew-pypi-poet/blob/master/poet/poet.py#L80.

Fails for entrypoints

I'm trying to generate resource blocks for jupyter, but receiving a failure for the entrypoints dependency. Note that I receive a similar message for poet -s entrypoints:

(tmp-bdd0011f9f08ece1) bash-3.2$ poet jupyter
/Users/chrislong/.virtualenvs/tmp-bdd0011f9f08ece1/lib/python2.7/site-packages/poet/poet.py:97: PackageVersionNotFoundWarning: Could not find an exact version match for entrypoints version 0.2.2; using newest instead
  format(name, version), PackageVersionNotFoundWarning)
Traceback (most recent call last):
  File "/Users/chrislong/.virtualenvs/tmp-bdd0011f9f08ece1/bin/poet", line 11, in <module>
    sys.exit(main())
  File "/Users/chrislong/.virtualenvs/tmp-bdd0011f9f08ece1/lib/python2.7/site-packages/poet/poet.py", line 250, in main
    print(resources_for([package] + args.also))
  File "/Users/chrislong/.virtualenvs/tmp-bdd0011f9f08ece1/lib/python2.7/site-packages/poet/poet.py", line 175, in resources_for
    nodes = merge_graphs(make_graph(p) for p in packages)
  File "/Users/chrislong/.virtualenvs/tmp-bdd0011f9f08ece1/lib/python2.7/site-packages/poet/poet.py", line 182, in merge_graphs
    for g in graphs:
  File "/Users/chrislong/.virtualenvs/tmp-bdd0011f9f08ece1/lib/python2.7/site-packages/poet/poet.py", line 175, in <genexpr>
    nodes = merge_graphs(make_graph(p) for p in packages)
  File "/Users/chrislong/.virtualenvs/tmp-bdd0011f9f08ece1/lib/python2.7/site-packages/poet/poet.py", line 142, in make_graph
    package_data = research_package(package, dependencies[package]['version'])
  File "/Users/chrislong/.virtualenvs/tmp-bdd0011f9f08ece1/lib/python2.7/site-packages/poet/poet.py", line 105, in research_package
    d['url'] = artefact['url']
TypeError: 'NoneType' object has no attribute '__getitem__'

Ship groom_resources with poet

It alphabetizes.

https://gist.github.com/d51e5cebfa822f0bfff7ffc91782cdb5

A cute but obscure name would be "dedeckle," referring to irregular deckle edges in printing.

poet sometimes fails to select latest version because its method of accessing PyPI returns the wrong version

I ran into this issue while using poet. I'm fairly confident it's not your fault, but I thought you'd be interested in following the issue:
pypi/warehouse#1996

Add tests for --also flag and merge_graphs

set up basic functional tests, integrate with tox and travis

http://docs.travis-ci.com/user/languages/python/

Inspiration:

poet -s broken with pinned packages

Hi there!

Awesome tool. Just wanted to flag this but in version 0.10.0.

Specifying a pinned package is no problem with the default behavior...

poet six==1.11.0
  resource "six" do
    url "https://files.pythonhosted.org/packages/16/d8/bc6316cf98419719bd59c91742194c111b6f2e85abac88e496adefaf7afe/six-1.11.0.tar.gz"
    sha256 "70e8a77beed4562e7f14fe23a786b54f6296e34344c23bc42f07b15018ff98e9"
  end

But seems to break with the -s flag. Looks like pypi no longer supports this url scheme https://pypi.io/pypi/{}/json

poet -s six==1.11.0
Traceback (most recent call last):
  File "/Users/jpvelez/.local/share/virtualenvs/science-box-cli-Dy8vfmAv/bin/poet", line 11, in <module>
    sys.exit(main())
  File "/Users/jpvelez/.local/share/virtualenvs/science-box-cli-Dy8vfmAv/lib/python3.6/site-packages/poet/poet.py", line 246, in main
    data = research_package(package)
  File "/Users/jpvelez/.local/share/virtualenvs/science-box-cli-Dy8vfmAv/lib/python3.6/site-packages/poet/poet.py", line 80, in research_package
    with closing(urlopen("https://pypi.io/pypi/{}/json".format(name))) as f:
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 564, in error
    result = self._call_chain(*args)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 756, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 564, in error
    result = self._call_chain(*args)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 756, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 564, in error
    result = self._call_chain(*args)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 756, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

alphabetize dependencies

The sort order doesn't matter for install_requires dependencies.

Deprecate Python2

Creating valid python@2 recipes will require manual/private kegs. Homebrew-core no longer hosts a valid python2 recipe.

Move this repo into the Homebrew organisation?

How would you feel about moving this repo into the Homebrew organisation? You would still have the same permissions to the repo.

This would allow for maintenance to be shared amongst the Homebrew maintainers and also improve its visibility.

Something to discuss if you are okay with the move would be if you are happy with it continuing to be published to PyPI using your personal account or if you would want to transfer it to a Homebrew account.

homebrew formula?

Is it possible to make a homebrew formula for this? I started making one (using poet itself, how meta!) and it will even run poet in the test do stanza of the formula, but it complains about pip when I try to run poet directly from the command line. Is this a hopeless task due to the interaction with pip and homebrew?

Here’s my Formula for references:

class HomebrewPypiPoet < Formula
  desc "Generate Homebrew resource stanzas for Python packages"
  homepage "https://github.com/tdsmith/homebrew-pypi-poet"
  url "https://pypi.python.org/packages/source/h/homebrew-pypi-poet/homebrew-pypi-poet-0.5.0.tar.gz"
  sha256 "a0e8a0db2852238cfc31eeabbf843b1093510aa393d9c2aab73d6981f2846bee"

  head "https://github.com/tdsmith/homebrew-pypi-poet.git"

  depends_on :python if MacOS.version <= :snow_leopard

  resource "Jinja2" do
    url "https://pypi.python.org/packages/source/J/Jinja2/Jinja2-2.7.3.tar.gz"
    sha256 "2e24ac5d004db5714976a04ac0e80c6df6e47e98c354cb2c0d82f8879d4f8fdb"
  end

  resource "MarkupSafe" do
    url "https://pypi.python.org/packages/source/M/MarkupSafe/MarkupSafe-0.23.tar.gz"
    sha256 "a4ec1aff59b95a14b45eb2e23761a0179e98319da5a7eb76b56ea8cdc7b871c3"
  end

  resource "tl.eggdeps" do
    url "https://pypi.python.org/packages/source/t/tl.eggdeps/tl.eggdeps-0.4.tar.gz"
    sha256 "a99de5e4652865224daab09b2e2574a4f7c1d0d9a267048f9836aa914a2caf3a"
  end

  def install
    ENV.prepend_create_path "PYTHONPATH", libexec/"vendor/lib/python2.7/site-packages"
    %w[Jinja2 MarkupSafe tl.eggdeps].each do |r|
      resource(r).stage do
        system "python", *Language::Python.setup_install_args(libexec/"vendor")
      end
    end

    ENV.prepend_create_path "PYTHONPATH", libexec/"lib/python2.7/site-packages"
    system "python", *Language::Python.setup_install_args(libexec)

    bin.install Dir[libexec/"bin/*"]
    bin.env_script_all_files(libexec/"bin", :PYTHONPATH => ENV["PYTHONPATH"])
  end

  test do
    assert_match /resource "FORD" do/, shell_output("#{bin}/poet -s FORD")
  end
end

poet should write correct formulas for python 3

Running poet with python2 should generate a formula for Python 2.7 that installs with system python. It may assume the Python major.minor version is always 2.7. It should declare depends_on :python if MacOS.version <= :snow_leopard.
Running poet with python3 should generate a formula that depends_on :python3 and uses Language::Python.major_minor_version "python3" (which returns e.g. "3.4") to determine e.g. the name of the versioned site-packages path.

poet should always be run with the version of python that the formula requires. Applications should always be installed against python 2 if they can, so that they will use system python.

Graceful fallback when sdist is not available

$ poet mitmproxy
/Users/zmwang/.pyenv/virtualenvs/tmp-a855353f32c7ddc/lib/python3.6/site-packages/poet/poet.py:97: PackageVersionNotFoundWarning: Could not find an exact version match for mitmproxy version 3.0.0; using newest instead
  format(name, version), PackageVersionNotFoundWarning)
Traceback (most recent call last):
  File "/Users/zmwang/.pyenv/virtualenvs/tmp-a855353f32c7ddc/bin/poet", line 11, in <module>
    sys.exit(main())
  File "/Users/zmwang/.pyenv/virtualenvs/tmp-a855353f32c7ddc/lib/python3.6/site-packages/poet/poet.py", line 250, in main
    print(resources_for([package] + args.also))
  File "/Users/zmwang/.pyenv/virtualenvs/tmp-a855353f32c7ddc/lib/python3.6/site-packages/poet/poet.py", line 175, in resources_for
    nodes = merge_graphs(make_graph(p) for p in packages)
  File "/Users/zmwang/.pyenv/virtualenvs/tmp-a855353f32c7ddc/lib/python3.6/site-packages/poet/poet.py", line 182, in merge_graphs
    for g in graphs:
  File "/Users/zmwang/.pyenv/virtualenvs/tmp-a855353f32c7ddc/lib/python3.6/site-packages/poet/poet.py", line 175, in <genexpr>
    nodes = merge_graphs(make_graph(p) for p in packages)
  File "/Users/zmwang/.pyenv/virtualenvs/tmp-a855353f32c7ddc/lib/python3.6/site-packages/poet/poet.py", line 142, in make_graph
    package_data = research_package(package, dependencies[package]['version'])
  File "/Users/zmwang/.pyenv/virtualenvs/tmp-a855353f32c7ddc/lib/python3.6/site-packages/poet/poet.py", line 105, in research_package
    d['url'] = artefact['url']
TypeError: 'NoneType' object is not subscriptable

The problem here is that mitmproxy 3.0.0 doesn't have an sdist, only a none-any bdist_wheel.[1] This is not great on their part, but we don't really care since we fetch the sdist from GitHub.

Maybe instead of failing right there, a more graceful approach could be taken here, e.g. print a warning and ignore the package?

[1] https://pypi.python.org/pypi/mitmproxy/3.0.0#downloads

poet crashes if no sdist available

Support combining resource blocks for additional packages

It should be possible to express "I want to install X's dependencies, and I additionally want to install module Y and Y's recursive dependencies" in a single command. This will help us handle annoying cases like ansible, where we need to install a bunch of extra packages to support "optional" functionality.

use the pip cache for downloads

Downloading packages a second time just to compute their checksum is annoying. Recent pip versions cache downloads. Can we exploit it? It looks kinda opaque.

There was distutils-sig list traffic about pypi/warehouse returning sha256 checksums in the JSON API, which would be nice, and would also save us from hitting the network a second time.

Flag packages that depend on Pillow for special installation steps

Homebrew requires a special installation procedure to ensure that the correction combination of dependencies from Apple system libraries and Homebrew libraries are selected. Pillow is also a popular and widely used library so this special case is likely to occur often enough in practice.

It would be helpful if poet advised users that Pillow needs special procedures and that one should search for a recently updated formula in homebrew-core and copy that.

Provided no other packages need special steps, it could also reproduce the boilerplate to install Pillow, which is as follows:

  depends_on "freetype"
  depends_on "jpeg"

  [...] snip

  def install
    venv = virtualenv_create(libexec, "python3")

    resource("Pillow").stage do
      inreplace "setup.py" do |s|
        sdkprefix = MacOS::CLT.installed? ? "" : MacOS.sdk_path
        s.gsub! "openjpeg.h", "probably_not_a_header_called_this_eh.h"
        s.gsub! "ZLIB_ROOT = None", "ZLIB_ROOT = ('#{sdkprefix}/usr/lib', '#{sdkprefix}/usr/include')"
        s.gsub! "JPEG_ROOT = None", "JPEG_ROOT = ('#{Formula["jpeg"].opt_prefix}/lib', '#{Formula["jpeg"].opt_prefix}/include')"
        s.gsub! "FREETYPE_ROOT = None", "FREETYPE_ROOT = ('#{Formula["freetype"].opt_prefix}/lib', '#{Formula["freetype"].opt_prefix}/include')"
      end

      # avoid triggering "helpful" distutils code that doesn't recognize Xcode 7 .tbd stubs
      ENV.append "CFLAGS", "-I#{MacOS.sdk_path}/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers" unless MacOS::CLT.installed?
      venv.pip_install Pathname.pwd
    end

    res = resources.map(&:name).to_set - ["Pillow"]

    res.each do |r|
      venv.pip_install resource(r)
    end
  end

poet doesn't grok setup_requires requirements

e.g. installing protobuf requires google-apputils at build time but not at runtime. Homebrew needs to install the setup_requires deps (and needs to install them into buildpath).

Sorting install_requires deps on the dependency graph is not helpful because they are not needed at install time. Sorting recursive setup_requires deps is important but hopefully rarely necessary.

Poet fails with HTTP Error 404

I'm trying to create a Brew formula for Gajim. I have grabbed the gajim source and am trying to run poet, but it fails:

# setup venv
python3 -m venv venv
source venv/bin/activate
# install dependencies
pip install keyring nbxmpp pyOpenSSL pyasn1 pyobjc pillow idna precis-i18n certifi cssutils
# download and install gajim
curl -O https://gajim.org/downloads/1.1/gajim-1.1.1.tar.bz2
tar xf gajim-1.1.1.tar.bz2
cd gajim-1.1.1
python setup.py install
# install and run poet
pip install homebrew-pypi-poet
poet -f gajim

Now poet fails with the following output:

Traceback (most recent call last):
  File "/Users/sebas/venv/bin/poet", line 11, in <module>
    sys.exit(main())
  File "/Users/sebas/venv/lib/python3.7/site-packages/poet/poet.py", line 243, in main
    print(formula_for(args.formula, args.also))
  File "/Users/sebas/venv/lib/python3.7/site-packages/poet/poet.py", line 161, in formula_for
    nodes = merge_graphs(make_graph(p) for p in [package] + also)
  File "/Users/sebas/venv/lib/python3.7/site-packages/poet/poet.py", line 187, in merge_graphs
    for g in graphs:
  File "/Users/sebas/venv/lib/python3.7/site-packages/poet/poet.py", line 161, in <genexpr>
    nodes = merge_graphs(make_graph(p) for p in [package] + also)
  File "/Users/sebas/venv/lib/python3.7/site-packages/poet/poet.py", line 147, in make_graph
    package_data = research_package(package, dependencies[package]['version'])
  File "/Users/sebas/venv/lib/python3.7/site-packages/poet/poet.py", line 80, in research_package
    with closing(urlopen("https://pypi.io/pypi/{}/json".format(name))) as f:
  File "/usr/local/Cellar/python/3.7.2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/Cellar/python/3.7.2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python/3.7.2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/Cellar/python/3.7.2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 563, in error
    result = self._call_chain(*args)
  File "/usr/local/Cellar/python/3.7.2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python/3.7.2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 755, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/local/Cellar/python/3.7.2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python/3.7.2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/Cellar/python/3.7.2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 563, in error
    result = self._call_chain(*args)
  File "/usr/local/Cellar/python/3.7.2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python/3.7.2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 755, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/local/Cellar/python/3.7.2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python/3.7.2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/Cellar/python/3.7.2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/local/Cellar/python/3.7.2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python/3.7.2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

Am I using poet right? Any idea what's going wrong?

tdsmith / homebrew-pypi-poet Goto Github PK

homebrew-pypi-poet's Introduction

homebrew-pypi-poet

License

Contributors

homebrew-pypi-poet's People

Stargazers

Watchers

Forkers

homebrew-pypi-poet's Issues

Recommend Projects

Recommend Topics

Recommend Org