dhatim / python-license-check Goto Github PK

View Code? Open in Web Editor NEW

154.0 14.0 55.0 197 KB

Check python packages from requirement.txt and report issues

License: Apache License 2.0

Python 100.00%

python license check tool

python-license-check's People

Contributors

Stargazers

Watchers

python-license-check's Issues

pkg_resources.DistributionNotFound: The 'None' distribution was not found and is required by the application

I get the following error when I try to run liccheck -s strategy.ini -r requirements.txt:

gathering licenses...
Traceback (most recent call last):
  File "/home/egegunes/Dev/arcelik/3d-backend/venv/bin/liccheck", line 11, in <module>
    load_entry_point('liccheck==0.3.12', 'console_scripts', 'liccheck')()
  File "/home/egegunes/Dev/arcelik/3d-backend/venv/lib64/python3.7/site-packages/liccheck/command_line.py", line 264, in main
    sys.exit(run(args))
  File "/home/egegunes/Dev/arcelik/3d-backend/venv/lib64/python3.7/site-packages/liccheck/command_line.py", line 259, in run
    return process(args.requirement_txt_file, strategy, args.level)
  File "/home/egegunes/Dev/arcelik/3d-backend/venv/lib64/python3.7/site-packages/liccheck/command_line.py", line 182, in process
    pkg_info = get_packages_info(requirement_file)
  File "/home/egegunes/Dev/arcelik/3d-backend/venv/lib64/python3.7/site-packages/liccheck/command_line.py", line 103, in get_packages_info
    packages = [transform(dist) for dist in pkg_resources.working_set.resolve(requirements)]
  File "/home/egegunes/Dev/arcelik/3d-backend/venv/lib64/python3.7/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'None' distribution was not found and is required by the application

strategy.ini

# Authorized and unauthorized licenses in LOWER CASE
[Licenses]
authorized_licenses:
        bsd
        new bsd
        bsd license
        new bsd license
        simplified bsd
        apache
        apache 2.0
        apache software license
        gnu lgpl
        lgpl with exceptions or zpl
        isc license
        isc license (iscl)
        mit
        mit license
        python software foundation license
        zpl 2.1

unauthorized_licenses:
        gpl v3

requirements.txt

applicationinsights
azure-batch==6.0.0
azure-storage-blob==1.4.0
celery==4.2.1
Django==2.2
django-cors-headers==2.4.0
django-environ==0.4.5
djangorestframework==3.9.0
gunicorn
psycopg2-binary
requests
whitenoise

git+https://github.com/jazzband/django-oauth-toolkit.git@846ab0ba8acaa3e4870b424700544aa6329511e4

Support SPDX identifiers

I'd like to be able to only allowlist/blocklist specific licenses by their SPDX identifiers (e.g. Apache-2.0), as opposed to the various forms that pop up in packages:

apache 2.0
apache software 2.0
apache license 2.0
apache software license 2.0
the idkapache software license 2.0

Full list of SPDX identifiers is at: https://spdx.org/licenses/

This project has a similar mapping of frequently used identifiers to their proper SPDX identifier:

https://github.com/NFJones/pipoe/blob/master/pipoe/licenses.py

Of course, this only gets you so far as some projects have ambiguous metadata that doesn't explicitly specify the license version (like dateutil for example). Perhaps this could be part of the normal/cautious/paranoid checking levels where "normal" might map "apache" to any version, but "paranoid" would not match unless it included a version number.

Create report of all dependencies with their license

Add an option to get a report of all dependencies with their license, ideally as HTML but text would be sufficient, too.

The report should include the name of the dependency, the version and the first matching license name from authorized_licenses plus the status OK.

If there is no match, it should look for a match in unauthorized_licenses. If there is one, the dependency should be reported as UNAUTH.

If there is no match, the status should be UNKNOWN

Custom proprietary license support

Library has following license metadata:

License: X Proprietary
Classifier: License :: Other/Proprietary License

And liccheck.ini file has following authorized license in it:

[Licenses]
authorized_licenses:
        x proprietary

But liccheck fails, because it needs Other/Proprietary License (as classifier license) to be in liccheck.ini file

However, this doesn't reproduce on versions below 0.5.2

I suppose changing:

python-license-check/liccheck/command_line.py

Line 119 in 668ea7b

licenses = get_license(dist) + get_license_OSI_classifiers(dist)

to:

python-license-check/liccheck/command_line.py

Line 119 in 3cbce20

licenses = get_licenses_from_classifiers(dist) or get_license(dist) or []

caused it.

Maybe liccheck should handle proprietary licenses differently to avoid allowing all Other/Proprietary License instead of custom proprietary licenses?

support pre-commit hook

as it is possible to run with pre-commit as one of hooks

Example:
https://github.com/pre-commit/pre-commit-hooks/blob/master/.pre-commit-hooks.yaml

Allow use on libraries built with Windows line endings.

We are currently having an issue with liccheck and version 17.3.0 of pyrsistent.

Liccheck always flags the package as unknown, due to the fact that there is a \r in the string, which is most likely an artifact from the Windows line break \r\n:

check unknown packages...
2 packages.
    pyrsistent (0.17.3): ['MIT License\r']
      dependency:
          pyrsistent << jsonschema

Other packages using the MIT license are parsed just fine.

I dug a bit into the metadata parsing and found that the metadata for pyrsistent: 17.3.0 indeed contains \r\n linebreaks, while all other packages have \n in their metadata. It probably got compiled on Windows with faulty settings regarding the line endings.

Is it possible to adjust the metadata parsing to support Windows line endings?

Normalized package names

I'm using pip-tools which, as of recently, outputs normalized package names into my requirements.txt. All special characters are replaced with -. This is absolutely fine for pip but causes liccheck to fail.

$ liccheck -s strategy.ini
gathering licenses...
Traceback (most recent call last):
  File "/usr/local/bin/liccheck", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/liccheck/command_line.py", line 344, in main
    sys.exit(run(args))
  File "/usr/local/lib/python3.7/site-packages/liccheck/command_line.py", line 339, in run
    return process(args.requirement_txt_file, strategy, args.level, args.reporting_txt_file, args.no_deps)
  File "/usr/local/lib/python3.7/site-packages/liccheck/command_line.py", line 249, in process
    pkg_info = get_packages_info(requirement_file, no_deps)
  File "/usr/local/lib/python3.7/site-packages/liccheck/command_line.py", line 165, in get_packages_info
    packages = [transform(dist) for dist in resolve_func(requirements)]
  File "/usr/local/lib/python3.7/site-packages/liccheck/command_line.py", line 165, in <listcomp>
    packages = [transform(dist) for dist in resolve_func(requirements)]
  File "/usr/local/lib/python3.7/site-packages/liccheck/requirements.py", line 51, in resolve
    for dist in pkg_resources.working_set.resolve(requirements):
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 783, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'ruamel-yaml-clib==0.2.2' distribution was not found and is required by the application
$ pip freeze | grep clib
ruamel.yaml.clib==0.2.2

The canonical name for this package is ruamel.yaml.clib but my requirements.txt contains ruamel-yaml-clib. Is there any way to make liccheck work with these normalized names?

Improve dual-licensing detection

Ran into an issue with python-dateutil where the license="Dual License" in setup.cfg, but the trove classifiers report Apache 2.0 and BSD 3-clause. My license check failed because "Dual" was not an approved license.

My suggestion here would be to ignore the plain License: * metadata if Classifier: License :: * metadata exists, instead of combining them into a list:

# Current:
licenses = get_license(dist) + get_license_OSI_classifiers(dist)

# Suggestion
licenses = get_license_OSI_classifiers(dist) or get_license(dist) or ['Unknown']

simplejson is another dual-license example, though their metadata specifies it a bit differently:

License: MIT License
License :: OSI Approved :: MIT License
License :: OSI Approved :: Academic Free License (AFL)

A VersionConflict exception when a system installed package conflicts with a pinned requirements package

Steps to reproduce:

pip install setuptools==40.0.0
cat strategy.ini:

[Licenses]
authorized_licenses:
        mit

cat requirements.txt

setuptools>=41.0.0

liccheck -s strategy.ini -r requirements.txt

gathering licenses...
Traceback (most recent call last):
  File "/private/tmp/liccheck/.venv/bin/liccheck", line 8, in <module>
    sys.exit(main())
  File "/private/tmp/liccheck/.venv/lib/python3.8/site-packages/liccheck/command_line.py", line 301, in main
    sys.exit(run(args))
  File "/private/tmp/liccheck/.venv/lib/python3.8/site-packages/liccheck/command_line.py", line 296, in run
    return process(args.requirement_txt_file, strategy, args.level)
  File "/private/tmp/liccheck/.venv/lib/python3.8/site-packages/liccheck/command_line.py", line 231, in process
    pkg_info = get_packages_info(requirement_file)
  File "/private/tmp/liccheck/.venv/lib/python3.8/site-packages/liccheck/command_line.py", line 152, in get_packages_info
    packages = [transform(dist) for dist in pkg_resources.working_set.resolve(requirements)]
  File "/private/tmp/liccheck/.venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 782, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (setuptools 40.0.0 (/private/tmp/liccheck/.venv/lib/python3.8/site-packages), Requirement.parse('setuptools>=41.0.0'))

Allow matching with regex

Hi,

Thanks for this great package ! I just set it up for a project but found that many of my (sub)dependencies mentioned their license in an ill-defined way :

License string
Apache Software License; BSD License
BSD
BSD 3-Clause
BSD License
BSD License; GNU Library or Lesser General Public License (LGPL)
BSD-3-Clause-LBNL
Copyright (c) 2014 Gavin M. Roy
GNU General Public License v3 (GPLv3)
GNU Library or Lesser General Public License (LGPL)
Historical Permission Notice and Disclaimer (HPND)
ISC
ISC License (ISCL)
MIT
MIT License
MPL-2.0
Mozilla Public License 2.0 (MPL 2.0)
Public Domain
Python Software Foundation License
http://www.apache.org/licenses/LICENSE-2.0

In this case, I thought it was more practical for me to provide literal strings in the authorized_licenses under unauthorized_licenses keys, from which two regex patterns are build.
I made a quick POC (see PR #94 ), if you judge this feature of interest I can finish it with tests and docs ;)

Improve error message in case a requirement is not installed.

In case a requirement found in the requirements.txt is not installed in the current environment, liccheck fails with an AttributeError like this:

(myenv) me@geoms:~$ liccheck --no-deps
gathering licenses...
Traceback (most recent call last):
  File "/home/me/mambaforge/envs/myenv/bin/liccheck", line 8, in <module>
    sys.exit(main())
  File "/home/me/mambaforge/envs/myenv/lib/python3.10/site-packages/liccheck/command_line.py", line 399, in main
    sys.exit(run(args))
  File "/home/me/mambaforge/envs/myenv/lib/python3.10/site-packages/liccheck/command_line.py", line 388, in run
    return process(args['requirement_txt_file'], strategy, args['level'], args['reporting_txt_file'],
  File "/home/me/mambaforge/envs/myenv/lib/python3.10/site-packages/liccheck/command_line.py", line 250, in process
    pkg_info = get_packages_info(requirement_file, no_deps)
  File "/home/me/mambaforge/envs/myenv/lib/python3.10/site-packages/liccheck/command_line.py", line 166, in get_packages_info
    packages = [transform(dist) for dist in resolve_func(requirements)]
  File "/home/me/mambaforge/envs/myenv/lib/python3.10/site-packages/liccheck/command_line.py", line 166, in <listcomp>
    packages = [transform(dist) for dist in resolve_func(requirements)]
  File "/home/me/mambaforge/envs/myenv/lib/python3.10/site-packages/liccheck/command_line.py", line 120, in transform
    licenses = get_licenses_from_classifiers(dist) or get_license(dist) or []
  File "/home/me/mambaforge/envs/myenv/lib/python3.10/site-packages/liccheck/command_line.py", line 146, in get_licenses_from_classifiers
    if dist.has_metadata(dist.PKG_INFO):
AttributeError: 'NoneType' object has no attribute 'has_metadata'

It would be nice to have a more useful error message.

release on PyPI

currently I'm using my own fork: https://pypi.org/project/liccheck2/

Broken on pip 20.1

With pip 20.1 (released today) and liccheck 0.4.2, we get this:

$ liccheck -l CAUTIOUS -s requirements/licenses.ini -r requirements.txt
gathering licenses...
Traceback (most recent call last):
  File "/Users/cdestigter/checkout/sno/venv/bin/liccheck", line 11, in <module>
    load_entry_point('liccheck==0.4.2', 'console_scripts', 'liccheck')()
  File "/Users/cdestigter/checkout/sno/venv/lib/python3.7/site-packages/liccheck/command_line.py", line 316, in main
    sys.exit(run(args))
  File "/Users/cdestigter/checkout/sno/venv/lib/python3.7/site-packages/liccheck/command_line.py", line 311, in run
    return process(args.requirement_txt_file, strategy, args.level)
  File "/Users/cdestigter/checkout/sno/venv/lib/python3.7/site-packages/liccheck/command_line.py", line 246, in process
    pkg_info = get_packages_info(requirement_file)
  File "/Users/cdestigter/checkout/sno/venv/lib/python3.7/site-packages/liccheck/command_line.py", line 127, in get_packages_info
    if req.markers:
AttributeError: 'ParsedRequirement' object has no attribute 'markers'

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

chore(deps): update dependency ubuntu to v22

Detected dependencies

github-actions

.github/workflows/build.yml

actions/checkout v4

actions/setup-python v5

codecov/codecov-action v4

ubuntu 20.04

.github/workflows/release.yml

actions/checkout v4

actions/setup-python v5

codecov/codecov-action v4

actions/checkout v4

ubuntu 20.04

pip_requirements

requirements.txt

pip >=9.0.1

test-requirements.txt

pytest >=3.6.3

pytest-mock >=1.10

pip_setup

setup.py

semantic_version >=2.7.0

Check this box to trigger a request for Renovate to run again on this repository

python-dateutil comes out as ['BSD License\r']


 ['BSD License\r']  (2.4.2) : Simplified BSD

      dependencies:

          python-dateutil << pandas

VersionConflict: zipp

$ pyenv local 3.8.4
$ python -m venv venv
$ source venv/bin/activate
$ pip install liccheck
$ liccheck -s license-strategy.ini -r requirements-dev.txt
gathering licenses...
Traceback (most recent call last):
  File "/home/moose/GitHub/MartinThoma/mpu/venv/bin/liccheck", line 8, in <module>
    sys.exit(main())
  File "/home/moose/GitHub/MartinThoma/mpu/venv/lib/python3.8/site-packages/liccheck/command_line.py", line 332, in main
    sys.exit(run(args))
  File "/home/moose/GitHub/MartinThoma/mpu/venv/lib/python3.8/site-packages/liccheck/command_line.py", line 327, in run
    return process(args.requirement_txt_file, strategy, args.level, args.reporting_txt_file, args.no_deps)
  File "/home/moose/GitHub/MartinThoma/mpu/venv/lib/python3.8/site-packages/liccheck/command_line.py", line 237, in process
    pkg_info = get_packages_info(requirement_file, no_deps)
  File "/home/moose/GitHub/MartinThoma/mpu/venv/lib/python3.8/site-packages/liccheck/command_line.py", line 153, in get_packages_info
    packages = [transform(dist) for dist in resolve_func(requirements)]
  File "/home/moose/GitHub/MartinThoma/mpu/venv/lib/python3.8/site-packages/liccheck/command_line.py", line 153, in <listcomp>
    packages = [transform(dist) for dist in resolve_func(requirements)]
  File "/home/moose/GitHub/MartinThoma/mpu/venv/lib/python3.8/site-packages/liccheck/requirements.py", line 47, in resolve
    for dist in pkg_resources.working_set.resolve(requirements):
  File "/home/moose/GitHub/MartinThoma/mpu/venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 787, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'zipp==3.1.0' distribution was not found and is required by the application

Authorized Packages don't bypass unauthorized licenses

With the following requirements.txt

wand
yoyo-migrations

and licence_strategy.ini

# Authorized and unauthorized licenses in lower case
[Licenses]
authorized_licenses:
        BSD
        new BSD
        BSD license
        new BDS license
        simplified BSD
        Apache
        Apache 2.0
        Apache software license
        gnu LGPL
        LGPL with exceptions or zpl
        ISC license
        ISC license (ISCL)
        MIT
        MIT license
        python software foundation license
        zpl 2.1

unauthorized_licenses:
        GPL v3
        GPL
        GNU General Public License v2 or later (GPLv2+)


[Authorized Packages]
# Python software license (see http://zesty.ca/python/uuid.README.txt)
uuid: 1.30
Unidecode: 0.4.20

I get the following result :

gathering licenses...
6 packages and dependencies.

check forbidden packages based on licenses...
1 forbidden packages :

    Unidecode (0.4.20) : GPL ['GNU General Public License v2 or later (GPLv2+)']

      dependencies:

          Unidecode << python-slugify << yoyo-migrations



check authorized packages based on licenses...
4 packages.

check authorized packages...
1 packages.

check unknown licenses...
none

Unidecode should not be matched in the forbidden packages and the execution should succeed

Read configuration from `pyproject.toml`

Thanks for your quick "fix" of #26

I have another proposal: It would be great if it wasn't required to add another configuration file but to include a liccheck section in the new pyproject.toml file ( https://www.python.org/dev/peps/pep-0518/ ), similar to how black is doing it:

A sample configuration might look like this:

[tool.liccheck]
authorized_licenses = [
  "bsd",
  "new bsd",
  "bsd license"
]
unauthorized_licenses = [
  "gpl v3"
]
authorized_packages = [
  "uuid: 1.30"
]

What do you think about this? I'll be happy to open a PR for this.

allow liccheck to be used as a pre-commit hook

Enabling liccheck as a pre-commit hook can be done by just adding simple yaml file in the root of the repository which declares the hook so users can use it.

See https://github.com/pre-commit/pre-commit/blob/master/.pre-commit-hooks.yaml

I may make a pull request to add if you want, it is very easy.

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Location: renovate.json
Error type: The renovate configuration file contains some invalid settings
Message: packageRules[0]: Each packageRule must contain at least one match* or exclude* selector. Rule: {"automerge":true}

Add usage mode to support not resolving dependencies

I am conceptually trying to use liccheck in a CI job as follows:

pipenv lock -r > requirements.txt
liccheck -r requirements.txt

However this fails due to #42 because liccheck tries to resolve all dependencies but they have not actually been installed in any virtualenv. It isn't necessary to resolve dependencies here because the requirements file was generated from the lockfile and has pinned dependencies that have already been resolved. Installing a bunch of dependencies is not desired for what should be a lightweight CI job.

Is it possible to add a usage mode to support this, or does liccheck require all the dependencies to be installed?

Support globs for license matches

Let's say you have a project which uses fluent-logger, stevedore, and typed-ast. These three all have an apache license, but running liccheck against them looks like this:

    fluent-logger (0.9.6): ['Apache License, Version 2.0']
      dependency:
          fluent-logger
    stevedore (2.0.1): ['Apache Software']
      dependency:
          stevedore 
    typed-ast (1.4.1): ['Apache License 2.0']
      dependency:
          typed-ast

which means that instead of this

authorized_licenses:
        apache

I have to do this

authorized_licenses:
        apache license, version 2.0
        apache software
        apache license 2.0

I think it would be nice to support globs, in order to match all three with something similar to the intuitive one while still keeping readability high

authorized_licenses:
        apache*

Feature Request: Option To Specify Where To Look for License Information

Packages on the python package index specify license information in one of two or both places:

As part of the trove classifiers
As metadata

In some packages, the classifier information is more general e.g: "BSD License", whereas the metadata contains information on the specific version of the license used "BSD License (BSD-3-Clause)". For example:

Since liccheck will always prefer the information retrieved from the classifiers, it is not possible to block specific versions of licenses. However, if it is possible to specify that I prefer to retrieve license information from the package metadata, I could whitelist (or blacklist) specific versions (like BSD-4-Clause). For example, the pip-licenses package supports this.

I'm happy to open a pull request with the necessary changes.

Create support to map license "keys" to names and URLs.

Add an API and config to map several license "keys" (i.e. apache, apache 2.0 and apache 2, apl2) to a single canonical name (example: Apache License v2).

For the HTML report (see issue #34), allow to specify the URL of the license.

Proposed config format:

[License Mapping]
Apache License v2: https://opensource.org/licenses/Apache-2.0
    apache  # One key per line
    apache 2.0
    apache 2
BSD: ...
GNU Public License v3: ...
ISC License: https://opensource.org/licenses/ISC
    isc license

[License Overrides]
# See http://zesty.ca/python/uuid.README.txt
uuid 1.3.0: isc license

If the first "line" of the mapping value starts with http:// or https://, it's a URL, otherwise a license key.

`or`-ed license check breaks on licenses with "or" in the name

The or-ed license check, added in #100 blindly splits the license name on the word "or".

This breaks the license check for projects like psycopg2 which has a license of GNU Library or Lesser General Public License (LGPL) (this gets turned into ["GNU Library", "Lesser General Public License (LGPL)"] by the get_license_names function).

liccheck should support all supported python version

As a linter, in order to be usable to most python package maintainers, liccheck should be tested and functioning on all python versions, thus includes todays production python versions: py27,py34,py35,py36,py37 as of July 2018.

PS. I was really keen to test it and adopt it for pycontribs projects but current requirements make it impossible to use.

broken on pip 10

    from pip.download import PipSession
ImportError: No module named 'pip.download'

0.5.2 is not installable: ParseVersionError Unable to parse "dev"

I am using poetry to manage my environment. poetry install throws this error:

ParseVersionError Unable to parse "dev"

#62 seems to be at fault. It works fine in the build pipeline to create the source tarball and upload to pypi, but it is not installable because setup.py still specifies the version as dev when installing

Some trove license classifiers are ignored

Not all packages have OSI approved licenses, so their metadata wont be found by a search for Classifier: License :: OSI Approved ::.

Here's the list of all standard trove classifiers: https://pypi.org/classifiers/

Seems like the solution would be to stop looking for OSI Approved specifically (make that part of the regex optional).

liccheck vs liccheck2

What is the difference between those two:

Failed builds on travis

Looks like #48 broke CI. Unfortunately, there were no CI jobs on PR, which could help track the issue earlier. Fix is on the way.

Editable requirements (-e) raises "DistributionNotFound: The 'None' distribution was not found"

Editable requirements, i.e: rows like this one
-e file:some_internal_package # via -r requirements.in

Are currently making this library fail with pkg_resources.DistributionNotFound: The 'None' distribution was not found and is required by the application

This is kind of related to #33 but although error message is the same, it is actually a different problem as that one relates to git+https:// construct, and this one, to the -e construct

Failing build

Currently, the first thing that I see on PyPI is that the build is failing: https://pypi.org/project/liccheck/

pre-commit hooks is not working as expected

First of all thanks for all effort for this lybrary.

  files: ^(.*requirements.*\.txt|setup\.cfg|setup\.py|pyproject\.toml|liccheck\.ini)$

This regex detect multiple requirements.txt file but when i deleted all others after it is still not working.

i set up .pre-commit-config.yaml as below:

repos:
  - repo: local
    hooks:
    - id: liccheck
      name: Run Python License Checker
      description: Check license compliance of python requirements
      entry: liccheck
      args: []
      language: python
      files: ^(.*requirements.*\.txt|setup\.cfg|setup\.py|pyproject\.toml|liccheck\.ini)$

After first attempt:

Run Python License Checker...............................................Failed
- hook id: liccheck
- exit code: 2

usage: liccheck.EXE [-h] [-s STRATEGY_INI_FILE]
                    [-l {STANDARD,CAUTIOUS,PARANOID}]
                    [-r [REQUIREMENT_TXT_FILE]] [-R [REPORTING_TXT_FILE]]
                    [--no-deps]
liccheck.EXE: error: unrecognized arguments: server/project/google cloud functions/function-name/requirements.txt server/project/google cloud functions/function-name2/requirements.txt server/requirements.txt

I delete all others except one requirements.txt, run again:

Run Python License Checker...............................................Failed
- hook id: liccheck
- exit code: 2

usage: liccheck.EXE [-h] [-s STRATEGY_INI_FILE]
                    [-l {STANDARD,CAUTIOUS,PARANOID}]
                    [-r [REQUIREMENT_TXT_FILE]] [-R [REPORTING_TXT_FILE]]
                    [--no-deps]
liccheck.EXE: error: unrecognized arguments: server/requirements.txt

That seems arguments are not enough fitting for running.

- repo: local
  hooks:
  - id: liccheck
    name: Run Python License Checker
    description: Check license compliance of python requirements
    entry: liccheck
    args: 
    - -s server/liccheck.ini
    - -r server/requirements.txt
    language: python
    files: ^(.*requirements.*\.txt|setup\.cfg|setup\.py|pyproject\.toml|liccheck\.ini)$

after run again:

Run Python License Checker...............................................Failed
- hook id: liccheck
- exit code: 2

usage: liccheck.EXE [-h] [-s STRATEGY_INI_FILE]
                    [-l {STANDARD,CAUTIOUS,PARANOID}]
                    [-r [REQUIREMENT_TXT_FILE]] [-R [REPORTING_TXT_FILE]]
                    [--no-deps]
liccheck.EXE: error: unrecognized arguments: server/requirements.txt

All of tested in Windows 10.

DistributionNotFound: The 'greenlet==1.1.3' distribution was not found and is required by the application

Running liccheck on this requirements.txt

alembic==1.8.1 
anyio==3.6.1 
aredis==1.1.8 
certifi==2022.6.15.2 
click==8.1.3 
colorama==0.4.5 
fastapi==0.83.0 
greenlet==1.1.3
h11==0.12.0 
httpcore==0.15.0 
httpx==0.23.0 
idna==3.4 
importlib-metadata==4.2.0 
importlib-resources==5.9.0 
mako==1.2.2 
markupsafe==2.1.1 
pydantic==1.10.2 
rfc3986[idna2008]==1.5.0 
sentry-sdk==1.9.8 
sniffio==1.3.0 
sqlalchemy-utils==0.38.3 
sqlalchemy==1.4.41 
starlette==0.19.1 
typing-extensions==4.3.0 
urllib3==1.26.12 
uvicorn==0.18.3 
zipp==3.8.1

raises

gathering licenses...
Traceback (most recent call last):
  File "/Users/marek/Library/Caches/pypoetry/virtualenvs/big-battery-Zhirr-W2-py3.7/bin/liccheck", line 8, in <module>
    sys.exit(main())
  File "/Users/marek/Library/Caches/pypoetry/virtualenvs/big-battery-Zhirr-W2-py3.7/lib/python3.7/site-packages/liccheck/command_line.py", line 399, in main
    sys.exit(run(args))
  File "/Users/marek/Library/Caches/pypoetry/virtualenvs/big-battery-Zhirr-W2-py3.7/lib/python3.7/site-packages/liccheck/command_line.py", line 389, in run
    args['no_deps'])
  File "/Users/marek/Library/Caches/pypoetry/virtualenvs/big-battery-Zhirr-W2-py3.7/lib/python3.7/site-packages/liccheck/command_line.py", line 250, in process
    pkg_info = get_packages_info(requirement_file, no_deps)
  File "/Users/marek/Library/Caches/pypoetry/virtualenvs/big-battery-Zhirr-W2-py3.7/lib/python3.7/site-packages/liccheck/command_line.py", line 166, in get_packages_info
    packages = [transform(dist) for dist in resolve_func(requirements)]
  File "/Users/marek/Library/Caches/pypoetry/virtualenvs/big-battery-Zhirr-W2-py3.7/lib/python3.7/site-packages/liccheck/command_line.py", line 166, in <listcomp>
    packages = [transform(dist) for dist in resolve_func(requirements)]
  File "/Users/marek/Library/Caches/pypoetry/virtualenvs/big-battery-Zhirr-W2-py3.7/lib/python3.7/site-packages/liccheck/requirements.py", line 51, in resolve
    for dist in pkg_resources.working_set.resolve(requirements):
  File "/Users/marek/Library/Caches/pypoetry/virtualenvs/big-battery-Zhirr-W2-py3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 795, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'greenlet==1.1.3' distribution was not found and is required by the application

Same goes for colorama==0.4.5

pyproject configuration is not included in readme

When project configuration was added the README was not updated to mention it, making use of it require some digging. Ideally the readme should cover both.

Version 0.4.1 is not on PyPI

According to https://pypi.org/project/liccheck/ the latest published version is 0.4.0, but here on Github I see that 0.4.1 was released. Could you please upload to PyPI ?

[Feature Request] Print reporting to stdout

I want to include liccheck into a CI pipeline. It would be nice if it created a formatted output. I propose to add a flag --report-to-stdout which would create this output:

Liccheck checked 63 packages:
* 62 OK
* 1 UNKNOWN

State Package name                License                
--------------------------------------------------------------------------------
OK    Flask 1.1.2                 BSD-3-Clause           
OK    cryptography 3.4.6          BSD or Apache License, Version 2.0
OK    paramiko 2.7.2              GNU Library or Lesser General Public Lice...

Attribute error with pyqt5 library

Really a nice and helpful tool. Unfortunately if fails for pyqt packages.

gathering licenses...
Traceback (most recent call last):
...
license = regex_license.search(metadata).group('license')
AttributeError: 'NoneType' object has no attribute 'group'

Reproduce: ini file from repo, requirements.txt with the following content

PyQt5==5.11.3
PyQt5-sip==4.19.13

authorized_packages doesn't work in pyproject.toml

Seems like authorized_packages is just ignored.

requirements.txt

aiofiles==0.7.0
[...]

$ pipenv run liccheck
Loading .env environment variables...
gathering licenses...
131 packages and dependencies.
check authorized packages...
118 packages.
check unauthorized packages...
11 packages.
    aiofiles (0.7.0): ['Other/Proprietary']
      dependency:
          aiofiles
[...]

[tool.liccheck]
authorized_licenses = [
  'BSD',
  'MIT',
  'Python Software Foundation',
]

unauthorized_licenses = [
  'Other/Proprietary',
]

[[tool.liccheck.authorized_packages]]
aiofiles = '0.7.0'

Here we see that liccheck is complaining about aiofiles version 0.7.0, but that is clearly incorrect because it's included in the authorized_packages section of pyproject.toml.

Support execution with "python -m liccheck"

I currently get

'liccheck' is a package and cannot be directly executed

Please change that. If you want, I can make a PR for it; it's pretty easy.

parameters should be optional and have default implicit values

There is no need to force user to add -r requirements.txt as this should be taken as default value unless someone defines it.

Same applies for the -s license_strategy.ini, which should be assumed to be license_strategy.ini. I would personally prefer a hidden file but I see you already picked this.

pkg_resources.VersionConflict: (pandas 1.3.5 (~/miniconda3/envs/myenv/lib/python3.8/site-packages), Requirement.parse('pandas==1.3.3'))

Hi there,

I'm getting this pkg_resources.VersionConflict error.

$ liccheck -s liccheck.ini

~/miniconda3/envs/myenv/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
gathering licenses...
Traceback (most recent call last):
  File "~/miniconda3/envs/myenv/bin/liccheck", line 8, in <module>
    sys.exit(main())
  File "~/miniconda3/envs/myenv/lib/python3.8/site-packages/liccheck/command_line.py", line 336, in main
    sys.exit(run(args))
  File "~/miniconda3/envs/myenv/lib/python3.8/site-packages/liccheck/command_line.py", line 331, in run
    return process(args.requirement_txt_file, strategy, args.level, args.reporting_txt_file, args.no_deps)
  File "~/miniconda3/envs/myenv/lib/python3.8/site-packages/liccheck/command_line.py", line 241, in process
    pkg_info = get_packages_info(requirement_file, no_deps)
  File "~/miniconda3/envs/myenv/lib/python3.8/site-packages/liccheck/command_line.py", line 157, in get_packages_info
    packages = [transform(dist) for dist in resolve_func(requirements)]
  File "~/miniconda3/envs/myenv/lib/python3.8/site-packages/liccheck/command_line.py", line 157, in <listcomp>
    packages = [transform(dist) for dist in resolve_func(requirements)]
  File "~/miniconda3/envs/myenv/lib/python3.8/site-packages/liccheck/requirements.py", line 51, in resolve
    for dist in pkg_resources.working_set.resolve(requirements):
  File "~/miniconda3/envs/myenv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 782, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (pandas 1.3.5 (~/miniconda3/envs/myenv/lib/python3.8/site-packages), Requirement.parse('pandas==1.3.3'))

I'll admit I don't fully understand the error. I'm also a bit confused as I didn't think liccheck would check for dependency version conflicts - is there a way to turn this off maybe?

Many thanks for any help, and for this great tool!

Different outputs on windows and linux executions

Using this requirements.txt

Flask>=0.12.1
flask_restful
jsonify
psycopg2>=2.7.1
nose
scipy
scikit-learn
pandas
numpy
argparse
uuid
sqlbuilder
proboscis

and this license_strategy.ini

# Authorized and unauthorized licenses in lower case
[Licenses]
authorized_licenses:
        BSD
        new BSD
        BSD license
        new BDS license
        simplified BSD
        Apache
        Apache 2.0
        Apache software license
        gnu LGPL
        LGPL with exceptions or zpl
        ISC license
        ISC license (ISCL)
        MIT
        MIT license
        python software foundation license
        zpl 2.1

unauthorized_licenses:
        GPL v3
        GPL
        GNU General Public License v2 or later (GPLv2+)


[Authorized Packages]
# Python software license (see http://zesty.ca/python/uuid.README.txt)
uuid: 1.30

I have two different output results when executedon windows and bash.
On Bash (Ubuntu), exit code = 0 (expected):

gathering licenses...
22 packages and dependencies.

check forbidden packages based on licenses...
none


check authorized packages based on licenses...
18 packages.

check authorized packages...
4 packages.

check unknown licenses...
none

On Windows, exit code = -1 (unexpected):

gathering licenses...
22 packages and dependencies.

check forbidden packages based on licenses...
none


check authorized packages based on licenses...
10 packages.

check authorized packages...
3 packages.

check unknown licenses...
9 unknown packages :

 ['BSD License\r'] NKNOWN

      dependencies:

          click << Flask << Flask-RESTful

 ['BSD License\r'] : BSD

      dependencies:

          Jinja2 << Flask << Flask-RESTful

 ['BSD License\r'] 3) : BSD

      dependencies:

          MarkupSafe << Jinja2 << Flask << Flask-RESTful

 ['GNU Library or Lesser General Public License (LGPL)\r']

      dependencies:

          nose

 [] numpy (1.11.3) : BSD

      dependencies:

          numpy << pandas

 [] pandas (0.19.2) : BSD

      dependencies:

          pandas

 ['BSD License\r']  (2.6.0) : Simplified BSD

      dependencies:

          python-dateutil << aniso8601 << Flask-RESTful

          python-dateutil << pandas

 ['BSD License\r'] : BSD

      dependencies:

          scipy

 ['BSD License\r'] 15) : BSD

      dependencies:

          Werkzeug << Flask << Flask-RESTful

Order licenses alphabetically

Hey, it looks like the script doesn't order licenses in any way, resulting in such diffs when running multiple times

Could we add alphabetic ordering to the licenses to make it more idempotent?

whitelisting packages across versions

Hey,

this is a great package, it just enriched our CI pipeline :)

Is it possible to whitelist packages without pinning them to specific versions? I presumed:

[Authorized Packages]
uuid: "*"

[Authorized Packages]
uuid: "<1.5"

but this wasn't working...

Support `poetry.lock`

poetry is a Python package manager that is gaining momentum as an alternative to pip.

It provides a concrete poetry.lock describing the resolved dependencies of a project.

Would it make sense to add a poetry.lock parsing module to this project to integrate smoothly with poetry projects?

It might be sensible to add the parse as on extra to the project that would be installed with liccheck[poetry]. Depends if we use the poetry project to read the lock file.

poetry does have the functionality to generate a requirements.txt so there is a work around possible.

v0.5.0 is tagged but not on PyPI

liccheck 0.3.11 installation is broken

It seems the installation of liccheck 0.3.11 is broken:

$ pip install liccheck                                                                                                                                                                       ~/tmp/liccheck {#}
Collecting liccheck
  Downloading https://files.pythonhosted.org/packages/91/a7/83118d3ba708ebdf29c6bdd7a593d7297311492fe7ca63854ae0c8cbff20/liccheck-0.3.11.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-kdjtfgh6/liccheck/setup.py", line 20, in <module>
        with open(path.join(here, 'requirements.txt'), encoding='utf-8') as f:
      File "/usr/lib/python3.7/codecs.py", line 898, in open
        file = builtins.open(filename, mode, buffering)
    FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-install-kdjtfgh6/liccheck/requirements.txt'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-kdjtfgh6/liccheck/

Broken on pip 20

pip 20 has moved some code around specifically _internal/download.py no longer exists and seems to have been split into _internal/network/download.py and _internal/network/session.py. I've created a PR that I believe resolves the issue: #37

Feedback welcome

dhatim / python-license-check Goto Github PK

python-license-check's People

Contributors

Stargazers

Watchers

Forkers

python-license-check's Issues

Ignored or Blocked

Detected dependencies

Recommend Projects

Recommend Topics

Recommend Org