dhatim / python-license-check Goto Github PK
View Code? Open in Web Editor NEWCheck python packages from requirement.txt and report issues
License: Apache License 2.0
Check python packages from requirement.txt and report issues
License: Apache License 2.0
Let's say you have a project which uses fluent-logger
, stevedore
, and typed-ast
. These three all have an apache license, but running liccheck against them looks like this:
fluent-logger (0.9.6): ['Apache License, Version 2.0']
dependency:
fluent-logger
stevedore (2.0.1): ['Apache Software']
dependency:
stevedore
typed-ast (1.4.1): ['Apache License 2.0']
dependency:
typed-ast
which means that instead of this
authorized_licenses:
apache
I have to do this
authorized_licenses:
apache license, version 2.0
apache software
apache license 2.0
I think it would be nice to support globs, in order to match all three with something similar to the intuitive one while still keeping readability high
authorized_licenses:
apache*
Thanks for your quick "fix" of #26
I have another proposal: It would be great if it wasn't required to add another configuration file but to include a liccheck
section in the new pyproject.toml
file ( https://www.python.org/dev/peps/pep-0518/ ), similar to how black is doing it:
A sample configuration might look like this:
[tool.liccheck]
authorized_licenses = [
"bsd",
"new bsd",
"bsd license"
]
unauthorized_licenses = [
"gpl v3"
]
authorized_packages = [
"uuid: 1.30"
]
What do you think about this? I'll be happy to open a PR for this.
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.
.github/workflows/build.yml
actions/checkout v4
actions/setup-python v5
codecov/codecov-action v4
ubuntu 20.04
.github/workflows/release.yml
actions/checkout v4
actions/setup-python v5
codecov/codecov-action v4
actions/checkout v4
ubuntu 20.04
requirements.txt
pip >=9.0.1
test-requirements.txt
pytest >=3.6.3
pytest-mock >=1.10
setup.py
semantic_version >=2.7.0
We are currently having an issue with liccheck
and version 17.3.0
of pyrsistent
.
Liccheck always flags the package as unknown, due to the fact that there is a \r
in the string, which is most likely an artifact from the Windows line break \r\n
:
check unknown packages...
2 packages.
pyrsistent (0.17.3): ['MIT License\r']
dependency:
pyrsistent << jsonschema
Other packages using the MIT license are parsed just fine.
I dug a bit into the metadata parsing and found that the metadata for pyrsistent: 17.3.0
indeed contains \r\n
linebreaks, while all other packages have \n
in their metadata. It probably got compiled on Windows with faulty settings regarding the line endings.
Is it possible to adjust the metadata parsing to support Windows line endings?
Add an API and config to map several license "keys" (i.e. apache
, apache 2.0
and apache 2
, apl2
) to a single canonical name (example: Apache License v2
).
For the HTML report (see issue #34), allow to specify the URL of the license.
Proposed config format:
[License Mapping]
Apache License v2: https://opensource.org/licenses/Apache-2.0
apache # One key per line
apache 2.0
apache 2
BSD: ...
GNU Public License v3: ...
ISC License: https://opensource.org/licenses/ISC
isc license
[License Overrides]
# See http://zesty.ca/python/uuid.README.txt
uuid 1.3.0: isc license
If the first "line" of the mapping value starts with http:// or https://, it's a URL, otherwise a license key.
Running liccheck
on this requirements.txt
alembic==1.8.1
anyio==3.6.1
aredis==1.1.8
certifi==2022.6.15.2
click==8.1.3
colorama==0.4.5
fastapi==0.83.0
greenlet==1.1.3
h11==0.12.0
httpcore==0.15.0
httpx==0.23.0
idna==3.4
importlib-metadata==4.2.0
importlib-resources==5.9.0
mako==1.2.2
markupsafe==2.1.1
pydantic==1.10.2
rfc3986[idna2008]==1.5.0
sentry-sdk==1.9.8
sniffio==1.3.0
sqlalchemy-utils==0.38.3
sqlalchemy==1.4.41
starlette==0.19.1
typing-extensions==4.3.0
urllib3==1.26.12
uvicorn==0.18.3
zipp==3.8.1
raises
gathering licenses...
Traceback (most recent call last):
File "/Users/marek/Library/Caches/pypoetry/virtualenvs/big-battery-Zhirr-W2-py3.7/bin/liccheck", line 8, in <module>
sys.exit(main())
File "/Users/marek/Library/Caches/pypoetry/virtualenvs/big-battery-Zhirr-W2-py3.7/lib/python3.7/site-packages/liccheck/command_line.py", line 399, in main
sys.exit(run(args))
File "/Users/marek/Library/Caches/pypoetry/virtualenvs/big-battery-Zhirr-W2-py3.7/lib/python3.7/site-packages/liccheck/command_line.py", line 389, in run
args['no_deps'])
File "/Users/marek/Library/Caches/pypoetry/virtualenvs/big-battery-Zhirr-W2-py3.7/lib/python3.7/site-packages/liccheck/command_line.py", line 250, in process
pkg_info = get_packages_info(requirement_file, no_deps)
File "/Users/marek/Library/Caches/pypoetry/virtualenvs/big-battery-Zhirr-W2-py3.7/lib/python3.7/site-packages/liccheck/command_line.py", line 166, in get_packages_info
packages = [transform(dist) for dist in resolve_func(requirements)]
File "/Users/marek/Library/Caches/pypoetry/virtualenvs/big-battery-Zhirr-W2-py3.7/lib/python3.7/site-packages/liccheck/command_line.py", line 166, in <listcomp>
packages = [transform(dist) for dist in resolve_func(requirements)]
File "/Users/marek/Library/Caches/pypoetry/virtualenvs/big-battery-Zhirr-W2-py3.7/lib/python3.7/site-packages/liccheck/requirements.py", line 51, in resolve
for dist in pkg_resources.working_set.resolve(requirements):
File "/Users/marek/Library/Caches/pypoetry/virtualenvs/big-battery-Zhirr-W2-py3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 795, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'greenlet==1.1.3' distribution was not found and is required by the application
Same goes for colorama==0.4.5
First of all thanks for all effort for this lybrary.
files: ^(.*requirements.*\.txt|setup\.cfg|setup\.py|pyproject\.toml|liccheck\.ini)$
This regex detect multiple requirements.txt
file but when i deleted all others after it is still not working.
i set up .pre-commit-config.yaml
as below:
repos:
- repo: local
hooks:
- id: liccheck
name: Run Python License Checker
description: Check license compliance of python requirements
entry: liccheck
args: []
language: python
files: ^(.*requirements.*\.txt|setup\.cfg|setup\.py|pyproject\.toml|liccheck\.ini)$
After first attempt:
Run Python License Checker...............................................Failed
- hook id: liccheck
- exit code: 2
usage: liccheck.EXE [-h] [-s STRATEGY_INI_FILE]
[-l {STANDARD,CAUTIOUS,PARANOID}]
[-r [REQUIREMENT_TXT_FILE]] [-R [REPORTING_TXT_FILE]]
[--no-deps]
liccheck.EXE: error: unrecognized arguments: server/project/google cloud functions/function-name/requirements.txt server/project/google cloud functions/function-name2/requirements.txt server/requirements.txt
I delete all others except one requirements.txt
, run again:
Run Python License Checker...............................................Failed
- hook id: liccheck
- exit code: 2
usage: liccheck.EXE [-h] [-s STRATEGY_INI_FILE]
[-l {STANDARD,CAUTIOUS,PARANOID}]
[-r [REQUIREMENT_TXT_FILE]] [-R [REPORTING_TXT_FILE]]
[--no-deps]
liccheck.EXE: error: unrecognized arguments: server/requirements.txt
That seems arguments are not enough fitting for running.
- repo: local
hooks:
- id: liccheck
name: Run Python License Checker
description: Check license compliance of python requirements
entry: liccheck
args:
- -s server/liccheck.ini
- -r server/requirements.txt
language: python
files: ^(.*requirements.*\.txt|setup\.cfg|setup\.py|pyproject\.toml|liccheck\.ini)$
after run again:
Run Python License Checker...............................................Failed
- hook id: liccheck
- exit code: 2
usage: liccheck.EXE [-h] [-s STRATEGY_INI_FILE]
[-l {STANDARD,CAUTIOUS,PARANOID}]
[-r [REQUIREMENT_TXT_FILE]] [-R [REPORTING_TXT_FILE]]
[--no-deps]
liccheck.EXE: error: unrecognized arguments: server/requirements.txt
All of tested in Windows 10.
currently I'm using my own fork: https://pypi.org/project/liccheck2/
I'm using pip-tools which, as of recently, outputs normalized package names into my requirements.txt
. All special characters are replaced with -
. This is absolutely fine for pip
but causes liccheck
to fail.
$ liccheck -s strategy.ini
gathering licenses...
Traceback (most recent call last):
File "/usr/local/bin/liccheck", line 10, in <module>
sys.exit(main())
File "/usr/local/lib/python3.7/site-packages/liccheck/command_line.py", line 344, in main
sys.exit(run(args))
File "/usr/local/lib/python3.7/site-packages/liccheck/command_line.py", line 339, in run
return process(args.requirement_txt_file, strategy, args.level, args.reporting_txt_file, args.no_deps)
File "/usr/local/lib/python3.7/site-packages/liccheck/command_line.py", line 249, in process
pkg_info = get_packages_info(requirement_file, no_deps)
File "/usr/local/lib/python3.7/site-packages/liccheck/command_line.py", line 165, in get_packages_info
packages = [transform(dist) for dist in resolve_func(requirements)]
File "/usr/local/lib/python3.7/site-packages/liccheck/command_line.py", line 165, in <listcomp>
packages = [transform(dist) for dist in resolve_func(requirements)]
File "/usr/local/lib/python3.7/site-packages/liccheck/requirements.py", line 51, in resolve
for dist in pkg_resources.working_set.resolve(requirements):
File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 783, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'ruamel-yaml-clib==0.2.2' distribution was not found and is required by the application
$ pip freeze | grep clib
ruamel.yaml.clib==0.2.2
The canonical name for this package is ruamel.yaml.clib
but my requirements.txt
contains ruamel-yaml-clib
. Is there any way to make liccheck
work with these normalized names?
Editable requirements, i.e: rows like this one
-e file:some_internal_package # via -r requirements.in
Are currently making this library fail with pkg_resources.DistributionNotFound: The 'None' distribution was not found and is required by the application
This is kind of related to #33 but although error message is the same, it is actually a different problem as that one relates to git+https://
construct, and this one, to the -e
construct
I am conceptually trying to use liccheck in a CI job as follows:
pipenv lock -r > requirements.txt
liccheck -r requirements.txt
However this fails due to #42 because liccheck tries to resolve all dependencies but they have not actually been installed in any virtualenv. It isn't necessary to resolve dependencies here because the requirements file was generated from the lockfile and has pinned dependencies that have already been resolved. Installing a bunch of dependencies is not desired for what should be a lightweight CI job.
Is it possible to add a usage mode to support this, or does liccheck require all the dependencies to be installed?
There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.
Location: renovate.json
Error type: The renovate configuration file contains some invalid settings
Message: packageRules[0]: Each packageRule must contain at least one match* or exclude* selector. Rule: {"automerge":true}
I get the following error when I try to run liccheck -s strategy.ini -r requirements.txt
:
gathering licenses...
Traceback (most recent call last):
File "/home/egegunes/Dev/arcelik/3d-backend/venv/bin/liccheck", line 11, in <module>
load_entry_point('liccheck==0.3.12', 'console_scripts', 'liccheck')()
File "/home/egegunes/Dev/arcelik/3d-backend/venv/lib64/python3.7/site-packages/liccheck/command_line.py", line 264, in main
sys.exit(run(args))
File "/home/egegunes/Dev/arcelik/3d-backend/venv/lib64/python3.7/site-packages/liccheck/command_line.py", line 259, in run
return process(args.requirement_txt_file, strategy, args.level)
File "/home/egegunes/Dev/arcelik/3d-backend/venv/lib64/python3.7/site-packages/liccheck/command_line.py", line 182, in process
pkg_info = get_packages_info(requirement_file)
File "/home/egegunes/Dev/arcelik/3d-backend/venv/lib64/python3.7/site-packages/liccheck/command_line.py", line 103, in get_packages_info
packages = [transform(dist) for dist in pkg_resources.working_set.resolve(requirements)]
File "/home/egegunes/Dev/arcelik/3d-backend/venv/lib64/python3.7/site-packages/pkg_resources/__init__.py", line 786, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'None' distribution was not found and is required by the application
strategy.ini
# Authorized and unauthorized licenses in LOWER CASE
[Licenses]
authorized_licenses:
bsd
new bsd
bsd license
new bsd license
simplified bsd
apache
apache 2.0
apache software license
gnu lgpl
lgpl with exceptions or zpl
isc license
isc license (iscl)
mit
mit license
python software foundation license
zpl 2.1
unauthorized_licenses:
gpl v3
requirements.txt
applicationinsights
azure-batch==6.0.0
azure-storage-blob==1.4.0
celery==4.2.1
Django==2.2
django-cors-headers==2.4.0
django-environ==0.4.5
djangorestframework==3.9.0
gunicorn
psycopg2-binary
requests
whitenoise
git+https://github.com/jazzband/django-oauth-toolkit.git@846ab0ba8acaa3e4870b424700544aa6329511e4
poetry
is a Python package manager that is gaining momentum as an alternative to pip
.
It provides a concrete poetry.lock
describing the resolved dependencies of a project.
Would it make sense to add a poetry.lock
parsing module to this project to integrate smoothly with poetry
projects?
It might be sensible to add the parse as on extra to the project that would be installed with liccheck[poetry]
. Depends if we use the poetry project to read the lock file.
poetry
does have the functionality to generate a requirements.txt
so there is a work around possible.
Using this requirements.txt
Flask>=0.12.1
flask_restful
jsonify
psycopg2>=2.7.1
nose
scipy
scikit-learn
pandas
numpy
argparse
uuid
sqlbuilder
proboscis
and this license_strategy.ini
# Authorized and unauthorized licenses in lower case
[Licenses]
authorized_licenses:
BSD
new BSD
BSD license
new BDS license
simplified BSD
Apache
Apache 2.0
Apache software license
gnu LGPL
LGPL with exceptions or zpl
ISC license
ISC license (ISCL)
MIT
MIT license
python software foundation license
zpl 2.1
unauthorized_licenses:
GPL v3
GPL
GNU General Public License v2 or later (GPLv2+)
[Authorized Packages]
# Python software license (see http://zesty.ca/python/uuid.README.txt)
uuid: 1.30
I have two different output results when executedon windows and bash.
On Bash (Ubuntu), exit code = 0
(expected):
gathering licenses...
22 packages and dependencies.
check forbidden packages based on licenses...
none
check authorized packages based on licenses...
18 packages.
check authorized packages...
4 packages.
check unknown licenses...
none
On Windows, exit code = -1
(unexpected):
gathering licenses...
22 packages and dependencies.
check forbidden packages based on licenses...
none
check authorized packages based on licenses...
10 packages.
check authorized packages...
3 packages.
check unknown licenses...
9 unknown packages :
['BSD License\r'] NKNOWN
dependencies:
click << Flask << Flask-RESTful
['BSD License\r'] : BSD
dependencies:
Jinja2 << Flask << Flask-RESTful
['BSD License\r'] 3) : BSD
dependencies:
MarkupSafe << Jinja2 << Flask << Flask-RESTful
['GNU Library or Lesser General Public License (LGPL)\r']
dependencies:
nose
[] numpy (1.11.3) : BSD
dependencies:
numpy << pandas
[] pandas (0.19.2) : BSD
dependencies:
pandas
['BSD License\r'] (2.6.0) : Simplified BSD
dependencies:
python-dateutil << aniso8601 << Flask-RESTful
python-dateutil << pandas
['BSD License\r'] : BSD
dependencies:
scipy
['BSD License\r'] 15) : BSD
dependencies:
Werkzeug << Flask << Flask-RESTful
With pip 20.1 (released today) and liccheck 0.4.2, we get this:
$ liccheck -l CAUTIOUS -s requirements/licenses.ini -r requirements.txt
gathering licenses...
Traceback (most recent call last):
File "/Users/cdestigter/checkout/sno/venv/bin/liccheck", line 11, in <module>
load_entry_point('liccheck==0.4.2', 'console_scripts', 'liccheck')()
File "/Users/cdestigter/checkout/sno/venv/lib/python3.7/site-packages/liccheck/command_line.py", line 316, in main
sys.exit(run(args))
File "/Users/cdestigter/checkout/sno/venv/lib/python3.7/site-packages/liccheck/command_line.py", line 311, in run
return process(args.requirement_txt_file, strategy, args.level)
File "/Users/cdestigter/checkout/sno/venv/lib/python3.7/site-packages/liccheck/command_line.py", line 246, in process
pkg_info = get_packages_info(requirement_file)
File "/Users/cdestigter/checkout/sno/venv/lib/python3.7/site-packages/liccheck/command_line.py", line 127, in get_packages_info
if req.markers:
AttributeError: 'ParsedRequirement' object has no attribute 'markers'
Library has following license metadata:
License: X Proprietary
Classifier: License :: Other/Proprietary License
And liccheck.ini
file has following authorized license in it:
[Licenses]
authorized_licenses:
x proprietary
But liccheck
fails, because it needs Other/Proprietary License
(as classifier license) to be in liccheck.ini
file
However, this doesn't reproduce on versions below 0.5.2
I suppose changing:
python-license-check/liccheck/command_line.py
Line 119 in 668ea7b
python-license-check/liccheck/command_line.py
Line 119 in 3cbce20
caused it.
Maybe liccheck
should handle proprietary licenses differently to avoid allowing all Other/Proprietary License
instead of custom proprietary licenses?
$ pyenv local 3.8.4
$ python -m venv venv
$ source venv/bin/activate
$ pip install liccheck
$ liccheck -s license-strategy.ini -r requirements-dev.txt
gathering licenses...
Traceback (most recent call last):
File "/home/moose/GitHub/MartinThoma/mpu/venv/bin/liccheck", line 8, in <module>
sys.exit(main())
File "/home/moose/GitHub/MartinThoma/mpu/venv/lib/python3.8/site-packages/liccheck/command_line.py", line 332, in main
sys.exit(run(args))
File "/home/moose/GitHub/MartinThoma/mpu/venv/lib/python3.8/site-packages/liccheck/command_line.py", line 327, in run
return process(args.requirement_txt_file, strategy, args.level, args.reporting_txt_file, args.no_deps)
File "/home/moose/GitHub/MartinThoma/mpu/venv/lib/python3.8/site-packages/liccheck/command_line.py", line 237, in process
pkg_info = get_packages_info(requirement_file, no_deps)
File "/home/moose/GitHub/MartinThoma/mpu/venv/lib/python3.8/site-packages/liccheck/command_line.py", line 153, in get_packages_info
packages = [transform(dist) for dist in resolve_func(requirements)]
File "/home/moose/GitHub/MartinThoma/mpu/venv/lib/python3.8/site-packages/liccheck/command_line.py", line 153, in <listcomp>
packages = [transform(dist) for dist in resolve_func(requirements)]
File "/home/moose/GitHub/MartinThoma/mpu/venv/lib/python3.8/site-packages/liccheck/requirements.py", line 47, in resolve
for dist in pkg_resources.working_set.resolve(requirements):
File "/home/moose/GitHub/MartinThoma/mpu/venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 787, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'zipp==3.1.0' distribution was not found and is required by the application
In case a requirement found in the requirements.txt is not installed in the current environment, liccheck fails with an AttributeError like this:
(myenv) me@geoms:~$ liccheck --no-deps
gathering licenses...
Traceback (most recent call last):
File "/home/me/mambaforge/envs/myenv/bin/liccheck", line 8, in <module>
sys.exit(main())
File "/home/me/mambaforge/envs/myenv/lib/python3.10/site-packages/liccheck/command_line.py", line 399, in main
sys.exit(run(args))
File "/home/me/mambaforge/envs/myenv/lib/python3.10/site-packages/liccheck/command_line.py", line 388, in run
return process(args['requirement_txt_file'], strategy, args['level'], args['reporting_txt_file'],
File "/home/me/mambaforge/envs/myenv/lib/python3.10/site-packages/liccheck/command_line.py", line 250, in process
pkg_info = get_packages_info(requirement_file, no_deps)
File "/home/me/mambaforge/envs/myenv/lib/python3.10/site-packages/liccheck/command_line.py", line 166, in get_packages_info
packages = [transform(dist) for dist in resolve_func(requirements)]
File "/home/me/mambaforge/envs/myenv/lib/python3.10/site-packages/liccheck/command_line.py", line 166, in <listcomp>
packages = [transform(dist) for dist in resolve_func(requirements)]
File "/home/me/mambaforge/envs/myenv/lib/python3.10/site-packages/liccheck/command_line.py", line 120, in transform
licenses = get_licenses_from_classifiers(dist) or get_license(dist) or []
File "/home/me/mambaforge/envs/myenv/lib/python3.10/site-packages/liccheck/command_line.py", line 146, in get_licenses_from_classifiers
if dist.has_metadata(dist.PKG_INFO):
AttributeError: 'NoneType' object has no attribute 'has_metadata'
It would be nice to have a more useful error message.
According to https://pypi.org/project/liccheck/ the latest published version is 0.4.0, but here on Github I see that 0.4.1 was released. Could you please upload to PyPI ?
Hi there,
I'm getting this pkg_resources.VersionConflict
error.
$ liccheck -s liccheck.ini
~/miniconda3/envs/myenv/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
gathering licenses...
Traceback (most recent call last):
File "~/miniconda3/envs/myenv/bin/liccheck", line 8, in <module>
sys.exit(main())
File "~/miniconda3/envs/myenv/lib/python3.8/site-packages/liccheck/command_line.py", line 336, in main
sys.exit(run(args))
File "~/miniconda3/envs/myenv/lib/python3.8/site-packages/liccheck/command_line.py", line 331, in run
return process(args.requirement_txt_file, strategy, args.level, args.reporting_txt_file, args.no_deps)
File "~/miniconda3/envs/myenv/lib/python3.8/site-packages/liccheck/command_line.py", line 241, in process
pkg_info = get_packages_info(requirement_file, no_deps)
File "~/miniconda3/envs/myenv/lib/python3.8/site-packages/liccheck/command_line.py", line 157, in get_packages_info
packages = [transform(dist) for dist in resolve_func(requirements)]
File "~/miniconda3/envs/myenv/lib/python3.8/site-packages/liccheck/command_line.py", line 157, in <listcomp>
packages = [transform(dist) for dist in resolve_func(requirements)]
File "~/miniconda3/envs/myenv/lib/python3.8/site-packages/liccheck/requirements.py", line 51, in resolve
for dist in pkg_resources.working_set.resolve(requirements):
File "~/miniconda3/envs/myenv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 782, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (pandas 1.3.5 (~/miniconda3/envs/myenv/lib/python3.8/site-packages), Requirement.parse('pandas==1.3.3'))
I'll admit I don't fully understand the error. I'm also a bit confused as I didn't think liccheck
would check for dependency version conflicts - is there a way to turn this off maybe?
Many thanks for any help, and for this great tool!
Ran into an issue with python-dateutil
where the license="Dual License"
in setup.cfg
, but the trove classifiers report Apache 2.0 and BSD 3-clause. My license check failed because "Dual" was not an approved license.
My suggestion here would be to ignore the plain License: *
metadata if Classifier: License :: *
metadata exists, instead of combining them into a list:
# Current:
licenses = get_license(dist) + get_license_OSI_classifiers(dist)
# Suggestion
licenses = get_license_OSI_classifiers(dist) or get_license(dist) or ['Unknown']
simplejson
is another dual-license example, though their metadata specifies it a bit differently:
License: MIT License
License :: OSI Approved :: MIT License
License :: OSI Approved :: Academic Free License (AFL)
['BSD License\r'] (2.4.2) : Simplified BSD
dependencies:
python-dateutil << pandas
There is no need to force user to add -r requirements.txt
as this should be taken as default value unless someone defines it.
Same applies for the -s license_strategy.ini
, which should be assumed to be license_strategy.ini
. I would personally prefer a hidden file but I see you already picked this.
Looks like #48 broke CI. Unfortunately, there were no CI jobs on PR, which could help track the issue earlier. Fix is on the way.
Really a nice and helpful tool. Unfortunately if fails for pyqt packages.
gathering licenses...
Traceback (most recent call last):
...
license = regex_license.search(metadata).group('license')
AttributeError: 'NoneType' object has no attribute 'group'
Reproduce: ini file from repo, requirements.txt with the following content
PyQt5==5.11.3
PyQt5-sip==4.19.13
What is the difference between those two:
As a linter, in order to be usable to most python package maintainers, liccheck should be tested and functioning on all python versions, thus includes todays production python versions: py27,py34,py35,py36,py37 as of July 2018.
PS. I was really keen to test it and adopt it for pycontribs projects but current requirements make it impossible to use.
I currently get
'liccheck' is a package and cannot be directly executed
Please change that. If you want, I can make a PR for it; it's pretty easy.
Hi,
Thanks for this great package ! I just set it up for a project but found that many of my (sub)dependencies mentioned their license in an ill-defined way :
License string |
---|
Apache Software License; BSD License |
BSD |
BSD 3-Clause |
BSD License |
BSD License; GNU Library or Lesser General Public License (LGPL) |
BSD-3-Clause-LBNL |
Copyright (c) 2014 Gavin M. Roy |
GNU General Public License v3 (GPLv3) |
GNU Library or Lesser General Public License (LGPL) |
Historical Permission Notice and Disclaimer (HPND) |
ISC |
ISC License (ISCL) |
MIT |
MIT License |
MPL-2.0 |
Mozilla Public License 2.0 (MPL 2.0) |
Public Domain |
Python Software Foundation License |
http://www.apache.org/licenses/LICENSE-2.0 |
In this case, I thought it was more practical for me to provide literal strings in the authorized_licenses
under unauthorized_licenses
keys, from which two regex patterns are build.
I made a quick POC (see PR #94 ), if you judge this feature of interest I can finish it with tests and docs ;)
Hey,
this is a great package, it just enriched our CI pipeline :)
Is it possible to whitelist packages without pinning them to specific versions? I presumed:
[Authorized Packages]
uuid: "*"
or
[Authorized Packages]
uuid: "<1.5"
but this wasn't working...
Currently, the first thing that I see on PyPI is that the build is failing: https://pypi.org/project/liccheck/
Seems like authorized_packages
is just ignored.
requirements.txt
aiofiles==0.7.0
[...]
$ pipenv run liccheck
Loading .env environment variables...
gathering licenses...
131 packages and dependencies.
check authorized packages...
118 packages.
check unauthorized packages...
11 packages.
aiofiles (0.7.0): ['Other/Proprietary']
dependency:
aiofiles
[...]
[tool.liccheck]
authorized_licenses = [
'BSD',
'MIT',
'Python Software Foundation',
]
unauthorized_licenses = [
'Other/Proprietary',
]
[[tool.liccheck.authorized_packages]]
aiofiles = '0.7.0'
Here we see that liccheck
is complaining about aiofiles
version 0.7.0
, but that is clearly incorrect because it's included in the authorized_packages
section of pyproject.toml
.
I want to include liccheck into a CI pipeline. It would be nice if it created a formatted output. I propose to add a flag --report-to-stdout
which would create this output:
Liccheck checked 63 packages:
* 62 OK
* 1 UNKNOWN
State Package name License
--------------------------------------------------------------------------------
OK Flask 1.1.2 BSD-3-Clause
OK cryptography 3.4.6 BSD or Apache License, Version 2.0
OK paramiko 2.7.2 GNU Library or Lesser General Public Lice...
When project configuration was added the README was not updated to mention it, making use of it require some digging. Ideally the readme should cover both.
Packages on the python package index specify license information in one of two or both places:
In some packages, the classifier information is more general e.g: "BSD License", whereas the metadata contains information on the specific version of the license used "BSD License (BSD-3-Clause)". For example:
Since liccheck will always prefer the information retrieved from the classifiers, it is not possible to block specific versions of licenses. However, if it is possible to specify that I prefer to retrieve license information from the package metadata, I could whitelist (or blacklist) specific versions (like BSD-4-Clause). For example, the pip-licenses package supports this.
I'm happy to open a pull request with the necessary changes.
It seems the installation of liccheck 0.3.11 is broken:
$ pip install liccheck ~/tmp/liccheck {#}
Collecting liccheck
Downloading https://files.pythonhosted.org/packages/91/a7/83118d3ba708ebdf29c6bdd7a593d7297311492fe7ca63854ae0c8cbff20/liccheck-0.3.11.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-kdjtfgh6/liccheck/setup.py", line 20, in <module>
with open(path.join(here, 'requirements.txt'), encoding='utf-8') as f:
File "/usr/lib/python3.7/codecs.py", line 898, in open
file = builtins.open(filename, mode, buffering)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-install-kdjtfgh6/liccheck/requirements.txt'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-kdjtfgh6/liccheck/
from pip.download import PipSession
ImportError: No module named 'pip.download'
The or
-ed license check, added in #100 blindly splits the license name on the word "or".
This breaks the license check for projects like psycopg2
which has a license of GNU Library or Lesser General Public License (LGPL)
(this gets turned into ["GNU Library", "Lesser General Public License (LGPL)"]
by the get_license_names
function).
With the following requirements.txt
wand
yoyo-migrations
and licence_strategy.ini
# Authorized and unauthorized licenses in lower case
[Licenses]
authorized_licenses:
BSD
new BSD
BSD license
new BDS license
simplified BSD
Apache
Apache 2.0
Apache software license
gnu LGPL
LGPL with exceptions or zpl
ISC license
ISC license (ISCL)
MIT
MIT license
python software foundation license
zpl 2.1
unauthorized_licenses:
GPL v3
GPL
GNU General Public License v2 or later (GPLv2+)
[Authorized Packages]
# Python software license (see http://zesty.ca/python/uuid.README.txt)
uuid: 1.30
Unidecode: 0.4.20
I get the following result :
gathering licenses...
6 packages and dependencies.
check forbidden packages based on licenses...
1 forbidden packages :
Unidecode (0.4.20) : GPL ['GNU General Public License v2 or later (GPLv2+)']
dependencies:
Unidecode << python-slugify << yoyo-migrations
check authorized packages based on licenses...
4 packages.
check authorized packages...
1 packages.
check unknown licenses...
none
Unidecode
should not be matched in the forbidden packages and the execution should succeed
I am using poetry to manage my environment. poetry install
throws this error:
ParseVersionError Unable to parse "dev"
#62 seems to be at fault. It works fine in the build pipeline to create the source tarball and upload to pypi, but it is not installable because setup.py still specifies the version as dev
when installing
Steps to reproduce:
pip install setuptools==40.0.0
cat strategy.ini
:[Licenses]
authorized_licenses:
mit
cat requirements.txt
setuptools>=41.0.0
liccheck -s strategy.ini -r requirements.txt
gathering licenses...
Traceback (most recent call last):
File "/private/tmp/liccheck/.venv/bin/liccheck", line 8, in <module>
sys.exit(main())
File "/private/tmp/liccheck/.venv/lib/python3.8/site-packages/liccheck/command_line.py", line 301, in main
sys.exit(run(args))
File "/private/tmp/liccheck/.venv/lib/python3.8/site-packages/liccheck/command_line.py", line 296, in run
return process(args.requirement_txt_file, strategy, args.level)
File "/private/tmp/liccheck/.venv/lib/python3.8/site-packages/liccheck/command_line.py", line 231, in process
pkg_info = get_packages_info(requirement_file)
File "/private/tmp/liccheck/.venv/lib/python3.8/site-packages/liccheck/command_line.py", line 152, in get_packages_info
packages = [transform(dist) for dist in pkg_resources.working_set.resolve(requirements)]
File "/private/tmp/liccheck/.venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 782, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (setuptools 40.0.0 (/private/tmp/liccheck/.venv/lib/python3.8/site-packages), Requirement.parse('setuptools>=41.0.0'))
pip 20 has moved some code around specifically _internal/download.py no longer exists and seems to have been split into _internal/network/download.py and _internal/network/session.py. I've created a PR that I believe resolves the issue: #37
Feedback welcome
as it is possible to run with pre-commit as one of hooks
Example:
https://github.com/pre-commit/pre-commit-hooks/blob/master/.pre-commit-hooks.yaml
Enabling liccheck as a pre-commit hook can be done by just adding simple yaml file in the root of the repository which declares the hook so users can use it.
See https://github.com/pre-commit/pre-commit/blob/master/.pre-commit-hooks.yaml
I may make a pull request to add if you want, it is very easy.
Add an option to get a report of all dependencies with their license, ideally as HTML but text would be sufficient, too.
The report should include the name of the dependency, the version and the first matching license name from authorized_licenses plus the status OK.
If there is no match, it should look for a match in unauthorized_licenses. If there is one, the dependency should be reported as UNAUTH.
If there is no match, the status should be UNKNOWN
Not all packages have OSI approved licenses, so their metadata wont be found by a search for Classifier: License :: OSI Approved ::
.
Here's the list of all standard trove classifiers: https://pypi.org/classifiers/
Seems like the solution would be to stop looking for OSI Approved
specifically (make that part of the regex optional).
I'd like to be able to only allowlist/blocklist specific licenses by their SPDX identifiers (e.g. Apache-2.0
), as opposed to the various forms that pop up in packages:
apache 2.0
apache software 2.0
apache license 2.0
apache software license 2.0
the idkapache software license 2.0
Full list of SPDX identifiers is at: https://spdx.org/licenses/
This project has a similar mapping of frequently used identifiers to their proper SPDX identifier:
https://github.com/NFJones/pipoe/blob/master/pipoe/licenses.py
Of course, this only gets you so far as some projects have ambiguous metadata that doesn't explicitly specify the license version (like dateutil
for example). Perhaps this could be part of the normal/cautious/paranoid checking levels where "normal" might map "apache" to any version, but "paranoid" would not match unless it included a version number.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.