webscopeio / license.sh Goto Github PK

View Code? Open in Web Editor NEW

41.0 17.0 6.0 9.58 MB

License checker tool - We're in a beta phase, any help is appreciated, especially reporting issues

Home Page: https://webscopeio.github.io/license.sh

License: MIT License

Python 99.70% JavaScript 0.30%

license-management license-checking python3

license.sh's Introduction

Check licenses of your software.

The goal of this repository is to create a simple utility that you can simply run in your repository to check compliance of your 3rd party dependencies.

https://webscopeio.github.io/license.sh/#/

License compliance tool for your software. We're currently in Beta phase, please feel free to help us with providing bugreports & submitting PRs.

Installation

🐍 Install pip https://pip.pypa.io/en/stable/installing/
💻 pip install license-sh

Usage

Run the following command inside your repository.

license-sh

Supported Lanaguages & Package managers

Javascript
- NPM
- Yarn
Python
- pipenv
Java
- maven

Contribution guide

Available commands:

pipenv run check-types
pipenv run lint
pipenv run test

You need to set-up a repository and install dependencies using pipenv.

# clone the repo
$ git clone [email protected]:webscopeio/license.sh.git
# install pipenv
$ pipenv install
# run the project
$ pipenv run ./license-sh <path_to_test_project>

Running tests

pipenv run python -m unittest

Documentation

pipenv run mkdocs serve

Packaging

Read https://packaging.python.org/tutorials/packaging-projects/

Run python3 setup.py sdist bdist_wheel.

It will generate .tar.gz file in dist/ directory which you can
install with pip install <file.tar.gz>

license.sh's People

Contributors

Stargazers

Watchers

Forkers

jayvdb jasonshih myhall pombredanne jankoritak jeremi

license.sh's Issues

Silent output

Add argument for silent output(Output without progress). So it is easily stored in file

Lets say test is not able to resolve certain license. However, I am able to find online its license and confirm that it is ok for my scenario. I should be able to add it to config file to prevent further fails.

ignore: [
'[email protected]',
'[email protected]'
]

When I try this on a directory without any config, and either prompt_toolkit 2.0.9 or 2.0.10 with questionary 1.4.0, I get an error

> PYTHONPATH=~/projects/python/license.sh python3 ~/projects/python/license.sh/license-sh config
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/cache.py", line 34, in get
    return self._data[key]
KeyError: 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jayvdb/projects/python/license.sh/license-sh", line 72, in <module>
    config_cmd(path, config)
  File "/home/jayvdb/projects/python/license.sh/license_sh/commands/__init__.py", line 69, in config_cmd
    answers = questionary.prompt(questions)
  File "/usr/lib/python3.7/site-packages/questionary/prompt.py", line 97, in prompt
    answer = question.unsafe_ask(patch_stdout)
  File "/usr/lib/python3.7/site-packages/questionary/question.py", line 59, in unsafe_ask
    return self.application.run()
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/application/application.py", line 709, in run
    return run()
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/application/application.py", line 683, in run
    return f.result()
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/eventloop/future.py", line 149, in result
    raise self._exception
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/eventloop/coroutine.py", line 90, in step_next
    new_f = coroutine.throw(exc)
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/application/application.py", line 658, in _run_async2
    result = yield f
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/eventloop/coroutine.py", line 86, in step_next
    new_f = coroutine.send(None)
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/application/application.py", line 601, in _run_async
    self._redraw()
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/application/application.py", line 444, in _redraw
    self.renderer.render(self, self.layout)
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/renderer.py", line 546, in render
    layout.container.preferred_height(size.columns, size.rows).preferred)
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/layout/containers.py", line 245, in preferred_height
    for c in self._all_children]
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/layout/containers.py", line 245, in <listcomp>
    for c in self._all_children]
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/layout/containers.py", line 2179, in preferred_height
    return self.content.preferred_height(width, max_available_height)
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/layout/containers.py", line 1362, in preferred_height
    dont_extend=self.dont_extend_height())
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/layout/containers.py", line 1380, in _merge_dimensions
    preferred = get_preferred()
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/layout/containers.py", line 1357, in preferred_content_height
    self.get_line_prefix)
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/layout/controls.py", line 308, in preferred_height
    content = self.create_content(width, None)
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/layout/controls.py", line 313, in create_content
    fragments_with_mouse_handlers = self._get_formatted_text_cached()
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/layout/controls.py", line 296, in _get_formatted_text_cached
    lambda: to_formatted_text(self.text, self.style))
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/cache.py", line 37, in get
    value = getter_func()
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/layout/controls.py", line 296, in <lambda>
    lambda: to_formatted_text(self.text, self.style))
  File "/usr/lib/python3.7/site-packages/prompt_toolkit/formatted_text/base.py", line 45, in to_formatted_text
    return to_formatted_text(value(), style=style)
  File "/usr/lib/python3.7/site-packages/questionary/prompts/common.py", line 306, in _get_choice_tokens
    tokens.pop()  # Remove last newline.
IndexError: pop from empty list

Note I get a different error if no args are given

> PYTHONPATH=~/projects/python/license.sh python3 ~/projects/python/license.sh/license-sh
Traceback (most recent call last):
  File "/home/jayvdb/projects/python/license.sh/license-sh", line 102, in <module>
    dep_tree, WHITELIST
  File "/home/jayvdb/projects/python/license.sh/license-sh", line 50, in print_dependency_tree_with_licenses
    dep_tree, whitelist=whitelist
  File "/home/jayvdb/projects/python/license.sh/license_sh/helpers.py", line 91, in annotate_dep_tree
    node.license_normalized = normalize_license_expression(node.license)
AttributeError: 'NoneType' object has no attribute 'license'

Originally posted by @jayvdb in #44 (comment)

More granular error reporting for misconfiguration

The package requires the user to provide .license-sh.json file. There are currently 3 points of failure that can occur.

User does not provide .license-sh.json at all
User does provide .license-sh.json, but misconfigures projects property
User does provide .license-sh.json, but misconfigures whitelist property

The package should be able to detect and report all 3 cases of misconfiguration

Maven dependencies

jar to resolve maven dependency tree is missing its dependencies

Prerequisites warning

#81

Check if prerequisites are met and throw an error if not

Fatal error on custom config path

I tried to specify custom config path by using:
license-sh --config ../.license-sh.json

I got this fatal error afterwards:

===========
Initiated License.sh check for Maven project hazelcast-cloud located at .
===========
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/resources.py", line 188, in path
    yield Path(reader.resource_path(resource))
  File "/usr/local/lib/python3.7/site-packages/license_sh/runners/maven/__init__.py", line 40, in get_dependency_tree_xml
    subprocess.run(["mvn", "install", f"-f={maven_path}"], capture_output=not debug)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 472, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 1522, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'mvn': 'mvn'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/license-sh", line 2, in <module>
    from license_sh import __main__
  File "/usr/local/lib/python3.7/site-packages/license_sh/__main__.py", line 25, in <module>
    run_license_sh(arguments)
  File "/usr/local/lib/python3.7/site-packages/license_sh/commands/run_license_sh.py", line 64, in run_license_sh
    dep_tree, license_map = runner.check()
  File "/usr/local/lib/python3.7/site-packages/license_sh/runners/maven/__init__.py", line 209, in check
    xml_tree = get_dependency_tree_xml(self.directory, self.debug)
  File "/usr/local/lib/python3.7/site-packages/license_sh/runners/maven/__init__.py", line 40, in get_dependency_tree_xml
    subprocess.run(["mvn", "install", f"-f={maven_path}"], capture_output=not debug)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py", line 161, in __exit__
    raise RuntimeError("generator didn't stop after throw()")
RuntimeError: generator didn't stop after throw()

It probably expects just a path to a directory, but I think it should handle a path to a config file too (parameter is called --config not --configFolder).

Npm run fail

https://gitlab.com/license_test_group/satnogs-db

Traceback (most recent call last):
  File "/usr/local/bin/license-sh", line 2, in <module>
    from license_sh import __main__
  File "/usr/local/lib/python3.8/site-packages/license_sh/__main__.py", line 24, in <module>
    run_license_sh(arguments)
  File "/usr/local/lib/python3.8/site-packages/license_sh/commands/run_license_sh.py", line 58, in run_license_sh
    dep_tree, license_map = runner.check()
  File "/usr/local/lib/python3.8/site-packages/license_sh/runners/npm/__init__.py", line 142, in check
    license_map = fetch_npm_licenses(flat_dependencies)
  File "/usr/local/lib/python3.8/site-packages/license_sh/runners/runners_shared.py", line 40, in fetch_npm_licenses
    asyncio.run(fetch_concurrent(urls))
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 612, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/license_sh/runners/runners_shared.py", line 33, in fetch_concurrent
    license_map[f"{page['name']}@{version}"] = extract_npm_license(
KeyError: 'name'

KeyError

[2020-02-28T13:31:49Z] Traceback (most recent call last):
  File "/usr/local/bin/license-sh", line 2, in <module>
    from license_sh import __main__
  File "/usr/local/lib/python3.8/site-packages/license_sh/__main__.py", line 26, in <module>
    run_license_sh(arguments)
  File "/usr/local/lib/python3.8/site-packages/license_sh/commands/run_license_sh.py", line 67, in run_license_sh
    dep_tree = run_check(project_to_check, path, silent, debug)
  File "/usr/local/lib/python3.8/site-packages/license_sh/runners/__init__.py", line 20, in run_check
    return runner(path, silent, debug).check()
  File "/usr/local/lib/python3.8/site-packages/license_sh/runners/npm/__init__.py", line 138, in check
    dep_tree = get_dependency_tree(package_json, all_dependencies)
  File "/usr/local/lib/python3.8/site-packages/license_sh/runners/npm/__init__.py", line 79, in get_dependency_tree
    dependency = package_lock_tree[dep_name]
KeyError: '@fortawesome/fontawesome'

pipenv check

I am getting error. Probably some package is missing from pipenv

Multiple licenses

Handle and how to handle multiple licenses.

Maven(Beta) now handles them with AND

Implement proper --debug mode

--debug mode should render all output from 3rd party dependencies (yarn, mvn, etc).

No such file or directory: 'yarn': 'yarn'

[theodore@milacik license.sh]$ license-sh --version
1.0.15

[theodore@milacik license.sh]$ license-sh ../devenv/
===========
Initiated License.sh check for YARN project devenv located at ../devenv/
===========
Traceback (most recent call last):
  File "/usr/bin/license-sh", line 2, in <module>
    from license_sh import __main__
  File "/usr/lib/python3.7/site-packages/license_sh/__main__.py", line 24, in <module>
    run_license_sh(arguments)
  File "/usr/lib/python3.7/site-packages/license_sh/commands/run_license_sh.py", line 68, in run_license_sh
    dep_tree, license_map = runner.check()
  File "/usr/lib/python3.7/site-packages/license_sh/runners/yarn/__init__.py", line 336, in check
    package_map = parse_yarn_lock(get_yarn_lock_json(self.directory))
  File "/usr/lib/python3.7/site-packages/license_sh/runners/yarn/__init__.py", line 58, in get_yarn_lock_json
    stderr=subprocess.PIPE,
  File "/usr/lib/python3.7/subprocess.py", line 488, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.7/subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.7/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'yarn': 'yarn'

"ignored_packages" non pipenv

Only pipenv value is used, others are ignored

Set up test coverage flow including the badge

We should be aware of our test coverage and try to get it to a reasonable level.
The badge on the GH frontpage could also be nice.

Human-readable 3rd party licenses declaration

It would be nice to have a feature that exports list of 3rd party packages along with their license in a human-readable format, that can be used as a declaration point.

For example plain-text.

Fatal error when using both NPM and YARN

I have a project that's using both NPM and YARN (not used anymore, packages not installed, we just forgot to delete the yarn.lock file).

After I removed yarn.lock file it worked.

Maybe it could handle the error with better UX?

Traceback (most recent call last):
  File "/usr/local/bin/license-sh", line 2, in <module>
    from license_sh import __main__
  File "/usr/local/lib/python3.7/site-packages/license_sh/__main__.py", line 25, in <module>
    run_license_sh(arguments)
  File "/usr/local/lib/python3.7/site-packages/license_sh/commands/run_license_sh.py", line 69, in run_license_sh
    dep_tree, license_map = runner.check()
  File "/usr/local/lib/python3.7/site-packages/license_sh/runners/yarn/__init__.py", line 336, in check
    package_map = parse_yarn_lock(get_yarn_lock_json(self.directory))
  File "/usr/local/lib/python3.7/site-packages/license_sh/runners/yarn/__init__.py", line 70, in get_yarn_lock_json
    stderr=subprocess.PIPE,
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Better UI to handle bad licenses

After the system detects bad license, we should provide a way to whitelist this license in the config.

Package-lock without dependencies

===========
Initiated License.sh check for pipenv project located at ../*****
===========
Traceback (most recent call last):
  File "/usr/bin/license-sh", line 2, in <module>
    from license_sh import __main__
  File "/usr/lib/python3.8/site-packages/license_sh/__main__.py", line 24, in <module>
    run_license_sh(arguments)
  File "/usr/lib/python3.8/site-packages/license_sh/commands/run_license_sh.py", line 58, in run_license_sh
    dep_tree, license_map = runner.check()
  File "/usr/lib/python3.8/site-packages/license_sh/runners/npm/__init__.py", line 121, in check
    all_dependencies = package_lock["dependencies"]
KeyError: 'dependencies'

Support license preset & whitelist & blacklist

The goal is to support something like a preset of licenses that are allowed (e.g. enterprise preset)

The resulting whitelist will be constructed as follows.

All licenses taken from the preset(s)
- licenses from the whitelist
- licenses taken from the blacklist

This needs to be properly documented in the documentation.

Report inconsistency for npm & yarn

We should have a special message about an inconsistency of yarn.lock / package-lock.json and package.json

Multi language repo

If you have a repo that have Pipfile.lock and 'package-lock.json' the output will look something like this

$ license-sh ../XXX/ -t
===========
Initiated License.sh check for pipenv project located at ../XXX/
===========
⠸ Fetching license info from pypi ...
===========
Initiated License.sh check for NPM project YYY located at ../XXX/
===========

All results on python check will be ignored and results of npm check will be displayed

Documentation is missing an "Installation" section

Documentation does not have an "Installation" section. It breaks the first usage flow because users need to search for it in Github repo.

Config path option

Add option to pass path to the config

license-sh ../superProject/frontend --config ../superConfig

Documentation links to Slack and Forum are dead

There are supposed to links to License.sh Slack and forum, both are dead.

We can keep the slack one, but update it.
Removed the forum one.

Init nodemodules

Init should ignore node_modules

{
  "projects": {
    ".": [
      "yarn"
    ],
    "./node_modules/combined-stream": [
      "yarn"
    ],
    "./node_modules/form-data": [
      "yarn"
    ],
    "./node_modules/mock-require": [
      "yarn"
    ],
    "./node_modules/tslint/node_modules/tsutils": [
      "yarn"
    ],
    "./node_modules/tsutils": [
      "yarn"
    ],
    "./node_modules/uri-js": [
      "yarn"
    ]
  }
}

Yarn run fail

https://github.com/license-test/devenv

Traceback (most recent call last):
  File "/usr/local/bin/license-sh", line 2, in <module>
    from license_sh import __main__
  File "/usr/local/lib/python3.8/site-packages/license_sh/__main__.py", line 24, in <module>
    run_license_sh(arguments)
  File "/usr/local/lib/python3.8/site-packages/license_sh/commands/run_license_sh.py", line 68, in run_license_sh
    dep_tree, license_map = runner.check()
  File "/usr/local/lib/python3.8/site-packages/license_sh/runners/yarn/__init__.py", line 344, in check
    license_map = fetch_npm_licenses(flat_dependencies)
  File "/usr/local/lib/python3.8/site-packages/license_sh/runners/runners_shared.py", line 40, in fetch_npm_licenses
    asyncio.run(fetch_concurrent(urls))
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 612, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/license_sh/runners/runners_shared.py", line 33, in fetch_concurrent
    license_map[f"{page['name']}@{version}"] = extract_npm_license(
  File "/usr/local/lib/python3.8/site-packages/license_sh/helpers.py", line 78, in extract_npm_license
    return get_npm_license_from_licenses_array(licenses_array)
  File "/usr/local/lib/python3.8/site-packages/license_sh/helpers.py", line 46, in get_npm_license_from_licenses_array
    license_item_type = license_item.get("type", UNKNOWN)
AttributeError: 'str' object has no attribute 'get'

pipenv install

I am getting

(license.sh) [theodore@milacik license.sh]$ pipenv installPipfile.lock (a61080) out of date, updating to (6bb451)…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
✘ Locking Failed! 
[pipenv.exceptions.ResolutionFailure]:   File "/home/theodore/.local/lib/python3.7/site-packages/pipenv/resolver.py", line 69, in resolve
[pipenv.exceptions.ResolutionFailure]:       req_dir=requirements_dir
[pipenv.exceptions.ResolutionFailure]:   File "/home/theodore/.local/lib/python3.7/site-packages/pipenv/utils.py", line 726, in resolve_deps
[pipenv.exceptions.ResolutionFailure]:       req_dir=req_dir,
[pipenv.exceptions.ResolutionFailure]:   File "/home/theodore/.local/lib/python3.7/site-packages/pipenv/utils.py", line 480, in actually_resolve_deps
[pipenv.exceptions.ResolutionFailure]:       resolved_tree = resolver.resolve()
[pipenv.exceptions.ResolutionFailure]:   File "/home/theodore/.local/lib/python3.7/site-packages/pipenv/utils.py", line 395, in resolve
[pipenv.exceptions.ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]:       pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches prompt-toolkit<2.1.0,==1.0.14,>=2.0.0
[pipenv.exceptions.ResolutionFailure]:       Tried: 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.8, 0.9, 0.10, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.20, 0.21, 0.22, 0.23, 0.25, 0.26, 0.28, 0.30, 0.31, 0.32, 0.32, 0.32, 0.33, 0.33, 0.33, 0.34, 0.34, 0.34, 0.35, 0.35, 0.35, 0.36, 0.36, 0.36, 0.37, 0.37, 0.37, 0.38, 0.38, 0.38, 0.39, 0.39, 0.39, 0.40, 0.40, 0.40, 0.41, 0.41, 0.41, 0.42, 0.42, 0.42, 0.43, 0.43, 0.43, 0.44, 0.44, 0.44, 0.45, 0.45, 0.45, 0.46, 0.46, 0.46, 0.47, 0.47, 0.47, 0.48, 0.48, 0.48, 0.49, 0.49, 0.49, 0.50, 0.50, 0.50, 0.51, 0.51, 0.51, 0.52, 0.52, 0.52, 0.53, 0.53, 0.53, 0.54, 0.54, 0.54, 0.55, 0.55, 0.55, 0.56, 0.56, 0.56, 0.57, 0.57, 0.57, 0.58, 0.58, 0.58, 0.59, 0.59, 0.59, 0.60, 0.60, 0.60, 1.0.0, 1.0.0, 1.0.0, 1.0.1, 1.0.1, 1.0.1, 1.0.2, 1.0.2, 1.0.2, 1.0.3, 1.0.3, 1.0.3, 1.0.4, 1.0.4, 1.0.4, 1.0.5, 1.0.5, 1.0.5, 1.0.6, 1.0.6, 1.0.6, 1.0.7, 1.0.7, 1.0.7, 1.0.8, 1.0.8, 1.0.8, 1.0.9, 1.0.9, 1.0.9, 1.0.10, 1.0.10, 1.0.10, 1.0.13, 1.0.13, 1.0.13, 1.0.14, 1.0.14, 1.0.14, 1.0.15, 1.0.15, 1.0.15, 1.0.16, 1.0.16, 1.0.16, 2.0.1, 2.0.1, 2.0.1, 2.0.2, 2.0.2, 2.0.2, 2.0.3, 2.0.3, 2.0.3, 2.0.4, 2.0.4, 2.0.4, 2.0.5, 2.0.5, 2.0.5, 2.0.6, 2.0.6, 2.0.6, 2.0.7, 2.0.7, 2.0.7, 2.0.8, 2.0.8, 2.0.8, 2.0.9, 2.0.9, 2.0.9
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ERROR: Could not find a version that matches prompt-toolkit<2.1.0,==1.0.14,>=2.0.0
Tried: 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.8, 0.9, 0.10, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.20, 0.21, 0.22, 0.23, 0.25, 0.26, 0.28, 0.30, 0.31, 0.32, 0.32, 0.32, 0.33, 0.33, 0.33, 0.34, 0.34, 0.34, 0.35, 0.35, 0.35, 0.36, 0.36, 0.36, 0.37, 0.37, 0.37, 0.38, 0.38, 0.38, 0.39, 0.39, 0.39, 0.40, 0.40, 0.40, 0.41, 0.41, 0.41, 0.42, 0.42, 0.42, 0.43, 0.43, 0.43, 0.44, 0.44, 0.44, 0.45, 0.45, 0.45, 0.46, 0.46, 0.46, 0.47, 0.47, 0.47, 0.48, 0.48, 0.48, 0.49, 0.49, 0.49, 0.50, 0.50, 0.50, 0.51, 0.51, 0.51, 0.52, 0.52, 0.52, 0.53, 0.53, 0.53, 0.54, 0.54, 0.54, 0.55, 0.55, 0.55, 0.56, 0.56, 0.56, 0.57, 0.57, 0.57, 0.58, 0.58, 0.58, 0.59, 0.59, 0.59, 0.60, 0.60, 0.60, 1.0.0, 1.0.0, 1.0.0, 1.0.1, 1.0.1, 1.0.1, 1.0.2, 1.0.2, 1.0.2, 1.0.3, 1.0.3, 1.0.3, 1.0.4, 1.0.4, 1.0.4, 1.0.5, 1.0.5, 1.0.5, 1.0.6, 1.0.6, 1.0.6, 1.0.7, 1.0.7, 1.0.7, 1.0.8, 1.0.8, 1.0.8, 1.0.9, 1.0.9, 1.0.9, 1.0.10, 1.0.10, 1.0.10, 1.0.13, 1.0.13, 1.0.13, 1.0.14, 1.0.14, 1.0.14, 1.0.15, 1.0.15, 1.0.15, 1.0.16, 1.0.16, 1.0.16, 2.0.1, 2.0.1, 2.0.1, 2.0.2, 2.0.2, 2.0.2, 2.0.3, 2.0.3, 2.0.3, 2.0.4, 2.0.4, 2.0.4, 2.0.5, 2.0.5, 2.0.5, 2.0.6, 2.0.6, 2.0.6, 2.0.7, 2.0.7, 2.0.7, 2.0.8, 2.0.8, 2.0.8, 2.0.9, 2.0.9, 2.0.9
There are incompatible versions in the resolved dependencies.
[pipenv.exceptions.ResolutionFailure]:       req_dir=requirements_dir
[pipenv.exceptions.ResolutionFailure]:   File "/home/theodore/.local/lib/python3.7/site-packages/pipenv/utils.py", line 726, in resolve_deps
[pipenv.exceptions.ResolutionFailure]:       req_dir=req_dir,
[pipenv.exceptions.ResolutionFailure]:   File "/home/theodore/.local/lib/python3.7/site-packages/pipenv/utils.py", line 480, in actually_resolve_deps
[pipenv.exceptions.ResolutionFailure]:       resolved_tree = resolver.resolve()
[pipenv.exceptions.ResolutionFailure]:   File "/home/theodore/.local/lib/python3.7/site-packages/pipenv/utils.py", line 395, in resolve
[pipenv.exceptions.ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]:       pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches prompt-toolkit<2.1.0,==1.0.14,>=2.0.0
[pipenv.exceptions.ResolutionFailure]:       Tried: 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.8, 0.9, 0.10, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.20, 0.21, 0.22, 0.23, 0.25, 0.26, 0.28, 0.30, 0.31, 0.32, 0.32, 0.32, 0.33, 0.33, 0.33, 0.34, 0.34, 0.34, 0.35, 0.35, 0.35, 0.36, 0.36, 0.36, 0.37, 0.37, 0.37, 0.38, 0.38, 0.38, 0.39, 0.39, 0.39, 0.40, 0.40, 0.40, 0.41, 0.41, 0.41, 0.42, 0.42, 0.42, 0.43, 0.43, 0.43, 0.44, 0.44, 0.44, 0.45, 0.45, 0.45, 0.46, 0.46, 0.46, 0.47, 0.47, 0.47, 0.48, 0.48, 0.48, 0.49, 0.49, 0.49, 0.50, 0.50, 0.50, 0.51, 0.51, 0.51, 0.52, 0.52, 0.52, 0.53, 0.53, 0.53, 0.54, 0.54, 0.54, 0.55, 0.55, 0.55, 0.56, 0.56, 0.56, 0.57, 0.57, 0.57, 0.58, 0.58, 0.58, 0.59, 0.59, 0.59, 0.60, 0.60, 0.60, 1.0.0, 1.0.0, 1.0.0, 1.0.1, 1.0.1, 1.0.1, 1.0.2, 1.0.2, 1.0.2, 1.0.3, 1.0.3, 1.0.3, 1.0.4, 1.0.4, 1.0.4, 1.0.5, 1.0.5, 1.0.5, 1.0.6, 1.0.6, 1.0.6, 1.0.7, 1.0.7, 1.0.7, 1.0.8, 1.0.8, 1.0.8, 1.0.9, 1.0.9, 1.0.9, 1.0.10, 1.0.10, 1.0.10, 1.0.13, 1.0.13, 1.0.13, 1.0.14, 1.0.14, 1.0.14, 1.0.15, 1.0.15, 1.0.15, 1.0.16, 1.0.16, 1.0.16, 2.0.1, 2.0.1, 2.0.1, 2.0.2, 2.0.2, 2.0.2, 2.0.3, 2.0.3, 2.0.3, 2.0.4, 2.0.4, 2.0.4, 2.0.5, 2.0.5, 2.0.5, 2.0.6, 2.0.6, 2.0.6, 2.0.7, 2.0.7, 2.0.7, 2.0.8, 2.0.8, 2.0.8, 2.0.9, 2.0.9, 2.0.9
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ERROR: Could not find a version that matches prompt-toolkit<2.1.0,==1.0.14,>=2.0.0
Tried: 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.8, 0.9, 0.10, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.20, 0.21, 0.22, 0.23, 0.25, 0.26, 0.28, 0.30, 0.31, 0.32, 0.32, 0.32, 0.33, 0.33, 0.33, 0.34, 0.34, 0.34, 0.35, 0.35, 0.35, 0.36, 0.36, 0.36, 0.37, 0.37, 0.37, 0.38, 0.38, 0.38, 0.39, 0.39, 0.39, 0.40, 0.40, 0.40, 0.41, 0.41, 0.41, 0.42, 0.42, 0.42, 0.43, 0.43, 0.43, 0.44, 0.44, 0.44, 0.45, 0.45, 0.45, 0.46, 0.46, 0.46, 0.47, 0.47, 0.47, 0.48, 0.48, 0.48, 0.49, 0.49, 0.49, 0.50, 0.50, 0.50, 0.51, 0.51, 0.51, 0.52, 0.52, 0.52, 0.53, 0.53, 0.53, 0.54, 0.54, 0.54, 0.55, 0.55, 0.55, 0.56, 0.56, 0.56, 0.57, 0.57, 0.57, 0.58, 0.58, 0.58, 0.59, 0.59, 0.59, 0.60, 0.60, 0.60, 1.0.0, 1.0.0, 1.0.0, 1.0.1, 1.0.1, 1.0.1, 1.0.2, 1.0.2, 1.0.2, 1.0.3, 1.0.3, 1.0.3, 1.0.4, 1.0.4, 1.0.4, 1.0.5, 1.0.5, 1.0.5, 1.0.6, 1.0.6, 1.0.6, 1.0.7, 1.0.7, 1.0.7, 1.0.8, 1.0.8, 1.0.8, 1.0.9, 1.0.9, 1.0.9, 1.0.10, 1.0.10, 1.0.10, 1.0.13, 1.0.13, 1.0.13, 1.0.14, 1.0.14, 1.0.14, 1.0.15, 1.0.15, 1.0.15, 1.0.16, 1.0.16, 1.0.16, 2.0.1, 2.0.1, 2.0.1, 2.0.2, 2.0.2, 2.0.2, 2.0.3, 2.0.3, 2.0.3, 2.0.4, 2.0.4, 2.0.4, 2.0.5, 2.0.5, 2.0.5, 2.0.6, 2.0.6, 2.0.6, 2.0.7, 2.0.7, 2.0.7, 2.0.8, 2.0.8, 2.0.8, 2.0.9, 2.0.9, 2.0.9
There are incompatible versions in the resolved dependencies.

Any idea whats wrong?

'LicenseWithExceptionSymbol' object has no attribute 'key'

GNU Library General Public License v2.1 OR later
Eclipse Public License v1.0 AND Eclipse Distribution License v. 1.0
MPL 1.1 AND LGPL 2.1 AND Apache License 2.0
GNU Lesser General Public License v2.1 OR later
Eclipse Public License - v 1.0 AND GNU Lesser General Public License
Eclipse Public License - v 1.0 AND GNU Lesser General Public License
CDDL + GPLv2 WITH classpath exception
CDDL AND Apache License 2.0
CDDL + GPLv2 WITH classpath exception

Are getting 'LicenseWithExceptionSymbol' object has no attribute 'key' in is_license_ok function

Ignore packages documentation

There is missing documentation on how to use "ignored_packages" feathure

Tree option exit

license-sh . exits with 0, but license-sh . -t exits with 1

Exit with 1 error code if bad licenses are found

Whitelisting unknown

One should not be able to whitelist unknown at least not via CLI

Npm licenses

Currently fetching of licenses search for license property. However, some packages uses licenses with an array.

"licenses":[{"type":"MIT","url":"http://github.com/mscdex/streamsearch/raw/master/LICENSE"}]

License.sh generate file

Right now, I'm creating a Django application and in the guide I need to generate a License file. https://docs.djangoproject.com/en/3.0/intro/reusable-apps/ step 4.

It'd be nice if we could do something like this with our utility.

Replace PyInquirer

PyInquirer requires prompt_toolkit 1.x , which is now incompatible with most other parts of the Python ecosystem, and it appears unlikely to become prompt_toolkit 2.x compatible any time soon.

https://github.com/tmbo/questionary was created by people who have given up waiting for PyInquirer to become prompt_toolkit 2.x compatible.

"ignored_packages" version

"ignored_packages" property should take version of the package as the license might change on version update

Support mono repo

Support mono repo, "projects" parameter in script.

It is confusing to set projects to check, but they are not checked. This is supported only in integration.

User has to set projects to check but to test it he has to go to every set up project and via config property set config.

Create temporary directory when checking mvn project

Hi,

While checking mvn projects, we create tmp folder .license-shTestDir in current directory. I think this is not the best approach and we should try to avoid touching code dir as much as possible.

Let's try to create temdir if possible https://docs.python.org/3.8/library/tempfile.html

yaspin is outputing text in json mode

When outputing json, we shuld not output anything to std output.

Since we're using yaspin, we can provide a null context in certain cases to fix this

from contextlib import nullcontext

with (open(file) if file else nullcontext()) as FILE:
    # Do something with `FILE`
    pass

CLI whitelisting complex licenses

If a package has license for example MIT AND Zlib the CLI will give you option to whitelist "MIT AND Zlib". However, is_license_ok function splits complex licenses on AND and OR and therefore this won't work.

You need to manually add "MIT" and "Zlib" license to the license whitelist config to make it work

Document ignore_packages option

#46

We need to add this to the documentation.

Npm unknown license

Npm runner says that license is unknown even when the license is specified in the version

Error after whitelisting licenses

After I selected licenses I want to whitelist I got this fatal error:

? 📋 Which licenses do you want to whitelist?  done (11 selections)                                                                                                                                                                                                                                                                                                              
Traceback (most recent call last):
  File "/usr/local/bin/license-sh", line 2, in <module>
    from license_sh import __main__
  File "/usr/local/lib/python3.7/site-packages/license_sh/__main__.py", line 24, in <module>
    run_license_sh(arguments)
  File "/usr/local/lib/python3.7/site-packages/license_sh/commands/run_license_sh.py", line 102, in run_license_sh
    get_full_tree=tree,
ValueError: too many values to unpack (expected 2)

Environment

macOS - 10.15.2 (19C57) (Catalina)
Python - 3.7.3
license-sh - 1.0.16

Duplicate config created

I had a license.sh config file in the root of my monorepo. I launched this command inside one of subdirectories (./frontend):
license-sh --config ../

Process finished successfully, but it created a duplicate config (./frontend/.license-sh.json) instead of modifying the config in the root of monorepo.

Add main

> python3 -m license_sh
/usr/bin/python3: No module named license_sh.__main__; 'license_sh' is a package and cannot be directly executed

Segmentation fault (core dumped)

[theodore@milacik license-sh]$ license-sh -o json
Segmentation fault (core dumped)

[theodore@milacik license-sh]$ license-sh
===========
Initiated License.sh check for NPM project license-sh located at .
===========
⠋ Fetching license info from npm ...Segmentation fault (core dumped)

This is not fully deterministic, happens in ~20%

[theodore@milacik license-sh]$ license-sh --version
1.0.13
[theodore@milacik license-sh]$ python3 --version
Python 3.8.1
[theodore@milacik license-sh]$ pip3 --version
pip 19.2.3 from /usr/lib/python3.8/site-packages/pip (python 3.8)

Yarn issue

S4n60w3n/phaser/master/9db9511bf106c0e9f7523b1b2d32eaf35ad322d3

Traceback (most recent call last):
  File "/usr/local/bin/license-sh", line 69, in <module>
    dep_tree, license_map = runner.check()
  File "/usr/local/lib/python3.8/site-packages/license_sh/runners/yarn/__init__.py", line 332, in check
    dep_tree = get_dependency_tree(flat_tree, package_json, package_map)
  File "/usr/local/lib/python3.8/site-packages/license_sh/runners/yarn/__init__.py", line 283, in get_dependency_tree
    version=dependency.get("version"),
AttributeError: 'NoneType' object has no attribute 'get'

Check without project

Running check on wrong directory gives

(license.sh) [theodore@milacik license.sh]$ ./license-sh ..
Config file '../.license-sh.json' not found...
Traceback (most recent call last):
  File "./license-sh", line 2, in <module>
    from license_sh import __main__
  File "/home/theodore/Workspace/license.sh/license_sh/__main__.py", line 25, in <module>
    run_license_sh(arguments)
  File "/home/theodore/Workspace/license.sh/license_sh/commands/run_license_sh.py", line 66, in run_license_sh
    dep_tree, whitelist, ignored_packages=ignored_packages, get_full_tree=tree
  File "/home/theodore/Workspace/license.sh/license_sh/helpers.py", line 218, in get_dependency_tree_with_licenses
    dep_tree, whitelist=whitelist, ignored_packages=ignored_packages
  File "/home/theodore/Workspace/license.sh/license_sh/helpers.py", line 159, in annotate_dep_tree
    node.license_normalized = normalize_license_expression(node.license)
AttributeError: 'NoneType' object has no attribute 'license'
(license.sh) [theodore@milacik license.sh]$

There should be more readable error

Package whitelisting

I am trying to white-list certain packages but they are still failing

bunyan-format - 0.2.1 - {'type': 'MIT', 'url': 'https://github.com/thlorenz/bunyan-format/blob/master/LICENSE'}

pako - 1.0.10 - (MIT AND Zlib)

They are present in the config

{
  "projects": [
    {
      "directory": ".",
      "type": "npm"
    }
  ],
  "whitelist": [
    "ISC",
    "Unlicense",
    "CC-BY-4.0",
    "BSD-3-Clause",
    "MIT",
    "BSD-2-Clause",
    "(BSD-3-Clause OR GPL-2.0)",
    "CC0-1.0",
    "Apache 2",
    "Apache-2",
    "(MIT AND Zlib)",
    "(MIT AND BSD-3-Clause)",
    "BSD",
    "MPL-2.0",
    "Apache-2.0",
    "CC-BY-3.0",
    "{'type': 'MIT', 'url': 'https://github.com/thlorenz/bunyan-format/blob/master/LICENSE'}",
    "MIT/X11",
    "(MIT AND CC-BY-3.0)"
  ]
}