saltstack / salt-pylint Goto Github PK

View Code? Open in Web Editor NEW

28.0 28.0 24.0 259 KB

PyLint Plugins Used In SaltStack's Projects

License: Apache License 2.0

Python 100.00%

python pylint saltstack

salt-pylint's Introduction

salt-pylint

salt-pylint's People

Contributors

Stargazers

Watchers

salt-pylint's Issues

New version of pylint breaks strings checker

Hello.

I used to use this plugin and it gave me correct errors on '{}'.format('abc') in Python 2.6. Now that check no longer works with new versions of pylint.

Code:

(pylint) sevagh:py-2.6-compat-linting $ cat example.py
#!/usr/bin/env python2

if __name__ == '__main__':
    mystr = 'Bad format string {} {} {}'.format('a',
                                                'b',
                                                'c')

    with open('/tmp/f1.txt', 'r') as a, open('/tmp/f2.txt', 'r') as b:
        print('Two withs in one line!')
(pylint) sevagh:py-2.6-compat-linting $

Pylintrc:

(pylint) sevagh:py-2.6-compat-linting $ cat .pylintrc
[MASTER]
load-plugins=saltpylint.minpyver,
    saltpylint.strings
minimum-python-version=2.6
[MESSAGES CONTROL]
disable=all
enable=E0598,
    W1320,
    E1320,
    W1321,
    E1231,
    E1322

Execution result:

(pylint) sevagh:py-2.6-compat-linting $ pylint ./example.py
************* Module example
E:  8, 0: Incompatible Python 2.6 code found: with statement with multiple contexts (minimum-python-version)

------------------------------------------------------------------
Your code has been rated at 0.00/10 (previous run: 0.00/10, +0.00)

(pylint) sevagh:py-2.6-compat-linting $

`saltpylint` release 2017.4.5 setup does not auto-install requirements

Install of 2017.4.5 (latest at this time) does NOT pull dependencies

> pip install saltpylint
Collecting saltpylint
  Downloading SaltPyLint-2017.4.5.tar.gz
Building wheels for collected packages: saltpylint
  Running setup.py bdist_wheel for saltpylint ... done
  Stored in directory: /home/blaine/.cache/pip/wheels/92/27/42/dd9a94cd74d389026644b4e9253f33ccc7881718c571262059
Successfully built saltpylint
Installing collected packages: saltpylint
Successfully installed saltpylint-2017.4.5

Install of 2017.3.6 pulls dependencies as expected

> pip install saltpylint==2017.3.6
Collecting saltpylint==2017.3.6
  Downloading SaltPyLint-2017.3.6-py2.py3-none-any.whl
Requirement already satisfied: modernize==0.5 in ./venv/lib/python2.7/site-packages (from saltpylint==2017.3.6)
Requirement already satisfied: pep8<=1.6.2,>=1.6.0 in ./venv/lib/python2.7/site-packages (from saltpylint==2017.3.6)
Requirement already satisfied: PyLint==1.6.5 in ./venv/lib/python2.7/site-packages (from saltpylint==2017.3.6)
Requirement already satisfied: mccabe in ./venv/lib/python2.7/site-packages (from PyLint==1.6.5->saltpylint==2017.3.6)
Requirement already satisfied: astroid<1.5.0,>=1.4.5 in ./venv/lib/python2.7/site-packages (from PyLint==1.6.5->saltpylint==2017.3.6)
Requirement already satisfied: configparser; python_version == "2.7" in ./venv/lib/python2.7/site-packages (from PyLint==1.6.5->saltpylint==2017.3.6)
Requirement already satisfied: backports.functools-lru-cache; python_version == "2.7" in ./venv/lib/python2.7/site-packages (from PyLint==1.6.5->saltpylint==2017.3.6)
Requirement already satisfied: six in ./venv/lib/python2.7/site-packages (from PyLint==1.6.5->saltpylint==2017.3.6)
Requirement already satisfied: isort>=4.2.5 in ./venv/lib/python2.7/site-packages (from PyLint==1.6.5->saltpylint==2017.3.6)
Requirement already satisfied: lazy-object-proxy in ./venv/lib/python2.7/site-packages (from astroid<1.5.0,>=1.4.5->PyLint==1.6.5->saltpylint==2017.3.6)
Requirement already satisfied: wrapt in ./venv/lib/python2.7/site-packages (from astroid<1.5.0,>=1.4.5->PyLint==1.6.5->saltpylint==2017.3.6)
Installing collected packages: saltpylint
Successfully installed saltpylint-2017.3.6

> pip --version
pip 9.0.1 from /home/blaine/development/DeepSea/venv/lib/python2.7/site-packages (python 2.7)
> python --version
Python 2.7.13

Black list and White list for python modules required to be in try/except block

It would be very useful to have a lint plugin to manage a black list and a white list of python modules that must be in a try/except block.

Black list example:
The pwd modules doesn't exist on Windows and so should be in a try/except block so it doesn't blow up the windows minion

White list example:
@s0undt3ch suggested we make all non-stdlib and non salt hard dep modules be required to be in a try/except block as well.
If we wanted to allow a module that's not stdlib or non salt hard dep be imported outside a try/except block, then we'd add it to the white list.

@s0undt3ch or @thatch45 do you have any thoughts or suggestions for this?

X-Ref: vmware-archive/salt-testing#27

AttributeError: 'zipimporter' object has no attribute 'path'

Pylint is failing with a stack trace:

Traceback (most recent call last):
  File "c:\python38\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\python38\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Python38\Scripts\pylint.exe\__main__.py", line 7, in <module>
  File "c:\python38\lib\site-packages\pylint\__init__.py", line 22, in run_pylint
    PylintRun(sys.argv[1:])
  File "c:\python38\lib\site-packages\pylint\lint\run.py", line 308, in __init__
    linter.load_plugin_modules(plugins)
  File "c:\python38\lib\site-packages\pylint\lint\pylinter.py", line 490, in load_plugin_modules
    module = modutils.load_module_from_name(modname)
  File "c:\python38\lib\site-packages\astroid\modutils.py", line 203, in load_module_from_name
    return load_module_from_modpath(dotted_name.split("."), path, use_sys)
  File "c:\python38\lib\site-packages\astroid\modutils.py", line 245, in load_module_from_modpath
    module = imp.load_module(curname, mp_file, mp_filename, mp_desc)
  File "c:\python38\lib\imp.py", line 234, in load_module
    return load_source(name, filename, file)
  File "c:\python38\lib\imp.py", line 171, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 702, in _load
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "c:\python38\lib\site-packages\saltpylint\thirdparty.py", line 50, in <module>
    class ThirdPartyImportsChecker(BaseChecker):
  File "c:\python38\lib\site-packages\saltpylint\thirdparty.py", line 72, in ThirdPartyImportsChecker
    all_modules = {m[1]: m[0].path for m in pkgutil.iter_modules()}
  File "c:\python38\lib\site-packages\saltpylint\thirdparty.py", line 72, in <dictcomp>
    all_modules = {m[1]: m[0].path for m in pkgutil.iter_modules()}
AttributeError: 'zipimporter' object has no attribute 'path'

SaltPyLint 2017.4.5 does not have Git tag

Hi @s0undt3ch !

The latest and greatest SaltPyLint release 2017.4.5 is available from pypi, but it appears that GitHub release (and appropriate tag) is missing:
https://github.com/saltstack/salt-pylint/releases
https://github.com/saltstack/salt-pylint/tags

This also does not match version info in saltpylint/version.py.

pylint fails for pytest modules

Running pylint on any module in tests/pytests/unit/modules produces a stack trace:

Traceback (most recent call last):
  File "/home/amendlik/.local/bin/pylint", line 10, in <module>
    sys.exit(run_pylint())
  File "/home/amendlik/.local/lib/python3.7/site-packages/pylint/__init__.py", line 24, in run_pylint
    PylintRun(sys.argv[1:])
  File "/home/amendlik/.local/lib/python3.7/site-packages/pylint/lint/run.py", line 358, in __init__
    linter.check(args)
  File "/home/amendlik/.local/lib/python3.7/site-packages/pylint/lint/pylinter.py", line 877, in check
    self.get_ast, self._iterate_file_descrs(files_or_modules)
  File "/home/amendlik/.local/lib/python3.7/site-packages/pylint/lint/pylinter.py", line 910, in _check_files
    self._check_file(get_ast, check_astroid_module, name, filepath, modname)
  File "/home/amendlik/.local/lib/python3.7/site-packages/pylint/lint/pylinter.py", line 936, in _check_file
    check_astroid_module(ast_node)
  File "/home/amendlik/.local/lib/python3.7/site-packages/pylint/lint/pylinter.py", line 1071, in check_astroid_module
    ast_node, walker, rawcheckers, tokencheckers
  File "/home/amendlik/.local/lib/python3.7/site-packages/pylint/lint/pylinter.py", line 1115, in _check_astroid_module
    walker.walk(ast_node)
  File "/home/amendlik/.local/lib/python3.7/site-packages/pylint/utils/ast_walker.py", line 75, in walk
    self.walk(child)
  File "/home/amendlik/.local/lib/python3.7/site-packages/pylint/utils/ast_walker.py", line 72, in walk
    callback(astroid)
  File "/home/amendlik/.local/lib/python3.7/site-packages/saltpylint/thirdparty.py", line 123, in visit_import
    self._check_third_party_import(node, name)
  File "/home/amendlik/.local/lib/python3.7/site-packages/saltpylint/thirdparty.py", line 135, in _check_third_party_import
    if is_relative(modname, module_file):
  File "/home/amendlik/.local/lib/python3.7/site-packages/astroid/modutils.py", line 581, in is_relative
    parent_spec = importlib.util.find_spec(name, from_file)
  File "/usr/lib/python3.7/importlib/util.py", line 94, in find_spec
    parent = __import__(parent_name, fromlist=['__path__'])
ModuleNotFoundError: No module named 'pytests'

lookup for setuptools in sys.modules is a no-op

Unless you import setuptools (which does not happen in setup.py), 'setuptools' will not be found in sys.modules, thus the lookup here:

https://github.com/saltstack/salt-pylint/blob/master/setup.py#L68

will never return True. This commit purports to fix the issue, but it does not.

pylint 2.17.4 crash after SaltPyLint is installed

When running pylint --rcfile=.pylintrc salt//.py in pipenv environment, with

pipenv run pylint --version
pylint 2.13.9
astroid 2.11.7
Python 3.9.16 (main, Dec  7 2022, 10:06:04)
[Clang 14.0.0 (clang-1400.0.29.202)]

and

SaltPyLint = "==2020.9.28"

Everything work fine. But when running pylint with following versions after SaltPyLint is installed, pylint will crash

pylint --version
pylint 2.17.4
astroid 2.15.5
Python 3.11.3 (main, Apr  7 2023, 20:13:31) [Clang 14.0.0 (clang-1400.0.29.202)]

or with

pylint --version
pylint 2.17.4
astroid 2.15.5
Python 3.9.16 (main, Dec  7 2022, 10:06:04)
[Clang 14.0.0 (clang-1400.0.29.202)]

The traceback is

Traceback (most recent call last):
  File "/opt/homebrew/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/opt/homebrew/lib/python3.9/site-packages/pylint/__init__.py", line 36, in run_pylint
    PylintRun(argv or sys.argv[1:])
  File "/opt/homebrew/lib/python3.9/site-packages/pylint/lint/run.py", line 169, in __init__
    args = _config_initialization(
  File "/opt/homebrew/lib/python3.9/site-packages/pylint/config/config_initialization.py", line 54, in _config_initialization
    linter.load_plugin_modules(utils._splitstrip(config_data["load-plugins"]))
  File "/opt/homebrew/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 384, in load_plugin_modules
    module.register(self)
  File "/opt/homebrew/lib/python3.9/site-packages/saltpylint/py3modernize/__init__.py", line 247, in register
    linter.register_checker(Py3Modernize(linter))
  File "/opt/homebrew/lib/python3.9/site-packages/saltpylint/checkers.py", line 20, in __init__
    super(BaseChecker, self).__init__(*args, **kwargs)
  File "/opt/homebrew/lib/python3.9/site-packages/pylint/checkers/base_checker.py", line 62, in __init__
    _ArgumentsProvider.__init__(self, linter)
  File "/opt/homebrew/lib/python3.9/site-packages/pylint/config/arguments_provider.py", line 48, in __init__
    self._arguments_manager._register_options_provider(self)
  File "/opt/homebrew/lib/python3.9/site-packages/pylint/config/arguments_manager.py", line 145, in _register_options_provider
    argument = _convert_option_to_argument(opt, optdict)
  File "/opt/homebrew/lib/python3.9/site-packages/pylint/config/utils.py", line 138, in _convert_option_to_argument
    return _StoreArgument(
  File "/opt/homebrew/lib/python3.9/site-packages/pylint/config/argument.py", line 261, in __init__
    self.type = _TYPE_TRANSFORMERS[arg_type]
KeyError: 'multiple_choice'

Package dependency graph has conflicts

When trying to install with pipenv (pipenv --two install saltpylint), following warning appears:

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
Could not find a version that matches astroid<2.0,>=1.6,>=2.0.0
Tried: 1.0.0, 1.0.1, 1.1.0, 1.1.1, 1.2.0, 1.2.1, 1.3.0, 1.3.0, 1.3.1, 1.3.1, 1.3.2, 1.3.2, 1.3.3, 1.3.3, 1.3.4, 1.3.4, 1.3.5, 1.3.5, 1.3.6, 1.3.6, 1.3.7, 1.3.7, 1.3.8, 1.3.8, 1.4.0, 1.4.0, 1.4.1, 1.4.1, 1.4.2, 1.4.2, 1.4.3, 1.4.3, 1.4.4, 1.4.4, 1.4.5, 1.4.5, 1.4.6, 1.4.6, 1.4.7, 1.4.7, 1.4.8, 1.4.8, 1.4.9, 1.4.9, 1.5.0, 1.5.0, 1.5.1, 1.5.1, 1.5.2, 1.5.2, 1.5.3, 1.5.3, 1.6.0, 1.6.0, 1.6.1, 1.6.1, 1.6.2, 1.6.2, 1.6.3, 1.6.3, 1.6.4, 1.6.4, 1.6.5, 1.6.5
There are incompatible versions in the resolved dependencies.

Incompatible with Pylint 3

ImportError: cannot import name 'IRawChecker' from 'pylint.interfaces' (/Users/user/.cache/pre-commit/repoz5pliu_c/py_env-python3.9/lib/python3.9/site-packages/pylint/interfaces.py)

Disallow logging in `virtual()`

There should never be any logging in any module's __virtual__() function. We need to yell at users that do that.

X-Ref: vmware-archive/salt-testing#34

Fix logging in virtual checks: dunders, conditional import blocks

Copying from saltstack/salt#57730 (comment).

Support was added for this in #37. However, I ran into a couple of obstacles trying to add it.

Salt's dunders blow up the check

E.g. https://github.com/saltstack/salt/blob/b95213ec903402f25c1e0aeb3990fe8452ab63ce/salt/modules/linux_service.py#L18-L56

$ pylint --rcfile=.pylintrc --disable=I --rcfile=.pylintrc --disable=I salt/modules/linux_service.py
Traceback (most recent call last):
  File "/home/ken/.virtualenvs/salt/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/home/ken/.virtualenvs/salt/lib/python3.6/site-packages/pylint/__init__.py", line 23, in run_pylint
    PylintRun(sys.argv[1:])
  File "/home/ken/.virtualenvs/salt/lib/python3.6/site-packages/pylint/lint.py", line 1731, in __init__
    linter.check(args)
  File "/home/ken/.virtualenvs/salt/lib/python3.6/site-packages/pylint/lint.py", line 1004, in check
    self._do_check(files_or_modules)
  File "/home/ken/.virtualenvs/salt/lib/python3.6/site-packages/pylint/lint.py", line 1165, in _do_check
    self.check_astroid_module(ast_node, walker, rawcheckers, tokencheckers)
  File "/home/ken/.virtualenvs/salt/lib/python3.6/site-packages/pylint/lint.py", line 1252, in check_astroid_module
    walker.walk(ast_node)
  File "/home/ken/.virtualenvs/salt/lib/python3.6/site-packages/pylint/utils/ast_walker.py", line 77, in walk
    self.walk(child)
  File "/home/ken/.virtualenvs/salt/lib/python3.6/site-packages/pylint/utils/ast_walker.py", line 74, in walk
    callback(astroid)
  File "/home/ken/.virtualenvs/salt/lib/python3.6/site-packages/saltpylint/virt.py", line 52, in visit_functiondef
    for inferred in functions.func.expr.infer():
  File "/home/ken/.virtualenvs/salt/lib/python3.6/site-packages/astroid/decorators.py", line 131, in raise_if_nothing_inferred
    yield next(generator)
  File "/home/ken/.virtualenvs/salt/lib/python3.6/site-packages/astroid/decorators.py", line 92, in wrapped
    generator = _func(node, context, **kwargs)
  File "/home/ken/.virtualenvs/salt/lib/python3.6/site-packages/astroid/inference.py", line 195, in infer_name
    name=self.name, scope=self.scope(), context=context
astroid.exceptions.NameInferenceError: '__grains__' not found in <FunctionDef.__virtual__ l.18 at 0x7f37b4840eb8>.

Log message in missing import blocks aren't found

https://github.com/saltstack/salt/blob/b95213ec903402f25c1e0aeb3990fe8452ab63ce/salt/returners/appoptics_return.py#L83-L108

$ pylint --rcfile=.pylintrc --disable=I --rcfile=.pylintrc --disable=I salt/returners/appoptics_return.py 

------------------------------------
Your code has been rated at 10.00/10

If I add:

    log.info('foo')

at the top of __virtual__, though:

$ pylint --rcfile=.pylintrc --disable=I --rcfile=.pylintrc --disable=I salt/returners/appoptics_return.py 
************* Module salt.returners.appoptics_return
salt/returners/appoptics_return.py:98: [E1401(log-in-virtual), __virtual__] Log statement detected inside __virtual__ function. Remove it.

-----------------------------------
Your code has been rated at 9.29/10

minpyver: not erroring on '{}'.format() for python 2.6

Hello.

'{}'.format(a) works in Python 2.7 but not earlier Python (in 2.6, for example, I need to use '{0}'.format(a).

I'm using the minpyver module from this repo to check Python 2.6 compatibility on my codebase, and noticed it does not catch this case.

Should I write a pull request to detect this condition? Was it omitted on purpose?

Thanks.

Some tests.

{}.format not working in Python 2.6:

Python 2.6.6 (r266:84292, Jan 22 2014, 09:42:36) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> string1 = '{}'.format('lol')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: zero length field name in format
>>> string1 = '{0}'.format('lol')
>>> print(string1)
lol
>>>

Pylint config:

$ cat setup.cfg 
[MASTER]
load-plugins=saltpylint.minpyver
minimum-python-version=2.6
[MESSAGES CONTROL]
disable=all
enable=E0598

Output on testfile:

$ cat testfile.py 
if __name__ == '__main__':
    str1 = '{}'.format('lol')
    str2 = '{0}'.format('wat')
$ pylint --rcfile=setup.cfg -e E0598 -r n testfile.py 
$ echo $?
0

[BUG] salt-pylint throws a TypeError

Running pylint against salt-pylint:

> pylint saltpylint/pep8.py
Traceback (most recent call last):
  File "/home/mark.ferrell/.local/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/home/mark.ferrell/.local/lib/python3.8/site-packages/pylint/__init__.py", line 24, in run_pylint
    PylintRun(sys.argv[1:])
  File "/home/mark.ferrell/.local/lib/python3.8/site-packages/pylint/lint/run.py", line 330, in __init__
    linter.load_plugin_modules(plugins)
  File "/home/mark.ferrell/.local/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 503, in load_plugin_modules
    module = astroid.modutils.load_module_from_name(modname)
  File "/home/mark.ferrell/.local/lib/python3.8/site-packages/astroid/modutils.py", line 212, in load_module_from_name
    return importlib.import_module(dotted_name)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/mark.ferrell/.local/lib/python3.8/site-packages/saltpylint/pep8.py", line 384, in <module>
    class PEP8WhitespaceWarning(_PEP8BaseChecker):
  File "/home/mark.ferrell/.local/lib/python3.8/site-packages/saltpylint/pep8.py", line 391, in PEP8WhitespaceWarning
    'trailing-whitespace' if pylint_version_info < (1, 0) else
TypeError: '<' not supported between instances of 'str' and 'int'

Importing pip install pylint:

> python3
Python 3.8.5 (default, Jan 27 2021, 15:41:15) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pylint.__pkginfo__ import numversion as pylint_version_info
>>> print(pylint_version_info)
('2', '8', '2')
>>>

It looks like prior to pylint-dev/pylint@0e63847 pylint returned a tuple of strings, but after this commit it now returns a tuple of Int's. From pylint.git:

pylint.git (master)> python3
Python 3.8.5 (default, Jan 27 2021, 15:41:15) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pylint.__pkginfo__ import numversion as pylint_version_info
>>> print(pylint_version_info)
(2, 8, 2)
>>>