diff-quality won't fail on new Pylint issues? about diff_cover HOT 7 CLOSED

Ill see if I can reproduce this. What you have there looks right to me. PyLint seems to be somewhat flawed in diff-quality. Hopefully I can figure this out sometime this week

from diff_cover.

Howdy.

I reproduced this issue. It seems passing in the report is breaking fail under which make me sad :-(

I verified that not passing in the report works as expected

(venv) ~/code/testproject   issue31  diff-quality --compare-branch=master --violations=pylint --fail-under=100
-------------
Diff Quality
Quality Report: pylint
Diff: master...HEAD, staged, and unstaged changes
-------------
woah.py (87.5%):
    4: C0103: (invalid-name), badlyNamedFunction: Invalid function name "badlyNamedFunction"
-------------
Total:   8 lines
Violations: 1 line
% Quality: 87%
-------------
Failure. Quality is below 100.0%.
(venv) ✘  ~/code/testproject   issue31  echo $?
1

I will will try to get a fix out to this quickly.

from diff_cover.

@butla hey, I realized that the problem could be how the pylint report was generated. Diff-quality depends on a very specific way of running the report to have a chance of working correctly. (honestly I probably should not allow people to pass in reports at all because of this.... sorry.)

So when you dont pass in the report diff-quality will run on individual files with this command

pylint --msg-template="{path}:{line}: [{msg_id}({symbol}), {obj}] {msg}"

You can try generating your report with that template. Is your project private? Or can you share the pylint report and the diff that diff-quality is working with?

from diff_cover.

Hey, thanks for the answer. It works! :)
I only have two issues with this:

• I'd also want to be able to see the old (not the newly introduced ones) pylint issues.
• Is there some way to exclude a folder from being checked? In my current setup I don't run Pylint on my tests.

Also, I've made a very crude "pylint-diff-quality.sh" for my use:

CURRENT_ISSUES=$(pylint $MODULE_NAME | wc -l)
git stash
NEW_ISSUES=$(pylint $MODULE_NAME | wc -l)
ISSUE_DIFF=$(($CURRENT_ISSUES - $NEW_ISSUES))
git stash pop
echo $ISSUE_DIFF
exit $ISSUE_DIFF

I know I should checkout origin/master and then the branch I was on originally instead of stashing. Does diff-cover also work like that?

from diff_cover.

By default diff-cover should reach out to origin and I dont think you need to pull it down. I inherited the codebase so I am still leaning things.

Diff-cover/diff-quality are designed to only work on changes so for the full report I think you just need to run pylint directly.

Finally for excluding a folder see if there is something that can go into your pylintrc file. I believe diff-quality respects that

from diff_cover.

Pylint needs to be used on a Python package or module. "ignore" in .pylintrc can be used to skip something in that selected context. If diff-quality runs on every file by itself then cutting out a module is unfeasible. But I'm no expert, I'm just telling you what I've worked out :)

I've also tried running diff-quality only in the folder that I want checked... It doesn't seem to detect any problems then.

Anyway, I guess I'll be using my custom script to do the checking, so you can close the issue if you want. Or you can leave this as some feature request :)

And good luck with the project, I really think that something like this can be useful to safeguard a project against sloppy code (and everyone makes that, sometimes).

from diff_cover.

@butla @Bachmann1234: diff-quality supports since version 5.1.0 an include option as well since 1.0.0 an exclude option. With this there is no need to use the pylint configuration to exclude stuff.

The other issues seem to be resolved. So, I guess this issue can be closed

from diff_cover.