adamtornhill / code-maat Goto Github PK

I am analyzing wikimedia/mediawiki with code-maat.

I seem to be getting parsing errors for the commit messages:

java -Xmx512M -Xms64M -jar \winmaat0.8.5\code-maat-0.8.5-standalone.jar -l ../mediawiki/maat_evo.log -c git -a summary
Invalid argument: input: [ebbf38d2ab] Antoine Musso 2010-10-29 Follow up r75682 : fix private function naming. Fix cruise control build 2010-10-29T22:18:27
3 3 maintenance/tests/phpunit/includes/UserIsValidEmailAddrTest.php
, reason: Parse error at line 1, column 122:
[ebbf38d2ab] Antoine Musso 2010-10-29 Follow up r75682 : fix private function naming. Fix cruise control build 2010-10-29T22:18:27

When I manually fix this comment, I run into the next parsing error:

java -Xmx512M -Xms64M -jar \winmaat0.8.5\code-maat-0.8.5-standalone.jar -l ../mediawiki/maat_evo.log -c git -a summary
Invalid argument: input: [c7723263fc] Niklas Laxstr▒m 2010-08-08 Removed $wgUseZhdaemon, $wgZhdaemonHost and $wgZhdaemonPort. Apparently unused since r6878 at 2004-12-29.

It would be nice if code-maat would be more robust in parsing comments.

I created a Perforce log file using the command given in the documentation.
It looks like the code-maat parser can't handle some lines I have, starting with 'Jobs fixed'.
The error from the parser says that it expected 'Affected' at that position.
Below is the log file content.

Change 401723 by ejbo@ejbo-home-fimbul on 2015/03/19 12:40:33
FIM-29 ##scription here>
Jobs fixed ...
FIM-29 on 2015/03/19 by ejbo closed
Fimbul unit test suite
Affected files ...
... //depot/fiks/fimbul/unittest/gmock-1.6.0/gtest/samples/test_fimbul_klip.cc#31 edit

Change 401722 by ejbo@ejbo-home-fimbul on 2015/03/19 10:21:54
FIM-175 proxy ##on here>
Jobs fixed ...
FIM-40 on 2015/03/19 by ejbo closed
Fjern hardkodet dato 01.01.2020 i army.cxx
Affected files ...
... //depot/fiks/fimbul/klip_syg.cxx#93 edit
... //depot/fiks/fimbul/klip_syg.hxx#16 edit

I think it would be great to avoid the need to manually generate the logs and then feed it into code-maat after..

After all the command that needs to be run is known and to find out what kind of repository is we just need to check for .git/.hg/.svn in the directory right?

I can probably give it a try and add something like "--auto $path" or "--repository-path $path" option that does the detection and parsing, what do you think?

Using the SVN log format mentioned in the README file for code-maat:

svn log -v --xml > logfile.log -r {YYYYmmDD}:HEAD

I have encountered the following error:

$ maat -l ~/workspace/project/logfile.log -c svn -a abs-churn
java.lang.IllegalArgumentException: Internal error - please report it. Details = churn analysis: the given VCS data doesn't contain modification metrics. Check the code-maat docs for supported VCS and correct log format.
    at code_maat.app.app$throw_internal_error.invoke(app.clj:158)
    at code_maat.app.app$run_with_recovery_point.invoke(app.clj:167)
    at code_maat.app.app$run.invoke(app.clj:188)
    at code_maat.cmd_line$_main.doInvoke(cmd_line.clj:63)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at code_maat.cmd_line.main(Unknown Source)
Error:  Internal error - please report it. Details = churn analysis: the given VCS data doesn't contain modification metrics. Check the code-maat docs for supported VCS and correct log format.

This issue arises from the choice of using SVN's XML log format, which does not support reporting the diffs. The text format supports diffs but may be harder to parse.

When i run the command java -jar code-maat.jar -l “maatfreqs.csv" -c git -a coupling, the program doesn´t show information.

Hi,

I am following the code samples from the book and the grouping commands output incorrect text.

For example:

1. maat -l maat_evo.log -c git -a coupling -g maat_src_test_boundaries.txt
   entity,coupled,degree,average-revs
   ",80,65est
2. maat -l maat_evo.log -c git -a coupling -g maat_src_detailed_test_boundaries.txt
   entity,coupled,degree,average-revs
   ",42,50to end Tests
   ",42,49is Test
   ",Parsers Test,41,49

I have tried both the latest version and 0.8.5 and I get the same results. I am running on Windows 8.

Thanks!

Hi,
Here is an engineer from Huawei Technologies Co.,Ltd.. We are working on a topic How to get the record of code change by VCS?
We want to know which lines of code were changed by who and at when through analyzing the log of VCS by some tools. Code-maat seems to be what we are looking for.

Is this product still available? Couldn't find it on the website.

Hey adam ,
I not able to generate log file with this command
git log --all --numstat --date=short --pretty=format:'--%h--%ad--%aN' --no-renames --after=2016-10-01
I am running it in a cloned project for which numstat is getting displayed but log isn't getting generated
can you just guide me how to run code-maat for a git project
I have lein install already using package manager on ubuntu 18.04
Do i need to create a a clj file for my git project or shall i include it in a lein app.

http://www.adamtornhill.com/code/codemaat.htm seems to be down currently

Hello,

I'm completly new to maat, and I'm having problems with the setup process.

I've made the lein script to a shell file and put it in my bin folder as I usually do. I've also put the project.clj in the same folder.
When I run the code I get the following error:
Madss-MacBook-Pro:bin madsgadehenrichsen$ ./lein run -l desktop/betaflight/log2.log -c Compiling code-maat.cmd-line Could not locate code_maat/cmd_line__init.class or code_maat/cmd_line.clj on classpath. Please check that namespaces with dashes use underscores in the Clojure file name. Exception in thread "main" java.io.FileNotFoundException: Could not locate code_maat/cmd_line__init.class or code_maat/cmd_line.clj on classpath. Please check that namespaces with dashes use underscores in the Clojure file name., compiling:(/private/var/folders/_g/67_46n712vxchn3rc17cch2m0000gn/T/form-init5180465450996050829.clj:1:125) at clojure.lang.Compiler.load(Compiler.java:7391) at clojure.lang.Compiler.loadFile(Compiler.java:7317) at clojure.main$load_script.invokeStatic(main.clj:275) at clojure.main$init_opt.invokeStatic(main.clj:277) at clojure.main$init_opt.invoke(main.clj:277) at clojure.main$initialize.invokeStatic(main.clj:308) at clojure.main$null_opt.invokeStatic(main.clj:342) at clojure.main$null_opt.invoke(main.clj:339) at clojure.main$main.invokeStatic(main.clj:421) at clojure.main$main.doInvoke(main.clj:384) at clojure.lang.RestFn.invoke(RestFn.java:421) at clojure.lang.Var.invoke(Var.java:383) at clojure.lang.AFn.applyToHelper(AFn.java:156) at clojure.lang.Var.applyTo(Var.java:700) at clojure.main.main(main.java:37) Caused by: java.io.FileNotFoundException: Could not locate code_maat/cmd_line__init.class or code_maat/cmd_line.clj on classpath. Please check that namespaces with dashes use underscores in the Clojure file name. at clojure.lang.RT.load(RT.java:456) at clojure.lang.RT.load(RT.java:419) at clojure.core$load$fn__5677.invoke(core.clj:5893) at clojure.core$load.invokeStatic(core.clj:5892) at clojure.core$load.doInvoke(core.clj:5876) at clojure.lang.RestFn.invoke(RestFn.java:408) at clojure.core$load_one.invokeStatic(core.clj:5697) at clojure.core$compile$fn__5682.invoke(core.clj:5903) at clojure.core$compile.invokeStatic(core.clj:5903) at clojure.core$compile.invoke(core.clj:5895) at user$eval20$fn__29.invoke(form-init5180465450996050829.clj:1) at user$eval20.invokeStatic(form-init5180465450996050829.clj:1) at user$eval20.invoke(form-init5180465450996050829.clj:1) at clojure.lang.Compiler.eval(Compiler.java:6927) at clojure.lang.Compiler.eval(Compiler.java:6917) at clojure.lang.Compiler.eval(Compiler.java:6917) at clojure.lang.Compiler.load(Compiler.java:7379) ... 14 more Compilation failed: Subprocess failed
Looks to me like it can't find the code_maat folder. I've tried placing it multiple places without luck. Where am I supposed to put it so the script can find it?

Thanks

Hi Adam

This is right at the edge of my technical envelope! =) Anyway, have a log file and am also pretty sure it's UTF-8 encoded. At least that's what it says when I open it in Notepad++ and Atom.

What other issues could be causing this error text?

Invalid argument: git2: Failed to parse the given file - is it a valid logfile?

Actually took the test simple_git2.txt file and tried to run it through this and it failed with the same error so I'm guessing this could be a case of user (has no clue) error 👍

Also - since I know NOTHING about clojure - is this warning relevant at all??
WARNING: update already refers to: #'clojure.core/update in namespace: incanter.core, being replaced by: #'incanter.core/update

Thanks!!

I am trying to use the new "messages" analysis but it doesn't seem to like the regular expression I am passing to it.
According to other tools it is a valid regex. Maybe Clojure uses a different syntax?
I tried to find an example under the test directory but couldn't find any.

java -jar C:\Dropbox\bin\code-maat-0.9.0-SNAPSHOT-standalone.jar -l C:\temp\d4vanrx.txt -c git -a messages -e "(S[0-9]{,7})|([)]|([A-Z]{1,}-[0-9]{1,})[:]" -o C:\temp\maat_output\messages.csv
Invalid argument: Internal error - please report it. Details = Illegal repetition near index 6
(S[0-9]{,7})|([)]|([A-Z]{1,}-[0-9]{1,})[:]

I am running the python script csv_as_enclosure_json.py and the json is always generated with 0 in the weight attribute.

In debug mode the function normalized_weight_for is not executing "if module_name in normalized_weights:"

These are my files

countlines-domina04012018.txt
revisions-domina04012018.txt

And this is the way i execute the script

python csv_as_enclosure_json.py
--structure "countlines-domina04012018.csv" --weights "revisions-domina04012018.csv" - weightcolumn 2 > jsond3Domina.json

When using code-maat on recent javascript (React) code bases unfortunately it's not possible to distinguish production code from test code, as the convention for React seems to be putting test files next to production files in the same folder. If I wanted to separate the production code from the test code in my code-maat analysis using the --group options I would liked to be able to write a file using regexp expressions to split the layer assignment, e.g.

app/frontend/.*/[^--]+\.js => Code
app/frontend/.*--test\.js => Unit tests

Hi, I did git clone for code maat then:

code-maat$ git log --all --numstat --date=short --pretty=format:'--%h--%ad--%aN' --no-renames > git.log
java -jar target/code-maat-1.0-SNAPSHOT-standalone.jar -l git.log -c git

and result:

Invalid argument:  java.lang.IllegalArgumentException: input: --788a0f3--2016-07-19--Adam Tornhill
1	1	README.md
, reason: Parse error at line 1, column 1:
--788a0f3--2016-07-19--Adam Tornhill
^
Expected:
[

This is Code Maat, a program used to collect statistics from a VCS.
Version: 1.0-SNAPSHOT

can you help please?

Hi,

On Docker Hub there is a code-maat image: https://hub.docker.com/r/stevenalexander/code-maat-python/

However that also contains Python and cloc. I could add my own bare image, but maybe it would be handier to do so under an adamtornhill account?

I'd suggest publishing every image version under three tags: a version tag, a major version tag, and 'latest', e.g.

adamtornhill/code-maat:1.0.2
adamtornhill/code-maat:1
adamtornhill/code-maat:latest

That way, people can choose to depend on a specific version, a major version, or to use the latest tag.

Using: Java 8, Leiningen 2.9.2

When I try to run the following command

java -jar C:\code-maat\target\code-maat-1.1-SNAPSHOT.jar -l logfile.log -c git

I get the following error message:

Exception in thread "main" java.lang.NoClassDefFoundError: clojure/lang/Var
at code_maat.cmd_line.(Unknown Source)
Caused by: java.lang.ClassNotFoundException: clojure.lang.Var
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
... 1 more

I still get the error message above after running the following command:

java -jar C:\code-maat\target\code-maat-1.1-SNAPSHOT.jar -cp .;C:\Users\Owner\Documents\WindowsPowerShell\Modules\ClojureTools\clojure-tools-1.10.1.561.jar -l logfile.log -c git

It would be wonderful if code-maat could be published on http://www.clojars.org or http://central.maven.org This would allow folks to leverage code-maat as a library and call its APIs.

Thanks,
Jim

How can we use code-maat with CI tools like Jenkins and codeship because code-maat is a part of code analysis. Now, the time, we need to maintain code analysis at CI time. Is possible to use with codeship of Jenkins?

how to exclude certain folders inside repository during generation of the log?

Hi adamtornhill
Wanted to know whether all scenarios of code-forensics are covered here like sloc,churn etc if not what are to be covered and when are you going to extend it.How many languages it supports currently
and how to extend it for other languages.Wanted to know how to extend in terms of code.

Hi, I am not well versed in java so bear with me. When I run this code from my repo directory, I get an error.

java -jar code-maat-0.5.1.jar -l logfile.log -c <vcs>

zsh: parse error near `\n'

Again, I am running this inside of my repository directory which I have a feeling I am doing wrong. Please let me know what it is that I need to do so that this will work. Thank you.

From the readme:

Generate a git log file using the following command:

git log --pretty=format:'[%h] %aN %ad %s' --date=short --numstat --after=YYYY-MM-DD

Note that there's a second supported Git format as well, imaginatively named git2. This format is more tolerant and faster to parse, so please prefer it over the plain git format described above:

git log --all -M -C --numstat --date=short --pretty=format:'--%h--%ad--%an' --no-renames

The second instruction uses --pretty=format:'--%h--%ad--%an', which results in this error:

$ java -jar code-maat-0.9.2-SNAPSHOT-standalone.jar -l test.log -c git -a entity-effort -o entity-test.csv
WARNING: update already refers to: #'clojure.core/update in namespace: incanter.core, being replaced by: #'incanter.core/update
Invalid argument:  java.lang.IllegalArgumentException: input: --afefec5--2015-11-07--JP Johansson
5       3       filename1.js
60      0       filename2.js
55      81      filename3.js
126     158     filename.css
, reason: Parse error at line 1, column 1:
--afefec5--2015-11-07--JP Johansson
^
Expected:
[

This is Code Maat, a program used to collect statistics from a VCS.
Version: 0.9.2-SNAPSHOT

It happens with all -a flags to code-maat. I changed it to use the --pretty=format:'[%h] %aN %ad %s' from the first instruction and it works, but then the readme text about a "second supported Git format" is a bit confusing.

I also didn't quite understand the other flags in the second instruction, especially the last parameter --no-renames which seems to contradict -M -C? That's how I read it on ExplainShell.

Hi,
I wonder if you would be interested in implementing the components library:
https://github.com/stuartsierra/component/

It's a bit tricky to adapt it to an existing codebase but it can be done, and as the project becomes bigger could become a great advantage..
I

Hi!

I tried to process a pretty large git log from a private git repo. I increased to max heap to 4GB but it still did not help. Much more heap would not go as my laptop's memory is limited.

Best Regards,
Robin

Hi,

First of all, thanks for creating this tool and putting in the effort and research to help bring some insight into analysing technical debts.

I'm trying to use code-maat to understand our codebases and i'm seeing very weird results which making me question whether I should use the results of this tool at all.

I ran this tool on a large codebase within my company and the results seem conflicting:

git log --all --numstat --date=short --pretty=format:'--%h--%ad--%aN' --no-renames --after=2019-03-01 -- . ":(exclude).staging/" ":(exclude).tmp_staging/" ":(exclude)tests/" ":(exclude)k8s/" ":(exclude)node_modules/" > logfile.log

Main Dev

Command:

docker run -v ~/Work/code-analysis/:/data -it code-maat-app -l /data/logfile.log -c git2 -a main-dev > main-dev.csv

Tagged over 2000 files to a single dev - with added=0, total-added=0 and ownsership=0. This means that no changes were made to these files? This dev has made no changes to the files at all. Verified this by looking at Github history.

Main Dev By Rev

Command:

docker run -v ~/Work/code-analysis/:/data -it code-maat-app -l /data/logfile.log -c git2 -a main-dev-by-revs > main-dev-by-revs.csv

Tagged over 2000 files to a single dev - This dev has made no changes to the files at all. Verified this by looking at Github history

Age

Command:

docker run -v ~/Work/code-analysis/:/data -it code-maat-app -l /data/logfile.log -c git2 -a age > age.csv

Result - Off the 5k files analysed, 4.7k files have been modified in the last 2 month. This is conflicting with the results under Main Dev. Also, I checked file history, some files tagged with age as 2 months has not been modified since 2018.

Is this because it looks at entire git history of all branches vs only master? How do I make it run against only master?

Hello,

I've been using the code-maat tool to mine features from a git repository, especially the logic coupling. What I'm facing is the scenario that I want to mine the coupling changes over a time. So what I'm wondering is if it's possible to get all couplings by just running on the latest revision? Or do I need to do as I'm doing now and run the tool on each revision and then track the coupling changes myself?

Best regards,

Oscar

I invoked code-maat with a wrong parameter (-a couplin instead of -a coupling) and it threw the following exception:

java.lang.IllegalArgumentException: Internal error - please report it. Details = Wrong number of args (1) passed to: app/make-analysis/fn--8124
    at code_maat.app.app$throw_internal_error.invoke(app.clj:158)
    at code_maat.app.app$run_with_recovery_point.invoke(app.clj:167)
    at code_maat.app.app$run.invoke(app.clj:188)
    at code_maat.cmd_line$_main.doInvoke(cmd_line.clj:63)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at code_maat.cmd_line.main(Unknown Source)
Error:  Internal error - please report it. Details = Wrong number of args (1) passed to: app/make-analysis/fn--8124

It would be good to know that "couplin" is invalid and what the valid options are :)

The following git log file leads to an error when invoking Code Maat (winmaat0.8.5) using maat -l evo.Sample.log -c git -a summary:

[611a2fe] User2 2016-03-11 (JIRA-789) Some text (see mails of 2016-03-11).
12  3   Project.UnitTests/Spec.cs
3   3   OtherProject.UnitTests/OtherSpec.cs

The error message is:

Invalid argument:  input: [611a2fe] User2 2016-03-11 (JIRA-789) Some text (see mails of 2016-03-11).
12      3       Project.UnitTests/Spec.cs
3       3       OtherProject.UnitTests/OtherSpec.cs
, reason: Parse error at line 1, column 73:
[611a2fe] User2 2016-03-11 (JIRA-789) Some text (see mails of 2016-03-11).
                                                                        ^
Expected:
#"\s"

This is Code Maat, a program used to collect statistics from a VCS.
[...]

Code Maat seems to interpret the date within the message as something to be parsed rather than treating it as an opaque part of the commit message.

This is just a suggestion, but might be nice to tag the git revision whenever, which makes github understand your project history a bit better and is also easier to understand what is the last release.
It can still be done even now for previous versions potentially if you're interested in doing that, thanks

I am trying to execute code maat in TFS but i get the following error:

Invalid argument: Unsupported TFS Date Format: Thursday, December 21, 2017 9:26:46 AM
This is Code Maat, a program used to collect statistics from a VCS.
Version: 1.0-SNAPSHOT

This is the command i am using

java -jar code-maat.jar -l "C:\TFS\Loc\Dev\Satrack.Loc.Core\Domain\tfslog.csv" -c tfs -a revisions

And this is the tfs log i am using.

tfslog.txt

I checked and the date is in the locale that you recommend.

Thanks!

I noticed that the two files are essentially the same, grouper.clj and layer_mapper.clj. So we should delete layer_mapper.clj (since the ns is grouper)

>diff code_maat/app/grouper.clj code_maat/app/layer_mapper.clj 
24c24
<     path      = #'^[\\w/\\\\\\.\\-]+'

---
>     path      = #'^[\\w/\\\\]+'

Thanks,
Rui

Hi @adamtornhill ,

I loved this tool and learning to understand more to use code-maat in my work environment. I want to know still what new features are you going to add up and how support will be delivered if i looking it to deploy it in my working place.

Any chance that code-maat could support Accurev depots?

Downloaded and unpacked code-maat release 1.0.1

Installed lein with:

$ brew install leiningen

Lein version that got installed:

$ lein --version
Leiningen 2.8.3 on Java 1.8.0_202-ea Java HotSpot(TM) 64-Bit Server VM

Launched

lein uberjar

Result:

$ lein uberjar
[...]
Retrieving com/github/rwl/AMDJ/1.0.1/AMDJ-1.0.1.jar from central
Retrieving org/clojure/data.csv/0.1.2/data.csv-0.1.2.jar from central
Retrieving joda-time/joda-time/2.6/joda-time-2.6.jar from central
Retrieving org/clojure/math.numeric-tower/0.0.4/math.numeric-tower-0.0.4.jar from central
Could not transfer artifact org.clojure:math.combinatorics:jar:0.1.1 from/to central (https://repo1.maven.org/maven2/): Socket closed
Could not transfer artifact org.clojure:math.combinatorics:jar:0.1.1 from/to clojars (https://repo.clojars.org/): Socket closed
Could not transfer artifact org.clojure:clojurescript:jar:1.9.493 from/to central (https://repo1.maven.org/maven2/): Socket closed
Could not find artifact org.clojure:clojurescript:jar:1.9.493 in clojars (https://repo.clojars.org/)
Could not transfer artifact com.google.javascript:closure-compiler-unshaded:jar:v20170218 from/to central (https://repo1.maven.org/maven2/): Socket closed
Could not find artifact com.google.javascript:closure-compiler-unshaded:jar:v20170218 in clojars (https://repo.clojars.org/)
Could not transfer artifact com.google.javascript:closure-compiler-externs:jar:v20170218 from/to central (https://repo1.maven.org/maven2/): Socket closed
Could not find artifact com.google.javascript:closure-compiler-externs:jar:v20170218 in clojars (https://repo.clojars.org/)
Could not transfer artifact args4j:args4j:jar:2.33 from/to central (https://repo1.maven.org/maven2/): Socket closed
Could not find artifact args4j:args4j:jar:2.33 in clojars (https://repo.clojars.org/)
Could not transfer artifact com.google.protobuf:protobuf-java:jar:3.0.2 from/to central (https://repo1.maven.org/maven2/): Socket closed
Could not transfer artifact com.google.protobuf:protobuf-java:jar:3.0.2 from/to clojars (https://repo.clojars.org/): Socket closed
Could not transfer artifact com.google.code.findbugs:jsr305:jar:3.0.1 from/to central (https://repo1.maven.org/maven2/): Socket closed
Could not transfer artifact com.google.code.findbugs:jsr305:jar:3.0.1 from/to clojars (https://repo.clojars.org/): Socket closed
Could not transfer artifact com.google.jsinterop:jsinterop-annotations:jar:1.0.0 from/to central (https://repo1.maven.org/maven2/): Socket closed
Could not transfer artifact com.google.jsinterop:jsinterop-annotations:jar:1.0.0 from/to clojars (https://repo.clojars.org/): Socket closed
Could not transfer artifact org.clojure:google-closure-library:jar:0.0-20160609-f42b4a24 from/to central (https://repo1.maven.org/maven2/): Socket closed
Could not transfer artifact org.clojure:google-closure-library:jar:0.0-20160609-f42b4a24 from/to clojars (https://repo.clojars.org/): Socket closed
Could not transfer artifact org.clojure:google-closure-library-third-party:jar:0.0-20160609-f42b4a24 from/to central (https://repo1.maven.org/maven2/): Socket closed
Could not transfer artifact org.clojure:google-closure-library-third-party:jar:0.0-20160609-f42b4a24 from/to clojars (https://repo.clojars.org/): Socket closed
Could not transfer artifact org.clojure:data.json:jar:0.2.6 from/to central (https://repo1.maven.org/maven2/): Socket closed
Could not transfer artifact org.clojure:data.json:jar:0.2.6 from/to clojars (https://repo.clojars.org/): Socket closed
Could not transfer artifact org.mozilla:rhino:jar:1.7R5 from/to central (https://repo1.maven.org/maven2/): Socket closed
Could not transfer artifact org.mozilla:rhino:jar:1.7R5 from/to clojars (https://repo.clojars.org/): Socket closed
Could not transfer artifact org.clojure:tools.reader:jar:1.0.0-beta3 from/to central (https://repo1.maven.org/maven2/): Socket closed
Could not transfer artifact org.clojure:tools.reader:jar:1.0.0-beta3 from/to clojars (https://repo.clojars.org/): Socket closed
This could be due to a typo in :dependencies, file system permissions, or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.
Uberjar aborting because jar failed: Could not resolve dependencies

OS version:
MacOS Hight Sierra 10.13.6 (17G5019)

Im getting the following error when running this with the docker container:

Invalid argument: git2: Failed to parse the given file - is it a valid logfile?

I built the container with the command below from the latest code on 9th august:

docker build -t code-maat-app .

I then ran

git log --all --numstat --date=short --pretty=format:'--%h--%ad--%aN' --no-renames --after=2018-10-31 > logfile.log

and after that:

docker run -v /PATH-TO-CODEBASE:/data -it code-maat-app -l logfile.log -c git2

my logfile looks just like the ones in your tests folder :/

I'm following along and tried to run the analysis tool on some logs generated from a ~3 year old project. there are about 81K lines in the file i'm trying to analyze -- here's the command and output:

$ ~/opt/ixmaat0.8.2/maat -l ignore_me/commit_log.txt -c git -a summary
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at clojure.lang.PersistentHashMap$ArrayNode.assoc(PersistentHashMap.java:368)
    at clojure.lang.PersistentHashMap$ArrayNode.assoc(PersistentHashMap.java:365)
    at clojure.lang.PersistentHashMap.assoc(PersistentHashMap.java:142)
    at clojure.lang.PersistentHashMap.assoc(PersistentHashMap.java:28)
    at clojure.lang.RT.assoc(RT.java:702)
    at clojure.core$assoc.invoke(core.clj:187)
    at clojure.lang.Atom.swap(Atom.java:65)
    at clojure.core$swap_BANG_.invoke(core.clj:2234)
    at instaparse.gll$node_get.invoke(gll.clj:221)
    at instaparse.gll$push_listener.invoke(gll.clj:267)
    at instaparse.gll$CatListener$fn__672.invoke(gll.clj:397)
    at instaparse.gll$push_message$f__602.invoke(gll.clj:173)
    at instaparse.gll$step.invoke(gll.clj:328)
    at instaparse.gll$run.invoke(gll.clj:344)
    at instaparse.gll$run.invoke(gll.clj:332)
    at instaparse.gll$parse.invoke(gll.clj:758)
    at instaparse.core$parse.doInvoke(core.clj:83)
    at clojure.lang.RestFn.invoke(RestFn.java:425)
    at code_maat.parsers.hiccup_based_parser$as_grammar_map.invoke(hiccup_based_parser.clj:23)
    at code_maat.parsers.hiccup_based_parser$parse_log.invoke(hiccup_based_parser.clj:90)
    at code_maat.parsers.git$parse_log.invoke(git.clj:62)
    at code_maat.app.app$git__GT_modifications$fn__8177.invoke(app.clj:111)
    at code_maat.app.app$run_parser_in_error_handling_context.invoke(app.clj:80)
    at code_maat.app.app$git__GT_modifications.invoke(app.clj:109)
    at code_maat.app.app$parse_commits_to_dataset.invoke(app.clj:168)
    at code_maat.app.app$run.invoke(app.clj:181)
    at code_maat.cmd_line$_main.doInvoke(cmd_line.clj:63)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at code_maat.cmd_line.main(Unknown Source)

git log --all --numstat --date=short --pretty=format:"--%h--%ad--%aN" --no-renames

produces a file
`
-aa208775--2017-10-26--araut
2 0 src/main/java/com/xxx/yyy/repository/z1.java
23 0 src/main/java/com/xxx/yy/rest/z2.java
30 0 src/main/java/com/xxx/yy/service/z3.java
47 0 src/main/resources/swagger.yml
37 0 src/test/java/com/xxx/yy/rest/z4.java

--c1ffc53b--2017-10-26--Prasad xxx
`
while with single quotes
git log --all --numstat --date=short --pretty=format:'--%h--%ad--%aN' --no-renames
produces

'--aa208775--2017-10-26--araut' 2 0 src/main/java/com/xxx/yyy/repository/z1.java 23 0 src/main/java/com/xxx/yy/rest/z2.java 30 0 src/main/java/com/xxx/yy/service/z3.java 47 0 src/main/resources/swagger.yml 37 0 src/test/java/com/xxx/yy/rest/z4.java '--c1ffc53b--2017-10-26--Prasad xxx'

(note the quotes around the author/date bit)..
the parser will not parse the 2nd option sadly.

I found about it by accident (in an error message). I tried to infer what it does from the code but it would be nicer to have an example and proper documentation.

Hi,

Thanks a lot for creating this awesome project. I'm trying it for our projects:

git log --all -M -C --numstat --date=short --pretty=format:'--%h--%ad--%an' --no-renames > log.log

java -jar target/code-maat-0.9.2-SNAPSHOT-standalone.jar -l log.log -c git -s summary

WARNING: update already refers to: #'clojure.core/update in namespace: incanter.core, being replaced by: #'incanter.core/update
The following errors occurred while parsing your command:
Error while parsing option "-s summary": java.lang.NumberFormatException: For input string: "summary"

Am I doing something wrong?

Btw, your presentation in EuroClojure was the best!

Thanks a lot.

When I try to build code-maatwith lein uberjar under OSX 10.11, the compilation fails:

[525]08:51:[email protected]:code-maat$ lein uberjar
Compiling code-maat.cmd-line
java.lang.UnsupportedClassVersionError: incanter/Matrix : Unsupported major.minor version 52.0, compiling:(incanter/core.clj:2277:1)
Exception in thread "main" java.lang.UnsupportedClassVersionError: incanter/Matrix : Unsupported major.minor version 52.0, compiling:(incanter/core.clj:2277:1)
        at clojure.lang.Compiler.analyze(Compiler.java:6688)
        at clojure.lang.Compiler.analyze(Compiler.java:6625)
        at clojure.lang.Compiler$HostExpr$Parser.parse(Compiler.java:1009)
        at clojure.lang.Compiler.analyzeSeq(Compiler.java:6868)
        at clojure.lang.Compiler.analyze(Compiler.java:6669)
        at clojure.lang.Compiler.analyze(Compiler.java:6625)
        at clojure.lang.Compiler.compile1(Compiler.java:7469)
        at clojure.lang.Compiler.compile(Compiler.java:7541)
        at clojure.lang.RT.compile(RT.java:406)
        at clojure.lang.RT.load(RT.java:451)
        at clojure.lang.RT.load(RT.java:419)
        at clojure.core$load$fn__5677.invoke(core.clj:5893)
        at clojure.core$load.invokeStatic(core.clj:5892)
        at clojure.core$load.doInvoke(core.clj:5876)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at clojure.core$load_one.invokeStatic(core.clj:5697)
        at clojure.core$load_one.invoke(core.clj:5692)
        at clojure.core$load_lib$fn__5626.invoke(core.clj:5737)
        at clojure.core$load_lib.invokeStatic(core.clj:5736)
        at clojure.core$load_lib.doInvoke(core.clj:5717)
        at clojure.lang.RestFn.applyTo(RestFn.java:142)
        at clojure.core$apply.invokeStatic(core.clj:648)
        at clojure.core$load_libs.invokeStatic(core.clj:5774)
        at clojure.core$load_libs.doInvoke(core.clj:5758)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.core$apply.invokeStatic(core.clj:648)
        at clojure.core$require.invokeStatic(core.clj:5796)
        at clojure.core$require.doInvoke(core.clj:5796)
        at clojure.lang.RestFn.invoke(RestFn.java:436)
        at code_maat.cmd_line$loading__5569__auto____36.invoke(cmd_line.clj:6)
        at clojure.lang.AFn.applyToHelper(AFn.java:152)
        at clojure.lang.AFn.applyTo(AFn.java:144)
        at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3652)
        at clojure.lang.Compiler.compile1(Compiler.java:7474)
        at clojure.lang.Compiler.compile1(Compiler.java:7464)
        at clojure.lang.Compiler.compile(Compiler.java:7541)
        at clojure.lang.RT.compile(RT.java:406)
        at clojure.lang.RT.load(RT.java:451)
        at clojure.lang.RT.load(RT.java:419)
        at clojure.core$load$fn__5677.invoke(core.clj:5893)
        at clojure.core$load.invokeStatic(core.clj:5892)
        at clojure.core$load.doInvoke(core.clj:5876)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at clojure.core$load_one.invokeStatic(core.clj:5697)
        at clojure.core$compile$fn__5682.invoke(core.clj:5903)
        at clojure.core$compile.invokeStatic(core.clj:5903)
        at clojure.core$compile.invoke(core.clj:5895)
        at user$eval20$fn__29.invoke(form-init3861789591659076358.clj:1)
        at user$eval20.invokeStatic(form-init3861789591659076358.clj:1)
        at user$eval20.invoke(form-init3861789591659076358.clj:1)
        at clojure.lang.Compiler.eval(Compiler.java:6927)
        at clojure.lang.Compiler.eval(Compiler.java:6917)
        at clojure.lang.Compiler.eval(Compiler.java:6917)
        at clojure.lang.Compiler.load(Compiler.java:7379)
        at clojure.lang.Compiler.loadFile(Compiler.java:7317)
        at clojure.main$load_script.invokeStatic(main.clj:275)
        at clojure.main$init_opt.invokeStatic(main.clj:277)
        at clojure.main$init_opt.invoke(main.clj:277)
        at clojure.main$initialize.invokeStatic(main.clj:308)
        at clojure.main$null_opt.invokeStatic(main.clj:342)
        at clojure.main$null_opt.invoke(main.clj:339)
        at clojure.main$main.invokeStatic(main.clj:421)
        at clojure.main$main.doInvoke(main.clj:384)
        at clojure.lang.RestFn.invoke(RestFn.java:421)
        at clojure.lang.Var.invoke(Var.java:383)
        at clojure.lang.AFn.applyToHelper(AFn.java:156)
        at clojure.lang.Var.applyTo(Var.java:700)
        at clojure.main.main(main.java:37)
Caused by: java.lang.UnsupportedClassVersionError: incanter/Matrix : Unsupported major.minor version 52.0
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:792)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
        at clojure.lang.DynamicClassLoader.loadClass(DynamicClassLoader.java:77)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
        at clojure.lang.DynamicClassLoader.loadClass(DynamicClassLoader.java:77)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
        at clojure.lang.DynamicClassLoader.loadClass(DynamicClassLoader.java:77)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
        at clojure.lang.DynamicClassLoader.loadClass(DynamicClassLoader.java:77)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
        at clojure.lang.DynamicClassLoader.loadClass(DynamicClassLoader.java:77)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
        at clojure.lang.DynamicClassLoader.loadClass(DynamicClassLoader.java:77)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
        at clojure.lang.DynamicClassLoader.loadClass(DynamicClassLoader.java:77)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
        at clojure.lang.DynamicClassLoader.loadClass(DynamicClassLoader.java:77)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
        at clojure.lang.DynamicClassLoader.loadClass(DynamicClassLoader.java:77)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:270)
        at clojure.lang.RT.classForName(RT.java:2168)
        at clojure.lang.RT.classForName(RT.java:2177)
        at clojure.lang.Compiler.resolveIn(Compiler.java:7145)
        at clojure.lang.Compiler.resolve(Compiler.java:7108)
        at clojure.lang.Compiler.analyzeSymbol(Compiler.java:7069)
        at clojure.lang.Compiler.analyze(Compiler.java:6648)
        ... 95 more
Compilation failed: Subprocess failed

Here is my Java version:

[527]09:01:[email protected]:code-maat$ java -version
java version "1.7.0_25"
Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)

I forked to implement a Perforce parser. I have it completed, but when I went to merge with your latest which splits the parsing into chunks, I'm unable to get it functioning again. I've found if I remove all newlines except those between change sets, I can get it parsing again. But, that involves quite a bit of data massaging to clean up the perforce log. Do you have any suggestions? If you look on my perforce branch, you can see my grammar.

Hi guys, I'm trying to run the tests for this project and I'm getting a very cryptic error.

I'm using these Java 1.8 and these other libraries, any idea about what is it?

; CIDER 0.10.0snapshot (package: 20150629.637) (Java 1.8.0_45, Clojure 1.6.0, nREPL 0.2.7)

$ lein test
java.lang.RuntimeException: EOF while reading
 at clojure.lang.Util.runtimeException (Util.java:221)
    clojure.lang.LispReader.readDelimitedList (LispReader.java:1127)
    clojure.lang.LispReader$ListReader.invoke (LispReader.java:972)
    clojure.lang.LispReader.read (LispReader.java:183)
    clojure.core$read.invoke (core.clj:3477)
    clojure.core$read.invoke (core.clj:3475)
    bultitude.core$read_ns_form$fn__525.invoke (core.clj:28)
    bultitude.core$read_ns_form.invoke (core.clj:28)
    bultitude.core$ns_form_for_file.invoke (core.clj:42)
    bultitude.core$namespace_forms_in_dir$iter__531__535$fn__536.invoke (core.clj:56)
    clojure.lang.LazySeq.sval (LazySeq.java:40)
    clojure.lang.LazySeq.seq (LazySeq.java:49)
    clojure.lang.RT.seq (RT.java:484)
    clojure.core$seq.invoke (core.clj:133)
    clojure.core$map$fn__4245.invoke (core.clj:2551)
    clojure.lang.LazySeq.sval (LazySeq.java:40)
    clojure.lang.LazySeq.seq (LazySeq.java:49)
    clojure.lang.Cons.next (Cons.java:39)
    clojure.lang.RT.length (RT.java:1646)
    clojure.lang.RT.seqToArray (RT.java:1587)
    clojure.lang.LazySeq.toArray (LazySeq.java:126)
    clojure.lang.RT.toArray (RT.java:1565)
    clojure.core$to_array.invoke (core.clj:333)
    clojure.core$sort.invoke (core.clj:2828)
    clojure.core$sort.invoke (core.clj:2825)
    leiningen.test$read_args.invoke (test.clj:146)
    leiningen.test$test.doInvoke (test.clj:192)
    clojure.lang.RestFn.invoke (RestFn.java:410)
    clojure.lang.Var.invoke (Var.java:379)
    clojure.lang.AFn.applyToHelper (AFn.java:154)
    clojure.lang.Var.applyTo (Var.java:700)
    clojure.core$apply.invoke (core.clj:626)
    leiningen.core.main$partial_task$fn__6094.doInvoke (main.clj:263)
    clojure.lang.RestFn.invoke (RestFn.java:410)
    clojure.lang.AFn.applyToHelper (AFn.java:154)
    clojure.lang.RestFn.applyTo (RestFn.java:132)
    clojure.lang.AFunction$1.doInvoke (AFunction.java:29)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invoke (core.clj:626)
    leiningen.core.main$apply_task.invoke (main.clj:313)
    leiningen.core.main$resolve_and_apply.invoke (main.clj:319)
    clojure.lang.AFn.applyToHelper (AFn.java:156)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invoke (core.clj:624)
    leiningen.githooks$auto_install.doInvoke (githooks.clj:144)
    clojure.lang.RestFn.invoke (RestFn.java:439)
    clojure.lang.Var.invoke (Var.java:388)
    clojure.lang.AFn.applyToHelper (AFn.java:160)
    clojure.lang.Var.applyTo (Var.java:700)
    clojure.core$apply.invoke (core.clj:626)
    robert.hooke$compose_hooks$fn__11735.doInvoke (hooke.clj:40)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invoke (core.clj:624)
    robert.hooke$run_hooks.invoke (hooke.clj:46)
    robert.hooke$prepare_for_hooks$fn__11740$fn__11741.doInvoke (hooke.clj:54)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.lang.AFunction$1.doInvoke (AFunction.java:29)
    clojure.lang.RestFn.invoke (RestFn.java:421)
    leiningen.core.main$_main$fn__6160.invoke (main.clj:392)
    leiningen.core.main$_main.doInvoke (main.clj:385)
    clojure.lang.RestFn.invoke (RestFn.java:408)
    clojure.lang.Var.invoke (Var.java:379)
    clojure.lang.AFn.applyToHelper (AFn.java:154)
    clojure.lang.Var.applyTo (Var.java:700)
    clojure.core$apply.invoke (core.clj:624)
    clojure.main$main_opt.invoke (main.clj:315)
    clojure.main$main.doInvoke (main.clj:420)
    clojure.lang.RestFn.invoke (RestFn.java:436)
    clojure.lang.Var.invoke (Var.java:388)
    clojure.lang.AFn.applyToHelper (AFn.java:160)
    clojure.lang.Var.applyTo (Var.java:700)
    clojure.main.main (main.java:37)

What does the "commits" column on the churn output represent? If it is meant to represent the total number of commits for the file, then it's reporting incorrectly.

Failing to run code-maat on linux, using lein uberjar or lein run. Can you update this dependency, or verify that this dependency should not be an issue? [Thanks]

[INFO] Unable to find resource 'bouncycastle:bcprov-jdk14:jar:138' in repository clojars (http://clojars.org/repo/)

I am running the Docker Container version of Code Maat and can see it is trying to work but it seems unable to cope with the Git log I have generated using the command you suggest in the documentation. I have tried saving it in various different encodings to see if that fixed the issue but it doesn't. Any other ideas on what might be going wrong?

I am running Docker for Windows for clarity.

codemaat2.log

I just started playing with code-maat, and I find I do something like this:
lein.bat run -l logfile.log -c git | grep -v pom.xml | grep -v test | head
It would be really nice to be able to specify these exclusions on the command-line of code-maat instead piping to commands

Do you have an example of how to analyze commit messages?

I tried -a messages -e fix but I get no results.