alfert / coverex Goto Github PK

I have a module with 0% of coverage:

0.00% | BankApiWeb.Endpoint

I've put it into my mix.exs:
test_coverage: [tool: Coverex.Task, ignore_modules: [BankApiWeb.Endpoint]]

The file is still there.. it's not ignored.

I have 2 apps inside my umbrella app.

It would be great if there was a way to add a list of modules to ignore from the cover report in the mix file. There are many libs out there that macro gen modules/function over basic structs. This makes cover reports look very empty because only one of the macro's functions are called.

For example we have a library that takes in a module struct and adds a bunch of serialize/deserialize macros for thrift. We only may use 2 function API out of 20 possible but the cover report shows only 10% coverage.

Thanks!

-Chris

This is a small feature request which would be great:

In excoveralls (link https://github.com/parroty/excoveralls) they have a nice feature that when a test is completed it will print out a summary report like so:

$ MIX_ENV=test mix coveralls
...
----------------
COV    FILE                                        LINES RELEVANT   MISSED
100.0% lib/excoveralls/general.ex                     28        4        0
 75.0% lib/excoveralls.ex                             54        8        2
 94.7% lib/excoveralls/stats.ex                       70       19        1
100.0% lib/excoveralls/poster.ex                      16        3        0
 95.5% lib/excoveralls/local.ex                       79       22        1
100.0% lib/excoveralls/travis.ex                      23        3        0
100.0% lib/mix/tasks.ex                               44        8        0
100.0% lib/excoveralls/cover.ex                       32        5        0
[TOTAL]  94.4%
----------------

This is nice for day to day testing because you can see your coverage increasing as you add tests without having to goto a web browsers. We like the visual reports coverex currently provides better, but it was the one feature some on our team thought was nice about excoveralls.

As a super nice to have would be to print this report out as a json or csv file so our build systems can parse this and report a build as a failure if test coverage drops bellow a pre-defined threshold.

Thanks,

-Chris

Hello. I'm trying to add coverage to my elixir project but I have a problem:

~/work/user(branch:master*) » mix test --cover
Coverex compiling modules ... 
compile_path is "/home/user/work/app/_build/test/lib/app/ebin"
Options are: [output: "cover", tool: Coverex.Task]
** (UndefinedFunctionError) function :cover.start/0 is undefined (module :cover is not available)
    :cover.start()
    lib/coverex/task.ex:23: Coverex.Task.start/2
    (mix) lib/mix/tasks/test.ex:193: Mix.Tasks.Test.run/1
    (mix) lib/mix/task.ex:296: Mix.Task.run_task/3
    (mix) lib/mix/cli.ex:58: Mix.CLI.run_task/2

I have the same problem when I download coverex code and run mix task --cover
I was looking into code and I wasn't able to find problem.

This would allow providing alternative outputs for coverage reports like XUnit-like or better HTML output.

We use the library amnesia which does some macro tricks to make using mnesia easier. However when I run coverex on it get a errors about mod_entry is nil. It would be nice to suppress these errors because it looks like the test is failing and the are not. Also the coverage reports are generating as well.

Example error:

15:31:55.002 [error] mod_entry is nil and cover = [{{Database.Builds.App, 0}, 0}, {{Database.Builds.App, 0}, 3}, {{Database.Builds.App, 0}, 0}, {{Database.Builds.App, 0}, 0}, {{Database.Builds.App, 0}, 1}, {{Database.Builds.App, 0}, 0}, {{Database.Builds.App, 0}, 0}, {{Database.Builds.App, 0}, 0}, {{Database.Builds.App, 0}, 1}, {{Database.Builds.App, 0}, 0}, {{Database.Builds.App, 0}, 0}, {{Database.Builds.App, 0}, 0}, {{Database.Builds.App, 0}, 2}, {{Database.Builds.App, 0}, 0}, {{Database.Builds.App, 0}, 0}, {{Database.Builds.App, 0}, 0}, {{Database.Builds.App, 0}, 0}, {{Database.Builds.App, 0}, 0}, {{Database.Builds.App, 0}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 1}, {{Database.Builds.App, 15}, 1}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 1}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, 15}, 0}, {{Database.Builds.App, ...}, 0}, {{...}, ...}, {...}, ...]

15:31:55.019 [error] mod_entry is nil and cover = [{{Database.Builds, 0}, 0}, {{Database.Builds, 0}, 0}, {{Database.Builds, 0}, 1}, {{Database.Builds, 0}, 0}, {{Database.Builds, 0}, 0}, {{Database.Builds, 0}, 0}, {{Database.Builds, 0}, 0}, {{Database.Builds, 1}, 0}, {{Database.Builds, 1}, 0}, {{Database.Builds, 6}, 1}, {{Database.Builds, 6}, 0}, {{Database.Builds, 6}, 1}, {{Database.Builds, 6}, 2}, {{Database.Builds, 6}, 0}, {{Database.Builds, 6}, 0}, {{Database.Builds, 6}, 0}, {{Database.Builds, 6}, 0}, {{Database.Builds, 6}, 0}, {{Database.Builds, 6}, 2}]

What is needed to get simultaneous output in umbrella project is that you need to add:

     test_coverage: [output: "../../cover"],

In your Mix config. However this still doesn't provide CLI output.

Hey there,

I'm the founder of Codecov and would like to have an upload option for customers to send reports to Codecov.

Here is an example upload:

curl -X POST -d '<json>' \
    https://codecov.io/upload/v2?service=travis&branch=$TRAVIS_BRANCH&commit=$TRAVIS_COMMIT&build=$TRAVIS_JOB_NUMBER&pr=$TRAVIS_PULL_REQUEST&job=$TRAVIS_JOB_ID&slug=$TRAVIS_REPO_SLUG&tag=$TRAVIS_TAG

{
  "coverage": {
    "filename": [null, 1, 0, ...], //option-1
    "filename_2": {  //option-2
      "1": 1,
      "3": 0
    }
  }
}

Option 1 list of [line_number_coverage] index [0] is always null, index[1] is line one, etc.
Option 2 object of linenum: coverage

Full list of supported CI providers

I'm happy to help our where I can!

Cheers,
Steve

I'm experiencing issues with coverex in a project using the official elixir docker image.

OS is Debian Jessie
Elixir version is 1.3.1
Erlang version is 19

The tests are run by the CircleCI bot within the Docker environment, but before they are executed, they crash with the following error report(s):

Coverex compiling modules ... 
compile_path is "/myproject.git/_build/test/lib/myproject/ebin"
Options are: [output: "cover", tool: Coverex.Task]

09:54:29.254 [error] Process #PID<0.338.0> raised an exception
** (ArgumentError) argument error
    (stdlib) erl_anno.erl:318: :erl_anno.set(:location, -1, [location: -1, generated: true])
    (stdlib) erl_parse.yrl:1516: anonymous fn/3 in :erl_parse.map_anno/2
    (stdlib) erl_parse.yrl:1645: :erl_parse.modify_anno1/3
    (stdlib) erl_parse.yrl:1658: :erl_parse.modify_anno1/3
    (stdlib) erl_parse.yrl:1637: :erl_parse.modify_anno1/3
    (stdlib) erl_parse.yrl:1658: :erl_parse.modify_anno1/3
    (stdlib) erl_parse.yrl:1648: :erl_parse.modify_anno1/3
    (stdlib) erl_parse.yrl:1658: :erl_parse.modify_anno1/3
** (exit) an exception was raised:
    ** (ArgumentError) argument error
        (stdlib) erl_anno.erl:318: :erl_anno.set(:location, -1, [location: -1, generated: true])
        (stdlib) erl_parse.yrl:1516: anonymous fn/3 in :erl_parse.map_anno/2
        (stdlib) erl_parse.yrl:1645: :erl_parse.modify_anno1/3
        (stdlib) erl_parse.yrl:1658: :erl_parse.modify_anno1/3
        (stdlib) erl_parse.yrl:1637: :erl_parse.modify_anno1/3
        (stdlib) erl_parse.yrl:1658: :erl_parse.modify_anno1/3
        (stdlib) erl_parse.yrl:1648: :erl_parse.modify_anno1/3
        (stdlib) erl_parse.yrl:1658: :erl_parse.modify_anno1/3
    cover.erl:583: :cover.call/1
    lib/coverex/task.ex:24: Coverex.Task.start/2
    (mix) lib/mix/tasks/test.ex:193: Mix.Tasks.Test.run/1
    (mix) lib/mix/task.ex:296: Mix.Task.run_task/3
    (mix) lib/mix/task.ex:328: Mix.Task.run_alias/3
    (mix) lib/mix/task.ex:261: Mix.Task.run/2
    (mix) lib/mix/cli.ex:58: Mix.CLI.run_task/2


09:54:29.257 [error] Process #PID<0.340.0> raised an exception
** (ArgumentError) argument error
    (stdlib) erl_anno.erl:318: :erl_anno.set(:location, -1, [location: -1, generated: true])
    (stdlib) erl_parse.yrl:1516: anonymous fn/3 in :erl_parse.map_anno/2
    (stdlib) erl_parse.yrl:1645: :erl_parse.modify_anno1/3
    (stdlib) erl_parse.yrl:1658: :erl_parse.modify_anno1/3
    (stdlib) erl_parse.yrl:1637: :erl_parse.modify_anno1/3
    (stdlib) erl_parse.yrl:1658: :erl_parse.modify_anno1/3
    (stdlib) erl_parse.yrl:1648: :erl_parse.modify_anno1/3
    (stdlib) erl_parse.yrl:1658: :erl_parse.modify_anno1/3

09:54:29.267 [error] Process #PID<0.350.0> raised an exception
** (ArgumentError) argument error
    (stdlib) :ets.insert(:cover_internal_data_table, {{:bump, Myproject.Mymodule, :some_func, 2, 1, 19}, 0})
    cover.erl:1789: :cover.munge_body/4
    cover.erl:1740: :cover.munge_clauses/4
    cover.erl:1719: :cover.munge/4
    cover.erl:1635: :cover.transform_2/5
    cover.erl:1620: :cover.transform/3
    cover.erl:1568: :cover.do_compile_beam2/5
    cover.erl:1478: :cover.do_compile_beam/3

The last paragraph gets printed multiple times for all the different modules that are being tested.

There should be no need to start httpoison by configuring mix.exs. Even more, this is not useful as test only dependency.

I've got a project that's got {:poison, "~> 1.2.1"}, and so I get this error if I try to add coverex to the project:

Running dependency resolution
Unlocked:   coverex
** (Mix) Hex dependency resolution failed, relax the version requirements or unlock dependencies

When running my test suite with mix test --cover I get these errors. This worked fine before upgrading to Elixir 1.3.2 and Erlang 19.

    13:23:51.609 [error] Process #PID<0.291.0> raised an exception
** (ArgumentError) argument error
    (stdlib) :ets.insert(:cover_internal_clause_table, {MyApp.TokenError, [{MyApp.TokenError, :__info__, 1, 1, 1}, {MyApp.TokenError, :__info__, 1, 2, 1}, {MyApp.TokenError, :__info__, 1, 3, 1}, {MyApp.TokenError, :__struct__, 0, 1, 1}, {MyApp.TokenError, :__struct__, 1, 1, 2}, {MyApp.TokenError, :exception, 1, 1, 1}, {MyApp.TokenError, :exception, 1, 2, 2}, {MyApp.TokenError, :message, 1, 1, 3}]})
    cover.erl:1584: :cover.do_compile_beam2/5
    cover.erl:1478: :cover.do_compile_beam/3
    (stdlib) lists.erl:1239: :lists.map/2
    cover.erl:2732: anonymous fn/2 in :cover.pmap_spawn/4

13:23:51.609 [error] Process #PID<0.286.0> raised an exception
** (ArgumentError) argument error
    (stdlib) :ets.insert(:cover_internal_clause_table, {MyApp.PageController, [{MyApp.PageController, :__info__, 1, 1, 1}, {MyApp.PageController, :__info__, 1, 2, 1}, {MyApp.PageController, :__info__, 1, 3, 1}, {MyApp.PageController, :action, 2, 1, 2}, {MyApp.PageController, :"action (overridable 2)", 2, 1, 2}, {MyApp.PageController, :call, 2, 1, 1}, {MyApp.PageController, :index, 2, 1, 3}, {MyApp.PageController, :init, 1, 1, 1}, {MyApp.PageController, :phoenix_controller_pipeline, 2, 1, 4}]})
    cover.erl:1584: :cover.do_compile_beam2/5
    cover.erl:1478: :cover.do_compile_beam/3
    (stdlib) lists.erl:1239: :lists.map/2
    cover.erl:2732: anonymous fn/2 in :cover.pmap_spawn/4

13:23:51.611 [error] Process #PID<0.288.0> raised an exception
** (ArgumentError) argument error
    (stdlib) :ets.insert(:cover_internal_clause_table, {MyApp.RegistrationView, [{MyApp.RegistrationView, :__info__, 1, 1, 1}, {MyApp.RegistrationView, :__info__, 1, 2, 1}, {MyApp.RegistrationView, :__info__, 1, 3, 1}, {MyApp.RegistrationView, :__phoenix_recompile__?, 0, 1, 1}, {MyApp.RegistrationView, :__resource__, 0, 1, 1}, {MyApp.RegistrationView, :__templates__, 0, 1, 1}, {MyApp.RegistrationView, :render, 1, 1, 1}, {MyApp.RegistrationView, :render, 2, 1, 1}, {MyApp.RegistrationView, :render, 2, 2, 1}, {MyApp.RegistrationView, :render, 2, 3, 1}, {MyApp.RegistrationView, :render, 2, 4, 1}, {MyApp.RegistrationView, :render_template, 2, 1, 1}, {MyApp.RegistrationView, :render_template, 2, 2, 1}, {MyApp.RegistrationView, :render_template, 2, 3, 1}, {MyApp.RegistrationView, :render_template, 2, 4, 1}, {MyApp.RegistrationView, :render_template, 2, 5, 1}, {MyApp.RegistrationView, :template_not_found, 2, 1, 1}]})
    cover.erl:1584: :cover.do_compile_beam2/5
    cover.erl:1478: :cover.do_compile_beam/3
    (stdlib) lists.erl:1239: :lists.map/2
    cover.erl:2732: anonymous fn/2 in :cover.pmap_spawn/4

Something I suspect is noteworthy, is that I get the same errors with excoveralls as I get with this project so I suspect it might be related to the underlying erlang cover library?

If you look at any of the coverage reports at https://coveralls.io/github/scouten/sqlite_ecto2, the lines reported as covered vs not covered (even executable vs not) appear to be almost completely random.

One particularly crazy example is https://coveralls.io/builds/9415936/source?filename=lib%2Fsqlite_ecto%2Fquery.ex, in which lines 117 and 119 are shown as being hit multiple times, but line 121 is missed. All three of those lines are comments.

When I look at HTML results for the same file generated locally (not on Coveralls), the results look reasonable. (I haven't analyzed line-by-line, but it looks at least credible.)

https://gist.github.com/alexrp/2cfc32f1fa71b7274357

There is a version 1.4.14 for coverex published on hex.pm, however when pulling that into a project, it fails to compile .. the deps directory ends up being empty!

Pinning the version to 1.4.13 works, however.

Also, there is no 1.4.14 release (or any release after 1.4.10?) in the git repository .. is this intentional?

Cheers, and thanks for the nice library!

I have multiple modules where Module Coverage Report says there are uncovered lines, but the Coverage Source code shows no red. Where are the missing lines? Is there some way to see how the count is being generated so I can find the missing lines outside of the Coverage Source code html?

Noticed these when building with Elixir 1.0.2:

lib/coverex/source.ex:101: warning: variable m is unused
lib/coverex/task.ex:81: warning: variable body is unused

Run coverage on a file that contains defstruct and that line will be reported as missed, rather than unreachable.

A simple example: https://coveralls.io/builds/12690375/source?filename=lib%2Frx%2Fobservable%2Fcreate_stage.ex

I see the following report from covered sometimes:

In this case it is saying the if statement never was executed however the code inside the if statement did. I am not sure if this is because of how elixir parses the "if" macro or if it is a bug in coverex, but I thought I would report it.

-Chris

I'd like to use coverex with my current project. Unfortunately installing creates a conflict with the latest version of poison (1.5).

I don't know if there will be breaking changes but I guess coverex should work fine both with 1.4 and 1.5 of poison.

Identify a CI format that can be easily interpreted by CI tools such as Jenkins etc.

Assume I have created a mix project 'covertest', the following module ./lib/covertest.ex:

defmodule Covertest do
  def x(1), do: 2
  def x(:z), do: 45
  def y(45), do: 34
end

and the following test file ./lib/covertest_test.exs:

defmodule CovertestTest do
  use ExUnit.Case

  test "covertest" do
    assert Covertest.x(1) == 2
    assert Covertest.x(:z) == 45
  end
end

Then running mix test --cover generates a report where all the lines are marked as class irrelevant in the html. I would have expected two to be marked as covered and one as not covered.

I haven't figured out where the problem is yet. I'm running Elixir 1.0.3.

When I run the coverage report for a module, I get uncovered lines in the special __info__ function. Is this function supposed to be tracked?

I'm using Elixir 1.2.2 and :coverex, "~> 1.4.7"

The annotated source code files are simple ascii files. We should provide a better looking format and provide anchor points for functions and modules such that the tables can link directly to the functions or modules, respectively