tyrrrz / githubactionstestlogger Goto Github PK

Version

2.2.0

Details

After updating to 2.2.0, if more than one test project is included in a GitHub Actions workflow where other content is also written to the step summary, the title of the second test run report does not render correctly.

Attached are two screenshots showing the rendering before and after the change.

Before

Workflow

After

Steps to reproduce

Clone justeattakeaway/httpclient-interception@3274754 and run build.ps1 in GitHub Actions.

Thanks for a great module!

This currently emits warnings, which is a great thing. Unfortunately, the Github Actions UI for annotations isn't very good, and simply displays warning and error annotations as a big list, without any sorting or grouping. This means that if I have many test skips, I have to scour the list for any errors in between those errors... A good example is this build.

Ideally Github will make their UI better (I sent a request on their help/support page), but in the meantime it may be good to provide an option to disable showing warnings altogether. Even without this UI bug, some people are likely not interested in these.

Version

2.0.1

Details

.NET 6

The error Could not find a test logger with AssemblyQualifiedName, URI or FriendlyName 'GitHubActions'. is being logged. Tests are outputted to the summary as expected, but I suspect this error is the reason my job is exiting with code 1.

Steps to reproduce

• Set up as explained in repo
• Run workflow

Just curious if you considered using a GitHub Actions problem matcher instead of a different logger. Seems like it might be better to just have a problem matcher enabled maybe in the dotnet GitHub Action that creates problems for failing tests.

Details

I'd love to be able to see which test suits are taking the longest to run without having to click to expand each section. I.e. instead of this:

I'd love to see this:

Version

2.3.3

Platform

.NET 7.0

Steps to reproduce

  - name: Test with the dotnet CLI
    run: >
      dotnet test
      --no-build
      --configuration Release
      --logger "GitHubActions;summary.includePassedTests=true;summary.includeSkippedTests=true"
      --
      RunConfiguration.CollectSourceInformation=true

See: https://github.com/passwordless/passwordless-server/actions/runs/6325650703

Details

• Expected behavior: If NuGet is not present, make sure tests still pass, but skip the reporting then for that project?

Checklist

• I have looked through existing issues to make sure that this bug has not been reported before
• I have provided a descriptive title for this issue
• I have made sure that that this bug is reproducible on the latest version of the package
• I have provided all the information needed to reproduce this bug as efficiently as possible
• I have sponsored this project

Details

Currently, the test runs are all listed sequentially without any separation. It would be nice to group them similarly to how it's done in IDEs and test runners.

I spent an entire day diagnosing why this package didn't work and found out that

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.5.0" />

is needed instead of the default old version 15.0.0...

Details

Resolve the repository URL and then link to test definitions from job summary. Make sure to include current commit hash in the URL.

Eine Lösung für die ToRomanNumerals Kata

Version

2.0

Details

I am getting duplicate github annotations for build warnings. Any ideas how I can fix them?

Here is a sample output:
https://github.com/exceptionless/Exceptionless/actions/runs/2458030356

Steps to reproduce

Run build with warnings using this library for output

Details

Add $framework

Details

I have a few test projects in my solution. I'd rather not add this project's package to each test project, but only install it in the workflow. I don't know if this can be achieved in some way, as there's a strong focus on nuget packages it seems. Ideally I'd only have to run something like dotnet tool install (-g) ... to be able to use this logger.

I've also tried using the dll directly, but that doesn't work as well:

dotnet test --logger=/Users/bouke/Downloads/githubactionstestlogger.1.2.0/lib/netstandard2.0/GitHubActions.TestLogger.dll

Starting test execution, please wait...
Could not find a test logger with AssemblyQualifiedName, URI or FriendlyName 'file:///Users/bouke/Downloads/githubactionstestlogger.1.2.0/lib/netstandard2.0/GitHubActions.TestLogger.dll'.

I've been testing out this project (works great btw 👍) and I think one thing that's missing is including the framework/runtime version in the annotation. I wanted to start a discussion here, first of all to say thanks for a great project 👏 👏 👏 - and in case anyone else ran into the same thing.

I'm in a situation where I need to test both netcoreapp3.1 and net5 and I don't want to be confusing to people why they see the same error reported twice:

It would be great if there was an option to include the runtime version, or if not to find some strategy where I could customize this. For instance right now I'm not specifying a target framework at the command line, and letting dotnet test do its magic. I could easily change this if it makes the error reporting experience better.

One of the ideas here would be to use a matrix and make that part of the job name.

Example:

test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        dotnet-version: [3.1.x, 5.0.x]

    name: Test ${{ matrix.dotnet-version }}

Thanks for the package - it's just what I was looking for.

Cant seem to get it working when i run my tests in docker-compose on github actions though. Perhaps because the lines are prefixed with the container name in the output?

tests_1  | [xUnit.net 00:00:00.77]   Finished:    xxxxx.Tests
tests_1  | 
tests_1  | Test run for /sln/tests/xxxxx/bin/Release/netcoreapp3.1/xxxxx.dll(.NETCoreApp,Version=v3.1)
tests_1  | ::error file=/sln/tests/xxxxx/Demo.cs,line=8::xxxxx.Tests.Demo.This_will_fail: Assert.True() Failure
tests_1  |   X xxxxxxxx.Tests.Demo.This_will_fail [2ms]
tests_1  |   Error Message:
tests_1  |    Assert.True() Failure
tests_1  | Expected: True
tests_1  | Actual:   False
tests_1  |   Stack Trace:
tests_1  |      at xxxxx.Tests.Demo.This_will_fail() in /sln/tests/xxxxx.Tests/Demo.cs:line 8

Version

2.0.2

Details

If deterministic builds are enabled for a project, links to failing tests from the stack traces 404 instead of linking correctly to the code as they contain an incorrect /_/ segment in the GitHub link.

See this example for such a broken link.

A comparison of a broken and working link is shown below:

- https://github.com/martincostello/dependabot-helper/blob/06003a012895c3b1d70f7f1829e3fdd128180d0c/_/tests/DependabotHelper.Tests/ApiTests.cs#L1363
+ https://github.com/martincostello/dependabot-helper/blob/06003a012895c3b1d70f7f1829e3fdd128180d0c/tests/DependabotHelper.Tests/ApiTests.cs#L1363

Steps to reproduce

Clone martincostello/dependabot-helper@74df38b and build in a GitHub Actions workflow with build.ps1 in the root of the repository.

It seems nunit (possibly xunit) test results are showing only in console as a summary of passed/failed - should any additional output be produced with GitHubActions enabled?

Details

I really like the summary emitted by the test logger. However, will the aggregate statistics (total pass/fail/skip) are great, the test-per-test listing is huge, for projects which have lots of test (see this as an example).

I'd suggest showing this collapsed by default (so users click to open it), or possibly to make it opt-in/opt-out in the build (but keep the aggregate statistics!).

A bit of an interesting issue since it's fair to say I'm not using this tool exactly as would be expected. Essentially we build in docker and test in docker, the problem here is that the file paths don't match the git repo.

What options are there for manipulating those paths which are output? The only way I can think of now is changing the output directories inside the docker container to more closely match the git repository.

I'm getting this error Could not find a test logger with AssemblyQualifiedName, URI or FriendlyName 'GitHubActions'

I'm referencing Microsoft.NET.Test.Sdk version 16.6.0 which I believe is the latest.

Any tips on how to debug this or what I might be missing?