zyborg / dotnet-tests-report Goto Github PK

<SNIP>

Build FAILED.

<SNIP>

    0 Warning(s)
    36 Error(s)

Time Elapsed 00:00:21.82
Warning: Execution of tests returned failure: 1

Warning: Execution of tests DID NOT PRODUCE a tests results file

the action does NOT report the check as failed in this situation

the last action is
Mapping check status to test outcome... from Publish-ToCheckRun. It seems this command is crashing out and aborting execution of the check?

Would it be possible to have an option to have a link to the generated GIST - and perhaps the summary of the test result - in the Check Run rather than the full results due to the 65K arbitrary limit on Check Runs?

It does not take that many unit tests to hit this limit with detailed reporting, and it's a pain to subdivide unit tests further.

The Check Run functionality is extremely valuable for easy feedback in the Actions themselves, but sadly it's very hard to use with the limit. The Gists are great, but they are very "non-discoverable" for a team working together. This would allow them to be exposed to the team easily.

I'm seeing this error in each of my calls to this task:

Run zyborg/[email protected]
"C:\Program Files\PowerShell\7\pwsh.exe" -f D:\a\_actions\zyborg\dotnet-tests-report\v1.3.0/action.ps1
New-Item: D:\a\_actions\zyborg\dotnet-tests-report\v1.3.0\action.ps1:40
Line |
  40 |  New-Item -Name $tmpDir -ItemType "directory" -Force
     |  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | The filename, directory name, or volume label syntax is incorrect. :
     | 'D:\a\myrepo\myrepo\D:\a\myrepo\myrepo'

Hello I was wondering if you are able to display app ui screenshots that are shown when running dotnet test. Is it possible to add these screen shots on the report?

Is it possible to stop the execution of an action if any test fails?

I see there's a no_restore but it would seem to make sense to also support a no_build as well?

I've got the action running against a bunch of tests and it runs them all correctly but I don't understand how to construct the badge reference.

It is possible that the build has not completed as the following is output near the bottom of the log;

Resolve Repo Full Name as TheGeneGenieProject/GeneGenie.DataQuality

Adding Check Run

Invoke-WebRequest: /home/runner/work/_actions/zyborg/dotnet-tests-report/v1.0.0/action.ps1:103
Line |
103 | Invoke-WebRequest -Headers $hdr $url -Method Post -Body ($bdy | C …
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| {"message":"Invalid request.\n\nOnly 65535 characters are
| allowed; 174429 were
| supplied.","documentation_url":"https://docs.github.com/rest/reference/checks#create-a-check-run"}

My workflow has the following step based on the sample. Am I doing something daft?

`
- name: Unit tests
uses: zyborg/[email protected]
with:
project_path: GeneGenie.DataQuality.Tests
report_name: unit_test_results
report_title: Unit test results
github_token: ${{ secrets.GITHUB_TOKEN }}
gist_name: unit_tests.md
gist_badge_label: 'Unit tests: %Counters_passed%/%Counters_total%'
gist_token: ${{ secrets.GIST_TOKEN }}

`

I need my tests to compile and run in release mode. An option needs to be added to do so.

It seems that the highest .NET runtime currently supported is .netcore3.1
How should I go about using this workflow action for a .NET 5 project ? Is there planned support for .NET 5 ?

Here is the console error:

error NETSDK1045: The current .NET SDK does not support targeting .NET Core 5.0. Either target .NET Core 3.1 or lower, or use a version of the .NET SDK that supports .NET Core 5.0. [/home/runner/work/DeepK8s/DeepK8s/UnitTests/UnitTests.csproj]

And the workflow file

[...]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Run test & Generate report
        uses: zyborg/[email protected]
        with:
          project_path: UnitTests
          report_name: DeepK8s - Unit Tests
          report_title: DeepK8s - Unit Tests
          github_token: ${{ secrets.GITHUB_TOKEN }}
          gist_name: deepk8s_unit_tests.md
          gist_badge_label: 'DeepK8s - Unit Tests: %Counters_passed%/%Counters_total%'
          gist_token: ${{ secrets.GIST_TOKEN }}
          set_check_status_from_test_outcome: true

The tests won't produce any details in reports:

I assume it's because of tests count, because i was using this action in my prev. project and it had the same behaviour. When there are about 20 tests - it works, but now my project has about 80 tests overall and all the reports produced by this action are empty.

how do I get this value in my workflow file?

Is GIST_TOKEN an existing one like GITHUB_TOKEN or do you have to create it?

The following error occurs:

Invoke-WebRequest: /home/runner/work/_actions/zyborg/dotnet-tests-report/v1/action.ps1:149
Line |
149 | Invoke-WebRequest -Headers $hdr $url -Method Post -Body ($bdy | C …
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| {"message":"Invalid request.\n\nOnly 65535 characters are
| allowed; 266941 were
| supplied.","documentation_url":"https://docs.github.com/rest/reference/checks#create-a-check-run"}

I would be nice to have an input to choose the kind of gist: public (by default) or secret.

Actually once created as public a gist cannot be turned to secret.

I've noticed that while the action doesn't fail (and indeed in the output I see a 201 Created status), the test report never appears when you try and manually re-run the jobs (using the re-run jobs button when viewing a workflow results.

I wonder if there's some extra tracking info that needs to be supplied to differentiate between different job runs?

I'm seeing some strange behaviour with my test process. It runs my tests successfully and seems to generate an output, but then tries and fails to add a check run, citing invalid JSON:

Invoke-WebRequest: /home/runner/work/_actions/zyborg/dotnet-tests-report/v1.3.0/action.ps1:144
Line |
 144 |      Invoke-WebRequest -Headers $hdr $url -Method Post -Body ($bdy | C …
     |      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | {"message":"Problems parsing
     | JSON","documentation_url":"https://docs.github.com/rest/reference/checks#create-a-check-run"}

It also fails to create a results gist, with the same error at line 235.

I did notice it failed to build the solution, but since the tests ran successfully I'm not sure if this is related:

 Results File: /home/runner/work/geodesy-csharp/geodesy-csharp/_TMP/test-results.trx

Test Run Failed.
Total tests: 81
     Passed: 75
     Failed: 6
 Total time: 1.6010 Seconds
     3>Done Building Project "/home/runner/work/geodesy-csharp/geodesy-csharp/geodesy-test/geodesy-test.csproj" (VSTest target(s)) -- FAILED.
     1>Done Building Project "/home/runner/work/geodesy-csharp/geodesy-csharp/geodesy-csharp.sln" (VSTest target(s)) -- FAILED.

Build FAILED.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:05.25
Warning: Execution of tests returned failure: 1

My tests are running successfully, but the creation of the check run is failing:

Adding Check Run

Mapping check status to test outcome...

All tests passed

Invoke-WebRequest: D:\a\_actions\zyborg\dotnet-tests-report\v1.2.0\action.ps1:123
Line |
 123 |      Invoke-WebRequest -Headers $hdr $url -Method Post -Body ($bdy | C …
     |      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | {"message":"Invalid request.\n\nOnly 65535 characters are allowed; 85289 were
     | supplied.","documentation_url":"https://docs.github.com/rest/reference/checks#create-a-check-run"}

I currently have 862 tests (of which 12 are skipped).

I would like to expose a link to the gist one way or another after the dotnet-tests-report stage.

We provide its filename with the gist_name input, but it is complicated to retrieve the resourcePath hash which looks like
https://gist.github.com/<username>/db44a432da467d8c021f26b3b8f4c074

Could the gist path be returned as an output?

If tests fail, next jobs (such as deployment) should not start. Therefore, if a unit test fails, the whole job needs to fail. I need an option to achieve this.

The mkdir line is failing when running on Ubuntu agents because while on Windows, mkdir is a PowerShell alias to New-Item, on Linux it is not aliased and instead runs the native mkdir command.

eg.

Hi I have a fairly simple project configuration. When using:

- name: use this action, test solution dir
  uses: zyborg/dotnet-tests-report@v1
  if: always()
  with:        
    report_name: my_project_tests
    report_title: My Project Tests
    github_token: ${{ secrets.GITHUB_TOKEN }}

it appends for some reason the WorkSpace path twice.

Noticed that it's not getting created relative to the root on Linux.

eg.

PS /home/david> $tmpDir = [System.IO.Path]::Combine($PWD, '_TMP')
PS /home/david> $tmpDir
/home/david/_TMP
PS /home/david> New-Item -Name $tmpDir -ItemType Directory -Force -ErrorAction Ignore

    Directory: /home/david/home/david

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d----          07/23/2021    15:44                _TMP

Looks like New-Item is always relative to current directory on Linux. Strange but oh well.

It would be great if there were a way to either add additional arbitrary arguments to the call to dotnet test, or specifically, if there were a way to cause the dotnet test run to collect code coverage information (i.e. pass --collect:"XPlat Code Coverage" for coverlet).

Because of the wide variety of tools available to collect coverage information, it would probably be significantly easier to implement a way to pass arbitrary arguments, and allow the user to determine the correct setup to use.

See Specify check suite when creating a checkrun #24616

First of all thanks a lot for this awesome action!
We are using multi-stage docker builds so our tests run inside docker.
I was trying to use the test_restults_path variable to point to a directory because we are writing all the tests to a directory.

It seems like test_results_path is expected to be an absolute path to a single file.
Is there a possibility to point to a directory and then generate a test report for all the trx files present in the directory?

If I do provide an absolute path, I am getting this error

Export-Clixml: /home/runner/work/_actions/zyborg/dotnet-tests-report/v1.0.0/action.ps1:279
Line |
 279 |      Export-Clixml -InputObject $testResult -Path $result_clixml_path
     |      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Could not find a part of the path
     | '/home/runner/work/auditlogas/auditlogas/_TMP/dotnet-test-result.clixml'.

The above error seems to be happening because the _TMP directory doesn't exist.
If I create $PWD/_TMP prior to running this action then it works as expected.

I would really appreciate some guidance

Thanks!

If you're separating out workflows like in https://docs.github.com/en/code-security/supply-chain-security/keeping-your-dependencies-updated-automatically/automating-dependabot-with-github-actions#handling-push-events then you might want to run this action from a workflow triggered by the workflow_run event.

In that case the current logic doesn't pick up the correct commit SHA. In this situation it would be great to have a property that can be set to the SHA..

eg.

        with:
          report_name: Unit Tests
          report_title: Unit Tests
          github_token: ${{ secrets.GITHUB_TOKEN }}
          test_results_path: ./TestResults.trx
          sha: ${{ github.event.workflow_run.head_sha }}

When this action runs on a PR, it fails. Example: https://github.com/madskristensen/RestClientVS/runs/4563113880?check_suite_focus=true#step:6:275

I got this error from the dotnet-tests-report action as run here:

Invoke-WebRequest: D:\a\_actions\zyborg\dotnet-tests-report\v1.3.0\action.ps1:144
Line |
 144 |      Invoke-WebRequest -Headers $hdr $url -Method Post -Body ($bdy | C …
     |      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | {"message":"Invalid request.\n\nOnly 65535 characters are allowed; 93137 were
     | supplied.","documentation_url":"https://docs.github.com/rest/reference/checks#create-a-check-run"}

There's got to be some other way to present the test results...

If running in a PR, it would be nice to add a comment after the test run. This would be a little more visible than the check report.

Maybe the badge and a list of any failed tests?