Deion Hello, I have an automated E2E test project on Azure a

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

microsoft,vstest

Comments (15)

nohwnd commented on June 9, 2024 2

This seems strange to reference <PackageReference Include="Microsoft.TestPlatform" Version="17.10.0-preview-24080-01" /> but it should not hurt anything, maybe you are using it to get the whole TP into your .nuget cache.

There is most likely no Foo.Tests.runtimeconfig.json in the output folder. Which means your test project is not built as exe for some reason. Can you try adding <OutputType>exe</OutputType> property to your .csproj?

from vstest.

ksidirop-laerdal commented on June 9, 2024 1

Facing the same issue with a dotnet8 / xunit project in Azure. In my localdev I have the test-dlls published properly under 'bin/Release/net8.0/publish/'. I then invoke:

dotnet  test  Foo.Tests/bin/Release/net8.0/publish/Foo.Tests.dll  --verbosity normal  --logger trx;LogFileName=Laerdal.Assets.Installers.Tests.trx  --results-directory TestResults

This works on localdev but fails when running it in Azure with the following error message:

  Starting test execution, please wait...
  A total of 1 test files matched the specified pattern.
##[error]Testhost process for source(s) 'D(0,0): Error : A fatal error was encountered. The library 'hostpolicy.dll' required to execute the application was not found in 'D:\a\1\s\Foo\bin\Release\net8.0\publish\'.
Testhost process for source(s) 'D : \a\1\s\Foo\bin/Release/net8.0/publish/Foo.Tests.dll' exited with error : A fatal error was encountered. The library 'hostpolicy.dll' required to execute the application was not found in 'D:\a\1\s\Foo\bin\Release\net8.0\publish\'. [D:\a\1\s\Laerdal.Scripts\Laerdal.Builder.targets]
  Failed to run as a self-contained app.
    - The application was run as a self-contained app because 'D:\a\1\s\Foo\bin\Release\net8.0\publish\testhost.runtimeconfig.json' was not found.
    - If this should be a framework-dependent app, add the 'D:\a\1\s\Foo\bin\Release\net8.0\publish\testhost.runtimeconfig.json' file and specify the appropriate framework.
  . Please check the diagnostic logs for more information.
  
  Test Run Aborted.
  Results File: D:\a\1\s\TestResults\Laerdal.Assets.Installers.Tests.trx
##[error]Laerdal.Scripts\Laerdal.Builder.targets(151,9): Error MSB3073: The command "..." exited with code 1.
D:\a\1\s\Laerdal.Scripts\Laerdal.Builder.targets(151,9): error MSB3073: The command "..." exited with code 1.

Using the following nugets:

        <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.10.0-preview-24080-01" />
        <PackageReference Include="Microsoft.TestPlatform" Version="17.10.0-preview-24080-01" />
        <PackageReference Include="Microsoft.TestPlatform.TestHost" Version="17.10.0-preview-24080-01" />

        <PackageReference Include="xunit" Version="2.4.2"/>
        <PackageReference Include="xunit.runner.visualstudio" Version="2.5.6">
            <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
            <PrivateAssets>all</PrivateAssets>
        </PackageReference>
        <PackageReference Include="coverlet.collector" Version="6.0.0">
            <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
            <PrivateAssets>all</PrivateAssets>
        </PackageReference>

from vstest.

ksidirop-laerdal commented on June 9, 2024 1

Ok here's how I solved it: I simply reverted to calling 'dotnet test' directly on the .csproj while making sure to specify the correct configuration in the build script - somehow this solved all the problems I was witnessing:

        <PropertyGroup>
            <Params>$(Params) &quot;Foo.Tests.csproj&quot;</Params>
            <Params>$(Params) --logger:&quot;trx;LogFileName=Foo.Tests.trx&quot;</Params>
            <Params>$(Params) --verbosity:&quot;minimal&quot;</Params>
            <Params>$(Params) --configuration:&quot;Release&quot;</Params>
            <Params>$(Params) --results-directory:&quot;TestResults&quot;</Params>
        </PropertyGroup>

        <Message Importance="High" Text=""/>
        <Message Importance="High" Text="dotnet  test  $(Params)"/>
        <Message Importance="High" Text=""/>

        <Exec Command="  dotnet  test  $(Params)  "
              EchoOff="false"
              ConsoleToMSBuild="true"
              WorkingDirectory="$(RootFolder)"/>

The moral for me is that 'publish' doesn't quite publish all assets needed for the test-suite to be run properly.

from vstest.

nohwnd commented on June 9, 2024 1

@dfederm Thanks for pointing this out. I've made this to be the default in vstest@2 task on azdo some time ago. And even updated the doc for vstest@2. But now there is a new version of the task and it does not have that change. Made a bug here: microsoft/azure-pipelines-tasks#19555 unfortunately we don't own this task . :/

from vstest.

nohwnd commented on June 9, 2024

Could you check that there are both runtimeconfig.json and deps.json files next to your test dll? Their name should start with the name of your dll.

from vstest.

jvu-parsec commented on June 9, 2024

Could you check that there are both runtimeconfig.json and deps.json files next to your test dll? Their name should start with the name of your dll.

Yes, runtimeconfig.json and deps.json are both in the same directory my Tests.dll is in

from vstest.

ksidirop-laerdal commented on June 9, 2024

This seems strange to reference <PackageReference Include="Microsoft.TestPlatform" Version="17.10.0-preview-24080-01" /> but it should not hurt anything, maybe you are using it to get the whole TP into your .nuget cache.

It was just me experimenting with adding a few more nugets just to be sure I'm not missing something.

There is most likely no Foo.Tests.runtimeconfig.json in the output folder. Which means your test project is not built as exe for some reason.

Yes you're right - in Azure I create the file manually:

bin\Release\net8.0\publish\Foo.Tests.runtimeconfig.json

This still doesn't resolve the issue completely however (see the output I get at the bottom)

Can you try adding <OutputType>exe</OutputType> property to your .csproj?

I get the following error:

CSC : error CS5001: Program does not contain a static 'Main' method suitable for an entry point [

I had to use the following in the .csproj:

        <OutputType>exe</OutputType>
        <GenerateProgramFile>false</GenerateProgramFile>

And I added a 'Main' inside Program.cs on the test project. When the tests are run on Azure however I get this:

  No test is available in D:\a\1\s\Foo.Tests/bin/Release/net8.0/publish/Foo.Tests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

While on localdev tests are discovered and run just fine. Hmmm ...

PS: The Foo.Tests.csproj looks like so:

<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <TargetFramework>net8.0</TargetFramework>
        
        <Nullable>enable</Nullable>
        <LangVersion>12.0</LangVersion>
        <ImplicitUsings>enable</ImplicitUsings>

        <IsPackable>false</IsPackable>
        <IsTestProject>true</IsTestProject>

        <!-- https://stackoverflow.com/questions/38085430/the-library-hostpolicy-dll-was-not-found -->
        <!--        <GenerateRuntimeConfigDevFile>true</GenerateRuntimeConfigDevFile>-->

        <OutputType>exe</OutputType>
        <GenerateProgramFile>false</GenerateProgramFile>
    </PropertyGroup>

    <ItemGroup>
        <PackageReference Include="Moq" Version="4.20.70" />
        <PackageReference Include="FluentAssertions" Version="7.0.0-alpha.3" />
        <PackageReference Include="System.Linq.Async" Version="6.0.1" />
        
        <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.10.0-preview-24080-01" />
        <PackageReference Include="Microsoft.TestPlatform" Version="17.10.0-preview-24080-01" />
        <PackageReference Include="Microsoft.TestPlatform.TestHost" Version="17.10.0-preview-24080-01" />

        <PackageReference Include="xunit" Version="2.4.2"/>
        <PackageReference Include="xunit.runner.visualstudio" Version="2.5.6">
            <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
            <PrivateAssets>all</PrivateAssets>
        </PackageReference>
        <PackageReference Include="coverlet.collector" Version="6.0.0">
            <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
            <PrivateAssets>all</PrivateAssets>
        </PackageReference>
    </ItemGroup>

    <ItemGroup>
      <ProjectReference Include="..\Foo.csproj" />
    </ItemGroup>

</Project>

Additional factoids:

The Foo.csproj is a netstandard2.1 project
I'm running the pipeline on Windows-2022 on Azure if that's of any consequence.
I also gave 'dotnet vstest' a shot but no dice (no tests get discovered)
I upgraded xunit to version 2.6.6 still no joy (no tests get discovered)

It turns out that the warning I get about the tests not being discoverable is because the following dlls are not outputted in the output directory when running the pipeline in Azure:

xunit.runner.reporters.netcoreapp10.dll
xunit.runner.utility.netcoreapp10.dll
xunit.runner.visualstudio.testadapter.dll

I have no idea why this happens.

from vstest.

nohwnd commented on June 9, 2024

How do you build this? Do you do just dotnet publish on the project / solution, or something else?

…

________________________________ Od: Kyriakos Sidiropoulos ***@***.***> Odesláno: Wednesday, January 31, 2024 6:59:59 PM Komu: microsoft/vstest ***@***.***> Kopie: Comment ***@***.***>; Subscribed ***@***.***> Předmět: Re: [microsoft/vstest] "The library 'hostpolicy.dll' required to execute the application was not found" - Azure vstest (Issue #4743) This seems strange to reference <PackageReference Include="Microsoft.TestPlatform" Version="17.10.0-preview-24080-01" /> but it should not hurt anything, maybe you are using it to get the whole TP into your .nuget cache. It was just me experimenting with adding a few more nugets just to be sure I'm not missing something. There is most likely no Foo.Tests.runtimeconfig.json in the output folder. Which means your test project is not built as exe for some reason. Yes you're right - in Azure I create the file manually: bin\Release\net8.0\publish\Foo.Tests.runtimeconfig.json This still doesn't resolve the issue completely however (see the output I get at the bottom) Can you try adding <OutputType>exe</OutputType> property to your .csproj? I get the following error: CSC : error CS5001: Program does not contain a static 'Main' method suitable for an entry point [ I had to use the following in the .csproj: <OutputType>exe</OutputType> <GenerateProgramFile>false</GenerateProgramFile> And I added a 'Main' inside Program.cs on the test project. When the tests are run on Azure however I get this: No test is available in D:\a\1\s\Foo.Tests/bin/Release/net8.0/publish/Foo.Tests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again. While on localdev tests are discovered and run just fine. Hmmm ... — Reply to this email directly, view it on GitHub<#4743 (comment)> or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABLYLYIYWVFGVKWP7CNVF3DYRKBCBBFKMF2HI4TJMJ2XIZLTSOBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLLDTOVRGUZLDORPXI6LQMWWES43TOVSUG33NNVSW45FGORXXA2LDOOJIFJDUPFYGLKTSMVYG643JORXXE6NFOZQWY5LFVA3DENZXGQ3TENUCUR2HS4DFUVUXG43VMWSXMYLMOVS2UMJZHA2DCNRRHAZTRJ3UOJUWOZ3FOKTGG4TFMF2GK>. You are receiving this email because you commented on the thread. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

from vstest.

ksidirop-laerdal commented on June 9, 2024

How do you build this? Do you do just dotnet publish on the project / solution, or something else?

I have a .targets msbuild-script that builds both Foo.csproj and Foo.csproj like so:

        <!-- RESTORE NUGETS -->
        <MSBuild Projects="Foo.csproj" Targets="Restore"/>

        <!-- COMPILE AND PACK -->
        <MSBuild Projects="Foo.csproj"  Targets="Build"/>

        <MSBuild Projects="Foo.Tests.csproj" Targets="Restore"/>

        <MSBuild Projects="Foo.Tests.csproj"  Targets="Publish"/>

        <Exec Command="  dotnet  test  &quot;Foo.Tests/bin/Release/net8.0/publish/Foo.Tests.dll&quot;  --verbosity normal  --logger &quot;trx;LogFileName=Foo.Tests.trx&quot;  --results-directory &quot;TestResults&quot;  "
              EchoOff="true"
              ConsoleToMSBuild="true"
              WorkingDirectory="$(RootFolder)"/>

from vstest.

ksidirop-laerdal commented on June 9, 2024

Even if I specify the test-adapter-path explicitly to have it point to the local nuget-folder in the filesystem, it's still unable to discover the tests (I get the same warning):

        <PropertyGroup>
            <Params>$(Params) &quot;Foo.Tests/bin/Release/net8.0/publish/Laerdal.Assets.Installers.Tests.dll&quot;</Params>
            <Params>$(Params) --verbosity:normal</Params>
            <Params>$(Params) --logger:&quot;trx;LogFileName=Foo.Tests.trx&quot;</Params>
            <Params>$(Params) --results-directory:&quot;TestResults&quot;</Params>
            <Params>$(Params) --test-adapter-path:&quot;C:\Users\VssAdministrator\.nuget\packages\xunit.runner.visualstudio\2.5.6\build\net6.0&quot;</Params>

            <!-- also tried -->
            <!-- <Params>$(Params) --test-adapter-path:&quot;/c/Users/VssAdministrator/.nuget/packages/xunit.runner.visualstudio/2.5.6/build/net6.0&quot;</Params> -->
        </PropertyGroup>

        <Exec Command="  dotnet  test  $(Params)  "
              EchoOff="false"
              ConsoleToMSBuild="true"
              WorkingDirectory="$(RootFolder)"/>

I'm absolutely baffled by this

from vstest.

nohwnd commented on June 9, 2024

The original error is coming from the fact that runtimeconfig.json is not present. I would start by generating a binlog from the build locally and on the server and comparing what is going on with runtimeconfig.json. Binary log viewer should make this rather easy, because you will see the targets that are skipped, and why.

All the other errors can be explained also by a "broken" build, so I would start investigating why runtimeconfig.json is not generated first.

I would also try deleting all the bins before every attempt, to make sure that you are not seeing some previous state leaving files in the folder (and making it work).

Your project references all the right nugets, and when I try it locally it builds just fine (I am building via msbuild /t:Publish on the project).

from vstest.

ksidirop-laerdal commented on June 9, 2024

The original error is coming from the fact that runtimeconfig.json is not present. I would start by generating a binlog from the build locally and on the server and comparing what is going on with runtimeconfig.json. Binary log viewer should make this rather easy, because you will see the targets that are skipped, and why.

All the other errors can be explained also by a "broken" build, so I would start investigating why runtimeconfig.json is not generated first.

I would also try deleting all the bins before every attempt, to make sure that you are not seeing some previous state leaving files in the folder (and making it work).

Your project references all the right nugets, and when I try it locally it builds just fine (I am building via msbuild /t:Publish on the project).

I did a deep-clean on localdev (deleted bin and obj folders) and re-run my command line scriptlet: You're right the problem can be reproduced on my localdev in this fashion. If I build through my IDE first and then run my scriptlet I can't reproduce this problem anymore - it seems the IDE is taking care of copying additional dlls into the output folders somehow.

from vstest.

dfederm commented on June 9, 2024

I also ran into this recently and had to add more exclusions to testAssemblyVer2. in particular !**\Microsoft.Testing.Extensions.*.dll

  - task: VSTest@3
    displayName: VSTest
    inputs:
      testSelector: 'testAssemblies'
      testAssemblyVer2: |
        **\*test*.dll
        !**\testhost.dll
        !**\*TestAdapter.dll
        !**\*TestPlatform*.dll
        !**\Microsoft.Testing.Extensions.*.dll
        !**\obj\**

from vstest.

nohwnd commented on June 9, 2024

@DFeder maybe you could benefit from having the filter search just your bin:

**\bin\**\*Tests.dll
**\bin\**\*Test.dll

#4516

from vstest.

dfederm commented on June 9, 2024

Maybe. It's unfortunate that the docs show examples of stuff that just doesn't work in many cases.

https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/vstest-v3?view=azure-pipelines

from vstest.

"The library 'hostpolicy.dll' required to execute the application was not found" - Azure vstest about vstest HOT 15 OPEN

Comments (15)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent