New test files require rebuild to be picked up by VS Test explorer about specflow.netcore HOT 15 CLOSED

As a workaround, you can use MSBuild CreateItem task.

Example:

  <Target Name="PrecompileScript" BeforeTargets="BeforeBuild">
    <Exec Command="dotnet SpecFlow.NetCore" />

    <CreateItem Include="$(ProjectDir)Features\*.feature.cs">
      <Output ItemName="Compile" TaskParameter="Include" />
    </CreateItem>
  </Target>

from specflow.netcore.

Thanks for the report, I'll have a look when I get some time.

from specflow.netcore.

I'm confident this is a tooling issue (problems between dotnetcore and the VS Test Runner). If you speak with @bradwilson he can probably confirm.

from specflow.netcore.

From the runner perspective, we never deal with source, only DLLs. It's definitely sounds like .NET CLI tooling is to blame.

My guess is that the build engine had already decided what files to compile before it runs your precompile step. This is why it works the second time: the file is already on disk before your precompile step runs.

from specflow.netcore.

I'm pretty sure I can repro this with xunit alone (let me know if you want repro steps). I don't for a second think this is xunit's fault, I'm very aware the test runner interaction can be interesting at times. More wondered if this is something you've already seen reported, @bradwilson? Perhaps spoken with some VS guys about it or would know who to point us at?

from specflow.netcore.

.NET CLI bugs should be reported here: https://github.com/dotnet/cli

What do you mean you can repro it with just xUnit.net alone?

from specflow.netcore.

I take that back. I remember in the past that I used to have to build before running tests in .Net Core (I have it in some old youtube videos I'm sure, but that was possibly in RC2 or even RC1). I see the behaviour has been fixed since. It used to be that everytime I added a new test (or renamed an existing one) I would have to build then run, despite the fact that running the tests does a build. That no longer seems to be the case.

It was never a great hardship so I never thought to investigate further.

from specflow.netcore.

Thanks for the input guys. It looks like this issue: https://github.com/dotnet/cli/issues/3807

And just to confirm it again, you have to also run dotnet test twice from the command line for tests to be picked up (so not just VS Test Explorer) indicating it is a .NET CLI issue.

from specflow.netcore.

Yup, I was about to comment based on testing for another issue. Any file created during precompile is ignored; I'd guess that the file lists for compilation are generated before the precompile step currently. Works fine after first build/creation of each designer file - worthy of a readme mention I'm sure.

from specflow.netcore.

I wonder... would it be worthwhile throwing an exception or returning a non-zero exit code if files are created by the specflow process so that the user can be informed that not all tests are going to be run until next build? It's yet another ugly-interim-hack, but it might be a useful one?

from specflow.netcore.

Yeah, definitely worth noting in the README. I'm also open to the idea of emitting a warning of some sort.

from specflow.netcore.

Accidentally closed. Reopening, even though the fix is with .NET CLI.

from specflow.netcore.

Note to self: the upstream issue has been closed; need to re-test this.

from specflow.netcore.

As I read it, the upstream issue was only fixed for MSBuild-based .NET Core projects. So I think the warning in the readme should stay for now 😓

from specflow.netcore.

#85 (comment)

from specflow.netcore.