Describe the bug
The URL for the User Manual appears to be incorrect in the README.md in the root of the repo, or file permissions may be preventing other users from viewing it.

To Reproduce
Clicking the "user manual" link or attempting to visit the URL (https://tcunit.org/tcunit-runner-user-manual/) manually results in a GitHub pages 404 File Not Found

Expected behavior
Expected behavior is that this link would redirect to external docs or perhaps to the docs README here, which appears to have the info I was looking for.

My assumption is this is just an outdated link from a webpage migration.

Screenshots

Software versions
n/a

Run environment
n/a

Additional context
n/a

TwinCAT3 Version: 3.1.4024.10
Visual Studio Development Tools Environment (DTE) version: TcXaeShell.DTE.15.0
TcUnit Version: 1.1.0.0
TcUnit-Runner Version: 0.9.3.0

When building the project in Visual Studio 2013, the XML file looks right. This is what it looks like:

<?xml version="1.0" encoding="utf-8"?>
<testsuites failures="0" tests="4">
  <testsuite id="0" name="PRG_MAIN.fb_Subtract_Test" tests="2" failures="0">
    <testcase name="FiveMinusThreeEqualsTwo" assertions="1" classname="PRG_MAIN.fb_Subtract_Test" status="PASS" />
    <testcase name="TwelveMinusFiveEqualsSeven" assertions="1" classname="PRG_MAIN.fb_Subtract_Test" status="PASS" />
  </testsuite>
  <testsuite id="1" name="PRG_MAIN.fb_Sum" tests="2" failures="0">
    <testcase name="TwoPlusTwoEqualsFour" assertions="1" classname="PRG_MAIN.fb_Sum" status="PASS" />
    <testcase name="ZeroPlusZeroEqualsZero" assertions="1" classname="PRG_MAIN.fb_Sum" status="PASS" />
  </testsuite>
</testsuites>

However, when it tries building in TCXaeShell, the XML file looks like this:

<?xml version="1.0" encoding="utf-8"?>
<testsuites failures="0" tests="4" />

When it tries to record that file in Jenkins, My pipeline outputs this:

[Pipeline] junit
Recording test results
None of the test reports contained any result

When I build and run the project in TCXaeShell itself (away from Jenkins and TcUnitRunner), I am able to see the test cases and details in the output and Error List.

I was using AdvancedExampleProject and the latest released TcUnitRunner (0.9.3.0).

I changed one test to fail and noticed that the xUnit result for the test suite had still 0 failures.

<testsuite id="4" name="PRG_TEST.fbDiagnosticMessageTimeStampParser_Test" tests="2" failures="0">
<testcase name="TestWithTimestampZeroTimeExpectCurrentTime" assertions="0" classname="PRG_TEST.fbDiagnosticMessageTimeStampParser_Test" status="PASS" />
<testcase name="TestWithValidTimestampExpectSameTimestamp" assertions="0" classname="PRG_TEST.fbDiagnosticMessageTimeStampParser_Test" status="FAIL">
<failure message="Test 'TimeStamp zero time' failed" type="BOOL" />
</testcase>
</testsuite>

Describe the bug
TcUnit-Runner documentation website has been broken.

To Reproduce
Go to TcUnit-Runner, and click on "user manual" hyperlinked text in the README.md, which is the same as clicking link https://tcunit.org/tcunit-runner-user-manual/.

Screenshots

Has anyone experienced issues with the LaunchTcUnit.bat script timing out during execution? I execute the script on an Azure VM running Windows 10, through Azure Pipelines, and my console output occasionally end up looking like this.

The VM is running 4 cores with 7 GB RAM, so I don't think this is a performance related issue, allthough I feel that some actions, like closing the Remote Desktop connection to the VM, or restarting the VM, seem to mitigate the issue.

One issue that might be a symptom of some underlying issue, is that the TcXaeShell application doesn't seem to be properly shut down after the script executes (and ends abruptedly?).

I'm attaching my azure-pipelines.yml file as well, as I'm unsure what is causing this behavior. Keep in mind that this occurs only some times, so it is kind of hard to reproduce and stabilize the error.

# Demo CI workflow with TcUnit-Runner
name: TcUnit-Runner  
 
trigger:
- main
pr:
- main
 
pool: 
    name: AC4Build   
    demands:
    - agent.name -equals TcUnitTests # self-hosted agent  

 
steps: 
 
- checkout: self
  displayName: Checkout repository  
 
- task: CmdLine@2
  displayName: Run PLC unit-tests with TcUnit-Runner 
  condition: succeeded()
  inputs:
    failOnStderr: true
    # Start TcUnit-Runner with a timeout of 3 min
    script: '"C:\Program Files (x86)\TcUnit-Runner\LaunchTcUnit.bat" -U 3 -t PlcTask -w 3.1.4024.20'
     
- task: PublishTestResults@2
  displayName: Publish TcUnit results
  condition: or(succeeded(),failed())     
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: '**/TcUnit_xUnit_results.xml'

Any help would be appreciated.

Hello Jakob Sagatowski,
Thank you very much for sharing and developing with Beckhoff twincat Test driven development (TDD),
I don't know if it would be possible instead of using Jenkins that it was with Azure Devops ?.
I am interested in this development if there was a TcUnit-Runner-Azure DevOps,
I don't know if you have any interest in doing it?
Thank you so much for everything.

Description - New feature

Expected behaviour

Jenkins Pipeline works with the right solution (Test- solution)

Informations

The TwinCat project has 2 solutions in 1 project. We use the TCUnit-Runner to check our code with a Jenkins Pipeline.

When we call the TcUnit-Runner we want to check the functionality of our code with our test solution. Right now TcUnit-Runner is trying to work with the productive code solution but we want to work with our test solution.

Is there already a way to order the Jenkins job to check a self chosen Solution (define the path to the solution in Jenkins) or is it only possible to find the correct solution by enhancing TCUnitRunner?

Software- Version

TCUnitRunner Version 0.9.3.0

Hi, i am trying to use the TcUnit-Runner to automate unit tests with jenkins. During the job execution I am getting the following error:

C:\Users\jenkins\AppData\Local\Jenkins.jenkins\workspace\LCS>"C:\Program Files (x86)\TcUnit-Runner\LaunchTcUnit.bat"
Task name of the TcUnit task not provided. Assuming only one task in TwinCAT solution
AmsNetId to run TwinCAT/TcUnit is not provided. Assuming TwinCAT/TcUnit will run locally '127.0.0.1.1.1'
A TwinCAT version is not provided. Assuming latest TwinCAT version should be used
Timeout not provided.
VISUAL_STUDIO_SOLUTION_PATH found!
The filepath to the visual studio solution file is: "path/to/file"
2021-02-23 10:50:23 - TcUnit-Runner build: 0.9.3.0
2021-02-23 10:50:23 - TcUnit-Runner build date: 22/12/2020
2021-02-23 10:50:23 - Visual Studio solution path: path/to/file
2021-02-23 10:50:23 -
2021-02-23 10:50:23 - In TwinCAT project file, found TwinCAT version 3.1.4024.11
2021-02-23 10:50:23 - In Visual Studio solution file, found visual studio version 16.0
2021-02-23 10:50:23 - Version is pinned: False
2021-02-23 10:50:23 - Trying to load the Visual Studio Development Tools Environment (DTE) version 'TcXaeShell.DTE.16.0' ...
2021-02-23 10:50:23 - ...FAILED!
2021-02-23 10:50:23 - Trying to load the Visual Studio Development Tools Environment (DTE) version 'VisualStudio.DTE.16.0' ...
2021-02-23 10:50:26 - ...SUCCESSFUL!
2021-02-23 10:50:33 - ERROR: Error loading VS DTE. Is the correct version of Visual Studio and TwinCAT installed? Is the TcUnit-Runner running with administrator privileges?
2021-02-23 10:50:33 - Exiting application...
Exit code is 11

I have followed the user manual steps and I have duble checked that both twinCAT and Visual Studio versions are correct, i have also added administrator privileges to the tcUnit-runner application via: properties -> compatibility -> run as administrator. But i keep getting the same output.

However, if I run the tcunit-runner from the command line, i get the following output:

2021-02-23 11:19:30 - TcUnit-Runner build: 0.9.3.0
2021-02-23 11:19:30 - TcUnit-Runner build date: 22/12/2020
2021-02-23 11:19:30 - Visual Studio solution path: C:\Projects\M2M3\M2\LCS\LCS_TestJenkins.sln
2021-02-23 11:19:30 -
2021-02-23 11:19:30 - In TwinCAT project file, found TwinCAT version 3.1.4024.11
2021-02-23 11:19:30 - In Visual Studio solution file, found visual studio version 16.0
2021-02-23 11:19:30 - Version is pinned: False
2021-02-23 11:19:30 - Trying to load the Visual Studio Development Tools Environment (DTE) version 'TcXaeShell.DTE.16.0' ...
2021-02-23 11:19:30 - ...FAILED!
2021-02-23 11:19:30 - Trying to load the Visual Studio Development Tools Environment (DTE) version 'VisualStudio.DTE.16.0' ...
2021-02-23 11:19:39 - ...SUCCESSFUL!
2021-02-23 11:20:04 - Using the TwinCAT remote manager to load TwinCAT version '3.1.4024.11'...
2021-02-23 11:20:38 - No task name provided. Assuming only one task exists
2021-02-23 11:20:38 - Found task with name 'PlcTask_Test'
2021-02-23 11:20:55 - Setting target NetId to '127.0.0.1.1.1'
2021-02-23 11:20:55 - Enabling boot project and setting BootProjectAutostart on 127.0.0.1.1.1
2021-02-23 11:21:23 - Waiting for results from TcUnit...
2021-02-23 11:21:34 - ... got 1 report lines so far.
2021-02-23 11:22:30 - ... got 105 report lines so far.
2021-02-23 11:22:40 - ... got 105 report lines so far.
2021-02-23 11:22:51 - ... got 105 report lines so far.
2021-02-23 11:23:02 - ... got 105 report lines so far.
2021-02-23 11:23:12 - ... got 105 report lines so far.
2021-02-23 11:23:22 - ... got 105 report lines so far.
2021-02-23 11:23:33 - ... got 105 report lines so far.
2021-02-23 11:23:43 - ... got 105 report lines so far.

Any idea on what am i doing wrong? Thanks for the help.

Hi all,

I want to run TcUnit-Runner on a Azure-hosted VM. Following the example under Azure Pipelines on the TcUnit-Runner User Manual I registered the VM as a self-hosted agent and installed all prerequisites on it, including TwinCAT 3, Git and TcUnit/TcUnit-Runner. This is my azure-pipelines.yml file

# Demo CI workflow with TcUnit-Runner
name: TcUnit-Runner  
 
trigger:
- simplified-decoupled-dependency
pr:
- simplified-decoupled-dependency
 
pool: 'AC4Build'    # self-hosted agent  
 
steps: 
 
- checkout: self
  displayName: Checkout repository  
 
- task: CmdLine@2
  displayName: Run PLC unit-tests with TcUnit-Runner 
  condition: succeeded()
  inputs:
    failOnStderr: true
    # Start TcUnit-Runner with a timeout of 5 min
    script: '"C:\Program Files (x86)\TcUnit-Runner\LaunchTcUnit.bat" -U 5'
     
- task: PublishTestResults@2
  displayName: Publish TcUnit results
  condition: or(succeeded(),failed())     
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: '**/TcUnit_xUnit_results.xml'

Now, testing my setup by changing my TC project on my developer machine, and pushing it to the Azure repo, I can see the pipeline job starting in Azure DevOps, but the job fails with the following console output

Starting: Run PLC unit-tests with TcUnit-Runner
==============================================================================
Task         : Command line
Description  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
Version      : 2.182.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
==============================================================================
Generating script.
Script contents:
"C:\Program Files (x86)\TcUnit-Runner\LaunchTcUnit.bat" -U 5
========================== Starting Command Output ===========================
"C:\windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "C:\agent\_work\_temp\f6f23cbc-b040-41b0-a738-8f209b523d44.cmd""
##[error]Systemet finner ikke angitt bane.
##[error]Cmd.exe exited with code '1'.
Finishing: Run PLC unit-tests with TcUnit-Runner

Sorry for the Norwegian language, but the first error basically translates to "The system cannot find the given path".

Now I am stuck, partly because I am confused as to what the script is trying to do at this point. What file doesn't it find? Any help would be much appreciated.

Add an (optional) parameter to select variant (available since 4024.0).
If a variant isn't provided, TwinCAT uses the latest selected variant by default (stored outside the project?).

Make sure to implement the following rules:

• if anything < 4024.0 is loaded AND the option is provided, the user gets an error message
• if a variant is provided, and that variant is not available in the configuration, the user gets an error message

A TwinCAT project can be either a full-blown executable (that you can do "activate configuration" with) or a library. Usually unit tests are added to the library, but sometimes it might be benefical to add the tests directly to the main program and adding a separate task and separate program for the tests and run that instead. This is today possible with the TcUnit-Runner. What might cause a problem however are the different I/Os (such as an EtherCAT master) which won't work if the hardware is not there.

This can be solved by TcUnit-Runner automatically disabling all I/Os prior to executing the unit tests. This can be achieved through the TwinCAT automation interface.

Since Oracle has deprecated the JavaWS (Java Web Start), the way to install the TwinCAT-Agent pass by the cli.
In the attached file, you will find how to install the TwinCAT-Agent.
I hope that will help somweone, fill free to adapt to your need.
Installing the Jenkins Agent by the command line.docx
.

Currently , the TcUnit-Runner process is exiting with exitcode 0 - RETURN_SUCCESSFULL, unless a error occurred.
The process should return an exiting code, depending on the test-results, i.e.

• all tests ok: RETURN_SUCCESSFULL
• at least on test failed: RETURN_TESTS_FAILED
• no test result at all (possible bug): RETURN_ERROR_NO_TEST_RESULTS

This would mimic the same behaviour as other test-suites offer, i.e. VsTest, i.e. in GitHub Actions.

Hi all,

At my workplace, we want to keep our production code as clean as possible. This means we don't want the TcUnit library reference to be compiled down into our libraries (the specific reason for this is that it seems that some hardware (small PLCs such as the CX7000 etc.) don't have the sufficient memory to run our biggest libraries if TcUnit is loaded into it as well).

Therefore, I tried to separate out our unit tests into a separate test project in the same solution file as our library code. In this project i reference TcUnit and our library, and run the test on the referenced blocks. Now, I want to run this on a remote VM using TcRunner, but when I do this (through Azure Pipelines), I run into the obvious problem that the library I try to run tests on is not available. How can I can I compile and save my (updated) library on the VM, before referencing it with my test project? Is this even posssible, or am I chasing rainbows here?

Describe the bug
When trying to run TcUnit-Runner via Jenkins, TcUnit-Runner hangs on vsInstance.BuildSolution. This bug appears to be fixed if vsInstance.CleanSolution is removed. There is another vsInstance.CleanSolution that causes hanging on automationInterface.StartRestartTwinCAT.

To Reproduce

1. Clone ExampleProjects
2. Have Visual Studio 2019 installed
3. Point Jenkins pipeline at ExampleProjects
4. Create Jenkins file that calls bat '"C:\\Program Files (x86)\\TcUnit-Runner\\LaunchTcUnit.bat" -t PlcTask' from within ExampleProjects/SimpleExampleProject folder.
5. Have Jenkins build off of the JenkinsFile
6. Wait for Jenkins build to hang

Expected behavior
It is expected that the Visual Studio Instance is able to successfully build and start/restart TwinCAT without hanging

Screenshots
Successfully running via PowerShell (completed in 90 seconds):

Getting hung in Jenkins (screenshot taken after running for about 13 minutes):

Software versions
TcUnit-Runner - 0.9.3
TcUnit 1.2.0.0
Visual Studio version 2019 (16.8.30717.126)
Which version of the applicable software did you use? (TcUnit-Runner, TcUnit library, TwinCAT XAE version, Visual Studio version etc)

Run environment
Bare Metal

Additional context
I'm able to run the LaunchTcUnit.bat file from PowerShell without any trouble. Everything works as expected but when trying to do so via Jenkins, it hangs every time. This points to there being a problem with Jenkins or Visual Studio but wanted to share in case anyone else has run into similar problems.
I haven't tried this setup with XAE Shell but that is my next step, to see how this behaves in different versions of Visual Studio

There might be a need to save the test result in a specific directory relative to the Visual Studio solution being run. This is useful for gathering test results neatly together in the same place. One particular use case I encountered was when using this tool together with Bitbucket pipeline, which requires test results to be inside directories named according to a set of supported names. See more info about that at https://support.atlassian.com/bitbucket-cloud/docs/test-reporting-in-pipelines/.

I could add a script within the pipeline to move the test results manually afterwards to my desired path, but having this as a feature in the TcUnit-Runner is a much cleaner solution.

Describe the bug
A clear and concise description of what the bug is.
Double lines in ErrorList causes TcUnitResultCollector to fail

To Reproduce
What steps are necessary to reproduce the behavior?
When you have enough tests that you need to loop though the while (true) loop in program.cs multiple times, the line
var newErrors = errorList.AddNew(errorItems);
gets called twice or more. this has the effect of errorList getting double messages.
I discovered this by printing the ErrorList to a file, and seeing double lines.
The output was fine in VS TwinCAT output console.

Expected behavior
A clear and concise description of what you expected to happen.
Errorlist shouldn't have double messages.

Screenshots
If applicable, add screenshots to help explain your problem.

Software versions
Which version of the applicable software did you use? (TcUnit-Runner, TcUnit library, TwinCAT XAE version, Visual Studio version etc)
TcUnit-Runner built from GitHub, TcUnit library latest release, VS 2017, TwinCAT v3.1.2024.12

Run environment
Under what environment did you run the software? PLC? Virtual machine?
Windows, localhost 'PLC'

Additional context
Have you tried running the software without TcUnit-Runner and making sure you get the results?
Add any other context about the problem here.
When I run the same test suite on the same machine without TcUnit-Runner, I get the correct output in the console.

After Jenkins job is executed, is that possible to add Expected and Actual values in Error Details which shows only Error Message.

Writing a Unitest that doesn't have an explicit message for an assert, results in a parsing error.
I know, the message is useful, but can you default to a message if none is given explicitly? I don't think it should be necessary to give a detailed message. It'd prefer a simple default like "assertion failed, 5!=4" (if message is empty).

AssertEquals(5, 4, '');

results in

ERROR: While parsing TcUnit results, expected TEST_ASSERT_MESSAGE but got TEST_ASSERT_TYPE

when used together with TcUnit-Runner, no xml file is generated although other tests use explicit messages.

I am using
TcUnit 1.1.0.0
TcUnit-Runner 0.9.1.0

The installed TwinCat versions are listed in the interface ITcRemoteManager remoteManager.Versions.
In the current sorting the last version is at position remoteManager.Versions[0]

In order not to rely on the sorting the last version should be determined from the interface remoteManager.Versions

maybe with the Versions class?

TwinCAT projects created in Visual Studio versions newer than 2013 don't have default ErrorList order setting, which means that the ErrorList entries are not ordered in the expected format for TcUnit-runner to parse (TcUnitResultCollector ln 122-375).

TcUnit-Runner does not support more complex solution structure
When a solution has more than one child item under it, the function FindTwinCATProjectFile may fail to find a TwinCAT project file if a non-TwinCAT-project item is listed at the top in the solution file.

To Reproduce
Consider the following solution structure,

├── Example.sln
│   ├── Solution Items
│   │   ├── Example.libcat.xml
│   ├── Example (XAE Solution)

Given this, the content of the Example.sln could be, for example,

Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio 15
VisualStudioVersion = 15.0.33403.129
MinimumVisualStudioVersion = 10.0.40219.1
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution Items", "{986D592B-6B07-4D9C-9910-4D3A5BBDD0A3}"
	ProjectSection(SolutionItems) = preProject
		Example.libcat.xml = Example.libcat.xml
	EndProjectSection
EndProject
Project("{B1E792BE-AA5F-4E3C-8C82-674BF9C0715B}") = "Examle", "Example\Example.tsproj", "{D6956552-7530-4208-832D-7CACFFFF6BE4}"
EndProject

In this scenario, FindTwinCATProjectFile will stop parsing at the first Project entry and mistakenly assume that it is an entry of a TwinCAT project. It then returns an empty string and TcUnit-Runner error out with this message,

Did not find TwinCAT project file in solution. Is this a TwinCAT project?

Expected behaviour
The program should parse the solution file differently to get the right TwinCAT project. It should iterate through all Project entries in the solution file and return the first TwinCAT project entry it finds, or an empty string if it doesn't find any at all. Within the solution file, the EndProject, used after each Project, should help with parsing by limiting the scope of each Project entry.

Software versions
TcUnit-Runner v0.9.3.0.

Run environment
Not applicable.

Additional context
Not applicable.

Sometimes - shit happens ;-) - somebody commits code to a project that causes a runtime exception in TwinCAT, most commonly this occurs with div0 or pointers to NULL. TcUnit-Runner should check the TwinCAT state and if an Exception occurs it should end. Another option would be to timeout after X minutes, where X is provided as a command line argument.

System.Timers.Timer timeout = new System.Timers.Timer(options.timeout);
timeout.Elapsed += KillProcess;
timeout.AutoReset = false; //One-shot-timer
timeout.Start();

static private void KillProcess(Object source, System.Timers.ElapsedEventArgs e)
{
    Console.WriteLine($"timeout occured ... killing processes");
    Environment.Exit(-1);
}

what do you think about this?

Would it be possible to include RunStaticAnalysis in TcUnit-Runner arguments/options when there are no BuildError

The use case as mentioned in TcUnit/issues/93.

1. The USB license stick in the server. Test server architecture is as mentioned Example_2_-_Separate_Jenkins_master_Jenkins_agent
2. Developers pushes the code to server, static analysis will be triggered by Jenkins in the Server.

I tested this in tcunit/ExampleProjects by adding below code and it worked well.

If this included in TcUnit-Runner , it will be a All in one Framework :)

Describe the bug
Turning on Static Analysis (light edition), gives build errors.

To Reproduce
Turn on Static Analysis (light edition).
Save project.
Call TcUnit-Runner on the project

Expected behavior
Execute test cases as defined in the Test_PLC task

Screenshots
Jenkins output:

00:00:03.035 The filepath to the visual studio solution file is: "C:\Jenkins_home\workspace<REDACTED><REDACTED>.sln"
00:00:03.177 2022-08-16 08:59:07 - TcUnit-Runner build: 0.9.3.0
00:00:03.179 2022-08-16 08:59:07 - TcUnit-Runner build date: 2020-12-22
00:00:03.179 2022-08-16 08:59:07 - Visual Studio solution path: C:\Jenkins_home\workspace<REDACTED><REDACTED>.sln
00:00:03.179 2022-08-16 08:59:07 -
00:00:03.181 2022-08-16 08:59:07 - In TwinCAT project file, found TwinCAT version 3.1.4024.32
00:00:03.182 2022-08-16 08:59:07 - In Visual Studio solution file, found visual studio version 15.0
00:00:03.190 2022-08-16 08:59:07 - Version is pinned: True
00:00:03.195 2022-08-16 08:59:07 - Trying to load the Visual Studio Development Tools Environment (DTE) version 'TcXaeShell.DTE.15.0' ...
00:00:06.032 2022-08-16 08:59:10 - ...SUCCESSFUL!
00:00:07.477 2022-08-16 08:59:11 - The pinned TwinCAT version is available
00:00:12.930 2022-08-16 08:59:17 - Using the TwinCAT remote manager to load TwinCAT version '3.1.4024.32'...
00:00:26.200 2022-08-16 08:59:30 - Setting task 'Test_Task' enable and autostart, and all other tasks (if existing) to disable and non-autostart
00:00:56.551 2022-08-16 09:00:00 - Description: SA0027: Variable name 'Param' in 'FB__Test' already used for an object in Library 'tc2_system, 3.4.25.0 (beckhoff automation gmbh)'
00:00:56.551 2022-08-16 09:00:00 - ErrorLevel: vsBuildErrorLevelHigh

00:00:57.248 2022-08-16 09:00:01 - Filename: C:\Jenkins_home\workspace<REDACTED>\S_aStruct.TcDUT
00:00:57.255 2022-08-16 09:00:01 - ERROR: Build errors in project

Software versions

Latest of TcUnit-Runner and TcUnit library.
TwinCAT 3.1.4024.32

Run environment
Jenkins, windows 10, real machine

Additional context
TcUnit works fine, to do the same tests

** Possible solution **
Tried to create a pull request but I could not figure it out.

Change line 290 in program.cs:

else if (item.ErrorLevel == vsBuildErrorLevel.vsBuildErrorLevelHigh)
else if (item.ErrorLevel == vsBuildErrorLevel.vsBuildErrorLevelHigh && !item.Description.ToString().StartsWith("SA"))

Hi,
I have the same issue described by another user and I have also read your blog about it (https://alltwincat.com/2018/10/04/ci-cd-with-twincat-part-four/), but the problem still remain. Instead of Git as a version control system, I used SubVersion if it can help you to understand the problem. Using the Shell it works fine, reporting the results in a XML file. Probably the versions of Visual Studio and TwinCAT installed are not ok?

Thanks for the help.

Clean up the code and replace TC with TwinCAT in the log output.

The buildSolution Method caused me some troubles using TcXaeShell, since it will block the code.
Sadly I was not able to solve the problem.

Used a workaround by calling SolutionBuild.Build(false) and waiting the buildingstate to become "done".
Worked like a charm, but there might be more sexier ways..

VisualStudioInstance.cs:

public void BuildSolution()
{
	visualStudioSolution.SolutionBuild.Build(false);
	System.Threading.SpinWait.SpinUntil(() => visualStudioSolution.SolutionBuild.BuildState== EnvDTE.vsBuildState.vsBuildStateDone);
}

Today, TcUnit-Runner tries to load the TwinCAT-version that the project is created in such as:
log.Info("Using the TwinCAT remote manager to load TwinCAT version '" + this.tcVersion + "'...");
ITcRemoteManager remoteManager = dte.GetObject("TcRemoteManager");
remoteManager.Version = this.tcVersion;

where tcVersion is for example 3.1.4022.32.

Now, for example loading 3.1.4022.32 on a machine that only has 3.1.4022.30 does not return any error. Should it be this way, or should the program abort? Or should it be a configuration parameter to select whether the user:

• Wants to override the project version of TwinCAT
• Or simply ignore the TwinCAT project version

One thing that has to be considered if is the user has pinned/locked the version of TwinCAT in the project file itself.

Running TcUnit-Runner on a machine that has 4022.30 but where the project is 4022.32 results in:
2020-09-05 14:59:59 - In TwinCAT project file, found TwinCAT version 3.1.4022.32
2020-09-05 14:59:59 - In Visual Studio solution file, found visual studio version 12.0
2020-09-05 14:59:59 - ... none existing Visual Studio process found. Creating new instance of visual studio DTE.
2020-09-05 14:59:59 - Trying to load the Visual Studio Development Tools Environment (DTE) version 'VisualStudio.DTE.12.0' ...
2020-09-05 15:00:02 - ...SUCCESSFUL!
2020-09-05 15:00:02 - Using the TwinCAT remote manager to load TwinCAT version '3.1.4022.32'...
2020-09-05 15:00:19 - No task name provided. Assuming only one task exists
2020-09-05 15:00:19 - Found task with name 'PlcTask'
2020-09-05 15:00:37 - Setting target NetId to '5.44.93.176.1.1'
2020-09-05 15:00:38 - Enabling boot project and setting BootProjectAutostart on 5.44.93.176.1.1
2020-09-05 15:01:12 - Waiting for results from TcUnit...

In the current implementation of the TcUnit-Runner, a new instance of visual studio is created when running the TwinCAT project. If the visual studio process is not killed correctly for some reason, this can lead to many zombie-instances of visual studio being running in the system. What should be done instead is to try to attach to an existing instance of visual studio (if one with the same version as the project was created with is existing).

Build errors
After executing the runner from Jenkins (using a master/slave configuration) I get several build errors such as those showed below.
It happens the same when I execute the batch file from the command prompt.

Nevertheless, when I build the project from the TwinCAT XAE it works without errors.
It also works when I execute the runner from the command prompt but using the original program (instead of the one stored in the Jenkins workspace folder).

Sample of the errors (there are many more similar to these):

2023-04-20 15:40:41 - Description: Cannot convert type 'POINTER TO STRING(255)' to type 'UDINT' [tc2_utilities, 3.3.54.0 (beckhoff automation gmbh)]
2023-04-20 15:40:41 - ErrorLevel: vsBuildErrorLevelHigh
2023-04-20 15:40:41 - Filename: DWORD_TO_BINSTR
2023-04-20 15:40:41 - Description: Cannot convert type 'POINTER TO ARRAY [0..32] OF BYTE' to type 'UDINT' [tc2_utilities, 3.3.54.0 (beckhoff automation gmbh)]
2023-04-20 15:40:41 - ErrorLevel: vsBuildErrorLevelHigh
2023-04-20 15:40:41 - Filename: DWORD_TO_BINSTR
2023-04-20 15:40:41 - Description: Cannot convert type 'POINTER TO STRING(255)' to type 'UDINT' [tc2_utilities, 3.3.54.0 (beckhoff automation gmbh)]
2023-04-20 15:40:41 - ErrorLevel: vsBuildErrorLevelHigh
2023-04-20 15:40:41 - Filename: DWORD_TO_BINSTR
2023-04-20 15:40:41 - Description: Cannot convert type 'POINTER TO STRING(255)' to type 'UDINT' [tc2_utilities, 3.3.54.0 (beckhoff automation gmbh)]
2023-04-20 15:40:41 - ErrorLevel: vsBuildErrorLevelHigh
2023-04-20 15:40:41 - Filename: DWORD_TO_DECSTR

Software versions
I am using TwinCAT 3.1.4024.22 with the XAE (not visual studio).

Run environment
The software is running in a Virtual Machine with Windows 10.

Describe the bug
Under certain circumstances a zombie instance of the TcXaeShell (or whatever IDE is being used) can be left. The problem is here:

LoadSolution(@filePath);
LoadProject();
loaded = true;

If the solution is correctly loaded but the project not, we will have one instance of the xaeshell running dead as its not properly cleaned up:

/// <summary>
/// Closes the DTE and makes sure the VS process is completely shutdown
 /// </summary>
        public void Close()
        {
            if (loaded) {
               ...
                dte.Quit();
            }
            loaded = false;
        }

Suggestion is to simply move loaded:

LoadSolution(@filePath);
loaded = true;
LoadProject();

I would also rename it solutionLoaded

Software versions
TcUnit-Runner 0.9.3

Run environment
TcXaeShell 3.1.4024.15