tinkoff / allure.xunit Goto Github PK
View Code? Open in Web Editor NEWAllure.XUnit is a library for displaying xunit .NET Core and .NET 5 tests in Allure reports.
License: Apache License 2.0
Allure.XUnit is a library for displaying xunit .NET Core and .NET 5 tests in Allure reports.
License: Apache License 2.0
If we use xunit's TestOutputHelper to log something, it also appear in allure, but only for success tests. For failed tests exceptions are logged but test output is missing in allure.
So if we have some debug output like "Sample test userId = 123", test start failing and we compare test results, we don't see useful debug info in comparison.
e.g. for imaginary test GetUserNameByUserId() with some logging inside:
Success test sample output :
"Sample test userId = 123"
"Received username is Oleg"
Current failed test output:
"Assert.Equal() Failure (pos 0) Expected: "Oleg" Actual: "MissingName"
Fixed new failed test output:
"Assert.Equal() Failure (pos 0) Expected: "Oleg" Actual: "MissingName"
"Sample test userId = 123"
Steps to reproduce:
[AllureXunitTheory]
[MemberData(nameof(GetData), parameters: "testParam")]
public void InformationApiTests(string scenario, Test test) {}
public static TheoryData GetData(string testParam)
{
//return TheoryData object
}
Expected result
Tests are discovered and can be run
Actual result
Tests are not discovered and running hangs without error. From cmd, the message is: "No test is available in C:\Repos\test\bin\Debug\net5.0.\test.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again."
Hello Team,
I have cloned the Allure.Xunit repository to my local host and opened the project in Visual Studio.
When I navigate to Allure.XUnit.Examples with git Bash and run dotnet test I do not see any allure-results being generated. Any advice ???
Thank you!!!
Hi,
Nice work on creating this library.
I've worked a lot with NUnit and NUnit.Allure and think their implementation is pretty nice to use. Most notably is the use of attributes at the class level.
Is there any technical reason you did not cater for class-level attributes and is this something you are looking to add?
Cheers
Craig
When trying to run a parameterized test using [AllureXunitTheory] attribute along with [MemberData] pointing to a static class member, tests cannot be found and executed.
As static class I use the TheoryData class instead of using IEnumerable for various reasons, but it seems it's not supported yet.
Please note that in my current solution I am using:
Allure version: 2.19.0
Allure.XUnit: 2.1.3
xunit: 2.4.1
xunit.runner.visualstudio: 2.4.5
Microsoft.AspNetCore.Mvc.Testing: 6.0.8
Microsoft.NET.Test.Sdk: 17.3.1
Visual Studio version: 4.8.04084
.NET 6
Current state and problem
When allure_result/*.json are uploaded to Allure TestOps, it performs automatic mapping between test case and result, based on <package>.<class name>.<method name>
. When test launch is closed, automatic test cases are created, and all next results are linked to it.
It's recommended to specify the id of test case to avoid wrong automapping after test code changes, using allureId in terminology, don't confuse with Allure.Commons "testCaseId" property, which is id from external TMS.
More details at https://docs.qameta.io/allure-testops/ecosystem/intellij-plugin/#link-tests-in-ide-to-tests-cases-in-allure-testops-via-allureid
Proposed solution
[AllureAllureId("1")] attribute is created, internally it adds label with name "as_id", same as java version do.
https://javadoc.io/static/io.qameta.allure/allure-java-commons/2.13.9/io/qameta/allure/AllureId.html https://github.com/allure-framework/allure-java/blob/708a1f113842fadfc6d0cb092bb084b319525a49/allure-java-commons/src/main/java/io/qameta/allure/util/ResultsUtils.java#L81
Testing
Before fix: get result*.json, upload to Allure TestOps test launch and finish it, it creates automatic test case. Upload another test run results again, it's automapped to same test case. Try to change test code namespace for example, upload results, then new automaped test case is created, no connection to a previous one.
After fix:
get result*.json, upload to Allure TestOps, it will always be mapped to specified test case id.
Unit tests:
One extra tests is created, in result*.json we have label with name="as_id" and value.
Trying to use your code example from README https://github.com/TinkoffCreditSystems/Allure.XUnit/blob/master/src/Allure.Xunit.StepExtensions.Examples/ExampleSteps.cs
Not possible because AllureStep, AllureBefore, AllureAfter attributes are missing in the package
Exception is thrown
System.InvalidOperationException: The test method expected 1 parameter value, but 0 parameter values were provided.
System.InvalidOperationException
The test method expected 1 parameter value, but 0 parameter values were provided.
Exception doesn't have a stacktrace
Note: It works fine with InlineData
If Custom type was declared and passed throw MemberData Attribute to Test InvalidOperationException is got.
Precondition:
class MyClass
{
}
public static IEnumerable<object[]> TestData = new[]
{
new object[] {new MyClass()}
};
[AllureXunitTheory]
[MemberData(nameof(TestData))]
public async Task GetCallInformation_ReturnsDuration(MyClass input)
{
}
Result:
System.InvalidOperationException: The test method expected 1 parameter values, but 0 parameter values were provided.
Expected Result:
Test Passed
subj.
XUnit theory works correctly.
AllureXUnitTheory does not work.
public class TestClassData : IEnumerable<object[]> { public IEnumerator<object[]> GetEnumerator() { yield return new object[] {DateOnly.MaxValue}; } IEnumerator IEnumerable.GetEnumerator() => GetEnumerator(); }
[AllureXunitTheory] [AllureParentSuite("AllTests")] [AllureSuite("Test AllureXunitTheory")] [AllureSubSuite("Test ClassData")] [ClassData(typeof(TestClassData))] public void TestTheoryWithClassData(DateOnly expected) { Assert.Equal(expected, DateOnly.MaxValue); }
Returns: System.InvalidOperationException: The test method expected 1 parameter value, but 0 parameter values were provided.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.