Failure not being reported up when last keyword passes about agent-python-robotframework HOT 22 CLOSED

@skinny-b it can be fixed on the agent side. But, firstly, I'd like to ask @DzmitryHumianiuk about limitations from the application side. Is that kind of expected behavior when TestSuite takes into account the result of the last nested element only?

from agent-python-robotframework.

If you populate status in FinishRQ RP set it to the item.
If status is not provided, than it calculated based on childs.

• if FAILED>0 then set FAILED

if you finish parent (suite) level before child (test), then child will receive status of parent, and in case of finishRQ will have status for child, it will be updates afterwards.

hope this helps,
or i didn't get the question

from agent-python-robotframework.

The problem is that RP shows up the wrong summary for the test. In Robot Framework test case consists of keywords that are being reported as children to the high-level keywords in a nested manner. On the screenshot below we can notice that test case reports 5 passed steps(children keywords) instead of one failed(high-level keyword). I think it's not correct. If we disable reporting for nested keywords in the agent, the results are reported as expected.

from agent-python-robotframework.

@iivanou in version 5 we have reportportal/reportportal#275 for this.
so you can just add steps into the test object.

we can have a call next week with dev team, to discuss the way it works

from agent-python-robotframework.

@DzmitryHumianiuk oke, let's do this. Getting back to the issue. Our problem is hidden in line 204. We consider the last element of the self.stack being the parent_id for the current item that makes us unable to control nested structures. We can't modify this line without breaking other Python agents.
The only thing we can do here is to make changes in the start_test_item interface of the client in order to provide agents with an ability to control parent_item_id value. Fix for this issue requires changes both in common client and RF agent. I do not see an easy way to work it out.

from agent-python-robotframework.

@skinny-b could you please run the same test suite against the v5.0 of RP? https://beta.demo.reportportal.io/

from agent-python-robotframework.

@iivanou does async capabilities in RPv5 will help to solve it?

now you can create tree structure of elements on client side, by generating item UUID on client side, and send it in HTTP request to server.
So client side knows where item should belong to, and server can accept create requests in any sequence. Even child can be sent before parent.

from agent-python-robotframework.

@DzmitryHumianiuk it should, definitely. But it will require RP to be upgraded to V5.

from agent-python-robotframework.

@iivanou

it should, definitely. But it will require RP to be upgraded to V5.

sure,
i guess we need to keep it as separate branch for v5

from agent-python-robotframework.

I don't believe I can test it on V5 yet, once reportportal/reportportal#715 is resolved we can deploy it to test it further.

from agent-python-robotframework.

@skinny-b It's already deployed for everyone: https://beta.demo.reportportal.io/. You can use your GitHub account to log in.

from agent-python-robotframework.

Saw the same against the beta.demo.reportportal.io server: Testcase data is altered slightly but resulting on failure take a screenshot which ends up passing seems to show the failure.

from agent-python-robotframework.

@skinny-b Unfortunately, we have to change a common client and agents to make use of new async reporting introduce in RP v5. I'll let u know when a list of necessary changes is ready to go.

from agent-python-robotframework.

@skinny-b Can u give a try for the changes in PR #40? Let us know the results. Thanks.

from agent-python-robotframework.

Tried with both #40 and reportportal/client-Python#62 and got the same result as before: #39 (comment)

from agent-python-robotframework.

Ok, in this case, I would consider reportportal/reportportal#275 feature is not working by design.

Status of test item with nested steps:
If all nested steps are passed - Parent item is passed
If one of the nested steps is failed - Parent item is failed

In our case, we have at least one failed children, but RP still reports all passed. @DzmitryHumianiuk, test items with type "STEP" are still being reported with method type "TEST". What type should be set for nested steps? I couldn't find this information at https://beta.demo.reportportal.io/ui/#api.

from agent-python-robotframework.

@Yumfriez Vanya, could you please help? Could you please add a link to the dev guide for Nested steps

from agent-python-robotframework.

@iivanou it should be STEP

Nested step marker is parameter:
"hasStats": FALSE

{
 "attributes": [
 {
 "key": "string",
 "system": true,
 "value": "string"
 }
 ],
 "description": "string",
"hasStats": FALSE,
 "launchId": "string",
 "location": "string",
 "name": "string",
 "parameters": [
 {
 "key": "string",
 "value": "string"
 }
 ],
 "retry": true,
 "startTime": "2019-06-21T14:50:15.600Z",
 "type": "SUITE",
 "uniqueId": "string"
}

from agent-python-robotframework.

Dev guide for Nested steps:
https://github.com/reportportal/client-java/wiki/Nested-steps
https://github.com/reportportal/client-java/wiki/Step-template-and-step-template-config

from agent-python-robotframework.

@iivanou Hi is there any update on this issue. kindly let me know when are we planning to fix this issue or how can I over come this. Thanks for your time

from agent-python-robotframework.

This issue will be fixed in new agents that will support RP version 5 only.

from agent-python-robotframework.

@iivanou Thank you.

from agent-python-robotframework.