reportportal / agent-python-robotframework Goto Github PK

View Code? Open in Web Editor NEW

59.0 36.0 34.0 632 KB

Robot Framework integration for Report Portal

License: Apache License 2.0

Python 100.00%

python robot-framework reportportal robot test-automation testing testing-framework testing-library testing-tools

agent-python-robotframework's Introduction

ReportPortal RobotFramework agent

Listener for RobotFramework to report results to ReportPortal

Installation

First you need to install RobotFramework:

pip install robotframework

The latest stable version of library is available on PyPI:

pip install robotframework-reportportal

Usage

Properties

For reporting results to ReportPortal you need to pass some variables to robot run:

REQUIRED:

--listener robotframework_reportportal.listener
--variable RP_API_KEY:"your_user_api_key"
--variable RP_ENDPOINT:"your_reportportal_url"
--variable RP_LAUNCH:"launch_name"
--variable RP_PROJECT:"reportportal_project_name"

NOT REQUIRED:

--variable RP_CLIENT_TYPE:"SYNC"
    - Type of the under-the-hood ReportPortal client implementation. Possible values: [SYNC, ASYNC_THREAD, ASYNC_BATCHED].
--variable RP_LAUNCH_UUID:"id_of_existing_rp_launch"
    - ID of existing ReportPortal launch
--variable RP_LAUNCH_DOC:"some_documentation_for_launch"
    - Description for the launch
--variable RP_LAUNCH_ATTRIBUTES:"RF tag_name:tag_value"
    - Space-separated list of tags/attributes for the launch
--variable RP_LAUNCH_UUID_PRINT:"True"
    - Default value is "False", enables printing Launch UUID on test run start.
--variable RP_LAUNCH_UUID_PRINT_OUTPUT:"stderr"
    - Default value is "stdout", Launch UUID print output. Possible values: [stderr, stdout].
--variable RP_TEST_ATTRIBUTES:"key1:value1 key1:value2 tag key2:value3"
    - Space-separated list of tags/attributes for the tests
--variable RP_CONNECT_TIMEOUT:"20"
    - Default value is "10.0", connection timeout to ReportPortal server.
--variable RP_READ_TIMEOUT:"20"
    - Default value is "10.0", response read timeout for ReportPortal connection.
--variable RP_LOG_BATCH_SIZE:"10"
    - Default value is "20", affects size of async batch log requests
--variable RP_LOG_BATCH_PAYLOAD_SIZE:"10240000"
    - Default value is "65000000", maximum payload size of async batch log
      requests
--variable RP_RERUN:"True"
    - Default is "False". Enables rerun mode for the last launch.
--variable RP_RERUN_OF:"xxxxx-xxxx-xxxx-lauch-uuid"
    - Default is "None". Enables rerun mode for the launch with the specified
      UUID. Should be used in combination with the RP_RERUN option.
--variable RP_SKIPPED_ISSUE:"True"
    - Default value is "True", marks skipped test items with 'To Investigate'
--variable RP_ATTACH_LOG:"True"
    - Default value is "False", attaches Robot Framework HTML log file to
      the launch.
--variable RP_ATTACH_REPORT:"True"
    - Default value is "False", attaches Robot Framework HTML report file to
      the launch.
--variable RP_ATTACH_XUNIT:"True"
    - Default value is "False", attaches Robot Framework XUnit result file to
      the launch.
--variable RP_VERIFY_SSL:"True"
    - Default value is "True", disables SSL verification for HTTP requests.
      Also, you can specify a full path to your certificate as the value.

Logging

Custom logger which supports attachments can be used in Python keywords. Usage of this logger is similar to the standard robot.api.logger with addition of an extra kwarg "attachment":

import subprocess
from robotframework_reportportal import logger


def log_free_memory():
    logger.debug("Collecting free memory statistics!")
    logger.debug(
        "Memory consumption report",
        attachment={
            "name": "free_memory.txt",
            "data": subprocess.check_output("free -h".split()),
            "mime": "text/plain",
        },
    )

Test case ID

It's possible to tag tests the following way test_case_id:12345 using default Robot Framework tagging functionality. ID specified after : will be sent to ReportPortal.

Send attachment (screenshots)

https://github.com/reportportal/client-Python#send-attachement-screenshots

Integration with GA

ReportPortal is now supporting integrations with more than 15 test frameworks simultaneously. In order to define the most popular agents and plan the team workload accordingly, we are using Google analytics.

ReportPortal collects information about agent name and its version only. This information is sent to Google analytics on the launch start. Please help us to make our work effective. If you still want to switch Off Google analytics, please change env variable the way below.

export AGENT_NO_ANALYTICS=1

Copyright Notice

Licensed under the Apache 2.0 license (see the LICENSE.txt file).

agent-python-robotframework's People

Contributors

Stargazers

Watchers

agent-python-robotframework's Issues

Option not to finish launch

RP_LAUNCH_UUID is introduced but the listener finishes launch on end of robot run.
If we have the option to define whether the launch will be finished or not, we can run multiple robot command for the same launch.
(Now we should rerun the launch before running a new robot command)

RP makes each keyword as a separate test in launch

Running single test-case from suite results in 100+ tests, RP counts each keyword as a separate test.

I have a suite (.txt file) with about 10 tests, and run single test via -t arg, like this:
sudo robot -x junit.xml --listener robotframework_reportportal.listener --variablefile=resources/stages/conf_main.py --variablefile=resources/stages/conf_rp.py --variable RP_LAUNCH:QA_German_TestJob #29 -t f321_watch_empty_asset mpx/create_assets/incomplete_packages.txt

where conf_rp.py is the file with RP parameters, mpx/create_assets/incomplete_packages.txt is the suite name.

So, I run it from Jenkins and assume that it will result in a launch on my project and set Total tests to 1 and it will be passed. But here is what I see in RP (launch_log.png), RP assumes each test instruction, each KEYWORD as a test :)

There are some conditional keywords that RP assumed as failed and it resulted in an error:

16:22:39 Traceback (most recent call last):
16:22:39   File "/usr/local/lib/python2.7/dist-packages/reportportal_client/service_async.py", line 204, in process_item
16:22:39     getattr(self.rp_client, method)(**kwargs)
16:22:39   File "/usr/local/lib/python2.7/dist-packages/reportportal_client/service.py", line 140, in finish_launch
16:22:39     return _get_msg(r)
16:22:39   File "/usr/local/lib/python2.7/dist-packages/reportportal_client/service.py", line 22, in _get_msg
16:22:39     return _get_data(response)["msg"]
16:22:39   File "/usr/local/lib/python2.7/dist-packages/reportportal_client/service.py", line 34, in _get_data
16:22:39     raise ResponseError(error_messages[0])
16:22:39 ResponseError: 4065: Cannot finish launch '595d12183cdea200067133e9' with calculated automatically status 'FAILED' as 'PASSED'. Did you provide correct status in request?

Here how it looks in RP:

I've attached Jenkins' output here
job_log.txt

Status is not displayed at prent level and Defect_type is null

Packages:
reportportal-client==5.0.2
robotframework-reportportal==5.0.1
robotframework==3.1.2

Here is the structure of my robot framework tests.

TestSuits
	-- Folder1
		-- suite1.robot
		-- suite2.robot
		-- __init__.robot ( With setup and Teardown)
	-- Folder2
		-- suite3.robot
		-- suite4.robot
		-- __init__.robot ( With setup and teardown)
	-- __init.robot ( With setup and Teardown)

Issues:

[ ERROR ] Calling method start_keyword of listener robotframework_reportportal.listener failed: IndexError: list index out of range
[ ERROR ] Calling method end_keyword of listener robotframework_reportportal.listener failed: IndexError: pop from empty list
BUT, all the levels of the tests and logs are uploaded and launch completes successfully.
I see that STATUS is null for all parent levels And STATUS is displayed only at TestCase level which are IN "TestSuite" files.
As noted in #54 ,
updated the service.py as in 5.0.2 to STEP -> I see that STATUS is displayed at SUITE level , but not above that.

All the time, defect type is null
If all the TESTS are PASSED. Launch is IN_PROGRESS for ever.

And upon 'Force Finish' gives

Failed to force finish the launch: Unclassified error [org.hibernate.exception.SQLGrammarException: could not execute statement]

Referencing Selenium Library is not reflected in script file in Eclipse RED ui IDE

We are trying to use selenium library for automating Web based application using Eclipse- RED ui.
Referencing the selenium library in red.xml is not referenced properly and library reference is not reflected in Script file

Requesting to provide solution to reference the library in Script,keyword directories

Thanks,
S.Thangavel

cli option -L TRACE leads to AttributeError: 'NoneType' object has no attribute 'put_nowait'

Below is command I use to run a test. This one works great. Results go to RP and test launch finished properly.

robot --listener robotframework_reportportal.listener --variable RP_ENDPOINT:"http://localhost:8080" --variable RP_UUID:"MYID" --variable RP_PROJECT:"IDP" --variable RP_LAUNCH:"STAGING SMOKE" --variable SUT:STAGE -e broken --extension robot .

But when I use the -L TRACE cli option bad things happen and test launch needs to be "forced finished" in RP. Same happens with -L trace. (no problem with -L INFO and -L DEBUG, though)

robot --listener robotframework_reportportal.listener --variable RP_ENDPOINT:"http://localhost:8080" --variable RP_UUID:"MYID" --variable RP_PROJECT:"IDP" --variable RP_LAUNCH:"STAGING SMOKE" -L TRACE --variable SUT:STAGE -e broken --extension robot .

Error message

[ ERROR ] Calling method 'start_suite' of listener 'robotframework_reportportal.listener' failed: AttributeError: 'NoneType' object has no attribute 'put_nowait'

some screenshots

My System:
Python 2.7.13
Ubuntu 17.04 VirtualBox VM on Windows 10 host
ReportPortal 3.0

docker images
REPOSITORY                           TAG                 IMAGE ID            CREATED
mongo                                3.4                 88b7188af865        3 weeks ago
redis                                3.2                 83744227b191        4 weeks ago
reportportal/service-ui              3.0.2               985de725c408        6 weeks ago
reportportal/service-rally           3.0.0               84f341e32ceb        2 months ago
reportportal/service-jira            3.0.0               5eb8c387592c        2 months ago
reportportal/service-authorization   3.0.0               5e5191d2f79e        2 months ago
reportportal/service-api             3.0.0               9f02a558f6e4        2 months ago
reportportal/service-registry        3.0.0               6bd796fa7c15        2 months ago
reportportal/service-gateway         3.0.0               18a5435b3845        2 months ago

Keyword names include binary prefixes

Keyword names pushed to the Report Portal should be formatted in Unicode. Currently, keyword names include binary literal prefixes:

Importing listener 'robotframework_reportportal.listener' failed: ImportError: cannot import name FinishExecutionRQ

I am trying to run a simple robot test but get this error

[ ERROR ]
Taking listener 'robotframework_reportportal.listener' into use failed:
Importing listener 'robotframework_reportportal.listener' failed:
ImportError: cannot import name FinishExecutionRQ

Test code in `simple_robot_test.robot` file

*** Settings ***
Documentation    This is a very simple Robot test!

*** Test Cases ***
001 Simple Log Test Case
    Log    Hello Python!
    Log Many          Robot  Framework  Rules!!!

Test execution command

robot --listener robotframework_reportportal.listener \
--variable RP_UUID:"XXXXXX-XXXX-XXXX-XXXX-XXXXXXXX" \
--variable RP_ENDPOINT:"http://0.0.0.0:8080" \
--variable RP_LAUNCH:"Robot Launch" \
--variable RP_PROJECT:"ROBOT_INTEGRATION" \
-d Results simple_robot_test.robot

Complete output

(ROBOT_2.7) wlad@wlad-VirtualBox:~/_GITHUB/reportportal/robotframework_integration$ robot --listener robotframework_reportportal.listener \
> --variable RP_UUID:"61bbbb1d-cad0-47e3-a783-31dfe0ba70b9" \
> --variable RP_ENDPOINT:"http://0.0.0.0:8080" \
> --variable RP_LAUNCH:"Robot Launch" \
> --variable RP_PROJECT:"ROBOT_INTEGRATION" \
> -d Results simple_robot_test.robot
[ ERROR ] Taking listener 'robotframework_reportportal.listener' into use failed: Importing listener 'robotframework_reportportal.listener' failed: ImportError: cannot import name FinishExecutionRQ
Traceback (most recent call last):
  File "/home/wlad/.virtualenvs/ROBOT_2.7/local/lib/python2.7/site-packages/robotframework_reportportal/listener.py", line 5, in <module>
    from .service import RobotService
  File "/home/wlad/.virtualenvs/ROBOT_2.7/local/lib/python2.7/site-packages/robotframework_reportportal/service.py", line 3, in <module>
    from reportportal_client import (ReportPortalService, FinishExecutionRQ,
PYTHONPATH:
  /home/wlad/.virtualenvs/ROBOT_2.7/bin
  /home/wlad/.virtualenvs/ROBOT_2.7/lib/python2.7
  /home/wlad/.virtualenvs/ROBOT_2.7/lib/python2.7/plat-x86_64-linux-gnu
  /home/wlad/.virtualenvs/ROBOT_2.7/lib/python2.7/lib-tk
  /home/wlad/.virtualenvs/ROBOT_2.7/lib/python2.7/lib-old
  /home/wlad/.virtualenvs/ROBOT_2.7/lib/python2.7/lib-dynload
  /usr/lib/python2.7
  /usr/lib/python2.7/plat-x86_64-linux-gnu
  /usr/lib/python2.7/lib-tk
  /home/wlad/.virtualenvs/ROBOT_2.7/local/lib/python2.7/site-packages
  /home/wlad/_GITHUB/catinabox
  /home/wlad/.virtualenvs/ROBOT_2.7/lib/python2.7/site-packages
==============================================================================
Simple Robot Test :: This is a very simple Robot test!                        
==============================================================================
001 Simple Log Test Case                                              | PASS |
------------------------------------------------------------------------------
Simple Robot Test :: This is a very simple Robot test!                | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Output:  /home/wlad/_GITHUB/reportportal/robotframework_integration/Results/output.xml
Log:     /home/wlad/_GITHUB/reportportal/robotframework_integration/Results/log.html
Report:  /home/wlad/_GITHUB/reportportal/robotframework_integration/Results/report.html

Overall stats don't appear after launch completes [v5]

Describe the bug
No summary stats (total, passed, failed etc.) are appearing for my launch

To Reproduce
I am facing this issue while using Robot Framework integration with Report Portal5
Issue occurs when Test teardown / Suite Teardown is used. Tests are updated when Test teardown / Suite Teardown is removed.

Screenshots

RP_LAUNCH_TAGS NOT appearing in key and value format on report portal

RP_LAUNCH_TAGS is NOT appearing in key value format on report portal while running the below pabot commands.

pabot --listener robotframework_reportportal.listener --escape quot:Q --variable RP_ENDPOINT:end_point_url --variable RP_UUID:3xyz--variable RP_LAUNCH:"AjeetDemoDryRun6" --variable RP_PROJECT:ABC_Project --variable RP_LAUNCH_TAGS:"DemoRegression:DemoTestTag" RobotTestsFileFolderName

Actual Result- DemoRegression:DemoTestTag is setting up as attribute values on the RP as shown in the screenshot. The attribute keys is empty
Expected Result - DemoRegression should show as attribute keys and DemoTestTag should show as attribute values

Just for reference old issue- #68

Below is the screenshot from report portal .

Wrong TC status when 'Run Keyword And Expect Error' KW is used

If a test uses Run Keyword And Expect Error keyword it is displayed as FAILED in Launches although it is PASSED

*** Test Cases ***
Expect Error
    Run Keyword And Expect Error    Expected to FAIL
    ...                             Fail    Expected to FAIL

Support for multithreaded execution with Pabot

The Robot Framework agent should have support for reporting results from multithreaded Pabot (https://pabot.org/) execution.

It should be done so, that the end result in RP looks excactly like it would in basic Robot FW execution.

result_visitor always use local timestamp but not timestamp generated by Robotframework

Whenever the timestamp is in report xml file, agent always generated a new timestamp.
This is OK if I use listener interface, but it becomes a problem if I use RobotResultvisitor to parse result xml file.

[help required] Error Parsing '--listener' failed: File or directory to execute does not exist.

Command => robot --variable BROWSER:Chrome "D:\Personal\Knowledge\Robot Framework\WebDemo-master\WebDemo-master\login_tests/valid_login.robot" --listener robotframework_reportportal.listener --variable RP_UUID:"*********" --variable RP_ENDPOINT:"http://ec2-.compute.amazonaws.com:8080/api/v1" --variable RP_LAUNCH:"Robot" --variable RP_PROJECT:"SUPERADMIN_PERSONAL"

Error => [ ERROR ] Parsing '--listener' failed: File or directory to execute does not exist.

Tried various options but its not working

Remove listener.ParentTypes class

Background: comment

Reportportal.io with Robot Framework In Maven

Hello All,

I am able to integrate Robot Framework with Report portal when I running as bat File/in cmd.

Please share do we have the capability to add the report portal to Robot Framework files i.e in .robot file or do we have the maven plugin.

If we do not have maven plugin how to combine the Report portal other plugins with Robot Framework.

[ ERROR ] not enough values to unpack (expected 2, got 1) during starting test execution

I have an issue when test cases are starting. This is my error message

[ ERROR ] not enough values to unpack (expected 2, got 1)

But test case's results are correctly imported to Report Portal io

reportportal-client 5.0.5
robotframework 3.2.1
robotframework-appiumlibrary 1.5.0.6
robotframework-reportportal 5.0.3
robotframework-reportportal-ng 2.0.0
robotframework-seleniumlibrary 4.4.0

I'm using python 3.7. Any idead why this is happening?

Not able pass the robot framework - report.html and log.html to Report Portal..

Calling method 'end_suite' of listener 'reportportal_listener' failed: TypeError: finish_test_item() missing 1 required positional argument: 'item_id'.

Need help guys.. i am trying to fix for some time but i couldn't do so.. If anyone can throw some light here it would be good.

AttributeError: 'ReportPortalService' object has no attribute 'get_system_information'

Hi folks,

I have ERROR when run Robot tests:

[ ERROR ] Calling method 'start_suite' of listener 'robotframework_reportportal.listener' failed: AttributeError: 'ReportPortalService' object has no attribute 'get_system_information'
[ ERROR ] Calling method 'start_keyword' of listener 'robotframework_reportportal.listener' failed: IndexError: list index out of range
[ ERROR ] Calling method 'start_keyword' of listener 'robotframework_reportportal.listener' failed: IndexError: list index out of range
[ ERROR ] Calling method 'log_message' of listener 'robotframework_reportportal.listener' failed: IndexError: list index out of range
[ ERROR ] Calling method 'end_keyword' of listener 'robotframework_reportportal.listener' failed: IndexError: pop from empty list
[ ERROR ] Calling method 'start_keyword' of listener 'robotframework_reportportal.listener' failed: IndexError: list index out of range
[ ERROR ] Calling method 'start_keyword' of listener 'robotframework_reportportal.listener' failed: IndexError: list index out of range
[ ERROR ] Calling method 'start_keyword' of listener 'robotframework_reportportal.listener' failed: IndexError: list index out of range
[ ERROR ] Calling method 'log_message' of listener 'robotframework_reportportal.listener' failed: IndexError: list index out of range
[ ERROR ] Calling method 'end_keyword' of listener 'robotframework_reportportal.listener' failed: IndexError: pop from empty list
...
...

My cmd:
robot --pythonpath . --argumentfile "DEBUG__Pixel_3_API_26_1" "Tests"

and my DEBUG__Pixel_3_API_26_1.txt file with all test variables:

...
--listener xxx/Execution/ResultsListener.py
--listener robotframework_reportportal.listener
--variable RP_UUID:"xxx"
--variable RP_ENDPOINT:"https://reportportal.epam.com"
--variable RP_LAUNCH:"xxx"
--variable RP_PROJECT:"xxx"
--variable screenshot_taking:True
--variable screen_recording:True
...

Python 3.8.6
robotframework-reportportal 5.0.3

Suite Setup is not reported properly

Test setup and tear down are reporting properly in report where as suites level setup and teardowns are not.

how to add new test case execution to existing Launch?

Consider I have test case "a" executed with Launch called "New Launch". Now i have a 2nd test case "b", how to make append the result of b to existing launch. If I give the same name in as variable to pybot, it creates new launch like "New Launch #2"

I can merge the launch in UI but i have very huge number of test cases running so could you please help me on the same to append test executions to existing Launch

Note: First of all, it is a great plugin to robot and its extremely useful to me. Many thanks for the same.

Include multiple test suites in same Launch ID when tests are run in parallel through RobotFramework

Actual:-
Currently, Suppose if my test folder has multiple test suite files, running these test suite files parallel through pabot command will instantiate multiple launch ID's for each of the test suite.

Expected:-
All these three suites IDPortal,UserPortal and HomePortal should show within same Launch ID

Robot's Error and Warning logs are not recognized by listener

I'm getting below warning when I log error or warning with robot framework and add listener to report portal.

Calling method 'log_message' of listener 'robotframework_reportportal.listener' failed: KeyError: 'WARN'

Calling method 'log_message' of listener 'robotframework_reportportal.listener' failed: KeyError: 'ERROR'

eportportal_client.errors.ResponseError: 4065: Cannot finish launch '5e304ff730c49e00015d6341' with calculated automatically status 'FAILED' as 'PASSED'

TC to reproduce issue

*** Test Cases ***
Failing non-critical TC for reportportal
    [Documentation]     Since this test is marked as 'not-ready' and executed
    ...                 with `--noncritical not-ready` option, it should not
    ...                 impact the overall test result.
    [Tags]    not-ready
    Fail    Expected to FAIL as non-critical

results in error below

==============================================================================
Ci Test Dummy                                                                 
==============================================================================
Failing non-critical TC for reportportal :: Since this test is mar... | FAIL |
Expected to FAIL as non-critical
------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service_async.py", line 234, in process_item
    getattr(self.rp_client, method)(**kwargs)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 171, in finish_launch
    status=status)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 167, in _finalize_launch
    return _get_msg(r)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 40, in _get_msg
    return _get_data(response)["msg"]
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 52, in _get_data
    raise ResponseError(error_messages[0])
reportportal_client.errors.ResponseError: 4065: Cannot finish launch '5e3051fc30c49e00015d6418' with calculated automatically status 'FAILED' as 'PASSED'. Did you provide correct status in request?
Ci Test Dummy                                                         | PASS |
0 critical tests, 0 passed, 0 failed
1 test total, 0 passed, 1 failed
==============================================================================

and has wrong status in RP

Interestingly there is no error if you have the following test cases:

*** Test Cases ***
Failing TC for reportportal
                        Fail    Expected to FAIL for reportportal  *tags


Failing non-critical TC for reportportal
    [Documentation]     Since this test is marked as 'not-ready' and executed
    ...                 with `--noncritical not-ready` option, it should not
    ...                 impact the overall test result.
    [Tags]              not-ready
                        Fail    Expected to FAIL as non-critical

==============================================================================
Ci Test Dummy                                                                 
==============================================================================
Failing TC for reportportal                                           | FAIL |
Expected to FAIL for reportportal
------------------------------------------------------------------------------
Failing non-critical TC for reportportal :: Since this test is mar... | FAIL |
Expected to FAIL as non-critical
------------------------------------------------------------------------------
Ci Test Dummy                                                         | FAIL |
1 critical test, 0 passed, 1 failed
2 tests total, 0 passed, 2 failed
==============================================================================

but still wrong status in RP:

Error when finishing the test executing the report portal

Hi,

Below, I describe the error that occurs when I execute an instruction when I run my tests using the Robot Framework

[ ERROR ] Calling method 'end_suite' of listener 'reportportal_listener' failed: TypeError: finish_test_item() missing 1 required positional argument: 'item_id'

[ ERROR ] Calling method 'output_file' of listener 'reportportal_listener' failed: AttributeError: 'ReportPortalService' object has no attribute 'base_url'

Command line:

robot --listener reportportal_listener
--variable RP_ENDPOINT:"http://000.000.0.0:8080"
--variable RP_UUID:"xxxx-xxxx-xxxx-xxxx"
--variable RP_LAUNCH:"superadmin_TEST_EXAMPLE"
--variable RP_PROJECT:"superadmin_personal"
-d ./log/results tests/login.robot

Could you help me with this, please?

The Report Portal runs on a VM.

The tag sent through the execution command is not reflecting in the report portal

We are integrating our robot framework run into the report portal , but the tag we are sending through the execution command is reflecting in the report portal.

Details :
Python version: Python 3.7.3
Robot framework: Robot Framework 3.1.2
Report Portal: 5.2
Command used for execution:
pabot --processes 6 --exclude env2env --exclude excluded -n noncritical -d results/regression/initial --listener robotframework_reportportal.listener --escape quot:Q --variable RP_ENDPOINT:http://reportportalip:8080 --variable RP_UUID: --variable RP_LAUNCH:"AjeetDemoDryRun3" --variable RP_ATTRIBUTES:"Regressiontest:Runbridges" --variable RP_PROJECT:phm try

Note: Used the same command with "robot" instead of "pabot" and still the tag is not displaying in the report portal

Running robot framework tests inside a container is causing ResponseError: 5000: Unclassified error [null]

I am facing an issue where I am unable to publish my test results to report portal and I suspect it could be because of networking. Below is a more detailed view of my problem.

Test Setup

Tests are running inside a ubuntu image on a gitlab-runner container , say hostname gitlab.network
The reportportal server is running on a different host, say report-portal.network
The tests are called in the docker container with the below command:

robot -d reports --pythonpath lib --listener robotframework_reportportal.listener --variable RP_ENDPOINT:"http://report-portal.network:8080" --variable RP_UUID:"XXXXX-XXXX-XXXX-XXXX-XXXXXXXX" --variable RP_LAUNCH:"GitLab Demo" --variable RP_PROJECT:"GITLAB_TEST" tests/

Issue

As each step is executed, I see the below log writing:

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/reportportal_client/service_async.py", line 208, in process_item
    getattr(self.rp_client, method)(**kwargs)
  File "/usr/local/lib/python2.7/dist-packages/reportportal_client/service.py", line 199, in finish_test_item
    return _get_msg(r)
  File "/usr/local/lib/python2.7/dist-packages/reportportal_client/service.py", line 22, in _get_msg
    return _get_data(response)["msg"]
  File "/usr/local/lib/python2.7/dist-packages/reportportal_client/service.py", line 34, in _get_data
    raise ResponseError(error_messages[0])
ResponseError: 5000: Unclassified error [null]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/reportportal_client/service_async.py", line 208, in process_item
    getattr(self.rp_client, method)(**kwargs)
  File "/usr/local/lib/python2.7/dist-packages/reportportal_client/service.py", line 199, in finish_test_item
    return _get_msg(r)
  File "/usr/local/lib/python2.7/dist-packages/reportportal_client/service.py", line 22, in _get_msg
    return _get_data(response)["msg"]
  File "/usr/local/lib/python2.7/dist-packages/reportportal_client/service.py", line 34, in _get_data
    raise ResponseError(error_messages[0])
ResponseError: 40022: Finish test item is not allowed. Test item '5bc5c17442eba400010da6be' has descendants with 'IN_PROGRESS' status. All descendants '[XvfbRobot.Start Virtual Display (1920, 1080)]'
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/reportportal_client/service_async.py", line 208, in process_item
    getattr(self.rp_client, method)(**kwargs)
  File "/usr/local/lib/python2.7/dist-packages/reportportal_client/service.py", line 199, in finish_test_item
    return _get_msg(r)
  File "/usr/local/lib/python2.7/dist-packages/reportportal_client/service.py", line 22, in _get_msg
    return _get_data(response)["msg"]
  File "/usr/local/lib/python2.7/dist-packages/reportportal_client/service.py", line 34, in _get_data
    raise ResponseError(error_messages[0])
ResponseError: 40022: Finish test item is not allowed. Test item '5bc5c17442eba400010da6bd' has descendants with 'IN_PROGRESS' status. All descendants '[login.Open Headless Browser (), XvfbRobot.Start Virtual Display (1920, 1080)]'
T

I am not really sure why this should occur. How can we work together to fix this ? Can I provide you with more information or logs ?

Issues setting up a demo

Hey guys, I'm trying to setup demo integration for a cool open source project: EHRbase. It has a nice and complex test-suite of REST API integration tests written in Robot Framework - thus it's a great candidate to back test your RF integration. I configured the agent to use your demo instance https://beta.demo.reportportal.io/

OS: Ubuntu 19.10 / x86_64 Linux 5.3.0-26-generic
Python 3.7.5
reportportal-client 3.2.3
robotframework-reportportal 3.1.0
robotframework 3.1.2
six 1.14.0

All I did was pip install, logged in with Github to your demo instance and added information from profile to robot command.

Have I missed something?

robot --listener robotframework_reportportal.listener \
      --variable RP_UUID:'871a4189-97c7-4efa-9b8f-xxxxxxxxx' \
      --variable RP_ENDPOINT:'https://beta.demo.reportportal.io' \
      --variable RP_LAUNCH:'DUMMY' \
      --variable RP_PROJECT:'TESTAUTOMATION_PERSONAL' \
      -d results robot/ci_test_dummy.robot

==============================================================================
Ci Test Dummy                                                                 
==============================================================================
Dummy Test For CircleCi Pipeline Testing/Debugging :: CI Pipeline ... ..

	UUID: e92c16bb-5346-4482-aa40-61f0421334aa

Dummy Test For CircleCi Pipeline Testing/Debugging :: CI Pipeline ... | PASS |
------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service_async.py", line 234, in process_item
    getattr(self.rp_client, method)(**kwargs)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 153, in start_launch
    self.launch_id = _get_id(r)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 32, in _get_id
    return _get_data(response)["id"]
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 52, in _get_data
    raise ResponseError(error_messages[0])
reportportal_client.errors.ResponseError: Incorrect Request. [Field 'attributes[].value' should have size from '1' to '128'.] [Field 'attributes[].value' should not contain only white spaces and shouldn't be empty.] 
Traceback (most recent call last):
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service_async.py", line 234, in process_item
    getattr(self.rp_client, method)(**kwargs)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 211, in start_test_item
    item_id = _get_id(r)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 32, in _get_id
    return _get_data(response)["id"]
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 52, in _get_data
    raise ResponseError(error_messages[0])
reportportal_client.errors.ResponseError: Incorrect Request. [Field 'launchUuid' should not be null.] 
Traceback (most recent call last):
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service_async.py", line 234, in process_item
    getattr(self.rp_client, method)(**kwargs)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 211, in start_test_item
    item_id = _get_id(r)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 32, in _get_id
    return _get_data(response)["id"]
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 52, in _get_data
    raise ResponseError(error_messages[0])
reportportal_client.errors.ResponseError: Incorrect Request. [Field 'launchUuid' should not be null.] 
Traceback (most recent call last):
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service_async.py", line 233, in process_item
    self._post_log_batch()
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service_async.py", line 203, in _post_log_batch
    self.rp_client.log_batch(self.log_batch)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 317, in log_batch
    return _get_data(r)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 52, in _get_data
    raise ResponseError(error_messages[0])
reportportal_client.errors.ResponseError: ReportPortalException: Launch 'null' not found. Did you use correct Launch ID?
Traceback (most recent call last):
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service_async.py", line 234, in process_item
    getattr(self.rp_client, method)(**kwargs)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 211, in start_test_item
    item_id = _get_id(r)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 32, in _get_id
    return _get_data(response)["id"]
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 52, in _get_data
    raise ResponseError(error_messages[0])
reportportal_client.errors.ResponseError: Incorrect Request. [Field 'launchUuid' should not be null.] 
Traceback (most recent call last):
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service_async.py", line 233, in process_item
    self._post_log_batch()
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service_async.py", line 203, in _post_log_batch
    self.rp_client.log_batch(self.log_batch)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 317, in log_batch
    return _get_data(r)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 52, in _get_data
    raise ResponseError(error_messages[0])
reportportal_client.errors.ResponseError: ReportPortalException: Launch 'null' not found. Did you use correct Launch ID?
Traceback (most recent call last):
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service_async.py", line 234, in process_item
    getattr(self.rp_client, method)(**kwargs)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 211, in start_test_item
    item_id = _get_id(r)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 32, in _get_id
    return _get_data(response)["id"]
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 52, in _get_data
    raise ResponseError(error_messages[0])
reportportal_client.errors.ResponseError: Incorrect Request. [Field 'launchUuid' should not be null.] 
Traceback (most recent call last):
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service_async.py", line 234, in process_item
    getattr(self.rp_client, method)(**kwargs)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 231, in finish_test_item
    return _get_msg(r)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 40, in _get_msg
    return _get_data(response)["msg"]
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 52, in _get_data
    raise ResponseError(error_messages[0])
reportportal_client.errors.ResponseError: Test Item 'None' not found. Did you use correct Test Item ID?
Traceback (most recent call last):
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service_async.py", line 234, in process_item
    getattr(self.rp_client, method)(**kwargs)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 227, in finish_test_item
    item_id = self.stack.pop()
IndexError: pop from empty list
Traceback (most recent call last):
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service_async.py", line 234, in process_item
    getattr(self.rp_client, method)(**kwargs)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 171, in finish_launch
    status=status)
  File "/home/osboxes/projects/ehrbase/tests/venv/lib/python3.7/site-packages/reportportal_client/service.py", line 165, in _finalize_launch
    self.stack.pop()
IndexError: pop from empty list
Ci Test Dummy                                                         | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Output:  /home/osboxes/projects/my/ehrbase/tests/results/output.xml
Log:     /home/osboxes/projects/my/ehrbase/tests/results/log.html
Report:  /home/osboxes/projects/my/ehrbase/tests/results/report.html

[v5] When click PASSED no test are shown.

@iivanou that's architectual desicion fail, made 7 years ago.
We named objects corresponding to Test Case as steps, assuming the option to involve BDD.

Now we have nested steps.

And change of STEP type to TEST will affect all variety of our agents, clients and UI as well.
We plan this big step for RP 6.

Originally posted by @DzmitryHumianiuk in reportportal/agent-python-pytest#182 (comment)

Tutorial for beginners

I would like to try ReportPortal with Robot-Framework and Python unittests/pytest.

In order to better understand things I would like to make a tutorial about that. But I am kind of lost and don't know where to start :-/. So please give me some guidance and I will publish a tutorial in a Github repo or Gist.

So far I have a Ubuntu 16.04 with Docker up and running as a VirtubalBox VM on a Windows host machine. Inside that VM I have ReportPortal running by following the installation steps from your README and your docs. I can access ReportPortal UI inside VM at http://localhost:8080. Further I have cloned Robotframework repository which has a great set of over 1500 Python unittests and over 4000 acceptance test in Robot-Framework format. I want to use this tests as example for the integration with ReportPortal.

What I have in short:

Ubuntu VM with ReportPortal started with docker-compose up
set of Python unittests
set of RobotFramework tests

What are the next steps I have to work on to see the results of above mentioned tests after execution inside ReportPortal?

Reportportal.io with Robot Framework In Maven/Jenkins Pipeline

Hi,

I am able to integrate Robot Framework with Report portal when I running as bat File/in cmd.

Please share do we have the capability to add the report portal to Robot Framework files i.e in .robot file or do we have the maven plugin.

If we do not have maven plugin how to combine the Report portal other plugins with Robot Framework.

Thanks,
Phartheeb Kandasamy

Listner not working

Moment I start the test, it keeps printing this message in loop. Not sure what is causing the issue or what I am doing wrong.

Version Info:
reportportal-client 5.0.3
robotframework 2.8.4
robotframework-reportportal 5.0.2
Report Portal Version: 5.2.3

[ ERROR ] Calling listener method 'start_suite' of listener 'robotframework_reportportal.listener' failed: KeyError: 'id'
[ ERROR ] Calling listener method 'start_keyword' of listener 'robotframework_reportportal.listener' failed: IndexError: list index out of range
[ ERROR ] Calling listener method 'log_message' of listener 'robotframework_reportportal.listener' failed: IndexError: list index out of range
[ ERROR ] Calling listener method 'start_keyword' of listener 'robotframework_reportportal.listener' failed: IndexError: list index out of range
[ ERROR ] Calling listener method 'log_message' of listener 'robotframework_reportportal.listener' failed: IndexError: list index out of range
[ ERROR ] Calling listener method 'log_message' of listener 'robotframework_reportportal.listener' failed: IndexError: list index out of range
[ ERROR ] Calling listener method 'log_message' of listener 'robotframework_reportportal.listener' failed: IndexError: list index out of range
[ ERROR ] Calling listener method 'end_keyword' of listener 'robotframework_reportportal.listener' failed: KeyError: 'libname'
[ ERROR ] Calling listener method 'start_keyword' of listener 'robotframework_reportportal.listener' failed: IndexError: list index out of range
[ ERROR ] Calling listener method 'log_message' of listener 'robotframework_reportportal.listener' failed: IndexError: list index out of range
[ ERROR ] Calling listener method 'log_message' of listener 'robotframework_reportportal.listener' failed: IndexError: list index out of range

Error - When finished test using Robot Framework

Hi!
For some reason, my task was closed. But I reopened with the intention of reevaluating the problem based on the beginning of the resolution #75

Below, I describe the error that occurs when I execute an instruction when I run my tests using the Robot Framework

[ ERROR ] Calling method 'end_suite' of listener 'reportportal_listener' failed: TypeError: finish_test_item() missing 1 required positional argument: 'item_id'

[ ERROR ] Calling method 'output_file' of listener 'reportportal_listener' failed: AttributeError: 'ReportPortalService' object has no attribute 'base_url'

Command line:

Could you help me with this, please?

The Report Portal runs on a VM.

reportportal-client --------> 5.0.5
robotframework-reportportal --------> 5.0.3
robotframework-reportportal-ng --------> 2.0.0
report-portal --------> 2.4 (version description in a docker file)

the report is the version docker, I downloaded and installed the project via docker compose. I performed the installation suggested on site.

All services started:
reportportal/service-api:5.3.1
reportportal/service-auto-analyzer:5.3.1
reportportal/service-authorization:5.3.1
reportportal/service-index:5.0.10
rabbitmq:3.7.16-management
postgres:12-alpine
minio/minio:RELEASE.2020-05-01T22-19-14Z
reportportal/service-ui:5.3.1
docker.elastic.co/elasticsearch/elasticsearch:7.3.0
traefik:v2.0.5

TKS

Not documented dependency - ModuleNotFoundError: No module named 'six'

pip install robotframework-reportportal automatically installs reportportal-client but six is missing. Might be mentioned in docs or installed automatically, too?

[ ERROR ] Taking listener 'robotframework_reportportal.listener' into use failed: Importing listener 'robotframework_reportportal.listener' failed: ModuleNotFoundError: No module named 'six'

Executing tests with pabot results in multiple launch entries

I've encountered the following problem:
When I execute the tests using pabot then there is a launch entry for each test case instead of just one.

How can I resolve this issue?

Question: What maps to SKIPPED status

What is the criteria that makes RP see robot tests as SKIPPED?

Listener does not work for Initialization files (init.robot)

When using an initialization file (__init__.robot) for global setups and tear-downs all the keywords/actions performed inside these global setup and teardowns result on error, see below:

[ ERROR ] Calling method 'start_keyword' of listener 'robotframework_reportportal.listener' failed: IndexError: list index out of range

These happens as there is no suite attached as a parent due to init.robot being an special "test suite" not recognized as a test suite.

These can be fixed by modifying the listener.py file more specifically the start suite method (See below the method code and the fix between commented lines)

def start_suite(name, attributes):
    print("start suite:" + name)
    suite = Suite(attributes=attributes)
    print("robot id: " + suite.robot_id + " - name: "+ name)
    if suite.robot_id == "s1":
        Variables.check_variables()
        RobotService.init_service(Variables.endpoint, Variables.project,
                                  Variables.uuid)
        suite.doc = Variables.launch_doc
        logging.debug("ReportPortal - Start Launch: {0}".format(attributes))
        RobotService.start_launch(launch_name=Variables.launch_name,
                                  launch=suite)
        #-------------------------------------------------------------------
        #Used to display top level folder to contain global setups and teardowns
        logging.debug("ReportPortal - Create global Suite: {0}".format(attributes))
        parent_id = items[-1][0] if items else None
        item_id = RobotService.start_suite(name=name, suite=suite,
                                           parent_item_id=parent_id)
        items.append((item_id, parent_id))
       #-------------------------------------------------------------------------------
        
        if not suite.suites:
            attributes['id'] = "s1-s1"
            start_suite(name, attributes)

    else:
        logging.debug("ReportPortal - Start Suite: {0}".format(attributes))
        parent_id = items[-1][0] if items else None
        item_id = RobotService.start_suite(name=name, suite=suite,
                                           parent_item_id=parent_id)
        items.append((item_id, parent_id))

Can the latest pull request also released?

Too long item test name causes RP server errors

RP server limits length of name field of a test to be 256 chars max.
But this agent may produce name value much longer than 256 chars. This leads to server error:

ResponseError: 4001: Incorrect Request. {} [[Field 'name' should have size from '1' to '256'.] ]

This is how generated test name looks like:
${options} = BuiltIn.Catenate (--variablefile ${VF1}, -V ${VF2}:arg, -V "${VF2}:arg2:value;with;semi;colons", -V "${VF2};semicolon;separator", -V "${VF2};semi:colon;separator:with:colons", --VariableFile ${VF2}, -V ${VF2}:FAIL, -V ${VF2}:too:many:args, --variablef ${VF3} --VARIABLEFILE ${VF4} --VariableFile pythonpath_varfile.py --VariableFile pythonpath_varfile.py:1:2:3 --pythonpath ${VARFILEDIR}/pythonpath_dir

How to reproduce:

clone robotframework source
execute robotframework's acceptance tests as follows:
./atest/run.py python --listener robotframework_reportportal.listener --variable RP_UUID:"uuid" --variable RP_ENDPOINT:"https://endpoint" --variable RP_LAUNCH:”ROBOT” --variable RP_PROJECT:"you_name_personal" atest/robot/variables/

Unicode is not defined error

I am running into error while trying to use the the plugin

[ ERROR ] Taking listener 'robotframework_reportportal.listener' into use failed: Importing listener 'robotframework_reportportal.listener' failed: NameError: name 'unicode' is not defined Traceback (most recent call last):

Here is my test command
robot --listener robotframework_reportportal.listener --variable RP_UUID:"52bed5ea-68a0-4f7b-bbd3-cb1d7fbae91a" --variable RP_ENDPOINT:"http://localhost:8080" --variable RP_LAUNCH:"km_TEST_EXAMPLE" --variable RP_PROJECT:"robot" -d results/ tests/pg.robot

Any help appreciated. Thanks,

How to send screenshot to report portal- help needed

Hi All @iivanou

I am very new to reportportal. I have installed agent for robotframework. I would like to send the failure cases screenshots to report portal i went through the code from here https://github.com/reportportal/client-Python#send-attachment-screenshots

but I am not able to follow it on where to add this code snippet. Kindly help me with the same.

RP_LAUNCH_TAGS names are Not reflecting on the report portal

RP_LAUNCH_TAGS names are Not reflecting on the report portal while running the below pabot commands.
Commands are running fine without any warning or issue except the given tag in the command is not reflecting on the report portal.
Could you please look at this issue and help me out ?

pabot --listener robotframework_reportportal.listener --escape quot:Q --variable RP_ENDPOINT:end_point_url --variable RP_UUID:3xyz--variable RP_LAUNCH:"AjeetDemoDryRun4" --variable RP_PROJECT:ABC_Project --variable RP_LAUNCH_TAGS:"TestTag" RobotTestsFolderName

pabot --listener robotframework_reportportal.listener --escape quot:Q --variable RP_ENDPOINT:end_point_url --variable RP_UUID:3xyz--variable RP_LAUNCH:"AjeetDemoDryRun4" --variable RP_PROJECT:ABC_Project --variable RP_LAUNCH_TAGS:"SmokeTest:TestTag" RobotTestsFolderName

Below is the screenshot from report portal and also my expected and result results view.

Listener has json issue on windows OS

When I ran robot framework with report portal listener on Windows, I got error which is 'No JSON object could decoded'. It worked on Mac OS.

[OS]
Windows 7 x86

[Ver]
Python 2.7.13
RF 3.0.2
report portal listener 2.5.4

[Test Data]
tp2.txt
*** Test Cases ***
test1
test keyword abcd

*** Keywords ***
test keyword
[Arguments] ${msg}
log ${msg}

[Repro]

run tp2.txt

pybot -v RP_UUID:"xxx" -v RP_ENDPOINT:"xxx" -v RP_LAUNCH:"xxx" -v RP_PROJECT:"xxx" --listener robotframework_reportportal.listener tp2.txt

Result

RP_REPORT_LEVEL is gone?

RP_REPORT_LEVEL is gone in gent-Python-RobotFramework 2.5.4?

Now default level might be Keyword. It cause long test running time.

Tests don't finish due to request to Report Portal even though the tests is done.

Before I solved this problem by test of RP_REPORT_LEVEL.

So what can I solve this problem without setting the level?

ascii error

I am running a robot framework from RIDE, trying to set up a report portal reporting to robot framework. My command line is:

--listener robotframework_reportportal.listener --variable RP_UUID:9a10eub9-e451-470d-a3f4-2409d5c45cbc --variable RP_ENDPOINT:http://192.168.99.100:8080 --variable RP_LAUNCH:pybot_TEST_EXAMPLE --variable RP_PROJECT:tbd_robot
And I am getting an error in ride console:

[ ERROR ] Calling method 'start_keyword' of listener 'robotframework_reportportal.listener' failed: UnicodeEncodeError: 'ascii' codec can't encode character u'\u0101' in position 29: ordinal not in range(128)

What could be wrong here?
After this error appears, all further activity and calling all methods goes with it, suite remains unfinished in ReportPortal.

Error "No message available"

Hello,

I have a problem when I run a test with the listener

=> robot --listener robotframework_reportportal.listener --variable RP_UUID:"XXXXXXXXXXXXXX" --variable RP_ENDPOINT:"http://localhost:8080/api/v1" --variable RP_LAUNCH:"superadmin_TEST_EXAMPLE" --variable RP_PROJECT:"Test" test.robot

File "/usr/local/lib/python2.7/site-packages/reportportal_client/service_async.py", line 208, in process_item
getattr(self.rp_client, method)(**kwargs)
  
File "/usr/local/lib/python2.7/site-packages/reportportal_client/service.py", line 129, in start_launch
    self.launch_id = _get_id(r)
  
File "/usr/local/lib/python2.7/site-packages/reportportal_client/service.py", line 14, in _get_id
    return _get_data(response)["id"]
  
File "/usr/local/lib/python2.7/site-packages/reportportal_client/service.py", line 37, in _get_data
    raise ResponseError(error_messages[0])
ResponseError: No message available

I do not understand where the mistake can be

Thank you

What is ReportPortal?

Could you please clarify what ReportPortal is?

Cheers
Tset

Report portal don't work correctly with library robotframework 3.1 and higher

Describe the bug
Hi,
I'm using ReportPoratl but I've write simple robotframework tests as example for this issue I have.

*** Settings ***
Library  SeleniumLibrary

Test Teardown  Close browser

*** Test Cases ***
Open google and fill search pass
    Open browser  http://www.seznam.cz  chrome
    Page should contain element  //*[@class="szn-input-with-suggest-list search-form__input input input--hp-search"]



Open google and fill search fail
    Open browser  http://www.seznam.cz  chrome
    Page should contain element  //*[@class="szn-input-with-suggest-list search-form__input input input--hp-search xcxccxxxc"]

First test is passing and second failing but in ReportPortal are both passed.

https://user-images.githubusercontent.com/26042586/62728078-263e2100-ba1b-11e9-9fcc-71ececcfd30e.png

If I open test that should fail I see this:
https://user-images.githubusercontent.com/26042586/62728168-600f2780-ba1b-11e9-8677-d503f9422192.png

But when I open failed step I see this:

https://user-images.githubusercontent.com/26042586/62728247-96e53d80-ba1b-11e9-879b-ef8fa0dc772c.png
Witch is good because it took screenshot but test case have to be failed.

> robot --listener robotframework_reportportal.listener --variable RP_UUID:0fe489c7-7598-4uo7-9uaf-460049jkd54f --variable RP_ENDPOINT:http://localhost:8080 --variable RP_LAUNCH:RF_Test_Run --variable RP_PROJECT:smart testReportPortal.robot 
> ==============================================================================
> testReportPortal                                                              
> ==============================================================================
> Open google and fill search pass                                      | PASS |
> ------------------------------------------------------------------------------
> Open google and fill search fail                                      | FAIL |
> Page should have contained element '//*[@class="szn-input-with-suggest-list search-form__input input input--hp-search xcxccxxxc"]' but did not.
> ------------------------------------------------------------------------------
> testReportPortal                                                      | FAIL |
> 2 critical tests, 1 passed, 1 failed
> 2 tests total, 1 passed, 1 failed
> ==============================================================================
>

Expected behavior
Test cases will be failed. ReportPortal will correctly represents results of RF. Issue is in robotframework-seleniumlibrary library 3.1 and higher.

For 3.0.0 it works good but we need to use higher version.

Screenshots

Versions:
I use libraries.

robotframework 3.1.2
robotframework-appiumlibrary 1.5.0.4
robotframework-databaselibrary 1.2.4
robotframework-reportportal 3.0.0
robotframework-reportportal-ng 1.0.4.9
robotframework-seleniumlibrary 3.3.1

API robot test execution is very slow [v5]

Describe the bug
API robot test execution is very slow

To Reproduce
Execute attached test case file ("RequestLib.txt") with report portal V5.

Actual Result
The test file RequestLib.txt took 1 min while executing in local without RP parameters.
Same test case took more than 7 mins while executing with RP parameters.

Please find the attached reports from above runs.

Library used in the attached test case file : RequestsLibrary - https://marketsquare.github.io/robotframework-requests/

RequestLib.txt
API_Run_WithoutRP.zip
API_Run_WithRP.zip

Failure not being reported up when last keyword passes

Was running tests that have an intermittent failure and wasn't seeing the failure being reported to the top level. It shows reduced testcase count, as test didn't complete, but no failures. With the help of @iivanou (thank you) were able to determine that the ending keyword pass seems to be the culprit.

Using the following:
(venv) bash-4.2$ pip list | grep report
pytest-reportportal 1.0.8
reportportal-client 3.2.3
robotframework-reportportal 3.1.0

Last image shows the red ones as failed and the last Capture as pass.

[Feature Request] Uploading the report.html and log.html of a test run to reportportal

As part of my test result, I would want to view the report.html created by the robot framework as these reports are very detailed. This may be provided in the report portal as a an attachment to the test result. I would like to know if there is a way that it can be done right now. For example, can we provide an option like --attachreport true to the run command ?