redhatqe / pong Goto Github PK

In the --help info, it indicates an example should include the query type such as 'title:' within the query. I agree with this...

-b [BASE_QUERIES [BASE_QUERIES ...]], --base-queries [BASE_QUERIES [BASE_QUERIES ...]]
A sequence of strings that will be used for TestCase
title searches eg 'title:<base_query>')

However when I specify --base-queries=title:rhsm..tests an exception is thrown...

com.polarion.subterra.index.QueryIndexException: Failed to parse query: (title:title:rhsm..tests AND type:testcase AND project.id:RHEL6) AND NOT IS_LOCAL:true'

notice that the code is automatically inserting another 'title:' I think you should not automatically insert another 'title'.

Need a new configuration option to allow setting a requirement query much as --base-queries does for providing a query for TestCases in Polarion.

Please troubleshoot this Pong traceback from jenkins build...
https://rhsm-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/QE-RHEL6.8/job/create-polarion-testrun/254/console

1460088869.86-pong.logger-INFO: Creating TestRecord for rhsm.cli.tests.RepoOverrideTests.AttemptToOverrideBaseurl_Test
1460088872.84-pong.logger-INFO: start time: 2016-04-08 00:14:29.862500
1460088872.84-pong.logger-INFO: end time: 2016-04-08 00:14:32.838732
1460088872.84-pong.logger-INFO: total time: 0:00:02.976232
Traceback (most recent call last):
File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/jenkins/workspace/create-polarion-testrun/pong/exporter.py", line 271, in
Exporter.export(config_map)
File "/home/jenkins/workspace/create-polarion-testrun/pong/exporter.py", line 257, in export
suite.create_test_run(config.testrun_template)
File "pong/decorators.py", line 41, in inner
result = fn(_args, *_kwargs)
File "/home/jenkins/workspace/create-polarion-testrun/pong/exporter.py", line 136, in create_test_run
tc.create_test_record(test_run, run_by=runner)
File "pong/core.py", line 131, in create_test_record
comment = "
".join(comment_string(i, step) for i,step in enumerate(self.step_results))
File "pong/core.py", line 131, in
comment = "
".join(comment_string(i, step) for i,step in enumerate(self.step_results))
File "pong/core.py", line 127, in comment_string
s.exception["stack_trace"] + "
"
KeyError: 'message'

If the project ID changes (ie, if you use --project-id) from what is currently in the .pylarion file, the .pylarion is zeroed out.

It looks like something has changed in Pylarion. Let's figure out how we should handle this...
From https://rhsm-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/QE-RHEL7.3/job/create-polarion-testrun/71/console

1465465643.54-pong.logger-INFO: Creating new Test Run ID: RHSM CLI Tier1 TestSuite Server x86_64 Run 18
1465465643.96-pong.logger-WARNING: Got exception These parameters are required: assignee
1465465643.96-pong.logger-WARNING: Manually adding in the plannedin field
1465465646.74-pong.logger-WARNING: Retrying 2 more times
1465465647.09-pong.logger-WARNING: Got exception These parameters are required: assignee
1465465647.09-pong.logger-WARNING: Manually adding in the plannedin field
1465465649.46-pong.logger-WARNING: Retrying 1 more times
1465465649.88-pong.logger-WARNING: Got exception These parameters are required: assignee
1465465649.88-pong.logger-WARNING: Manually adding in the plannedin field
1465465652.24-pong.logger-WARNING: Retrying 0 more times
Traceback (most recent call last):
File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/jenkins/workspace/create-polarion-testrun/pong/exporter.py", line 330, in
Exporter.export(config_map)
File "/home/jenkins/workspace/create-polarion-testrun/pong/exporter.py", line 316, in export
suite.create_test_run(config.testrun_template)
File "pong/decorators.py", line 41, in inner
result = fn(_args, *_kwargs)
File "/home/jenkins/workspace/create-polarion-testrun/pong/exporter.py", line 184, in create_test_run
raise Exception("Could not create a new TestRun")
Exception: Could not create a new TestRun

Look at these two polarion test runs uploaded from the same jenkins job https://rhsm-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/create-polarion-testrun/76/ ...

RHSM GUI Acceptance TestSuite RHEL-6 8-20160224 n 0 Server x86_64 Run 1 (RHSM RHEL6-8)
RHSM GUI Acceptance TestSuite RHEL-6 8-20160224 n 0 Server x86_64 Run 2 (RHSM RHEL6-8)

https://polarion.engineering.redhat.com/polarion/#/project/RHEL6/testrun?id=RHSM%20GUI%20Acceptance%20TestSuite%20RHEL%2D6%208%2D20160224%20n%200%20Server%20x86%5F64%20Run%201

https://polarion.engineering.redhat.com/polarion/#/project/RHEL6/testrun?id=RHSM%20GUI%20Acceptance%20TestSuite%20RHEL%2D6%208%2D20160224%20n%200%20Server%20x86%5F64%20Run%202

Respectively, they should have been...
RHSM GUI Acceptance TestSuite RHEL-6 8-20160224 n 0 Server x86_64 Run 1 (RHSM RHEL6-8)
RHSM CLI Acceptance TestSuite RHEL-6 8-20160224 n 0 Server x86_64 Run 1 (RHSM RHEL6-8)

Rather than appending a counting " #" to the title of a testrun, I think you should append " Run #".
I think this makes the title of the test run more pleasant to read especially when the testrun-suffix ends in a RHEL COMPOSE_ID. This will prevent the run number from getting interpreted by the user as part of the compose number.
Moreover, the regex that you use to find existing testruns by the same name would be more robust if you are searching for matches to ^(.+)\s+Run (\d+)$

When a query like title:RHSM-TC AND title:rhsm..tests is done, but there already exists TestCases with the title in the form class.method_name (but without the prefix of "RHSM-TC"), then there will be zero matches.

That in turn means pong will assume that no TestCase in polarion exists so it will autogenerate a new one, resulting in 2 TestCases:

• RHSM-TC class.method_name
• class.method_name

A temporary workaround to this problem is to only perform a query of title:rhsm..tests. This will match on existing TestCases. Then pong can check if the title of the existing TestCase startswith the matching --testcase-prefix argument. If it doesn't, pong will change it to include the prefix, otherwise it will leave the TestCase alone.

However, the ideal solution to this problem is to map in the source code what the TestCase work_item_id actually is. This will require:

• Make an Annotation class (or clojure metadata) that maps the method to the ID
• Write a custom IReporter interface in TestNG, and have it insert this into element

Currently, pong is hard coded to use stoner as the user who ran the TestRun. This should be changed to whomever the user actually is (as determined by .pylarion file, environment variable, or CLI option).

See failed job console...
https://rhsm-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/QE-RHEL6.8/job/create-polarion-testrun/113/console

This failure was encountered while... Creating TestRecord for rhsm.cli.tests.TranslationTests.AttemptLocalizedRegistrationWithInvalidCredentials_Test

I believe this failure came from this section of the testng-results.xml file which contains non-ascii parameters... (pasted from https://rhsm-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhsm-rhel-6.8-x86_64-Tier1Tests/54/artifact/test-output/testng-results.xml/*view*/) (Note the xml markup below is causing the guthub issue to be mis-formatted Click the edit message to see the raw comment.)

    <test-method status="FAIL" signature="AttemptLocalizedRegistrationWithInvalidCredentials_Test(java.lang.Object, java.lang.String, java.lang.String, java.lang.String, java.lang.Integer, java.lang.String, java.lang.String)[pri:0, instance:rhsm.cli.tests.TranslationTests@754ba872]" name="AttemptLocalizedRegistrationWithInvalidCredentials_Test" duration-ms="3088" started-at="2016-03-03T00:38:08Z" description="subscription-manager-cli: attempt to register to a Candlepin server using bogus credentials and check for localized strings results" data-provider="getInvalidRegistrationWithLocalizedStringsData" finished-at="2016-03-03T00:38:11Z">
      <params>
        <param index="0">
          <value>
            <![CDATA[ **Blocked by bugzilla bug 615362 1119688]]>
          </value>
        </param>
        <param index="1">
          <value>
            <![CDATA[de_DE.UTF-8]]>
          </value>
        </param>
        <param index="2">
          <value>
            <![CDATA[testuser11699992474]]>
          </value>
        </param>
        <param index="3">
          <value>
            <![CDATA[password793623205]]>
          </value>
        </param>
        <param index="4">
          <value>
            <![CDATA[70]]>
          </value>
        </param>
        <param index="5">
          <value is-null="true"/>
        </param>
        <param index="6">
          <value>
            <![CDATA[Ungültige Berechtigungsnachweise]]>
          </value>
        </param>
      </params>
      <exception class="java.lang.AssertionError">
        <message>
          <![CDATA[Stderr from command 'LANG=de_DE.UTF-8 subscription-manager register --username=testuser11699992474 --password=password793623205' contains matches to regex 'Ungültige Berechtigungsnachweise', expected:<true> but was:<false>]]>
        </message>
        <full-stacktrace>
          <![CDATA[java.lang.AssertionError: Stderr from command 'LANG=de_DE.UTF-8 subscription-manager register --username=testuser11699992474 --password=password793623205' contains matches to regex 'Ungültige Berechtigungsnachweise', expected:<true> but was:<false>
at com.redhat.qe.Assert.fail(Assert.java:186)
at com.redhat.qe.Assert.failNotEquals(Assert.java:606)
at com.redhat.qe.Assert.assertTrue(Assert.java:131)
at com.redhat.qe.Assert.assertContainsMatch(Assert.java:92)
at rhsm.cli.tasks.SubscriptionManagerTasks.runCommandWithLangAndAssert(SubscriptionManagerTasks.java:7741)
at rhsm.cli.tasks.SubscriptionManagerTasks.runCommandWithLangAndAssert(SubscriptionManagerTasks.java:7706)
at rhsm.cli.tests.TranslationTests.AttemptLocalizedRegistrationWithInvalidCredentials_Test(TranslationTests.java:109)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
at org.testng.SuiteRunner.run(SuiteRunner.java:240)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1198)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1123)
at org.testng.TestNG.run(TestNG.java:1031)
at org.testng.TestNG.privateMain(TestNG.java:1338)
at org.testng.TestNG.main(TestNG.java:1307)

]]>




Starting Test: AttemptLocalizedRegistrationWithInvalidCredentials_Test([ *_Blocked by bugzilla bug 615362 1119688, de_DE.UTF-8, testuser11699992474, password793623205, 70, null, Ungültige Berechtigungsnachweise])]]>


This automated test implements Nitrate test http://tcms.engineering.redhat.com/case/41691/]]>


This test was previously blocked by CLOSED Bugzilla bug ''ascii' codec can't decode byte 0xc3 in position 3: ordinal not in range(128)'. (https://bugzilla.redhat.com/show_bug.cgi?id=919584)]]>


This test was previously blocked by RELEASE_PENDING Bugzilla bug 'candlepin server is not returning translated strings for German'. (https://bugzilla.redhat.com/show_bug.cgi?id=615362)]]>


This test was previously blocked by CLOSED Bugzilla bug '[RFE] subscription-manager better usability for scripts'. (https://bugzilla.redhat.com/show_bug.cgi?id=1119688)]]>


Starting Test: AttemptLocalizedRegistrationWithInvalidCredentials_Test([ *_Blocked by bugzilla bug 615362 1119688, de_DE.UTF-8, testuser11699992474, password793623205, 70, null, Ungültige Berechtigungsnachweise])]]>


This automated test implements Nitrate test http://tcms.engineering.redhat.com/case/41691/]]>


This test was previously blocked by CLOSED Bugzilla bug ''ascii' codec can't decode byte 0xc3 in position 3: ordinal not in range(128)'. (https://bugzilla.redhat.com/show_bug.cgi?id=919584)]]>


This test was previously blocked by RELEASE_PENDING Bugzilla bug 'candlepin server is not returning translated strings for German'. (https://bugzilla.redhat.com/show_bug.cgi?id=615362)]]>


This test was previously blocked by CLOSED Bugzilla bug '[RFE] subscription-manager better usability for scripts'. (https://bugzilla.redhat.com/show_bug.cgi?id=1119688)]]>


ssh [email protected] ls -1 -v /etc/pki/entitlement/_.pem | grep -v key.pem]]>


Stdout: ]]>


Stderr: ls: cannot access /etc/pki/entitlement/_.pem: No such file or directory
]]>


ExitCode: 1]]>


ssh [email protected] subscription-manager unregister]]>


Stdout: ]]>


Stderr: This system is currently not registered.
]]>


ExitCode: 1]]>


Asserted: The unregister command was not necessary. Stderr indicates it was already unregistered.]]>


ssh [email protected] test -e /etc/pki/consumer/key.pem]]>


Stdout: ]]>


Stderr: ]]>


ExitCode: 1]]>


Asserted: Consumer key file '/etc/pki/consumer/key.pem' does NOT exist after unregister.]]>


ssh [email protected] test -e /etc/pki/consumer/cert.pem]]>


Stdout: ]]>


Stderr: ]]>


ExitCode: 1]]>


Asserted: Consumer cert file '/etc/pki/consumer/cert.pem does NOT exist after unregister.]]>


ssh [email protected] ls -1 -v /etc/pki/entitlement/_.pem | grep -v key.pem]]>


Stdout: ]]>


Stderr: ls: cannot access /etc/pki/entitlement/_.pem: No such file or directory
]]>


ExitCode: 1]]>


Asserted: All of the entitlement certificates have been removed after unregister.]]>


Attempting to register to a candlepin server using invalid credentials and expecting output in language de_DE.UTF-8]]>


ssh [email protected] LANG=de_DE.UTF-8 subscription-manager register --username=testuser11699992474 --password=password793623205]]>


Stdout: Registriere bei: jsefler-f22-6candlepin.usersys.redhat.com:8443/candlepin
]]>


Stderr: Ung��ltige Berechtigungsnachweise
]]>


ExitCode: 70]]>


Asserted: Actual value of '70' matches expected value: ExitCode from command 'LANG=de_DE.UTF-8 subscription-manager register --username=testuser11699992474 --password=password793623205'.]]>


Test Failed: AttemptLocalizedRegistrationWithInvalidCredentials_Test]]>


Test Failed: AttemptLocalizedRegistrationWithInvalidCredentials_Test]]>

See https://rhsm-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/QE-RHEL6.8/job/create-polarion-testrun/45/console

1455774850.7-pong.logger-INFO: Created test run for RHSM CLI Tier1 TestSuite Server x86_64 1

Expected: RHSM CLI Tier1 TestSuite Server x86_64 Run 1

Make more documentation for installing pylarion, how to install all deps for pong, and what needs to be put on the jenkins slaves.

Right now, when a TestCase or Requirement are autogenerated, it uses for the title the class.method_name. For example:

rhsm.cli.tests.UnregisterTests.UnregisterShouldNotThrowUnauthorizedRequests_Test

Since we now have a requirement to prefix TestCases with RHSM-TC, the title for a TestCase should be:

RHSM-TC rhsm.cli.tests.UnregisterTests.UnregisterShouldNotThrowUnauthorizedRequests_Test

The same needs to be done with Requirements using RHSM-REQ. Since other teams may use pong, these prefixes should be configurable

• Add --testcase-prefix to configuration in CLIConfigurator and YAMLConfigurator
• Add --requirements-prefix as above
• Pass the testcase_prefix arg down to when we actually query for existing TestCases
• As above, but for whenever we need to autogenerate a TestCase
• Pass the requirements_prefix arg down to where we actually query for existing Requirements
• As above, but for whenever we need to autogenerate a Requirement
• Verify that if the TestCase with that prefix doesn't exist, a new TestCase with the prefix is generated
• As above but for Requirements

Here, I am getting the Template TestRun to show that the plannedin field exists
for the Template. However, when I try to create a TestRun based off of it, the
create call fails

>>> TEMP_ID
'RHSM RHEL\\-7 3'
>>> templates = TestRun.search(TEMP_ID, fields=["test_run_id", "created", "status"], sort="created", search_templates=True)
>>> templates
[<pylarion.test_run.TestRun object at 0x7fc3a78c4290>]
>>> template = templates[0]
>>> template = TestRun(uri=template.uri)
>>> template.plannedin
RHEL_7_3
>>> template.test_run_id
RHSM RHEL-7 3
>>> NEW_ID = "RHSM example run"
>>> PROJECT_ID = "RedHatEnterpriseLinux7"
>>> new_tr = TestRun.create(PROJECT_ID, NEW_ID, TEMP_ID)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "<decorator-gen-158>", line 2, in create
  File "/home/stoner/venvs/pong/lib/python2.7/site-packages/pylarion/logstasher.py", line 134, in wrapper
    res = func(*args, **kwargs)
  File "/home/stoner/venvs/pong/lib/python2.7/site-packages/pylarion/test_run.py", line 195, in create
    tr.verify_required(**kwargs)
  File "<decorator-gen-169>", line 2, in verify_required
  File "/home/stoner/venvs/pong/lib/python2.7/site-packages/pylarion/logstasher.py", line 104, in wrapper
    return func(*args, **kwargs)
  File "/home/stoner/venvs/pong/lib/python2.7/site-packages/pylarion/test_run.py", line 1112, in verify_required
    format(fields))
PylarionLibException: These parameters are required: plannedin

However, I can workaround this problem by manually setting the plannedin field like this

>>> plannedin = template.plannedin
>>> plannedin
RHEL_7_3
>>> new_tr = TestRun.create(PROJECT_ID, NEW_ID, TEMP_ID, plannedin=plannedin)
>>> new_tr.test_run_id
RHSM example run
>>> new_tr.plannedin
RHEL_7_3

It would be nice to have a link somewhere so we know what jenkins build this test run was done under

I've noticed that some teams are starting to populate the "Build ID" property on their polarion testruns.
I would like to do the same. I only need Pong to provide a new command line option for...

--testrun-groupId

Then the value that I provide you will by uploaded to the polarion testrun-property:groupId

I think this is a simple string property and should operate the same way as pong --testrun-notes uploads to polarion testrun-property:notes.

The list of variants in incomplete. It should include computenode.

This is really a Pylarion bug. Opening it here because pong found it and because Sean knows where to submit the Jira ticket to get Sim Zacks to fix it....

See the following traceback at the end of the console here: https://rhsm-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/QE-RHEL6.8/job/create-polarion-testrun/168/console

1458759540.81-pong.logger-INFO: Creating new Test Run ID: RHSM CLI Acceptance TestSuite RHEL-6 8-20160322 0 ComputeNode x86_64 Run 1
/home/jenkins/venv/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:791: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
Traceback (most recent call last):
File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/jenkins/workspace/create-polarion-testrun/pong/exporter.py", line 271, in
Exporter.export(config_map)
File "/home/jenkins/workspace/create-polarion-testrun/pong/exporter.py", line 257, in export
suite.create_test_run(config.testrun_template)
File "pong/decorators.py", line 41, in inner
result = fn(_args, *_kwargs)
File "/home/jenkins/workspace/create-polarion-testrun/pong/exporter.py", line 134, in create_test_run
test_run.variant = self.transformer.config.distro.variant.lower()
File "/home/jenkins/venv/lib/python2.7/site-packages/pylarion/base_polarion.py", line 345, in
self._custom_setter(val, field_name)))
File "/home/jenkins/venv/lib/python2.7/site-packages/pylarion/logstasher.py", line 129, in wrapper
res = func(_args, *_kwargs)
File "/home/jenkins/venv/lib/python2.7/site-packages/pylarion/base_polarion.py", line 612, in _custom_setter
csm.get("control"))
File "/home/jenkins/venv/lib/python2.7/site-packages/pylarion/logstasher.py", line 99, in wrapper
return func(_args, *_kwargs)
File "/home/jenkins/venv/lib/python2.7/site-packages/pylarion/base_polarion.py", line 841, in check_valid_field_values
"{1}".format(enum_id, valid_values))
pylarion.exceptions.PylarionLibException: Acceptable values for variant are:[server, workstation, client]

When automatically creating a polarion testcase, there are several testcase properties, such as:
Status
Level
Component
Subcomponent
Test Type
Subtype 1

In my opinion, these values should be read from the automated testcase Java/Clojure meta-data. Maybe we need to create a new Annotation that the pong tool can read.
Maybe we can create a TestNG listener than can read these values.
Is there a way we could stuff this info into the testng results.xml that pong parses?
I'm really not sure how to implement this, but I believe the data should come from the source code.

Right now, trying to map a test method to a Polarion TestCase and Requirement is tricky. This is because titles are not unique, only ID's are. For legacy reasons, we have autogenerated Polarion TestCases and Requirements, but there needs to be a way to decisively and uniquely map a test method from automation source code to its matching TestCase and Requirement.

Proposed Solution:
Java

• Create a new Annotation class @PolarionTestCase that will reference the Polarion TestCase
• Create a new Annotation class @PolarionRequirement that will reference the Polarion Requirement
• Write a custom class that implements IReporter in TestNG to insert in the the matching annotations

Clojure

• Include in the metadata for a test method, the above PolarionTestCase and PolarionRequirement

Ramifications:
This will require manually going through all the existing code and adding these annotations.

Also, pong should no longer autogenerate TestCases or Requirements. Sim's team is working on an exporter so that given an XML file, it will generate TestCases and Requirements.

For some reason, on my laptop, I am unable to download files with download_url() function. This does not seem to be happening on the jenkins slaves, only from my laptop. However, this makes debugging more difficult because I have to download the testng-result.xml file locally manually

Even if only one Requirement is linked to a TestCase, pong is detecting duplicates. It will then remove the linked work item, but also add it back in.

The user won't see any harm, but this is causing an extra load on Polarion

Some characters are not permitted in the testrun ID in Polarion. Since the testrun_prefix + testrun_base + testrun_suffix are used to generate the testrun ID, these arguments should be sanitized

See https://rhsm-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/QE-RHEL6.8/job/create-polarion-testrun/64/console

I have the following test description which contains a character that pylarion is choking on...
@test( description="verify that translation msgstr does NOT contain paragraph character ¶ unless also in msgid",

What do you suggest? Should I just change the description? Or do you have a workaround within pong?

The example given for setting custom fields doesn't work. Need to figure out how to make it work