MOVED: https://github.com/ARMmbed/mbed-os-tools (Flash, reset and run host supervised tests on mbed platforms)
The development of Htrun has been moved into the mbed-os-tools repository. The mbed-host-tests package is still available on the Python Package Index (PyPI).
If you have any questions or encounter issues, please raise an issue in the mbed-os-tools repository.
The root cause of this problem is that python fails to return a correct fully qualified domain name for a mac on a windows LAN.
For example, a mac with a hostname of: bremoran-mac will return:
>>> socket.getfqdn()
'bremoran-mac'
getaddrinfo() does not know how to parse this, which causes a gai error:
socket.gaierror: [Errno 8] nodename nor servname provided, or not known
This name does resolve correctly on our DNS, which points to an error in the implementation of gethostbyname which seems not to support aliased DNS entries.
There are few obvious solutions to this problem.
An interim solution is to use the host's internet-facing port to collect an IP address when getfqdn fails:
ipaddr = [(s.connect(('8.8.8.8', 80)), s.getsockname()[0], s.close()) for s in [socket.socket(socket.AF_INET, socket.SOCK_DGRAM)]][0][1]This will not deal with machines that use multiple network interfaces, however and neither does the original solution.
I believe that the correct solution is to move the point of address resolution to a later stage in the flow. When the mbed has reported its IP address, mbed-host-tests should run the code above with a slight modification:
ipaddr = [(s.connect((mbedIPaddr, 80)), s.getsockname()[0], s.close()) for s in [socket.socket(socket.AF_INET, socket.SOCK_DGRAM)]][0][1]Below log shows that host test has failed to receive the sync correctly but since greentea works on test start and end prints it reads tests as pass. Perhaps it should make sure sync has worked correctly before parsing test case logs.
12:40:33 [1470311095.82][HTST][INF] starting host test process...
12:40:34 [1470311096.42][CONN][INF] starting serial connection process...
12:40:34 [1470311096.42][CONN][INF] notify event queue about extra 60 sec timeout for serial port pooling
12:40:34 [1470311096.42][CONN][INF] initializing serial port listener...
12:40:34 [1470311096.42][HTST][INF] setting timeout to: 60 sec
12:40:34 [1470311096.42][SERI][INF] serial(port=COM18, baudrate=9600, timeout=0)
12:40:34 Plugin info: HostTestPluginBase::BasePlugin: Waiting up to 60 sec for '0240000029304e45002a500878a200311951000097969900' serial port (current is 'COM18')...
12:40:34 [1470311096.52][SERI][INF] reset device using 'default' plugin...
12:40:34 [1470311096.78][SERI][INF] waiting 1.00 sec after reset
12:40:35 [1470311097.78][SERI][INF] wait for it...
12:40:35 [1470311097.78][SERI][TXD] mbedmbedmbedmbedmbedmbedmbedmbedmbedmbed
12:40:35 [1470311097.78][CONN][INF] sending up to 2 __sync packets (specified with --sync=2)
12:40:35 [1470311097.78][CONN][INF] sending preamble '4cb117f6-44e1-481d-b583-1093bab337cf'
12:40:35 [1470311097.78][SERI][TXD] {{__sync;4cb117f6-44e1-481d-b583-1093bab337cf}}
12:40:35 [1470311097.78][CONN][RXD] estcase_count;1}}
12:40:35 [1470311097.78][CONN][RXD] >>> Running 1 test cases...
12:40:35 [1470311097.92][CONN][RXD] {{__testcase_name;{{__sync;4cb117f6-44e1-481d-b583-1093bab337cf}}
12:40:35 [1470311097.92][CONN][WRN] found KV pair in stream: {{__testcase_name;{{__sync;4cb117f6-44e1-481d-b583-1093bab337cf}}, ignoring...
12:40:35 [1470311097.94][CONN][RXD] {{__version;1.3.0}}
12:40:35 [1470311097.94][CONN][WRN] found KV pair in stream: {{__version;1.3.0}}, ignoring...
12:40:35 [1470311097.95][CONN][RXD] {{__timeout;5}}
12:40:35 [1470311097.96][CONN][WRN] found KV pair in stream: {{__timeout;5}}, ignoring...
12:40:35 [1470311097.99][CONN][RXD] {{__host_test_name;default_auto}}
12:40:35 [1470311097.99][CONN][WRN] found KV pair in stream: {{__host_test_name;default_auto}}, ignoring...
12:40:35 [1470311098.02][CONN][RXD] {{__testcase_count;8}}
12:40:35 [1470311098.02][CONN][WRN] found KV pair in stream: {{__testcase_count;8}}, ignoring...
12:40:35 [1470311098.05][CONN][RXD] >>> Running 8 test cases...
12:40:35 [1470311098.08][CONN][RXD] {{__testcase_name;C strings: strtok}}
12:40:35 [1470311098.08][CONN][WRN] found KV pair in stream: {{__testcase_name;C strings: strtok}}, ignoring...
12:40:35 [1470311098.12][CONN][RXD] {{__testcase_name;C strings: strpbrk}}
12:40:35 [1470311098.12][CONN][WRN] found KV pair in stream: {{__testcase_name;C strings: strpbrk}}, ignoring...
12:40:35 [1470311098.18][CONN][RXD] {{__testcase_name;C strings: %i %d integer formatting}}
12:40:35 [1470311098.18][CONN][WRN] found KV pair in stream: {{__testcase_name;C strings: %i %d integer formatting}}, ignoring...
12:40:35 [1470311098.24][CONN][RXD] {{__testcase_name;C strings: %u %d integer formatting}}
12:40:35 [1470311098.24][CONN][WRN] found KV pair in stream: {{__testcase_name;C strings: %u %d integer formatting}}, ignoring...
12:40:35 [1470311098.30][CONN][RXD] {{__testcase_name;C strings: %x %E integer formatting}}
12:40:35 [1470311098.30][CONN][WRN] found KV pair in stream: {{__testcase_name;C strings: %x %E integer formatting}}, ignoring...
12:40:36 [1470311098.36][CONN][RXD] {{__testcase_name;C strings: %f %f float formatting}}
12:40:36 [1470311098.36][CONN][WRN] found KV pair in stream: {{__testcase_name;C strings: %f %f float formatting}}, ignoring...
12:40:36 [1470311098.41][CONN][RXD] {{__testcase_name;C strings: %e %E float formatting}}
12:40:36 [1470311098.41][CONN][WRN] found KV pair in stream: {{__testcase_name;C strings: %e %E float formatting}}, ignoring...
12:40:36 [1470311098.47][CONN][RXD] {{__testcase_name;C strings: %g %g float formatting}}
12:40:36 [1470311098.47][CONN][WRN] found KV pair in stream: {{__testcase_name;C strings: %g %g float formatting}}, ignoring...
12:40:36 [1470311098.52][CONN][RXD] >>> Running case #1: 'C strings: strtok'...
12:40:36 [1470311098.56][CONN][RXD] {{__testcase_start;C strings: strtok}}
12:40:36 [1470311098.56][CONN][WRN] found KV pair in stream: {{__testcase_start;C strings: strtok}}, ignoring...
12:40:36 [1470311098.60][CONN][RXD] {{__testcase_finish;C strings: strtok;1;0}}
12:40:36 [1470311098.60][CONN][WRN] found KV pair in stream: {{__testcase_finish;C strings: strtok;1;0}}, ignoring...
12:40:36 [1470311098.65][CONN][RXD] >>> 'C strings: strtok': 1 passed, 0 failed
12:40:36 [1470311098.70][CONN][RXD] >>> Running case #2: 'C strings: strpbrk'...
12:40:36 [1470311098.74][CONN][RXD] {{__testcase_start;C strings: strpbrk}}
12:40:36 [1470311098.74][CONN][WRN] found KV pair in stream: {{__testcase_start;C strings: strpbrk}}, ignoring...
12:40:36 [1470311098.78][CONN][INF] resending new preamble 'b028c99b-5297-434f-be73-1de8d47abbb4' after 1.00 sec
12:40:36 [1470311098.78][SERI][TXD] {{__sync;b028c99b-5297-434f-be73-1de8d47abbb4}}
12:40:36 [1470311098.78][CONN][RXD] {{__testcase_finish;C strings: strpbrk;1;0}}
12:40:36 [1470311098.78][CONN][WRN] found KV pair in stream: {{__testcase_finish;C strings: strpbrk;1;0}}, ignoring...
12:40:36 [1470311098.83][CONN][RXD] >>> 'C strings: strpbrk': 1 passed, 0 failed
12:40:36 [1470311098.90][CONN][RXD] >>> Running case #3: 'C strings: %i %d integer formatting'...
12:40:36 [1470311098.96][CONN][RXD] {{__testcase_start;C strings: %i %d integer formatting}}
12:40:36 [1470311098.96][CONN][WRN] found KV pair in stream: {{__testcase_start;C strings: %i %d integer formatting}}, ignoring...
12:40:36 [1470311099.02][CONN][RXD] {{__testcase_finish;C strings: %i %d integer formatting;1;0}}
12:40:36 [1470311099.02][CONN][WRN] found KV pair in stream: {{__testcase_finish;C strings: %i %d integer formatting;1;0}}, ignoring...
12:40:36 [1470311099.08][CONN][RXD] >>> 'C strings: %i %d integer formatting': 1 passed, 0 failed
12:40:36 [1470311099.15][CONN][RXD] >>> Running case #4: 'C strings: %u %d integer formatting'...
12:40:36 [1470311099.21][CONN][RXD] {{__testcase_start;C strings: %u %d integer formatting}}
12:40:36 [1470311099.21][CONN][WRN] found KV pair in stream: {{__testcase_start;C strings: %u %d integer formatting}}, ignoring...
12:40:36 [1470311099.28][CONN][RXD] {{__testcase_finish;C strings: %u %d integer formatting;1;0}}
12:40:36 [1470311099.28][CONN][WRN] found KV pair in stream: {{__testcase_finish;C strings: %u %d integer formatting;1;0}}, ignoring...
12:40:37 [1470311099.34][CONN][RXD] >>> 'C strings: %u %d integer formatting': 1 passed, 0 failed
12:40:37 [1470311099.40][CONN][RXD] >>> Running case #5: 'C strings: %x %E integer formatting'...
12:40:37 [1470311099.46][CONN][RXD] {{__testcase_start;C strings: %x %E integer formatting}}
12:40:37 [1470311099.46][CONN][WRN] found KV pair in stream: {{__testcase_start;C strings: %x %E integer formatting}}, ignoring...
12:40:37 [1470311099.53][CONN][RXD] {{__testcase_finish;C strings: %x %E integer formatting;1;0}}
12:40:37 [1470311099.53][CONN][WRN] found KV pair in stream: {{__testcase_finish;C strings: %x %E integer formatting;1;0}}, ignoring...
12:40:37 [1470311099.59][CONN][RXD] >>> 'C strings: %x %E integer formatting': 1 passed, 0 failed
12:40:37 [1470311099.66][CONN][RXD] >>> Running case #6: 'C strings: %f %f float formatting'...
12:40:37 [1470311099.71][CONN][RXD] {{__testcase_start;C strings: %f %f float formatting}}
12:40:37 [1470311099.71][CONN][WRN] found KV pair in stream: {{__testcase_start;C strings: %f %f float formatting}}, ignoring...
12:40:37 [1470311099.78][CONN][RXD] {{__testcase_finish;C strings: %f %f float formatting;1;0}}
12:40:37 [1470311099.78][CONN][WRN] found KV pair in stream: {{__testcase_finish;C strings: %f %f float formatting;1;0}}, ignoring...
12:40:37 [1470311099.84][CONN][RXD] >>> 'C strings: %f %f float formatting': 1 passed, 0 failed
12:40:37 [1470311099.90][CONN][RXD] >>> Running case #7: 'C strings: %e %E float formatting'...
12:40:37 [1470311099.96][CONN][RXD] {{__testcase_start;C strings: %e %E float formatting}}
12:40:37 [1470311099.96][CONN][WRN] found KV pair in stream: {{__testcase_start;C strings: %e %E float formatting}}, ignoring...
12:40:37 [1470311100.02][CONN][RXD] {{__testcase_finish;C strings: %e %E float formatting;1;0}}
12:40:37 [1470311100.02][CONN][WRN] found KV pair in stream: {{__testcase_finish;C strings: %e %E float formatting;1;0}}, ignoring...
12:40:37 [1470311100.09][CONN][RXD] >>> 'C strings: %e %E float formatting': 1 passed, 0 failed
12:40:37 [1470311100.15][CONN][RXD] >>> Running case #8: 'C strings: %g %g float formatting'...
12:40:37 [1470311100.20][CONN][RXD] {{__testcase_start;C strings: %g %g float formatting}}
12:40:37 [1470311100.21][CONN][WRN] found KV pair in stream: {{__testcase_start;C strings: %g %g float formatting}}, ignoring...
12:40:37 [1470311100.27][CONN][RXD] {{__testcase_finish;C strings: %g %g float formatting;1;0}}
12:40:37 [1470311100.27][CONN][WRN] found KV pair in stream: {{__testcase_finish;C strings: %g %g float formatting;1;0}}, ignoring...
12:40:38 [1470311100.33][CONN][RXD] >>> 'C strings: %g %g float formatting': 1 passed, 0 failed
12:40:38 [1470311100.37][CONN][RXD] >>> Test cases: 8 passed, 0 failed
12:40:38 [1470311100.40][CONN][RXD] {{__testcase_summary;8;0}}
12:40:38 [1470311100.40][CONN][WRN] found KV pair in stream: {{__testcase_summary;8;0}}, ignoring...
12:40:38 [1470311100.41][CONN][RXD] {{end;success}}
12:40:38 [1470311100.41][CONN][WRN] found KV pair in stream: {{end;success}}, ignoring...
12:40:38 [1470311100.43][CONN][RXD] {{__exit;0}}
12:40:38 [1470311100.43][CONN][WRN] found KV pair in stream: {{__exit;0}}, ignoring...
12:41:34 [1470311156.42][HTST][INF] test suite run finished after 60.00 sec...
12:41:34 [1470311156.42][CONN][INF] received special even '__host_test_finished' value='True', finishing
12:41:34 [1470311156.43][HTST][INF] CONN exited with code: 0
12:41:34 [1470311156.43][HTST][INF] No events in queue
12:41:34 [1470311156.43][HTST][INF] stopped consuming events
12:41:34 [1470311156.43][HTST][INF] host test result(): None
12:41:34 [1470311156.43][HTST][WRN] missing __exit event from DUT
12:41:34 [1470311156.43][HTST][ERR] missing __exit event from DUT and no result from host test, timeout...
12:41:34 [1470311156.43][HTST][INF] calling blocking teardown()
12:41:34 [1470311156.43][HTST][INF] teardown() finished
12:41:34 [1470311156.43][HTST][INF] {{result;timeout}}
12:41:34 mbedgt: checking for GCOV data...
12:41:34 mbedgt: mbed-host-test-runner: stopped
12:41:34 mbedgt: mbed-host-test-runner: returned 'TIMEOUT'
12:41:34 mbedgt: test on hardware with target id: 0240000029304e45002a500878a200311951000097969900
12:41:34 mbedgt: test suite 'tests-mbed_drivers-c_strings' .................................................... TIMEOUT in 70.51 sec
12:41:34 test case: 'C strings: %e %E float formatting' ............................................... OK in 0.06 sec
12:41:34 test case: 'C strings: %f %f float formatting' ............................................... OK in 0.07 sec
12:41:34 test case: 'C strings: %g %g float formatting' ............................................... OK in 0.06 sec
12:41:34 test case: 'C strings: %i %d integer formatting' ............................................. OK in 0.06 sec
12:41:34 test case: 'C strings: %u %d integer formatting' ............................................. OK in 0.07 sec
12:41:34 test case: 'C strings: %x %E integer formatting' ............................................. OK in 0.07 sec
12:41:34 test case: 'C strings: strpbrk' .............................................................. OK in 0.04 sec
12:41:34 test case: 'C strings: strtok' ............................................................... OK in 0.04 sec
Executing
mbedhtrun -c stlink -r stlink ...
on OS X results in this error:
-p: ST-LINK_CLI.exe: command not found
This is not surprising, since in both plug-ins, the CLI is hardcoded to this version.
For OS X (and Linux) this should be st-flash.
However the command format needs to be different too:
st-flash --debug write path/to/image.bin 0x08000000
04:05:55 calling mbedhtrun: mbedhtrun -d /mnt/DAPLINK -p /dev/ttyACM1:9600 -f "./build/frdm-k64f-gcc/test/configuration-store-test-find.bin" -C 4 -c shell -m K64F -t 0240000033514e450054500585d4001de981000097969900
04:05:55 mbedgt: mbed-host-test-runner: started
04:05:57 [1462935927.28][HTST][INF] host test executor ver. 0.2.8
04:05:57 [1462935927.28][HTST][INF] copy image onto target...
04:06:37 Plugin info: HostTestPluginCopyMethod_Shell::CopyMethod: Waiting for '0240000033514e450054500585d4001de981000097969900' mount point (current is '/mnt/DAPLINK')...
04:06:37 [1462935967.19][HTST][INF] starting host test process...
04:06:37 [1462935967.21][CONN][INF] starting connection process...
04:06:37 [1462935967.21][CONN][INF] initializing serial port listener...
04:06:37 [1462935967.22][SERI][INF] serial(port=/dev/ttyACM1, baudrate=9600)
04:06:44 [1462935974.08][SERI][ERR] [Errno 2] could not open port /dev/ttyACM1: [Errno 2] No such file or directory: '/dev/ttyACM1'
04:06:44 [1462935974.08][CONN][INF] sending preamble '40a38485-e62e-4c96-b8e7-bdddff50ba1f'...
04:06:44 [1462935974.08][SERI][TXD] {{__sync;40a38485-e62e-4c96-b8e7-bdddff50ba1f}}
04:06:44 [1462935974.08][HTST][ERR] connection lost, serial.Serial(/dev/ttyACM1. 9600, 0): [Errno 2] could not open port /dev/ttyACM1: [Errno 2] No such file or directory: '/dev/ttyACM1'
04:06:44 [1462935974.08][HTST][WRN] stopped to consume events due to __notify_conn_lost event
04:06:44 [1462935974.08][HTST][INF] test suite run finished after 6.89 sec...
04:06:44 [1462935974.09][HTST][INF] CONN exited with code: 0
04:06:44 [1462935974.09][HTST][INF] No events in queue
04:06:44 [1462935974.09][HTST][INF] host test result() call skipped, received: ioerr_serial
04:06:44 [1462935974.09][HTST][WRN] missing __exit event from DUT
04:06:44 [1462935974.09][HTST][INF] calling blocking teardown()
04:06:44 [1462935974.09][HTST][INF] teardown() finished
04:06:44 [1462935974.09][HTST][INF] {{result;ioerr_serial}}
See: http://e108747.cambridge.arm.com:8080/job/gt/3671/console
When running mbed-host-tests on Mac OS X, I frequently get unstable results from the serial driver.
e.g.
info: using filter "mbedgt --digest stdin -v -V" for tests in /Users/bremor01/dev/yotta/import/mbed-net-lwip/build/frdm-k64f-gcc/test
info: test mbed-net-lwip: mbed-net-lwip-test-tcp_echo_client
MBED: Instrumentation: "/dev/tty.usbmodem14242" and disk: "/Volumes/MBED"
HOST: Copy image onto target...
HOST: Initialize serial port...
...port ready!
HOST: Reset target...
HOST: Property 'timeout' = '5'
HOST: Property 'host_test_name' = 'default'
HOST: Property 'description' = 'Socket Abstraction Layer construction and utility test'
HOST: Property 'test_id' = 'SAL_INIT_UTIL'
HOST: Start test...
{{summary socket_api_test_create_destroy [PA
{{ioerr_serial}}
{{end}}
mbed-host-test-runner: Stopped
Here, the serial was working fine at first, but part-way through the test, the serial port has died.
This issue does not appear in minicom.
This code is not guaranteed to work across platforms:
https://github.com/ARMmbed/htrun/blob/master/mbed_host_tests/host_tests/mbed_client_auto.py#L41
OWN_PC_ADDRESS = socket.gethostbyname(socket.gethostname())Please use this instead:
SERVER_IP = str(socket.gethostbyname(socket.getfqdn()))`Currently yotta test command executes in module subdirectory and mbedgt (greentea) can't deduce and enumerate host tests in default module/test/host_tests.
How to reproduce:
$ cd C:\Work\mbed-drivers
$ yotta test
yotat test execution directory of tests (mbedgt): c:\Work\mbed-drivers\build\frdm-k64f-gcc\test
c:\Work\mbed-drivers\build\frdm-k64f-gcc\test\test\host_testsc:\Work\mbed-drivers\build\test\host_testsOn networks without DHCP/DNS integration, htrun fails to load network tests, which causes all of htrun to fail to load. This is hidden by ARMmbed/greentea#45.
The load failure is because hosts do not have resolvable names on networks without a DHCP/DNS integration, which causes the call to: gethostbyname to fail.
Many home networks do not have DHCP/DNS integration.
On a ubuntu platforms, it is found that remnants from previous runs of htrun in the serial buffer could messup the current run.
Currently we are using custom logger to log all events in htrun threads / processes we are using.
We want to replace custom logger which is using custom locking with python module logging.
mbed-drivers) must be virtually the same.htrun must report the same line format with custom logger.$python setup.py test to run basic unit tests for htrun.More here: #46
In short, pyserial module in latest version 3.0 (previous was 2.7) pupated APIs and deprecated existing ones.
We need to support both for compatibility.
The code reference for what I understand is the test runner which expects the result to be returned from a test():
htrun/mbed_host_tests/__init__.py
Line 410 in a8435d2
Shall we consider async support for our test cases (for host tests)? This might be useful to write async host test, which would be terminated once {{end}} is received.
For instance, client has 2 test cases, which send data. Host test echoes them back. Once client is done and the tests passes, sends {{end}}, this would terminate host test() method, call teardown().
When I execute:
mbedhtrun -p /dev/tty.usbmodem14242 -d /Volumes/MBED -f <binary>
The copy appears to succeed, but the mbed drive contains the following FAIL.TXT:
Flash algorithm write command FAILURE
When I execute the following command line, the copy succeeds:
mbedhtrun -p /dev/tty.usbmodem14242 -d /Volumes/MBED -f <binary> -c cp
Is it possible to default to a shell copy on mac?
OS: Windows 7
file htrun/mbed_host_tests/host_tests/test_socket_server_udp.py
line 24
Issue: socket.gethostbyname(socket.getfqdn()) fails to resolve to IP address
On Windows 7 call to socket.getfqdn() gives . (Fully qualified domain name)
In order for socket.gethostbyname() to work it expects that computer's full name is like ..
The full computer name may not be in . format. In that case socket.gethostbyname() fails to resolve socket.getfqdn() (.) into the IP address.
Possible solutions:
By configuration: Follow the link for steps for configuring DNS suffix: https://technet.microsoft.com/en-us/library/cc786695(v=ws.10).aspx
Add this to documentation.
In code: If gethostbyname() fail for FQDN then try host name. As hostname resolution to IP succeeds.
Anecdotal evidence suggests that the boards don't like it when you send them 🍞.
Have you tried sending 🍕 or 🍔 instead?
https://github.com/ARMmbed/htrun/blob/master/mbed_host_tests/__init__.py#L145
Due to issue ARMmbed/greentea#55 it is desired that host test enumeration is explained in the mbed host test README.
We falsely assumed pySerial will have 3.x version numbers.
======================================================================
FAIL: test_pyserial_version_as_float (test.host_test_plugins.HostOSDetectionTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "c:\work\pypi\htrun\test\host_test_plugins.py", line 61, in test_pyserial_version_as_float
self.assertTrue(self.re_float.findall(pyserial_version))
AssertionError: [] is not true
----------------------------------------------------------------------
Ran 22 tests in 0.034s
I think @adbridge disabled on the https://github.com/ARMmbed/greentea repository. Can this be done here as well?
mbed-host-tests/README.md:177
-C COPY_METHOD, --program_cycle_s=COPY_METHOD
current usage of mbedflsh -d E: -f /path/to/file.bin
It shouldn't be too hard to figure out the target path using mbedls. A search of current directory can help identify the binary path. These improvements can greatly improve the user experience.
When invoking htrun through greentea, the connection to remote board often timeout. This makes it difficult to maintain a proper CI flow.
Greentea called from Jenkins:
execute("mbedgt --grm ${target}:raas_client:54.229.57.51:8000 -V -v --test-spec ${build_dir}/test_spec.json -n ${test_filters.join(',')} -e TESTS/powercut/host_tests")
Output from failed run:
10:42:08 [K64F GCC_ARM default] [1494585728.90][requests.packages.urllib3.connectionpool]Starting new HTTP connection (1): 54.229.57.51
10:42:08 [K64F GCC_ARM default] [1494585728.97][requests.packages.urllib3.connectionpool]http://54.229.57.51:8000 "PUT /resource/0240000034544e45002d00028aa900022011000097969900/disconnect HTTP/1.1" 200 85
10:42:08 [K64F GCC_ARM default] [1494585728.97][requests.packages.urllib3.connectionpool]Starting new HTTP connection (1): 54.229.57.51
10:42:10 [K64F GCC_ARM default] [1494585731.08][requests.packages.urllib3.connectionpool]http://54.229.57.51:8000 "PUT /resource/0240000034544e45002d00028aa900022011000097969900/release HTTP/1.1" 200 85
10:42:10 [K64F GCC_ARM default] [1494585731.08][requests.packages.urllib3.connectionpool]Starting new HTTP connection (1): 54.229.57.51
10:42:10 [K64F GCC_ARM default] [1494585731.18][requests.packages.urllib3.connectionpool]http://54.229.57.51:8000 "PUT /resource/0240000034544e45002d00028aa900022011000097969900/disconnect HTTP/1.1" 403 193
10:42:10 [K64F GCC_ARM default] [1494585731.18][GLRM][ERR] RemoteConnectorPrimitive.finish() failed, reason: Disconnect request failed
Related comments:
https://jira.arm.com/browse/IOTSYST-2282
When running all tests from mbed-os I've notices some issues inside (traceback). Please check.
Command to reproduce:
~/mbed-os$ mbedgt -V --report-html new03.html
tests-mbed_drivers-rtc testI was running some tests with mbed-os and saw this issue in even loop:
Traceback (most recent call last): File "c:\Python27.11\Scripts\mbedhtrun-script.py", line 9, in
load_entry_point('mbed-host-tests', 'console_scripts', 'mbedhtrun')()
File "c:\work\pypi\htrun\mbed_host_tests\mbedhtrun.py", line 34, in main
result = test_selector.execute()
File "c:\work\pypi\htrun\mbed_host_tests\host_tests_runner\host_test_default.py", line 164, in execute
test_result = self.run_test()
File "c:\work\pypi\htrun\mbed_host_tests\host_tests_runner\host_test_default.py", line 133, in run_test
return event_loop.run_loop()
File "c:\work\pypi\htrun\mbed_host_tests\host_tests_event_loop\base_event_loop.py", line 88, in run_loop
self.run_finish()
File "c:\work\pypi\htrun\mbed_host_tests\host_tests_event_loop\default_event_loop.py", line 227, in run_finish
self.logger.prn_inf("host test result(): %s"% str(result))
NameError: global name 'result' is not defined
See HTML report for more details.
check_serial_port_ready always returns true. There is logic to handle a timeout, but check_serial_port_ready doesn't notify its caller (SerialConnectorPrimitive.__init__) that no serial port was found after a timeout. As such, htrun attempts to use a serial port that isn't yet ready, to send a sync message, leading to failures to sync with greentea-client.
htrun shouldn't attempt to use a serial port until the port is ready. If waiting for the serial port times out, then htrun should fail right then and there and not continue.
On the MPS2+, the module_reset_mps2.py file handles the rebooting process. It happens copying a reboot.txt file onto the board.
However sometimes the test process continues because the file is not actually copied to the board and the board stays mounted.
A sync command, as done in module_copy_mps2.py, prevents this error and forces the file to be copied onto the board.
A pull request is going to be sent to fix this.
@malcolm-brooks-arm
mbedhtrun behaves correctly not causing any serial port issues.mbedhtrun is not handling serial port operations correctly causing serial port access issues:mbedgt: selecting test case observer...
calling mbedhtrun: mbedhtrun -d E: -p COM149 -f ".\build\frdm-k64f-gcc\test\mbed-drivers-test-asynch_spi.bin" -C 4 -c shell -m K64F
mbedgt: mbed-host-test-runner: started
MBED: Instrumentation: "COM149" and disk: "E:"
HOST: Copy image onto target...
1 file(s) copied.
HOST: Initialize serial port...
........................................MBED: poll_for_serial_init(9600, 1, 40, 0.5, 0.25)
MBED: could not open port 'COM149': WindowsError(5, 'Access is denied.')
{{ioerr_serial}}
{{end}}
mbedgt: mbed-host-test-runner: stopped
mbedgt: mbed-host-test-runner: returned 'IOERR_SERIAL'
test 'mbed-drivers-test-asynch_spi' .................................................... IOERR_SERIAL in 17.51 sec
See:
MBED: could not open port 'COM149': WindowsError(5, 'Access is denied.')
Version: 0226
Build: Aug 24 2015 17:06:30
Git Commit SHA: 27a236b9fe39c674a703c5c89655fbd26b8e27e1
Git Local mods: Yes
Running a test with more output than usual results in a cut-off of serial output and the test ultimately times out.
The serial output buffer is apparently also not properly emptied, as the same partial output appears twice.
https://gist.github.com/salkinium/4d46cb487bbb8a292e46
This does not happen when reducing the serial output of the test.
It also does not happen when using a non-Python terminal emulation program, like picocom.
There is also buffered output in picocom when opening it on the same port any time after the yt test timed out:
https://gist.github.com/salkinium/0391bbd70f9071580d5e
Pull requests #85 and #86 add support for checking the mount point and serial port for a platform. Since this information can be derived from mbed-ls (when supplied a target ID), it seems like the only time you would provide the serial port or mount point is when you don't know the target ID, the platform doesn't have a valid target ID, or you want to override one or many of the values that were resolved through mbed-ls.
If no target ID is provided, then it should default to requiring a provided serial port, baud rate, and mount point. If a target ID is provided, then you should be able to override the serial port, baud rate, and mount point through command line arguments.
It also seems valid to say sometimes you would want to override the baud rate without providing a serial port as part of the -p option (especially if you don't know the serial port). I would suggest adding a --baud-rate options to specify the serial port's baud rate specifically. The -p can still accept a baud rate component (<port>:<baud rate>) for backward compatibility. In the case that -p <port>:<baud rate> --baud-rate <different baud rate> are both specified, then --baud-rate would be used.
The priority for platform options from lowest to highest:
-d,--disk-path for disk/mount point, -p <port>:<baud rate> for serial port (:<baud rate> bit being optional)--baud-rate for serial baud rateI am working with ARM_CM3DS_MPS2 target, running on MPS2 board.
On Linux, while executing greentea tests, there is a bug when the binary image is copied from the BUILD folder to the board.
Here is the htrun log when executed with greentea:
[1511173577.37][HTST][INF] host test executor ver. 1.2.0
[1511173577.37][HTST][INF] copy image onto target...
[1511173577.37][COPY][INF] Waiting up to 60 sec for '50040200074D652F3828F333' mount point (current is '/media/ /V2M-MPS2')...
[1511173577.37][mbedls.lstools_base]Call to deprecated function list_mbeds_by_targetid. List formatting methods are deprecated to simplify the API. Please use 'list_mbeds' instead.
[1511173577.37][mbedls.lstools_base]Call to deprecated function list_mbeds_ext. Functionality has been moved into 'list_mbeds'. Please use list_mbeds with 'unique_names=True' and 'read_details_txt=True'
[1511173577.37][root]Found htm target id, 50040200074D652F3828F333, for usb target id 000000000000
[1511173577.40][COPY][ERR] shutil.copy('BUILD/tests/ARM_CM3DS_MPS2/ARM/TESTS/mbed_drivers/echo/echo.bin', '/media/ /V2M-MPS2/mbed.bin')
[1511173577.40][COPY][ERR] Error: run_command() takes at most 3 arguments (5 given)
[1511173581.41][COPY][INF] Waiting up to 60 sec for '50040200074D652F3828F333' mount point (current is '/media/ /V2M-MPS2')...
[1511173581.41][mbedls.lstools_base]Call to deprecated function list_mbeds_by_targetid. List formatting methods are deprecated to simplify the API. Please use 'list_mbeds' instead.
[1511173581.41][mbedls.lstools_base]Call to deprecated function list_mbeds_ext. Functionality has been moved into 'list_mbeds'. Please use list_mbeds with 'unique_names=True' and 'read_details_txt=True'
[1511173581.42][root]Found htm target id, 50040200074D652F3828F333, for usb target id 000000000000
[1511173581.46][COPY][ERR] shutil.copy('BUILD/tests/ARM_CM3DS_MPS2/ARM/TESTS/mbed_drivers/echo/echo.bin', '/media/ /V2M-MPS2/mbed.bin')
[1511173581.46][COPY][ERR] Error: run_command() takes at most 3 arguments (5 given)
^C[mbed] User aborted!
mbedgt: ctrl+c keyboard interrupt!
The cause of the bug has been identified and a pull request is soon going to be made.
Hi,
mbed-drivers-test-echo contains one test suite which contains three test cases.
In the first test case, mbed-host-tests will send one more KV pair than specified and interfere with next test case. So the second test case fails. This issue is met on Nuvoton's board but not on K64F board. I check with K64F and find mbed-host-tests still send one more KV pair as above, but I don't know why K64F can pass it.
Test environment
mbed-drivers: 1.4.0
mbed-host-tests: 0.2.6
Test flow
mbedgt -n mbed-drivers-test-echo -V
Test log
[1459490609.50][HTST][INF] starting host test process...
[1459490609.95][CONN][INF] starting connection process...
[1459490609.95][CONN][INF] initializing serial port listener...
[1459490609.95][SERI][INF] serial(port=COM31, baudrate=9600)
[1459490609.95][SERI][INF] reset device using 'default' plugin...
[1459490610.22][SERI][INF] wait for it...
[1459490611.23][CONN][INF] sending preamble 'a6d4ccb1-30c7-4a68-ba76-ebc8e51525d2'...
[1459490611.23][SERI][TXD] mbedmbedmbedmbedmbedmbedmbedmbedmbedmbed
[1459490611.23][SERI][TXD] {{__sync;a6d4ccb1-30c7-4a68-ba76-ebc8e51525d2}}
[1459490611.37][CONN][INF] found SYNC in stream: {{__sync;a6d4ccb1-30c7-4a68-ba76-ebc8e51525d2}}, queued...
[1459490611.37][HTST][INF] sync KV found, uuid=a6d4ccb1-30c7-4a68-ba76-ebc8e51525d2, timestamp=1459490611.373000
[1459490611.37][CONN][RXD] {{__sync;a6d4ccb1-30c7-4a68-ba76-ebc8e51525d2}}
[1459490611.39][CONN][INF] found KV pair in stream: {{__version;1.0.0}}, queued...
[1459490611.39][HTST][INF] DUT greentea-client version: 1.0.0
[1459490611.39][CONN][RXD] {{__version;1.0.0}}
[1459490611.40][CONN][INF] found KV pair in stream: {{__timeout;10}}, queued...
[1459490611.40][HTST][INF] setting timeout to: 10 sec
[1459490611.40][CONN][RXD] {{__timeout;10}}
[1459490611.43][CONN][INF] found KV pair in stream: {{__host_test_name;echo}}, queued...
[1459490611.43][HTST][INF] host test setup() call...
[1459490611.43][HTST][INF] CALLBACKs updated
[1459490611.43][HTST][INF] host test detected: echo
[1459490611.43][CONN][RXD] {{__host_test_name;echo}}
[1459490611.45][CONN][INF] found KV pair in stream: {{__testcase_count;3}}, queued...
[1459490611.45][CONN][RXD] {{__testcase_count;3}}
[1459490611.48][CONN][RXD] >>> Running 3 test cases...
[1459490611.50][CONN][INF] found KV pair in stream: {{echo_count;16}}, queued...
[1459490611.50][SERI][TXD] {{echo_count;16}}
[1459490611.50][CONN][RXD] {{echo_count;16}}
[1459490611.50][SERI][TXD] {{echo;918a603b-f5f4-4c21-ad6d-c280cbb44622}}
[1459490611.62][CONN][INF] found KV pair in stream: {{echo;918a603b-f5f4-4c21-ad6d-c280cbb44622}}, queued...
[1459490611.62][SERI][TXD] {{echo;f8a44d38-3513-4a16-971b-2509404387d3}}
[1459490611.62][CONN][RXD] {{echo;918a603b-f5f4-4c21-ad6d-c280cbb44622}}
[1459490611.72][CONN][INF] found KV pair in stream: {{echo;f8a44d38-3513-4a16-971b-2509404387d3}}, queued...
[1459490611.72][SERI][TXD] {{echo;ef663022-f2a4-455e-9b6f-989892399b52}}
[1459490611.72][CONN][RXD] {{echo;f8a44d38-3513-4a16-971b-2509404387d3}}
[1459490611.81][CONN][INF] found KV pair in stream: {{echo;ef663022-f2a4-455e-9b6f-989892399b52}}, queued...
[1459490611.81][CONN][RXD] {{echo;ef663022-f2a4-455e-9b6f-989892399b52}}
[1459490611.81][SERI][TXD] {{echo;77b30340-e301-44b6-9649-f50e3f3806d5}}
[1459490611.90][CONN][INF] found KV pair in stream: {{echo;77b30340-e301-44b6-9649-f50e3f3806d5}}, queued...
[1459490611.90][SERI][TXD] {{echo;eb36ac12-b56d-445a-8c67-cc70c06607aa}}
[1459490611.90][CONN][RXD] {{echo;77b30340-e301-44b6-9649-f50e3f3806d5}}
[1459490612.00][CONN][INF] found KV pair in stream: {{echo;eb36ac12-b56d-445a-8c67-cc70c06607aa}}, queued...
[1459490612.00][SERI][TXD] {{echo;1e024901-98ff-4622-b335-603cbffba496}}
[1459490612.00][CONN][RXD] {{echo;eb36ac12-b56d-445a-8c67-cc70c06607aa}}
[1459490612.09][CONN][INF] found KV pair in stream: {{echo;1e024901-98ff-4622-b335-603cbffba496}}, queued...
[1459490612.09][SERI][TXD] {{echo;7e5e1742-0c89-4218-b606-67e41ada8406}}
[1459490612.09][CONN][RXD] {{echo;1e024901-98ff-4622-b335-603cbffba496}}
[1459490612.18][CONN][INF] found KV pair in stream: {{echo;7e5e1742-0c89-4218-b606-67e41ada8406}}, queued...
[1459490612.18][SERI][TXD] {{echo;050ad3e1-e610-4fc1-8024-4b40b10c1655}}
[1459490612.18][CONN][RXD] {{echo;7e5e1742-0c89-4218-b606-67e41ada8406}}
[1459490612.28][CONN][INF] found KV pair in stream: {{echo;050ad3e1-e610-4fc1-8024-4b40b10c1655}}, queued...
[1459490612.28][SERI][TXD] {{echo;aa065b99-f197-4577-a43b-2efe4dd51a3f}}
[1459490612.28][CONN][RXD] {{echo;050ad3e1-e610-4fc1-8024-4b40b10c1655}}
[1459490612.37][CONN][INF] found KV pair in stream: {{echo;aa065b99-f197-4577-a43b-2efe4dd51a3f}}, queued...
[1459490612.37][SERI][TXD] {{echo;a324735f-621c-4b11-86b5-e79f38e6eaaa}}
[1459490612.37][CONN][RXD] {{echo;aa065b99-f197-4577-a43b-2efe4dd51a3f}}
[1459490612.46][CONN][INF] found KV pair in stream: {{echo;a324735f-621c-4b11-86b5-e79f38e6eaaa}}, queued...
[1459490612.46][SERI][TXD] {{echo;3606d3d6-e2d3-4a49-95f2-cbc1e92f7c52}}
[1459490612.46][CONN][RXD] {{echo;a324735f-621c-4b11-86b5-e79f38e6eaaa}}
[1459490612.56][CONN][INF] found KV pair in stream: {{echo;3606d3d6-e2d3-4a49-95f2-cbc1e92f7c52}}, queued...
[1459490612.56][SERI][TXD] {{echo;b3f9b48e-8983-43f6-95a8-132dd2329013}}
[1459490612.56][CONN][RXD] {{echo;3606d3d6-e2d3-4a49-95f2-cbc1e92f7c52}}
[1459490612.65][CONN][INF] found KV pair in stream: {{echo;b3f9b48e-8983-43f6-95a8-132dd2329013}}, queued...
[1459490612.67][SERI][TXD] {{echo;8af73d75-6609-4513-ba3e-4fe4b829bb53}}
[1459490612.67][CONN][RXD] {{echo;b3f9b48e-8983-43f6-95a8-132dd2329013}}
[1459490612.76][CONN][INF] found KV pair in stream: {{echo;8af73d75-6609-4513-ba3e-4fe4b829bb53}}, queued...
[1459490612.76][SERI][TXD] {{echo;cbcce0be-c7b8-4222-8096-62b7a820cf33}}
[1459490612.76][CONN][RXD] {{echo;8af73d75-6609-4513-ba3e-4fe4b829bb53}}
[1459490612.86][CONN][INF] found KV pair in stream: {{echo;cbcce0be-c7b8-4222-8096-62b7a820cf33}}, queued...
[1459490612.86][SERI][TXD] {{echo;b2b13b58-ed3a-4c0a-af0b-3fed4ab6c9f5}}
[1459490612.86][CONN][RXD] {{echo;cbcce0be-c7b8-4222-8096-62b7a820cf33}}
[1459490612.95][CONN][INF] found KV pair in stream: {{echo;b2b13b58-ed3a-4c0a-af0b-3fed4ab6c9f5}}, queued...
[1459490612.95][SERI][TXD] {{echo;d0476bc5-2622-45a7-b7da-71d335aa25f9}}
[1459490612.95][CONN][RXD] {{echo;b2b13b58-ed3a-4c0a-af0b-3fed4ab6c9f5}}
[1459490613.04][CONN][INF] found KV pair in stream: {{echo;d0476bc5-2622-45a7-b7da-71d335aa25f9}}, queued...
[1459490613.04][SERI][TXD] {{echo;0ea5f1f4-168f-48c3-b7b2-dee97cbd1f96}}
[1459490613.04][CONN][RXD] {{echo;d0476bc5-2622-45a7-b7da-71d335aa25f9}}
[1459490613.06][CONN][INF] found KV pair in stream: {{echo_count;32}}, queued...
[1459490613.06][SERI][TXD] {{echo_count;32}}
[1459490613.06][SERI][TXD] {{echo;7317ef92-fe8e-4d88-8363-c678031f0c61}}
[1459490613.06][CONN][RXD] {{echo_count;32}}
[1459490613.12][CONN][RXD] :36::FAIL: Expected 32 Was 0
[1459490613.18][CONN][RXD] >>> failure with reason 'Assertion Failed' during 'Case Handler'
[1459490621.42][HTST][INF] test suite run finished after 10.01 sec...
[1459490621.42][HTST][INF] CONN exited with code: 0
[1459490621.42][HTST][INF] No events in queue
[1459490621.42][HTST][INF] stopped consuming events
[1459490621.42][HTST][INF] host test result(): False
[1459490621.42][HTST][WRN] missing __exit event from DUT
[1459490621.42][HTST][INF] calling blocking teardown()
[1459490621.42][HTST][INF] teardown() finished
[1459490621.42][HTST][INF] {{result;failure}}
We are only calling setup() / teardown() when we handle ---run switch (not used in CI). We are not calling setup/teardown in normal workflow:
def execute_run(self):
...
try:
self.test_supervisor = get_host_test("run_binary_auto")
# Call to setup if function is implemented
if hasattr(self.test_supervisor, 'setup') and callable(getattr(self.test_supervisor, 'setup')):
self.test_supervisor.setup()
# Call to test function
result = self.test_supervisor.test(self) # This is blocking, waits for {end}
# Call to teardown if function is implemented
if hasattr(self.test_supervisor, 'teardown') and callable(getattr(self.test_supervisor, 'teardown')):
self.test_supervisor.teardown()
except Exception, e:
print str(e)
self.print_result(self.RESULT_ERROR)We should be calling it also here:
def execute(self):
...
try:
CONFIG = self.detect_test_config(verbose=True) # print CONFIG
result = None
if "host_test_name" in CONFIG:
if is_host_test(CONFIG["host_test_name"]):
#self.notify("HOST: CONFIG['host_test_name'] is '%s'" % CONFIG["host_test_name"])
self.test_supervisor = get_host_test(CONFIG["host_test_name"])
result = self.test_supervisor.test(self) #result = self.test()
else:
self.notify("HOST: Error! Unknown host test name '%s' (use 'mbedhtrun --list' to verify)!"% CONFIG["host_test_name"])
self.notify("HOST: Error! You can use switch '-e <dir>' to specify local directory with host tests to load")
self.print_result(self.RESULT_ERROR)
else:
self.notify("HOST: Error! No host test name defined in preamble")
self.print_result(self.RESULT_ERROR)probably missing code in above snippet:
self.test_supervisor.setup()
result = self.test_supervisor.test(self) #result = self.test()
self.test_supervisor.teardown()class DefaultAuto(BaseHostTest):
""" Simple, basic host test's test runner waiting for serial port
output from MUT, no supervision over test running in MUT is executed.
"""
def setup(self):
stdout.write("!!!")
stdout.flush()
def teardown(self):
stdout.write("@@@")
stdout.flush()
def test(self, selftest):
stdout.write("%%%")
stdout.flush()
return "XXX" self.test_supervisor = get_host_test(CONFIG["host_test_name"])
self.test_supervisor.setup()
result = self.test_supervisor.test(self) #result = self.test()
self.test_supervisor.teardown()mbedgt:$ mbedgt -V -n mbed-drivers-test-basic
...
mbedgt: selecting test case observer...
calling mbedhtrun: mbedhtrun -d F: -p COM184:9600 -f ".\build\frdm-k64f-gcc\test\mbed-drivers-test-basic.bin" -C 4 -c shell -m K64F -e "./test/host_tests"
mbedgt: mbed-host-test-runner: started
HOST: Instrumentation: "COM184" and disk: "F:"
HOST: My PID is 7516
HOST: Copy image onto target...
1 file(s) copied.
HOST: Initialize serial port...
...port ready!
HOST: Reset target...
HOST: Detecting test case properties...
HOST: Property 'timeout' = '20'
HOST: Property 'host_test_name' = 'default_auto'
HOST: Property 'description' = 'Basic'
HOST: Property 'test_id' = 'MBED_A1'
HOST: Start test...
!!!%%%@@@
{{XXX}}
{{end}}
This is the problem generated by trying to run the mbed-example-async-serial.bin on k64f as follows:
mbedhtrun d E: -p COM8 -f "build/frdm-k64f-gcc/source/mbed-example-asy
nch-serial.bin "
simhug01@E107851:/d/datastore/public/jobs/yr2015/jobs/2224/work/20150518/mbed-example-asynch-serial-private$ mbedls
+---------------------+-------------------+-------------------+--------------------------------+
|platform_name |mount_point |serial_port |target_id |
+---------------------+-------------------+-------------------+--------------------------------+
|K64F |E: |COM8 |02400221B1110E424CCFF3FA |
+---------------------+-------------------+-------------------+--------------------------------+
simhug01@E107851:/d/datastore/public/jobs/yr2015/jobs/2224/work/20150518/mbed-example-asynch-serial-private$ mbedhtrun -h
Usage: mbedhtrun-script.py [options]
Options:
-h, --help show this help message and exit
-m MICRO, --micro=MICRO
Target microcontroller name
-p PORT, --port=PORT Serial port of the target
-d DISK_PATH, --disk=DISK_PATH
Target disk (mount point) path
-f IMAGE_PATH, --image-path=IMAGE_PATH
Path with target's binary image
-c COPY_METHOD, --copy=COPY_METHOD
Copy method selector. Define which copy method (from
plugins) should be used
-C COPY_METHOD, --program_cycle_s=COPY_METHOD
Program cycle sleep. Define how many seconds you want
wait after copying binary onto target
-r FORCED_RESET_TYPE, --reset=FORCED_RESET_TYPE
Forces different type of reset
-R NUMBER, --reset-timeout=NUMBER
When forcing a reset using option -r you can set up
after reset idle delay in seconds
--list Prints registered host test and exits
simhug01@E107851:/d/datastore/public/jobs/yr2015/jobs/2224/work/20150518/mbed-example-asynch-serial-private$ mbedhtrun d E: -p COM8 -f build/frdm-k64f-gcc/
.ninja_deps CMakeCache.txt CMakeLists.txt build.ninja rules.ninja toolchain.cmake yotta_config.h
.ninja_log CMakeFiles/ CTestTestfile.cmake cmake_install.cmake source/ ym/
simhug01@E107851:/d/datastore/public/jobs/yr2015/jobs/2224/work/20150518/mbed-example-asynch-serial-private$ mbedhtrun d E: -p COM8 -f build/frdm-k64f-gcc/source/
CMakeFiles/ CTestTestfile.cmake mbed-example-asynch-serial mbed-example-asynch-serial.map
CMakeLists.txt cmake_install.cmake mbed-example-asynch-serial.bin
simhug01@E107851:/d/datastore/public/jobs/yr2015/jobs/2224/work/20150518/mbed-example-asynch-serial-private$ mbedhtrun d E: -p COM8 -f build/frdm-k64f-gcc/source/mbed
mbed-example-asynch-serial mbed-example-asynch-serial.bin mbed-example-asynch-serial.map
simhug01@E107851:/d/datastore/public/jobs/yr2015/jobs/2224/work/20150518/mbed-example-asynch-serial-private$ mbedhtrun d E: -p COM8 -f "build/frdm-k64f-gcc/source/mbed-example-asy
nch-serial.bin "
Traceback (most recent call last):
File "c:\mbed_tools\Python27\Scripts\mbedhtrun-script.py", line 9, in <module>
load_entry_point('mbed-host-tests==0.1.5', 'console_scripts', 'mbedhtrun')()
File "build\bdist.win32\egg\mbed_host_tests\mbedhtrun.py", line 26, in main
File "build\bdist.win32\egg\mbed_host_tests\__init__.py", line 87, in __init__
File "build\bdist.win32\egg\mbed_host_tests\host_tests_runner\host_test.py", line 112, in __init__
File "build\bdist.win32\egg\mbed_host_tests\host_tests_runner\host_test.py", line 45, in __init__
File "build\bdist.win32\egg\mbed_host_tests\host_tests_runner\mbed_base.py", line 45, in __init__
TypeError: float() argument must be a string or a number
simhug01@E107851:/d/datastore/public/jobs/yr2015/jobs/2224/work/20150518/mbed-example-asynch-serial-private$
Currently DefaultTestSelector class implements event loop for Key-Value protocol.
This forces htrun to rely on this functionality. As htrun changes we want to be able to have interchangeable even loop mechanism where event loop communicates with connection process using queues.
This refactoring should allow us to replace event loop implementation keeping current interface to it.
Event loop can be abstracted and current event loop can be implemented as part of "htrun event loop" interface.
This refactoring must be backward compatible with current htrun implementation which means if we do not specify any event loop name from command line etc. we will use current event loop to process all events.
htrun should be abstracted (separate class ?)htrun (it uses greentea-client features like __sync to communicate with device)
default and greentea-client.(at least on OS X and linux, not sure if this is possible on windows)
This would prevent "this disk was not ejected properly" notifications when running each test. To reproduce, run:
mbedhtrun -d /Volumes/MBED -f <pathtotest> -p <serialport device> -t 10 -C 4 -m K64F
MBED: Instrumentation: "/dev/tty.usbmodem1412" and disk: "/Volumes/MBED"
HOST: Copy image onto target...
HOST: Initialize serial port...
HOST: Reset target...
# disk not ejected error happens at this point
HOST: Unknown property:
Not calling BaseHostTest.__init__(self) in your explicitly defined host test class may cause state machine not to work properly (callbacks are not registered and properly called).
htrun should check if derived from BaseHostTest called its constructor and warn user if it doesn't.
Soluton:
__init__(self).__init__(self) in host test derived from BaseHostTest please call BaseHostTest.__init__(self) explicitly.Bad code:
from mbed_host_tests import BaseHostTest, event_callback
class TestJig(BaseHostTest):
def __init__(self):
self.match_state = 0xA9
self.pup_delay = 500000
self.pdwn_delay = 0Good code:
from mbed_host_tests import BaseHostTest, event_callback
class TestJig(BaseHostTest):
def __init__(self):
BaseHostTest.__init__(self) # Call to BaseHostTest constructor
self.match_state = 0xA9
self.pup_delay = 500000
self.pdwn_delay = 0
name = 'state_monitor_jig'[1507222815.67][urllib3.connectionpool]Starting new HTTP connection (1): austin-ci-linux-001.austin.arm.com
[1507222815.68][urllib3.connectionpool]http://austin-ci-linux-001.austin.arm.com:8000 "PUT /resource/0240000026334e45003840175c1a0048cb11000097969900/allocate HTTP/1.1" 200 1763
[1507222815.68][urllib3.connectionpool]Starting new HTTP connection (1): austin-ci-linux-001.austin.arm.com
[1507222815.68][urllib3.connectionpool]http://austin-ci-linux-001.austin.arm.com:8000 "PUT /resource/0240000026334e45003840175c1a0048cb11000097969900/disconnect HTTP/1.1" 400 74
[1507222815.68][GLRM][ERR] Disconnect request failed
[1507222815.68][urllib3.connectionpool]http://austin-ci-linux-001.austin.arm.com:8000 "PUT /resource/0240000026334e45003840175c1a0048cb11000097969900/disconnect HTTP/1.1" 400 74
[1507222815.68][GLRM][ERR] Disconnect request failed
Process Process-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/home/arm/repo/htrun/mbed_host_tests/host_tests_conn_proxy/conn_proxy.py", line 176, in conn_process
if not connector.write("mbed" * 10, log=True):
File "/home/arm/repo/htrun/mbed_host_tests/host_tests_conn_proxy/conn_primitive_remote.py", line 138, in write
self.selected_resource.write(payload)
File "/home/arm/repo/raas/raas-pyclient/raas_client/resources/Base.py", line 545, in write
format(Resource.CHANNEL_DEFAULT))
ResourceError: Attempted write when serial0 channel not open!
Not certain if this is a problem in the htrun or our tests and they don't send proper signal to htrun to complete its operation. Here's the command + the output
mbedhtrun -p COM62 -r stlink -c stlink -f build\stm32f429i-disco-gcc\test\mbed-drivers-test-basic.bin
MBED: Instrumentation: "COM62" and disk: "None"
HOST: Copy image onto target...
STM32 ST-LINK CLI v2.3.0
STM32 ST-LINK Command Line Interface
ST-LINK SN : 48FF70066770535335351767
ST-LINK Firmware version : V2J23S0
Connected via SWD.
SWD Frequency = 4000K.
Target voltage = 2.9 V.
Connection mode : Normal.
Device ID:0x419
Device flash Size : 2048 Kbytes
Device family :STM32F42xx/F43xx
Loading file...
Flash Programming:
File : build\stm32f429i-disco-gcc\test\mbed-drivers-test-basic.bin
Address : 0x08000000
Memory programming...
██████████████████████████████████████████████████ 100%
Memory programmed in 1s and 451ms.
Verification...OK
Programming Complete.
HOST: Initialize serial port...
...port ready!
HOST: Reset target...
STM32 ST-LINK CLI v2.3.0
STM32 ST-LINK Command Line Interface
ST-LINK SN : 48FF70066770535335351767
ST-LINK Firmware version : V2J23S0
Connected via SWD.
SWD Frequency = 4000K.
Target voltage = 2.9 V.
Connection mode : Normal.
Device ID:0x419
Device flash Size : 2048 Kbytes
Device family :STM32F42xx/F43xx
MCU Reset.
Application started.
HOST: Property 'timeout' = '20'
HOST: Property 'timeout' = '20'
HOST: Property 'host_test_name' = 'default_auto'
HOST: Property 'description' = 'Basic'
HOST: Property 'test_id' = 'MBED_A1'
HOST: Start test...
{{success}}
{{end}}
// here I had to do use control+c for windows to get out of the app, otherwise it would be just trapped here. It prints then ioerr serial
{{ioerr_serial}}
{{end}}
I added a comment 3 lines above in the code snippet
Currently the script wait_us_auto (I figured this it the file for ticker tests for example), prints this:
HOST: Start test...
* in 0.99 sec (0.01) [OK]
And I had another platform with this output:
HOST: Start test...
* in 2.00 sec (1.00) [FAIL]
What are those 2 numbers? The first is the actual time it took to get a character (0.99) and the second is deviation? It helped me to look at here to get more info what is the output (https://github.com/ARMmbed/htrun/blob/master/mbed_host_tests/host_tests/wait_us_auto.py#L62).
It would help us to get more descriptive output.
TBD but till now I know that Greentea spawns two processes which are using too much CPU.
This has to be debugged and tracked.
Originally reported here by @jeromecoutant:
In order to debug, I am following this page:
https://github.com/ARMmbed/mbed-os/blob/master/docs/testing_mbed_OS_5.md#debugging-testsIt seems that the skip-reset is not working...
$ mbedhtrun --skip-flashing --skip-reset -p COM62:9600 -vv
[1483627073.14][HTST][INF] host test executor ver. 1.1.6
[1483627073.14][HTST][INF] copy image onto target... SKIPPED!
[1483627073.15][HTST][INF] starting host test process...
[1483627074.41][CONN][INF] starting connection process...
[1483627074.41][CONN][INF] notify event queue about extra 60 sec timeout for serial port pooling
[1483627074.41][CONN][INF] initializing serial port listener...
[1483627074.41][SERI][INF] serial(port=COM62, baudrate=9600, timeout=0.01)
[1483627074.42][SERI][INF] reset device using 'default' plugin...
...Regards,
test_selector = DefaultTestSelector()
Traceback (most recent call last):
File "..\test.py", line 96, in <module>
test_selector = DefaultTestSelector()
File "C:\Python27\lib\site-packages\mbed_host_tests\__init__.py", line 256, in __init__
DefaultTestSelectorBase.__init__(self, options)
File "C:\Python27\lib\site-packages\mbed_host_tests\host_tests_runner\host_test.py", line 216, in __init__
Test.__init__(self, options=options)
File "C:\Python27\lib\site-packages\mbed_host_tests\host_tests_runner\host_test.py", line 50, in __init__
self.mbed = Mbed(options)
File "C:\Python27\lib\site-packages\mbed_host_tests\host_tests_runner\mbed_base.py", line 50, in __init__
self.port = self.options.port
AttributeError: 'NoneType' object has no attribute 'port'
Most noticable on echo tests. Causes the echo tests to fail without a timeout of >2 minutes.
I found the 30s timeout to be too small for some grm as there is network delays and resource overheads.
Can it be at least 60s. Or better yet, configurable.
I am probably misusing htrun at the moment, but should not allow me to do this:
mbedhtrun -d E: -p COM64 -e test\host_tests\tcp_echo_client.py
HOST: Copy image onto target...
.\.gitignore
.\.yotta.json
.\module.json
.\README.md
4 file(s) copied.
HOST: Initialize serial port...
...port ready!
HOST: Reset target...
HOST: Detecting test case properties...
HOST: Error! No host test name defined in preamble
{{error}}
{{end}}
HOST: Passive mode...
mbedhtrun -p COM64 -e test\host_tests\tcp_echo_client.py
MBED: Instrumentation: "COM64" and disk: "None"
HOST: Copy image onto target...
New file 'mbedhtrun.py' introduces new command line for host test.
I am running mbedgt -V -n socket-test-test-tcp_client_echo_v0. The test timeouts, and I can still see socket open (listening), and python process is running even though test time-outed.
Using mbedhtrun 0.1.18, mbedgt 0.1.14. I created an issue here, but might be mbedgt ? Let me know what do you think
Hi. What is the right way to debug a test firmware (using mbedos test framework) inside a IDE (say KEIL MDK UVISION 5) while running mbed test procedure?
Is device supposed to "wait" for a sync packet through serial port? Say I would like to set breakpoints at start of test methods and run a "mbed test..." command hoping those breakpoints will be hit :)
Am I wrong or mbed test (i.e. mbedhtrun) should skip both flashing and resetting device...right?
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.