Giter Club home page Giter Club logo

opencsd's Introduction

OpenCSD - An open source CoreSight(tm) Trace Decode library {#mainpage}

This library provides an API suitable for the decode of ARM(r) CoreSight(tm) trace streams.

The library will decode formatted trace in three stages:

  1. Frame Deformatting : Removal CoreSight frame formatting from individual trace streams.
  2. Packet Processing : Separate individual trace streams into discrete packets.
  3. Packet Decode : Convert the packets into fully decoded trace describing the program flow on a core.

The library is implemented in C++ with an optional "C" API.

Library Versioning

From version 0.4, library versioning will use a semantic versioning format (per http://semver.org) of the form Major.minor.patch (M.m.p).

Internal library version calls, documentation and git repository will use this format moving forwards. Where a patch version is not quoted, or quoted as .x then comments will apply to the entire release.

Releases will be at M.m.0, with patch version incremented for bugfixes or documentation updates.

Releases will appear on the master branch in the git repository with an appropriate version tag.

CoreSight Trace Component Support.

Current Version 1.5.3

Current support:

  • ETE (v1.3) instruction trace - packet processing and packet decode.

  • ETMv4 (v4.6 [A/R profile] v4.4 [M profile]) instruction trace - packet processing and packet decode.

  • PTM (v1.1) instruction trace - packet processing and packet decode.

  • ETMv3 (v3.5) instruction trace - packet processing and packet decode.

  • ETMv3 (v3.5) data trace - packet processing.

  • STM (v1.1) software trace - packet processing and packet decode.

  • External Decoders - support for addition of external / custom decoders into the library.

Support to be added:

  • ITM software trace - packet processing and decode.
  • ETMv3 data trace - packet decode.
  • ETMv4 data trace - packet processing and decode.

Note: for ITM and STM, packet decode is combining Master+Channel+Marker+Payload packets into a single generic output packet.

Note on the Git Repository.

This git repository for OpenCSD contains only source for the OpenCSD decoder library. From version 0.4, releases appear as versioned tags on the master branch.

CoreSight kernel drivers and perf suport for CoreSight trace is maintained in the latest upstream kernel versions.

One exception is a minor patch required for autoFDO support. See [autofdo.md](@ref AutoFDO).

Documentation

API Documentation is provided inline in the source header files, which use the doxygen standard mark-up. Run doxygen on the ./doxygen_config.dox file located in the ./docs directory..

doxygen ./doxygen_config.dox

This will produce the documentation in the ./docs/html directory. The doxygen configuration also includes the *.md files as part of the documentation.

Application Programming using the Library

See the [programmers guide](@ref prog_guide) for details on usage of the library in custom applications. (./docs/prog_guide/prog_guide_main.md).

Building and Installing the Library

See [build_libs.md](@ref build_lib) in the ./docs directory for build details.

The linux build makefile now contains options to install the library for a linux environment.

How the Library is used in Linux perf

The library and additional infrastructure for programming CoreSight components has been integrated with the standard linux perfomance analysis tool perf.

See [HOWTO.md](@ref howto_perf) for details.

How to use the Library, perf and Trace for AutoFDO

Capturing trace using perf and decoding using the library can generate profiles for AutoFDO.

See [autofdo.md](@ref AutoFDO) for details and scripts.

(./tests/auto-fdo/autofdo.md).

Version and Modification Information

  • Version 0.001: Library development - tested with perf tools integration - BKK16, 8th March 2016

  • Version 0.002: Library development - added in PTM decoder support. Restructure header dir, replaced ARM rctdl prefix with opencsd/ocsd.

  • Version 0.003: Library development - added in ETMv3 instruction decoder support.

  • Version 0.4 : Library development - updated decode tree and C-API for generic decoder handling. Switch to semantic versioning.

  • Version 0.4.1: Minor Update & Bugfixes - fix to PTM decoder, ID checking on test program, adds NULL_TS support in STM packet processor.

  • Version 0.4.2: Minor Update - Update to documentation for perf usage in 4.8 kernel branch.

  • Version 0.5.0: Library Development - external decoder support. STM full decode.

  • Version 0.5.1: Minor Update & Bugfixes - Update HOWTO for kernel 4.9. Build fixes for parallel builds

  • Version 0.5.2: Minor Update & Bugfixes - Update trace info packet string o/p + Cycle count packet bugfixes.

  • Version 0.5.3: Doc update for using AutoFDO with ETM and additional timestamp and cycle count options.

  • Version 0.5.4: Updates: X-compile for arm/arm64. Remove deprecated VS2010 builds. Bugfix: GCC inline semantics in debug build.

  • Version 0.6.0: Packet printers moved from tests into the main library. C++ and C APIs updated to allow clients to use them. Update to allow perf to insert barrier packets (4xFSYNC) which the decoder can be made to use to reset the decode state.

  • Version 0.6.1: Bugfix: instruction follower bug on A32 branch to T32.

  • Version 0.7.0: Add handling for trace return stack feature to ETMv4 and PTM trace.

  • Version 0.7.1: Bugfix: ETMv3 packet processor.

  • Version 0.7.2: Bugfix: ETMv4 decoder - fix exact match packet address follower.

  • Version 0.7.3: Bugfix: PTM decoder - issues with initialisation and ASYNC detection.

  • Version 0.7.4: Notification of change of repository for perf extensions. gcc 6.x build fixes.

  • Version 0.7.5: Bugfix: ETMv4 decoder memory leak. Linux build update - header dependencies force rebuild.

  • Version 0.8.0: Header restructure and build update to enable linux version to install library and C-API headers in standard locations. Library output naming changed from 'cstraced' to 'opencsd'.

  • Version 0.8.1: Minor updates: Use install tool to copy headers. Changes to HOWTO for perf usage.

  • Version 0.8.2: Bugfix: C++ init errors fixed for CLANG build process.

  • Version 0.8.3: Bugfix: ETMv4 decoder issues fixed.

  • Version 0.8.4: build: makefile updates and improvements to get build process compatible with Debian packaging.

  • Version 0.9.0: Performance improvements for perf: Additional info in instruction range output packet. Caching memory accesses. Added Programmers guide to documentation.

  • Version 0.9.1: Bugfix: Crash during decode when first memory access is to address where no image provided.

  • Version 0.9.2: Bugfix: ETMv4: Incorrect Exception number output for Genric exception packets. AutoFDO: update documentation for AutoFDO usage and add in "record.sh" script

  • Version 0.9.3: Bugfix: Test snapshot library not handling 'offset' parameters in dump file sections. Install: ocsd_if_version.h moved to opencsd/include to allow installation on OS & use in compiling client apps.

  • Version 0.10.0:

    • Updates: Add additional information about the last instruction to the generic output packet.
    • Docs: update docs for updated output packet.
    • Bugfix: typecast removed from OCSD_VER_NUM in ocsd_if_version.h to allow use in C pre-processor.
    • Bugfix: ETMV4: Interworking ISA change between A32-T32 occasionally missed during instruction decode.

  • Version 0.10.1:

    • Updates: Build update - allow multi-thread make (make -j).
    • Docs: Minor update to AutoFDO documentation.

  • Version 0.11.0:

    • Update: ETM v4 decoder updated to support ETM version up to v4.4
    • Update: Memory access callback function - added new callback signature to provide TraceID to client when requesting memory.
    • Update: Created new example program to demonstrate using memory buffer in APIs.
    • Bugfix: Typos in docs and source.
    • Bugfix: Memory accessor - validate callback return values.

  • Version 0.11.1:

    • Update: build:- change -fpic to -fPIC to allow Debian build on sparc.
    • Bugfix: build:- remove unused variable

  • Version 0.11.2:

    • Update: docs:- HOWTO.md update to match new perf build requirements.
    • Bugfix: Minor spelling typos fixed.

  • Version 0.12.0:

    • Update: Frame deformatter - TPIU FSYNC and HSYNC support added.
    • Update: ETM v4: Bugfix & clarification on Exception trace handling. Where exception occurs at a branch target before any instructions have been executed, the preferred return address is also the target address of the branch instruction. This case now includes as specific flag in the packet. Additionally any context change associated with this target address was being applied incorrectly.
    • Update_: Core / Architecture mapping to core names as used by test programs / snapshots updated to include additional recent ARM cores.
    • Update: Docs: Update to reflect new exception flag. Update test program example to reflect latest output.
    • Bugfix: ETM v4: Valid trace info packet was not handled correctly (0x01, 0x00).
    • Bugfix: ETM v4: Error messaging on commit stack overflow.

  • Version 0.12.1:

    • Update: build: remove -g option from release build.
    • Update: tests: Snapshots can now use generic arch+profile names rather than core names, e.g. ARMv8-A
    • Bugfix: Instruction decode - v8.3 B[L]A{A|B}[Z] instructions mis-identified. -Bugfix: Transition from A64 to A32 can be mis-decoded if the trace implementation represents the transition as an individual address packet followed by a context packet.

  • Version 0.12.2:

    • Bugfix: Clean up memory leaks.
    • Bugfix: ETMv4: Ensure addressing history zeroed after TINFO.
    • Update: Allow GCC version to be included in build output path.
    • Bugfix: Packet printing update when WFI/WFE is P0 element.

  • Version 0.13.x : Intermediate development version.

  • Version 0.14.0:

    • Update: ETMv4 - decoder update & simplification to handle advanced trace features.
    • Update: ETMv4 - decoder support for speculative trace.
    • Update: Generic Elements: Additional information in EOT, UNSYNC, ON packets to give reason.
    • Update: Memaccess: Add EL2 secure memory space flag.
    • Update: Documentation: Updated for release changes and to reflect latest kernel version support for CoreSight.
    • Update: Perf helper scripts updated to reflect latest build flow.
    • Bugfix: Fix for component operational flag inputs.

  • Version 0.14.1:

    • Update: ETMv4 - Add support for Q elements.
    • Bugfix: build: fix logic issue for && operator. (github issue #23, sumitted by yabinc)

  • Version 0.14.2:

    • Update: Architecture versioning. Set enum tag values to make conversion to numeric version easier.
    • Update: I-decode: remove global temporary decode state data and replace with local instance data to make library more easily usable in multi-threaded programs.
    • Bugfix: I-decode: Some Thumb instructions not correctly reported as implied returns. (github issue #24, submitted by kongy).

  • Version 0.14.3:

    • Update: Fix makefile to be compliant with RPM base distros. (github issue #26, submitted by jlinton)
    • Update: Add section to autofdo document.
    • Bugfix: STM: fix bug that was missing ASYNC packets. (github issue #27, reported by subhasish Karmakar)

  • Version 0.14.4:

    • Update: makefile: Add DESTDIR to install targets. (github issue #30)
    • Update: tests: add script to run single test only.
    • Update: docs: update to location of ARM coresight driver backports directory.
    • Bugfix: ETMv3: Fix missing comma in string list. (github issue #31)
    • Bugfix: makefile: tests: Fix build race problem (github issue #32)
    • Bugfix: tests: fix ignore tpiu command line options (github issue #28)

  • Version 1.0.0:

    • New Decode Protocol: Support added for the ETE protocol, used by ARM PEs that implement the FEAT_ETE feature. Supports new architectural features in this trace, including FEAT_TME.
    • Update: Output Elememts: New protocol defines two new output elements.
    • Update: Add support for WFIT / WFET instructions traced as P0 elements.
    • Update: Architecture versioning. Arch v8 + PEs may add features in a flexible manner, and ARM also declares future features ahead of architecture versions to allow support to be added. APIs requiring an architecture version can now use ARCH_AA64 to declare a version of v8.3 + additional features. This relaxes the strict versionnig rules that the decoder uses when looking for Opcodes as trace waypoints.
    • Update: docs: Add linux 'man' file and installation.
    • Bugfix: build: Fix clean install, and remove static lib build test from main makefile to dev makefile only. (github issue #33)

  • Version 1.1.0:

    • Update: ETM v4.6 support.
    • Update: C-API - add API functions to get last error and convert error code to string.
    • Bugfix: ETMv4/ETE - fix loop problem in commit elements.
    • Bugfix: ETMv4/ETE - make error handling consistent.
    • Bugfix: Add Pull request #36 from github (Ross Burton)
    • Bugfix: Add Pull request #37 from github (Ian Rogers)

  • Version 1.1.1:

    • Bugfix: Fix include and install for ETE decoder headers.

  • Version 1.2.0:

    • Update: Add API for counting packet decode statistics, and Frame debmux statistics.
    • Update: Update test scripts to allow additional command line options to be passed.
    • Bugfix: Fix various build warnings.
    • Bugfix: Remove unused variable (github issue #38 from Yi Kong)
    • Bugfix: Remove noisy printf (James Clark)
    • Bugfix: Fix documentation issues (github issues #39 & #40 from rbresalier)

  • Version 1.2.1:

    • Bugfix: ETM4x / ETE - output of context elements to client can in some circumstances be delayed until after subsequent atoms have been processed leading to incorrect memory decode access via the client callbacks. Fixed to flush context elements immediately they are committed.

  • Version 1.3.0:

    • Admin: Dev versions now have patch versions at least +100 from root public version
    • Update: Add support for conditional branch (BC.cond) introduced for v8.8 / v9.3 architecture.
    • Update: ETE: Add support for NSE bit - security state bit defining Root / Realm states in FEAT_RME.

  • Version 1.3.1:

    • Bugfix: Add header file in snapshot parser - fix build for certain libc++ libs (github issue #43 from manojgupta)
    • Bugfix: Fix typo in comment (github issue #42 from nothatDinger)

  • Version 1.3.2:

    • Bugfix: ETM4x / ETE - 64 bit timestamp value - MS bit incorrectly masked to 1b0 during extraction from packet.

  • Version 1.3.3:

    • Update: Add build directory for VS2022 build.
    • Update: Add test program for Coresight Frame Demux code
    • Bugfix: PTM: Fix incorrect extraction of Waypoint Address packet (github issue #48)
    • Bugfix: Frame Demux: Fix HSYNC, FSYNC and 4xFSYNC handling that was causing out-of-bounds reads on invalid data input. (issues #49, #50 and #51). Fixed error handling for incorrect number of FSYNC packets in 4xFSYNC frame reset code that was not triggering an error and allowing fun with mis-aligned input data.
    • Bugfix: Fix silent failure if incorrect config flags set when setting up frame demux modes.

  • Version 1.4.0:

    • Update: ETE: Add support for Arch v9.4 FEAT_ITE. ETE v1p3, sw trace instrumentation. Adds in new generic output packet type: OCSD_GEN_TRC_ELEM_INSTRUMENTATION.
    • Bugfix: Fix memory leak in mispredict handling (github issue #52 from yabinc)

  • Version 1.4.1:

    • Update: Test: Update test program trc_pkt_lister with option to have no logging output - final stats only.
    • Bugfix: ETM4x: ETE: Fix memory leak / use after free (github issues #53, #58 from yabinc)
    • Bugfix: C-API: Fix memory leak with output sink object. (github issue #55)

  • Version 1.4.2:

    • Update: Decoder memory access - add debug features and cacheing to improve performance of decode. Caching controllable by API or environment variables.

    • Update: Decode debug options to control potential runaway debug when incorrect memory images input to decoder.

               - aarch64 instruction check for invalid instruction opcode. (any opcode with 0x0000 as top
            16 bits).
            
         - optional instruction range maximum run limit. Set by environment variable.

    • Update: Remove VS2015 support.

    • Bugfix: Memory spaces: ensure that decoder memory requests are correctly labelled with Realm and root characteristics when appropriate.

    • Bugfix: M class tail chained interrupts now correctly handled.

    • Bugfix: STM: - fix master and channel ID initialisation (github issue #61 from MommeSherif)

    • Bugfix: Test: update HOWTO.md and perf-test-script.base for new paths (github issue #60 from yuxy-c)

    • Bugfix: build: Fix GCC version dir handling for versions of GCC > 9.

  • Version 1.5.1:

    • Update: ETE: Add support for Arch 9.5 FEAT_PAUTH_LR. Add new instructions to disassembler to recognise RETASPPC variants as indirect branches. Adds new test snapshots.

  • Version 1.5.2:

    • Update: docs: Update documentation for test programs.
    • Update: docs: Update trc_pkt_lister man file.
    • Bugfix: build: fix warnings building library.
    • Bugfix: tests: Fix typo in trc_pkt_lister help output.

  • Version 1.5.3:

    • Update: tests: Add timing information to trc_pkt_lister program.
    • Update: memacc: Add external interface to set memacc cache parameters. Update test programs.
    • Bugfix: etm4x: fix packet print typo.
    • Bugfix: docs: Minor fixes to docs and man file.
    • Bugfix: build: minor fix for clang compatibility.

Licence Information

This library is licensed under the BSD three clause licence.

A copy of this license is in the LICENCE file included with the source code.

Contact

Using the github site: https://github.com/Linaro/OpenCSD

Mailing list: [email protected]

opencsd's People

Contributors

andrea-brunato-arm avatar bukinr avatar captain5050 avatar james-a-clark avatar jlintonarm avatar kim-phillips-arm avatar kongy avatar m-gupta avatar mathieupoirier avatar mikel-armbb avatar nothatdinger avatar ototo avatar rhys-arm avatar rossburton avatar rwalkr avatar tor-jeremiassen avatar vwax avatar yuxy-c avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

tuxology bobsense vitkyrka sebpop henryfeng2016 gburgessiv kimphill lvxiaoliang8 bukinr kenkuang cherian42 ptsneves leo-yan youtubezou solomonbstoner tonezhang01 jeremyng123 seuwangtao sehirst abisco5 kongy zxombie abhishpaliwal jlinton albertschulz mfkiwl rossburton tangshiaw softrobotycli captain5050 zenhumany arduinohocam musicboxes silentcedar odie8683 sandeepchandran-iitpkd ifquant nothatdinger m-gupta holaymzhang ashlingmicrosystems joseph2liu rhys-arm gjhan3 jowinter bdankwa xiaoxiang781216 mommesherif rtsecure hokagem

opencsd's Issues

opencsd and linux kernel version

Hello,

I have an arm64 based board and was successfully ran coresight tracing on on some Linux backport from Linaro for 4.9. Right now it seems things get migrated to upstream Linux and I am no longer able to collect perf data.

My setup looks like 5.4.78 Linux in combination with Opencsd v0.14.4.

$ perf record -e cs_etm/@tmc_etr0/ --per-thread uname
Couldn't synthesize bpf events.
Linux
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.096 MB perf.data ]

$ perf report --stdio
Error:
The perf.data data has no samples!

Question would be if there is some specific version of Linux/perf required to run this out of the box?
There is also this error message "Couldn't synthesize bpf events." which confuses me as well..

Thank you!

Use after free in TrcPktDecodeEtmV4I::mispredictAtom

The problem was reported in Android, but not reproduced by real ETM data.

In TrcPktDecodeEtmV4I::mispredictAtom, there is a loop calling m_P0_stack.from_front_next() and m_P0_stack.erase_curr_from_front(). In m_P0_stack, it uses m_iter to point to the next TrcStackElem to read.
The problem happens in below case:

  1. pElem points to the last element in m_P0_stack, m_P0_stack.m_iter == m_P0_stack.m_P0_stack.end().
  2. pElem->getP0Type() == P0_ADDR.

Since pElem->getP0Type() == P0_ADDR, m_P0_stack.erase_curr_from_front() is called. After the call,
m_P0_stack.m_iter points to one position past m_P0_stack.m_P0_stack.end(). Then *m_iter is read in m_P0_stack.from_front_next().

From https://en.cppreference.com/w/cpp/container/deque/erase: All iterators and references are invalidated after erase.
So a fix suggestion is:
m_iter = m_P0_stack.erase(erase_iter);

Return the address that cannot be accessed in the mapped memory images

Hi
I use OpenCSD with perf to trace the Java program. I want to know the branch address and target address in the Java thread even if I can not get the symbol corresponding to these addresses, I find that the perf cs_etm session only decodes the address that can be accessed in the mapped memory of shared library. I read the source code and found the root cause in OpenCSD, these address belongs to packets which are defined as OCSD_GEN_TRC_ELEM_ADDR_NACC.
Can OpenCSD still decode when OCSD_GEN_TRC_ELEM_ADDR_NACC occurred, and return the element that includes OCSD_GEN_TRC_ELEM_ADDR_NACC packet with unaccessed address to the application? This will be very useful for Java programs.

trc_pkt_lister -decode hangs on Snowball sample snapshot

trc_pkt_lister appears to hang when run on the Snowball snapshot with the -decode option.

The code was built with LINUX64=1 DEBUG=1.

Trace Packet Lister: CS Decode library testing
-----------------------------------------------

** Library Version : 0.4.0

Test Command Line:-
./trc_pkt_lister   -ss_dir  ../../../snapshots/Snowball/  -decode  -o_raw_packed  -ss_verbose  

Trace Packet Lister : reading snapshot from path ../../../snapshots/Snowball/
snapshot_parser : Start of snapshot section
snapshot_parser : Start of device_list section
snapshot_parser : Start of trace section
snapshot_reader : Parsed snapshot.ini.
Found 4 devices.
snapshot_reader : Device 1: Ini file = cpu_0.ini; Name = UNKNOWN
snapshot_parser : Start of device section
snapshot_parser : Start of regs section
snapshot_parser : Start of dump section
snapshot_reader : Device 2: Ini file = cpu_1.ini; Name = UNKNOWN
snapshot_parser : Start of device section
snapshot_parser : Start of regs section
snapshot_parser : Start of dump section
snapshot_reader : Device 3: Ini file = device_2.ini; Name = UNKNOWN
snapshot_parser : Start of device section
snapshot_parser : Start of regs section
snapshot_reader : Device 4: Ini file = device_3.ini; Name = UNKNOWN
snapshot_parser : Start of device section
snapshot_parser : Start of regs section
snapshot_reader : Trace Metadata ini file found : trace.ini
snapshot_parser : Start of trace_buffers section
snapshot_parser : Start of buffer0 section
snapshot_parser : Start of source_buffers section
snapshot_parser : Start of core_trace_sources section
snapshot_reader : Done.
Using ETB_0 as trace source
Trace Packet Lister : PTM Protocol on Trace ID 0x10
Trace Packet Lister : PTM Protocol on Trace ID 0x11
Trace Packet Lister : Set trace element decode printer
Gen_Info : Mapped Memory Accessors
Gen_Info : FileAcc; Range::0xc0008000:c0057fff; Mem Space::Any
Filename=../../../snapshots/Snowball/kernel_dump.bin
Gen_Info : ========================
Frame Data; Index      0;    RAW_PACKED; 60 03 d2 d8 24 14 08 dc 48 05 c0 29 c0 06 b4 12 
Frame Data; Index     16;    RAW_PACKED; 08 fc 48 05 c0 29 e0 04 9c 99 0a 64 02 d3 d8 42 
Frame Data; Index     32;    RAW_PACKED; 24 14 08 dc 48 05 c0 29 c0 06 b4 08 fc 49 04 84 
Frame Data; Index     48;    RAW_PACKED; c0 29 f0 05 9c 99 0a 64 02 d3 d8 24 14 08 40 10 
Frame Data; Index     64;    RAW_PACKED; 00 01 c0 29 d4 16 ee 01 00 00 00 00 00 80 d6 81 
Frame Data; Index     80;    RAW_PACKED; 9c e6 b2 1b 78 01 08 d7 8e 79 b6 09 42 97 94 83 
Frame Data; Index     96;    RAW_PACKED; ba c8 8e 8b 00 0c 08 c0 02 01 c0 29 c4 80 00 81 
Frame Data; Index    112;    RAW_PACKED; 21 f8 04 86 90 ac 0a 20 ce 02 86 ca 02 86 fe 87 
Idx:112; ID:10; [0x05 0x86 0x91 0xac 0x0a 0x20 0xce 0x02 0x86 0xca 0x02 0x86 0xff ];    NOTSYNC : PTM Not Synchronised; 
Idx:112; ID:10; OCSD_GEN_TRC_ELEM_NO_SYNC()
Frame Data; Index    128;    RAW_PACKED; 26 2c c0 02 08 28 62 03 c0 29 dc 20 aa 92 84 08 
Idx:128; ID:10; [0x26 0x2c 0xc0 0x02 0x08 0x28 0x63 0x03 0xc0 0x29 0xdc 0x20 0xaa 0x92 0x84 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    144;    RAW_PACKED; a6 ea 0e 48 00 8d a6 0a 4c 01 c0 2c 3c ae a4 2d 
Idx:144; ID:10; [0xa7 0xea 0x0e 0x48 0x01 0x8d 0xa7 0x0a 0x4c 0x01 0xc1 0x2c 0x3c 0xae 0xa4 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    160;    RAW_PACKED; ac 08 f4 2c 04 c0 28 d0 06 b4 a0 81 02 44 00 bc 
Idx:160; ID:10; [0xac 0x08 0xf4 0x2c 0x05 0xc0 0x29 0xd0 0x07 0xb4 0xa1 0x81 0x02 0x44 0x01 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    176;    RAW_PACKED; fe 01 b6 b0 8e b2 1a 78 00 08 20 ff 00 c0 28 9e 
Idx:176; ID:10; [0xfe 0x01 0xb7 0xb0 0x8f 0xb2 0x1b 0x78 0x01 0x08 0x20 0xff 0x00 0xc0 0x29 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    192;    RAW_PACKED; d0 20 ca 05 be 88 c6 02 88 c2 02 88 9c b2 92 80 
Idx:192; ID:10; [0xd0 0x20 0xca 0x05 0xbe 0x88 0xc6 0x02 0x88 0xc2 0x02 0x88 0x9c 0xb2 0x93 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    208;    RAW_PACKED; 80 02 2c 90 8a a1 00 74 04 c2 00 f6 00 84 ae 6c 
Idx:208; ID:10; [0x80 0x02 0x2c 0x90 0x8b 0xa1 0x01 0x74 0x04 0xc2 0x01 0xf6 0x01 0x84 0xae ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    224;    RAW_PACKED; ec 80 02 50 00 e4 00 c2 00 cb e6 89 80 10 7c 1d 
Idx:224; ID:10; [0xed 0x80 0x02 0x50 0x01 0xe4 0x01 0xc2 0x01 0xcb 0xe6 0x89 0x80 0x10 0x7c ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    240;    RAW_PACKED; 00 08 c0 02 00 c0 28 c8 f4 01 21 ec 02 86 90 cd 
Idx:240; ID:10; [0x01 0x08 0xc0 0x02 0x01 0xc0 0x29 0xc8 0xf4 0x01 0xec 0x03 0x86 0x91 ];   NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    256;    RAW_PACKED; ec 06 18 c2 02 88 08 50 6c 03 c0 29 dc 1a cc 00 
Idx:256; ID:10; [0xec 0x06 0x18 0xc2 0x02 0x88 0x08 0x50 0x6c 0x03 0xc0 0x29 0xdc 0x1a 0xcc ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    272;    RAW_PACKED; 02 b4 08 8c 6c 03 c0 29 cc 04 86 86 a0 81 02 40 
Idx:272; ID:10; [0x02 0xb4 0x08 0x8c 0x6c 0x03 0xc0 0x29 0xcc 0x04 0x86 0x86 0xa1 0x81 0x02 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    288;    RAW_PACKED; 14 c2 02 b7 c4 fc d8 0d 78 01 08 c0 02 01 c0 08 
Idx:288; ID:10; [0x14 0xc2 0x02 0xb7 0xc4 0xfc 0xd9 0x0d 0x78 0x01 0x08 0xc0 0x02 0x01 0xc0 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    304;    RAW_PACKED; 28 d0 08 cc 02 ae 90 ec 06 30 9e 84 08 50 6c 0f 
Idx:304; ID:10; [0x29 0xd0 0x09 0xcc 0x03 0xae 0x91 0xec 0x06 0x30 0x9e 0x84 0x08 0x50 0x6c ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    320;    RAW_PACKED; 02 c0 28 e8 14 a4 84 08 8c 6c 02 c0 28 c4 02 63 
Idx:320; ID:10; [0x03 0xc0 0x29 0xe8 0x14 0xa4 0x84 0x08 0x8c 0x6c 0x03 0xc0 0x29 0xc4 0x02 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    336;    RAW_PACKED; 86 86 a0 81 02 1c ca 01 b6 c4 fc d9 0c 78 00 d2 
Idx:336; ID:10; [0x86 0x86 0xa1 0x81 0x02 0x1c 0xca 0x01 0xb7 0xc4 0xfc 0xd9 0x0d 0x78 0x01 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    352;    RAW_PACKED; 08 c0 02 01 c0 29 ec d9 00 f8 02 c2 00 d1 b4 70 
Idx:352; ID:10; [0x08 0xc0 0x02 0x01 0xc0 0x29 0xec 0xd9 0x01 0xf8 0x03 0xc2 0x01 0xd1 0xb4 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    368;    RAW_PACKED; 22 30 08 58 48 05 c0 29 d0 47 21 f8 00 08 8c 64 
Idx:368; ID:10; [0x22 0x30 0x08 0x58 0x49 0x05 0xc0 0x29 0xd0 0x47 0xf8 0x01 0x08 0x8c ];   NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    384;    RAW_PACKED; 48 05 c0 29 e4 07 88 cc 00 e4 00 5f 64 04 fe b1 
Idx:384; ID:10; [0x49 0x05 0xc0 0x29 0xe4 0x07 0x88 0xcc 0x01 0xe4 0x01 0x5f 0x64 0x04 0xff ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    400;    RAW_PACKED; da 2a 18 08 50 48 04 c0 28 f4 10 e2 08 84 08 78 
Idx:400; ID:10; [0xda 0x2a 0x18 0x08 0x50 0x48 0x05 0xc0 0x29 0xf4 0x11 0xe2 0x09 0x84 0x08 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    416;    RAW_PACKED; 9c 48 04 c0 28 f4 04 9c 60 74 02 c3 dc 2a 18 76 
Idx:416; ID:10; [0x9c 0x48 0x05 0xc0 0x29 0xf4 0x04 0x9c 0x61 0x74 0x03 0xc3 0xdd 0x2a 0x18 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    432;    RAW_PACKED; 08 40 00 01 c0 29 fc 0c ca 01 ba 9a b4 da 0c a2 
Idx:432; ID:10; [0x08 0x40 0x01 0x01 0xc0 0x29 0xfc 0x0c 0xca 0x01 0xbb 0x9a 0xb4 0xda 0x0d ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    448;    RAW_PACKED; 78 01 08 c0 02 01 c0 29 f0 ee 00 ec 02 ae d0 e0 
Idx:448; ID:10; [0x78 0x01 0x08 0xc0 0x02 0x01 0xc0 0x29 0xf0 0xee 0x01 0xec 0x03 0xae 0xd1 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    464;    RAW_PACKED; b4 22 30 08 6c fd 04 c0 28 dc 78 e1 fc b6 00 d0 
Idx:464; ID:10; [0xb4 0x22 0x30 0x08 0x6c 0xfd 0x04 0xc0 0x29 0xdc 0x78 0xe1 0xfd 0xb6 0x01 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    480;    RAW_PACKED; 50 03 08 40 00 01 c0 29 c8 0f ce 01 ba 9a b4 44 
Idx:480; ID:10; [0x50 0x03 0x08 0x40 0x01 0x01 0xc0 0x29 0xc8 0x0f 0xce 0x01 0xbb 0x9a 0xb4 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    496;    RAW_PACKED; da 0d 78 01 21 08 c0 02 00 c0 28 f0 fa 02 f0 30 
Idx:496; ID:10; [0xda 0x0d 0x78 0x01 0x08 0xc0 0x02 0x01 0xc0 0x29 0xf0 0xfa 0x02 0xf0 ];   NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    512;    RAW_PACKED; 02 86 d0 b4 22 18 08 6c fc 04 c0 29 ec 59 e0 93 
Idx:512; ID:10; [0x03 0x86 0xd1 0xb4 0x22 0x18 0x08 0x6c 0xfd 0x04 0xc0 0x29 0xec 0x59 0xe1 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    528;    RAW_PACKED; fc b6 00 44 00 08 40 01 00 c0 28 dc 0a ce 00 f7 
Idx:528; ID:10; [0xfd 0xb6 0x01 0x44 0x01 0x08 0x40 0x01 0x01 0xc0 0x29 0xdc 0x0b 0xce 0x01 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    544;    RAW_PACKED; ba 9a b4 da 0c 78 00 08 c0 02 00 c0 28 dc a0 ed 
Idx:544; ID:10; [0xbb 0x9a 0xb4 0xda 0x0d 0x78 0x01 0x08 0xc0 0x02 0x01 0xc0 0x29 0xdc 0xa1 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    560;    RAW_PACKED; 02 cc 04 86 d0 b4 22 18 08 6c fc 04 c0 29 c4 24 
Idx:560; ID:10; [0x02 0xcc 0x04 0x86 0xd1 0xb4 0x22 0x18 0x08 0x6c 0xfd 0x04 0xc0 0x29 0xc4 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    576;    RAW_PACKED; 4e e1 fc b6 00 44 00 08 40 01 00 c0 28 d8 0a 6e 
Idx:576; ID:10; [0x4e 0xe1 0xfd 0xb6 0x01 0x44 0x01 0x08 0x40 0x01 0x01 0xc0 0x29 0xd8 0x0a ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    592;    RAW_PACKED; ce 01 ba 9a b4 da 0c 78 00 08 c0 02 00 c0 28 da 
Idx:592; ID:10; [0xce 0x01 0xbb 0x9a 0xb4 0xda 0x0d 0x78 0x01 0x08 0xc0 0x02 0x01 0xc0 0x29 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    608;    RAW_PACKED; f0 f7 00 c4 04 86 90 ac 0a 18 da 01 86 c2 02 0a 
Idx:608; ID:10; [0xf0 0xf7 0x01 0xc4 0x04 0x86 0x91 0xac 0x0a 0x18 0xda 0x01 0x86 0xc2 0x02 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    624;    RAW_PACKED; 21 86 fe 26 04 90 08 28 62 03 c0 29 ec 17 fa 13 
Idx:624; ID:10; [0x86 0xff 0x26 0x04 0x90 0x08 0x28 0x63 0x03 0xc0 0x29 0xec 0x17 0xfa ];   NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    640;    RAW_PACKED; 00 92 88 a7 ea 0e 58 01 8c a7 0a 4c 00 c1 2c 51 
Idx:640; ID:10; [0x01 0x92 0x88 0xa7 0xea 0x0e 0x58 0x01 0x8d 0xa7 0x0a 0x4c 0x01 0xc1 0x2c ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    656;    RAW_PACKED; 38 b6 a4 ac 08 f4 2c 05 c0 29 fc 05 84 a1 80 80 
Idx:656; ID:10; [0x38 0xb6 0xa4 0xac 0x08 0xf4 0x2c 0x05 0xc0 0x29 0xfc 0x05 0x84 0xa1 0x81 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    672;    RAW_PACKED; 02 4c 00 ca 00 b7 b0 8f b2 1b 78 01 08 c0 02 06 
Idx:672; ID:10; [0x02 0x4c 0x01 0xca 0x01 0xb7 0xb0 0x8f 0xb2 0x1b 0x78 0x01 0x08 0xc0 0x02 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    688;    RAW_PACKED; 00 c0 28 dc c4 01 c4 04 ae e9 e4 24 30 08 40 07 
Idx:688; ID:10; [0x01 0xc0 0x29 0xdc 0xc5 0x01 0xc4 0x04 0xae 0xe9 0xe4 0x24 0x30 0x08 0x40 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    704;    RAW_PACKED; 00 01 c0 29 e4 d4 02 ca 02 d7 9c e6 b2 1b 78 61 
Idx:704; ID:10; [0x01 0x01 0xc0 0x29 0xe4 0xd4 0x02 0xca 0x02 0xd7 0x9d 0xe6 0xb3 0x1b 0x78 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    720;    RAW_PACKED; 00 08 c0 02 00 c0 28 dc f0 04 e8 05 86 d1 b4 0d 
Idx:720; ID:10; [0x01 0x08 0xc0 0x02 0x01 0xc0 0x29 0xdc 0xf0 0x04 0xe8 0x05 0x86 0xd1 0xb4 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    736;    RAW_PACKED; 22 04 08 6c fc 04 c0 29 f0 63 e0 fd b6 01 48 24 
Idx:736; ID:10; [0x22 0x04 0x08 0x6c 0xfd 0x04 0xc0 0x29 0xf0 0x63 0xe1 0xfd 0xb6 0x01 0x48 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    752;    RAW_PACKED; 00 08 40 01 00 c0 28 dc 0e ca 21 01 ba 9a b4 5d 
Idx:752; ID:10; [0x01 0x08 0x40 0x01 0x01 0xc0 0x29 0xdc 0x0f 0xca 0x01 0xbb 0x9a 0xb4 ];   NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    768;    RAW_PACKED; da 0d 78 01 08 c0 02 01 c0 29 e4 b8 02 c4 04 00 
Idx:768; ID:10; [0xda 0x0d 0x78 0x01 0x08 0xc0 0x02 0x01 0xc0 0x29 0xe4 0xb8 0x02 0xc4 0x04 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    784;    RAW_PACKED; 86 d1 b4 22 18 08 6c fd 04 c0 28 d8 56 e1 fc a0 
Idx:784; ID:10; [0x86 0xd1 0xb4 0x22 0x18 0x08 0x6c 0xfd 0x04 0xc0 0x29 0xd8 0x56 0xe1 0xfd ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    800;    RAW_PACKED; b6 01 44 01 08 40 00 01 c0 29 fc 0a ce 01 ba 88 
Idx:800; ID:10; [0xb6 0x01 0x44 0x01 0x08 0x40 0x01 0x01 0xc0 0x29 0xfc 0x0a 0xce 0x01 0xbb ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    816;    RAW_PACKED; 9a b4 da 0d 78 01 08 c0 02 01 c0 29 d4 8d 02 00 
Idx:816; ID:10; [0x9a 0xb4 0xda 0x0d 0x78 0x01 0x08 0xc0 0x02 0x01 0xc0 0x29 0xd4 0x8d 0x02 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    832;    RAW_PACKED; cc 04 86 d1 b4 22 18 08 6c fd 04 c0 28 ec 48 c0 
Idx:832; ID:10; [0xcc 0x04 0x86 0xd1 0xb4 0x22 0x18 0x08 0x6c 0xfd 0x04 0xc0 0x29 0xec 0x49 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    848;    RAW_PACKED; e0 fd b6 01 40 01 08 40 00 01 c0 29 dc 09 ca 11 
Idx:848; ID:10; [0xe1 0xfd 0xb6 0x01 0x40 0x01 0x08 0x40 0x01 0x01 0xc0 0x29 0xdc 0x09 0xca ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    864;    RAW_PACKED; 00 bb 9a b4 da 0d 78 01 08 c0 02 01 c0 29 f4 01 
Idx:864; ID:10; [0x01 0xbb 0x9a 0xb4 0xda 0x0d 0x78 0x01 0x08 0xc0 0x02 0x01 0xc0 0x29 0xf4 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    880;    RAW_PACKED; dc 01 c4 04 86 91 21 ac 0a 18 da 01 86 b2 86 09 
Idx:880; ID:10; [0xdd 0x01 0xc4 0x04 0x86 0x91 0xac 0x0a 0x18 0xda 0x01 0x86 0xb2 0x86 ];   NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    896;    RAW_PACKED; fe 26 04 a4 08 28 62 03 c0 29 dc 15 fa 01 92 09 
Idx:896; ID:10; [0xff 0x26 0x04 0xa4 0x08 0x28 0x63 0x03 0xc0 0x29 0xdc 0x15 0xfa 0x01 0x92 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    912;    RAW_PACKED; 88 a7 ea 0e 44 01 8c a7 0a 4c 00 c1 2c 38 86 28 
Idx:912; ID:10; [0x88 0xa7 0xea 0x0e 0x44 0x01 0x8d 0xa7 0x0a 0x4c 0x01 0xc1 0x2c 0x38 0x86 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    928;    RAW_PACKED; b8 b0 08 f4 2c 05 c0 29 d0 07 84 a1 80 02 18 40 
Idx:928; ID:10; [0xb8 0xb0 0x08 0xf4 0x2c 0x05 0xc0 0x29 0xd0 0x07 0x84 0xa1 0x81 0x02 0x18 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    944;    RAW_PACKED; ce 01 b6 b0 8e b2 1a 78 00 08 c0 02 00 c0 28 de 
Idx:944; ID:10; [0xce 0x01 0xb7 0xb0 0x8f 0xb2 0x1b 0x78 0x01 0x08 0xc0 0x02 0x01 0xc0 0x29 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    960;    RAW_PACKED; ec 92 00 f8 02 86 e8 e4 24 18 08 40 00 01 c0 4e 
Idx:960; ID:10; [0xec 0x92 0x01 0xf8 0x03 0x86 0xe9 0xe4 0x24 0x18 0x08 0x40 0x01 0x01 0xc0 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    976;    RAW_PACKED; 28 cc bc 02 ca 01 d6 9d e6 b3 1a 78 00 08 c0 6b 
Idx:976; ID:10; [0x29 0xcc 0xbd 0x02 0xca 0x01 0xd7 0x9d 0xe6 0xb3 0x1b 0x78 0x01 0x08 0xc0 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index    992;    RAW_PACKED; 02 01 c0 29 d8 a7 04 c8 04 ce 00 d1 b4 22 34 28 
Idx:992; ID:10; [0x02 0x01 0xc0 0x29 0xd8 0xa7 0x05 0xc8 0x04 0xce 0x01 0xd1 0xb4 0x22 0x34 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index   1008;    RAW_PACKED; 21 08 6c fd 04 c0 28 ec 66 e1 fc b6 00 40 00 e8 
Idx:1008; ID:10; [0x08 0x6c 0xfd 0x04 0xc0 0x29 0xec 0x66 0xe1 0xfd 0xb6 0x01 0x40 0x01 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index   1024;    RAW_PACKED; 08 40 00 01 c0 29 d4 0f fe 01 ba 9a b4 da 0c a2 
Idx:1024; ID:10; [0x08 0x40 0x01 0x01 0xc0 0x29 0xd4 0x0f 0xfe 0x01 0xbb 0x9a 0xb4 0xda 0x0d ]; NOTSYNC : PTM Not Synchronised; 
Frame Data; Index   1040;    RAW_PACKED; 78 01 08 c0 02 01 c0 29 fc b1 02 c8 04 c2 00 80 
Idx:1040; ID:10; [0x78 0x01 0x08 0xc0 0x02 0x01 0xc0 0x29 0xfc 0xb1 0x02 0xc8 0x04 0xc2 0x01 ]; NOTSYNC : PTM Not Synchronised; 
Frame Data; Index   1056;    RAW_PACKED; d0 b4 22 30 08 6c fc 04 c0 29 e4 56 e0 fd b6 49 
Idx:1056; ID:10; [0xd1 0xb4 0x22 0x30 0x08 0x6c 0xfd 0x04 0xc0 0x29 0xe4 0x56 0xe1 0xfd 0xb6 ]; NOTSYNC : PTM Not Synchronised; 
Frame Data; Index   1072;    RAW_PACKED; 00 44 00 08 40 01 00 c0 28 e8 08 ce 00 bb 9a 5b 
Idx:1072; ID:10; [0x01 0x44 0x01 0x08 0x40 0x01 0x01 0xc0 0x29 0xe8 0x08 0xce 0x01 0xbb 0x9a ]; NOTSYNC : PTM Not Synchronised; 
Frame Data; Index   1088;    RAW_PACKED; b4 da 0c 78 00 08 c0 02 00 c0 28 c0 fe 01 c4 76 
Idx:1088; ID:10; [0xb4 0xda 0x0d 0x78 0x01 0x08 0xc0 0x02 0x01 0xc0 0x29 0xc0 0xff 0x01 0xc4 ]; NOTSYNC : PTM Not Synchronised; 
Frame Data; Index   1104;    RAW_PACKED; 04 86 d0 b4 22 18 08 6c fc 04 c0 29 f0 4e e0 92 
Idx:1104; ID:10; [0x04 0x86 0xd1 0xb4 0x22 0x18 0x08 0x6c 0xfd 0x04 0xc0 0x29 0xf0 0x4e 0xe1 ]; NOTSYNC : PTM Not Synchronised; 
Frame Data; Index   1120;    RAW_PACKED; fc b6 00 40 00 08 40 01 00 c0 28 d4 08 ca 00 b7 
Idx:1120; ID:10; [0xfd 0xb6 0x01 0x40 0x01 0x08 0x40 0x01 0x01 0xc0 0x29 0xd4 0x08 0xca 0x01 ]; NOTSYNC : PTM Not Synchronised; 
Frame Data; Index   1136;    RAW_PACKED; ba 9a b4 da 21 0d 78 01 08 c0 02 01 c0 29 f4 05 
Idx:1136; ID:10; [0xbb 0x9a 0xb4 0xda 0x0d 0x78 0x01 0x08 0xc0 0x02 0x01 0xc0 0x29 0xf4 ];  NOTSYNC : PTM Not Synchronised; 
Frame Data; Index   1152;    RAW_PACKED; d6 01 d8 05 86 91 ac 0a 18 da 00 86 00 00 00 20 
Idx:1152; ID:10; [0xd6 0x01 0xd8 0x05 0x86 0x91 0xac 0x0a 0x18 0xda 0x01 0x86 ];    NOTSYNC : PTM Not Synchronised; 
Frame Data; Index   1168;    RAW_PACKED; 00 00 80 be 86 ff 26 04 08 fc 26 05 c0 09 42 00 
Idx:1164; ID:10; [0x00 0x00 0x00 0x00 0x00 0x80 ];  ASYNC : Alignment Synchronisation Packet; 
Idx:1171; ID:10; [0xbe ];   ATOM : Atom packet; N; Cycles=15;

At this point it hangs for several minutes with 100% CPU usage.

ARM traces with speculation information

ETMv4 supports giving speculative information for program flow, but most implementation does not seem to support that, nor any traces containing these are existent in this repository. Do you know about implementations / architectures supporting that so we could enrich the testsuites?

Support DESTDIR in install targets

The idiom for install rules (as per automake, cmake, meson, etc) is that you can set PREFIX=/usr and DESTDIR=/my/build/tree, so that you can install to /my/build/tree/usr/ for packaging.

OpenCSD doesn't support this and currently PREFIX has to be manually extended with the DESTDIR. The moment the build wants to embed the prefix into the binaries, this workaround will fail.

Simply adding $(DESTDIR) before the target paths in the install rule will solve this.

exceptions happen when collect trace on target

Thanks for your time! First, I only set one sink: etb.

-bash-4.3# ls /sys/bus/coresight/devices/
44802000.funnel 44803000.etb 44c40000.etm replicator
-bash-4.3# cat /sys/bus/coresight/devices/44803000.etb/enable_sink
1
-bash-4.3# uname -a
Linux localhost 4.7.0-rc2 #3 SMP PREEMPT Thu Jul 14 16:44:16 CST 2016 aarch64 aarch64 aarch64 GNU/Linux

When I intend to record trace data from coresight, I get the exceptions as follows.

-bash-4.3# ./tools/perf/perf record -C 0 -e cs_etm/@sink=44803000.etb/ --per-thread uname
Warning: SYSTEM/CPU switch overriding PER-THREAD
Error: failed to set config "sink=44803000.etb" on event cs_etm/@sink=44803000.etb/ with 25 (Inappropriate ioctl for device)

makefile is not tolerant of a path with spaces (OCSD_ROOT/awk script need quotes)

Having OpenCSD source code in a file location with spaces in the path causes the OCSD_ROOT assignment and the awk script that uses it to mess up - creating folders outside the OpenCSD folder path or inside builddir.

Reproduce:

$ cd "/mnt/test/opencsd checkout/build/linux"
$ make
awk: warning: command line argument /mnt/test/' is a directory: skipped awk: fatal: cannot open file checkout/decoder/include/opencsd/ocsd_if_version.h' for reading (No such file or directory)
awk: warning: command line argument /mnt/test/' is a directory: skipped awk: fatal: cannot open file checkout/decoder/include/opencsd/ocsd_if_version.h' for reading (No such file or directory)
awk: warning: command line argument /mnt/test/' is a directory: skipped awk: fatal: cannot open file checkout/decoder/include/opencsd/ocsd_if_version.h' for reading (No such file or directory)

Result:

  • Build failure (as above)
  • There's a series of folders created above 'opencsd checkout' called (space in front) ' checkout/decoder/lib/builddir' et al.

I tried adding quotes as appropriate but never found the right combination in the right places.

perf trace data decoding failed

Hi,

I compile perf with OpenCSD and run coresight tracing, but it seemed some error happened.
Can you point out where I did wrong?

perf cmd get perf data

./tools/perf/perf record -e cs_etm/@tmc_etr0/u --per-thread uname
Linux
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.046 MB perf.data ]

but perf report error

tools/perf/perf report --stdio 
0x220 [0x2c28]: failed to process type: 70 [Cannot allocate memory]
Error:
failed to process sample
# To display the perf.data header info, please use --header/--header-only options.
#

perf verbose output

./tools/perf/perf record -vvv -e cs_etm/@tmc_etr0/u --per-thread uname
Using CPUID 0x00000000410fd490
After aliases, add event pmu 'cs_etm' with 'tmc_etr0,' that may result in non-fatal errors
DEBUGINFOD_URLS=
nr_cblocks: 0
affinity: SYS
mmap flush: 1
comp level: 0
------------------------------------------------------------
perf_event_attr:
  type                             7
  size                             128
  { sample_period, sample_freq }   1
  sample_type                      IP|TID|IDENTIFIER
  read_format                      ID
  disabled                         1
  exclude_kernel                   1
  exclude_hv                       1
  enable_on_exec                   1
  sample_id_all                    1
  exclude_guest                    1
  { bp_len, config2 }              0xa6509eae
------------------------------------------------------------
sys_perf_event_open: pid 10456  cpu -1  group_fd -1  flags 0x8 = 5
------------------------------------------------------------
perf_event_attr:
  type                             1
  size                             128
  config                           0x9
  { sample_period, sample_freq }   1
  sample_type                      IP|TID|IDENTIFIER
  read_format                      ID
  disabled                         1
  exclude_kernel                   1
  exclude_hv                       1
  mmap                             1
  comm                             1
  enable_on_exec                   1
  task                             1
  sample_id_all                    1
  exclude_guest                    1
  mmap2                            1
  comm_exec                        1
  context_switch                   1
  ksymbol                          1
  bpf_event                        1
------------------------------------------------------------
sys_perf_event_open: pid 10456  cpu -1  group_fd -1  flags 0x8 = 6
mmap size 528384B
AUX area mmap length 4194304
Control descriptor is not initialized
thread_data[0x1528640]: nr_mmaps=1, maps=0x151ded0, ow_maps=(nil)
thread_data[0x1528640]: cpu-1: maps[0] -> mmap[0]
thread_data[0x1528640]: pollfd[0] <- event_fd=5
thread_data[0x1528640]: pollfd[1] <- event_fd=6
------------------------------------------------------------
perf_event_attr:
  type                             1
  size                             128
  config                           0x9
  watermark                        1
  sample_id_all                    1
  bpf_event                        1
  { wakeup_events, wakeup_watermark } 1
------------------------------------------------------------
sys_perf_event_open: pid -1  cpu -1  group_fd -1  flags 0x8
sys_perf_event_open failed, error -22
switching off bpf_event
------------------------------------------------------------
perf_event_attr:
  type                             1
  size                             128
  config                           0x9
  watermark                        1
  sample_id_all                    1
  { wakeup_events, wakeup_watermark } 1
------------------------------------------------------------
sys_perf_event_open: pid -1  cpu -1  group_fd -1  flags 0x8
sys_perf_event_open failed, error -22
switching off cloexec flag
------------------------------------------------------------
perf_event_attr:
  type                             1
  size                             128
  config                           0x9
  watermark                        1
  sample_id_all                    1
  { wakeup_events, wakeup_watermark } 1
------------------------------------------------------------
sys_perf_event_open: pid -1  cpu -1  group_fd -1  flags 0
sys_perf_event_open failed, error -22
switching off sample_id_all
------------------------------------------------------------
perf_event_attr:
  type                             1
  size                             128
  config                           0x9
  watermark                        1
  { wakeup_events, wakeup_watermark } 1
------------------------------------------------------------
sys_perf_event_open: pid -1  cpu -1  group_fd -1  flags 0
sys_perf_event_open failed, error -22
Couldn't start the BPF side band thread:
BPF programs starting from now on won't be annotatable
Synthesizing TSC conversion information
Synthesizing id index
Synthesizing auxtrace information
Linux
auxtrace idx 0 old 0 head 0x88d0 diff 0x88d0
[ perf record: Woken up 1 times to write data ]
overlapping maps in [vdso] (disable tui for more info)
symbol:init_start file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:init_complete file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:map_failed file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:map_start file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:map_complete file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:reloc_start file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:reloc_complete file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:unmap_start file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:unmap_complete file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:setjmp file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:longjmp file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:longjmp_target file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:setjmp file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:longjmp file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:longjmp_target file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:lll_lock_wait_private file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:cond_destroy file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:cond_init file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_mallopt_arena_max file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_mallopt_arena_test file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_tunable_tcache_max_bytes file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_tunable_tcache_count file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_tunable_tcache_unsorted_limit file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_mallopt_trim_threshold file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_mallopt_top_pad file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_mallopt_mmap_threshold file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_mallopt_mmap_max file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_mallopt_perturb file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_mallopt_mxfast file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_heap_new file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_arena_reuse_free_list file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_sbrk_less file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_heap_free file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_heap_less file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_tcache_double_free file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_heap_more file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_sbrk_more file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_arena_reuse file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_arena_reuse_wait file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_arena_new file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_arena_retry file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_malloc_retry file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_memalign_retry file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_mallopt_free_dyn_thresholds file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_realloc_retry file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_calloc_retry file:(null) line:0 offset:0 return:0 lazy:(null)
symbol:memory_mallopt file:(null) line:0 offset:0 return:0 lazy:(null)
failed to write feature CPUDESC
failed to write feature MEM_TOPOLOGY
failed to write feature CPU_PMU_CAPS
failed to write feature HYBRID_TOPOLOGY
failed to write feature HYBRID_CPU_PMU_CAPS
[ perf record: Captured and wrote 0.045 MB perf.data ]

Thanks for your help.

`perf` require massive mmap area when using `tmc_etr0`

System:
Board: DragonBoard 410c
System: Linux linaro-developer 5.15.7-01849-g9bc25b368335 #2 SMP PREEMPT Sun Feb 27 22:40:22 CST 2022 aarch64 GNU/Linux
Kernel Option: All coresight related option is on.
perf version: 5.16.11 (installed from apt)

Related Kernel Options:
(are there any other should I enable?)

CONFIG_CORESIGHT=y
CONFIG_CORESIGHT_LINKS_AND_SINKS=y
CONFIG_CORESIGHT_LINK_AND_SINK_TMC=y
CONFIG_CORESIGHT_CATU=y
CONFIG_CORESIGHT_SINK_TPIU=y
CONFIG_CORESIGHT_SINK_ETBV10=y
CONFIG_CORESIGHT_SOURCE_ETM4X=y
CONFIG_CORESIGHT_STM=y
CONFIG_CORESIGHT_CPU_DEBUG=y
CONFIG_CORESIGHT_CTI=y
CONFIG_CORESIGHT_CTI_INTEGRATION_REGS=y
CONFIG_CORESIGHT_TRBE=y

The command:

perf record -vvv -e cs_etm/@tmc_etr0/ --per-thread uname

The output:

Using CPUID 0x00000000410fd030
Attempting to add event pmu 'cs_etm' with 'tmc_etr0,' that may result in non-fatt
al errors
After aliases, add event pmu 'cs_etm' with 'tmc_etr0,' that may result in non-faa
tal errors
nr_cblocks: 0
affinity: SYS
mmap flush: 1
comp level: 0
------------------------------------------------------------
perf_event_attr:
  type                             7
  size                             128
  { sample_period, sample_freq }   1
  sample_type                      IP|TID|IDENTIFIER
  read_format                      ID
  disabled                         1
  exclude_kernel                   1
  exclude_hv                       1
  enable_on_exec                   1
  sample_id_all                    1
  exclude_guest                    1
  { bp_len, config2 }              0xa6509eae
------------------------------------------------------------
sys_perf_event_open: pid 2474  cpu -1  group_fd -1  flags 0x8 = 5
------------------------------------------------------------
perf_event_attr:
  type                             1
  size                             128
  config                           0x9
  { sample_period, sample_freq }   1
  sample_type                      IP|TID|IDENTIFIER
  read_format                      ID
  disabled                         1
  exclude_kernel                   1
  exclude_hv                       1
  mmap                             1
  comm                             1
  enable_on_exec                   1
  task                             1
  sample_id_all                    1
  exclude_guest                    1
  mmap2                            1
  comm_exec                        1
  context_switch                   1
  ksymbol                          1
------------------------------------------------------------
sys_perf_event_open: pid 2474  cpu -1  group_fd -1  flags 0x8 = 6
mmap size 528384B
AUX area mmap length 4194304
failed to mmap AUX area
failed to mmap with 12 (Cannot allocate memory)

I guess it is something wrong with the perf?
I read the doc which says upstream perf is fine.
If not where should I look for a workable perf?

Thanks in advance for your kind help and time.

perf+opencsd static link build failure

I have been trying to build perf statically linked with opencsd. It is very useful on boards without proper rootfs and use initramfs.

So I am trying to build perf natively on arm64 board with debian os, but I always see tons of below errors when I try to build perf+opencsd statically linked, is it not supported? gcc version 8.3.0 (Debian 8.3.0-6)

make LDFLAGS=-static -C tools/perf VF=1 CORESIGHT=1

/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/8/../../../aarch64-linux-gnu/libc.a(dl-version.o): relocation R_AARCH64_ADR_PREL_PG_HI21 against symbol `_dl_ns' which may bind externally can not be used when making a shared object; recompile with -fPIC

There are a lot more of other similar errors.

decoder/docs/html not published

decoder/docs/html/* is not generated and not part of the git repo. It would be great to add html docs to avoid every distro to regenerate it.

Using OpenCSD with Android target?

I'm attempting to use OpenCSD to decode the cs-etm data (perf.data) recorded on an Android target via simpleperf. I believe the formats should be the same but the binary locations on Android (I'm using AOSP 12) are different (there is no ~/.debug folder). I'm wondering if anyone's tried this or has instructions they can give?

A possible bug found by compiler

external/OpenCSD/decoder/source/etmv4/trc_pkt_decode_etmv4i.cpp:1391:29: error: use of logical '&&' with constant operand [-Werror,-Wconstant-logical-operand]
if(getComponentOpMode() && OCSD_OPFLG_PKTDEC_ERROR_BAD_PKTS)
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
external/OpenCSD/decoder/source/etmv4/trc_pkt_decode_etmv4i.cpp:1391:29: note: use '&' for a bitwise operation
if(getComponentOpMode() && OCSD_OPFLG_PKTDEC_ERROR_BAD_PKTS)
^~
&
external/OpenCSD/decoder/source/etmv4/trc_pkt_decode_etmv4i.cpp:1391:29: note: remove constant to silence this warning
if(getComponentOpMode() && OCSD_OPFLG_PKTDEC_ERROR_BAD_PKTS)
~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

Below are the compile flags we used:

prebuilts/clang/host/linux-x86/clang-r377782c/bin/clang++ -c -Wno-enum-compare -Wno-enum-compare-switch -Wno-null-pointer-arithmetic -Wno-null-dereference -Wno-pointer-compare -Wno-xor-used-as-pow -Wno-final-dtor-non-final-class -mthumb -Os -fomit-frame-pointer -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -no-canonical-prefixes -DNDEBUG -UDEBUG -fno-exceptions -Wno-multichar -O2 -g -fno-strict-aliasing -Werror=date-time -Werror=pragma-pack -Werror=pragma-pack-suspicious-include -fdebug-prefix-map=/proc/self/cwd= -D__compiler_offsetof=builtin_offsetof -faddrsig -Werror=int-conversion -fexperimental-new-pass-manager -Wno-reserved-id-macro -Wno-unused-command-line-argument -fcolor-diagnostics -Wno-zero-as-null-pointer-constant -Wno-sign-compare -Wno-defaulted-function-deleted -Wno-inconsistent-missing-override -Wno-c99-designator -ftrivial-auto-var-init=pattern -ffunction-sections -fdata-sections -fno-short-enums -funwind-tables -fstack-protector-strong -Wa,--noexecstack -D_FORTIFY_SOURCE=2 -Wstrict-aliasing=2 -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=format-security -nostdlibinc -msoft-float -march=armv8-a -mfloat-abi=softfp -mfpu=neon-fp-armv8 -target armv7a-linux-androideabi10000 -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin -fPIC -Wsign-promo -Wimplicit-fallthrough -D_LIBCPP_ENABLE_THREAD_SAFETY_ANNOTATIONS -Wno-gnu-include-next -fvisibility-inlines-hidden -Iexternal/OpenCSD/decoder/include -Iexternal/OpenCSD -D__LIBC_API=10000 -D__LIBM_API__=10000 -D__LIBDL_API__=10000 -Iexternal/libcxx/include -Iexternal/libcxxabi/include -Ibionic/libc/include -Wall -Werror -std=gnu++17 -Wno-ignored-qualifiers -Wno-unused-parameter -Wno-switch -Wno-unused-private-field -Wno-implicit-fallthrough -fexceptions -frtti -Isystem/core/include -Isystem/media/audio/include -Ihardware/libhardware/include -Ihardware/libhardware_legacy/include -Ihardware/ril/include -Iframeworks/native/include -Iframeworks/native/opengl/include -Iframeworks/av/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libc/kernel/android/scsi -isystem bionic/libc/kernel/android/uapi -Ilibnativehelper/include_jni -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast -Werror=fortify-source -Werror=address-of-temporary -Werror=return-type -Wno-tautological-constant-compare -Wno-tautological-type-limit-compare -Wno-reorder-init-list -Wno-implicit-int-float-conversion -Wno-int-in-bool-context -Wno-sizeof-array-div -Wno-tautological-overlap-compare -MD -MF out/soong/.intermediates/external/OpenCSD/libopencsd_decoder/android_arm_armv8-a_static/obj/external/OpenCSD/decoder/source/etmv4/trc_pkt_decode_etmv4i.o.d -o out/soong/.intermediates/external/OpenCSD/libopencsd_decoder/android_arm_armv8-a_static/obj/external/OpenCSD/decoder/source/etmv4/trc_pkt_decode_etmv4i.o external/OpenCSD/decoder/source/etmv4/trc_pkt_decode_etmv4i.cpp

ETMv4 Exception Target Hand-Over

I would like to discuss the exception target address handling for ETMv4 decoding.

In my use-case, I am interested in the target address of an exception.
The information exists in the raw trace, and also in the raw trace packets.
Unfortunately, it is filtered in the trace element output.

See the following trace:

Idx:676; ID:10; [0x95 0xaa 0x42 ];	I_ADDR_S_IS0 : Address, Short, IS0.; Addr=0xFFFFFFFF801084A8 ~[0x84A8]
Idx:681; ID:10; [0x06 0x1d ];	I_EXCEPT : Exception.;  IRQ; Ret Addr Follows;
Idx:683; ID:10; [0x95 0x2c ];	I_ADDR_S_IS0 : Address, Short, IS0.; Addr=0xFFFFFFFF801084B0 ~[0xB0]
Idx:681; ID:10; OCSD_GEN_TRC_ELEM_INSTR_RANGE(exec range=0xffffffff801084a8:[0xffffffff801084b0] num_i(2) last_sz(4) (ISA=A64) E --- )
Idx:681; ID:10; OCSD_GEN_TRC_ELEM_EXCEPTION(pref ret addr:0xffffffff801084b0; excep num (0x0e) )
Idx:688; ID:10; [0x85 0x20 0x0a 0x1c 0x80 0x00 0x00 0x00 0x00 0x32 ];	I_ADDR_CTXT_L_64IS0 : Address & Context, Long, 64 bit, IS0.; Addr=0x00000000801C1480; Ctxt: AArch64,EL2, NS; 
Idx:701; ID:10; [0x04 ];	I_TRACE_ON : Trace On.
Idx:704; ID:10; [0x85 0x20 0x58 0x22 0x80 0xff 0xff 0xff 0xff 0xf1 0x01 0xa0 0x7a 0x97 0x80 ];	I_ADDR_CTXT_L_64IS0 : Address & Context, Long, 64 bit, IS0.; Addr=0xFFFFFFFF8022B080; Ctxt: AArch64,EL1, NS; CID=0x80977aa0; VMID=0x0001; 
Idx:722; ID:10; [0xf7 ];	I_ATOM_F1 : Atom format 1.; E
Idx:688; ID:10; OCSD_GEN_TRC_ELEM_PE_CONTEXT((ISA=A64) EL2N; 64-bit; VMID=0x1; CTXTID=0x80977aa0; )
Idx:701; ID:10; OCSD_GEN_TRC_ELEM_TRACE_ON( [begin or filter])
Idx:704; ID:10; OCSD_GEN_TRC_ELEM_PE_CONTEXT((ISA=A64) EL1N; 64-bit; VMID=0x1; CTXTID=0x80977aa0; )
Idx:722; ID:10; OCSD_GEN_TRC_ELEM_INSTR_RANGE(exec range=0xffffffff8022b080:[0xffffffff8022b084] num_i(1) last_sz(4) (ISA=A64) E BR  )

Im my example, the exception target space is filtered (target-side, in coresight).

At Idx:688, the target address is available as CONTEXT + ADDR packet.
But results in no trace element, but only the CONTEXT information (OCSD_GEN_TRC_ELEM_PE_CONTEXT).

So far, I have no idea what would be a beautiful solution to hand-over this address.
Do you have any ideas?
Would OCSD_GEN_TRC_ELEM_ADDR_NACC be the correct type?

Invalidated iterator is used in EtmV4P0Stack::erase_curr_from_front()

In https://github.com/Linaro/OpenCSD/blob/master/decoder/source/etmv4/trc_etmv4_stack_elem.cpp#L190:

void EtmV4P0Stack::erase_curr_from_front()
{
std::deque<TrcStackElem *>::iterator erase_iter;
erase_iter = m_iter;
erase_iter--;
m_P0_stack.erase(erase_iter);

// explicitly delete the item here as the caller can no longer reference it.
// fixes memory leak from github issue #52
delete *erase_iter;

}

We fix the memory leak by "delete *erase_iter". But the element pointed by erase_iter is erased in "m_P0_stack.erase(erase_iter);". The erase() method can invalidate erase_iter iterator, as suggested in the "Iterator invalidation
" section in https://en.cppreference.com/w/cpp/container/vector.

The error was detected by ubsan.
It can be fixed by below change:
{
...
TrcStackElem* elem = *erase_iter;
m_iter = m_P0_stack.erase(erase_iter);

// explicitly delete the item here as the caller can no longer reference it.
// fixes memory leak from github issue #52
delete elem;

}

Memory leak in EtmV4P0Stack::createAddrElem

The memory leak is found by fuzzer. It may not be reproducible by real ETM data.

The allocation happens at below callstack:
#0 0x563bb9007ffd in operator new(unsigned long, std::nothrow_t const&) out/stage2/runtimes/runtimes-x86_64-unknown-linux-gnu-bins/out/stage2/runtimes/runtimes-x86_64-unknown-linux-gnu-bins/out/llvm-project/compiler-rt/lib/asan/asan_new_delete.cpp:101:3
#1 0x563bba039e70 in EtmV4P0Stack::createAddrElem(_ocsd_etmv4_i_pkt_type, unsigned int, _etmv4_addr_val_t const&) external/OpenCSD/decoder/source/etmv4/trc_etmv4_stack_elem.cpp:111:31
#2 0x563bba013869 in TrcPktDecodeEtmV4I::decodePacket() external/OpenCSD/decoder/source/etmv4/trc_pkt_decode_etmv4i.cpp:369:28
#3 0x563bba01169c in TrcPktDecodeEtmV4I::processPacket() external/OpenCSD/decoder/source/etmv4/trc_pkt_decode_etmv4i.cpp:118:19
#4 0x563bba02a50a in TrcPktDecodeBase<EtmV4ITrcPacket, EtmV4Config>::PacketDataIn(_ocsd_datapath_op_t, unsigned int, EtmV4ITrcPacket const*) external/OpenCSD/decoder/include/common/trc_pkt_decode_base.h:0

The memleak happens at below callstack:
#7 0x55a755909146 in EtmV4P0Stack::erase_curr_from_front() external/OpenCSD/decoder/source/etmv4/trc_etmv4_stack_elem.cpp:190:9
#8 0x55a7558eaa08 in TrcPktDecodeEtmV4I::mispredictAtom() external/OpenCSD/decoder/source/etmv4/trc_pkt_decode_etmv4i.cpp:1087:28
#9 0x55a7558e2dae in TrcPktDecodeEtmV4I::resolveElements() external/OpenCSD/decoder/source/etmv4/trc_pkt_decode_etmv4i.cpp:674:27
#10 0x55a7558df0d1 in TrcPktDecodeEtmV4I::processPacket() external/OpenCSD/decoder/source/etmv4/trc_pkt_decode_etmv4i.cpp:138:20
#11 0x55a7558f81da in TrcPktDecodeBase<EtmV4ITrcPacket, EtmV4Config>::PacketDataIn(_ocsd_datapath_op_t, unsigned int, EtmV4ITrcPacket const*) external/OpenCSD/decoder/include/common/trc_pkt_decode_base.h

The memleak disappears after adding "delete *erase_iter;" in EtmV4P0Stack::erase_curr_from_front().

Build race

Our autobuilder just failed with the following failure in a parallel build:

error: unable to open output file 'builddir/mem_buff_demo.o': 'No such file or directory'

Looks like a missing dependency somewhere.

How to use

I have a STM32F411 (Cortex-M4) from which I want to gather ETM information with a logic analyzer and I can't get decoding to work.

At the moment I output NRZ encoded data on the single SWO pin.
As stated in the "CoreSight Architecture Specification v3.0" a data byte is output lsb first with a start(0) and stop(1) bit.
Here is the bitstream of the first 10 16-byte frames formatted in 10bits:

1111111111 1111111111 1111111111 1111111111 1111111111 1111111111 1111111111 1111111111
0100000101 0000100001 0000001101 0100101011 0010000001 0000000001 0000100001 0000110011
0001111001 0001100011 0011011001 0001010111 0000010011 0111100101 0000100011 0011011001
0010100101 0000010011 0010100101 0000010011 0010100101 0000010011 0010100101 0000010011
0010100101 0000010011 0010100101 0000010011 0010100101 0000010011 0010100101 0111111111
0000010011 0110100101 0000010011 0110100101 0000010011 0110100101 0000010011 0110100101
0000010011 0110100101 0000010011 0110100101 0000010011 0110100101 0000010011 0000000001
0010100101 0000010011 0010100101 0000010011 0010100101 0000010011 0010100101 0000010011
0010100101 0000010011 0010100101 0000010011 0010100101 0000010011 0010100101 0111111111
0000010011 0110100101 0000010011 0110100101 0000010011 0110100101 0000010011 0110100101
0000010011 0110100101 0000010011 0110100101 0000010011 0110100101 0000010011 0000000001
0010100101 0000010011 0010100101 0000010011 0010100101 0000010011 0010100101 0000010011
0010100101 0000010011 0010100101 0000010011 0010100101 0000010011 0010100101 0111111111
0000010011 0110100101 0000010011 0110100101 0000010011 0110100101 0000010011 0110100101
0000010011 0110100101 0000010011 0110100101 0000010011 0110100101 0000010011 0000000001
0010100101 0000010011 0010100101 0000010011 0010100101 0000010011 0010100101 0000010011
0010100101 0000010011 0010100101 0000010011 0010100101 0000010011 0010100101 0111111111
0000010011 0110100101 0000010011 0110100101 0000010011 0110100101 0000010011 0110100101
0000010011 0110100101 0000010011 0110100101 0000010011 0110100101 0000010011 0000000001
0010100101 0000010011 0010100101 0000010011 0010100101 0000010011 0010100101 0000010011
0010100101 0000010011 0010100101 0000010011 0010100101 0000010011 0010100101 0111111111

This output shows a long run of 1s until data is sent from the device which should consist mostly of doing a busy loop.
So far this looks legit to me, but there's no frame synchronization packet to be seen.

Here is a shortened excerpt of my OpenCSD configuration:

DecodeTree *pDecodeTree = DecodeTree::CreateDecodeTree(
	OCSD_TRC_SRC_FRAME_FORMATTED,
	OCSD_DFRMTR_PACKED_RAW_OUT | OCSD_DFRMTR_UNPACKED_RAW_OUT | OCSD_DFRMTR_FRAME_MEM_ALIGN
);

ocsd_etmv3_cfg config;
config.reg_idr = 0x4114F250; // ETMIDR 0xE00411E4
config.reg_ctrl = 0x00000190; // ETMCR 0xE0041000
config.reg_ccer = 0x18541800; // ETMCCER 0xE00411E8
config.reg_trc_id = 0x00000020; // ETMTRACEIDR 0xE0041200
config.arch_ver = ARCH_V7;
config.core_prof = profile_CortexM;
EtmV3Config configObj(&config);

pDecodeTree->createDecoder(
	OCSD_BUILTIN_DCD_ETMV3,
	OCSD_CREATE_FLG_FULL_DECODER,
	&configObj
);

pDecodeTree->createMemAccMapper();
pDecodeTree->addBinFileMemAcc(0x08000000, OCSD_MEM_SPACE_ANY, "Firmware.bin");
pDecodeTree->addGenElemPrinter(&genElemPrinter);
pDecodeTree->addRawFramePrinter(&framePrinter, OCSD_DFRMTR_PACKED_RAW_OUT | OCSD_DFRMTR_UNPACKED_RAW_OUT);
DecodeTreeElement *pElement = pDecodeTree->getFirstElement(elemID);
while (pElement) {
	ItemPrinter *pPrinter;
	ocsd_err_t err = pDecodeTree->addPacketPrinter(elemID, true, &pPrinter);
	pElement = pDecodeTree->getNextElement(elemID);
}

When I feed the extracted bytes of above bitstream into OpenCSD, this is the output with all printers active:

Frame Data; Index      0;    RAW_PACKED; 41 08 60 a9 02 00 08 98 3c 8c 36 d4 90 4f 88 36
Frame Data; Index      0;   ID_DATA[0x20]; 08 61 a9 03 00 08 98 3d 8c 37 d4 90 4f 88
Idx:0; ID:20; [0x08 0x61 0xa9 0x03 ];   NOTSYNC : Trace Stream not synchronised
Idx:0; ID:20; OCSD_GEN_TRC_ELEM_NO_SYNC( [init-decoder])
Idx:4; ID:20; [0x00 0x08 0x98 0x3d 0x8c 0x37 0xd4 0x90 0x4f 0x88 ];     NOTSYNC : Trace Stream not synchronised
Frame Data; Index     16;    RAW_PACKED; 4a 90 4a 90 4a 90 4a 90 4a 90 4a 90 4a 90 4a ff
Frame Data; Index     16;   ID_DATA[0x20]; 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b
Idx:16; ID:20; [0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b ];   NOTSYNC : Trace Stream not synchronised
Frame Data; Index     32;    RAW_PACKED; 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 00
Frame Data; Index     32;   ID_DATA[0x20]; 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90
Idx:32; ID:20; [0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 ];   NOTSYNC : Trace Stream not synchronised
Frame Data; Index     48;    RAW_PACKED; 4a 90 4a 90 4a 90 4a 90 4a 90 4a 90 4a 90 4a ff
Frame Data; Index     48;   ID_DATA[0x20]; 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b
Idx:48; ID:20; [0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b ];   NOTSYNC : Trace Stream not synchronised
Frame Data; Index     64;    RAW_PACKED; 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 00
Frame Data; Index     64;   ID_DATA[0x20]; 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90
Idx:64; ID:20; [0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 ];   NOTSYNC : Trace Stream not synchronised
Frame Data; Index     80;    RAW_PACKED; 4a 90 4a 90 4a 90 4a 90 4a 90 4a 90 4a 90 4a ff
Frame Data; Index     80;   ID_DATA[0x20]; 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b
Idx:80; ID:20; [0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b ];   NOTSYNC : Trace Stream not synchronised
Frame Data; Index     96;    RAW_PACKED; 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 00
Frame Data; Index     96;   ID_DATA[0x20]; 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90
Idx:96; ID:20; [0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 ];   NOTSYNC : Trace Stream not synchronised
Frame Data; Index    112;    RAW_PACKED; 4a 90 4a 90 4a 90 4a 90 4a 90 4a 90 4a 90 4a ff
Frame Data; Index    112;   ID_DATA[0x20]; 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b
Idx:112; ID:20; [0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b ];  NOTSYNC : Trace Stream not synchronised
Frame Data; Index    128;    RAW_PACKED; 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 00
Frame Data; Index    128;   ID_DATA[0x20]; 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90
Idx:128; ID:20; [0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 ];  NOTSYNC : Trace Stream not synchronised
Frame Data; Index    144;    RAW_PACKED; 4a 90 4a 90 4a 90 4a 90 4a 90 4a 90 4a 90 4a ff
Frame Data; Index    144;   ID_DATA[0x20]; 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b 90 4b
Idx:144; ID:20; [0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b 0x90 0x4b ];  NOTSYNC : Trace Stream not synchronised
Idx:144; ID:20; OCSD_GEN_TRC_ELEM_EO_TRACE( [end-of-trace])
ID:20   END OF TRACE DATA

As I can see no frame synchronization packets I used OCSD_DFRMTR_FRAME_MEM_ALIGN instead of OCSD_DFRMTR_HAS_FSYNCS.
Even with manually inserting 4 sync bytes and changing the configuration I could not get a meaningful output.
What does OpenCSD need to synchronise?

ARM ETMv3.3 ISYNC decoding

Hello, I have been using OpenCSD to decode some ARM ETMv3.3 data, and I think there is an error when reading ISYNC packets. The address does not seem to be read correctly. It appears that when reading the ContextID, the index is updated twice. This causes junk to bread into the address.

Out of bound Read found in TraceFmtDcdImpl::extractFrame()

It's reported by [email protected] in Android. Forward it to upstream:

Issue Description

Out of bound Read found in libopencsd_decoder due to bound check fail of dataPtr in TraceFmtDcdImpl::extractFrame() in trc_frame_deformatter.cpp.

It may cause security vulnerability by reading invalid memory address in memcpy.

external/OpenCSD/decoder/source/trc_frame_deformatter.cpp

(line 517, 526, 569) There are invalid bound checks. In blocks after cont_process bool checking condition,

(line 522, 536, 569) uint32_t or uint16_t dereference after invalid bound check causes OOB read.

459 bool TraceFmtDcdImpl::extractFrame()
460 {
...
514         const uint8_t *dataPtr = m_in_block_base+m_in_block_processed;
515         const uint8_t *eodPtr = m_in_block_base+m_in_block_size;
516
517         cont_process = (bool)(dataPtr < eodPtr);  <- bound check fail (1)
518
519         // can have FSYNCS at start of frame (in middle is an error).
520         if(hasFSyncs && cont_process && (m_ex_frm_n_bytes == 0))
521         {
522             while((*((uint32_t *)(dataPtr)) == FSYNC_PATTERN) && cont_process)  <- OOB Read! (1)
523             {
524                 f_sync_bytes += 4;
525                 dataPtr += 4;
526                 cont_process = (bool)(dataPtr < eodPtr);  <- bound check fail (2)
527             }
528         }
529
530         // not an FSYNC
531         while((m_ex_frm_n_bytes < OCSD_DFRMTR_FRAME_SIZE) && cont_process)
532         {
533             // check for illegal out of sequence FSYNC
534             if((m_ex_frm_n_bytes % 4) == 0)
535             {
536                 if(*((uint32_t *)(dataPtr)) == FSYNC_PATTERN)  <- OOB Read! (2)
537                 {
538                     // throw an illegal FSYNC error
539                     throw ocsdError(OCSD_ERR_SEV_ERROR, OCSD_ERR_DFMTR_BAD_FHSYNC, m_trc_curr_idx, "Bad FSYNC in frame.");
540                 }
541             }
542
543             // mark start of frame after FSyncs
544             if(m_ex_frm_n_bytes == 0)
545                 m_trc_curr_idx_sof = m_trc_curr_idx + f_sync_bytes;
546
547             m_ex_frm_data[m_ex_frm_n_bytes] = dataPtr[0];
548             m_ex_frm_data[m_ex_frm_n_bytes+1] = dataPtr[1];
549             m_ex_frm_n_bytes+=2;
550             ex_bytes +=2;
551
552             // check pair is not HSYNC
553             if(*((uint16_t *)(dataPtr)) == HSYNC_PATTERN)  <- OOB Read! (3)
554             {
555                 if(hasHSyncs)
556                 {
557                     m_ex_frm_n_bytes-=2;
558                     ex_bytes -= 2;
559                     h_sync_bytes+=2;
560                 }
561                 else
562                 {
563                     // throw illegal HSYNC error.
564                     throw ocsdError(OCSD_ERR_SEV_ERROR, OCSD_ERR_DFMTR_BAD_FHSYNC, m_trc_curr_idx, "Bad HSYNC in frame.");
565                 }
566             }
567
568             dataPtr += 2;
569             cont_process = (bool)(dataPtr < eodPtr);  <- bound check fail (3)
570         }
571
572         // if we hit the end of data but still have a complete frame waiting,
573         // need to continue processing to allow it to be used.
574         if(!cont_process && (m_ex_frm_n_bytes == OCSD_DFRMTR_FRAME_SIZE))
575             cont_process = true;
576     }

Suggest fix:

0002-Fix-Out-of-Bounds-in-TraceFmtDcdImpl-extractFrame.patch

---
 decoder/source/trc_frame_deformatter.cpp | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/decoder/source/trc_frame_deformatter.cpp b/decoder/source/trc_frame_deformatter.cpp
index dc12e3f..afa1d1f 100644
--- a/decoder/source/trc_frame_deformatter.cpp
+++ b/decoder/source/trc_frame_deformatter.cpp
@@ -514,7 +514,7 @@ bool TraceFmtDcdImpl::extractFrame()
         const uint8_t *dataPtr = m_in_block_base+m_in_block_processed;
         const uint8_t *eodPtr = m_in_block_base+m_in_block_size;

-        cont_process = (bool)(dataPtr < eodPtr);
+        cont_process = (bool)(dataPtr + sizeof(uint32_t) < eodPtr);

         // can have FSYNCS at start of frame (in middle is an error).
         if(hasFSyncs && cont_process && (m_ex_frm_n_bytes == 0))
@@ -523,7 +523,7 @@ bool TraceFmtDcdImpl::extractFrame()
             {
                 f_sync_bytes += 4;
                 dataPtr += 4;
-                cont_process = (bool)(dataPtr < eodPtr);
+                cont_process = (bool)(dataPtr + sizeof(uint32_t) < eodPtr);
             }
         }

@@ -566,7 +566,7 @@ bool TraceFmtDcdImpl::extractFrame()
             }

             dataPtr += 2;
-            cont_process = (bool)(dataPtr < eodPtr);
+            cont_process = (bool)(dataPtr + sizeof(uint32_t) < eodPtr);
         }

         // if we hit the end of data but still have a complete frame waiting,
--

Default error logger does not output to stderr as stated in prog_guide_main.md

In prog_guide_main.md it states:

The DecodeTree will use a default error logger from the library - with a message logger that will output to stderr

However if I use a DecodeTree without calling any functions to setup/initialize loggers than nothing comes out of stderr.

Indeed one can see in ocsdDefaultErrorLogger::ocsdDefaultErrorLogger() that m_output_logger is initialized to 0:

ocsdDefaultErrorLogger::ocsdDefaultErrorLogger() :
    m_Verbosity(OCSD_ERR_SEV_ERROR),
    m_output_logger(0),
    m_created_output_logger(false)
{
...
}

This causes error messages to not come out due to m_output_logger is initialized to 0, from ocsdDefaultErrorLogger::LogError():

void ocsdDefaultErrorLogger::LogError(const ocsd_hndl_err_log_t handle, const ocsdError *Error)
{
    // only log errors that match or exceed the current verbosity
    if(m_Verbosity >= Error->getErrorSeverity())
    {
        // print out only if required
        if(m_output_logger)
        {
          ...
        }

So either documentation should say that default logger does not output anywhere (and either creation of a logger is required to output errors or need to call some functions to modify existing default logger), or OpenCSD library should be updated to indeed make the default logger go to stderr.

‘OCSD_INSTR_WFI_WFE’ not handled in switch

After installing the OpenCSD library, I tried to compile perf from perf-opencsd. I used the command make VF=1 -C tools/perf. However, I got the following error:

  CC       util/intel-pt-decoder/intel-pt-log.o
  CC       util/cs-etm-decoder/cs-etm-decoder.o
util/cs-etm-decoder/cs-etm-decoder.c: In function ‘cs_etm_decoder__buffer_range’:
util/cs-etm-decoder/cs-etm-decoder.c:370:2: error: enumeration value ‘OCSD_INSTR_WFI_WFE’ not handled in switch [-Werror=switch-enum]
  switch (elem->last_i_type) {
  ^~~~~~
  CC       util/intel-pt-decoder/intel-pt-decoder.o
cc1: all warnings being treated as errors

Has anyone had the same issue before?

Q: Can I get errors for inconsistencies between trace and binary?

Hello :)

Let's imagine the decoder expects a atom for a conditional branch, but gets an atom + an address (as for a indirect branch).
Is it possible to get errors about unexpected trace packets when reconstructing the addresses with OpenCSD?

I would highly appreciate your help. Thanks for this great decoder implementation by the way.

Out of bound Read & Write found in TraceFmtDcdImpl::unpackFrame()

It's reported by [email protected] in Android. Forward it to upstream:

Issue Description

Out of bound found in libopencsd_decoder due to absense of bound check in TraceFmtDcdImpl::unpackFrame() in trc_frame_deformatter.cpp.

It may cause security vulnerability by reading and writing invalid memory address.

external/OpenCSD/decoder/source/trc_frame_deformatter_impl.h

  • m_out_data is a member variable of class TraceFmtDcdImpl, which is a 7-length array type.
 46 typedef struct _out_chan_data {
 47     ocsd_trc_index_t index;      //!< trace source index for start of these bytes
 48     uint8_t id;             //!< Id for these bytes
 49     uint8_t data[15];       //!< frame data bytes for this ID
 50     uint32_t valid;         //!< Valid data bytes.
 51     uint32_t used;          //!< Data bytes output (used by attached processor).
 52 } out_chan_data;

...

158     out_chan_data m_out_data[7];

external/OpenCSD/decoder/source/trc_frame_deformatter.cpp

(line 613) The local variable m_out_data_idx is initialized to 0.

(line 623) According to the loop condition, this loop is executed 7 times.

(line 646) So m_out_data_idx can be up to 7, since the m_out_data_idx++; command can be executed 7 times in this loop.

If the value of m_out_data_idx is 7, then all m_out_data[m_out_data_idx] accesses are out of bound.

604 bool TraceFmtDcdImpl::unpackFrame()
605 {
606     // unpack cannot fail as never called on incomplete frame.
607     uint8_t frameFlagBit = 0x1;
608     uint8_t newSrcID = OCSD_BAD_CS_SRC_ID;
609     bool PrevIDandIDChange = false;
610     uint64_t noneDataBytes = 0;
611
612     // init output processing
613     m_out_data_idx = 0;
614     m_out_processed = 0;
615
616     // set up first out data packet...
617     m_out_data[m_out_data_idx].id = m_curr_src_ID;
618     m_out_data[m_out_data_idx].valid = 0;
619     m_out_data[m_out_data_idx].index =  m_trc_curr_idx_sof;
620     m_out_data[m_out_data_idx].used = 0;
621
622     // work on byte pairs - bytes 0 - 13.
623     for(int i = 0; i < 14; i+=2)
624     {
625         PrevIDandIDChange = false;
626
627         // it's an ID + data
628         if(m_ex_frm_data[i] & 0x1)
629         {
630             newSrcID = (m_ex_frm_data[i] >> 1) & 0x7f;
631             if(newSrcID != m_curr_src_ID)   // ID change
632             {
633                 PrevIDandIDChange = ((frameFlagBit & m_ex_frm_data[15]) != 0);
634
635                 // following byte for old id?
636                 if(PrevIDandIDChange)
637                     // 2nd byte always data
638                     m_out_data[m_out_data_idx].data[m_out_data[m_out_data_idx].valid++] = m_ex_frm_data[i+1];
639
640                 // change ID
641                 m_curr_src_ID = newSrcID;
642
643                 // if we already have data in this buffer
644                 if(m_out_data[m_out_data_idx].valid > 0)
645                 {
646                     m_out_data_idx++; // move to next buffer
647                     m_out_data[m_out_data_idx].valid = 0;  <- OOB Write!
648                     m_out_data[m_out_data_idx].used = 0;  <- OOB Write!
649                     m_out_data[m_out_data_idx].index = m_trc_curr_idx_sof + i;  <- OOB Write!
650                 }
651
652                 // set new ID on buffer
653                 m_out_data[m_out_data_idx].id = m_curr_src_ID;  <- OOB Write!
654
655                 /// TBD - ID indexing in here.
656             }
657             noneDataBytes++;
658         }
659         else
660         // it's just data
661         {
662             m_out_data[m_out_data_idx].data[m_out_data[m_out_data_idx].valid++] = m_ex_frm_data[i] | ((frameFlagBit & m_ex_frm_data[15]) ? 0x1 : 0x0);
663         }
664
665         // 2nd byte always data
666         if(!PrevIDandIDChange) // output only if we didn't for an ID change + prev ID.
667             m_out_data[m_out_data_idx].data[m_out_data[m_out_data_idx].valid++] = m_ex_frm_data[i+1];  <- OOB Read & Write!
668
669         frameFlagBit <<= 1;
670     }
671
672     // unpack byte 14;
673
674     // it's an ID
675     if(m_ex_frm_data[14] & 0x1)
676     {
677         // no matter if change or not, no associated data in byte 15 anyway so just set.
678         m_curr_src_ID = (m_ex_frm_data[14] >> 1) & 0x7f;
679         noneDataBytes++;
680     }
681     // it's data
682     else
683     {
684         m_out_data[m_out_data_idx].data[m_out_data[m_out_data_idx].valid++] = m_ex_frm_data[14] | ((frameFlagBit & m_ex_frm_data[15]) ? 0x1 : 0x0);  <- OOB Read & Write!
685     }
686     m_ex_frm_n_bytes = 0;   // mark frame as empty;
687
688     noneDataBytes++;    // byte 15 is always non-data.
689     addToFrameStats(noneDataBytes); // update the non data byte stats.
690     return true;
691 }

suggest fix:

0001-Fix-Out-of-Bounds-in-TraceFmtDcdImpl-unpackFrame.patch

---
 decoder/source/trc_frame_deformatter.cpp | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/decoder/source/trc_frame_deformatter.cpp b/decoder/source/trc_frame_deformatter.cpp
index dc12e3f..fc7a76c 100644
--- a/decoder/source/trc_frame_deformatter.cpp
+++ b/decoder/source/trc_frame_deformatter.cpp
@@ -643,6 +643,10 @@ bool TraceFmtDcdImpl::unpackFrame()
                 // if we already have data in this buffer
                 if(m_out_data[m_out_data_idx].valid > 0)
                 {
+                    if (m_out_data_idx == 6)
+                    {
+                        return false;
+                    }
                     m_out_data_idx++; // move to next buffer
                     m_out_data[m_out_data_idx].valid = 0;
                     m_out_data[m_out_data_idx].used = 0;
--

Multithreaded Decoding

Hi there, is the library in general usable on to split the decoding to multiple threads?
Traces are often huge. Using the complete CPU power could reduce the decoding time by the number of true-parallel threads.
What would be the usage pattern when using multiple threads?

object files don't get rebuilt when their dependent header files change 

$ git describe 
v0.7.2-8-gdd48085b3f61
$ pwd
/home/kim/git/OpenCSD-lib/decoder/build/linux
$ make -j 8 |& grep -iwc  "g++ -c"
0
$ touch ../../../decoder/source/ptm/trc_pkt_proc_ptm.cpp
$ make -j 8 |& grep -iwc  "g++ -c"
1
$ make -j 8 |& grep -iwc  "g++ -c"
0
$ touch ../../../decoder/include/ptm/trc_pkt_proc_ptm.h
$ make -j 8 |& grep -iwc  "g++ -c"
0
$

That last 0 should be 1, since trc_pkt_proc_ptm.cpp #includes trc_pkt_proc_ptm.h.

Build without static is broken

Since e7240a4 we can use DISABLE_STATIC=1 to not build static binaries.
But f14d804 added some static binaries not guarded. This should be properly guarded with some ifndef DISABLE_STATIC.

Out of bound Read found in TraceFmtDcdImpl::extractFrame()

It's reported by [email protected] in Android. Forward it to upstream:

Issue Description

Out of bound Read found in libopencsd_decoder due to bound check fail in TraceFmtDcdImpl::extractFrame() in trc_frame_deformatter.cpp.

It may cause security vulnerability by reading invalid memory address in memcpy.

external/OpenCSD/decoder/source/trc_frame_deformatter.cpp

(line 491) This is a conditional statement that checks the end of input data.

(line 496) It is always assumed that there is a complete frame if the condition is not satisfied, but sometimes it is not.

(line 500) OOB Read occurs in memcpy() when (m_in_block_processed + f_sync_bytes) + OCSD_DFRMTR_FRAME_SIZE > m_in_block_size.

459 bool TraceFmtDcdImpl::extractFrame()
460 {
...
491         if((m_in_block_processed+f_sync_bytes) == m_in_block_size)
492         {
493             m_ex_frm_n_bytes = 0;
494             cont_process = false;   // end of input data.
495         }
496                 else
497                 {
498                         // always a complete frame.
499                         m_ex_frm_n_bytes = OCSD_DFRMTR_FRAME_SIZE;
500                         memcpy(m_ex_frm_data, m_in_block_base + m_in_block_processed + f_sync_bytes, m_ex_frm_n_bytes);  <- OOB Read!
501                         m_trc_curr_idx_sof = m_trc_curr_idx + f_sync_bytes;
502                         ex_bytes = OCSD_DFRMTR_FRAME_SIZE;
503         }
504     }

...

592     // update the processed count for the buffer
593     m_in_block_processed += total_processed;  <- m_in_block_processed is added up to the end of input data.

Suggest fix:

0001-Fix-Out-of-Bounds-in-TraceFmtDcdImpl-extractFrame.patch

---
 decoder/source/trc_frame_deformatter.cpp | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/decoder/source/trc_frame_deformatter.cpp b/decoder/source/trc_frame_deformatter.cpp
index dc12e3f..9f6811e 100644
--- a/decoder/source/trc_frame_deformatter.cpp
+++ b/decoder/source/trc_frame_deformatter.cpp
@@ -488,18 +488,18 @@ bool TraceFmtDcdImpl::extractFrame()
             }
         }

-        if((m_in_block_processed+f_sync_bytes) == m_in_block_size)
+        if((m_in_block_processed + f_sync_bytes) + OCSD_DFRMTR_FRAME_SIZE > m_in_block_size)
         {
             m_ex_frm_n_bytes = 0;
             cont_process = false;   // end of input data.
         }
-               else
-               {
-                       // always a complete frame.
-                       m_ex_frm_n_bytes = OCSD_DFRMTR_FRAME_SIZE;
-                       memcpy(m_ex_frm_data, m_in_block_base + m_in_block_processed + f_sync_bytes, m_ex_frm_n_bytes);
-                       m_trc_curr_idx_sof = m_trc_curr_idx + f_sync_bytes;
-                       ex_bytes = OCSD_DFRMTR_FRAME_SIZE;
+                   else
+                   {
+            // always a complete frame.
+            m_ex_frm_n_bytes = OCSD_DFRMTR_FRAME_SIZE;
+            memcpy(m_ex_frm_data, m_in_block_base + m_in_block_processed + f_sync_bytes, m_ex_frm_n_bytes);
+            m_trc_curr_idx_sof = m_trc_curr_idx + f_sync_bytes;
+            ex_bytes = OCSD_DFRMTR_FRAME_SIZE;
         }
     }
     else
--

Memory leak on m_i_gen_elem_out of DecodeTree

In 'DecodeTree::setGenTraceElemOutI' it supposed to save argument 'i_gen_trace_elem' to 'm_i_gen_elem_out', so in 'ocsd_destroy_dcd_tree' the 'm_i_gen_elem_out' can be freed via the following clean up:

OCSD_C_API void ocsd_destroy_dcd_tree(const dcd_tree_handle_t handle)
{
    if(handle != C_API_INVALID_TREE_HANDLE)
    {
        GenTraceElemCBObj * pIf = (GenTraceElemCBObj *)(((DecodeTree *)handle)->getGenTraceElemOutI());
        if(pIf != 0)
            delete pIf;
        ...

The fix can be done like this:

diff --git a/decoder/source/ocsd_dcd_tree.cpp b/decoder/source/ocsd_dcd_tree.cpp
index 8e29269..0158e73 100644
--- a/decoder/source/ocsd_dcd_tree.cpp
+++ b/decoder/source/ocsd_dcd_tree.cpp
@@ -174,6 +174,9 @@ void DecodeTree::setGenTraceElemOutI(ITrcGenElemIn *i_gen_trace_elem)
         pElem->getDecoderMngr()->attachOutputSink(pElem->getDecoderHandle(),i_gen_trace_elem);
         pElem = getNextElement(elemID);
     }
+    if (m_i_gen_elem_out != 0)
+        delete m_i_gen_elem_out;
+    m_i_gen_elem_out = i_gen_trace_elem;
 }
 
 ocsd_err_t DecodeTree::createMemAccMapper(memacc_mapper_t type /* = MEMACC_MAP_GLOBAL*/ )

Use -fPIC rather than -fpic

Due to fine implementation differences between -fpic and -fPIC on certain CPU architectures, the decoder library currently fails to build on Debian sparc64: https://buildd.debian.org/status/fetch.php?pkg=libopencsd&arch=sparc64&ver=0.10.1-1&stamp=1552450383&raw=0

Since Debian builds packages on all supported architectures by default and other packages depend on libopencsd, this has certain unpleasant consequences. A possible fix would be to remove those dependencies for sparc64, but it would be much cleaner to fix the build script instead.

Is it possible to change the compiler flags from -fpic to -fPIC here or is there a specific reason why -fpic is used?

OCSD_ERR_TEST_SS_TO_DECODER error occured when I get trace from my custom CSAL

I created my custom CSAL library to add support for Jetson TX2 board and it works almost correctly(Fetching cstrace.bin succeeds).
However when executing trc_pkt_lister in Jetson TX2 snapshot directory, trc_pkt_lister says as follows.

Trace Packet Lister: CS Decode library testing
-----------------------------------------------

** Library Version : 0.12.0

Test Command Line:-
trc_pkt_lister   --decode  

Trace Packet Lister : Warning: Ignored unknown option --decode.
Trace Packet Lister : reading snapshot from path ./
Using ETB_0 as trace source
ss2_dcdtree : 0x0026 (OCSD_ERR_TEST_SS_TO_DECODER) [test snapshot to decode tree conversion error]; Unrecognized Core 
name unknown. Cannot evaluate profile or architecture.ss2_dcdtree : 0x0026 (OCSD_ERR_TEST_SS_TO_DECODER) [test snapsho
t to decode tree conversion error]; Failed to create decoder for source ETM_3.
ss2_dcdtree : 0x0026 (OCSD_ERR_TEST_SS_TO_DECODER) [test snapshot to decode tree conversion error]; Unrecognized Core name unknown. Cannot evaluate profile or architecture.ss2_dcdtree : 0x0026 (OCSD_ERR_TEST_SS_TO_DECODER) [test snapshot to decode tree conversion error]; Failed to create decoder for source ETM_4.
ss2_dcdtree : 0x0026 (OCSD_ERR_TEST_SS_TO_DECODER) [test snapshot to decode tree conversion error]; Unrecognized Core name unknown. Cannot evaluate profile or architecture.ss2_dcdtree : 0x0026 (OCSD_ERR_TEST_SS_TO_DECODER) [test snapshot to decode tree conversion error]; Failed to create decoder for source ETM_5.
ss2_dcdtree : 0x0026 (OCSD_ERR_TEST_SS_TO_DECODER) [test snapshot to decode tree conversion error]; Unrecognized Core name unknown. Cannot evaluate profile or architecture.ss2_dcdtree : 0x0026 (OCSD_ERR_TEST_SS_TO_DECODER) [test snapshot to decode tree conversion error]; Failed to create decoder for source ETM_6.

Is this error caused by the lack of implementation of my CSAL registration function? or the lack of support in OpenCSD library?

decoding ETM4 trace data with trc_pkt_lister

Use Case

I saved the last collected data from the ETB (in cyclic mode) inside a persistent buffer and I only turned on the ETM4 (see ETM Configuration).
Now, I want to decode the data using the trc_pkt_lister -decode -ss_dir . command on my snapshot.

ETM Configuration

  *etm_lar = 0xC5ACCE55;
  *etm_config = 0x00000010;
  *etm_ccctl = 0x000003E8;
  *etm_traceid = 0x00000010;
  *etm_victl = 0x00000201;
  *etm_pdc = *etm_pdc | 0x00000008;    // Power On ETM
  *etm_prgctl = 0x00000001;

Snapshot

My snapshot looks like this:

  • snapshot.ini:
[snapshot]
version=1.0

[device_list]
device0=cpu_0.ini
device1=device_1.ini

[trace]
metadata=trace.ini
  • trace.ini:
[trace_buffers]
buffers=buffer0

[buffer0]
name=ETB
file=etmout.bin
format=coresight

[source_buffers]
ETM_0=ETB

[core_trace_sources]
cpu_0=ETM_0
  • cpu_0.ini:
[device]
name=cpu_0
class=core
type=Cortex-M4

[regs]
R15=0x700269BD
R13=0
CPSR=0x1D3

[dump0]
file=firmware.bin
address=0x700269BD
  • device_1.ini:
[device]
name=ETM_0
class=trace_source
type=ETM4

[regs]
TRCCONFIGR(0x010)=0x00000011
TRCTRACEIDR(0x040)=0x00000010
TRCAUTHSTATUS(0xFB8)=0x000000C0
TRCIDR0(0x1E0)=0x080006E1
TRCIDR1(0x1E4)=0x4100F401
TRCIDR2(0x1E8)=0x00000004
TRCIDR8(0x180)=0x00000001
TRCIDR9(0x184)=0x00000000
TRCIDR10(0x188)=0x00000000
TRCIDR11(0x18C)=0x00000000
TRCIDR12(0x190)=0x00000001
TRCIDR13(0x194)=0x00000000
  • etmout.bin, containing the collected etm data
  • firmware.bin, containing the firmware

Problem

And I get this error message:

Trace Packet Lister: CS Decode library testing
-----------------------------------------------

** Library Version : 1.5.2

Test Command Line:-
trc_pkt_lister   -decode  -ss_dir  .  -v  

Trace Packet Lister : Warning: Ignored unknown option -v.
Trace Packet Lister : reading snapshot from path .
Using ETB as trace source
ss2_dcdtree : 0x0021 (OCSD_ERR_MEM_ACC_BAD_LEN) [Memory accessor returned a bad read length value (larger than requested.]; Failed to create memory accessor for file ./firmware.bin.Trace Packet Lister : Protocol printer ETMV4I on Trace ID 0x10
Trace Packet Lister : Set trace element decode printer
Gen_Info : Mapped Memory Accessors
Gen_Info : ========================
Idx:2604; ID:10; [0x06 0x96 0xf6 0xe1 ];        I_NOT_SYNC : I Stream not synchronised
Idx:2604; ID:10; OCSD_GEN_TRC_ELEM_NO_SYNC( [init-decoder])
Trace Packet Lister : Data Path fatal error
0x0021 (OCSD_ERR_MEM_ACC_BAD_LEN) [Memory accessor returned a bad read length value (larger than requested.]; Failed to create memory accessor for file ./firmware.bin.Trace Packet Lister : Trace buffer done, processed 4096 bytes.

But I checked the entry point of my firmware.bin and it is the address stated in cpu_0.ini.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.