cr / hx870 Goto Github PK

Also the designation tty is sub-optimal when it comes to legibility and usability. And I wouldn't be surprised if forcing model detection with --model works neither.

The device protocol is sufficiently well-understood by now that we can write a protocol-level HX device emulator to run implementation tests against.

This should be a good weekend's day of work.

Hello, nice that there is some code for the HX870 radio.

/dev/ttyACM0

is there, when I connect the radio via USB to my Debian Linux box.

Could You provide a one liner to get the logger data?

I enabled the logger in the radio for some time to record the movement and now would like to get the logger data. Unfortunateley, Yaseu only provides the software for Windoze which I do not have. Thanks

When I run hxtool --help I'm getting the following syntax error. Any thoughts on what may be causing this? I'm Assuming its something in my python environment. This is on OS X Mojave. I have installed the latest python version from the python website. Thanks!

Traceback (most recent call last): File "/usr/local/bin/hxtool", line 9, in <module> load_entry_point('hxtool==0.2.1a0', 'console_scripts', 'hxtool')() File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 565, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 2697, in load_entry_point return ep.load() File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 2370, in load return self.resolve() File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 2376, in resolve module = __import__(self.module_name, fromlist=['__name__'], level=0) File "/Library/Python/2.7/site-packages/hxtool/__init__.py", line 19 logger.warning(f"Multiple devices detected, using {devices[0].tty}") ^ SyntaxError: invalid syntax

Close this once progress reporting has landed.

With my HX870E (firmware 2.04), decode_gps_log() fails to decode more than 201 GPS log points, even though the log is much longer than that.

It looks like the raw log data is chunked. Each chunk ends with twelve ff bytes (starting at offset 0xff4), causing the current implementation to break the decode loop and return after only the first chunk has been read.

Each chunk is 4 KiB long. Each chunk starts with 64 header bytes. 01 00 01 0b doesn’t look like a magic constant to me: The first byte is incremented each time the header is repeated.

These are the first 16 bytes in each of the header sections that my log currently contains:

0x0000:  01 00 01 0b 7f 00 00 00 3c 00 00 00 00 00 43 0b
0x1000:  02 00 01 0b 7f 00 00 00 05 00 00 00 00 00 79 0b
0x2000:  03 00 01 0b 7f 00 00 00 05 00 00 00 00 00 78 0b
0x3000:  04 00 01 0b 7f 00 00 00 05 00 00 00 00 00 7f 0b

The bitfield looks identical for the first three chunks, each beginning with 201 zeros. The last bitfield has 176 zeros. This adds up to 779, which is exactly the number of points my log currently contains.

The “unknown” data looks identical for the first three chunks with 00 0c 8c 14 for each. The last chunk has 00 fc 8c 1c.

In the wild we have seen HX870 handsets sending unexpected messages after EraseLog, resulting in the following error:

hxtool.protocol.ProtocolError: Unexpected EraseLog acknowledgement from device: $PMTK011,MTKGPS*08

See #24 for an error report.

According to https://cdn-shop.adafruit.com/datasheets/PMTK_A11.pdf, certain GPS chipsets send the following sentences after boot-up:

• $PMTK011,MTKGPS
• $PMTK010,001

This may be related to some HX firmware versions issuing a GPS module restart after EraseLog while others do not.

Have you tried entering waypoints from a GPX file into a hex dump? I'd like to enter a couple dozen waypoints into the radio and don't really feel like doing it manually...

The new CLI offers a -d flag, and so does the config command. As a result, for that command the meaning of -d depends upon the position of that flag in the command line:

hx870 -d config test.dat  # means --debug
hx870 config -d test.dat  # means --dump

While the former triggers an error because config requires either --dump or --flash, this situation seems confusing to me.

Personally, I’d much prefer --read/-r and --write/-w over --dump and --flash. I do understand that “read” and “write” are in theory ambiguous with regards to the direction of the data transfer (i. e. “write” might mean writing to the HX870’s flash memory, or writing to the computer’s hard drive), but in practice, when working with pyhx870, I think about the radio, not the computer. (YMMV, obviously.)

Besides, I’m not sure “dump” is any clearer (i. e., will it dump data into the HX870 or into a local file?).

#29 introduces a new issue with dumping elevation data to GPX:

  <trkpt lat="…" lon="…">
    <ele>0</ele>
    <time>2019-10-23T12:10:05Z</time>
  </trkpt>
  <trkpt lat="…" lon="…">
    <ele>65535</ele>
    <time>2019-10-23T12:10:10Z</time>
  </trkpt>

Looks like the elevation field is interpreted as unsigned when read from the device.

The LOCUS GPS log can contain multiple 4kB blocks, but this was unknown when the JSON log dump format was conceived. Consequently, fixing the log truncation bug in #35 resulted in a minor regression in that area, specifically that dumps of GPS logs with many entries spanning multiple blocks do not fully represent the binary in-memory format any longer.

At the moment, no workflows in the code rely on this requirement, so this issue is merely documenting that the format's bijectivity promise is broken for the time being.

Observed:

• when running hxtool gpslog with my device (HX890), I see the log reported as "11% full"
• looking at the GPS raw log, it looks more like 90% full (726 total trackpoints, out of an apparent capacity of 4 sectors * 201 trackpoints per sector = 804 trackpoints)

Expected: hxtool gpslog reports the log as "90% full" (unless I'm getting something else wrong)

Todo: continue logging until memory is reported as full by the device, and try again to confirm.

Spatial data serialisation using JSON should preferably follow the GeoJSON standard (RFC 7946), for which widespread support exists in mapping software such as QGIS or Leaflet.

Implementing GeoJSON log export by hand shouldn’t be too difficult. But apparently there are also multiple Python libraries available:
https://duckduckgo.com/?q=geojson+python

Go through all the files and prune unused code. Everything related to pyusb is dead code now and must go.

We currently use gpxpy for handling GPX files. Currently only GPX 1.0 is supported by hxtool. GPX 1.0 defines a fixed set of XML nodes with a fixed set of properties to cover the most widespread use cases. So far these cover well what we need to do in hxtool.

GPX 1.1 is a whole different story. It only implements a subset of GPX 1.0 which does not cover all of the data we can extract from an HX870/890 GPS log. For that, GPX 1.1 implements extensions which we could theoretically use for fully covering our log data export. However, those extensions are more or less free form and different GPX 1.1 consumers expect different property names and formats while GPX 1.1 producers are all over the place.

This makes GPX 1.1 a de-facto broken standard, as every big vendor uses it a bit differently and that makes them incompatble amongst each other. hxtool would end up targeting only one specific log consumer or having to support a multitude. Also gpxpy would not let me add the extensions I wanted last time I looked.

So I think that unless the GPX 1.1 standard finally, well, standardizes – and I don't expect this to happen, as major vendors revel in the user lock-in the current situation creates – we should remain with GPX 1.0 as long as it enjoys the universal support it currently does.

Please ping me if you can help with testing.

NMEA messages are trailed by varying appendices like *7D in the dump. I am suspecting that they may be HX message protocol checksums, but they could be part of the NMEA standard as well. If it's not standard, they need to be validated and stripped from the output.

This requires some quick NMEA spec digging for verification.

The config writer currently happily rewrites the flash ID at offset 0x100 with a mismatching value, but it should warn the user if they try and ask if they are --really sure.