Broken training file(s) about v800_downloader HOT 14 CLOSED

Hi @ababacz,

I have:

1. manually renamed the files that v800_downloader didn't get to (due to
   the parse error).
2. added a fake zero-byte -exercise--create file (ie would have come from
   the missing BASE.BPB file).
3. uncompressed the ROUTE and SAMPLE files (the other files were already
   raw protobuf, without compression)
4. truncated ROUTE and SAMPLE at 131895 and 200630 bytes respectively.
5. processed the session with Bipolar.

The resulting GPX file looks good. As do both HRM files. The TCX file is
essentially empty - unfortunately, it will be very hard to get the TCX
output without the proper exercise-create (ie BASE.BPB) file.

I've uploaded the GPX file to Strava - looks good. It works out to be
25.7km... so hopefully the watch didn't lose too much of your activity.

The resulting GPX, HRM and TCX files are attached in 7zip, zip, and
TAR/bzip2 formats. Hopefully one of those will work for you.

Cheers.

pc.

On Mon, Aug 3, 2015 at 8:29 PM Aleksander Babacz <
[email protected]> wrote:

Hello Paul,

I used both 7-zip and one more program (I cannot recall the name of it – I
have it on my second PC) however both of them gave me different results as
decompressed files have 2 different file sizes.
Attached ROUTE and SAMPLES are taken from 7-zip

Later one I will send you files from that second decompressor.

Thank you.

Pozdrawiam,
Aleksander Babacz
[email protected]
mob 603134040

From: Paul Colby [email protected]
Sent: Monday, August 03, 2015 11:44 AM
To: profanum429/v800_downloader [email protected]
Cc: ababacz [email protected]
Subject: Re: [v800_downloader] Broken training file(s) (#11)

Finaly i have ROUTE unpacked.

How did you manage to unpack it? Normally, the following should work:

cat ROUTE.GZB | gunzip | protoc --decode_raw

gzip: stdin: unexpected end of file
Failed to parse input.

Or if you've managed to unpack it first:

cat ROUTE | protoc --decode_raw
// lots of protocol buffer data here...

(note, the above examples are using the protoc compiler from the Google
Protocol Buffers project).

At this stage, I can't decompress the data (since gunzip won't recognise
it). But if you can (as you suggested above), then let me know how you did
so. Then we can see if the uncompressed data is valid protobuf data or not.

Cheers.

—
Reply to this email directly or view it on GitHub
#11 (comment)
.

from v800_downloader.

my V800 crashed during the training. I got FATAL ERROR on it.

That sucks! I know how important that data can be!

but v800_downloader gives me some error during conversion to TCX.

What's the actual error message?

from v800_downloader.

Hello,

The error is just:
20150730155424: Error making Bipolar names.

In the result program manages to download attached files (see attachment.
Directory is packed with zip and extension is changed to r__ to pass email
filters)

Inside the directory, i see key Route file which i'm even able to un-gzip.
Again, not all the decompressors are able to do it... Maybe file itself is
corrupted.

Finaly i have ROUTE unpacked. How may i concert it to any standard gps file
known for endo or strava? :)

Thank you in advance for help! :)

Pozdrawiam
Aleksander Babacz

my V800 crashed during the training. I got FATAL ERROR on it.

That sucks! I know how important that data can be!

but v800_downloader gives me some error during conversion to TCX.

What's the actual error message?

—
Reply to this email directly or view it on GitHub
#11 (comment)
.

from v800_downloader.

Disclaimer: I'm not all that familiar with the V800_download code... (other than the code shared with Bipolar, of course).

20150730155424: Error making Bipolar names.

It appears that this occurs when the V800export::make_bipolar_names function fails to rename files to match those expect by the Bipolar code.

Specifically, it appears to require all of the following files:

• TSESS.BPB
• PHYSDATA.BPB
• BASE.BPB
• ROUTE.GZB

Presumably one (or more) of these are missing due to the "FATAL ERROR" that occurred on the V800.

It will take an awful lot of work to convert the ROUTE.GZB to GPX by hand, so the best option is going to be use either V800_downloader or Bipolar, if possible, both probably requiring some modification to deal with the missing data. But it really does depend on which data is missing. For example, if either TSESS.BPB or BASE.BPB is missing, then its going to me much harder. Whereas, if PHYSDATA.PBP is missing, then it'll probably just mean some info would be missing from TCX output (not normally included in GPX anyway).

see attachment. Directory is packed with zip and extension is changed to r__ to pass email
filters

Unfortunately the attachment didn't come through (I suspect github doesn't allow email attachments, and if it did, I'm pretty sure they only allow images to be attached to issues anyway).

If you email the attachment directly to me, then I'd be happy to take a look sometime: git at colby dot id dot au

Cheers.

from v800_downloader.

Hello Paul,

here is mentioned archive (your personal emmail address id TO’ed)

Thank you for your help.

Pozdrawiam,
Aleksander Babacz
[email protected]
mob 603134040

From: Paul Colby
Sent: Sunday, August 02, 2015 11:35 PM
To: profanum429/v800_downloader
Cc: ababacz
Subject: Re: [v800_downloader] Broken training file(s) (#11)

Disclaimer: I'm not all that familiar with the V800_download code... (other than the code shared with Bipolar, of course).

20150730155424: Error making Bipolar names.

It appears that this occurs when the V800export::make_bipolar_names function fails to rename files to match those expect by the Bipolar code.

Specifically, it appears to require all of the following files:

a.. TSESS.BPB
b.. PHYSDATA.BPB
c.. BASE.BPB
d.. ROUTE.GZB
Presumably one (or more) of these are missing due to the "FATAL ERROR" that occurred on the V800.

It will take an awful lot of work to convert the ROUTE.GZB to GPX by hand, so the best option is going to be use either V800_downloader or Bipolar, if possible, both probably requiring some modification to deal with the missing data. But it really does depend on which data is missing. For example, if either TSESS.BPB or BASE.BPB is missing, then its going to me much harder. Whereas, if PHYSDATA.PBP is missing, then it'll probably just mean some info would be missing from TCX output (not normally included in GPX anyway).

see attachment. Directory is packed with zip and extension is changed to r__ to pass email
filters

Unfortunately the attachment didn't come through (I suspect github doesn't allow email attachments, and if it did, I'm pretty sure they only allow images to be attached to issues anyway).

If you email the attachment directly to me, then I'd be happy to take a look sometime: git at colby dot id dot au

Cheers.

—
Reply to this email directly or view it on GitHub.

from v800_downloader.

Got it. Thanks.

So it looks like the TSESS.BPB and PHYSDATA.BPB files were both present, and successfully renamed (to *-create and *-physical-information respectively). And ROUTE.GZB is obviously present, as you said.

The problem is that BASE.BPB is missing... the file provides all of the non-sample data for the actual exercise (the watch / flow.polar.com protocol regards a session as consisting of one or more exercises).

I'll have a look at what (if anything) can be done without the base / exercise data being present...

from v800_downloader.

Finaly i have ROUTE unpacked.

How did you manage to unpack it? Normally, the following should work:

cat ROUTE.GZB | gunzip | protoc --decode_raw 

gzip: stdin: unexpected end of file
Failed to parse input.

Or if you've managed to unpack it first:

cat ROUTE | protoc --decode_raw 
// lots of protocol buffer data here...

(note, the above examples are using the protoc compiler from the Google Protocol Buffers project).

At this stage, I can't decompress the data (since gunzip won't recognise it). But if you can (as you suggested above), then let me know how you did so. Then we can see if the uncompressed data is valid protobuf data or not.

Cheers.

from v800_downloader.

Hello Paul,

I used both 7-zip and one more program (I cannot recall the name of it – I have it on my second PC) however both of them gave me different results as decompressed files have 2 different file sizes.
Attached ROUTE and SAMPLES are taken from 7-zip

Later one I will send you files from that second decompressor.

Thank you.

Pozdrawiam,
Aleksander Babacz
[email protected]
mob 603134040

From: Paul Colby
Sent: Monday, August 03, 2015 11:44 AM
To: profanum429/v800_downloader
Cc: ababacz
Subject: Re: [v800_downloader] Broken training file(s) (#11)

Finaly i have ROUTE unpacked.

How did you manage to unpack it? Normally, the following should work:

cat ROUTE.GZB | gunzip | protoc --decode_raw

gzip: stdin: unexpected end of file
Failed to parse input.
Or if you've managed to unpack it first:

cat ROUTE | protoc --decode_raw
// lots of protocol buffer data here...
(note, the above examples are using the protoc compiler from the Google Protocol Buffers project).

At this stage, I can't decompress the data (since gunzip won't recognise it). But if you can (as you suggested above), then let me know how you did so. Then we can see if the uncompressed data is valid protobuf data or not.

Cheers.

—
Reply to this email directly or view it on GitHub.

from v800_downloader.

The bad news is that both the ROUTE and SAMPLE files are corrupt.

for f in ROUTE SAMPLES; do echo -n "$f: "; cat $f | protoc --decode_raw; done
ROUTE: Failed to parse input.
SAMPLES: Failed to parse input.

The good news, is that the corruption appears to be a truncation, so we should be able to get most of the samples...

for i in `seq 131902 -1 1`; do echo -n "$i: "; cat ROUTE | dd bs=1 count=$i 2>/dev/null | protoc --decode_raw >/dev/null && echo good && break; done
131902: Failed to parse input.
131901: Failed to parse input.
131900: Failed to parse input.
131899: Failed to parse input.
131898: Failed to parse input.
131897: Failed to parse input.
131896: Failed to parse input.
131895: good

for i in `seq 200635 -1 1`; do echo -n "$i: "; cat SAMPLES | dd bs=1 count=$i 2>/dev/null | protoc --decode_raw >/dev/null && echo good && break; done
200635: Failed to parse input.
200634: Failed to parse input.
200633: Failed to parse input.
200632: Failed to parse input.
200631: Failed to parse input.
200630: good

Now we'll either need to manually truncate the files to valid protobuf boundaries, or make the shared Bipolar code ignore trailing broken protobuf entries (if it doesn't already).

from v800_downloader.

Hmm.. not sure why github recorded the above comment as being from @ababacz ... it was from me ;)

Cheers.

from v800_downloader.

Hello Paul,

you are amazing! It works! Strava has successfully imported GPX file so
something from me left on few very nice segments ;)
Thank you for your time and efforts.
Do guys from Polar know about you and your skills? :)

Thank you and regards from Poland! :)

2015-08-04 1:50 GMT+02:00 Paul Colby [email protected]:

Hmm.. not sure why github recorded the above comment as being from
@ababacz https://github.com/ababacz ... it was from me ;)

Cheers.

—
Reply to this email directly or view it on GitHub
#11 (comment)
.

Pozdrawiam,
Aleksander Babacz
[email protected]
Mob. 603134040

from v800_downloader.

Hi @ababacz,

Strava has successfully imported GPX file

I'm glad we could get some data for you 😄 I know how incredibly frustrating it is to even come close to losing this sort of data!! I've come close a couple of times 😦

Thank you for your time and efforts.

You're very welcome. It was actually fun to get back into the protobuf level for a bit (I've been stuck at much higher / more abstract levels of software engineering lately).

Do guys from Polar know about you and your skills? :)

No idea. They are aware of the v800_downloader and Bipolar projects based on chatter on their forums 😉

Cheers.

from v800_downloader.

I've been away for the past few weeks, sorry I couldn't reply but it looks like Paul got things sorted out for you!

from v800_downloader.

Hello,

that is correct. Thank you. I close this issue.

from v800_downloader.