moderation / garmin-connect-export Goto Github PK

On Thursday 2018-04-05 the login to Garmin Gonnect has stopped working (the day before it still worked):

(Stacktrace from my develop branch

Traceback (most recent call last):
  File "./gcexport.py", line 201, in <module>
    hostname_url = http_req('http://connect.garmin.com/gauth/hostname')
...
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 558, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 503: Service Unavailable

There are some activities where there are no GPX data. For these activities you get this error:

Garmin Connect activity: [2753488155] Some Running
https://connect.garmin.com/modern/proxy/download-service/export/gpx/activity/2753488155?full=true
    Downloading file... Writing empty file since there was no GPX activity data...
Traceback (most recent call last):
  File "./gcexport3.py", line 454, in <module>
    write_to_file(data_filename, decoding_decider(data), file_mode)
  File "./gcexport3.py", line 128, in decoding_decider
    data = data.decode()
AttributeError: 'str' object has no attribute 'decode'

When trying to download the file, I get a return code 204

        Downloading file...
Traceback (most recent call last):
  File "garmin-connect-export_moderation/gcexport.py", line 357, in <module>
    data = http_req(download_url)
  File "garmin-connect-export_moderation/gcexport.py", line 86, in http_req
    raise Exception('Bad return code (' + str(response.getcode()) + ') for: ' + url)
Exception: Bad return code (204) for: https://connect.garmin.com/modern/proxy/download-service/export/gpx/activity/1539681677?full=true

Hi, sice some days the download did not work anymore. I saw that the script from pe-st is fixed and working again, seems that Garmin changed someting during login.
Thanks for yore work!

Regards
Hary

After using python3 gcexport3.py and logging in I get the error 'TypeError: the JSON object must be str, not 'bytes''

I have solved this issue by detecting the encoding :
Instead of LIST = json.loads(ACTIVITY_LIST) use:
LIST = json.loads(ACTIVITY_LIST.decode(chardet.detect(ACTIVITY_LIST)["encoding"]))

and instead of #JSON_SUMMARY = json.loads(ACTIVITY_SUMMARY) use:
JSON_SUMMARY = json.loads(ACTIVITY_SUMMARY.decode(chardet.detect(ACTIVITY_SUMMARY)["encoding"]))

you have to install and use the module chardet:
pip3 install chardet
add 'import chardet' to gcexport3.py

no backslashes before doublequotes and digits in the displayname value

ex:
,"displayName":"36e29d65-715c-456b-9115-84f0b9a0c0ba","

Worked for me with this edit in the script: (line 326)
PATTERN = re.compile(
r"."displayName":"([-.\w\d]+)".", re.MULTILINE | re.DOTALL
)

Not sure if this is due to a Garmin API change or a particular case on my account setup (did not find how to change this display name in my account)

After switching back to master here is still the same issue

Traceback (most recent call last):
  File "/Users/tomek/Garmin/garmin-connect-export//gcexport2.py", line 108, in <module>
    hostname_url = http_req('http://connect.garmin.com/gauth/hostname')
  File "/Users/tomek/Garmin/garmin-connect-export//gcexport2.py", line 82, in http_req
    response = opener.open(request, data=post)  # This line may throw a urllib2.HTTPError.
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 469, in error
    result = self._call_chain(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 656, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 475, in error
    return self._call_chain(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 558, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 404: Not Found

Exception: Did not get a ticket in the login response. Cannot log in. Did you enter the correct username and password?

Password and userid work fine in browser.

The current gcexport3.py has still the problem from pr #6.

• With LIMIT_MAXIMUM = 1000 I get a HTTP error 400 from the activity-search-service-1.2 endpoint

• With LIMIT_MAXIMUM = 100 I get a HTTP error 500 from the activity-search-service-1.2 endpoint

• When changing LIMIT_MAXIMUM to 18 or below I manage to retrieve about 7 activities, but then I get another 500, see this stacktrace:

  Activity details URL: https://connect.garmin.com/modern/proxy/activity-service-1.3/json/activityDetails/2620788460
  Traceback (most recent call last):
  File "gcexport3.py", line 388, in
  ACTIVITY_DETAIL = http_req(URL_GC_ACTIVITY_DETAIL + str(a['activity']['activityId']))

Hi!

Since a few days I get this error:

File "Z:\Sport\Script\gcexport.py", line 365, in
ACTIVITY_LIST = http_req(URL_GC_LIST + urllib.parse.urlencode(SEARCH_PARAMS))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "Z:\Sport\Script\gcexport.py", line 165, in http_req
response = OPENER.open(request, data=post)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\urllib\request.py", line 525, in open
response = meth(req, response)
^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\urllib\request.py", line 634, in http_response
response = self.parent.error(
^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\urllib\request.py", line 563, in error
return self._call_chain(*args)
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\urllib\request.py", line 496, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\urllib\request.py", line 643, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

Same Error with Python 3.10, has Garmin changed the Login?

Thanks for help.

Regards from Vienna
Hary

I find the url links end with .com in the scrips are not applicable to Chinese users. Our website is end with cn

I have replaced all the urls and logged in successfully, however, issue occurs when making request.

Finished authentication
Making activity request ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
https://connect.garmin.cn/proxy/activity-search-service-1.2/json/activities?start=0&limit=1
Traceback (most recent call last):
File "gcexport3.py", line 283, in
RESULT = http_req(URL_GC_SEARCH + urllib.parse.urlencode(SEARCH_PARAMS))
File "gcexport3.py", line 101, in http_req
response = OPENER.open((request), data=post)
File "D:\InstallFolder\Anaconda\lib\urllib\request.py", line 532, in open
response = meth(req, response)
File "D:\InstallFolder\Anaconda\lib\urllib\request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "D:\InstallFolder\Anaconda\lib\urllib\request.py", line 570, in error
return self._call_chain(*args)
File "D:\InstallFolder\Anaconda\lib\urllib\request.py", line 504, in _call_chain
result = func(*args)
File "D:\InstallFolder\Anaconda\lib\urllib\request.py", line 650, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

It seems the URL_GC_SEARCH is not applicable. Do you know how to get the right link for this issue.

Thanks

I have tested code from "activity search url " branch. It worked OK, as we talking about HTTP Errors.
But there is a problem connected with activities without 'distance' - as Yoga, or Aerobic

Traceback (most recent call last):
  File "/Users/tomek/Garmin/garmin-connect-export//gcexport.py", line 463, in <module>
    print "{0:.3f}".format(a['distance']/1000)
TypeError: unsupported operand type(s) for /: 'NoneType' and 'int'

Garmin now has 2FA implemented. Is there a way to use the script with 2FA enabled?

I'am getting this error since today:
Traceback (most recent call last):
File "gcexport.py", line 318, in
print '\t' + a['activity']['activitySummary']['BeginTimestamp']['display'] + ',',
KeyError: 'activitySummary'

Thanks for your great Too!

Thanks for this script - yours was the only fork that worked for me, after I'd tried about 5! I was able to download all my activities but had to restart the script lots of times...

I was running it with the options --format original and --unzip and received the error message:

Garmin Connect activity: [12345678] Untitled swim
	Mon, 10 Jan 2011 6:30 PM, ??:??:??, 0.80 Kilometers
	Downloading file... Writing empty file since there was no original activity data... Unzipping and removing original files...
Traceback (most recent call last):
  File "gcexport-moderation.py", line 464, in <module>
    z = zipfile.ZipFile(zip_file)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/zipfile.py", line 770, in __init__
    self._RealGetContents()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/zipfile.py", line 811, in _RealGetContents
    raise BadZipfile, "File is not a zip file"
zipfile.BadZipfile: File is not a zip file

So the script fails when it tried to unzip the 0Kb zip file for activities where there is no original file. E.g. manually created activities, such as my swims!

Once the empty zip file is there though, the next time I ran the script it would keep going past this point.

Appreciated it's in the Description:

If the original format is used, Garmin may not provide a file at all and an empty file will be created

But perhaps the script could be amended to not try and unzip 0Kb files? Or some other suitable approach to enable it to keep going. I don't know Python and have downloaded all my files with the manual workaround, but it would be beneficial for future users 👍

Hi again. Unfortunately I identified another issue in python3 version. Maybe it's connected with local characters?

Garmin Connect activity: [2632311679] Aerobics, general - MyFitnessPal
	Śr, kwi 18, 2018 2:55 PM, 	Downloading file... Traceback (most recent call last):
  File "/Users/tomek/Garmin/garmin-connect-export//gcexport3.py", line 378, in <module>
    write_to_file(data_filename, data.decode(), file_mode)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x85 in position 11: invalid start byte

$ /Users/xxx/bin/gcexport2.py -d /Users/xxx/Dropbox/Garmin -c 1 -f original -u --username xxxx --password xxxxx
Welcome to Garmin Connect Exporter!
Warning: Output directory already exists. Will skip already-downloaded files and append to the CSV file.
Traceback (most recent call last):
File "/Users/xxx/bin/gcexport2.py", line 108, in
hostname_url = http_req('http://connect.garmin.com/gauth/hostname')
File "/Users/xxx/bin/gcexport2.py", line 82, in http_req
response = opener.open(request, data=post) # This line may throw a urllib2.HTTPError.
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 437, in open
response = meth(req, response)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 469, in error
result = self._call_chain(*args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 656, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 437, in open
response = meth(req, response)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 475, in error
return self._call_chain(*args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 558, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 404: Not Found

If the display name in garmin connect contains any "." (dot), like "John.Doe", script ends with:

Traceback (most recent call last):
  File "gcexport3.py", line 327, in <module>
    raise Exception("Did not find the display name in the profile page.")
Exception: Did not find the display name in the profile page.

Hi,

I have some activities imported into my Garmin Connect account from GPX files (generated with Endomondo/MyTracks before I bought my Garmin device). In this case the original file is a GPX file and not FIT. It means that isfile(fit_filename) will never be true and the script will re-download the already existing file.
I can't see any information in the json response which would help identifying the original activity file, so it seems to be impossible to fix. I just wanted you to know this problem.

Best regards,
kukukk

When executing multiple downloads I would run into WIN ERROR - File in use and also http incomplete read errors

I added an

import time
and in the list loop I added a time.sleep(3) delay.

for a in LIST:
# Display which entry we're working on.
time.sleep(3)
print("Garmin Connect activity: [" + str(a["activityId"]) + "]", end=" ")

The delay cured all the error I ran into and have just completed a 150 file download without problem

Hi, another error case for you - in case you are interested... I will need to fix the activity itself I think...

https://connect.garmin.com/modern/activity/15254925

Looks like I didn't stop the Garmin after the activity and it's moving time is ~2.5hrs vs ~46hrs elapsed!

The Python scripts exits with:

Garmin Connect activity: [15254925] North Downs - Leith Hill and Holmbury Hill
	Fri, 10 Apr 2009 12:46 PM, 46:17:06, 21.00 Kilometers
	Downloading file...
Traceback (most recent call last):
  File "./gcexport.py", line 370, in <module>
    raise Exception('Failed. Got an unexpected HTTP error (' + str(e.code) + download_url +').')
Exception: Failed. Got an unexpected HTTP error (403http://connect.garmin.com/proxy/download-service/files/activity/15254925).

Going to that URL is the same as clicking 'Export original' from the activities settings icon and gives:

{"message":null,"error":"WebApplicationException"}

I guess because either the file/activity is corrupt or just too large.

I hope that cropping the original file (if I can find it!) and re-uploading will solve the issue for me, but just in case you wanted to handle this exception I thought I'd report it.

Adrian.