stefal / mapillary_download Goto Github PK

downloading.. this process will take a while. please wait
Traceback (most recent call last):
  File "mapillary_download.py", line 148, in <module>
  File "os.py", line 215, in makedirs
  File "os.py", line 225, in makedirs
OSError: [Errno 5] Input/output error: 'data'
[28073] Failed to execute script 'mapillary_download' due to unhandled exception!

The error appeared following a request for 21 sequences, it was the first time that I exceeded 20.
So I thought it was too much, but the error persisted when I removed some id sequences...
I also thought it was a disk space problem, but that is far from the case since there was 4.5 TB free.
Nor even a problem with the right to create folders, I created and deleted other test folders in the open shell and I had used this same window for the same script for some time before.

Downloading stops on some sequences, on a specific photo it seems. I tried to restart the download and sometime it download some additional photos on the first attempt but then the download stops, maybe not always on the same photo cause the download doesn't seem to follow an order.
The message is the following:

Traceback (most recent call last):
  File "mapillary_download.py", line 162, in <module>
KeyError: 'camera_type'
[21287] Failed to execute script 'mapillary_download' due to unhandled exception!

It appends on several sequences like this one: a1eg384w8wcw1ybyw3shju

When the token is bad or outdated, there is no error checking and thus...

13 images in sequence 1 of 1  - id : 6IhCFqv7N5bokwu3VWnPiH
getting images data
downloading.. this process will take a while. please wait
Traceback (most recent call last):
  File "/home/cquest/src/mapillary_download/download.py", line 139, in <module>
    date_time_image_filename = datetime.utcfromtimestamp(int(image_data['captured_at'])/1000).strftime('%Y-%m-%d_%HH%Mmn%Ss%f')[:-3]
KeyError: 'captured_at'

Here is what image_data contained:
{'error': {'message': 'Veuillez vous connecter pour voir cette page.', 'type': 'OAuthException', 'code': 368, 'error_data': {'sentry_block_data': 'Aej24p-bf0W3cKKCcH26Nz_zQr1Ozs33o96R-wXWh5WJEV5eo275rB1rZqS_Fx7uPlLY2dbq1EGUW47X5ubYLN2OCYhllLAfJvKTAf7piOVy9lNoAFNRhAplswNLF7ogt6UXS3bHHIxV9BHPnkzuWhkoJE_cVmB8G8ye0HRhVIY0r8f3p6qtUlGKDBIDIfAHGZocVUERYLszDBX86JjlznuAC6O9awzxs9V1AKUlwlU_PBQcNMZmUkD_9_ROKPDMCMkLfNmpcdM8ZLZuadkZSgIBfu8AlH--4TBbd91woihdyuCPASsDwut8sYs_K1Y8eTDKewlwNfPx7ZD2CT5rI7M44kt6XRP87-jUi9r1NZsZM0NT7-gZ8KsaJMIdCbmGGvZju2YAhax9APRJqVkZFL9H', 'help_center_id': 0, 'is_silent': False}, 'error_subcode': 1348007, 'error_user_msg': '', 'fbtrace_id': 'AyM--aqHldjdS_IpeXNw2Vv'}, 'sequence_id': 'JN12Zqej4b5cTELxoCOtzX'}

Inside writer.py localize function can't work on image without existing coordinates, and won't use new but not written coordinates.

https://github.com/Stefal/mapillary_download/blob/66d49c001b5d9aa5e33fa6d68f8b515ceb9ab0cd/writer.py#L184C1-L208C1

Right now, download.py download images in a data subfolder. We should add an argument to let the user specify the destination.

Personally, I use YYYYMMDD-HHmmss as shorter format for Date-Time, so in the present application I suggest to change

strftime('%Y-%m-%d_%HH%Mmn%Ss%f')[:-3] + '.jpg'
to
strftime('%Y%m%d-%H%M%S:%f')[:-3] + '.jpg'

Example:
20161011-152350:987.jpg

If you like I can send a regular pull request for this small change.

In case anyone needs a script for creating a video, and a deshaked video, here is my version of getmapillary as a proposal.

You need to insert your API credentials into the script.

#!/usr/bin/bash

# script for downloading a mapillary image sequence
#
# postprocessing: creates a deshaked video
#
# init 20240302
#

if [ "$#" -eq 1 ]; then

function deshake {
	# pass1
	ffmpeg -i "$1" -vf vidstabdetect=result="$1.trf" -f null -
	# pass2
	ffmpeg -i "$1" -vf vidstabtransform=input="$1.trf":crop=black:smoothing=20,unsharp=5:5:0.8:3:3:0.4 "${1%.*}-deshaked.mp4"
	# rm "$1"
	rm "$1.trf"
}

# $1 sequenceid
# $2 outdir
# $3 size 640x360 | 960x540 | 1920x1080

function makevideo {
	inrate=1
	outrate=25
	outfile=$2/$1/$1-$3.mp4

	ffmpeg -framerate $inrate -pattern_type glob -i "$2/$1/*.jpg" -filter:v minterpolate -vf scale=$3 -c:v libx264 -pix_fmt yuv420p -r $outrate $outfile
	deshake $outfile

}

outdir=/var/www/html/mapillary
source /opt/mapillary_download/venv/bin/activate
python /opt/mapillary_download/mapillary_download.py "$APIKEYCREDENTIALS" --destination $outdir --sequence_ids $1
deactivate


makevideo $1 $outdir 640x360
# makevideo $1 $outdir 960x360
# makevideo $1 $outdir 1920x1080

else
	echo "Usage:"
	echo "getmapillary <imagesequence-id>"
fi