olivierhagolle / sentinel-download Goto Github PK

Hello,

are there any known workarounds or attempts to run this script on windows?

wget --no-check-certificate --user="guest" --password="guest" --output-document=query_results.xml "https://scihub.copernicus.eu/apihub/search?q=footprint:"Intersects(POLYGON((-23.0 51.75, 23.75 51.75, 23.75 52.25, -23.0 52.25,-23.0 51.75)))" filename:S1AGRDH&rows=100"

'wget' is not recognized as an internal or external command,
operable program or batch file.
Traceback (most recent call last):
File "Sentinel_download.py", line 279, in
xml = minidom.parse("query_results.xml")
File "C:\Users\vaak\AppData\Local\conda\conda\envs\gdal\lib\xml\dom\minidom.py", line 1918, in parse
return expatbuilder.parse(file)
File "C:\Users\vaak\AppData\Local\conda\conda\envs\gdal\lib\xml\dom\expatbuilder.py", line 922, in parse
fp = open(file, 'rb')
IOError: [Errno 2] No such file or directory: 'query_results.xml'

Hi Olivier,

I am experiencing problems with downloading S2A data on a per tile basis (-t). All elements of the .SAFE folder are downloading fine, but the general .xml file in the root of the S2A image .SAFE folder is completely blank. This is the case for all S2A images I am trying to download. I tried typing in the url link to the server in my browser to download the .xml manually, and an "error 404 not found" shows up.

Here is the command I use (linux CentOS):
python Sentinel_download.py --lat 50.42 --lon 6.02 -a apihub.txt -w /data -o 8 -t 31UES

Could it be that the the location of the .xml file on scihub has changed and the link that your script points to is outdated?

Thanks for you script in any case, a life saver :)

Regards,

William

Hi Oliver, not really an issue, a suggestion:

extending line 193 with checking for darwin, makes the script compatible for OSX as well:
193: if sys.platform.startswith('linux') or sys.platform.startswith('darwin'):

cheers, Uwe

Hi,

The script for getting data from 1. April 2017 to the 1. November 2017 downloads me only data for October and a few Septembers.
This is the code I'm running:
python Sentinel_download.py -s S2 -m 50 --latmin 46.3 --latmax 46.7 --lonmin 13.8 --lonmax 14.1 -a apihub.txt -d 20170401 -f 20171101 -w /home/alen/Downloads/sentinel2

When trying to get just data from April to May, there is an error.
I run:
python Sentinel_download.py -s S2 -m 50 --latmin 46.3 --latmax 46.7 --lonmin 13.8 --lonmax 14.1 -a apihub.txt -d 20170401 -f 20170501 -w /home/alen/Downloads/sentinel2
And I get:
`wget --no-check-certificate --user="XXX" --password="XXX" --output-document=query_results.xml "https://scihub.copernicus.eu/apihub/search?q=footprint:\"Intersects(POLYGON((13.8 46.3, 14.1 46.3, 14.1 46.7, 13.8 46.7,13.8 46.3)))" filename:S2* producttype:S2MSI1C &rows=100"
--2018-02-01 15:23:42-- https://scihub.copernicus.eu/apihub/search?q=footprint:%22Intersects(POLYGON((13.8%2046.3,%2014.1%2046.3,%2014.1%2046.7,%2013.8%2046.7,13.8%2046.3)))%22%20filename:S2*%20producttype:S2MSI1C%20&rows=100
Resolving scihub.copernicus.eu (scihub.copernicus.eu)... 131.176.236.28
Connecting to scihub.copernicus.eu (scihub.copernicus.eu)|131.176.236.28|:443... connected.
HTTP request sent, awaiting response... 401 Unauthorized
Authentication selected: Basic realm="Sentinels Scientific Data Hub Search"
Reusing existing connection to scihub.copernicus.eu:443.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/xml]
Saving to: ‘query_results.xml’

query_results.xml [ <=> ] 288,43K --.-KB/s in 0,1s

2018-02-01 15:23:47 (2,78 MB/s) - ‘query_results.xml’ saved [295353]
`

I am running the code in Linux Mint 18.3. I tried to do it also without -w.
The listing -n from 1.4.2017 to 1.11.2017 stop with the dataset from the 28.9.2017.

What's going on, could somebody help me please? :)

Hello Olivier,

when I am trying to run the following line in the command line I always receive "error with the password file". As a result the problem is between line 160 - 168. What I figured out ithat if I add a print statement <<(print ("error"))>> before the code <<f=file(options.apihub)>> line 161 and I run the following code in the command line <<Sentinel_download.py --lat 43.6 --lon 1.44 -a apihub>> I get in my screen the following:

However, if I add a print statement after the <<f=file(options.apihub)>>, it never appears on my screen. So, I believe the line <<f=file(options.apihub)>> does not work.

Any help would be much appreciated. Thank you!!

Regards,
Andronikos

Hi,

Any chance you can help me with this?

sentinel search --sentinel2 -s 20160201 -e 20160211 <u> <p> vienna_bbox.geojson

Error: API returned unexpected response 401 .
Traceback (most recent call last):
  File "/usr/local/bin/sentinel", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/sentinelsat/scripts/cli.py", line 99, in search
    for product in api.get_products():
  File "/usr/local/lib/python2.7/site-packages/sentinelsat/sentinel.py", line 129, in get_products
    raise ValueError('API response not valid. JSON decoding failed.')
ValueError: API response not valid. JSON decoding failed.

Thanks!

Salut,

ligne 283 dans Sentinel_download.py:
ident=prod.getElementsByTagName("id")[0].firstChild.data
dead code: ident n'est jamais utilisé, qui plus est, ça génère une erreur :

ident=prod.getElementsByTagName("id")[0].firstChild.data
AttributeError: 'NoneType' object has no attribute 'data'

En effet, il semblerait que le champ id n'ait pas toujours de fils, du moins dans l'exemple suivant:
(extrait de query_results.xml)

... <entry> <title>S2A_MSIL1C_20171004T153621_N0205_R068_T18QWF_20171004T153616</title> <link href="https://scihub.copernicus.eu/apihub/odata/v1/Products('b74bc209-b6a7-4161-8862-16103f49ed2d')/$value"/> <link href="https://scihub.copernicus.eu/apihub/odata/v1/Products('b74bc209-b6a7-4161-8862-16103f49ed2d')/" rel="alternative"/> <link href="https://scihub.copernicus.eu/apihub/odata/v1/Products('b74bc209-b6a7-4161-8862-16103f49ed2d')/Products('Quicklook')/$value" rel="icon"/> <id>b74bc209-b6a7-4161-8862-16103f49ed2d</id> ...
Commenter la ligne suffit à empêcher que ça replante.

Cordialement

Hello Olivier, I was wondering if it is an issue or it is how it should work, but the script does not query Sentinel 2 B images.

Thanks,

Gonzalo

          I have implemented again possibility to make queries with constraints on ingestion date
          python  Sentinel_download.py --lat 43.6 --lon 1.44 -a apihub.txt -o 51 --id 2016-11-01 --if 2016-12-01

Originally posted by @olivierhagolle in #16 (comment)

Hi @olivierhagolle,
I am using the above code for downloading images between specific date ranges, but there are no files getting downloaded, although it is detecting some files in that ranges. Can it be because the images are offline in the Open Access Hub?
Is there any alternate method to download such files?

===============================================
20181218 20150613 20230306
S2A_MSIL2A_20181218T052221_N0211_R062_T43QEU_20181218T074117.SAFE
https://apihub.copernicus.eu/apihub/odata/v1/Products('c4ab8da3-c01f-4a95-b2ec-2f686dd7b85d')/\$value
cloud percentage = 32.89 %
===============================================
wget --no-check-certificate --continue --output-document=./Products/S2A_MSIL2A_20181218T052221_N0211_R062_T43QEU_20181218T074117.SAFE.zip "https://apihub.copernicus.eu/apihub/odata/v1/Products('c4ab8da3-c01f-4a95-b2ec-2f686dd7b85d')/\$value"
--2023-03-06 17:37:05-- https://apihub.copernicus.eu/apihub/odata/v1/Products('c4ab8da3-c01f-4a95-b2ec-2f686dd7b85d')/$value
Resolving apihub.copernicus.eu (apihub.copernicus.eu)... 146.59.215.225
Connecting to apihub.copernicus.eu (apihub.copernicus.eu)|146.59.215.225|:443... connected.
HTTP request sent, awaiting response... 401
Authentication selected: Basic realm="OData service"
Reusing existing connection to apihub.copernicus.eu:443.
HTTP request sent, awaiting response... 202
Length: 0 [text/plain]
Saving to: ‘./Products/S2A_MSIL2A_20181218T052221_N0211_R062_T43QEU_20181218T074117.SAFE.zip’
./Products/S2A_MSIL2A_20181218T052221_N0211_R06 [ <=> ] 0 --.-KB/s in 0s
2023-03-06 17:37:07 (0.00 B/s) - ‘./Products/S2A_MSIL2A_20181218T052221_N0211_R062_T43QEU_20181218T074117.SAFE.zip’ saved [0/0]

Thank you
Girish

Bonjour,

J'ai constaté qu'utiliser l'argument -o (orbit) empêche le script de télécharger les données S1.

Dans le fichier query_results.xml, on retrouve la variable :

filename:S1ASLCR161*

Or les fichiers S1 ont des noms du format suivant désormais (exemple), l'information relative à l'orbite n'apparaît plus dans le nom du fichier :

S1A_IW_SLC__1SDV_20180703T173124_20180703T173151_022633_0273C0_7F06

Cordialement,
Eric

Hello,

I'm trying to run Sentinel-download on my windows machine.
When I try python Sentinel_download.py --lat 43.6 --lon 1.44 -a apihub.txt -o 51 -s S2 i get an error query_results.xml not found and wget is not recognized as an internal or external command

Do I have to change anything if I want to use aria2 instead of wget?

Hi Olivier,
Thanks for your code! It helps a lot. when I tried downloading the S1A SLC product, I found this issue.
no matches found: S1A*SLC

Hello Olivier! my question is next, I am trying to get a query of all available images of S2 for a point, it works correctly but the latest image is missing, ie, (did it on 19/Jul) for point lon -61.52651 lat -35.76756 I get that the las available is 2017-06-16 but in the site If I check the same spot I get that the last is on 2017-07-06 and I could not find on the Sentinel site anything about it.

Let me know if you have some clue about it.

Thanks!

Gonzalo

hello, I have cloned this repo on both a windows 10 environment and an Ubuntu environment (through Docker). On the windows environment, I have ensured that both wget and aria2 are installed. This is the output of that command (this is Git Bash, but same as CMD) and the contents of the XML file. After running the same command (not specifying --downloader aria2) on the Ubuntu environment, I get results from the search. Why would it not work on windows??

<?xml version="1.0" encoding="utf-8"?><feed xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/" xmlns="http://www.w3.org/2005/Atom">
<title>Sentinels Scientific Data Hub search results for: footprint:Intersects(39.982811,-105.469371) filename:S2* producttype:S2MSI1C </title>
<subtitle>Displaying  results. Request done in 0.001 seconds.</subtitle>
<updated>2018-01-23T22:59:39.517Z</updated>
<author>
<name>Sentinels Scientific Data Hub</name>
</author>
<id>https://scihub.copernicus.eu/apihub/search?q=footprint:Intersects(39.982811,-105.469371) filename:S2* producttype:S2MSI1C </id>
<opensearch:totalResults/>
<opensearch:startIndex>0</opensearch:startIndex>
<opensearch:itemsPerPage>100</opensearch:itemsPerPage>
<opensearch:Query role="request" searchTerms="footprint:Intersects(39.982811,-105.469371) filename:S2* producttype:S2MSI1C " startPage="1"/>
<link rel="self" type="application/atom+xml" href="https://scihub.copernicus.eu/apihub/search?q=footprint:Intersects(39.982811,-105.469371) filename:S2* producttype:S2MSI1C &amp;start=0&amp;rows=100"/>
<link rel="first" type="application/atom+xml" href="https://scihub.copernicus.eu/apihub/search?q=footprint:Intersects(39.982811,-105.469371) filename:S2* producttype:S2MSI1C &amp;start=0&amp;rows=100"/>
<link rel="last" type="application/atom+xml" href="https://scihub.copernicus.eu/apihub/search?q=footprint:Intersects(39.982811,-105.469371) filename:S2* producttype:S2MSI1C &amp;start=NaN&amp;rows=100"/>
<link rel="search" type="application/opensearchdescription+xml" href="opensearch_description.xml"/>
</feed>

I download the codes of Sentinel_download and try it in windows 10,I have installed aria2c,but I run into questions,it doesn't work in windows 10?What's the solutions?
C:\sentinel2_python_erdas\Sentinel_download_by_python>python ./Sentinel_download.py --lat 43.6 --lon 1.44 -a apihub.txt -o 51 --id 2016-11-01 --if 2016-12-01 --downloader aria2
aria2c --check-certificate=false --http-user="guest" --http-passwd="guest" --continue -o query_results.xml "https://scihub.copernicus.eu/apihub/search?q=footprint:"Intersects(43.600000,1.440000)" filename:S2R051 ingestiondate:[2016-11-01T00:00:00.000Z TO 2016-12-01T23:59:50.000Z] producttype:S2MSI1C &rows=100"
'aria2c' is not recognized as an internal or external command,
operable program or batch file.
Traceback (most recent call last):
File "./Sentinel_download.py", line 282, in
xml=minidom.parse("query_results.xml")
File "C:\Python27\ArcGIS10.5\lib\xml\dom\minidom.py", line 1918, in parse
return expatbuilder.parse(file)
File "C:\Python27\ArcGIS10.5\lib\xml\dom\expatbuilder.py", line 922, in parse
fp = open(file, 'rb')
IOError: [Errno 2] No such file or directory: 'query_results.xml'

Hello Olivier,

Thank you very much for creating this tool!

I have the issue that my internet connection is very slow, so I want to minimise the amount of bites send/received. That's why I only use the -t option.
When you try to download a complete download for the second time, it is skipped. But this is not working when using the tiles/granule option.

Gert

Hi there,

Great script:

My cmd:

ubuntu@ubuntu:~$ Sentinel_download.py --latmin 0.904282606986 --latmax -0.90428260986 --lonmin 34.797275229 --lonmax 36.592770943 -a '/home/ubuntu/anaconda3/bin/apihub.txt' -m 20 --start_date 20151010 --end_date 20161230 --tile 36MZE -w /media/ubuntu/HD-PZFU3/Kenya/S2_training/L1C_tiles_mau/ --dhus -r 200

It seems to run fine but exits eventually with this error (although it has successfully downloaded plenty granules).

HTTP request sent, awaiting response... 502 Bad Gateway 2016-08-16 16:00:09 ERROR 502: Bad Gateway

Traceback (most recent call last): File "/home/ubuntu/anaconda3/envs/python2/bin/Sentinel_download.py", line 361, in <module> get_dir('DATASTRIP',url_datastrip_dir,product_dir_name,wg,auth,wg_opt,value) File "/home/ubuntu/anaconda3/envs/python2/bin/Sentinel_download.py", line 67, in get_dir download_tree(product_dir_name+'/'+dir_name,"temp.xml",wg,auth,wg_opt,value) File "/home/ubuntu/anaconda3/envs/python2/bin/Sentinel_download.py", line 50, in download_tree download_tree(nom_rep,'files.xml',wg,auth,wg_opt,value) File "/home/ubuntu/anaconda3/envs/python2/bin/Sentinel_download.py", line 40, in download_tree urls,types,names=get_elements(xml_file) File "/home/ubuntu/anaconda3/envs/python2/bin/Sentinel_download.py", line 29, in get_elements line=fic.readlines()[0].split('<entry>') IndexError: list index out of range
`
This happens every time it runs. Do you know why this may be? I boosted the limit on number of images to download just in case although I admit this is not likely related to the error.

Thanks

I am trying to download Sentinel images over the UK using the command:
python Sentinel_download.py --lat 52.192620 --lon -2.239065 -a apihub.txt -d 2016-01-01 -s S2

It results in the error message "IndexError: list index out of range" from line 281 in Sentinel_Download.py. It appears to be the negative longitude value that is causing the troubles, since it works flawlessly when using positive longitude values. I have also tried with specifying --latmin and --lonmin, with the same result.

Hi Olivier,
When the aria2-argument is provided in the request, e.g.
python Sentinel_download.py --downloader aria2 -s S1*IW_SLC --lat 39 --lon 68.3 -a apihub.txt

the resulting call to aria2c does not consider apihub.txt:
aria2c --check-certificate=false --http-user="guest" --http-passwd="guest" --continue -o query_results.xml "https://scihub.copernicus.eu/apihub/search?q=footprint:\"Intersects(39.000000,68.300000)\" filename:S1IW_SLC&rows=100"

The "guest" account does not seem to work any more, I get an authentication error (code 24).

Best wishes,
David

Anyone had this problem?
import os is at the start of the script so no idea why it's not defined....

python Sentinel_download.py --lat 43.6 --lon 1.44 --downloader aria2 -a apihub.txt -o 51 -s S2

Traceback (most recent call last):
File "Sentinel_download.py", line 178, in
# prepare wget command line to search catalog
NameError: name 'os' is not defined

I'm trying to download subset of the Sentinel-1 data to a specific folder.
It seems that Sentinel-download.py returns some query results, but do not start download.

Is it a bug or should I manually use query_results.xml to download my files?
And why in the messages there's an "401 Unauthorized" if it gets some results?

git clone https://github.com/olivierhagolle/Sentinel-download.git


Cloning into 'Sentinel-download'...
remote: Counting objects: 261, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 261 (delta 0), reused 0 (delta 0), pack-reused 258
Receiving objects: 100% (261/261), 78.58 KiB | 0 bytes/s, done.
Resolving deltas: 100% (129/129), done.
Checking connectivity... done.


cd Sentinel-download/

python Sentinel_download.py --latmin 51.75 --latmax 52.25 --lonmin -23.00 --lonmax 23.75 -a apihub.txt -s S1A*GRDH -w /home/mateuszk/Testy -d 20160508


wget --no-check-certificate --user="guest" --password="guest" --output-document=query_results.xml "https://scihub.copernicus.eu/apihub/search?q=footprint:\"Intersects(POLYGON((-23.0 51.75, 23.75 51.75, 23.75 52.25, -23.0 52.25,-23.0 51.75)))\" filename:S1A*GRDH*&rows=100"
--2016-06-23 13:27:16--  https://scihub.copernicus.eu/apihub/search?q=footprint:%22Intersects(POLYGON((-23.0%2051.75,%2023.75%2051.75,%2023.75%2052.25,%20-23.0%2052.25,-23.0%2051.75)))%22%20filename:S1A*GRDH*&rows=100
Translacja scihub.copernicus.eu (scihub.copernicus.eu)... 131.176.236.28
Łączenie się z scihub.copernicus.eu (scihub.copernicus.eu)|131.176.236.28|:443... połączono.
Żądanie HTTP wysłano, oczekiwanie na odpowiedź... 401 Unauthorized
Wybrano uwierzytelnienie: Basic realm="Sentinels Scientific Data Hub Search"
Łączenie się z scihub.copernicus.eu (scihub.copernicus.eu)|131.176.236.28|:443... połączono.
Żądanie HTTP wysłano, oczekiwanie na odpowiedź... 401 Unauthorized

Uwierzytelnienie użytkownik/hasło nie powiodło się.
Traceback (most recent call last):
  File "Sentinel_download.py", line 228, in <module>
    xml=minidom.parse("query_results.xml")
  File "/usr/lib64/python2.7/xml/dom/minidom.py", line 1918, in parse
    return expatbuilder.parse(file)
  File "/usr/lib64/python2.7/xml/dom/expatbuilder.py", line 924, in parse
    result = builder.parseFile(fp)
  File "/usr/lib64/python2.7/xml/dom/expatbuilder.py", line 211, in parseFile
    parser.Parse("", True)
xml.parsers.expat.ExpatError: no element found: line 1, column 0



echo Now I will edit apihub.txt to provide credentials to my SciHub account
Now I will edit apihub.txt to provide credentials to my SciHub account


gedit apihub.txt


python Sentinel_download.py --downloader aria2 --latmin 51.75 --latmax 52.25 --lonmin -23.00 --lonmax 23.75 -a apihub.txt -s S1A*GRDH -w /home/mateuszk/Testy -d 20160508
wget --no-check-certificate --user="MyAccountName" --password="myPasswordWithSpecialCharactersSuchAs!and@" --output-document=query_results.xml "https://scihub.copernicus.eu/apihub/search?q=footprint:\"Intersects(POLYGON((-23.0 51.75, 23.75 51.75, 23.75 52.25, -23.0 52.25,-23.0 51.75)))\" filename:S1A*GRDH*&rows=100"
--2016-06-23 13:31:35--  https://scihub.copernicus.eu/apihub/search?q=footprint:%22Intersects(POLYGON((-23.0%2051.75,%2023.75%2051.75,%2023.75%2052.25,%20-23.0%2052.25,-23.0%2051.75)))%22%20filename:S1A*GRDH*&rows=100
Translacja scihub.copernicus.eu (scihub.copernicus.eu)... 131.176.236.28
Łączenie się z scihub.copernicus.eu (scihub.copernicus.eu)|131.176.236.28|:443... połączono.
Żądanie HTTP wysłano, oczekiwanie na odpowiedź... 401 Unauthorized
Wybrano uwierzytelnienie: Basic realm="Sentinels Scientific Data Hub Search"
Łączenie się z scihub.copernicus.eu (scihub.copernicus.eu)|131.176.236.28|:443... połączono.
Żądanie HTTP wysłano, oczekiwanie na odpowiedź... 200 OK
Długość: nieznana [application/atom+xml]
Zapis do: `query_results.xml'

query_results.xml       [  <=>               ] 248,48K  1,12MB/s    in 0,2s    

2016-06-23 13:31:38 (1,12 MB/s) - zapisano `query_results.xml' [254447]

77b079c1-084c-4a90-81e5-2aeb36eec0ce
bf219593-4c0c-4c9c-ae07-87050d9665ed
7f826694-07df-4f5c-a046-1a7eccfe86ac
8ae33ebf-8e3c-4e51-9592-8c6df5e76b2d
47de62c1-e8ac-48bb-bc34-c5cea9990670
a3aececd-3837-4f93-89a9-382b8bfe70d8
25a7455e-b4b1-490e-af17-22f4a0a6b835
344a752d-06ad-4a2e-8cb4-554a39bf717f
39f14dc7-357f-48b8-89ef-3f5ae67591fa
61c80884-afe1-40c9-b409-df5cd09fda50
178d8651-3086-42ed-af06-2d2f2afc484f
bfa88079-38e4-4d3d-aa03-2b0015fb7c96
caff0a71-aa3c-4327-a224-b69dbb001842
b66ec94c-9df1-4b0d-b6a4-0bb337352407
151756a1-f5eb-408f-9307-b5dacb7c7a8a
bc0728e2-cca3-4454-8cd7-796de2ee0c86
b6dca0c2-f1dc-427c-9222-3a42e16b0f75
715a0689-8ee7-41d3-95a7-46eac2c6e3d3
b4cea1d6-9fd8-47a3-bb6c-1d281fbb8455
314c5061-faff-430d-8d8b-aff578fc3a79
cbaf9e6b-c105-4419-aeef-d1c253976a1d
438265d4-5e1d-4392-811d-4c96999482eb
9c85b737-1e85-4445-96b4-e6a6bd742593
08bfe4ef-2553-4552-bb63-a895d8f8edc2
28949bdc-53f1-4238-abfa-2b8d0448c5b8
0dd46910-1513-4c96-a000-0bf19113c9d6
0082668c-cf31-4059-86e6-cc18af07c0c6
e33544c1-b6d9-40d0-b74b-b7c7b0f1986a
a1d3d89a-0b4d-4a81-a98f-22e091dfb20b
b8c3dc6b-ff07-4e3a-8da2-5f90842ac65d
035f3e04-86cb-45b8-a9a0-abef1fb4a2e1
60051ab2-0ea0-4ae2-974f-cb157b5bca19
be4304b3-bbc1-4ff4-8c8f-24e610d9e21d
ca1c2bb0-b819-404c-95a8-5743d1eee831
68ba81e6-4e50-4348-9e5a-15cf9006c00c
9b68dbbd-7d49-420a-93cf-3da183a5a830
40c68fc3-a077-499e-a022-c729044246d2
e90ce463-ba24-4547-bde6-f230a53c9a3a
30c5625f-81d3-41b6-8a18-b36dcd108aa0
ef855941-bf32-4116-9e06-6e9267268f1e
2c7eb763-c933-4a6f-b69d-bda09b12c090
370fac88-b4cb-439e-ba7e-da692b7fa272
f8d3185c-5727-4e18-ba80-52ad13177112
51e75ea5-b923-477b-941b-b06a9f6f2b9b
2f21231e-b20e-4469-af59-fde7513d6fda
17758a75-8e54-4c73-8f4a-a09ff40e3306
526a2919-f29b-404d-83ec-197a8c69b782
d23566e7-02a8-4378-a4ec-b32b7d240773
73272b69-69b0-4ecd-b878-0cd462e28a97
13c5be69-728b-496c-910e-e0833424574e
2fc6d817-a933-441d-a292-b120b85ffd95
932fb469-fdd8-4f39-8210-2046d7e5a527
ce362374-36b4-41eb-8774-9a5cefc5f1d0
da74214b-30ad-4425-90d9-a10454113e7b
d171e19b-64ef-46e6-bcbf-6d3ff873eba1
8c10351c-9305-46e1-aab9-90591bd851ae
d4213bd0-206c-4b64-9af7-494c670e5220
ba23bb88-9c4a-423d-8e0e-3ff05b9b4c40
97b04321-43af-4115-a666-c36a92759582
7383b950-50ef-4def-8de6-73d0439e44f7
e560973a-dbb5-4d3c-a17a-9bfd51814ace
ea450601-edc7-44c4-84b1-26ca14d324b5
9ed8cda5-5e8b-4167-9eb0-77b2caa374ff
b1604df3-e6e7-4ca4-ba9c-8acc1889eae8
f1487f2e-a7b2-45b4-ad72-5b4546ffc40f
d7247646-08af-4b72-8eb4-8cbaedce606c
58537076-82e4-48e9-b1f1-b1cf59dd8eee
30650919-6a99-4c86-bf9a-d4fab71add14
e79c7174-e17f-4062-8d6d-7ed3060a01b7
1cafdd7f-f872-43a5-8a4d-0e64c939ca76
9d7b3327-421e-4494-98db-34c061e5fb5a
72a82e9e-1464-490e-9ccf-aeddfc702b56
f58464ce-2d90-4002-8a11-b20498569be8
c505b853-ea57-4f0d-8168-1ca70978426e
cac4f1b4-f166-46d0-80f5-a93567051ca6
8cd6632a-926d-4732-a0da-a603d99e3ccb
47467643-54ea-42a9-b208-ff02dbea509b
e9d91fc0-46cd-4686-b3e8-e2907cad6a6e
7faa8cab-5f22-4326-b8b1-e0f395ead117
e9cc9aa1-9f5f-46df-8a3c-96e548d2dd31
02395fe6-8839-4225-a297-e2ce5801d1a1
2f7ff185-51b6-4b01-9478-298ecbee4a72
29d26e21-a53e-4483-999b-222306dd8609
da10ef2a-7de9-43ae-a449-c7f00a1d9cc2
b574fb96-70d7-419a-983a-43324eccd896
995afd03-6614-4172-8cac-8abd0b60eeff
f5b45ab6-d384-4d2b-a7c4-53d329762a7a
85257b7a-5969-43fc-a034-79421ba2b86e
6bb7ea28-591b-4c90-be76-f08a47f1adfb
e87edac9-5adf-437f-880d-b5d0fc61d8d6
37ff0cc9-462e-4d97-9f13-cf824050d0f2
3af6197f-67d1-4d63-91c0-685570af40cf
f0b9828e-5fcb-4ada-86de-1f217b98339e
d50796b4-11b9-4cd5-b998-f47f215df634
eebb913c-b2a0-4a87-8025-5fa87e7fd0db
49742d08-9227-465f-808d-531ba9f75c32
415ed0c9-f9f0-4076-afc0-c3af2605e5fb
c6102ed7-db02-46de-92ac-1a8576ec07cf
b4264397-44db-4371-86a9-74949e5c35c7
66cac5c8-1bcf-4f95-b469-6f9bbbe10593


ls /home/mateuszk/Testy/

For the new product format " or names[i].find('MSIL1C')>0 " needs to be added to if clause.
And also I think " .replace("eu/odata","eu/apihub/odata") " is no longer required, since they have corrected the urls.

When the download starts, file starts to download:
query_results.xml, file_dir.xml.
Then on the following link for example:
https://scihub.copernicus.eu/apihub/odata/v1/Products('326b6316-f7ae-4ae6-91e5-2121f3aedfe1')/Nodes('S2A_OPER_PRD_MSIL1C_PDMC_20160526T094552_R051_V20160521T105553_20160521T105553.SAFE')/Nodes('GRANULE')/Nodes
It says: Unable an to establish SSL connection. And "granule_dir.xml" is empty.

When I set https_proxy="" then there is no error.
But he's getting more time to send request for file.

I couldn't find your email to ask you, so I attach this question here. Sorry.

Hello Olivier,

Great software!
I does indeed reduce the time and headach of downloading images directly from Scihub, especially when one is only interested in one Granule rather than the whole tile.

I was wondering if there is a possibility to download specific bands only, like 432 to create a true color composite image. Are there plans to do that in the future?

Best,
Mohammad

I use the script to automatically download S1 and S2 data (thank you for sharing). When downloading S2 data everything is fine. When downloading S1 data, the script seems to interpret Lat(min,max) and Long(min,max) in reverse, Lat for Long and Long for Lat.

I try to use in win10 64bit os,and I only get the "query_results.xml" file.The images are not downloaded from ESA Sentinel distribution. When I run in win10,I use this command line "python Sentinel_download.py --downloader aria2 --lat 43.6 --lon 1.44 -a apihub.txt -o 51 -s S2".What's wrong with me?

Hello,

i try to run the command python Sentinel_download.py --downloader aria2 --lat 43.6 --lon 1.44 -a apihub.txt -s S2 -d 20170201 -m 75

and get an output like

aria2c --check-certificate=false --http-user="user" --http-passwd="password"  --continue -o query_results.xml "https://scihub.copernicus.eu/apihub/search?q=footprint:"Intersects(43.600000,1.440000)" filename:S2*&rows=100"
 *** Download Progress Summary as of Mon Feb 27 09:24:39 2017 ***
======================================================================================================================
[#3ec4a7 0B/0B CN:1 DL:0B]
FILE: D:/s2_download/query_results.xml
-----------------------------------------------------------------------------------

(querey_results.xml is not generated)

this repeats several times and ends in an error:

[#3ec4a7 0B/0B CN:1 DL:0B]
02/27 09:28:41 [ERROR] CUID#7 - Download aborted. URI=https://scihub.copernicus.eu/apihub/search?q=footprint:Intersects(43.600000,1.440000)%20filename:S2*&rows=100
Exception: [AbstractCommand.cc:339] errorCode=2 Timeout.

02/27 09:28:41 [NOTICE] Download GID#3ec4a752a29477f0 not complete: D:/s2_download/query_results.xml

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
3ec4a7|ERR |       0B/s|D:/s2_download/query_results.xml

Status Legend:
(ERR):error occurred.

and
IOError: [Errno 2] No such file or directory: 'query_results.xml

Hi,

I want to download only "TCI.jp2 file just like "https://github.com/DHI-GRAS/s2-tci".

Is it possible?

Thanks.

Hello,

i receive the following error when running
python Sentinel_download.py --lat 43.6 --lon 1.44 -a apihub.txt -o 51 -s S2 --downloader aria2

aria2c --check-certificate=false --http-user="guest" --http-passwd="guest"  --continue -o query_results.xml "https://scihub.copernicus.eu/apihub/search?q=footprint:"Intersects(43.600000,1.440000)" filename:S2*R051*&rows=100"

07/25 08:29:12 [ERROR] CUID#7 - Download aborted. URI=https://scihub.copernicus.eu/apihub/search?q=footprint:Intersects(43.600000,1.440000)%20filename:S2*R051*&rows=100
Exception: [AbstractCommand.cc:350] errorCode=22 URI=https://scihub.copernicus.eu/apihub/search?q=footprint:Intersects(43.600000,1.440000)%20filename:S2*R051*&rows=100
  -> [HttpSkipResponseCommand.cc:239] errorCode=22 The response status is not successful. status=500

07/25 08:29:12 [NOTICE] Download GID#fec8660e908f0856 not complete: D:/ohagolle/query_results.xml

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
fec866|ERR |       0B/s|D:/ohagolle/query_results.xml

Status Legend:
(ERR):error occurred.

aria2 will resume download if the transfer is restarted.
If there are any errors, then see the log file. See '-l' option in help/man page for details.
Traceback (most recent call last):
  File "Sentinel_download.py", line 230, in <module>
    xml=minidom.parse("query_results.xml")
  File "C:\hagelprogramme\Anaconda\lib\xml\dom\minidom.py", line 1918, in parse
    return expatbuilder.parse(file)
  File "C:\hagelprogramme\Anaconda\lib\xml\dom\expatbuilder.py", line 922, in parse
    fp = open(file, 'rb')
IOError: [Errno 2] No such file or directory: 'query_results.xml'

any ideas hwo to solve this?
Win10

Hi,

Thanks for sharing this repository. I really liked your code for bulk downloading Landsat data as well.

Do you have any plans to make this code compatible with Sentinel 3 data?

Hi Looks like S2MSI2Ap is no longer available and stops in may.

To avoid the problem you have to specify the product S2MSI2A without p that means preview.

replacing the code

if producttype !=None:    
    query_producttype=" producttype:%s "%producttype
    query=query+query_producttype
    

by:

if producttype !=None: 
    if producttype=="S2MSI2Ap":
        query_producttype = " (producttype:%s OR producttype:S2MSI2A)" % producttype
    else:
        query_producttype=" producttype:%s "%producttype
    query=query+query_producttype

should be enough.

Hi and thank you for sharing this software.

When I download more than one file at the same time (in my case because I'm specifying a rectangle), the first one is downloaded to the -w folder.
But the rest are downloaded where the Sentinel-download script is located.

Kind Regards

Carlos

I just tried downloading Sentinel 2 images with the following command:
python Sentinel_download.py --latmin 40.591 --latmax 60.095 --lonmin -19.245 --lonmax 34.146 -d 20160601 -f 20160901.
The script connects to the server, but doesn't start any downloads. When I remove the date arguments, the downloads start flawlessly.

Hello Olivier,
I first want to thank you for this script. I am having issues making it work. Apparently I having problems with autenthication certificate. Below you can see the error from the terminal.

What would you recommend me to try out?
Thank you very much,

Inti

Dear,
I have tried to use the script for downloading all sen2 images over Belgium.
However, since the .xml file is cut after 100 entries (0 to 99), it never searches any further back into time.
The parser mentions the '-r' option as number of records to download, but actually it refers to the number of records to put into the xml, causing far less records to actually download.
Is it possible to remove the constraint on the number of records in the xml?

Kind regards,
Stien