Giter Club home page Giter Club logo

s2preprocessing's Introduction

Sentinel 2 preprocessing Pipeline

This repository is a collection of bash and python scripts for query, download and upload (to local postgresql db) of Sentinel 2 images in the scope of my Master Thesis

Processing steps

  • query (with awsdownload)
  • download (with sentinelhub)
  • (optional) atmospheric correction (sen2cor)
  • conversion to .tif (with GDAL)
  • upload to PostgreSQL+PostGIS

Installation

clone repository

git clone https://github.com/MarcCoru/S2preprocessing.git

Dependencies

satellite data query

https://github.com/kraftek/awsdownload/releases/download/1.7/ProductDownload-1.7

satellite data download

https://github.com/sentinel-hub/sentinelhub-py

Pipeline

Query available products from Amazon Web Service (AWS)
sh query.sh sites/munich.site

creates $queryfile containing product names

Download products from $queryfile
sh download.sh sites/munich.site
Optional: Delete unnecessary tiles to reduce file size and processing time
python delGranules.py sites/munich.site

Dry run flag -d only simulates deletion

python delGranules.py sites/munich.site -d

the config file must include a granules variable e.g.

granules="32UPU 32UQU 33UTP 33UUP"

Here is an online tool to determine Granules

Atmospheric correction with Sen2Cor
sh sen2cor.sh sites/munich.site

The Sentinel 2 atomspheric correction config file L2A_GIPP.xml is by default stored in the project root directory. However, the location can be changed in the config file using the L2A_GIPP_path variable

PostgresSQL

Scripts

Add a "is evaluate" column to a polygon dataset

sh addEvalField.sh sites/bavaria.site tiles eval "0.8" ""

Add a "is train" column to a polygon dataset (eval=1 are excluded)

sh addEvalField.sh sites/bavaria.site tiles train "0.75" "where eval=0"

PostGIS extension

enable gdal drivers via (as sudo user: $sudo -u postgres -d <database>)

SET postgis.gdal_enabled_drivers = 'ENABLE_ALL';
ALTER SYSTEM SET postgis.gdal_enabled_drivers TO 'GTiff PNG JPEG';
SELECT pg_reload_conf();

test if drivers are enabled via

SELECT short_name, long_name FROM ST_GDALDrivers();

s2preprocessing's People

Contributors

marccoru avatar

Stargazers

Raphaël Meignant avatar  avatar Kostas Vlachos avatar Martin Kolarik avatar  avatar JB KOPECKY avatar Aron avatar omparkhomchuk avatar Abe Snowman avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.