Giter Club home page Giter Club logo

s2preprocessing's Introduction

Sentinel 2 preprocessing Pipeline

This repository is a collection of bash and python scripts for query, download and upload (to local postgresql db) of Sentinel 2 images in the scope of my Master Thesis

Processing steps

  • query (with awsdownload)
  • download (with sentinelhub)
  • (optional) atmospheric correction (sen2cor)
  • conversion to .tif (with GDAL)
  • upload to PostgreSQL+PostGIS

Installation

clone repository

git clone https://github.com/MarcCoru/S2preprocessing.git

Dependencies

satellite data query

https://github.com/kraftek/awsdownload/releases/download/1.7/ProductDownload-1.7

satellite data download

https://github.com/sentinel-hub/sentinelhub-py

Pipeline

Query available products from Amazon Web Service (AWS)
sh query.sh sites/munich.site

creates $queryfile containing product names

Download products from $queryfile
sh download.sh sites/munich.site
Optional: Delete unnecessary tiles to reduce file size and processing time
python delGranules.py sites/munich.site

Dry run flag -d only simulates deletion

python delGranules.py sites/munich.site -d

the config file must include a granules variable e.g.

granules="32UPU 32UQU 33UTP 33UUP"

Here is an online tool to determine Granules

Atmospheric correction with Sen2Cor
sh sen2cor.sh sites/munich.site

The Sentinel 2 atomspheric correction config file L2A_GIPP.xml is by default stored in the project root directory. However, the location can be changed in the config file using the L2A_GIPP_path variable

PostgresSQL

Scripts

Add a "is evaluate" column to a polygon dataset

sh addEvalField.sh sites/bavaria.site tiles eval "0.8" ""

Add a "is train" column to a polygon dataset (eval=1 are excluded)

sh addEvalField.sh sites/bavaria.site tiles train "0.75" "where eval=0"

PostGIS extension

enable gdal drivers via (as sudo user: $sudo -u postgres -d <database>)

SET postgis.gdal_enabled_drivers = 'ENABLE_ALL';
ALTER SYSTEM SET postgis.gdal_enabled_drivers TO 'GTiff PNG JPEG';
SELECT pg_reload_conf();

test if drivers are enabled via

SELECT short_name, long_name FROM ST_GDALDrivers();

s2preprocessing's People

Contributors

marccoru avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.