Giter Club home page Giter Club logo

datacube-core's Introduction

Open Data Cube Core

Build Status Coverage Status Documentation Status

Overview

The Open Data Cube Core provides an integrated gridded data analysis environment for decades of analysis ready earth observation satellite and related data from multiple satellite and other acquisition systems.

Documentation

See the user guide for installation and usage of the datacube, and for documentation of the API.

Join our Slack if you need help setting up or using the Open Data Cube.

Please help us to keep the Open Data Cube community open and inclusive by reading and following our Code of Conduct.

Requirements

System

  • PostgreSQL 9.5+
  • Python 3.6+

Developer setup

  1. Clone:
    • git clone https://github.com/opendatacube/datacube-core.git
  2. Create a Python environment for using the ODC. We recommend conda as the easiest way to handle Python dependencies.
conda create -n odc -c conda-forge python=3.6 datacube pre_commit
conda activate odc
  1. Install a develop version of datacube-core.
cd datacube-core
pip install --upgrade -e .
  1. Install the pre-commit hooks to help follow ODC coding conventions when committing with git.
pre-commit install
  1. Run unit tests + PyLint ./check-code.sh

    (this script approximates what is run by Travis. You can alternatively run pytest yourself). Some test dependencies may need to be installed, attempt to install these using:

    pip install --upgrade -e '.[test]'

    If install for these fails please lodge them as issues.

  2. (or) Run all tests, including integration tests.

    ./check-code.sh integration_tests

    • Assumes a password-less Postgres database running on localhost called

    agdcintegration

    • Otherwise copy integration_tests/agdcintegration.conf to ~/.datacube_integration.conf and edit to customise.

Alternatively one can use the opendatacube/datacube-tests docker image to run tests. This docker includes database server pre-configured for running integration tests. Add --with-docker command line option as a first argument to ./check-code.sh script.

./check-code.sh --with-docker integration_tests

Developer setup on Ubuntu

Building a Python virtual environment on Ubuntu suitable for development work.

Install dependencies:

sudo apt-get update
sudo apt-get install -y \
  autoconf automake build-essential make cmake \
  graphviz \
  python3-venv \
  python3-dev \
  libpq-dev \
  libyaml-dev \
  libnetcdf-dev \
  libudunits2-dev

Build the python virtual environment:

pyenv="${HOME}/.envs/odc"  # Change to suit your needs
mkdir -p "${pyenv}"
python3 -m venv "${pyenv}"
source "${pyenv}/bin/activate"
pip install -U pip wheel cython numpy
pip install -e '.[dev]'
pip install flake8 mypy pylint autoflake black

Development on Colab

The following script allows reproducing the datacube library on Google Colaboratory, since this is currently working on a conda environment, individual libraries and correspondent dependencies must be installed independently. For reproducibility on Colab, execute the following commands on a single cell.

import os
!git clone https://github.com/opendatacube/datacube-core.git
% cd datacube-core
! pip install --upgrade -e .
os.system('/content/datacube-core/check-code.sh')
!pip install --upgrade -e '.[test]'
os.system("./check-code.sh integration_tests")
!sudo apt-get install -y \
  autoconf automake build-essential make cmake \
  graphviz \
  python3-venv \
  python3-dev \
  libpq-dev \
  libyaml-dev \
  libnetcdf-dev \
  libudunits2-dev
!pip install --extra-index-url="https://packages.dea.ga.gov.au" \
  odc-ui \
  odc-index \
  odc-geom \
  odc-algo \
  odc-io \
  odc-aws \
  odc-aio \
  odc-dscache \
  odc-dtools
!git clone https://github.com/ceos-seo/odc-gee.git
!pip install -e odc-gee
!wget -nc https://raw.githubusercontent.com/ceos-seo/odc-colab/master/odc_colab.py
from odc_colab import odc_colab_init
odc_colab_init(install_odc_gee=True)

from datacube import Datacube

datacube-core's People

Contributors

alex-ip avatar alexgleith avatar andrewdhicks avatar ashoka1234 avatar awalshie avatar bellemae avatar benjimin avatar ceos-seo avatar cronosnull avatar gypsybojangles avatar harshurampur avatar jeremyh avatar kirill888 avatar max-ar avatar mpaget avatar omad avatar petewa avatar richardscottoz avatar robbibt avatar rtaib avatar santoshamohan avatar sebasmos avatar simonaoliver avatar snowman2 avatar spaxe avatar uchchwhash avatar v0lat1le avatar whatnick avatar woodcockr avatar zhang01ga avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.