Giter Club home page Giter Club logo

pdal-parallelizer's Introduction

PDAL-PARALLELIZER

Some processing on point clouds can be very time consuming, this problem can be solved by using several processes on a machine to run calculations in parallel. The pdal-parallelizer tool will allow you to fully use the power of your machine very simply to put it at the service of your processing.

pdal-parallelizer is a tool that allows you to process your point clouds through pipelines that will be executed on several cores of your machine. This tool uses the flexible open-source Python library Dask for the multiprocess side and allows you to use the power of the Point Data Abstraction Library, PDAL to write your pipelines.

It also protect you from any problem during the execution. Indeed, as the points clouds treatments can be really long, if something goes wrong during the execution you don’t want to restart this from the beginning. So pdal-parallelizer will serialize each pipeline to protect you from this.

Read the documentation for more details : https://pdal-parallelizer.readthedocs.io/

Installation

Using Pip

pip install pdal-parallelizer

Using Conda

conda install -c clementalba pdal-parallelizer

GitHub

The repository of pdal-parallelizer is available at https://github.com/meldig/pdal-parallelizer

Usage

Config file

Your configuration file must be like that :

{
    "input": "The folder that contains your input files (or a file path)",
    "output": "The folder that will receive your output files",
    "temp": "The folder that will contains your temporary files"
    "pipeline": "Your pipeline path"
}

Processing pipelines with API

from pdal_parallelizer import process_pipelines as process

process(config="./config.json", input_type="single", timeout=500, n_workers=5, diagnostic=True)

Processing pipelines with CLI

pdal-parallelizer process-pipelines -c <config file> -it dir -nw <n_workers> -tpw <threads_per_worker> -dr <number of files> -d
pdal-parallelizer process-pipelines -c <config file> -it single -nw <n_workers> -tpw <threads_per_worker> -ts <tiles size> -d -dr <number of tiles> -b <buffer size>

Requirements (only for pip installs)

Python 3.9+ (eg conda install -c anaconda python)

PDAL 2.4+ (eg conda install -c conda-forge pdal)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.