Giter Club home page Giter Club logo

gridfourc's Introduction

The Gridfour Software Project Port to C

A C library for raster data including scientific and geophysical applications.

Welcome

Welcome to the Gridfour software project's port to the C programming language. We are just getting started and aspects of this project are still in flux. Currently, we are focused on a C-langage implementation of the GVRS API.

What is GVRS?

The Gridfour Virtual Raster Store (GVRS, pronounced "givers") is a file-backed system that provides memory-efficient access to large and very large raster (grid) data sets. GVRS was created with three main purposes in mind:

1. Authoring Data: For applications that produce data sets, GVRS provides a high-performance virtual management system for handling content. Applications may store their final results in persisent GVRS files, or transcribe them to conventional formats (NetCDF, HDF5, TIFF and GeoTIFF, etc.).

2. Experiments in Data Compression: GVRS provides a convenient testbed for developers who are exploring new ways of performing data compression for raster data sources.

3. Distribution: GVRS provides a light-weight API and data format suitable for distributing data to other systems. GVRS was originally conceived as a way of providing environment data for small systems, single-board computers, and platforms such as Autonomous Underwater Vehicles. But it is a feasible solution for many other use cases.

Project Status

The original Gridfour project was implemented in Java. More information about the original implementation is available at the Gridfour Software Project.

Currently, we have completed a read-only version of GVRS that runs under Windows and Linux. A cmake file is available for building the GVRS library. We are working on a series of how-to articles on our GridfourC wiki. Got something you'd like to learn more about? Let us know.

We are now turning our attention to implementing the ability for the C API to write data to a GVRS data store. The new GvrsBuilder functions provide a partial implementation of the ability to write files. Currently, they are limited to non-compressed formats and do not yet support metadata.
When that task is complete, the GVRS C API will be fully functional.

We are also in the process of adding doxygen-style documentation to our primary header files.

Documentation

Please see our Wiki page for more information on the C API.

For the Gridfour project in general, we have two main documentation pages:

  1. The Gridfour Project Notes give information on the underlying concepts and algorithms used by this project. The Notes page isn't just about Gridfour. It covers ideas and topics related to raster data processing in general.

  2. The Gridfour Wiki gives lots of helpful information on using Gridfour software including our Gridfour Virtual Raster Store (GVRS). It also gives information about our project goals and roadmap.

gridfourc's People

Contributors

gwlucastrig avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar

gridfourc's Issues

Implement ability to write data to a GVRS data store

At this time, the GvrsC API can read GVRS files, but does not implement the ability to write to them.

Currently, users who wish to prepare data for access by GvrsC must do so using the original Java implementation. This constraint is inconvenient. But the real disadvantage to not having a write capability is that it means that the C API cannot be used in programs that are authoring data. And authoring data is one of the most important use cases for GVRS.

Restore zlib to build scripts

The GVRS format specifies two broad categories of data compression: Huffman coding and the Deflate (zlib) algorithm. Code for Huffman is included in the standard build and does not depend on external libraries. Code that uses Deflate does depend on the zlib library. Unfortunately, some of the most important data compression tools in GVRS depend on Deflate. These tools include the files GvrsCodecDeflate.c, GvrsCodecLsop.c, and GvrsCodecFloat.c.

At this time, zlib-dependent code is not included in the build (cmake) scripts. The relevant files are included in the distribution, but are not part of the build specification.

We wish to investigate the best way to restore zlib capability to our cmake files. And, critically, we also need to have some simple procedures our users can apply to make sure that zlib is installed on their systems. On Linux, that is straightforward. Under Windows, we have encountered some roadblocks.

Ensure Linux compatibility for GvrsC library

To date, all GvrsC development has been conducted under Windows using Visual Studio. Although the codebase is intended to be cross-platform compatible, I have not yet tried building and testing under Linux. I am sure that there will be a few glitches in doing so.

I am particularly concerned about some of the functions that Visual Studios requires, such as _strdup() which I have not seen under Linux (I've seen strdup, but not with the underscore). I have not yet encountered a Big-Endian architecture, and haven't implemented code to address that byte-order (though I have a plan for doing so).

I do not have access to a Linux development environment at this time, but am looking into two possibilities:

  1. Setting up a trial account on Amazon AWS (the Amazon cloud)
  2. Purchasing and configuring a Raspbery PI to use as a build and test environment.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.