Giter Club home page Giter Club logo

loam's Introduction

A wrapper for running GDAL in the browser using gdal-js

Installation

npm install loam

Assuming you are using a build system, the main loam library should integrate into your build the same as any other library might. However, in order to correctly initialize the Emscripten environment for running GDAL, there are other assets that need to be accessible via HTTP request at runtime, but which should not be included in the main application bundle. Specifically, these are:

  • loam-worker.js: This is the "backend" of the library; it initializes the Web Worker and translates between the Loam "frontend" and GDAL.
  • gdal.js: This initializes the Emscripten runtime and loads the GDAL WebAssembly.
  • gdal.wasm: The GDAL binary, compiled to WebAssembly.
  • gdal.data: Contains configuration files that GDAL expects to find on the host filesystem.

All of these files will be included in the node_modules folder after running npm install loam, but it is up to you to integrate them into your development environment and deployment processes. Unfortunately, support for WebAssembly and Web Workers is still relatively young, so many build tools do not yet have a straightforward out-of-the-box solution that will work. However, in general, treating the four files above similarly to static assets (e.g. images, videos, or PDFs) tends to work fairly well. An example for Create React App is given below.

Create React App

When integrating Loam with a React app that was initialized using Create React App, the simplest thing to do is probably to copy the assets above into the /public folder, like so:

cp node_modules/gdal-js/gdal.* node_modules/loam/lib/loam-worker.js public/

This will cause the CRA build system to copy these files into the build folder untouched, where they can then be accessed by URL (e.g. http://localhost:3000/gdal.wasm). However, this has the disadvantage that you will need to commit the copied files to source control, and they won't be updated if you update Loam. A way to work around this is to put symlinks in /public instead:

ln -s ../node_modules/loam/lib/loam-worker.js public/loam-worker.js
ln -s ../node_modules/gdal-js/gdal.wasm public/gdal.wasm
ln -s ../node_modules/gdal-js/gdal.data public/gdal.data
ln -s ../node_modules/gdal-js/gdal.js public/gdal.js

API Documentation

Basic usage

import loam from "loam";

// Load WebAssembly and data files asynchronously. Will be called automatically by loam.open()
// but it is often helpful for responsiveness to pre-initialize because these files are fairly large. Returns a promise.
loam.initialize();

// Assuming you have a `Blob` object from somewhere. `File` objects also work.
loam.open(blob).then((dataset) => {
  dataset.width()
    .then((width) => /* do stuff with width */);

Functions

loam.initialize(pathPrefix, gdalPrefix)

Manually set up web worker and initialize Emscripten runtime. This function is called automatically by other functions on loam. Returns a promise that is resolved when Loam is fully initialized.

Although this function is called automatically by other functions, such as loam.open(), it is often beneficial for user experience to manually call loam.initialize(), because it allows pre-fetching Loam's WebAssembly assets (which are several megabytes uncompressed) at a time when the latency required to download them will be least perceptible by the user. For example, loam.initialize() could be called when the user clicks a button to open a file-selection dialog, allowing the WebAssembly to load in the background while the user selects a file.

This function is safe to call multiple times.

Parameters

  • pathPrefix (optional): The path or URL that Loam should use as a prefix when fetching its Web Worker. If left undefined, Loam will make a best guess based on the source path of its own <script> element. If Loam has no <script> element (e.g. because you are using dynamic imports), then autodetecting a prefix will fail and this parameter must be provided. URLs with domains may be used to enable Loam to be loaded from CDNs like unpkg, but the file name should be left off.
  • gdalPrefix (optional): The path or URL that Loam should use as a prefix when fetching WebAssembly assets for GDAL. If left undefined, Loam will use the same value as pathPrefix. URLs with domains may be used to enable loading from CDNs like unpkg, but the file name should be left off. If Loam fails to work properly and you see requests resulting in 404s or other errors for the gdal.* assets listed above, you will need to set pathPrefix, or this parameter, or both, to the correct locations where Loam can find those assets.

Return value

A promise that resolves when Loam is initialized. All of the functions described in this document wait for this promise's resolution when executing, so paying attention to whether this promise has resolved or not is not required. However, it may be helpful to do so in some circumstances, for example, if you want to display a visual indicator that your app is ready.


loam.open(file, sidecars)

Creates a new GDAL Dataset.

Parameters

  • file: A Blob or File object that should be opened with GDAL. GDAL is compiled with TIFF, PNG, and JPEG support. If you have a Blob, you may also control the name of the file that is shown to GDAL on the virtual filesystem by passing an object with the shape {name: string, data: Blob}. This can be useful if you are relying on GDAL behavior that uses file extensions to determine formats.
  • sidecars: An array of additional files that will be made present in the virtual file system when opening file. Some data formats are composed of multiple files (for example, Shapefiles have .shp, .shx, and .prj files, among others). If you need to include multiple files in order to open a dataset, pass the "main" file as file, and pass the others to sidecars. For a Shapefile, this would mean passing the .shp file as file and the .shx, .prj, and friends to sidecars. If file is a File, then sidecars must be an Array<File>. If file is a Blob or Object (see above), then sidecars must be an Array<Object> where each element has the shape {name: string, data: Blob}.

Return value

A promise that resolves with an instance of GDALDataset.


loam.rasterize(geojson, args)

Burns vectors in GeoJSON format into rasters. This is the equivalent of the gdal_rasterize command.

Note: This returns a new GDALDataset object but does not perform any immediate calculation. Instead, calls to .rasterize() are evaluated lazily (as with convert() and warp(), below). The rasterization operation is only evaluated when necessary in order to access some property of the dataset, such as its size, bytes, or band count. Successive calls to .warp() and .convert() can be lazily chained onto datasets produced via loam.rasterize().

Parameters

  • geojson: A Javascript object (not a string) in GeoJSON format.
  • args: An array of strings, each representing a single command-line argument accepted by the gdal_rasterize command. The src_datasource and dst_filename parameters should be omitted; these are handled internally by Loam. Example (assuming you have a properly structured GeoJSON object): loam.rasterize(geojson, ['-burn', '1.0', '-of', 'GTiff', '-ts', '200', '200'])

Return value

A promise that resolves to a new GDALDataset.


loam.reproject(fromCRS, toCRS, coords)

Reproject coordinates from one coordinate system to another using PROJ.4.

Parameters

  • fromCRS: A WKT-formatted string representing the source CRS.
  • toCRS: A WKT-formatted string representing the destination CRS.
  • coords: An array of [x, y] coordinate pairs.

Return value

A promise that resolves with an array of transformed coordinate pairs.


loam.reset()

Tear down Loam's internal Web Worker. This will cause initialize() to create a new Web Worker the next time it is called.

Note: This exists primarily to enable certain types of unit testing. It should not be necessary to call this function during normal usage of Loam. If you find that you are encountering a problem that loam.reset() solves, please open an issue

Parameters

  • None

Return value

A promise that resolves when the Web Worker has been terminated. This function waits for initialize() to complete or fail before tearing down the worker.


GDALDataset.close()

This used to be required in order to avoid memory leaks in earlier versions of Loam, but is currently a no-op. It has been maintained to preserve backwards compatibility, but has no effect other than to display a console warning.

Return value

A promise that resolves immediately with an empty list (for historical reasons).


GDALDataset.count()

Get the number of raster bands in the dataset.

Return value

A promise which resolves to the number of raster bands in the dataset.


GDALDataset.layerCount()

Get the number of vector layers in the dataset.

Return value

A promise which resolves to the number of vector layers in the dataset.


GDALDataset.width()

Get the width of the dataset, in pixels.

Return value

A promise which resolves to the width of the dataset, in pixels.


GDALDataset.height()

Get the height of the dataset, in pixels.

Return value

A promise which resolves to the height of the dataset, in pixels.


GDALDataset.wkt()

Get the coordinate reference system of the dataset, as a WKT-formatted string.

Return value

A promise which resolves with a WKT-formatted string representing the dataset's coordinate reference system.


GDALDataset.transform()

Get the affine transform of the dataset, as a list of six coefficients. This allows converting between pixel coordinates and geographic coordinates. See the GDAL documentation for further details.

Return value

A promise which resolves to the affine transform.


GDALDataset.bandMinimum(bandNum)

Get the actual minimum value or the minimum possible value of a band (depending on format).

Parameters

  • bandNum: The number of the band for which to get the minimum value. Band numbering starts at 1.

Return value

A promise which resolves to a minimum value for the specified band.


GDALDataset.bandMaximum(bandNum)

Get the actual maximum value or the maximum possible value of a band (depending on format).

Parameters

  • bandNum: The number of the band for which to get the maximum value. Band numbering starts at 1.

Return value

A promise which resolves to a maximum value for the specified band.


GDALDataset.bandStatistics(bandNum)

Get statistics about the values in a band.

Parameters

  • bandNum: The number of the band for which to get statistics. Band numbering starts at 1.

Return value

A promise which resolves to an object containing statistics. The shape of the object will be:

{
  minimum: The calculated minimum value of the band
  maximum: The calculated minimum value of the band
  median: The calculated median value of the band
  stdDev: The calculated standard deviation of the band
}

GDALDataset.bandNoDataValue(bandNum)

Get the value representing "no data" within the band.

Parameters

  • bandNum: The number of the band for which to get the no-data value. Band numbering starts at 1.

Return value

A promise which resolves to the no-data value for the specified band.


GDALDataset.bandDataType(bandNum)

Get the data type of the band (Byte, UInt16, Float32, etc.)

Parameters

  • bandNum: The number of the band for which to get the data type. Band numbering starts at 1.

Return value

A promise which resolves to a string containing the name of the data type for the specified band. For example, 'Byte', 'Float32', etc.


GDALDataset.bytes()

Get the on-disk representation of the dataset, as an array of bytes.

Return value

A promise which resolves to a Uint8Array containing the bytes of the dataset.


GDALDataset.convert(args)

Converts raster data between different formats. This is the equivalent of the gdal_translate command.

Note: This returns a new GDALDataset object but does not perform any immediate calculation. Instead, calls to .convert() and .warp() are evaluated lazily. Each successive call to .convert() or .warp() is stored in a list of operations on the dataset object. These operations are only evaluated when necessary in order to access some property of the dataset, such as its size, bytes, or band count.

Parameters

  • args: An array of strings, each representing a single command-line argument accepted by the gdal_translate command. The src_dataset and dst_dataset parameters should be omitted; these are handled by GDALDataset. Example: ds.convert(['-outsize', '200%', '200%'])

Return value

A promise that resolves to a new GDALDataset.


GDALDataset.vectorConvert(args)

Converts vector data between different formats. This is the equivalent of the ogr2ogr command.

Note: This returns a new GDALDataset object but does not perform any immediate calculation. Instead, calls to .vectorConvert() are evaluated lazily. Each successive call to .vectorConvert() is stored in a list of operations on the dataset object. These operations are only evaluated when necessary in order to access some property of the dataset, such as its size, bytes, or layer count.

Parameters

  • args: An array of strings, each representing a single command-line argument accepted by the ogr2ogr command. The dst_datasource_name and src_datasource_name parameters should be omitted; these are handled by GDALDataset. Example: ds.vectorConvert(['-f', 'GeoJSON']).

Return value

A promise that resolves to a new GDALDataset.


GDALDataset.warp(args)

Image reprojection and warping utility. This is the equivalent of the gdalwarp command.

Note: This returns a new GDALDataset object but does not perform any immediate calculation. Instead, calls to .convert() and .warp() are evaluated lazily. Each successive call to .convert() or .warp() is stored in a list of operations on the dataset object. These operations are only evaluated when necessary in order to access some property of the dataset, such as its size, bytes, or band count.

Parameters

  • args: An array of strings, each representing a single command-line argument accepted by the gdalwarp command. The srcfile and dstfile parameters should be omitted; these are handled by GDALDataset. Example: ds.warp(['-s_srs', 'EPSG:3857', '-t_srs', 'EPSG:4326'])

Return value

A promise that resolves to a new GDALDataset.


GDALDataset.render(mode, args, colors)

Utility for rendering and computing DEM metrics. This is the equivalent of the gdaldem command.

Note: This returns a new GDALDataset object but does not perform any immediate calculation. Instead, calls to .render() are evaluated lazily (as with convert() and warp(), above). The render operation is only evaluated when necessary in order to access some property of the dataset, such as its size, bytes, or band count. Successive calls to .warp() and .convert() can be lazily chained onto datasets produced by .render(), and vice-versa.

Parameters

  • mode: One of ['hillshade', 'slope','aspect', 'color-relief', 'TRI', 'TPI', 'roughness']. See the gdaldem documentation for an explanation of the function of each mode.
  • args: An array of strings, each representing a single command-line argument accepted by the gdaldem command. The inputdem and output_xxx_map parameters should be omitted; these are handled by GDALDataset. Example: ds.render('hillshade', ['-of', 'PNG'])
  • colors: If (and only if) mode is equal to 'color-relief', an array of strings representing lines in the color text file. Example: ds.render('color-relief', ['-of', 'PNG'], ['993.0 255 0 0']). See the gdaldem documentation for an explanation of the text file syntax.

Return value

A promise that resolves to a new GDALDataset.

Developing

Yarn and NVM are required.

After cloning,

  1. nvm use
  2. yarn install
  3. yarn dev and in another session yarn test:watch

Built assets are placed in lib.

Demo page

There is a (very!) simple demo page available that utilizes Loam to print info about a GeoTIFF. To view it in a browser, run yarn demo, and then navigate to http://localhost:8080/ . You can use this site for things like:

  • Playing around with Loam by editing the source code in demo/index.js
  • Validating changes that are difficult to test fully in CI

Editing Loam or the source in demo/ should auto-reload.

Contributing

Contributions are welcomed! Please feel free to work on any of the open issues or open an issue describing the changes you'd like to make. All contributions will be licensed under the Apache License, as per the GitHub Terms of Service.

loam's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

loam's Issues

loam.initialize throws Invalid base URL exception

I'm importing loam dynamically (I don't have an explicit script tag in the html) but it keeps failing when I call initialize. I get the error Failed to construct 'URL': Invalid base URL.

I am passing an absolute url as the first argument in initialize('http://localhost:3000') (initialize('/') also fails) else the webworker call will fail because it seems it can't import relatively due to security (Uncaught SyntaxError: Failed to execute 'importScripts' on 'WorkerGlobalScope': The URL '/loam-worker.js' is invalid.).

I've searched through the code and it seems it happens because of this function in the loam lib code:

function getPathPrefix() {
    return THIS_SCRIPT.src.substring(0, THIS_SCRIPT.src.lastIndexOf('/')) + '/';
}

This will always try to get the first script tag in the html and looks for the src attribute and it will paste '/' behind it causing the new URL call here new URL(loamPrefix, defaultPrefix) to fail when you don't import loam via a script tag in the html, because '/' is an invalid fallback base url. I don't have this because I'm importing dynamically in the code like this:

const loam = await import('loam');
loam.initialize('http://localhost:3000');

If I remove the defaultPrefix from the new URL function in the worker lib code it actually works new URL(loamPrefix)
A possible solution to this is to check if a script tag exists to allow for dynamic imports with absolute path prefix, something like this: b791b26

GDALDataset.bytes() is not documented

Is there a way to access the converted Geotiff file when using GDALDataset.convert(args) the same way as gdal_translate cli command returns a Geotiff file?

Remote COG support?

Can this read COGs via vsis3? The gdal.js examples seem to all be local files.

Loam cannot be run under Node.js

Hi

I want to use this in node.JS environment on local Windows PC, not on web browser.

Just for test, I wrote such ad-hoc code (without webpack or any packaging) and run:

const loam = require("loam")
loam.initialize()

const main = async () => {

  const file = 'png/Nara-Genpyo.png'
  const dataset = await loam.open(file)
  const width = await dataset.width()
  console.log(width)
}

main()

The result is error follow:

ReferenceError: self is not defined
    at Object.<anonymous> (F:\github\loam\node_modules\loam\lib\loam.js:1:197)
    at Module._compile (node:internal/modules/cjs/loader:1101:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (F:\github\loam\index.js:1:14)
    at Module._compile (node:internal/modules/cjs/loader:1101:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10)

I changed the code like this, but still same error occurs.

const main = async () => {
  const loam = await import("loam")
  loam.initialize()

  const file = 'png/Nara-Genpyo.png'
  const dataset = await loam.open(file)
  const width = await dataset.width()
  console.log(width)
}

main()

Is this expected result?
Loam don't work in node.js environment or it always must be used with webpack-like packager?

Unable to use Loam in React

Hello hi
i have react app i need to load .tif file in it that was my requirement after goggling i got loam npm package ,and i got working example in typescript here is linkand repo link i tried to clone same code logic in react, facing loam library error

import loam from "loam"
`function App() {
require('loam/lib/loam-worker.js');
require('gdal-js/gdal.js');
require('gdal-js/gdal.wasm');
require('gdal-js/gdal.data');
useEffect(()=>{
loam.initialize("/")

},[])
const ReadFile = (e) => {
const files = e.target.files
loam.open(files).then((dataset)=>{
console.log(dataset,"d")
}).catch((er)=>{
console.log(er,"er")
})
}
return `
above the simple code i have tried and throws error i have added screenshot this is about gdal i guess
Screenshot 2023-07-21 114015
and second error is about loam.open
Screenshot 2023-07-21 114127
where did i went wrong can you please tell how can i integrate loam in my project
i have
react v17+
"loam": "^1.2.0",
"gdal-js": "^2.2.0",

Make CI testing more robust

The current GitHub actions do not test whether the PR would break develop after being merged. If this repo is anticipated to see more activity, we should add testing for that condition.

Error in GDALOpen

When giving a png or jpeg file in loam.open() it throughs an error :

Error: Error in GDALOpen: The JPEG/PNG driver does not support update access to existing datasets.

While giving a GeoTIFF file loam works absolutely fine.

Code playground

It would be useful to have a playground where users can test out Loam without integrating it into their application. However, the easy paths to this, such as JSFiddle, are probably blocked by #58 .

Allow passing exact file path to loam.initialize()

Because the file might be named differently, or because the file might be on a remote origin, in which case Chrome won't allow creating a web worker using a script that is on the remote origin.

The latter can be worked around by fetch()ing the file and creating an object URL, like so:

const loamWorkerJS = await fetch('https://unpkg.com/[email protected]/lib/loam-worker.js')
  .then(res => res.blob())
  .then(blob => URL.createObjectURL(blob)) + "#";

Not the trailing #, which is really a hack so that when loam.initialize() appends the filename (loam-worker.js), it would end up being ignored. But that just so happens to work with object URLs.

I'd suggest adding a second parameter for backwards compatibility, for the filename, i.e. something like:

function initialize(pathPrefix, fileName) {
  if (fileName === undefined) {
    fileName = 'loam-worker.js';
  }

  …;
}

I'll send a PR later on, just wanted to note this down first.

Does loam compromise the image quality ?

Hey hi, i just i want to know that does loam compromise the image quality,Below i have given code that is used to parse .tif file and return some O/P in object

import loam from "loam";
import * as L from "leaflet";
import { v4 as uuidv4 } from "uuid";
loam.initialize("/");
export const CreateLayer = async (files, map) => { 
  const dataset = await loam.open(files); 
  const wktDest =
    'GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.01745329251994328,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]]';
  const smallDataset = await dataset.convert([
    "-of",
    "GTiff",
    "-outsize",
    "1000",
    "0",
    "-r",
    "nearest",
  ]);
  dataset.close();
  const newDataset = await smallDataset.warp([
    "-co",
    "PREDICTOR=2",
    "-dstnodata",
    "0",
    "-of",
    "GTiff",
    "-t_srs",
    "EPSG:3857",
    "-r",
    "near",
  ]);
  smallDataset.close();
  const width = await newDataset.width();
  const height = await newDataset.height();
  const geoTransform = await newDataset.transform();
  const wktSource = await newDataset.wkt();
  const corners = [
    [0, 0],
    [width, 0],
    [width, height],
    [0, height],
  ];
  const geoCorners = corners.map((coords) => {
    const x = coords[0];
    const y = coords[1];
    return [
      geoTransform[0] + geoTransform[1] * x + geoTransform[2] * y,
      geoTransform[3] + geoTransform[4] * x + geoTransform[5] * y,
    ];
  });
  const wgs84GeoCornersGdal = await loam.reproject(
    wktSource,
    wktDest,
    geoCorners
  );
  const wgs84GeoCorners = wgs84GeoCornersGdal.map((coords) => [
    coords[1],
    coords[0],
  ]);
  const pngDataset = await newDataset.convert(["-of", "PNG","-co", "COMPRESSION=deflate","-co", "PREDICTOR=2",]);
  const imageBytes = await pngDataset.bytes();
  const outputBlob = new Blob([imageBytes], { type: "image/png" });
  const imageURL = window.URL.createObjectURL(outputBlob);
  const imageOverlay = L.imageOverlay(imageURL, wgs84GeoCorners);
  const imageBounds = imageOverlay.getBounds();
  // const zoom = map.target.getBoundsZoom(imageBounds);
  const center = imageBounds.getCenter();
  const TiffParsedata = {
    id: uuidv4(),
    url: `${imageURL}`,
    bounds: wgs84GeoCorners,
    imageBounds: imageBounds,
    center: center,
  };
  let center1 = [center.lat, center.lng]; 
  // map.target.setView(center1, zoom);
  return TiffParsedata;
};

The out put of this above method will return imageUrl,Bounds,ImageBounds and Center so here image url is being overlayed on react-leaflet map for further implement, but the image is being generated by loam will have poor quality then its original (original image size can be around 200mb to 600mb .tif file) my question is there any way that loam can persist image original size as it is,do we have any work arounds here

I have tried this below code to improve the image quality but no use !
const pngDataset = await newDataset.convert(["-of", "PNG","-co", "COMPRESSION=deflate","-co", "PREDICTOR=2",]);
Please let us know if there are any possibility. Thanks you...

Originally posted by @syncthreads-sammith in #120 (comment)

Wrap additional methods

  • GDALRasterCount
  • GDALGetGeoTransform
  • GDALGetProjectionRef
  • GDALGetRasterXSize
  • GDALGetRasterYSize
  • GDALClose

Switch to VRTs for open(), convert(), and warp()

This will allow managing datasets without requiring the non-worker JS to remember to call close() on each dataset after finishing with it, which is more idiomatic.

Instead of each GDALDataset object storing a dataset pointer and filename, it should store a sequence of VRT XML files, as strings. Whenever an operation is performed on the dataset, the VRT strings will be written into files on the Emscripten filesystem with the appropriate file names, and then the operation will be called on the last VRT in the list.

If a new dataset is generated by the operation, then the resulting GDALDataset will have its "parent" dataset's VRT list, plus the VRT returned by the operation.

After the result is computed, the VRT files will be cleaned from the Emscripten filesystem, leaving it the same as it was before the operation was called.

This ensures that calling Emscripten functions have no lasting side effects, and that GDALDatasets can be chained without keeping track of the intermediate datasets for closing later.

Eliminate the need to call `close` on opened Datasets

This happens because each Dataset on the Javascript side is backed by an open file in the Emscripten file system and a live pointer to a GDALDatasetH on the WebAssembly heap. If a Dataset goes out of scope without notifying the web worker to clean up its backing resources, memory will be leaked.

In order to resolve this, operations that return new datasources should operate lazily and build up a sequence of operations, rather than generating persistent resources on the WebAssembly side. Once an operation is triggered that requires an actual file (such as getting the bytes of a file), the entire operation sequence should be executed on the web worker, with each intermediate dataset being closed, and its backing file removed, as soon as it is no longer needed. The final dataset should be closed prior to returning the result of the operation.

This will ensure that no persistent resources are created on the WebAssembly side, which will prevent memory from leaking if a Dataset goes out of scope and will eliminate the need to call close() entirely.

Add OGR wrappers

OGR methods are exported by the latest release of gdal-js, so we should add wrappers for them to Loam. Related: #82

Convert (gdal_translate) to mbtiles

Is it possible to convert to mbtiles? I currently have GTiff working but when I try to convert to mbtiles I get an error when calling the bytes() function.

const dataset = await file.convert(['-of', 'GTiff', '-a_srs', 'EPSG:4326', '-a_ullr', upperLeftX, upperLeftY, upperRightX, upperRightY]);
const fileBytes: Uint16Array = await dataset.bytes();
loam.js:1 Uncaught (in promise) Error: Error in GDALTranslate: ZMap: ZMap Plus Grid 
   at Worker.o2.onmessage (loam.js:1) 
   o2.onmessage @ loam.js:1 
   await in o2.onmessage (async)

Refactor string-parameter allocation

Several functions (currently GDALWarp, GDALTranslate, and GDALRasterize) follow a pattern where they store strings in Emscripten memory space in a format required by GDAL. Although this interface is stable and unlikely to change, it would speed up adding new GDAL commands if this code were factored out into a utility function.

Compiled assets in 'lib' folder are not in the release bundles

I downloaded the latest release from https://github.com/azavea/loam/archive/1.0.0-alpha.8.tar.gz

It did not include the lib folder minified bundles:

andrew@tango ~/Downloads: ls -l loam-1.0.0-alpha.8
total 400
-rw-r--r--@  1 andrew  staff   11343 Apr  6 17:07 LICENSE
-rw-r--r--@  1 andrew  staff    1251 Apr  6 17:07 README.md
-rw-r--r--@  1 andrew  staff    1753 Apr  6 17:07 karma.conf.js
-rw-r--r--@  1 andrew  staff    1616 Apr  6 17:07 package.json
drwxr-xr-x@ 10 andrew  staff     320 Apr  6 17:07 src
drwxr-xr-x@  4 andrew  staff     128 Apr  6 17:07 test
-rw-r--r--@  1 andrew  staff    1186 Apr  6 17:07 webpack.config.js
-rw-r--r--@  1 andrew  staff  172960 Apr  6 17:07 yarn.lock

Instead as a workaround, I cloned the repo, then ran:

yarn install --pure-lockfile
yarn build

and then copied the contents of the lib folder to my project.

How to install loam without react

Hi,
I'm working on a simple web app without using any framework, after the package installation I tried to import with
import loam from "loam";
But at the loam.initialize(); I get an error Uncaught ReferenceError: loam is not defined

Guard against non-string parameters to `convert` and `warp`

This doesn't always fail early enough to prevent bad parameters from making it into GDAL, which can cause difficult-to-diagnose errors like WebAssembly traps converting floats to integers.

A first step to resolve this is to verify that all parameters to these two functions are strings and throw an error if they're not.

Can't open a Tiff file in CRA with react 17 on Brave

Hey.

I am running a project with create-react-app with React 17
I want to open a tiff file into loam, but nothing happens.

Using the latest version of loam 1.0.0

import loam from 'loam';

loam.initialize()

const dataset = await loam.open(this.files[0]);

where this.files[0] is the tiff file the user uploaded.

In the console, the following error is also returned:

image

Is there anything that I should take into consideration? Also please note I am not loading the following files as I thought they are loaded automatically with loam.initialize()

  • gdal.js: This initializes the Emscripten runtime and loads the GDAL WebAssembly.
  • gdal.wasm: The GDAL binary, compiled to WebAssembly.
  • gdal.data: Contains configuration files that GDAL expects to find on the host filesystem.

Thanks for your work

Fix paths

Currently the WebWorker is loaded from /loam-worker.js; this should be configurable so that this library can be served from any path.

It looks like Emscripten already does this so the logic at the top of gdal.js can probably be used.

Further bundling cleanup

At the very least, remove babel-loader because it shouldn't be necessary for browsers that support WebAssembly, but also see if there are ways to simplify the testing setup and enable #66 to move forward.

Add ability to open remote files

This could either be accomplished by modifying loam.open() to detect URLs, or by creating a new method specifically for the purpose of opening remote files.

Either way, the corresponding Emscripten file will need to be created using FS.createLazyFile().

Asset locations

"All of these files will be included in the node_modules folder after running npm install loam, but it is up to you to integrate them into your development environment and deployment processes."

Any recommendations, best practices or examples regarding deployment structure?

Switch to Typescript

Given that this project interfaces with C code, it would benefit from use of Typescript because it would be easier to model the GDAL API interfaces that this is designed to connect to.

There are also some defined interfaces that Loam uses for internal communication that would benefit from being able to be formally defined somewhere.

How to rotate image?

Is there a way to rotate a georeferenced geotiff?
Getting lost in all warp options or should we use GDALSetGeoTransform from gdal.js?

Gcps to point work for external positioning but not internal when warping/ transforming

Firstly many thanks for loam, I have been playing around with it for the past fer months and enjoying using it.

Having an issue which I am unsure if loam or my lack of knowledge with gdal but from what I have read on gdal warping and transform it should be working.

When i apply commands and send it through loam it outputs a tiff but the gcps seem to only apply to the external bounds where I have adjusted points internally but no changes occur.

This is the core function that I am using which works for the outer gcps but does not work when trying to change internal points and gcps.

redacted in favor of below suggestion and example supplied

the example below is a small image but same results when trying with a large image are the same

example gcps -

-gcp 432.00 0.00 6.085175871849061 50.776186145801745
-gcp 432.00 357.00 6.085175871849061 50.77497511582229
-gcp 0.00 357.00 6.082858443260194 50.77497511582229
-gcp 209.20 113.00 6.083984973205042 50.77584862099022
-gcp 293.36 170.00 6.084666347114017 50.77561794936608
-gcp  227.44 234.00 6.08403323354001 50.77528550884761
-gcp 149.33 187.00 6.083367946409385 50.775560281282274```

the warp arguments 

`-rnear-order2-s_srsEPSG:4326-dstnodata-999-t_srsEPSG:3857-overwrite`

any thoughts, links to read or assistance would be appreciated as it is so close to working correctly

Many thanks   

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.