Comments (9)
I'm having trouble with this workaround, particularly using geotiff
and ESM.
Error [ERR_REQUIRE_ESM]: require() of ES Module /workspaces/test-proj/node_modules/quick-lru/index.js from /workspaces/test-proj/node_modules/geotiff/dist-node/source/blockedsource.js not supported.
I found this thread you and Tim posted in, but was curious if these are still the most up to date solutions.
from geoblaze.
Welp, we can rule out georaster initliazation as the issue. I ran the same performance tests with just the initial parseGeoraster step (excluding the statistical calculations) and the results were as expected.
Singleband raster analysis took 287 ms
Multiband raster analysis took 21 ms
Parsing all the single band rasters will naturally take longer because we'll have to issue a GET request for each of the 20 or so files' metadata whereas the multiband scenario will require just one fetch of the metadata (usually first 1024 bytes).
However, and most importantly, this initial metadata parsing contributes very little to the overall length of time it takes to run stats on the multiband raster ( 0.021 seconds vs. 23 seconds).
That being said, the initial metadata parsing does take up more than a full half of the time for the single band raster analysis (~287ms out of ~506ms). Meaning that, in theory, running stats on a multiband raster, could be faster someday once the main issue is fixed.
from geoblaze.
Further investigation reveals the bulk of the extra time is spent calling georaster.getValues. It takes on average 4ms for the single-band rasters, but takes 24,143 ms for the multiband raster. In other words, 99.9% of the time (24143ms of the total 24165ms) is spent fetching the data with getValues.
from geoblaze.
The plot thickens. I ran a test on geotiff.js readRasters method, which is what georaster uses and it doesn't seem like there's any critical issue with geotiff.js.
Singleband raster analysis took 186 ms
Multiband raster analysis took 928 ms
I'm still curious why it takes more times to read a multiband raster than all rasters individually. In theory, reading the multiband should be faster for reasons expressed above. But it's a very small factor in the abnormally large geoblaze time.
from geoblaze.
oof, so when I copy the getValues function directly from the georaster source code, it runs very fast.
HOWEVER, when I run georaster.getValues from the compiled georaster code it's SUPER slow.
My current hypothesis is that the webpack building process is adding in a backwards compatability polyfill that is massively slowing things down. I'll keep digging!
from geoblaze.
well I ran the compiled version of the georaster code and it's fast, so I'm at a bit of a loss. It might be an issue with the way the georaster build compiled geotiff.js or another dependency. I'll look into it more.
from geoblaze.
Okay. I can confirm that the issue is with the way GeoRaster compiled geotiff.js. GeoRaster has a private property called _geotiff
that gives you access to a geotiff.js instance of your raster. It's technically not the same geotiff.js instance as the one initially created to read the georaster-relevant metadata bc that's off the main thread.
const georaster = await parseGeoRaster(url);
const tiff = georaster._geotiff;
await tiff.readRasters({...})
from geoblaze.
Fixing the georaster build process or getting the next major release out the door is going to be a large lift. However, here's the quick hacky fix for georasters that are loaded via url:
import { fromUrl } from "geotiff";
async function fix(georaster) {
if (typeof georaster._data === "string" && georaster._geotiff) {
georaster._geotiff = await fromUrl(georaster._data);
}
}
const georaster = await parseGeoRaster(multiUrl);
await fix(georaster);
await geoblaze.sum(georaster, geometry);
from geoblaze.
Hi, is there any news on this? I have a cog of around ~8MB and it easily takes 30+ seconds to identify a single point from multiple bands (including fetching).
from geoblaze.
Related Issues (20)
- add support for skewed images
- `rasterCalculator` with polygon clipping? HOT 3
- Remove worker_threads warning on initial import HOT 1
- add support for precise geometry expressed with numerical strings
- replace most logic in utils.getBoundingBox with bbox-fns/calc.js HOT 1
- Gnarly results that I can't explain HOT 3
- NaN values cause unexpected results
- improve isBbox
- Can't get stats within a geojson geometry HOT 2
- validate GeoJSON Input
- A multipolygon and 2 single polygons give differing results HOT 2
- Large error when raster extent and geometry bbox don't overlap HOT 4
- bbox per polygon HOT 1
- Improve accuracy of re-projected geometries HOT 1
- Multipolygon producing unexpected sum - antimeridian crossing scenario HOT 5
- similarity
- area
- How i create a new georaster form array of pixel values (geoblaze.get)
- geoblaze.sum produces unexpected result in 2.7.0 HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from geoblaze.