Giter Club home page Giter Club logo

Comments (3)

DavisVaughan avatar DavisVaughan commented on July 29, 2024

I'm not sure how big simpleFeatureCollection is but one thing to keep in mind with your current approach is that you (probably) get 16 copies of it, one for each worker, and that could be expensive

from furrr.

DavisVaughan avatar DavisVaughan commented on July 29, 2024

Are you sure mediumSizeRaster = rast("twoGBraster.tif") is actually resulting in an object? It looks like it uses a relative path so the working directory on the worker may be different. You could try supplying an absolute path instead.

from furrr.

twest820 avatar twest820 commented on July 29, 2024

If the error message for workers > 1 is to be believed, it appears somehow the call to rast() is getting skipped—even if the statement was executed and rast() had some silent error leading it to somehow return NULL instead of failing properly that should still result in the parser adding mediumSizeRaster as a workspace variable. So it seems like something might be going pretty badly wrong though, given future's limitations for flowing diagnostics from workers back to their caller, we might be stuck. (I find myself often wishing for plan(multithread) but that's not on furrr.)

If it was a pathing issue, which it presumably isn't since there's no issue with workers = 1, I'd expect to see something like the usual

Error: [rast] file does not exist: twoGBraster.tif
In addition: Warning message:
twoGBraster.tif: No such file or directory (GDAL error 4) 

come back. But future may not be able to route that.

I'm not sure how big simpleFeatureCollection is

Good question! It's only a couple MB, so negligible in this context—32 workers would be better but even 8 GB per worker is maybe asking too much (if this approach to the task had worked I was prepared to kill the future_map() and try with eight workers to get 16 GB DDR per worker if physical memory was going to be exceeded).

from furrr.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.