Comments (3)
ericgyounkin
commented on July 17, 2024
I think this might require a rewrite of the xarray_helpers.ZarrWrite class. I created this class due to limitations with xarray.Dataset.to_zarr, but it could be that there are new features coming to Xarray that make going back to it a viable option:
Support parallel writes to regions of zarr stores
Allow nested dictionaries in the Zarr backend (#3517)
from kluster.
ericgyounkin
commented on July 17, 2024
New issue I've submitted for xarray, figuring out how to overwrite a subset of an array using to_zarr
Writing subset of array with to_zarr
from kluster.
ericgyounkin
commented on July 17, 2024
No traction on submitted issue. I spent this morning figuring this out with my existing system. It actually turned out to be not that bad. See xarray_helpers.get_write_indices_zarr. Users can now easily append during conversion, and process a subset of time within any of the main building blocks (subset_time, see get_orientation_vectors for instance)
from kluster.
Related Issues (20)
- set a custom port for the Dask cluster HOT 10
- Shoalest surface is broken HOT 4
- Flying vessel HOT 6
- Drag & Drop in the '2d View' should add a new data file HOT 3
- integrate CUBE into bathygrid/kluster
- Include ability to run filters on points in Points View and subsets of FQPR
- include ability to filter time series data
- Investigate more accurate beam correction methods HOT 1
- expand Accuracy Test to work with GDAL supported formats as well as Bathygrid data
- rework the output progress bars to maybe merge/replace progress information, keep the output text to a minimum
- variable for tracking realtime navigation solution? Does that exist in raw mbes?
- Export shapefile for ship tracklines, grid cells
- add customizable kluster_variables
- UI in linux HOT 17
- Error converting .all files v0.8.8 HOT 20
- Docker file won´t built a working image HOT 10
- Getting an error while processing/georeferenceing .all file HOT 6
- Creating a 'plugin' for Kluster HOT 12
- Unexpected filter behaviour? HOT 7
- Error redrawing 3d points after filtering HOT 23
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kluster.