Probably could use https://typedoc.org/.

I think https://developer.mozilla.org/en-US/docs/Web/API/ReadableStreamBYOBReader can help us reduce memory copies. It only works in Chrome right now, though.

Check whether https://rustwasm.github.io/docs/wasm-bindgen/reference/reference-types.html helps with performance.

Can we improve the new type performance or usability with #[repr(transparent)]?

Right now, we only release an UMD bundle for browsers. It would be good to also add ESM support as well as a commonjs module for node.

See https://rustwasm.github.io/docs/book/game-of-life/code-size.html

Support reading CSV files into a table. We currently only support IPC.

👋 I've become convinced that there's a use for a library like arrow-wasm... not designed to be used directly from JavaScript (assuming no perf gain over JS) but instead to be used as a building block library for rust-wasm applications that want to use Arrow but don't want to manage wrapping core Arrow data structures to wasm themselves. And especially to simplify FFI between JS and Rust.

I started on a library at https://github.com/kylebarron/arrow-wasm using parts of your code as inspiration, but thought I should check if you were interested in any upstream PRs to this project 🙂

Add support for

• date
• dictionary encoded
• list and struct vectors
• string vectors

What I really miss in the Arrow JS lib, that I have to write row based accumulators or lookups in JS to achieve synthetic accumulators (sum, avg, cumsum). As DataFusion and Polars already support this, I assume it's available on the base Arrow (Rust) too. Could you add a few examples in this area (sort, groupby, sum).

p.s. While I understand that Datafusion capabilities (SQL engine) could be cumbersome and overkill, did you consider exposing @ritchie46's lazy Polars API?

Should be a lot faster: https://docs.rs/arrow2/0.1.0/arrow2/.

Inlining makes bundles bigger. However, we need to figure out a way to get the correct path to the WASM file.

Right now, we get arrow = ReferenceError: Can't find variable: BigInt64Array.

Loading arrow from ipc will be a super common use case so it should be fast. We want to avoid as many memory copies as possible. We should look into what is happening and which copies we can avoid.

Try rustwasm/wasm-bindgen#1643 and rustwasm/wasm-bindgen#1993 and rustwasm/wasm-bindgen#2456

Needs rustwasm/wasm-bindgen#1591

https://github.com/cloudflare/serde-wasm-bindgen

Not needed it we can get #4 to work

How much slower/ faster is wasm compared to raw js?

List of things we want to compare with Arrow js and native typed arrays or js arrays:

• Serializing a list of record batches
• Creating a table from CSV/JSON/IPC
• Creating a record batch from a schema and typed js arrays
• Scanning a vector
• Converting a vector into a typed array
• Aggregations
• ... more? Add a comment below!

We already have the schema information so we should be able to write a method (at least in JS) that automatically casts a vector for a record batch to the right type.