Why, for encoding of parquet page, only read and write operations are supported? about arrow2 HOT 4 CLOSED

Could you clarify the question? I am not sure I am understanding. Atm this only supports writing PLAIN, and, in PLAIN with nullable values, only valid slots should be written in parquet.

from arrow2.

Thanks reply, Viewed:
https://github.com/jorgecarleitao/parquet2/blob/f86ba6f65fede75dd7b9f7dd6b0fe3e271d2aa58/src/serialization/read/mod.rs#L50

parquet2: page_iter_to_array vs arrow2:page_to_array

I don’t understand:
Who should be subject to actual use

from arrow2.

ahh, got it.

Who should be subject to actual use

it depends on what you are looking for: parquet2 makes no assumption about which in-memory format the consumer wants to use. It offers a minimal deserializer for a simple in-memory format that is mostly used to run integration tests against other producers and consumers.

This format does not have the complexities of arrow, and it is thus easier to understand for normal rust users (e.g. it uses Vec<Option<bool>>, not two bitmaps for validity and values of a boolean array).

Arrow2 on the other hand offers a deserializer specifically for arrow's in-memory format.

So, if you wish to use parquet without arrow, use parquet's version; else, use arrow's version. :)

For arrow, there is a guide on how to read and write via parquet:

• https://jorgecarleitao.github.io/arrow2/io/parquet_read.html
• https://jorgecarleitao.github.io/arrow2/io/parquet_write.html

and also examples in examples/.

from arrow2.

thanks

from arrow2.