`DataFrame.read_parquet` and `DataFrame.write_parquet` about explorer HOT 3 CLOSED

FWIW, I think raising an exception on unimplemented for a specific backend would be fine until it is implemented, but doesn't hold up the progression of the explorer package.

For example, Nx didn't hold back on implementing features based on XLA in order to keep libtorch on parity, which is fine.

from explorer.

I had actually done some research on this a while back trying to leverage Parquet files from within Elixir. Here are some resources I found but never got a chance to dig deeply into beyond generating the Erlang bindings for the Thrift definition (I was more focused on getting the data processed than tool-building at the time):

• https://github.com/apache/parquet-format
• https://thrift.apache.org/docs/Languages.html
  • https://github.com/apache/thrift/blob/master/lib/erl/README.md

from explorer.

Thanks for that @arpieb! Those are great resources to come back to for the pure Elixir implementation.

from explorer.