Support for Decimal128 & Decimal256 without downcasting to f64 about duckdb-rs HOT 6 OPEN

Hi! I don't know if helper-c.cpp matters, struct, union and list types aren't present there either, but It's true that the c-api doesn't expose a way to create decimal types using the standard duckdb_create_logical_type. The issue is that since decimal types are "parameterized" you'd need a different constructor function that allows you to pass the width and scale, (similar to e.g. duckdb_create_list_type), but thats definitely something we could look into adding.

We do have a duckdb_create_decimal_type and a duckdb_create_decimal function, so adding support for this in the rust api not be that complex.

In the case of 256-width decimals however, thats more of an expected limitation as DuckDB itself doesn't support 256-width decimals internally.

from duckdb-rs.

The vtab module does have some support already: https://docs.rs/duckdb/latest/duckdb/vtab/struct.LogicalType.html#method.decimal

from duckdb-rs.

Even if DuckDB-rs has some typing for Decimal128, I think the core issue is that duckdb_execute_prepared_arrow (i.e. https://github.com/duckdb/duckdb/blob/5b36f520df72be12f90ae31e6f8ed797a7b78e99/src/main/capi/arrow-c.cpp#L163) uses helper-c.cpp and if that doesn't support Decimal128, no change to only duckdb-rs will fix it.

I don't have the best understanding of duckdb specifically.

from duckdb-rs.

Where are you seeing that function used in the codepath duckdb-rs uses?

from duckdb-rs.

Running

let schema = Schema::new(vec![Field::new("a", input_array.data_type().clone(), false)]);
let rb = RecordBatch::try_new(Arc::new(schema), vec![Arc::new(input_array.clone())])?;
let mut stmt = db.prepare("select a from arrow(?, ?)")?;
let rb = stmt.query_arrow(param)?

stmt.query_arrow calls duckdb_execute_prepared_arrow, via:

• duckdb-rs/src/statement.rs

  Line 107 in a1aa55a

  pub fn query_arrow<P: Params>(&mut self, params: P) -> Result<Arrow<'_>> {

• duckdb-rs/src/statement.rs

  Line 63 in a1aa55a

  pub fn execute<P: Params>(&mut self, params: P) -> Result<usize> {

• duckdb-rs/src/statement.rs

  Line 510 in a1aa55a

  fn execute_with_bound_parameters(&mut self) -> Result<usize> {

• duckdb-rs/src/raw_statement.rs

  Line 223 in a1aa55a

  pub fn execute(&mut self) -> Result<usize> {

• duckdb-rs/src/raw_statement.rs

  Line 227 in a1aa55a

  let rc = ffi::duckdb_execute_prepared_arrow(self.ptr, &mut out);

from duckdb-rs.

As mentioned DuckDB doesn't support decimal256 or decimals with negative scale, but I've added support for positive decimal128 conversion here #328

from duckdb-rs.