Agnostic data container proposal about mlj.jl HOT 15 CLOSED

I was wondering if data container are not well handeled in packages like https://github.com/queryverse/Queryverse.jl

from mlj.jl.

https://github.com/queryverse/Query.jl

from mlj.jl.

Currently Query.jl supports only in-memory data sources, so not JuliaDB

from mlj.jl.

Sorry mixed it up with https://github.com/queryverse/IterableTables.jl/blob/master/docs/src/integrationguide.md which does

from mlj.jl.

I've tried IterableTables a bit and it is much easier to get along with.

from mlj.jl.

Here is another challenge for agnostic data containers, which concerns categorical data. Some data containers do not handle factors well.

In a DataFrame and other column-based containers, a column for a categorical feature can store metadata encoding all possible classes for the features, so that if I restrict to some subset of rows, I still know what all the possible values are, even if they do not occur in the restricted frame. If a model needs such data one-hot encoded, for example, the encoder can get all the information it needs just from, say, the training set, and later we can apply the same transformation to the test set. This means wrapping such models in the encoding transformation is not problematic.

However, in JuliaDB for example, which is row-based, there is no such column meta data. One must see all the data before one can know how to one-hot encode a categorical. That's fine if we do our preprocessing "a priori" but precludes wrapping a model in an encoder because we only train on training data and the training data may not contain all classes seen in test.

from mlj.jl.

I simply think a "proper" data container (for data scientific modelling) needs to be type aware. I.e., it needs to provide a method by which the user can query what type they would expect at a position within - including whether it is a factor variable and if yes which levels it has.

from mlj.jl.

A problem with using Query.jl, Tables.jl or similar as our interface point to data is that these are designed to handle very general source types. In particular, they do not allow random access to rows. They just generate row iterators, meaning random access is slow. Since the dominant use-case is data we load into memory, this is not satisfactory. I see we have three options: (a) Write our own interface for each type we want to support (not too bad, as we mostly have DataFrames and JuiliaDB on our wish list) (b) Abandon our desire to support multiple containers; or (c) use IterableTables or similar to convert any datatype into our favourite kind (which we could just as well require the user to do, and revert to (b)).

The other possibility is that there is a common random-access interface for in-memory data types out there, but I am not aware of one.

from mlj.jl.

Our agnostic data interface doesn't have to do much, but at minimum I think MLJ needs to be able to access arbitrary rows by index or splice (for, e.g., resampling) and request a subset of columns by name or index/splice. We should be able to extract the column names. It would be nice to be able change columns in-place (e.g., to standardise data) and to add columns (e.g., one-hot encode). Vertical and horizontal concatenation would also be nice. We probably don't need general joins or group-by.

from mlj.jl.

Regarding previous discussion regarding nominal features, the proposal is now the following: MLJ will require nominal features to have a nominal feature element type. Ie, each element knows all the possible levels the feature can take, as in R, and as in CategoricalArrays.jl. We could use CategoricalArrays.CategoricalValue and CategoricalArrays.CategoricalString as the required types. (CSV.jl can read specified columns into these formats.) So we would not require this information to be encoded in column metadata.

from mlj.jl.

from mlj.jl.

Yes, I think CategoricalArrays are the most widely adopted.

from mlj.jl.

To clarify, CategoricalArrays also have column metadata, but as the elements also know the levels, we just need the element type defined in that package. So, I could have a JuliaDB Table (which has no column metadata) with a column whose element type is CategoricalArrays.CategoricalValue and that's fine.

from mlj.jl.

I have lodged a feature request at Tables.jl for a row getindex method.

from mlj.jl.

After revisiting Tables.jl (and properly reading the docs :-) ) I have concluded that this package provides us with the most we can presently expect from a generic tabular interface. I had been flirting with Query/TableTraits in MLJBase for a while, but have now switched in an upcoming PR. The impact on MLJ will, for practical purposes, be nill.

from mlj.jl.