Semantics of unique and levels about dataarrays.jl HOT 8 OPEN

In c8e1653, I implemented the semantics proposed above. If people are happy with that, this should be done.

from dataarrays.jl.

Thoughts on this, @simonster?

from dataarrays.jl.

This seems right given that unique has isequal semantics. I wonder whether unique(da; skipna=true) would be more discoverable than levels, but then we'd have to return a DataArray and not an Array for type stability since we don't get type inference for kwargs.

from dataarrays.jl.

I like that idea, although the absence of type specialization is frustrating.

from dataarrays.jl.

I find it problematic that unique(pda) gives all levels, even those that don't actually exist in the array. This is a problem when using unique on a sub-DataFrame with PDA columns. I currently have to work around it by converting to DataArray before calling unique.

from dataarrays.jl.

Agreed. Even for levels, it is often practical to automatically drop those that do not appear in a subset (though that's yet another issue).

from dataarrays.jl.

I definitely think that levels should just return the pool, as it
currently does (and R does) but that unique should actually go through
the effort of checking. At some level, levels should be the domain of the
array. (See a bunch of very old discussion about metadata that could be
associated with columns/DataArrays, somewhere in the DataFrames issue
backlog...)

On Thu, Jan 2, 2014 at 4:56 PM, Milan Bouchet-Valat <
[email protected]> wrote:

Agreed. Even for levels, it is often practical to automatically drop
those that do not appear in a subset (though that's yet another issue).

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/29#issuecomment-31488294
.

from dataarrays.jl.

Yeah, this is how it works in R and that makes sense.

from dataarrays.jl.