Giter Club home page Giter Club logo

juxta.jl's Introduction

juxta

Build Status Codecov

This package facilitates creation and manipulation of multi-dimensional labeled arrays. The eventual aim of this package is to emulate something similar to Xarray in python.

A container, JuxtArray, is provided. Only slicing is currently implemented.

Examples

Creation:

julia> ja = juxta.JuxtArray(randn(5,10), ["x","y"], Dict("x"=>collect(1:5),"y"=>collect(1:10)))
Dimensions  : ["x", "y"]
Array       : 5×10 Array{Float64,2}
Coordinates :
    x: 5-element Array{Number,1}
    y: 10-element Array{Number,1}
Attributes  : Dict{Any,Any}()

Index-based slicing:

julia> ja = juxta.JuxtArray(randn(5,10), ["x","y"], Dict("x"=>collect(1:5),"y"=>collect(1:10)))
Dimensions  : ["x", "y"]
Array       : 5×10 Array{Float64,2}
Coordinates :
    x: 5-element Array{Number,1}
    y: 10-element Array{Number,1}
Attributes  : Dict{Any,Any}()

julia> juxta.isel!(ja, x=2:4, y=3:7)
Dimensions  : ["x", "y"]
Array       : 3×5 Array{Float64,2}
Coordinates :
    x: 3-element Array{Number,1}
    y: 5-element Array{Number,1}
Attributes  : Dict{Any,Any}()

Indexing based on physical values of dimensions:

julia> ja = juxta.JuxtArray(randn(5,10), ["x","y"], Dict("x"=>collect(1:5),"y"=>collect(1:10) .* 2))
julia> juxta.sel!(ja, y=3.7:7.9)
Dimensions  : ["x", "y"]
Array       : 5×2 Array{Float64,2}
Coordinates :
    x: 5-element Array{Number,1}
    y: 2-element Array{Number,1}
Attributes  : Dict{Any,Any}()

This returns a subset where the dimension y ranges from 4:2:6.

julia> ja = juxta.JuxtArray(randn(5,10), ["x","y"], Dict("x"=>collect(1:5),"y"=>collect(1:10) .* 2))
julia> juxta.sel!(ja, "nearest", y=3.7:7.9)
Dimensions  : ["x", "y"]
Array       : 5×3 Array{Float64,2}
Coordinates :
    x: 5-element Array{Number,1}
    y: 3-element Array{Number,1}
Attributes  : Dict{Any,Any}()

This returns a subset where the dimension y ranges from 4:2:8.

julia> ja = juxta.JuxtArray(randn(5,10), ["x","y"], Dict("x"=>collect(1:5),"y"=>collect(1:10) .* 2))
julia> juxta.sel!(ja, y=3.7)
Dimensions  : ["x", "y"]
Array       : 5×2 Array{Float64,2}
Coordinates :
    x: 5-element Array{Number,1}
    y: 2-element Array{Number,1}
Attributes  : Dict{Any,Any}()

This returns a subset where the dimension y ranges from 2:2:4.

julia> ja = juxta.JuxtArray(randn(5,10), ["x","y"], Dict("x"=>collect(1:5),"y"=>collect(1:10) .* 2))
julia> juxta.sel!(ja, "nearest", y=3.7)
Dimensions  : ["x", "y"]
Array       : 5×1 Array{Float64,2}
Coordinates :
    x: 5-element Array{Number,1}
    y: 1-element Array{Number,1}
Attributes  : Dict{Any,Any}()

This returns a subset where the dimension y ranges from 4:4.

Dropping singleton dimensions:

julia> ja = juxta.JuxtArray(randn(5,10), ["x","y"], Dict("x"=>collect(1:5),"y"=>collect(1:10) .* 2))
julia> ja = juxta.sel!(ja, "nearest", x=0.8)
julia> ja = dropdims(ja, ["x"])
Dimensions  : ["y"]
Array       : 10-element Array{Float64,1}
Coordinates :
    x: 1-element Array{Number,1}
    y: 10-element Array{Number,1}
Attributes  : Dict{Any,Any}()

All of the above operations can be combined using Julia's piping functionality:

julia> ja = juxta.JuxtArray(randn(5,10,1), ["x","y","za"],
                     Dict("x"=>collect(1:5),"y"=>collect(1:10) .* 2,"za"=>[2]))
julia> ja = (ja
               |> j->juxta.isel!(j,x=2)
               |> j->dropdims(j,["x","za"])
               |> j->size(j))
(10,)

or by using Pipe.jl:

julia> using Pipe
julia> ja = juxta.JuxtArray(randn(5,10,1), ["x","y","za"],
                     Dict("x"=>collect(1:5),"y"=>collect(1:10) .* 2,"za"=>[2]))
julia> ja = @pipe (ja
                     |> juxta.isel!(_,x=2)
                     |> dropdims(_,["x","za"])
                     |> size(_))
(10,)

or by using Lazy.jl:

julia> using Lazy
julia> ja = juxta.JuxtArray(randn(5,10,1), ["x","y","za"],
                     Dict("x"=>collect(1:5),"y"=>collect(1:10) .* 2,"za"=>[2]))
julia> ja = @> ja juxta.isel!(x=2) dropdims(["x","za"]) size()
(10,)

TODO

  • Test with datetime dimensions
  • Implement plotting

juxta.jl's People

Contributors

beta-effect avatar

Stargazers

avatar

Watchers

avatar avatar avatar

juxta.jl's Issues

Can use multiple dispatch to make `isel!` more flexible.

juxta.jl/src/juxta.jl

Lines 70 to 76 in 7162bef

if String(k) in ja.dims
if typeof(v) <: OrdinalRange
push!(ja.indices, String(k)=>v)
elseif typeof(v) <: Integer
push!(ja.indices, String(k)=>v:v)
end
end

can be rewritten as something like

push_dim!(ja, k, v::OrdinalRange) = push!(ja.indices, String(k) => v)
push_dim!(ja, k, v::Integer)      = push!(ja.indices, String(k) => v:v)

...

String(k) in ja.dims && push_dim!(ja, k, v)

which should be easier to extend to other types of v. It also seems to be the more Julian way of doing things.

If ja.dims was of type Vector{Symbol} you might be able to just use k instead of String(k).

Could probably dispatch on `method` for `get_start_stop_indices`

Could probably dispatch on method if it was a type or via the Val type although I think the latter is discouraged.

juxta.jl/src/juxta.jl

Lines 89 to 109 in 7162bef

function get_start_stop_indices(dim_vector::Vector,
physical_start::Real,
physical_stop::Real,
method::String)
if method == "subset"
dim_vector_temp = dim_vector .- physical_start
start = findfirst(x -> x>=0, dim_vector_temp)
dim_vector_temp = dim_vector .- physical_stop
stop = findlast(x -> x<=0, dim_vector_temp)
if physical_start == physical_stop
return (stop, start)
end
elseif method == "nearest"
dim_vector_temp = abs.(dim_vector .- physical_start)
start = findmin(dim_vector_temp)[2]
dim_vector_temp = abs.(dim_vector .- physical_stop)
stop = findmin(dim_vector_temp)[2]
end
return (start, stop)
end

Not super clear why `sel!` and `isel!` modify the `JuxtArray`?

I'm probably just misunderstanding but I thought when you select part of a JuxtArray, either through physical slicing or index-based slicing, that you would get back a new JuxtArray containing just the selected portion, which shouldn't modify the existing JuxtArray.

But it seems that these functions actually modify the existing JuxtArray and overwrite ja.array? Don't you lose data this way?

Formatting examples

If you include code examples in the README with

```julia <code>```

then you also get syntax highlighting which might make them a bit more readable.

Some suggestions and questions about `JuxtArray`

juxta.jl/src/juxta.jl

Lines 14 to 29 in 7162bef

mutable struct JuxtArray
array::AbstractArray
dims::Vector{String}
coords::Dict{String, Vector{Number}}
attribs::Dict
indices::Dict{String, AbstractRange}
function JuxtArray(array, dims, coords, attribs=Dict())
@assert ndims(array) == length(dims) "Number of dims should be equal to the number of dims of the array"
indices = Dict{String, AbstractRange}()
for (i, array_dim_length) in enumerate(Base.size(array))
@assert array_dim_length == size(coords[dims[i]])[1] "Mismatch between length of array and coordinate"
push!(indices,dims[i]=>1:array_dim_length)
end
new(array, dims, coords, attribs, indices)
end
end

Was just wondering about a few things:

  1. Why make the struct mutable? Looks like the array contents may change but the struct itself won't so making it immutable might make more sense? You can append attributes as well, just not change what attribs is. Also could help the compiler make certain optimizations as it knows the struct properties won't change.
  2. I'm wondering if it would be better to use Symbols instead of Strings as the key. Not sure how helpful it would be for a labelled array but other packages like DataFrames.jl seem to use symbols. Might be a good read: https://stackoverflow.com/questions/23480722/what-is-a-symbol-in-julia
  3. I used to use @assert but apparently in C/C++, Python, and Julia they seem more for checking developer mistakes and may actually be disabled by certain optimization levels. So might be better to throw exceptions or something else.

Also, I might strongly type the struct to get more flexible JuxtArray (i.e. keys can be symbols or strings, types sort themselves out, indices don't need to be abstract ranges, etc.):

struct JuxtArray{A, D, C, AT, I}
     array::A
     dims::D
     coords::C
     attribs::AT
     indices::I
end

or you can be a bit more specific if you want to:

struct JuxtArray{A<:AbstractArray, K, C, I}
     array::A
     dims::Vector{K}
     coords::Dict{K, C} 
     attribs::Dict
     indices::Dict{K, I} 
end

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.