use AbstractArray about onlinestats.jl HOT 12 CLOSED

That seems fine to me. Is the idea to provide support for sparse matrices? Or what are the gains you're thinking of?

I'm not familiar with view. What would be a good example where standard Julia code creates a copy whereas view doesn't?

from onlinestats.jl.

ArrayViews is a package which allows different AbstractArrays to share the same underlying data. Right now, x[2:3] will return a copy of the values in slots 2 and 3 of the array x. view(x,2:3) will return an efficient ArrayViews.ContiguousView which points to the same underlying memory as x. Here's an example, which shows taking a view, and that the view is an AbstractVector but NOT a Vector:

julia> using ArrayViews

julia> x = rand(5)
5-element Array{Float64,1}:
 0.371029
 0.538613
 0.855282
 0.667787
 0.624045

julia> y = x[2:3]
2-element Array{Float64,1}:
 0.538613
 0.855282

julia> z = view(x,2:3)
2-element ArrayViews.ContiguousView{Float64,1,Array{Float64,1}}:
 0.538613
 0.855282

julia> x[2] = 999
999

julia> x
5-element Array{Float64,1}:
   0.371029
 999.0     
   0.855282
   0.667787
   0.624045

julia> y
2-element Array{Float64,1}:
 0.538613
 0.855282

julia> z
2-element ArrayViews.ContiguousView{Float64,1,Array{Float64,1}}:
 999.0     
   0.855282

julia> map(a->isa(a,AbstractVector), (x,y,z))
(true,true,true)

julia> map(a->isa(a,Vector), (x,y,z))
(true,true,false)

All of this is to prevent unnecessary allocation and copying of temporary vectors/matrices, for example when looping through rows of a matrix.

from onlinestats.jl.

As part of the Adagrad stuff that I'm working on right now, I'll be adding the AVecF and AMatF aliases, and I'll start to replace method signatures, etc with abstract versions where it makes sense, so that the functions could be called with arrays, views, sparse arrays, or anything else like it.

from onlinestats.jl.

I've been operating under the impression that x[2:3] does not create a copy. Oh well, time to start using view.

After changing this line in common.jl,

# update!{T<:Real}(o::OnlineStat, y::Vector{T}) = (for yi in y; update!(o, yi); end)
update!{T<:Real}(o::OnlineStat, y::AbstractVector{T}) = (for yi in y; update!(o, yi); end)

I have no broken tests from making your proposed VecF/MatF change. I say we make the change. Users should expect ArrayViews types to work.

from onlinestats.jl.

Just a heads up... I'm trying to use both VecF and AVecF (same with matrices). There are times when you want a regular Vector, and times when you want an AbstractVector. We can still do VecF as abstract if you want... I'm not 100% on it.

from onlinestats.jl.

Check out the streamstats/adagrad.jl file in the tom branch to see how I'm using it.

from onlinestats.jl.

I was going to write some more tests for OnlinePCA and it looks like changing the definition of row to use AMatF has broken a thing or two. Even if smooth!() didn't restrict the vectors you're smoothing to contain the same type, this would be trying to assign values to a StridedView.

EDIT: Nevermind the second part...I was thinking you couldn't do that, but it looks like you can with no problem.

julia> o = OnlineStats.OnlinePCA(x, 5)
ERROR: `smooth!` has no method matching smooth!(::StridedView{Float64,1,0,Array{Float64,2}}, ::Array{Float64,1}, ::Float64)
 in update! at /Users/jtday/.julia/v0.3/OnlineStats/src/linearmodel/opca.jl:84
 in update! at /Users/jtday/.julia/v0.3/OnlineStats/src/linearmodel/opca.jl:106
 in OnlinePCA at /Users/jtday/.julia/v0.3/OnlineStats/src/linearmodel/opca.jl:45

from onlinestats.jl.

I'm fine with removing the same-type restriction for smooth!.

from onlinestats.jl.

I think smooth! can accept AVecF and AMatF.

On Jul 11, 2015, at 10:13 AM, Josh Day [email protected] wrote:

I was going to write some more tests for OnlinePCA and it looks like changing the definition of row to use AMatF has broken a thing or two. Even if smooth!() didn't restrict the vectors you're smoothing to contain the same type, this would be trying to assign values to a StridedView.

julia> o = OnlineStats.OnlinePCA(x, 5)
ERROR: smooth! has no method matching smooth!(::StridedView{Float64,1,0,Array{Float64,2}}, ::Array{Float64,1}, ::Float64)
in update! at /Users/jtday/.julia/v0.3/OnlineStats/src/linearmodel/opca.jl:84
in update! at /Users/jtday/.julia/v0.3/OnlineStats/src/linearmodel/opca.jl:106
in OnlinePCA at /Users/jtday/.julia/v0.3/OnlineStats/src/linearmodel/opca.jl:45
—
Reply to this email directly or view it on GitHub.

from onlinestats.jl.

function smooth!{T}(avg::Vector{T}, v::Vector{T}, λ::Float64) ...

was the implementation before, which doesn't appear to work with AbstractArrays

julia> OnlineStats.smooth!(view(x, :, 1), view(x, :, 2), .5)
ERROR: `smooth!` has no method matching smooth!(::ContiguousView{Float64,1,Array{Float64,2}}, ::ContiguousView{Float64,1,Array{Float64,2}}, ::Float64)

from onlinestats.jl.

I think this should work:

function smooth!(avg::AVecF, v::AVecF, lambda::Float64) ...

which is equivalent to:

function smooth!(avg::AbstractVector{Float64}, v::AbstractVector{Float64}, lambda::Float64) ...

from onlinestats.jl.

Cool. Yep, at least all the tests still pass with that.

from onlinestats.jl.