Generic types/methods for normalizing input data about onlinestats.jl HOT 11 CLOSED

I'm creating a Normalized class with functions

normalize(o::Normalized, x::Float64) = ...  # normalizes and returns float
normalize!(o::Normalized, x::Float64) = ... # calls update! on underlying Var and normalizes
denormalize(...) = ...
...

Is there standard notation/lettering for signifying raw vs normalized variables? what should statenames return?

from onlinestats.jl.

Sorry for answering almost none of your questions and added a few more questions. Typically notation is X and Z (z-score), but should state/statenames just return mean and standard deviation?

Side note: Should we change Var to MeanVar to be more clear on what it estimates?

Could we do something along these lines? Does Normalized store any fields other than Var objects?

normalize(o::Var, x::Float64) = (x - mean(o)) / std(o))
normalize!(o::Var, x::Float64) = (update!(o, x); normalize(o, x))

from onlinestats.jl.

I also made a Vars type last night, which may be easier to use than a vector of Var types.

from onlinestats.jl.

How about something like

type Normalized{W <: Weighting}
    xdata::Vars{W}
    ydata::Var{W}
end

normalize() could return a tuple of standardized x and standardized y

xnew, ynew = normalize!(o.Normalized, x::VecF, y::Float64)

from onlinestats.jl.

• Could we just add normalize/denormalize to Var? Yes I think so. I'll trash my normalize.jl file.
• I don't think we need to change Var to MeanVar because it also tracks the mean... however I do think that it might improve readability to call it Variance??
• Side question... does it make sense to return stddev instead of variance in the state? Or both? Which do you find is more useful?

from onlinestats.jl.

I don't think we need that normalized class at all... Var is plenty as you suggested

from onlinestats.jl.

• I do like Variance a bit better
• stddev is more useful, but is it strange if the Variance type returns mean and standard deviation?

from onlinestats.jl.

Yes... that is a bit strange I suppose. I'll leave it up to you, since I can always map(sqrt, ...)

from onlinestats.jl.

Also, if you use the Vars type, it has a std() method which returns a vector of stddevs.

from onlinestats.jl.

Did you add Vars to the repo? I see means.jl, but no vars.jl

from onlinestats.jl.

Sorry, just pushed it now.

from onlinestats.jl.