Comments (5)
aggregate
is made to be used with a grouping variable. colwise!
would just be an in-place equivalent of colwise
, so something very simple. Cf. https://discourse.julialang.org/t/announcement-dataframes-future-plans/266/25.
Though thinking about it more, I'm not sure how useful such a function would be. colwise
is mainly useful for summary functions which return one or two values for each column. OTC, colwise!
would only work for functions which return the vectors or the same length as the input. So probably better give this function a different name.
The other issue with such a function is that colwise!(col -> col .- mean(col), df)
it would encourage an inefficient pattern which allocates a copy of col
before replacing the old column. A manual loop with x0 = mean(col); map!(x -> x - x0, col)
is much more efficient in that regard. So if we offer such a helper function, it should be smarter than that. We could have it call the function for each element of a column, with additional column-wise arguments à la broadcast
. For example: colwise!((x, x0) -> x - x0, df, colwise(mean, df)
.
from datatables.jl.
Reading your concerns here and thinking about it more, I am also not sure this functionality would be broadly useful. It came out of a natural impulse to work with a DataTable inplace because this is so ideomatic in julia.
Maybe if map! could simply take an eachcol
iterator as argument?
from datatables.jl.
Leaving a breadcrumb to https://stackoverflow.com/questions/44235201/shortcut-to-transforming-a-dataframe, which provides an example of a use case for this.
from datatables.jl.
Are you imagining that colwise!
would work like an in-place aggregate?
example:
julia> dt
10×2 DataTables.DataTable
│ Row │ A │ B │
├─────┼────┼────┤
│ 1 │ 1 │ 1 │
│ 2 │ 2 │ 2 │
│ 3 │ 3 │ 3 │
│ 4 │ 4 │ 4 │
│ 5 │ 5 │ 5 │
│ 6 │ 6 │ 6 │
│ 7 │ 7 │ 7 │
│ 8 │ 8 │ 8 │
│ 9 │ 9 │ 9 │
│ 10 │ 10 │ 10 │
julia> colwise!(normalize, dt)
10×2 DataTables.DataTable
│ Row │ A │ B │
├─────┼───────────┼───────────┤
│ 1 │ 0.0509647 │ 0.0509647 │
│ 2 │ 0.101929 │ 0.101929 │
│ 3 │ 0.152894 │ 0.152894 │
│ 4 │ 0.203859 │ 0.203859 │
│ 5 │ 0.254824 │ 0.254824 │
│ 6 │ 0.305788 │ 0.305788 │
│ 7 │ 0.356753 │ 0.356753 │
│ 8 │ 0.407718 │ 0.407718 │
│ 9 │ 0.458682 │ 0.458682 │
│ 10 │ 0.509647 │ 0.509647 │
# compared to
julia> aggregate(dt, normalize)
10×2 DataTables.DataTable
│ Row │ A_normalize │ B_normalize │
├─────┼─────────────┼─────────────┤
│ 1 │ 0.0509647 │ 0.0509647 │
│ 2 │ 0.101929 │ 0.101929 │
│ 3 │ 0.152894 │ 0.152894 │
│ 4 │ 0.203859 │ 0.203859 │
│ 5 │ 0.254824 │ 0.254824 │
│ 6 │ 0.305788 │ 0.305788 │
│ 7 │ 0.356753 │ 0.356753 │
│ 8 │ 0.407718 │ 0.407718 │
│ 9 │ 0.458682 │ 0.458682 │
│ 10 │ 0.509647 │ 0.509647 │
That should be pretty straight forward to do. What would be the preferable behavior for functions that reduce to scalars like sum/length or extrema? Fail on a dimension mismatch or resize the DataTable to be a single row? (I'm not sure if resize!(column, 1)
will work, in which case it'd have to throw an error because it couldn't modify in place).
example
julia> colwise!(length, dt)
1×2 DataTables.DataTable
│ Row │ A │ B │
├─────┼────┼────┤
│ 1 │ 10 │ 10 │
# or
Error: DimensionMismatch: columns cannot be resized in place, use aggregate(function, dt) instead
from datatables.jl.
Yes, that would make sense. Maybe a bit surprising that you would be able to modify an iterator, but doesn't sound like a big issue.
from datatables.jl.
Related Issues (20)
- documentation link broken HOT 1
- Recommended date to make DataTables branches master HOT 4
- Overwrites DataFrames describe function HOT 11
- "correct" join results HOT 9
- Strange error message at REPL with the latest Julia nightly HOT 5
- complete_cases for selected columns HOT 21
- How do I use CSV.read to return a DataTable instead of a DataFrame? HOT 3
- Make DataTables a major release of DataFrames HOT 3
- duplicate functionality in merge! and hcat! HOT 2
- Tag a new version? HOT 4
- Don't reexport Statsbase HOT 5
- describe does not function HOT 1
- Start replacing Nullable with Null HOT 3
- Throw error for type mismatch using join? HOT 5
- Join on columns of different name HOT 4
- Compiling error HOT 1
- Backporting to DataFrames HOT 7
- Future of DataTables HOT 13
- `==` does not compare columns of `ZonedDateTime`s correctly HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from datatables.jl.