vh-d / rflow Goto Github PK
View Code? Open in Web Editor NEWRflow is a general-purpose workflow management framework for R
License: Other
Rflow is a general-purpose workflow management framework for R
License: Other
Trying to create a new node with "ENV.id-1" id results in "NA.NA" id. The system should issue an error instead.
File node would benefit from a method that would allow user to move target without triggering rebuild of all downstream nodes.
For profiling and other purposes, it would be nice to have a record of time-to-complete everytime a job finishes.
If make()
tracks nodes it visited it can skip these and finish sooner.
Some update_definition()
methods ignore/skip properties set to NULL. This may be solved by using default values in update()
so removing a property from definition would lead to using the default value.
If we store cache objects together with its hash we would not need to recompute the hash again when restoring objects from the cache.
Currently we cannot initialize r_node with a target object in an environment outside of global environment.
Currently a list of parameters for setting up cache is passed to all constructors including those that ignore it.
Currently, cache is stored into a subfolder of the main rflow folder. This folder is often indexed by IDE (e.g. RStudio) which slows things down.
And expressions may be converted to functions during initialization so only functions would be stored.
Support for RMarkdown documents may be implemented in several ways.
One way is to create a rmarkdown class (subclass of file_node probably). This would take a RMarkdown document and inject code loading its dependencies at the beginning of the script.
Another way is to use current file_node and implement API to access values of dependencies from RMarkdown documents easily (without loading the whole Rflow etc...).
Instead of .RFLOW[["node_id"]]$value
we could use some shortcut operator. Candidates are:
.DATA[["target_id"]]
or just .D[["target_id"]]
.DATA("target_id")
or just .D("target_id")
It would be nice to have the option to automatically trigger one or many tests whenever a target is evaluated/rebuild.
Scripts would need to be tracked for changes.
Nodes such as db tables or csv files may need custom settings when reading and/or writing. We need some way to specify these in node definitions or to pass them to reader/writer functions during interactive sessions.
Currently, when disconnecting a DBI connector, all nodes representing targets from the associated DB has to be re-initiated. It would be convenient to have dedicated environment classes for wrapping DBI (and other) connectors. Reconnecting to a DB would happen through these objects.
Cache for large R objects may be loaded when needed instead of when the object is initialized. This would speed up the initial setup at the cost of slower first builds. We would save the time of loading cache for nodes that are not touched during the session.
The self$sql_code
property is no longer used so the print()
method should concatenate self$sql[[i]]$code
and use that instead.
Ideally user would have an option to pass a styler function as a global option or specifically to a particular job.
Perhaps .last_changed private property may be removed. It is only used in R nodes where it is just an alias for timestamp of last hash anyway.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.