joergen7 / cuneiform-legacy Goto Github PK

Also in the visualization, only relevant nodes should be drawn.

Macro calls should look more like apply calls. Thus

• keyword arguments
• allow expressions, not only ids

Conditionals should be first level citizens.

Task names can be displayed inside the boxes if there is only one of them.

Use declare statement to create .dot graphs with meaningful names.

Each task invocation writes things to the stdout and the stderr. Instead of just losing this information, the controlled side-effects of the invocation should appear in the controlled side effect of the workflow execution: the worklfow trace.

One gets the following error message when starting a task body with a tab:

missing BODY at '*'

Assigning the same variable multiple times should be forbidden, because all data types are immutable by definition.

Task nodes have ids which are stored in the log. Since these ids may not be the same in a subsequent run, all task node ids have to be exchanged for signatures, as done for invocations.

Check whether all occurrences of task node ids can be replaced by signatures.

Packages should have dependencies only in one direction.

A test case has been added. This is a crucial bug.

Cuneiform has support of higher order functions. Consider the following example

declare higher-order-fn;

deftask calculate-fn( fn : ) {
echo my-task > $fn
}

deftask my-task( out : a b ) {
cat $a $b > $out
}

target x;

x = apply(
task: apply( task: calculate-fn )
a: ~'hello'
b: ~'world'
);

fa = apply(
task : connect
...
ext : fa
);

The invocations that are supposed to be started after an output reduce task, are never invoked.

Readlink, the tool that resolves the full file name of an output variable, does not check for the existence of a file. Funny errors result.

i -> __I

• Find attipatterns in code
• Issue warning at compile time
• optimize task graph

• Altering the body of a function must also alter its hash.
• Altered intermediate data should affect downstream invocations.
• If an invocation did not produce an output it has to be recomputed.

This is especially important when workflow leaves are heavily altered. Because the current state forces the user to delete all cached results prior to his next run.

Invocation signatures must reliably represent invocations.

• if a task body is altered, the signature is altered too
• if an invocation signature changes, all downstream signatures also change
• use long signatures
• before any intermediate results are reused, compare all logs with all created directories and delete any directories, that aren't mentioned in any log

Some people prefer include. Make them interchangeable. Also can an import eat a string expression instead of just an identifier.

Data items don't have a type, variables in a prototype do have. Don't confuse them.

To ease the parse of a possible provenance software, input and output declarations should have the same form.

Make the log a bag of JSON objects. In the following, it might be useful to create a KNIME node that extracts a table out of the log.

What macro hasn't been found?

Is importing macros erroneous.

• Bootstrapping
• Expectation Maximization
• Simulated Annealing

Right now, the LocalDispatcher filters the report entries for the invocation output. This can be pushed into the Invocation class itself.

Currently, when an error occurs in a Bash script, it may be, that the workflow just continues.

It collects the error messages, but does not halt.

By default, store the log in a separate file in the build directory.

In order to allow provenance-explicit filters we need to be able to produce the empty list as a task result.

However, as task input it shouldn't be allowed.

It should be possible to create foreign code in any given scripting language. Consider the example below:

declare python-example;

deftask + in python( ~result : ~<operand> ) *{
    #!/usr/bin/env python

    result = 0
    for line in operand:
        result += float( line )
}*

target x;

x = apply(
    task: +
    operand: ~'5' ~'10' ~'-7' ~'1'
);

Caching obsoletes the task ids and invocation ids since these can be different for multiple workflow run. Log instead the cacheIds.

This also makes it easier to find the results in the file system.

Types should be checked.

Literals must not anymore be decorated. It suffices to have the task definition prototypes typed.