I have a test suite that includes a bunch of test suites/cases that takes about 15 minutes to complete.
It would be great if there will be an option to use N processes simultaneously, similar to make -j 8 option which uses 8 processes to build project.

Now that there exists a PROGRAM syntax element, it is natural to define the file/source executor in terms of such a PROGRAM.

The actor is specified in [conf]; a program can be defined in terms of symbols; but [act] cannot contain symbols for the moment. It would be natural to add symbol support to [conf].

It should parse a PROGRAM and execute it as the "action to check".
Any type of program should be supported, as "run" does.

Rename it to a name that indicates that it runs a PROGRAM, e.g. "program-executor".

Any transformation defined for the program should be applied to stdout.

A transformation in terms of a PROGRAM, works like shell pipes - transformation is done by supplying input on stdin and collecting output from stdout.

Currently, the concept for transforming text is "lines transformer". This does not fit so well with a transformation in terms of a PROGRAM, since such a transformation do not necessarily operate on lines, but rather strings (that may represent a sequence of lines).
So, as a preparation for this issue, the "lines transformer" concept maybe should be changed to a "string transformer" concept.

How should non-zero exitcode from PROGRAM be handled? Ignored, or treated as an error, causing the test to HARD_ERROR?

Ability to use an externa program as a file-matcher.

run PROGRAM

Supplying the file to match

The program must be given the name of the file to match.

Would be nice to be able to use different ways to do this, since there is no way that works well in every case.

run -argument first P
run -argument last P
run -replace-argument '{}' P
run -env-var FILE_ARG_VAR P

Add an option
-from PROGRAM
as has been done for "stdout" and "stderr"

dev-manfre:~/repos/git/playground/jaxlondon/consumer [master] $ exactly test.case
Traceback (most recent call last):
File "/usr/local/bin/exactly", line 7, in
from exactly_lib.default.default_main_program_setup import main
File "/usr/local/lib/python2.7/site-packages/exactly_lib/default/default_main_program_setup.py", line 14
def default_main_program() -> main_program.MainProgram:
^
SyntaxError: invalid syntax

Ability to use a program as files-matcher

run PROGRAM

Passing files to the program

There are different options for passing the files to match to the program.

• supplying each file as it's absolute path, one per line on stdin
• the above but separate paths with null, as xargs -0
• make the root dir the cwd for the program, and pass relative path names on stdin
• the above with the root dir passed as argument or env var (see corresponding issue for file-matcher)

It would be convenient to be able to share configuration with shell scripts.

Shell scripts (bash) declare string values by
VAR=VAL

A directive - including-string-defs - could read such a file, and make each VAR=VAL a definition of a string symbol.

File inclusion mechanism

The file inclusion mechanism of shell scripts - source, . - should be implemented as well.

Variable reference mechanism

${NAME}, and maybe also $NAME,
should be a symbol reference (to a string symbol)

Empty lines and space

Empty lines and shell comments should be ignored - it would be good to interpret as much of the shell syntax used for defining simple string variables.

$ xly empty-interval.case 
IMPLEMENTATION_ERROR
In [assert]
empty-interval.case, line 42

contents airports.csv
         -transformed-by select line-num <= 10
         equals
         -file expected/airports.csv

Exception:
'ascii' codec can't decode byte 0xc3 in position 118: ordinal not in range(128)

One failing part is the application of the string-transformer in

instructions.assert_.utils.file_contents.parts.file_assertion_part.FileToCheck#_write_transformed_contents

This will give HARD_ERROR

def _write_transformed_contents(self):
    try:
        with self._transformed_file_path.open('w') as dst_file:
            with self.lines() as lines:
                for line in lines:
                    dst_file.write(line)
    except ValueError as ex:
        raise PfhHardErrorException(str(ex))

... but there is also an unidentified failing part

which is, at least, not

exactly_lib.instructions.assert_.utils.file_contents.parts.equality.EqualityContentsAssertionPart#_do_compare

codec handling

solution might be to call open() with an "encoding" argument

See "encoding" attribute of open() in Python doc

see codecs module

Each instruction in a section document should have a name.

This name should be stored together with the instruction in the Section Content Element.

Motivation

Some instructions need to have a name (for different purposes). But each instruction, and instruction config, should not need to handle this themselves. Instead, the framework should supply this information.