lemon24 / linesieve Goto Github PK

For each line, output named groups of first matching pattern (of many) as a list of key-value pairs (as returned by groupdict()).

TODO: A better name than "extract".

Output format should be at least:

• tab-separated values, like split/cut (TBD how to delimit key from value)
• JSON or JSON Lines (whichever jq can read)

Nice to have:

• specify preferred ordering of keys (maybe not for JSON)
• handle lines that don't match anything in a sane way
• a mode where you give a pattern with "key" and "value" groups and it iterfinds them in the "line"

In-memory fine, initially.

Sort key regex (named, first group, whole match; like --section marker). Maybe multiple groups too?

Would be nice to have natural/human sort (see Unix sort flags).

Update:

Relevant Unix sort flags:

• -n, --numeric-sort
• -h, --human-numeric-sort (can be added later)
• -r, --reverse
• -k, --key=POS1[,POS2], "POS is F[.C][OPTS], where F is the field number and C the character position in the field"
• -t, --field-separator=SEP (arguably, both this and --key should work like linesieve split)

It should also be possible to pass a regex key:

• no groups: full match
• numeric groups: ???
• named groups: custom order, with flags like --key OPTS (a la linesieve parse name__value feature)

It should be possible to treat the following as 3 "lines":

start abcd
start ef
gh
start third

This is useful for log files that start with e.g. a timestamp.

It should like be possible to specify both a start and/or an end separator; also see how awk does it. Worth considering what happens "outside" of separators.

If multi-line is enabled, it should probably imply re.DOTALL (? if yes, we can just enable it everywhere, since single-line cannot use it).