fusion-jena / abecto Goto Github PK

An additional parameter should enable the optional use of TDB2 instead of the memory storage to enable processing of large datasets.

Open questions:

• Where should the TDB2 storage be located?
  • temporary folder
  • folder specified by the user
  • folder optionally specified by the user, otherwise temporary folder
• If the location was specified by the user, what should happen to the storage after processing?
  • always delete
  • if location was specified, keep, otherwise delete
  • if location was specified, optionally keep, depending on another parameter, otherwise delete
• If the location was specified by the user, what should happen if it already contains an TDB2 storage?
  • always throw an error
  • always delete beforehand
  • reuse, if parameter --loadOnly is set, otherwise throw an error
    • What to do about the Plan Dataset File parameter in this case?
  • reuse, if parameter --loadOnly is set, otherwise delete beforehand
    • What to do about the Plan Dataset File parameter in this case?

The association to an aspect with av:relevantResource for resources mapped in predefined metadata graphs should get calculated automatically, to ease the manual adjustment of mappings. This would make maintaining of av:relevantResource in plan configurations superfluous.

Approach:

• search for the belonging aspect of each mapped resource in Step#run()

It should be possible to reuse step results to be able to avoid re-execution of expensive steps.
With new parameters --continueWith and --continueAfter it should become possible to specify one or multiple steps and their predecessor not to execute, if the loaded dataset already contains results of them.