predict-idlab / tsflex-benchmarking Goto Github PK

It appears that the memory-usage of the sub-threads aren't properly logged with viztracer.

feature extraction:

• performance in terms of number of extracted features
• performance in terms of multiple-window stride combinations
• memory peak reducing with chunking

processing:

• added value of chunking (multithreading for speed-up) & memory peak reducing capabilities

The current benchmarks compare the full feature processing pipeline of different libraries. Those benchmarks are representative of a real-word usage but fail to separate the time to for the feature calculation itself from the overhead of the library itself. This means that depending on the efficiency of the feature implementation in the libraries, the runtime will be different. Consequently, the results could also be very different accordingly to which specific features you pick for the benchmarks themselves.
Therefore, it would be interesting to benchmark the overhead of the different libraries independently of the features. For this, a dummy feature could be implemented in every library that simple returns some random data.