Test performance drops quickly after running LSQB test about kuzu HOT 2 CLOSED

The new feature recently deployed, CSV_TO_PARQUET, converts CSV files into PARQUET files. The idea is to temporarily create the parquet files while executing the tests and then remove the files after all tests run.
However, ctest executes the binary e2e_test multiple times because it separates in multiple processes. So it's removing and creating the parquet_temp multiple times instead of re-using the generated parquet files.
It's not possible to set up a custom environment with a teardown callback directly in the script, so I'm checking other alternatives to solve this issue.
A temporary workaround is either comment e2e_test.cpp:FileUtils::removeDir(parquetDatasetTempDir); to keep the files, or rename interactive-short-parquet.test to interactive-short-parquet.disabled and also lsqb_queries_parquet.test to lsqb_queries_parquet.disabled

from kuzu.

Update:

ctest command calls the binary e2e_test for each test. We currently have 222 tests, so for each test ctest runs somethling like:

Test command: /Users/rfdavid/Devel/waterloo/kuzu/build/release/test/runner/e2e_test "--gtest_filter=BinderErrorTest.AddPropertyDuplicateName" "--gtest_also_run_disabled_tests"

e2e_test recursively scans test_files directory, parses all .test files, convert to parquet the necessary CSV, registers all the tests, and then it executes only the test passed in the argument. This is executed 222 times, but only now we noticed the performance due to the csv -> parquet conversion.

from kuzu.