Slow copying of large MAGICC distributions about pymagicc HOT 4 CLOSED

Creating the out dir separately would be totally reasonable!
It would only apply to temporary copies, right?
I think it would be good if Readme/License files were copied as well, but they could be moved into run if necessary.

from pymagicc.

It would only apply to temporary copies, right?

I would have thought that it applies to every call to create_temp_copy

It would be great to keep the current structure of having the run directory as a subdir mimicking the MAGICC folder structure. It should be noted that eventually, the magicc binary will be distributed in a bin subdirectory instead of in the run dir. So for MAGICC7 the 'bin' and 'run' dir will need copied.

temp_dir
+--  run
      |    MAGCFG_USER.CFG
      |    MAGTUNE_XXX.CFG
      |    ...
+--  bin
      |    magicc
      |    magicc.exe
+--  out ( Empty)
|    Installation_and_Use_Instructions.txt
|    License_MAGICC6Executable.txt

from pymagicc.

As a baseline, I have timed the time take to setup and teardown copies using the following snippet:

import time
import numpy as np
from pymagicc import MAGICC7

NUM_TRIALS = 100
times = []

for i in range(NUM_TRIALS):
    t0 = time.time()

    with MAGICC7() as magicc:
        pass
    times.append(time.time() - t0)

print('MEAN: {:.5f}s'.format(np.mean(times)))
print('MIN: {:.5f}s'.format(np.min(times)))
print('MAX: {:.5f}s'.format(np.max(times)))

MAGICC7 (only 10 iterations):

MEAN: 4.44909s
MIN: 2.11318s
MAX: 6.34478s

MAGICC6:

MEAN: 0.01886s
MIN: 0.01594s
MAX: 0.05642s

from pymagicc.

With the new implementation

MAGICC7 (only 10 iterations):

MEAN: 0.36197s
MIN: 0.33411s
MAX: 0.54252s

MAGICC6:

MEAN: 0.02210s
MIN: 0.01646s
MAX: 0.03500s

Order of magnitude reduction for MAGICC7. MAGICC7 is still an order of magnitude slower than MAGICC6 due to more and larger configuration files

from pymagicc.