cadcad-org / cadcad Goto Github PK

I have a query about cadCAD version 0.4.23 and parameter sweeps.

As far as I can remember the default behaviour of cadCAD is to pre-process the parameters, and extend all parameter lists to match the length of the longest parameter list, for example:

params = {
 'a': [1, 2, 3],
 'b': [0]
}

which is processed and becomes the following, for the last subset:

params = {
 'a': 3
 'b': 0
}

Instead, what currently happens is that the parameter 'b' isn't passed to the policy/state update functions, only for the last subset. If the behaviour has changed, that's okay, but I believe the default behaviour was to extend the parameter lists using the last entry as the default - this worked well.

The "Genesis" value of all states (both internal and exogenous) must be applied at the beginning of every run, not just the first one. Currently, the first mech_step of run 2 builds on top of the last value of the states in run 1. See timestamp and s1 in https://github.com/BlockScience/DiffyQ-SimCAD/blob/5bfa9d0674640c7ba90f791986740133ad0ef5ee/notebooks/test.ipynb

I get this error when trying to use a nix-shell:

cadCAD (master) λ:nix-shell
error: numpy-1.20.3 not supported for interpreter python3.6
(use '--show-trace' to show detailed location information)

As can be seen in the following simulation result, when running monte carlo runs with cadCAD 0.4.21, when subsetting runs by the run column, there is no difference between the runs, even when stochastic processes are present. This issue has been present since cadCAD 0.4.17 I believe. Please help

https://gitlab.com/grassrootseconomics/cic-modeling/-/blob/cadCAD_upgrade/Simulation/CIC_Network_cadCAD_model.ipynb

How to reproduce

Duplicate line 15 of the sys_model_AB_exec example

config_names = ['sys_model_A', 'sys_model_B']
sys_model_AB_raw_result, sys_model_AB_tensor_field, sessions = sys_model_AB_simulation.execute()
sys_model_AB_raw_result, sys_model_AB_tensor_field, sessions = sys_model_AB_simulation.execute()
sys_model_AB_result = pd.DataFrame(sys_model_AB_raw_result)

Output

$ python3 documentation/examples/sys_model_AB_exec.py 

                  ___________    ____
  ________ __ ___/ / ____/   |  / __ \
 / ___/ __` / __  / /   / /| | / / / /
/ /__/ /_/ / /_/ / /___/ ___ |/ /_/ /
\___/\__,_/\__,_/\____/_/  |_/_____/
by cadCAD

Execution Mode: local_proc
Configuration Count: 4
Dimensions of the first simulation: (Timesteps, Params, Runs, Vars) = (1, 1, 1, 5)
Execution Method: local_simulations
Execution Mode: parallelized
Total execution time: 0.12s

                  ___________    ____
  ________ __ ___/ / ____/   |  / __ \
 / ___/ __` / __  / /   / /| | / / / /
/ /__/ /_/ / /_/ / /___/ ___ |/ /_/ /
\___/\__,_/\__,_/\____/_/  |_/_____/
by cadCAD

Execution Mode: local_proc
Configuration Count: 4
Dimensions of the first simulation: (Timesteps, Params, Runs, Vars) = (1, 1, 1, 5)
Execution Method: local_simulations
Execution Mode: parallelized
ValueError: sim_configs' N must > 0
Traceback (most recent call last):
  File "documentation/examples/sys_model_AB_exec.py", line 16, in <module>
    sys_model_AB_raw_result, sys_model_AB_tensor_field, sessions = sys_model_AB_simulation.execute()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cadCAD/engine/__init__.py", line 140, in execute
    final_result = get_final_results(simulations_results, partial_state_updates, eps, sessions, remote_threshold)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cadCAD/engine/__init__.py", line 104, in get_final_results
    for sim_result, psu, ep in list(zip(simulations, psus, eps)):
TypeError: zip argument #1 must support iteration

Investigation

The print statement ValueError: sim_configs' N must > 0 shown in the output above is being generated by this exception handling bit of code, which seems to be masking the problem and allowing execution to continue when it should have been halted. Replacing the print statement with a raise shows that the ValueError exception is being raised by this line in execution.py:
results = flatten(pp.map(lambda params: threaded_executor(params), new_params))

Apparently, this is due to the fact that the pool that was instantiated the first time execute was called is being reused on the second call to execute. See this comment in the pathos repo:

pathos persists it's pools in an internal registry... so that if you create a pool that is "the same" as a previous pool (i.e. same number of nodes, same other configuration parameters), it reuses the old one instead of creating a new pool instance. Hence, you either have to restart an old pool instance that has been "closed", or you have to "delete" the old instance from the registry.

Adding a few lines of code to execution.py try to restart the pool in case it already exists seems to solve the problem

pp = PPool()
# NEW: try to restart the pool
try:
    pp.restart()
except:
    pass #maybe do something?
# \NEW
results = flatten(pp.map(lambda params: threaded_executor(params), new_params))

File "C:\Users\mbarl\AppData\Local\Continuum\anaconda3\lib\site-packages\cadCAD\engine\simulation.py", line 39, in policy_scope_tuner return f(sweep_dict, sub_step, sL, s, kwargs_1=None) TypeError: p_marketing_rate() got an unexpected keyword argument 'kwargs_1'

Working experiment in version 0.4.18: https://github.com/matttyb80/adoption_funnel
Viewed in notebook: https://github.com/matttyb80/adoption_funnel/blob/master/adoption_funnel.ipynb

Error in 0.4.19, same model new branch: https://github.com/matttyb80/adoption_funnel/tree/4.19
Viewed in notebook: https://github.com/matttyb80/adoption_funnel/blob/4.19/adoption_funnel.ipynb

When setting the initial parameters of the system to a single value, i.e. no parameter sweep, the params are provided as a list to the state update and policy functions rather than a dictionary. This isn't expected correct?

The following error is thrown TypeError: list indices must be integers or slices, not str which is simply solved by accessing the specific parameter after selecting the first list index:

m = {
    'a': [2]
}

...

def policy_function_1(_params, substep, sH, s, **kwargs):
    a = _params[0]['a']
    return {'a': a}

From the cadCAD Telegram group

Stephen Young
Hey guys, just did a clean install of anaconda and then did pip install cadCAD
Running the first tutorial I had to manually install the following modules: fn
funcy
pathos
Wasn't an issue, just did a pip install for each. I'm quite new to python but seems like missing dependency config somewhere?

I am able to import cadCAD in normal python files, but not in Jupyter. The error is that the library is missing, however a !pip freeze confirms the cadCAD library is present.

I have recently done a fresh distro install, so I can not say with confidence whether or not I was able to import cadCAD on this setup previously or not, so it might be distro related.

For reference:
Distro: Artix Linux
pip freeze: https://gist.github.com/rogervs/062de8c08172fd2c2f015a31d5477ad9
Python version: 3.9.1

The solution was to add __init__.py to the base of the library.

Hello,

I'm running an updated version of robot-marbles-part-4.ipynb notebook from the old repository so it is compatible with the latest version of the library however I'm getting the same results on every simulation when running with N > 1. Seems that the error starts in the following line:

This does not happen If you send the Ns variable instead of the RunIDs as the last parameters. However, the run columns get wrong values and the timestep column doesn't match with the results of each simulation

When adding multiple configuration objects to the global cadCAD configs list, the results are unexpected. I'm sure it relates to this statement in the docs:

As of now it does not support multi - system model simulation because configurations are still appended globally despite append_config being a method of Experiment.

https://gist.github.com/BenSchZA/7b7052ffcd65c5645d65b3e6bb56b329

At the end of the above notebook, I calculated the expected number of rows, if A/B testing was possible. This is also related to issue #247 (comment) I believe.

In the following notebook, the variables dictionary keys are not checked to match against the state update function keys: https://github.com/cadCAD-org/snippets/blob/main/snippets/execution_time_decorator.ipynb

We'd expect the following code, where the key 'a' has a state update function for key 'b', to throw an error when executed by cadCAD, but no error is thrown.

psubs = [
  {
    'policies': {},
    'variables': {
      'a': update_b,
      'b': update_a,
    }
  }
]

Hey Ben,
can you write a quick "thing" about your change given the format in this doc?
https://github.com/cadCAD-org/cadCAD/blob/staging/CHANGELOG.md

On the output dataset, repeat genesis state at the beginning of each run

We often reuse previously defined variables related to the system we're modelling. See for example the original robot and marbles tutorial part 6 how in cell 6 we reuse simulation_parameters originally defined in cell 1 and used in cell 2.

The new config_sim and append_configs implementations modify the content of the variables passed to them as arguments

I might be missing something and maybe there's a more straightforward workaround, but it seems this means that we now need to redefine and pre-process the same configuration before each append_config, like here. In any case, seems this could be avoided by implementing config_sim and append_configs so that they don't mutate mutable reference arguments.

Context

• When State Update Functions and Policy Functions are called, the engine passes arguments s and sL are passed to them.
• s is a dict containing the state of the system prior to the current Partial State Update Block
• sL is a list of dicts containing the historical states of the system from the beginning of the current timestep until the last Partial State Update Block
• If the user requires historical information from previous timesteps, they must encode that information in one or more state variable created specifically for this use
  • A possible general implementation is to create a variable that is updated at each Partial State Update Block and holds a Pandas Dataframe with the full history of all state variables (except itself) (example)

Enhancement Request
Make the engine pass as an argument to State Update Functions and Policy Functions a Pandas Dataframe containing the full state history of the current run. This argument can replace sL as far as the user is concerned, though sL may be kept under the hood if the engine still needs it.

Hello,

I'm running an updated version of robot-marbles-part-6.ipynb notebook from the old repository so it is compatible with the latest version of the library, however, I'm getting the following error when trying to run the A/B testing simulation:

Configurations Length: 2
Execution Method: parallelize_simulations
Execution Mode: parallelized
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-11-ff051015a8c7> in <module>
      1 # %%capture
      2 results = []
----> 3 for raw_result, tensor, _ in executor.execute():
      4     df = pd.DataFrame(raw_result)
      5     results.append(df)

~/anaconda3/envs/cadcad/lib/python3.8/site-packages/cadCAD/engine/__init__.py in execute(self)
    127 
    128             print("Execution Method: " + self.exec_method.__name__)
--> 129             simulations_results = self.exec_method(
    130                 sim_executors, var_dict_list, states_lists, configs_structs, env_processes_list, Ts, SimIDs, RunIDs #Ns
    131             )

~/anaconda3/envs/cadcad/lib/python3.8/site-packages/cadCAD/engine/execution.py in parallelize_simulations(simulation_execs, var_dict_list, states_lists, configs_structs, env_processes_list, Ts, SimIDs, Ns)
     80     len_new_configs_structs = len(new_configs_structs)
     81     pp = PPool(len_new_configs_structs)
---> 82     results = flatten(pp.map(lambda params: threaded_executor(params), new_params))
     83     pp.close()
     84     return results

~/anaconda3/envs/cadcad/lib/python3.8/site-packages/pathos/multiprocessing.py in map(self, f, *args, **kwds)
    135         AbstractWorkerPool._AbstractWorkerPool__map(self, f, *args, **kwds)
    136         _pool = self._serve()
--> 137         return _pool.map(star(f), zip(*args)) # chunksize
    138     map.__doc__ = AbstractWorkerPool.map.__doc__
    139     def imap(self, f, *args, **kwds):

~/anaconda3/envs/cadcad/lib/python3.8/site-packages/multiprocess/pool.py in map(self, func, iterable, chunksize)
    362         in a list that is returned.
    363         '''
--> 364         return self._map_async(func, iterable, mapstar, chunksize).get()
    365 
    366     def starmap(self, func, iterable, chunksize=None):

~/anaconda3/envs/cadcad/lib/python3.8/site-packages/multiprocess/pool.py in _map_async(self, func, iterable, mapper, chunksize, callback, error_callback)
    471         Helper function to implement map, starmap and their async counterparts.
    472         '''
--> 473         self._check_running()
    474         if not hasattr(iterable, '__len__'):
    475             iterable = list(iterable)

~/anaconda3/envs/cadcad/lib/python3.8/site-packages/multiprocess/pool.py in _check_running(self)
    348     def _check_running(self):
    349         if self._state != RUN:
--> 350             raise ValueError("Pool not running")
    351 
    352     def apply(self, func, args=(), kwds={}):

ValueError: Pool not running

test issue

If a run a simulation with the following definition, I'll get a TypeError: cannot convert dictionary update sequence element #0 to a sequence error, which can be difficult for point what's the problem with the code.

My suggestion is that we throw a different error, like TypeError: State Update Functions must have 5 positional arguments. The provided function has N positional arguments

f = lambda a, b, c, d: ('variable', None)

partial_state_update_blocks = [
    {
        'policies': {
        },
        'variables': {
            'variable': f
        }
    }
]

Description:
Parameter Sweep using ExecutionMode().single_mode or ExecutionMode().single_proc

Observed Behavior:

• Simulation results return truncated results (exclude the results of the last Configuration appended to configs) when using Single_Proc & Single_Mode (related issue: #195)
• M (Params) mutated to be different dimensions when using ExecutionMode().single_mode / ExecutionMode().single_proc
• Execution Method & Mode need to print as single threaded for such experiments:

Execution Method: parallelize_simulations
Execution Mode: parallelized

Hi,
I'm using the cadCAD for a while, and I have some simulation with it on my old machine. Today, I installed it on another machine which I think the installed version is now the latest version. but it doesn't work! At first, I thought maybe it's because of different versions of python, so I made sure both machines are using the same version(3.8.10), but it didn't solve the problem. Then I checked with the tutorial code, the first one, and I got this error:

                  ___________    ____
  ________ __ ___/ / ____/   |  / __ \
 / ___/ __` / __  / /   / /| | / / / /
/ /__/ /_/ / /_/ / /___/ ___ |/ /_/ /
\___/\__,_/\__,_/\____/_/  |_/_____/
by cadCAD

cadCAD Version: 0.4.28
Execution Mode: local_proc
Simulation Dimensions:
Entire Simulation: (Models, Unique Timesteps, Params, Total Runs, Sub-States) = (0, 0, 0, 0, 0)
Execution Method: local_simulations
Traceback (most recent call last):
  File "test.py", line 64, in <module>
    raw_system_events, tensor_field, sessions = simulation.execute() # The `execute()` method returns a tuple; its first elements contains the raw results
  File "/home/user/.local/lib/python3.8/site-packages/cadCAD/engine/__init__.py", line 157, in execute
    final_result = get_final_results(simulations_results, partial_state_updates, eps, sessions, remote_threshold)
  File "/home/user/.local/lib/python3.8/site-packages/cadCAD/engine/__init__.py", line 121, in get_final_results
    for sim_result, psu, ep in list(zip(simulations, psus, eps)):
TypeError: 'NoneType' object is not iterable ```

Description

We want to continue the Robot and Marbles tutorial series with a simple tutorial of the Parameter Sweeping feature found in cadCAD.

Parameter Sweeping is a cadCAD feature that runs a model under different possible parameterizations. In a cadCAD model, parameters might refer to intrinsic settings of the system being designed that are still subject to change. They might also refer to assumptions made about properties of the agents that interact with the system, for example. Parameter sweeping is especially helpful in sensitivity analysis, where we are interested in exploring the effects of changing a parameter, or if the desired goals of the system are achieved under a range of assumptions about external agents.

Instructions

1. Fork the Robot and Marbles part 4 notebook and name it Robot and Marbles Part 7 - Parameter Sweeps.
2. Modify it to make use of the Parameter Sweep feature where the robots_probabilities global constant is turned into a parameter to be swept. (Similar to part 6 being a modified fork of part 4 introducing the A/B Testing feature),
3. Compare the results of the sweeps with plots (as was done in Robot and Marbles part 6)

Acceptance Criteria

• robots_probabilities is parameterized
• Variable and function naming conventions match those in the previous robot and marbles tutorials
• The difference in the resulting datasets is clearly visible in the plots
• Plot Style is consistent with earlier robot and marbles tutorials
• The range of the parameter sweep should be appropriate for obvious difference in the resulting datasets
• Documentation assumes readers have completed Parts 1 through 6, and builds on these materials:
• Markdown cells throughout the notebook guiding the reader through the feature
• Code cells contain comments highlighting relevant pieces of the code

References

• https://github.com/BlockScience/cadCAD/blob/master/tutorials/robot-marbles-part-4/robot-marbles-part-4.ipynb
• https://github.com/BlockScience/cadCAD/blob/master/tutorials/robot-marbles-part-6/robot-marbles-part-6.ipynb
• https://github.com/BlockScience/cadCAD/blob/master/documentation/examples/param_sweep.py

My policy function was raising ValueError due to a mis-use of a numpy function which was raising a ValueError. The "'sim_configs' N must > 0" message was thus very confusing.

Suggestion: cadCAD should define it's own exceptions and raise them explicitly before having a final generic exception for cases like this. Cheers 🌵

I've noticed that docstrings are not implemented in some functions. They are very helpful to developers and would also enable auto-generated documentation. Anyone working on this? Thanks

Context:

• In the current implementation of SimCAD, Partial State Update Blocks ("mechanisms") are executed in series;
• Whenever the system model requires State Update Functions to be executed in parallel, the user must collapse them into a single Partial State Update Block;
• The process of collapsing multiple State Update Functions into a single Partial State Update Block requires careful design of Policy Functions, so that they are isolated from each other and the contents of their results do not get inadvertently aggregated. This is accomplished by assigning non-conflicting keys to the returns of those functions;
• The more complex the system, the more difficult it is to avoid making mistakes in said process.

Request
Enhance the SimCAD Engine so that the user can determine in the configuration file if Partial State Update Blocks are to be executed in series or in parallel. Any combination should be possible.

Example diagram:
https://www.lucidchart.com/documents/edit/53e450c2-5f96-42dd-9c6d-90ad07b5e133/5

Suggested interface: list of lists of dictionaries

partial_state_update_blocks = [
    [ #begin mech_step_1
        { #begin partial_state_update_block_1
            'behaviors': {
                'Policy1': policy_func_1,
                'Policy2': policy_func_2
            },
            'states': {
                'var1': state_func_1
            }
        }, #end partial_state_update_block_1
        { #begin partial_state_update_block_2
            'behaviors': {
                'Policy3': policy_func_3,
                'Policy4': policy_func_4,
                'Policy5': policy_func_5
            },
            'states': {
                'var2': state_func_2,
                'var3': state_func_3
            }
        } #end partial_state_update_block_2
    ], #end mech_step_1
    [ #begin mech_step_2
       { #begin partial_state_update_block_3
            'behaviors': {
                'Policy6': policy_func_6,
                'Policy7': policy_func_7
            },
            'states': {
                'var1': state_func_4,
                'var2': state_func_5
            }
        }  #end partial_state_update_block_3
    ] #end mech_step_2
]

For the MVP, we settled on Python dictionaries as the standard output for behaviors, in order to allow for multi-dimensional and multi-type behavior outputs. But as long as they are consistent with each other (ie, behaviors that are "summed" must be of the same type), ideally the user should be allowed to configure their behaviors to output whatever datatype they wanted to.

PoC: 01e70c3

When building the Nix package derivation for cadCAD, the tests fail because of references to a local dev file /Users/jjodesty/Projects/DiffyQ-SimCAD/simulations/external_data/output.csv 📦

Not sure if it's a case of committing that file, or updating those tests some other way.

As part of the following hot-fix #186 , parallelization of simulations was disabled through the commenting out of the following lines

The following simulation, for example, is run in series:

Execution Mode: local_proc
Configuration Count: 1
Dimensions of the first simulation: (Timesteps, Params, Runs, Vars) = (329, 55, 3, 49)
Execution Method: local_simulations
SimIDs   : [0, 0, 0]
SubsetIDs: [0, 1, 2]
Ns       : [0, 1, 2]
ExpIDs   : [0, 0, 0]
Execution Mode: parallelized

By un-commenting the lines in execution.py, the runs are parallelized again, but it's unclear what effect that has on the hot-fixed issue.

A possible feature request for the cadCAD core-tech. It'll reduce the barriers to learn cadCAD. Converting frequently used and cadCAD syntactic functions, methods to a graphical programming language.

Update all the robot and marbles tutorials so that they immediately run in Google Colab from a URL like https://colab.research.google.com/github/BlockScience/cadCAD/blob/master/tutorials/robot-marbles-part-1/robot-marbles-part-1.ipynb

This would require adding a cell in the notebook that pip installs cadCAD

%%capture #avoid printing the output of pip install
!pip install cadCAD

Add direct links to the top of every notebook and to the README

Hello,

I had an issue trying to execute cadCAD with multiple configurations. It works properly when running up to 3 configs. However, when the number of configs is increased to 4, the last configuration's result doesn't appear on the resulting dataframe, although the configuration count indicates the right number when executing run().

This is the run.py script:

# The following imports NEED to be in the exact order
from cadCAD.engine import ExecutionMode, ExecutionContext, Executor

# Simulation configs, input any new simulations here
from covid_19_sir import config
from covid_19_seir import config
from covid_19_seird import config
from covid_19_seir_3778 import config


from cadCAD import configs
import pandas as pd


def run(drop_midsteps: bool=True) -> pd.DataFrame:
    """
    Run all experiments and return their output on the dataset column.
    Each line represents an iteration of the parameter-sweep combinations.
    """
    exec_mode = ExecutionMode()
    multi_mode_ctx = ExecutionContext(context=exec_mode.local_mode)
    run = Executor(exec_context=multi_mode_ctx, configs=configs)
    raw_result, _, _ = run.execute()
    results = pd.DataFrame(raw_result)
    return results

The config scripts follow the same pattern:

MONTE_CARLO_RUNS = 1 # N monte carlo runs

from cadCAD.configuration.utils import config_sim
from cadCAD.configuration import Experiment
from .model.state_variables import genesis_states
from .model.partial_state_update_block import partial_state_update_block
from .model.sys_params import sys_params as sys_params
from .sim_params import SIMULATION_TIME_STEPS


sim_config = config_sim (
    {
        'N': MONTE_CARLO_RUNS,
        'T': range(SIMULATION_TIME_STEPS), # number of timesteps
        'M': sys_params,
    }
)

exp = Experiment()

exp.append_configs(
    sim_configs=sim_config,
    initial_state=genesis_states,
    partial_state_update_blocks=partial_state_update_block
)

If I change the order they are imported, the last config that was imported is still not on the dataframe.

Has anyone had a similar problem?

boop bop

#84

when runs are greater than one, the execution mode defaults to multi-threaded execution when local mode is selected.

when single mode is specified, simulation execution should be single- threaded

The execution mode needs to be clear

• when using jupyter notebook, it should be multi-proccessing (related #242)
• data pipelines for digital twins, should be multi-threading

note:
jupyter has control of the threads and uses tornado for multi-threading
@rogervs

First mentioned in #145 (comment)

As @matttyb80 said somewhere else we would like the ability to override the limit, since config count is not indicative of a memory/time overload. Also would want the error handling for this to be more explicit (this is the current behavior)

A user of cadCAD, @rogervs, recently commented on the fact that run == subset + 1. This doesn't make much sense, as it doesn't provide useful additional info, instead there should be a hierarchy from simulation to subset to run to timestep, where the maximum of the former should never be larger than the maximum of the latter.

Illustrative example: an A/B test of 3 simulations, each with a parameter sweep of 3 parameter sets, 3 Monte Carlo runs, and 100 timesteps.

It should have a hierarchy of a simulation per config that’s simulated, subset per parameter set in the sweep for each simulation, and a run per Monte Carlo run for each subset.

In total, we'd expect 333 or 27 sets of data of 100 timesteps each.

This is probably just a bug or oversight?

In the case where a mechanism like:

def account_for_tax(params, step, sL, s, inputs):
key = 'GRT'
delegation_tokens_quantity = inputs['delegation_tokens_quantity']
delegation_tax_rate = params['delegation_tax_rate']

tax = delegation_tax_rate * delegation_tokens_quantity
value = s['GRT'] - tax
**return key, value**

does not have the bold part (return key, value), you get the following error message:

c:\Users\Josh\anaconda3\envs\py396\lib\site-packages\cadCAD\engine\simulation.py in state_update_pipeline(self, sweep_dict, simulation_list, configs, env_processes, time_step, run, additional_objs)
172 sub_step += 1
173 for [s_conf, p_conf] in configs:
--> 174 states_list: List[Dict[str, Any]] = self.partial_state_update(
175 sweep_dict, sub_step, states_list, simulation_list, s_conf, p_conf, env_processes, time_step, run,
176 additional_objs
c:\Users\Josh\anaconda3\envs\py396\lib\site-packages\cadCAD\engine\simulation.py in partial_state_update(self, sweep_dict, sub_step, sL, sH, state_funcs, policy_funcs, env_processes, time_step, run, additional_objs)
138 return destination
139
--> 140 last_in_copy: Dict[str, Any] = transfer_missing_fields(last_in_obj, dict(generate_record(state_funcs)))
141 last_in_copy: Dict[str, Any] = self.apply_env_proc(sweep_dict, env_processes, last_in_copy)
142 last_in_copy['substep'], last_in_copy['timestep'], last_in_copy['run'] = sub_step, time_step, run
TypeError: cannot convert dictionary update sequence element #1 to a sequence

It would be more helpful to say which mechanism had the issue and what the issue was.

Context

• If multiple State Update Functions require the same computation to be executed based on the current state of the system, that computation will be executed multiple times;
• The more complex the computation is, the more inefficient the simulation will be.

Example

def complex_computation(s):
    res = []
    # complex code here
    return res

def state_update_1(step, sL, s, _input):
    y = 'var_1'
    x = complex_computation(s)[0]
    return (y, x)

def state_update_2(step, sL, s, _input):
    y = 'var_2'
    x = complex_computation(s)[1]
    return (y, x)

partial_state_update_blocks = {
    'PSUB1': { 
        'behaviors': { 
        },
        'states': { 
            'var_1': state_update_1,
            'var_2': state_update_2
        }
    }
}

Current Solutions

>1 - Execute the computation inside a Policy Function

def complex_computation(step, sL, s):
    res = []
    # complex code here
    return {'a': res[0], 'b': res[1]}

def state_update_1(step, sL, s, _input):
    y = 'var_1'
    x = _input['a']
    return (y, x)

def state_update_2(step, sL, s, _input):
    y = 'var_2'
    x = _input['b']
    return (y, x)

partial_state_update_blocks = {
    'PSUB1': { 
        'behaviors': { 
            'complex': complex_computation
        },
        'states': { 
            'var_1': state_update_1,
            'var_2': state_update_2
        }
    }
}

Drawbacks:

1. the computation might not semantically belong within a policy;

>2 - Store the result of the computation in a state variable

def complex_computation(step, sL, s, _input):
    y = 'complex'
    x = []
    # complex code here
    return (y, x)

def state_update_1(step, sL, s, _input):
    y = 'var_1'
    x = s['complex'][0]
    return (y, x)

def state_update_2(step, sL, s, _input):
    y = 'var_2'
    x = s['complex'][1]
    return (y, x)

partial_state_update_blocks = {
    'PSUB0': { 
        'behaviors': { 
        },
        'states': { 
            'complex': complex_computation
        }
    }
    'PSUB1': { 
        'behaviors': { 
        },
        'states': { 
            'var_1': state_update_1,
            'var_2': state_update_2
        }
    }
}

Drawbacks:

1. unnecessarily increases memory usage, because we're storing transient information throughout the whole simulation

Possible Solutions

1. Enhance the engine so as to allow for a single State Update Function to update multiple state variables simultaneously. Something like this:

def state_update_1_2(step, sL, s, _input):
    temp = complex_computation(s)
    return {'var_1': temp[0], 'var_2': temp[1]}

2. Enhance the engine so as to allow fine control of the duration for which state variables are kept in memory?

Relevant Issues:

Tracking:

• Related:
  ** #248
• Closed:
  ** #202 (Contains Work)
  ** #195

New Feature:

• #157

we need multi-processing execution mode

currently, multi-proc is incorrectly mapped to multi-threading, causing confusion. We need to rename the method appropriately

Acceptance Criteria

• we have multi-processing
• multi-proc execution mode should no longer be mapped to multi-threading
• we know the relationship between experimental and production ('digital twins') workflows

Links:
#253
#242

previous conversation:
@rogervs, @BenSchZA

It seems the single_proc Execution mode is only available if you have only one config, else it get switched by auto_mode_switcher() to multimode, parallelize_simulations (line 133 in engine/init.proc ) . Is there a branch or older version that allows single threading for multiple configs, processing them consecutively?

Benjamin Scholtz, [28.12.20 04:22]
[In reply to Roger]
Roger you might find this open issue relevant, about multiprocessing of runs. Might be worth adding the above comment to the issue or opening a new one. #242

cadCAD v0.4.27 (patch update) introduces a breaking change without backwards compatibility. If cadCAD is being used in production, with semantic versioning, it would be expected that users can safely update cadCAD if only the patch version is updated. I would suggest sticking to semantic versioning and in future making v0.4.27 either a minor update with backwards compatibility, or a major update.

On cadCAD 0.4.23, calling the 'timestep' variable on the first PSUB (substep=1) and on the first timestep will get you 0 instead of the expected 1. If you move the PSUB to let's say the second position (substep=2), then you get the timestep as being 1.

This can introduce unexpected behaviour when changing the location of a given PSUB.

To recreate the bug:

1. Pull code from default branch.
2. Run all steps in the robot-marbles-part-6 notebook.

Fails on step 17 with
NameError: name 'rand' is not defined

Expected:
results of both configurations should be appended to the "results" variable.

Hello, I know it is not an issue, but I have no place to find the answer to this question.

I want to know why the name is not “complex adaptive dynamics for Computer-Aided Design”, is there any reason to not use a "for" between "complex adaptive dynamics" and "Computer-Aided Design".

When I introduce cadCAD to my team, my teacher asks this question/(ㄒoㄒ)/~， Could u please tell me.

Simulation results return truncated results (exclude the results of the last Configuration appended to configs) when using Single_Proc & Single_Mode

Hello,

When trying to run "BlockScience/ZarghamPapers" notebooks locally (anaconda/windows), I got some errors when using multiprocessor execution mode. (see:[https://github.com/BlockScience/ZarghamPapers/issues/1]). This could be a generic issue for cadCAD.
Do you have any feedback on the subject?

Regards,

The link underlying the word "Executed" on the following page gives a 414 error:

https://github.com/cadCAD-org/cadCAD/blob/master/documentation/README.md

It should presumably link to the Simulation Execution page:

https://github.com/cadCAD-org/cadCAD/blob/master/documentation/Simulation_Execution.md

Hello!

After having updated the predator/prey model from the demos folder for cadCAD 0.4.18, I'm getting the following error. Does this error look familiar? I'm sure there's some more context I could give, I'll update this issue if I find it.

Execution Mode: local_proc
Configuration Count: 1
Dimensions of the first simulation: (Timesteps, Params, Runs, Vars) = (1000, 6, 1, 2)
Execution Method: local_simulations

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-1fa93129baf3> in <module>
      2 # Tweak the prey_predator_abm/sim_params.py file if you want it to take longer (or not).
      3 start_time = time()
----> 4 experiments = run.run()
      5 end_time = time()
      6 print("Execution in {:.1f}s".format(end_time - start_time))

~/workspace/cadCAD/demos/prey-predator/run.py in run(drop_midsteps)
     22     results = pd.DataFrame()
     23     i = 0
---> 24     for raw_result, _tensor_field, _sessions in run.execute():
     25         params = configs[i].sim_config['M']
     26         result_record = pd.DataFrame.from_records([tuple([i for i in params.values()])], columns=list(params.keys()))

~/Library/Caches/pypoetry/virtualenvs/demos-KUP5ZvRJ-py3.8/lib/python3.8/site-packages/cadCAD/engine/__init__.py in execute(self)
    149 
    150             print("Execution Method: " + self.exec_method.__name__)
--> 151             simulations_results = self.exec_method(
    152                 sim_executors, var_dict_list, states_lists, configs_structs, env_processes_list, Ts, SimIDs, RunIDs,
    153                 ExpIDs, SubsetIDs, SubsetWindows, original_N

~/Library/Caches/pypoetry/virtualenvs/demos-KUP5ZvRJ-py3.8/lib/python3.8/site-packages/cadCAD/engine/execution.py in local_simulations(simulation_execs, var_dict_list, states_lists, configs_structs, env_processes_list, Ts, SimIDs, Ns, ExpIDs, SubsetIDs, SubsetWindows, exec_method)
    138     try:
    139         if len(configs_structs) == 1:
--> 140             return single_proc_exec(
    141                 simulation_execs, var_dict_list, states_lists, configs_structs, env_processes_list, Ts, SimIDs, Ns,
    142                 ExpIDs, SubsetIDs, SubsetWindows, exec_method

~/Library/Caches/pypoetry/virtualenvs/demos-KUP5ZvRJ-py3.8/lib/python3.8/site-packages/cadCAD/engine/execution.py in single_proc_exec(simulation_execs, var_dict_list, states_lists, configs_structs, env_processes_list, Ts, SimIDs, Ns, ExpIDs, SubsetIDs, SubsetWindows, exec_method)
     34         map(lambda x: x.pop(), params)
     35     )
---> 36     result = simulation_exec(var_dict_list, states_list, config, env_processes, T, sim_id, N, subset_id, subset_window)
     37     return flatten(result)
     38 

TypeError: simulation() missing 1 required positional argument: 'configured_N'

"Example Simulation Executions:" under Execution Examples should be on a new line

https://github.com/cadCAD-org/cadCAD/blob/master/documentation/Simulation_Execution.md