example_calm_repop not running about populationsim HOT 8 OPEN

Repop works once I skip the summarize.repop module in the settings file.

from populationsim.

Hi @jawadmhoque, are you running the example as is, or did you update any of the settings? I noticed the settings.yaml file for example_calm_pop is configured to resume from summarize step: resume_after: summarize
Perhaps try setting it to blank like this resume_after: for it to run through all steps.

from populationsim.

Hi @jawadmhoque, are you running the example as is, or did you update any of the settings? I noticed the settings.yaml file for example_calm_pop is configured to resume from summarize step: resume_after: summarize Perhaps try setting it to blank like this resume_after: for it to run through all steps.

Hey @binnympaul . I am running the example as is. I tried setting the resume_after step to blank and it fails at the repop_setup_data_structures step with a Key Error on households. Here is the command line outputs:

INFO - Read logging configuration from: configs\logging.yaml
INFO - GROUP_BY_INCIDENCE_SIGNATURE: True
INFO - INTEGERIZE_WITH_BACKSTOPPED_CONTROLS: True
INFO - SUB_BALANCE_WITH_FLOAT_SEED_WEIGHTS: False
INFO - meta_control_data: None
INFO - control_file_name: None
INFO - USE_CVXPY: False
INFO - USE_SIMUL_INTEGERIZER: True
INFO - Time to execute open_pipeline : 0.033 seconds (0.0 minutes)
INFO - Time to execute preload_injectables : 0.016 seconds (0.0 minutes)
INFO - #run_model running step input_pre_processor
Running step 'input_pre_processor'
INFO - Using table list: [{'filename': 'repop_control_totals_taz.csv', 'tablename': 'TAZ_control_data'}]
INFO - Reading CSV file data\repop_control_totals_taz.csv
INFO - registering table TAZ_control_data
Time to execute step 'input_pre_processor': 0.00 s
Total time to execute iteration 1 with iteration value None: 0.00 s
INFO - #run_model running step repop_setup_data_structures
Running step 'repop_setup_data_structures'
Traceback (most recent call last):
  File "run_populationsim.py", line 63, in <module>
    pipeline.run(models=steps, resume_after=resume_after)
  File "C:\Users\USJH706661\Anaconda3\envs\popsim_2\lib\site-packages\activitysim\core\pipeline.py", line 617, in run
    run_model(model)
  File "C:\Users\USJH706661\Anaconda3\envs\popsim_2\lib\site-packages\activitysim\core\pipeline.py", line 476, in run_model
    orca.run([step_name])
  File "C:\Users\USJH706661\Anaconda3\envs\popsim_2\lib\site-packages\orca\orca.py", line 2168, in run
    step()
  File "C:\Users\USJH706661\Anaconda3\envs\popsim_2\lib\site-packages\orca\orca.py", line 972, in __call__
    expressions=self._argspec.defaults)
  File "C:\Users\USJH706661\Anaconda3\envs\popsim_2\lib\site-packages\orca\orca.py", line 1131, in _collect_variables
    thing = all_variables[expression]
KeyError: 'households'
Closing remaining open files:output\pipeline.h5...done

But running it with the pipeline from the example_calm output and skipping the summarize step still works. As far as I can understand from here , repop mode starts from the last step of the regular run, which is the summarize step. So shouldn't it be specified like that?

from populationsim.

Can confirm I'm getting the same error as @jawadmhoque. And also that the workaround of skipping summarize.repop works for me (keeping in mind that you need to re-copy the h5 file from the original run after the initial crash).

from populationsim.

@jawadmhoque , @conrad-mac : Thanks for confirming the issue. You are right, the repop run must be configured to resume from the summarize step.

I reviewed the code and settings, and looks like the list of geographies should list only the lowest geography among the sub-seed geographies. This is because the controls are only specified at the lowest geographic level for a repop run and other sub-seed geographies are not available for summaries. The geographies settings in the settings.yaml file can be updated as follows for example_calm_repop:
geographies: [REGION, PUMA, TAZ]

The workaround of skipping summarize.repop also works but will not generate any summaries.

from populationsim.

@jawadmhoque , @conrad-mac : Thanks for confirming the issue. You are right, the repop run must be configured to resume from the summarize step.

I reviewed the code and settings, and looks like the list of geographies should list only the lowest geography among the sub-seed geographies. This is because the controls are only specified at the lowest geographic level for a repop run and other sub-seed geographies are not available for summaries. The geographies settings in the settings.yaml file can be updated as follows for example_calm_repop: geographies: [REGION, PUMA, TAZ]

The workaround of skipping summarize.repop also works but will not generate any summaries.

Confirming that editing the geographic setting to have the lowest geography among the sub-seeds work with the summarize.repop step.

from populationsim.

Updated repop settings lead to null values in the second level geographies. Created new issue #170 .

from populationsim.

While we are updating the example(s) #90. It looks like we need to resolve the re-pop example and clean up issues #169 and #170.

from populationsim.