Giter Club home page Giter Club logo

Comments (2)

toliwaga avatar toliwaga commented on May 25, 2024

Currently the code is hard-wired to know that there are two levels of geography below seed, and there are two dedicated processors for balancing those sub_levels.

We can make geographies flexible by specifying them in settings:

geographies: [REGION, PUMA, TRACT, TAZ]
seed_geography: PUMA

Automating the process of iterating down through the tree of sub-geographies should not be difficult, but it might be nice to retain the current feature that each level of sub-geography is handled as a separate step, with data pipeline checkpointing and restartability.

You could imagine moving the _MODELS list to the settings file (which might make sense architecturally for asim generally) and/or introducing a notion of sub-steps to be iterated through either inside or outside the checkpointing.

Interestingly, orca has a similar, but inverted, concept of step iteration built in, but which doesn't play well with out pipelining architecture. In orca, the entire list of steps is iterated over each iter_var.

We want it the other way round. Something like:

models:
- input_pre_processor
- setup_data_structures
- initial_seed_balancing
- meta_control_factoring
- final_seed_balancing
- integerize_final_seed_weights
- {step_name: sub_balancing, iter_vars: [TRACT, TAZ]}
- sub_balancing.TAZ
- expand_population
- summarize
- write_results

(Though we might want to use anchors in the settings file to avoid redundancy.)

from populationsim.

bstabler avatar bstabler commented on May 25, 2024

We decided to add to the pipeliner the ability to read an optional dictionary of arguments for each step. This has lots of uses and means the pipeline acts somewhat like DOS or Bash as well. So, for the example above, the model list will look something like:

models:
- input_pre_processor
- setup_data_structures
- initial_seed_balancing
- meta_control_factoring
- final_seed_balancing
- integerize_final_seed_weights
- sub_balancing, GEOG=TRACT
- sub_balancing, GEOG=TAZ
- expand_population
- summarize
- write_results

from populationsim.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.