Giter Club home page Giter Club logo

urban-analytics / dust Goto Github PK

View Code? Open in Web Editor NEW
20.0 6.0 6.0 1.45 GB

Data Assimilation for Agent-Based Models - A research project at the University of Leeds, funded by the European Research Council

License: MIT License

Kotlin 0.01% Jupyter Notebook 98.70% TeX 0.27% Makefile 0.01% Python 0.91% Java 0.09% R 0.01% Shell 0.01% HTML 0.02% Batchfile 0.01%
agent-based-modeling agent-based-simulation urban-analytics data-assimilation particle-filter kalman-filter

dust's Introduction

dust

Main dust website: https://urban-analytics.github.io/dust/

The DUST project is an initiative at the University of Leeds that has been funded with €1.5M from the European Research Council. It started in Janurary 2018 and was featured as one of the Research Council’s highlighted projects in a recent press release. For more information please contact the Principal Investigator: Nick Malleson (http://www.geog.leeds.ac.uk/people/n.malleson/).

Project Results

The project ended in December 2023. If you would like to learn more about what the project did, the best places to look are on the publications and presentations pages of the main project website.

Most of the project code is in this repository, but there are some other related repositories as well, particularly looking at the footfall analysis work that we did in the final stages:

Project Aims

Civil emergencies such as flooding, terrorist attacks, fire, etc., can have devastating impacts on people, infrastructure, and economies. Knowing how to best respond to an emergency can be extremely difficult because building a clear picture of the emerging situation is challenging with the limited data and modelling capabilities that are available. Agent-based modelling (ABM) is a field that excels in its ability to simulate human systems and has therefore become a popular tool for simulating disasters and for modelling strategies that are aimed at mitigating developing problems. However, the field suffers from a serious drawback: models are not able to incorporate up-to-date data (e.g. social media, mobile telephone use, public transport records, etc.). Instead they are initialised with historical data and therefore their forecasts diverge rapidly from reality.

To address this major shortcoming, this new research project will develop dynamic data assimilation methods for use in ABMs. These techniques have already revolutionised weather forecasts and could offer the same advantages for ABMs of social systems. There are serious methodological barriers that must be overcome, but this research has the potential to produce a step change in the ability of models to create accurate short-term forecasts of social systems.

The project will evidence the efficacy of the new methods by developing a cutting-edge simulation of a city – entitled the Dynamic Urban Simulation Technique (DUST) – that can be dynamically optimised with streaming ‘big’ data. The model will ultimately be used in three areas of important policy impact: (1) as a tool for understanding and managing cities; (2) as a planning tool for exploring and preparing for potential emergency situations; and (3) as a real-time management tool, drawing on current data as they emerge to create the most reliable picture of the current situation.

dust's People

Contributors

aw-west avatar benjaminisaac0111 avatar kevinminors avatar ksuchak1990 avatar ld-archer avatar ldarcher94 avatar leminhkieu avatar nickmalleson avatar patricia-ternes avatar robertclay avatar vijay47kumar avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

dust's Issues

Write Thesis

  • Abstract
  • Acknowledgements
  • Contents
  • Introduction
  • Literature Review
  • Methodology
  • Results
  • Discussion
  • Conclusions
  • References
  • Appendices

Add stationsim environment figures to ODD

Produce the following figures and introduce them into the ODD for stationsim:

  • Layout of original stationsim environment
  • Layout of new stationsim environment
  • Concourse of Grand Central Station

Should we get rid of 'wiggles'?

They kind of made sense in the original version when movement was always left to right, but were largely a fudge. Now that there is better collision detection, should we redesign the collision avoidance routine?

Test clock setup

  • Test that station clock is set up as expected (i.e. location, size, etc.)

Test gate setup

  • Check that gates are where they are supposed to be when we create an instance of the model.

Test agent distance calculation

  • Test that calculation of distance between agents returns correct values.
  • e.g. if Agent 1 is at $(5, 5)$ and Agent 2 is at $(10, 15)$, then the distance returned should be $5\sqrt{5}$.

Pass `Model` reference to `Agent` once in constructor

Include a pointer to the parent model in the stationsim Agent constructor. Then it will no longer be necessary to pass references to the model in Agent functions (e.g. Agent.activate(self, model) becomes Agent.activate(self)

Write ODD section on Input Data

  • Does the model use input from external sources such as data files or other models to represent processes that change over time?
  • Note that ‘Input data’ does not refer to parameter values or initial values of state variables.

Write ODD section on Process Overview and Scheduling

  • Who (i.e., what entity) does what, and in what order?
  • When are state variables updated?
  • How is time modelled, as discrete steps or as a continuum over which both continuous processes and discrete events can occur?
  • Except for very simple schedules, one should use pseudo-code to describe the schedule in every detail, so that the model can be re-implemented from this code.
  • Ideally, the pseudo-code corresponds fully to the actual code used in the program implementing the ABM.

Kill particles that are taking too long to complete

(Patricia I'm happy to look at this, but would be interested in your thoughts).

When running large numbers of particles, you end up with a few that take ages to complete. E.g. see the screenshot. At the start of the DA window there were 16 python processes all running simultaneously, working their way through a population of 1000 particles. However, after a few hours, most particles have finished but a few remain and take ages. Here you can see that there are only 4 processes remaining. This is because the other 996 particles finished hours ago, but these last four are still going.

Screenshot 2020-05-05 at 16 33 46

I'm not sure why this is happenning. It might be to do with loads of random collisions taking place, but maybe there's a bug. Either way, one way round this might be to keep track of how long the particles are taking to complete and if any take too long, say more 10* the mean runtime of the whole population then we kill the process and discard them...

Get EnKF running with StationSim-GCS

Following on from a review of the new version of StationSIm, I need to see how my Ensemble Kalman Filter runs with it and fix any compatibility issues.

Refactor EnKF analysis code

As with the EnKF code that that it uses, the code for running EnKF experiments and analysis could use some refactoring to make it more readable and concise. This should ultimately culminate in experiments being moved into a notebook similar to those used for other DA methods; utility functions for experiments may be left in an analysis module.

Refactor EnKF code

After a cursory glance at the EnKF it appears that the code could do with a little refactoring. This is partly be an exercise in reacquainting myself with the code, but also should help with the following:

  • help to make it more readable (as an when papers are published that refer to this code, it should be in a state whereby others outside DUST can pick it up),
  • ensure that it is compliant with both the original StationSim and the new StationSim_GCS.

Test gate_out allocation

  • Test to ensure that upon setup, each agent is allocated an appropriate exit gate.
  • Importantly, this exit gate should not be the same as the entrance gate.

Write ODD section on Initialisation

  • What is the initial state of the model world, i.e., at time t=0 of a simulation run?
  • In detail, how many entities of what type are there initially, and what are the exact values of their state variables (or how were they set stochastically)?
  • Is initialization always the same, or is it allowed to vary among simulations?
  • Are the initial values chosen arbitrarily or based on data?
  • References to those data should be provided.

Review StationSim-GCS developments

Since I have been away on placement, StationSim has received an overhaul. I should have a look to see what has changed and how it runs.

Identify StationSim submodels for ODD

  • What, in detail, are the submodels that represent the processes listed in ‘Process overview and scheduling’?
  • What are the model parameters, their dimensions, and reference values?
  • How were submodels designed or chosen, and how were they parameterised and then tested?

Review project roadmap

  • Following a chat with Minh last, it is worth reviewing the structure of roadmap.
  • Should we leave Testing EnKF Assumptions to end? Perhaps this should be earlier?
  • Should there be a section here for a comparison with a basic PF?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.