Calibrating an Agent-Based Model using Machine Learning

Roadmap according to Herwig (2021)

Objective:

This research topic aims to extend an existing Covid-19 model with Machine Learning surrogate model to calibrate the roadmap values used by the Dutch government during the Covid pandemic.
Simulated versus actual number of disease cases (Herwig, 2021)

Description:

One of the shortcomings of empirical Agent-Based Models (ABMs) is that they are difficult to calibrate. They often have many variables and, therefore, many parameter combinations that must be checked to find the optimal parameter set. The Surrogate Analysis Approach (SAA) is a way to overcome this problem.

SAA generates a surrogate machine learning model that can reduce the original model's dimensionality while maintaining the original ABM's complexity. This model is not a "copy" of the original model but uses much less CPU time (Angione et al., 2022) and helps in the calibration phase.

The key to the successful application of this method is the selection of the learning algorithm (Zhang et al., 2020). Agent-Based models are often used to model complex systems, and these systems are known to produce emerging patterns like state transitions and counterintuitive results. A small change in the value of one variable can lead to a significant change in outputs. When applying an SAA approach, the chosen Machine Learning model should be able to produce these non-linear outcomes. Some authors claim that Artificial Neural Networks (ANN) are particularly suitable for this (Angione et al., 2022).

Recently we developed an ABM to simulate Covid-19 in the Netherlands. This model mimics the "Route Kaart" (RoadMap) of the government. Based on the number of disease cases and the hospitalization, this roadmap determines the Covid-19 risk level. Each risk level is associated with intervention measures like wearing face masks, working from home, closing schools etc. Threshold values decide when to move to the next risk level. In the current model version, we use the threshold values defined by the government. However, the government "updated" the roadmap frequently, and calibration of the threshold values can improve the model.

Recently we developed an ABM to simulate Covid-19 in the Netherlands (Herwig et al., 2021). This model mimics the "Route Kaart" (RoadMap) of the government. Based on the number of disease cases and the hospitalization, this roadmap determines the Covid-19 risk level. Each risk level is associated with intervention measures like wearing face masks, working from home, closing schools etc. Threshold values decide when to move to the next risk level. In the current model version, we use the threshold values defined by the government. However, the government "updated" the roadmap frequently, and calibration of the threshold values can improve the model.

This research topic aims to extend an existing Covid-19 model (created in Netlogo) with an SAA part using a Machine Learning algorithm to calibrate the roadmap values used by the Dutch government during the Covid pandemic. The research splits into the following phases;

  1. Gain an understanding of the existing NetLogo Covid-19 model
  2. Select a Surrogate Machine Learning Algorithm, Fast Sampling Procedure and the Performance Measure of the Surrogate Model
  3.  Implement and test the SAA model
Calibration of ABM via Surrogate ML model

References:

  • Angione, C., Silverman, E., & Yaneske, E. (2022). Using machine learning as a surrogate model for agent-based simulations. PLOS ONE, 17(2), e0263150. https://doi.org/10.1371/journal.pone.0263150

  • Zhang, Y., Li, Z., & Zhang, Y. (2020). Validation and Calibration of an Agent-Based Model: A Surrogate Approach. Discrete Dynamics in Nature and Society, 2020, 6946370. https://doi.org/10.1155/2020/6946370

  • Herwig, A. (2021). Modelling governmental risk perception and coping appraisal during the COVID-19 pandemic in the Netherlands. (MSc). University of Utrecht, 2021. (UU student number: 6419151)

Domain(s):

Study Program(s):

Researchers working on this field: