Scientists are used to create and to work with simplified views of reality, also known as models. These models are typically evaluated and/or validated through a limited set of statistics such as the root mean square error (RMSE), the mean absolute error (MAE), the coefficient of determination (R2), etc.

In some cases, various statistics are integrated into an index, or into a new metric, or into a “fancy” visualization (see, for instance, the Taylor diagram). Yet, statistics and model evaluation metrics have different properties and it is well-known among modelers that model evaluation or validation heavily depends on the selected statistic (i.e. RMSE, MAE, R2, etc.).

This MSc thesis will start by reviewing the existing literature on this topic (see for instance papers from Willmott and co-authors). This review should highlight the pros and cons of using non spatio-temporal metrics for model validation. Then, the student will find, test and compare methods that allow validating spatio-temporal models by the patterns that they produce. Lots of literature exist on this topic in the domain of weather forecasting (see, for instance, references and URLs linked to this topic).

A variety of agent-based and of data-driven (machine learning based) models is available at the GIP department to design one or more case studies for this MSc topic.