AI for Real-World Crisis Management

What if we knew in advance the optimal public policy in case of a pandemic, natural disaster or economic crisis?

What if we could give answers to our “what if” questions, with the help of statistics and AI? Imagine scenarios like, tsunami prediction after an earthquake, the effect of specific policies on climate change or the efficacy of public health guidelines during a pandemic. For example, what if there is a partial lockdown instead of a global one? What would be the outcome of a specific policy instead of another? The difficulty with such questions, scientifically called counterfactual, is that one cannot do experiments in advance in order to gain insights. The process of answering such questions is called counterfactual inference and it has become a ubiquitous tool in online advertisement, recommendation systems, biomedical sciences, and econometrics.

EURECOM’s Professor Motonobu Kanagawa, an expert in statistical modelling and machine learning (ML) is working on designing mathematical tools for the statistical modelling of the probability distributions of such potential scenario outcomes.

Q. Could your describe the context and motivation of your research?

MK. Counterfactual inference has become a universal tool in several fields such as online advertisement, recommendation systems, biomedical sciences, and econometrics. A counterfactual question states a hypothetical reality that contradicts the observed facts and asks for the alternative outcome under certain assumed conditions. Accurate modelling of outcome distributions associated with different interventions — known as counterfactual distributions — is crucial for the success of such applications.

The goal of this approach is to assess candidate policies, before committing to consequential decisions. For example, policy makers would like to know the possible effects, before implementing lockdowns during the pandemic. In other words, assess before implementing by analysing observational data from the past. Of course, there are certain conditions that the observational data need to satisfy in order to be used.

Q. What are the tools and techniques that you propose?

MK. We propose a new approach that offers more flexibility to counterfactual inference. Our method can be applied if we can define a positive definite kernel on the output space. This allows for the outcome space to be multivariate, and also structured domains (time-series, graphs, images, texts, etc.), quantifying the discrepancy between the counterfactual and observed outcome distributions. The novelty of our approach is that we don’t base our evaluation only on the statistical mean but also on how the whole probability distribution is altered. Also, previous studies propose approaches that can treat only scalar outcome data, but in our case we can use structured outcome data like images, graphs, time series. Our theoretical tools are not yet applied, but they are ready to be used.

The field of observational studies has been criticised with respect to assumptions verification. The successful use of the technique depends largely on the problem, since we need to make strong assumptions for observational studies, in contrast to randomised ones.

Q. So, how could counterfactual inference tools be used in real-life scenarios?

MK. Let’s say we are using our technique to study a public policy question and provide insight on a candidate policy. Our results can be used to motivate choosing an optimal policy instead of blindly make a decision, pointing to a direction among the vast amount of different possible scenarios. For example, as the effects of climate change are becoming more apparent with global warming and more frequently appearing natural disasters, our “what if”-type counterfactual questions urgently need insightful answers to help policy makers decide on possible strategies. This is an application where counterfactual inference tools would be ideal to help shaping an eco-friendly plan for our planet!

Counterfactual questions arise in other fields as well. For example, what is the effect of education with respect to your future income? Going to university is beneficial for your financial perspective? Furthermore, another application field for counterfactual inference which is now trending is the so-called policy evaluation of recommended systems. For example, companies like Amazon, it is crucial to have an excellent recommendation system which is usually based on customer characteristics. Before any modification aiming to improve the current system, it is not possible for companies to actually implement and test in practice the new system’s efficiency. In fact, with our statistical methods, a company could predict the answer to the question “What if we changed our recommendation system with these specific modifications? “

Q. What are the research topics you are working on?

MK. I’m involved in another project, with scientists at NEC, whose goal is to better tune simulators thanks to machine learning, in order to help design cities so people can escape disasters, like typhoons or earthquakes. Regarding the tsunami prediction, the idea is to build powerful tools that can quickly predict the evolution of the disaster once the earthquake occurs in the ocean. The simulator has to be able to quickly emulate different scenarios based on the recorded data of the underwater earthquake. It is easy to understand that this can be improved with better and more efficient machine learning algorithms. Regarding climate change, the objective is to make algorithms easier to interpret by humans, especially the ones who make environmental decisions. But there’s a big challenge: Can we truly make an interpretation about the real world from simulation results, given that a simulator is only an approximation to the real world? This question is both philosophical and practical, and I try to answer by addressing related technical challenges. Our experimental results on synthetic data and off-policy evaluation tasks demonstrate the advantages of the proposed estimator.

Q. What is the message you would like to share?

MK. Let’s say we are using our technique to study a public health question based on observational data. This would give us a small evidence about a candidate policy and motivate specific randomised experiments, which are costly but can provide a more reliable answer. In this sense, observational studies could be useful to suggest some effectiveness and point to a direction, among the vast amount of different possible scenarios.

For a causal inference point of view, it can be applied from psychology to economics. For example, last year’s economics Nobel prize was awarded to scientist worked related to causal inference in economics. Causal inference is becoming trendy in machine learning, since remaining key challenges are linked to causality. In the future, we would like to use these ML techniques to support human decision making, developing tools to assess the decisions of interest especially in the context of public policy, natural disasters and crisis management, but not only.

by Dora Matzakou for EURECOM

References

Muandet, K., Kanagawa, M., Saengkyongam, S., & Marukatat, S. (2021). Counterfactual Mean Embeddings. Journal of Machine Learning Research, 22, 162–1.

Kisamori, K., Kanagawa, M., & Yamazaki, K. (2020). Simulator calibration under covariate shift with kernels. In International Conference on Artificial Intelligence and Statistics (pp. 1244–1253). PMLR.