[TMP-065] Intelligent crowdsourcing of geolocation tasks in natural disasters

Contact person: Jesus Cerquides (j.cerquides@csic.es) 

Internal Partners:

1. Consejo Superior de Investigaciones Científicas (CSIC), Jesus Cerquides, cerquide@iiia.csic.es

External Partners:

1. University of Geneva, Jose Luis Fernandez Marquez  

Social media generates large amounts of almost real-time data which can turn out extremely valuable in an emergency situation, especially for providing information within the first 72 hours after a disaster event. Despite there being abundant state-of-the-art machine learning techniques to automatically classify social media images and some work for geolocating them, the operational problem in the event of a new disaster remains unsolved.

Currently the state-of-the-art approach for dealing with these first response mapping is first filtering and then submitting the images to be geolocated to a crowd of volunteers [1], assigning the images randomly to the volunteers.

The project is aimed at leveraging the power of crowdsourcing and artificial intelligence (AI) to assist emergency responders and disaster relief organizations in building a damage map from a zone recently hit by a disaster. Specifically, the project involves the development of a platform that can intelligently distribute geolocation tasks to a crowd of volunteers based on their skills. The platform uses machine learning to determine the skills of the volunteers based on previous geolocation experiences.

Thus, the project concentrates on two different tasks:

• Profile Learning. Based on the previous geolocations of a set of volunteers, learn a profile of each of the volunteers which encodes its geolocation capabilities. These profiles should be understood as competency maps of the volunteer, representing the capability of the volunteer to provide an accurate geolocation for an image coming from a specific geographical area.
• Active Task Assignment. Use the volunteer profiles efficiently in order to maximize the geolocation quality while maintaining a fair distribution of geolocation tasks among volunteers.

In the first stage, we envision an experimental framework with realistically generated artificial data, which acts as a feasibility study. This will be published as a paper in a major conference or journal. Simultaneously we plan to integrate both the profile learning and the active task assignment with the crowdnalysis library, a software outcome of our previous micro-project. Furthermore, we plan to organize a geolocation workshop to take place in Barcelona with participation from the JRC, University of Geneva, United Nations, and IIIA-CSIC.

In the near future, the system will generate reports and visualizations to help these organizations quickly understand the distribution of damages. The resulting platform could enable more efficient and effective responses to natural disasters, potentially saving lives and reducing the impact of these events on communities.

[1] Fathi, Ramian, Dennis Thom, Steffen Koch, Thomas Ertl, and Frank Fiedrich. “VOST: A Case Study in Voluntary Digital Participation for Collaborative Emergency Management.” Information Processing and Management 57, no. 4 (July 1, 2020): 102174. https://doi.org/10.1016/j.ipm.2019.102174 

Results Summary

The project focused on improving the accuracy and efficiency of geolocating social media images during emergencies by using crowdsourced volunteers. Key results include the development of two models: a profile-learning model to gauge volunteers’ geolocation abilities and a task assignment model that optimizes image distribution based on volunteer skills. These models outperform traditional random assignment approaches by reducing annotation requirements and improving the quality of geolocation consensus without sacrificing accuracy. This method holds promise for disaster response applications. We had 3 main outputs:

1. Open-source implementation of the volunteer profiling and consensus geolocation algorithms into the crowd analysis library.
2. Papers with the evaluation of the different geolocation consensus and active strategies for geolocation:
3. an online workshop to collect expert feedback about the topic

Tangible Outcomes

1. Ballester, Rocco, Yanis Labeyrie, Mehmet Oguz Mulayim, Jose Luis Fernandez-Marquez, and Jesus Cerquides. “Crowdsourced Geolocation: Detailed Exploration of Mathematical and Computational Modeling Approaches.” Cognitive Systems Research 88 (December 1, 2024): 101266. https://doi.org/10.1016/j.cogsys.2024.101266 .
2. Ballester, R., Labeyrie, Y., Mulayim, M.O., Fernandez-Marquez, J.L. and Cerquides, J., 2023. Mathematical and Computational Models for Crowdsourced Geolocation. In Artificial Intelligence Research and Development (pp. 301-310). IOS Press. https://ebooks.iospress.nl/doi/10.3233/FAIA230699
3.  Firmansyah, H. B., Bono, C. A., Lorini, V., Cerquides, J., & Fernandez-Marquez, J. L. (2023). Improving Disaster Response by Combining Automated Text Information Extraction from Images and Text on Social Media. In Artificial Intelligence Research and Development (pp. 320-329). IOS Press. https://ebooks.iospress.nl/doi/10.3233/FAIA230701
4. Cerquides J., Mülâyim M.O. Crowdnalysis: A software library to help analyze crowdsourcing results (2024), 10.5281/zenodo.5898579 https://github.com/IIIA-ML/crowdsourced_geolocation