OpenStreetMap and Nominatim

The location based game Badge(r)s relies on a political structure of the world to determine played regions, we rely on OpenStreetmap data via the Nominatim Services.

This is a great free and public service and another sample what open source and crowd sourcing can accomplish. We consume their service but we make sure to play fair, i.e. throttle request frequency, cache the retrieved results etc. Beyond the requirements they state, we also shift of our badged update requests preferably to less busy times like (European) night and weekend.

Beyond friendly consumption Badge(r)s also contributes back to OSM/Nominatim, in form of corrections to OSM data and bug reports to Nominatim software on Github, quite a few of both being resolved by now and back in the public.

We truly recommend these systems. However there are issues, we have to cope with. We mention them here, not to talk bad about OSM and Nominatim, but to document chances and boundaries of such services. Note that in preliminary experiments Nominatim was more reliable than Google Maps for Badgers purposes.

Some mentionable Issues:

• Wrong data. The amount of data in OSM is huge, and the quality and speed of contribution varies from region to region, therefore sometimes data is not always correct in time.
• Wrong data. We've already run in instances of OSM vandalism, i.e. sick people renamed Finland into Alcoholia, an error which propagated into our database.
• Interface. The machine readable Nominatim API unfortunately occasionally delivers wrong intermediate political structures, which are sometimes historic, sometimes something else and which are not displayed in the human UI. Several Github issues have been filed so far, unfortunately without a general solution. For applications be sure to check the differences between Json, geocodeJson and other technical output formats, they also vary in result quality.
• Interface. The Nominatim API does not deliver IDs of found structures forcing us, to work on structure names, which is usually a very bad idea.
• Interface. As we retrieve Nominatim data on demand, and try to reread fresh data, we really struggle with updates which are too different from the data few weeks ago.
• Downtimes. Occasionally Nominatim is down.

Considerable programming effort has gone to make our Nominatim interface more robust against above issues. The scripts fetching and updating our database from Nominatim/OSM sources are one of the crown jewels of Badgers programming. Still there is continuous manual effort going into this, to keep things tidy, we call this Mapmin (Map-Admin) work. An tool set has been developed to support manually spotting, diagnosing and interactive fixing further issues in the obtained data.

We check data especially carefully before we establish Ruler or Mapper badges, we have to do some manual work before we establish awards based on bugs. The major part of the queue for this work is summarized on the Suggested Regions page. And, to confess, despite the given manual and automatic effort, we occasionally create broken such badges.