Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision Both sides next revision
nominatim [2018/09/24 17:22]
Frank Nestel
nominatim [2018/10/03 11:10]
Frank Nestel
Line 3: Line 3:
 ====== OpenStreetMap and Nominatim ====== ====== OpenStreetMap and Nominatim ======
  
-As Badgers relies on a political structure of the world to determine [[rules:​regions|regions]],​ we relay on  [[https://​www.openstreetmap.org/​|OpenStreetmap data]] via the [[https://​nominatim.openstreetmap.org/​|Nominatim Services]]. ​+As Badgers relies on a political structure of the world to determine [[rules:​regions|regions]],​ we rely on  [[https://​www.openstreetmap.org/​|OpenStreetmap data]] via the [[https://​nominatim.openstreetmap.org/​|Nominatim Services]]. ​
  
-This is a **great** free and public service and another sample what open source and crowd sourcing can accomplish. We consume their service but we make sure to play fair, i.e. throttle request frequency, cache the retrieved results etc. Beyond the requirements they state, we also shift some of our requests ​on less busy times like (European) night and weekend. Beyond friendly consumption Badgers also contributed a little bit back to OSM/​Nominatim,​ mainly in form of data corrections to OSM data and bug reports to Nominatim.+This is a **great** free and public service and another sample what open source and crowd sourcing can accomplish. We consume their service but we make sure to play fair, i.e. throttle request frequency, cache the retrieved results etc. Beyond the requirements they state, we also shift of our badged update ​requests ​preferably to less busy times like (European) night and weekend. ​
  
-We truly recommend these systems. However there are issues, we have to cope with and I though I mention them herenot to say bad things about OSM and Nominatim, ​but to document the chances and boundaries ​of such services. Note that in preliminary experiments Nominatim was more reliable than Google Maps for our purpose.+//Beyond friendly consumption Badgers also **contributes** back to OSM/​Nominatimin form of corrections ​to OSM data and bug reports to Nominatim ​software on Githubquite a few of both being resolved by now and back in the public. Major agent but not only for this is [[user:​mega_chip|MegaChip]]. 
 +//
  
-Issues:+We truly recommend these systems. However there are issues, we have to cope with. We mention them here, not to talk bad about OSM and Nominatim, but to document chances and boundaries of such services. Note that in preliminary experiments Nominatim was more reliable than Google Maps for Badgers purposes. 
 + 
 +Some mentionable ​Issues:
   * Wrong data. The amount of data in OSM is huge, and the quality and speed of contribution varies from region to region, therefore sometimes data is not always correct in time.   * Wrong data. The amount of data in OSM is huge, and the quality and speed of contribution varies from region to region, therefore sometimes data is not always correct in time.
-  * Wrong data. We've already run in instances of OSM vandalism, i.e. some unbearable ​people ​renaming ​Finland into Alcoholia, an error which propagated into our database. +  * Wrong data. We've already run in instances of **OSM vandalism**, i.e. sick people ​renamed ​Finland into Alcoholia, an error which propagated into our database. 
-  * Interface. The machine readable Nominatim API unfortunately occasionally delivers wrong intermediate political structures, which are sometimes historic, sometimes something else and which are not displayed in the human UI. Several Github issues have been filed so far, unfortunately without a general solution.+  * Interface. The machine readable Nominatim API unfortunately occasionally delivers wrong intermediate political structures, which are sometimes historic, sometimes something else and which are not displayed in the human UI. Several Github issues have been filed so far, unfortunately without a general solution. For applications be sure to check the differences between ''​Json'',​ ''​geocodeJson''​ and other technical output formats, they also vary in result quality.
   * Interface. The Nominatim API does not deliver IDs of found structures forcing us, to work on structure names, which is usually a very bad idea.   * Interface. The Nominatim API does not deliver IDs of found structures forcing us, to work on structure names, which is usually a very bad idea.
   * Interface. As we retrieve Nominatim data on demand, and try to reread fresh data, we really struggle with updates which are too different from the data few weeks ago.    * Interface. As we retrieve Nominatim data on demand, and try to reread fresh data, we really struggle with updates which are too different from the data few weeks ago. 
   * Downtimes. Occasionally Nominatim is down.   * Downtimes. Occasionally Nominatim is down.
  
-Considerable programming effort has been gone into making ​our end of the Nominatim interface robust against above issues, but still there is also continuous manual effort going into this, to keep things tidy. Especially ​before we establish [[badge:​ruler|Ruler]] or [[badge:​mapper|Mapper]] badges, we have to do some manual work before we establish awards based on bugs. The major part of the queue for this work is summarized on the [[https://​www.badge-r-s.de/​suggestedRegions2.php|Suggested Regions page]].+Considerable programming effort has gone to make our Nominatim interface ​more robust against above issues. The scripts fetching and updating our database from Nominatim/​OSM sources are one of the **crown jewels of Badgers programming**. Still there is **continuous manual effort** going into this, to keep things tidy. An tool set has been developed to support manually spotting, diagnosing and interactive fixing further issues in the obtained data.  
 + 
 +We check data especially carefully ​before we establish [[badge:​ruler|Ruler]] or [[badge:​mapper|Mapper]] badges, we have to do some manual work before we establish awards based on bugs. The major part of the queue for this work is summarized on the [[https://​www.badge-r-s.de/​suggestedRegions2.php|Suggested Regions page]]. ​And, to confess, despite the given manual and automatic effort, we occasionally create broken such badges.