A simple reverse geocoder that resolves a location to a country

Overview

Reverse Geocoder

This repository holds a small web service that performs reverse geocoding to determine whether a user specified location is within the geographic bounds of a country. If it is then the response will contain attributes associated with the matched country such as name, iso label, etc.

I created this simple demonstrator app to learn about FastAPI and PostGIS. It is built using Python and a PostGIS database loaded with country outline polygons obtained from shape files.

API Endpoints

All endpoints are located at /v1/reverse-geocoder/ and are accessible by HTTP.

The OpenAPI specification can be read from /v1/reverse-geocoder/openapi.json.

A SwaggerUI that renders the OpenAPI specification can be found at /v1/reverse-geocoder/docs. The root / will also redirect to the SwaggerUI docs page.

Reverse Geocoder Endpoint

To access the reverse geocoder send a POST to /v1/reverse-geocoder/ with a payload containing the location of interest. See further down for examples using curl.

When the user supplied point is within a country then the response contains attributes of the country that matched. When the user supplied point is not within a country then the response indicates that no country matched.

The response will always contain a copy of the query parameters so that it provides some context for the data. When the location is within a country bounds then the response payload will look like this:

{
  "location":{
    "latitude":-42.239392,
    "longitude":146.558384,
    "altitude":null
  },
  "country":{
    "name":"Australia",
    "iso2":"AU",
    "iso3":"AUS"
  }
}

If the point is not within a country boundary then the response will look like this:

{
  "location":{
    "latitude":-35.031741,
    "longitude":138.119541,
    "altitude":null
  },
  "country":null
}

Demo

To simplify running the reverse geocoding service a Docker compose configuration example is included. However, a few set up steps need to be run first to prepare data that will go into the database - as it is not stored in this repository.

Clone Repo

Start by cloning this repo.

$ git clone https://github.com/claws/reverse-geocoder.git
$ cd reverse-geocoder

Preparation

This demonstration uses world country boundary outlines in shape file format (.shp, .dbf, .shx files) which contain encoded polygons along with other attributes. However, the files are not stored in this repo so they need to be downloaded and then converted into SQL statements.

The following steps show how to download the content from here, decompress it and then convert it into SQL statements that can later be run to insert the contents into the database.

It is important that the name of the generated SQL file is 90-shapes.sql to ensure it gets run after the builtin PostGIS initialization script (which is 10_postgis.sh). This file will be used as a volume mount in the Docker compose configuration.

shp2pgsql is a command line tool that comes with PostGIS. It converts shape files into a SQL format that can be imported into a PostGIS database. The '-G' specifies the use of the geography data type. The '-I' option creates a spatial index after the table is created. This is strongly recommended for improved performance. To run the last command you may need to install PostGIS on your host machine to get the shp2pgsql tool - there may even be a Docker container that has it too.

$ cd database
$ wget http://thematicmapping.org/downloads/TM_WORLD_BORDERS-0.3.zip
$ unzip TM_WORLD_BORDERS-0.3.zip
$ shp2pgsql -G -I TM_WORLD_BORDERS-0.3.shp countries > 90-shapes.sql
$ cd ..

The shape file produces columns containing the following structures:

COLUMN TYPE DESCRIPTION
fips String(2) FIPS 10-4 Country Code
iso2 String(2) ISO 3166-1 Alpha-2 Country Code
iso3 String(3) ISO 3166-1 Alpha-3 Country Code
un Short Integer(3) ISO 3166-1 Numeric-3 Country Code
name String(50) Name of country/area
area Long Integer(7) Land area, FAO Statistics (2002)
pop2005 Double(10,0) Population, World Population Prospects (2005)
region Short Integer(3) Macro geographical (continental region), UN Statistics
subregion Short Integer(3) Geographical sub-region, UN Statistics
lon FLOAT (7,3) Longitude
lat FLOAT (6,3) Latitude
geog Polygon Country/area border as polygon(s)

Start services

Start the database and web server using Docker compose. As part of the startup steps the database runs initialisation script - which will run SQL file created in the setup steps above.

$ docker-compose up --build

The FastAPI framework is built upon OpenAPI and it supports a builtin viewer for the interface specification. These allow developers to perform manual tests.

Once the services start up you should be able to access the SwaggerUI web user interface here which should redirect you to the docs site.

screenshot

Click the POST /v1/reverse-geocoder/ row to expose details about the endpoint. This endpoint is implemented to accept a payload, rather than URL parameters, as it can then rely on the specification and Pydantic to simplify the interface between the API endpoint and the database layer.

Click the Try it out button. This changes the form to allow user input. Leave the default payload structure then click Execute. The Responses section should get populated with the data returned from the web service - which in this case indicates that the point was in Australia.

Try changing the location to some of those in the table below. The last two should return 'null' for the country as they are located in a sea.

Rough Location Position
Tasmania 146.558384 -42.239392
New Zealand 167.930127 -47.033240
Hawaii -157.935416 21.460822
Iceland -21.478138 64.113670
Cypress 34.321495 35.556301
St Vincent Gulf 138.119541 -35.031741
Black Sea 34.402367 43.400157

The Swagger UI also shows the equivalent curl command to use too.

Check using curl

The Reverse Geocoder Service REST API can be used from curl too.

When the location is within a country bounds then the response payload will contain country attributes.

$ curl -X POST "http://localhost:8000/v1/reverse-geocoder/" \
  -H "accept: application/json" \
  -H "Content-Type: application/json" \
  -d "{\"location\":{\"longitude\":146.558384,\"latitude\":-42.239392}}"
{
  "location":{
    "latitude":-42.239392,
    "longitude":146.558384,
    "altitude":null
  },
  "country":{
    "name":"Australia",
    "iso2":"AU",
    "iso3":"AUS"
  }
}

When the user supplied point is not within a country then the response indicates that no country matched.

$ curl -X POST "http://localhost:8000/v1/reverse-geocoder/" \
  -H "accept: application/json" \
  -H "Content-Type: application/json" \
  -d "{\"location\":{\"longitude\":138.119541,\"latitude\":-35.031741}}"
{
  "location":{
    "latitude":-35.031741,
    "longitude":138.119541,
    "altitude":null
  },
  "country":null
}

Developer Notes

Data Server

It can be useful during development to run the Python web server locally. The instructions below show how to do that.

Create a Python virtual environment.

$ cd web-service
$ python3.8 -m venv venv --prompt fast
$ source venv/bin/activate
(fast) $ pip install pip -U
(fast) $ pip install -r requirements.dev.txt
(fast) $ pip install -r requirements.txt

Apply code style

$ black app

Run the data server. Use --reload to enable automatic reloads on code changes.

(fast) $ uvicorn app.main:app --reload --log-level info
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [2617]
INFO:     Started server process [2619]
INFO:     Waiting for application startup.
INFO:     Connected to database postgresql://postgres:********@localhost:54321/postgres
INFO:     Application startup complete.

Database Inspection

It can be useful to attach to the database directly to test out queries. Use docker-compose to connect to the running database by attaching to the container. Once attached the psql tool can be used to execute a query identical to that done by the web service to check if a point lies within a polygon.

The PostGIS ST_Covers function which returns TRUE if A covers B (i.e. no points of B are outside A).

$ docker-compose run database bash
[email protected]:/$
[email protected]:/$ psql --host database -U postgres postgres
Password for user postgres:
postgres=#
postgres=# SELECT name,fips,iso2,iso3 FROM countries WHERE ST_Covers(countries.geog, ST_GeographyFromText('POINT(146.558384 -42.239392)'));
   name    | fips | iso2 | iso3
-----------+------+------+------
 Australia | AS   | AU   | AUS
(1 row)
postgres=#
postgres=# SELECT name,fips,iso2,iso3 FROM countries WHERE ST_Covers(countries.geog, ST_GeographyFromText('POINT(138.119541 -35.031741)'));
 name | fips | iso2 | iso3
------+------+------+------
(0 rows)
postgres=# \q
[email protected]:/$ exit
gpdvega is a bridge between GeoPandas and Altair that allows to seamlessly chart geospatial data

gpdvega gpdvega is a bridge between GeoPandas a geospatial extension of Pandas and the declarative statistical visualization library Altair, which all

Ilia Timofeev 49 Jul 25, 2022
A Jupyter - Leaflet.js bridge

ipyleaflet A Jupyter / Leaflet bridge enabling interactive maps in the Jupyter notebook. Usage Selecting a basemap for a leaflet map: Loading a geojso

Jupyter Widgets 1.3k Dec 27, 2022
Python renderer for OpenStreetMap with custom icons intended to display as many map features as possible

Map Machine project consists of Python OpenStreetMap renderer: SVG map generation, SVG and PNG tile generation, Röntgen icon set: unique CC-BY 4.0 map

Sergey Vartanov 0 Dec 18, 2022
Satellite imagery for dummies.

felicette Satellite imagery for dummies. What can you do with this tool? TL;DR: Generate JPEG earth imagery from coordinates/location name with public

Shivashis Padhi 1.8k Jan 03, 2023
LEOGPS - Satellite Navigation with GPS on Python!

LEOGPS is an open-source Python software which performs relative satellite navigation between two formation flying satellites, with the objective of high accuracy relative positioning. Specifically,

Samuel Low 50 Dec 13, 2022
A package to fetch sentinel 2 Satellite data from Google.

Sentinel 2 Data Fetcher Installation Create a Virtual Environment and activate it. python3 -m venv venv . venv/bin/activate Install the Package via pi

1 Nov 18, 2021
Fiona reads and writes geographic data files

Fiona Fiona reads and writes geographic data files and thereby helps Python programmers integrate geographic information systems with other computer s

987 Jan 04, 2023
Python package for earth-observing satellite data processing

Satpy The Satpy package is a python library for reading and manipulating meteorological remote sensing data and writing it to various image and data f

PyTroll 882 Dec 27, 2022
Open GeoJSON data on geojson.io

geojsonio.py Open GeoJSON data on geojson.io from Python. geojsonio.py also contains a command line utility that is a Python port of geojsonio-cli. Us

Jacob Wasserman 114 Dec 21, 2022
Spatial Interpolation Toolbox is a Python-based GUI that is able to interpolate spatial data in vector format.

Spatial Interpolation Toolbox This is the home to Spatial Interpolation Toolbox, a graphical user interface (GUI) for interpolating geographic vector

Michael Ward 2 Nov 01, 2021
Geographic add-ons for Django REST Framework. Maintained by the OpenWISP Project.

Geographic add-ons for Django REST Framework. Maintained by the OpenWISP Project.

OpenWISP 982 Jan 06, 2023
Python module and script to interact with the Tractive GPS tracker.

pyTractive GPS Python module and script to interact with the Tractive GPS tracker. Requirements Python 3 geopy folium pandas pillow usage: main.py [-h

Dr. Usman Kayani 3 Nov 16, 2022
Automated download of LANDSAT data from USGS website

LANDSAT-Download It seems USGS has changed the structure of its data, and so far, I have not been able to find the direct links to the products? Help

Olivier Hagolle 197 Dec 30, 2022
Python library to decrypt Airtag reports, as well as a InfluxDB/Grafana self-hosted dashboard example

Openhaystack-python This python daemon will allow you to gather your Openhaystack-based airtag reports and display them on a Grafana dashboard. You ca

Bezmenov Denys 19 Jan 03, 2023
A service to auto provision devices in Aruba Central based on the Geo-IP location

Location Based Provisioning Service for Aruba Central A service to auto provision devices in Aruba Central based on the Geo-IP location Geo-IP auto pr

Will Smith 3 Mar 22, 2022
Evaluation of file formats in the context of geo-referenced 3D geometries.

Geo-referenced Geometry File Formats Classic geometry file formats as .obj, .off, .ply, .stl or .dae do not support the utilization of coordinate syst

Advanced Information Systems and Technology 11 Mar 02, 2022
A bot that tweets info and location map for new bicycle parking added to OpenStreetMap within a GeoJSON boundary.

Bike parking tweepy bot app A twitter bot app that searches for bicycle parking added to OpenStreetMap. Relies on AWS Lambda/S3, Python3, Tweepy, Flas

Angelo Trivisonno 1 Dec 19, 2021
Enable geospatial data mining through Google Earth Engine in Grasshopper 3D, via its most recent Hops component.

AALU_Geo Mining This repository is produced for a masterclass at the Architectural Association Landscape Urbanism programme. Requirements Rhinoceros (

4 Nov 16, 2022
Processing and interpolating spatial data with a twist of machine learning

Documentation | Documentation (dev version) | Contact | Part of the Fatiando a Terra project About Verde is a Python library for processing spatial da

Fatiando a Terra 468 Dec 20, 2022
Color correction plugin for rasterio

rio-color A rasterio plugin for applying basic color-oriented image operations to geospatial rasters. Goals No heavy dependencies: rio-color is purpos

Mapbox 111 Nov 15, 2022