Publishing an OpenStreetMap
Check out the demonstration map to see the final product!
OpenStreetMap (OSM) contains a wealth of data, kept up to date by a world-wide community of mappers. It is to map data what open source is to software: a unique and collaborative way to build shared knowledge.
Building a map using OSM data can be daunting: you have to extract data from the central server, or find a download package; choose and use tools convert it into the working database; choose and use a rendering chain to produce a cartographic product; and, choose and use a tool to publish that product.
This tutorial will explore deploying a cartographic product using a small set of easy-to-install tools:
- OpenGeo Suite, for rendering the data and publishing it to the world.
- bash scripts and curl to automate map configuration.
Download the Data¶
OSM City Data¶
We will be building a street map of a single city. To keep data volumes small, we will be working with Victoria, British Columbia for this example.
Inside the zip archive are a point file, line file and polygon file. In order to line the table names up with our processing later in this tutorial, we will name them planet_osm_point, planet_osm_line and planet_osm_polygon during import. If you import with the pgShapeLoader GUI, remember to
- set the SRID of the data to 4326 (longitude/latitude)
- set the table names appropriately as above
- set the geometry column name to way
Or, you can use the commandline shp2pgsql utility and do the uploads this way:
shp2pgsql -g way -s 4326 -I -D -i -S victoria.osm-point.shp planet_osm_point | psql osm shp2pgsql -g way -s 4326 -I -D -i -S victoria.osm-line.shp planet_osm_line | psql osm shp2pgsql -g way -s 4326 -I -D -i victoria.osm-polygon.shp planet_osm_polygon | psql osm
Note that we’re “piping” (using the “|” character) the output of the conversion directly to the psql utility. You might need to add some connection flags there to connect to your local database. There are a lot of shp2pgsql loader flags in play here, so it’s worth listing what they all do:
- -g controls the column name to use for geometries, we use “way”
- -s controls the SRID to apply to the data, we use “4326” for “WGS84 lon/lat”
- -I adds a spatial index to the table after loading
- -D uses “dump” mode for a faster loading process
- -i ensures that all integer types use 32-bit integers
- -S ensures that geometry column types are “simple” not “aggregate” (eg, “linestring”, not “multilinestring”)
OSM Ocean Data¶
The OSM city files include roads and land features, but not coastlines or ocean features. In order to get mapping-ready ocean, we need to download a different file.
You can download the whole ocean file from http://openstreetmapdata.com. However, it’s really large, and takes some processing to use in a small project. You can download a clipped set of oceans for our test area from here:
Once you have downloaded the ocean file, unzip it, then load it into the database:
shp2pgsql -g geom -s 4326 -I -D ocean.shp ocean | psql osm
Clipping Your Own Ocean¶
If you want to download the big file and clip it yourself, here are the steps.
Download the full ocean polygon file, from http://openstreetmapdata.com/data/water-polygons
Load the file into an ocean_all table, so it can be clipped down to size in the database:
shp2pgsql -s 4326 -I -D water_polygons.shp ocean_all | psql osm
Clip the ocean_all table to the working area ocean table using this SQL. It first calculates the extent of the working area, then uses that extent to clip the world table down to just the local extent.
CREATE TABLE ocean AS WITH bounds AS ( SELECT ST_SetSRID(ST_Extent(way)::geometry,4326) AS geom FROM planet_osm_line ) SELECT 1 AS id, ST_Intersection(b.geom, o.geom) AS geom FROM bounds b, ocean_all o WHERE ST_Intersects(b.geom, o.geom);
Create Layer Tables¶
Right now we have just four OSM tables: points, lines, polygons and (oddball) ocean. To build a map, we are going to subset the generic tables into more “layer-like” tables, using queries against the OSM tags.
Run the SQL commands in create_tables.sql. They will create the following tables, and associated spatial indexes:
If you want to get rid of the tables, here’s a drop_tables.sql file to use. And of course, we have already loaded an ocean table, so that is in the database too!
There is also a create_views.sql file (and drop_views.sql too), which does the same transformation, only without copying data. Using views instead of tables will cause your map rendering to run slower, however.
Use osm as the workspace name, to match up to the rest of the tutorial.
Create Data Store¶
We need to connect to our PostGIS database that holds all our tables, so define a new PostGIS data store.
- Choose osm as the workspace
- Use openstreetmap as the store name
- Add the other connection parameters to your database below
Add Layers and Styles¶
Because building a cartographic product from 25 input layers is a complex undertaking, we aren’t going to go layer by layer, but rather upload an existing configuration into GeoServer to product the output.
- All the styles for this exercise are in sld.zip. Download and unzip it.
These styles were automatically derived from the “OSM streets” stylesheet.
Now, there are two approaches to applying the styles, a manual approach and an automatic approach.
Manual Style Loading¶
For each SLD file in the directory, carry out the following steps:
- Go to the Styles configuration panel.
- Add a new style, using the name of your SLD file (without the .sld extension)
- Select the osm workspace.
- Upload or copy the content from the SLD file into the editor.
- Go to the Layers configuration panel.
- Add a new resource from the PostGIS openstreetmap store.
- Select the table that matches your SLD file name.
- Calculate the native and geographic bounds by clicking the right links.
- Go to the “Publishing” tab.
- Select as the default style, the new style you just created that shares a name with this layer.
Repeat until every SLD file has been used to create a style, and associated with an appropriate layer/table in the database.
Automatic Style Loading¶
If you have Linux or OSX, you can use the automatic approach (if you are good at Windows scripting, you can probably adapt this approach). This requires the curl utility, which is available by default on Linux and OSX.
This script assumes you are using a standard bash shell. It uses the GeoServer REST API to alter the configuration catalogue without requiring pointing and clicking.
- Enter the directory that contains the SLD files.
- Run the SLD_create.sh script below. (If you have changed your “admin” password, or used different names for the workspace or datastore, you may need to alter the script.)
# script to add layer/style information # for every SLD file in our collection # restapi=http://localhost:8080/geoserver/rest login=admin:geoserver workspace=osm store=openstreetmap for sldfile in *.sld; do # strip the extension from the filename to use for layer/style names layername=`basename $sldfile .sld` # create a new featuretype in the store, assuming the table # already exists in the database and is named $layername # this step automatically creates a layer of the same name # as a side effect curl -v -u $login -XPOST -H "Content-type: text/xml" \ -d "<featureType><name>$layername</name></featureType>" \ $restapi/workspaces/$workspace/datastores/$store/featuretypes?recalculate=nativebbox,latlonbbox # create an empty style object in the workspace, using the same name curl -v -u $login -XPOST -H "Content-type: text/xml" \ -d "<style><name>$layername</name><filename>$sldfile</filename></style>" \ $restapi/workspaces/$workspace/styles # upload the SLD definition to the style curl -v -u $login -XPUT -H "Content-type: application/vnd.ogc.sld+xml" \ -d @$sldfile \ $restapi/workspaces/$workspace/styles/$layername # associate the style with the layer as the default style curl -v -u $login -XPUT -H "Content-type: text/xml" \ -d "<layer><enabled>true</enabled><defaultStyle><name>$layername</name><workspace>$workspace</workspace></defaultStyle></layer>" \ $restapi/layers/$workspace:$layername done
The script runs one iteration for each SLD file, using the file name to create a layer/featuretype that matches the table in the database with that name, and populates an associated style with it. (If you want to reverse the process, there’s an SLD_delete.sh script too.)
Once you’re done, click on the Styles and Layers panes to see the results.
Create a Layer Group¶
Manual Layer Group¶
Now we have 25 layers, all nicely styled... but we don’t want 25 layers, we want one map!
Again, you can hand-build a layer group, go to the Layer Groups pane, and create a new layer group named osm, then follow the screenshot below, adding and ordering layers.
Automatic Layer Group¶
We can use a single curl call to add a layer group definition.
Create a layergroup.xml file with the following content:
<layerGroup> <name>osm</name> <title>OpenStreetMap Base</title> <layers> <layer>agriculture</layer> <layer>beach</layer> <layer>forest</layer> <layer>grass</layer> <layer>water-outline</layer> <layer>water</layer> <layer>ocean</layer> <layer>park</layer> <layer>aero-poly</layer> <layer>parking-area</layer> <layer>amenity-areas</layer> <layer>building</layer> <layer>route-tunnels</layer> <layer>route-bridge-0</layer> <layer>route-bridge-1</layer> <layer>route-bridge-2</layer> <layer>route-bridge-3</layer> <layer>route-bridge-4</layer> <layer>route-bridge-5</layer> <layer>route-line</layer> <layer>route-fill</layer> <layer>placenames-medium</layer> <layer>highway-label</layer> </layers> </layerGroup>
Now, run the following command to upload it to the server (you may need to change the login credentials if you have altered them from the default):
curl -v -u admin:geoserver -XPOST -firstname.lastname@example.org -H "Content-type: text/xml" \ http://localhost:8080/geoserver/rest/layergroups
If you find you want to change the definition, you can delete it from the catalogue with the command below, and then run the upload again:
curl -v -u admin:geoserver -XDELETE \ http://localhost:8080/geoserver/rest/layergroups/osm
View the Result¶
Now that all the layers and styles and layer groups are configured, you can see the result!
Go to the Layer Preview panel, and and find the entry for the osm layer group. It may be on the second page of results.
Open it up (Click “Go” after the OpenLayers dropdown), and you’ll see the result of our styling.
Alter the SLDs¶
You can edit the SLDs directly from the Styles pane, one at a time. This has the advantage of providing access to the “Validate” function, which is important to ensure the changes you are making are valid SLD.
For batch changes (global changes to things like font names, for example) sometimes it’s easier to update all the SLD files in your local directory, and then re-upload them. For the re-upload process, the following script provides a quick way to refresh all the SLD files on the server in one pass:
# Update SLDs restapi=http://localhost:8080/geoserver/rest login=admin:geoserver for sldfile in *.sld; do layername=`basename $sldfile .sld` # upload the SLD definition to the style curl -v -u $login -XPUT -H "Content-type: application/vnd.ogc.sld+xml" \ -d @$sldfile $restapi/workspaces/osm/styles/$layername done
Nicer Web Interface¶
Check out the demonstration map online to see the final product!
In this tutorial, we have seen:
- How to create an attractive cartographic base map from OSM
- How to carry out bulk operations on the GeoServer catalogue using the REST API
- How to wrap a layer group in a simple OpenLayers3 map window
To go on learning, you might want to alter the SLD definitions. Add some labelling at lower scales, change colors, and so on.
The possibilities are endless!
Practical walkthroughs of app development examples using OpenGeo Suite.
Course materials given at our various training classes in NYC and conferences around the world.
Boundless reduces the cost and time of deploying and managing open source geospatial software. Learn more about Suite, plus our Support, Developer Tools, Maintenance, Pro Services, and Training offerings.
Boundless provides commercial open source software for internet mapping and geospatial application development. We are dedicated to the growth and support of open source software.
This work is licensed under a Creative Non Commercial-Commons Attribution-Share Alike 3.0 United States License. Feel free to use this material, but we ask that you please retain the Boundless branding, logos and style.