We connect to a custom server that provides an OpenRailwayMap view of railroads worldwide, showing railway, tram, and subway infrastructure based on OpenStreetMap data. Our first try at creating a data source does not work. After consulting the Log Window we try again with a slight adjustment and our second try works.
This topic uses a connection
to a web server that is operated by a third party. Web servers
operated by third parties may change their access policies, use different
URLs, go offline, or be very slow. If you have any connection problems
with the server in this topic, review the Notes
at the end of the Web
Servers topic for tips.
It is usually convenient to work with web server layers within a map, so we can always have a "known good" background layer for context.
Web server layers are often very slow as well, so it is especially convenient to be able to use a background layer in a map, like Bing streets, to pan and zoom into our particular area of interest before dragging and dropping a web server layer into the map.
Therefore, many experienced users will begin a project using a new web server layer by first creating a map.
We create a map by right-clicking into a blank portion of the Project pane, and choosing Create - New Map.
For a base layer, we use the default Bing streets background layer, one of the standard Favorites built into Manifold.
Press Create Map.
A new map appears in the Project pane. The system also creates a new Bing Maps Street Map data source for the Bing streets base layer used in the map.
We double-click the Map to open it.
The map opens up zoomed to fit, with the Bing layer showing the entire world.
One of the richest lists of web servers of various kinds to use is at https://josm.openstreetmap.de/wiki/Maps The list provides potential sources that contributors to OpenStreetMap might use as a source of data to copy or trace over using the OpenStreetMap JOSM editor.
We scroll down to the Worldwide section at the end, and we right-click on the OpenRailwayMap - Default link (not on the "view" link), and then we choose Open link in new window.
The full entry in text form is:
<entry overlay="true">
<name>OpenRailwayMap - Default</name>
<id>openrailwaymap</id>
<category>osmbasedmap</category>
<type>tms</type>
<description lang="en">Overlay imagery showing railway infrastructure based on OpenStreetMap data</description>
<url>https://{switch:a,b,c}.tiles.openrailwaymap.org/standard/{zoom}/{x}/{y}.png</url>
<attribution-text mandatory="true">Rendering: OpenRailwayMap, © Map data OpenStreetMap contributors</attribution-text>
<attribution-url>https://www.openrailwaymap.org/</attribution-url>
<privacy-policy-url>https://www.openrailwaymap.org/en/imprint</privacy-policy-url>
<icon>https://www.openrailwaymap.org/img/openrailwaymap-64.png</icon>
<max-zoom>20</max-zoom>
<valid-georeference>true</valid-georeference>
</entry>
The URL we copy is:
https://{switch:a,b,c}.tiles.openrailwaymap.org/standard/{zoom}/{x}/{y}.png
We might guess from the <type>tms</type> tag specification that we should use a TMS server. That would be a good first guess, but it does not work. From experience, we know that when we see control sequences, also called patterns, that use curly { } brackets, we should next try a custom imageserver data source.
Manifold's imageserver dataport provides a Custom setting that allows utilizing a very wide range of web servers that provide image tiles using connection strings that contain some more-or-less typically encountered, but far from standard, control sequences. Control sequences are tokens within curly { } brackets that are substituted during the action of the server with values that control the result. To learn how a specific server uses control sequences, as well as learning other required information such as the tile size used and the zoom range supported, we must consult whatever documentation the server operator has provided.
We will try creating a data source using the imageserver Custom setting.
In Manifold, choose File - Create - New Data Source from the main menu, or right-click in the Project pane and from the menu choose Create - New Data Source.
Choose More... to launch the New Data Source dialog.
Enter OpenRailwayMap as the Name. We can enter whatever name we like, but it is wise to specify a name that reminds us of what this data source contains.
Choose Web Server: imageserver as the Type. In the subtype box just below the Type box, we choose Custom.
In the Source box we paste the URL connection string we copied from the OpenRailwayMap entry in the JOSM servers page:
https://{switch:a,b,c}.tiles.openrailwaymap.org/standard/{zoom}/{x}/{y}.png
Make sure to check the Cache data box, which is checked by default. This stores in cache memory everything Manifold downloads from the server, so that subsequent operations, such as panning and zooming, can take the data from cache and not have to download it again.
We leave the other settings at their default values, which usually work.
Press Create Data Source.
A new data source called OpenRailwayMap appears in the Project pane.
We can expand the data source to see it contains an image called Web Image and the image's table, called Web. This particular Custom server has only a single image layer.
To see what that server does, we first zoom our map to our first area of interest, the region surrounding Manila, in the Philippines.
We drag and drop the Web Image layer into the map:
The Web Image layer appears as a tab in the map, with a blue underline indicating it is loading, or trying to load, but nothing appears in the map for that layer.
We choose View - Panes - Log Window to open the Log Window to take a look at what is going on.
Very slowly, we see line after line appear, every time the image server dataport tries to fetch another tile, with an error message saying Invalid image size. (512x512).
That tells us the custom server data source we created is not working as we would like, and that perhaps the Tile size we used, 256, when we created the server is incorrect. Perhaps we should have used a tile size of 512.
We will delete the data source, and then we will create it again using a Tile size of 512.
In the Project pane we click the OpenRailwayMap data source to highlight it, and then we click the Delete button on the Project pane toolbar to delete it.
We also right-click the Web Image layer tab in the map and choose Delete from Map to delete the layer.
We will now create the data source again, but using a different Tile size.
We again choose File - Create - New Data Source from the main menu, or right-click in the Project pane and from the menu choose Create - New Data Source.
Choose More... to launch the New Data Source dialog.
We again enter OpenRailwayMap as the Name, and we choose Web Server: imageserver as the Type. In the subtype box just below the Type box, we choose Custom.
In the Source box we again paste the URL connection string we copied from the OpenRailwayMap entry in the JOSM servers page:
https://{switch:a,b,c}.tiles.openrailwaymap.org/standard/{zoom}/{x}/{y}.png
This time, we change the Tile size value from the default of 256 to 512.
Press Create Data Source.
A new data source called OpenRailwayMap appears in the Project pane.
When we expand the data source we see it is exactly the same as before, containing an image called Web Image and the image's table, called Web. There is no surprise to that since the custom server is exactly the same, but only now fetching tiles that are 512 pixels by 512 pixels in size instead of the original default value of 256 pixels by 256 pixels.
We drag and drop the Web Image layer into our map.
After what seems to be an absurdly long time, tens of seconds, the layer slowly fills in.
If we are following along in this example and repeating it, we should be aware that layers served from some custom servers can be incredibly slow. This is not Manifold being slow: it is simply a slow server.
The OpenRailway server is a cool server in that it serves images, but it is styled so that pixels other than those used to show rail lines and labels all use transparent color. This allows other layers underneath an OpenRailwayMap layer to show though. We can see that the server provides highly useful information on rail systems and that the layer is perfectly georegistered, lining up exactly with the Bing background layer.
We can zoom out to see the OpenRailwayMap server provides remarkably good coverage throughout the world.
It is a pain to wait for the server to send tiles, but one way we can deal with that is to turn off the OpenRailwayMap layer by double-clicking the tab, and then panning and zooming to a desired area of interest using the Bing layer. Once we have our desired area of interest in view, we can turn the OpenRailwayMap layer back on by double-clicking the tab again.
For example, we can turn off the OpenRailwayMap layer, pan and zoom into a view of Chartes, in France, and then turn he OpenRailwayMap layer back on to get the display seen above.
The OpenRailwayMap server adjusts the detail it shows based on the zoom level used. For example, we can zoom in to see more detail.
In the illustration above, we have zoomed farther into the view, and the server now provides a more detailed label for the orange line going through the main Chartres rail station, showing it is line to the Montparnasse station in Paris.
In the illustrations that follow, in all cases we turn off the OpenRailwayMap layer, pan and zoom to the desired view using Bing, and then turn on the OpenRailwayMap layer and wait while it fills in.
We pan and zoom to downtown Paris, in the triangular region between the Arc de Triomphe, the Eiffel Tower, and the Orsay museum, to see that the OpenRailwayMap server also provides information on metro (subway, or underground) lines. However, the styling used makes it difficult to see where the stations are located. We can do better by zooming in.
Zoomed further in, the server adds labels for individual stations and metro line numbers.
Styles seem to vary for different parts of the world.
For example, zooming into Hong Kong, seen above, shows the main tram lines (in pastel magenta color) along the high rise district on the North shore of Hong Kong island, with MTR subway lines and underwater connections across Victoria Harbor to the Kowloon mainland peninsula at the upper part of the window. Individual stations are shown as boxes on the main line, with only a few stations labeled in blue.
The example for Hong Kong shows a possible issue for OpenRailwayMap for some uses, in that labels use the local language of the region shown.
That can be an issue for English speakers in a place like Beijing, seen above. The Bing layer uses whatever language is used for the web server chosen, but the OpenRailwayMap layer uses local language for the region shown.
A staggering range of choices - Visit pages like https://josm.openstreetmap.de/wiki/Maps to get an idea of the absolutely staggering range of data sources that we can use in Manifold. Most of those data sources will be immediately useful using default or obvious choices in the Create New Data Source dialog, and almost all can be used with just a bit of street smarts to deal with minor issues, as we applied in this example to dial up the Tile size from a default of 256 to the required tile size value of 512.
Yep, slow is sure slow - One downside of the many choices for web served data is that most of those servers are painfully slow. Manifold helps deal with that by checking the Cache data box by default. We waited for tens of seconds to get every display illustrated above, but we only had to wait once for each. We could save each as a location, and then jump between locations with instantaneous display, as Manifold pulled tiles for the OpenRailwayMap layer not from the server but from local cache. Once we download a tile from the server the system does not need to download it again.
Missing tiles - Sometimes the OpenRailwayMap server gets confused and stops sending tiles before a scene is completely filled. That is usually obvious, with parts of the layer such as labels or lines cut off along the straight lines of tile edges. To deal with that, create another data source (not using copy and paste, but using the New Data Source dialog) using exactly the same connection string and settings and try again for that same location. We cannot simply copy/paste the data source because doing that copies the local cache, including local cache information that says the tiles in the problem location were missing.
File - Create - New Data Source
Example: Spectacular Images and Data from Web Servers - A must see topic providing a gallery of views illustrating how Manifold can use web servers such as image servers and other free resources to provide a seemingly endless selection of spectacular background maps, satellite images and GIS data with nearly zero effort.
Example: Vector Layers from an ArcGIS REST Feature Server - Visit an ESRI web site, copy a URL, and then use that URL to connect to an ArcGIS REST web server that shows petroleum fields in Kansas, getting the data as a vector drawing layer. Style the layer as if it were local. ESRI refers to ArcGIS REST servers that provide vector data as feature servers.
Example: Connect to a WFS Server for State Government Data - Gathering our courage, we connect to a WFS server that provides 1200 vector layers, run by the state of Massachusetts. We open a layer showing airports and then scrape the vector data into our own local storage.
Example: Connect to a WMS Server for National Map Layers - Visit the National Map services web page, copy a URL for a shaded relief layer from USGS, and then use Style to enhance that shaded relief data for combination with other layers and really spectacular effects.
Example: Connect to an OSM Vector Server - We connect to an OSM Server that provides a vector layer containing points and lines in the OpenStreetMap database. We then show how to scrape (copy) data from the OpenStreetMap server into local storage. We extract building footprints from the local copy.
Example: Raster Layers from an ArcGIS REST Image Server - Visit an ESRI web site, copy a URL, and then use that URL to connect to an ArcGIS REST web server that provide a raster layer showing a mosaic of aerial photographs near Portland, Oregon.