OSM Servers

Manifold's OSM Server dataport connects to OpenStreetMap web servers that provide vector data via OSM protocols. The layers brought in from OSM Server data sources are drawings with attribute tables, not images with tile tables. They provide layers with points, lines, and area objects. They do not provide images or image tiles.

Another way to get map layers from OSM is to use one of the built-in OSM image servers, which are very convenient for background, base layers for maps. See the Image Servers topic.

We can bring data from OSM servers into Manifold using two, related techniques:

Create a new data source of type Web Server: osm using the File - Create - New Data Source command. Creating a data source from an OSM server leaves the data on the OSM server and links a read-only drawing and table into the project.

Create a new data source of type Web Server: osm using the File - Create - New Data Source command. Create a map and zoom to the area of interest. Drop the drawing from the OSM data source into the map. Select the objects of interest in the OSM layer, and choose Edit - Copy. Next, click into the Project pane into the local part of the project and choose Edit - Paste to paste those objects as a new table. Create a new drawing from the table, and the result is a local drawing and table that contains only those objects that were selected. This copies the selected data from the OSM server and saves it in the Manifold .map project. We can then add a key field and index to the table to make the drawing and table selectable and fully read/write.

It is unrealistic to attempt to copy the drawing and table from within the OSM data source, because that would be an attempt to copy all of the vector data, worldwide, from OSM's servers. Instead, we must pan and zoom into a region of interest and select only the objects of interest within that region.

Connecting to a OSM server:

Choose File - Create - New Data Source in the main menu, or right-click in the Project pane and choose New Data Source.
In the dropdown menu choose More... to launch the New Data Source dialog.
Choose Web Server: osm in the Type box.
Enter the URL for the OSM data in the Source box.
Default settings for other options will usually work for most OSM data. Press Create Data Source.
A new data source appears in the project. Open it to see the drawing and table that is linked in from the OSM server.

Drawings and tables that are linked from OSM servers are neither selectable nor editable.

Creating a local copy of OSM data within a limited region:

Create a map, and pan and zoom into a limited area of interest.
Drag and drop the OSM drawing from the data source into the map.
Select desired objects in the OSM layer.
Choose Edit - Copy or press Ctrl-C to copy the objects.
Click anywhere in the main, Manifold .map part of the project outside of the OSM data source's hierarchy.
Choose Edit - Paste or press Ctrl-V to paste the objects into a new table.
Create a new drawing from the table.
Use the Schema dialog to add a key field and index to the table to make the drawing and table selectable and fully read/write.

A local table and drawing with objects that have been copied and pasted from an OSM layer are fully capable, native Manifold components.

We can easily add a key field and index to the table to enable editing and selection in both the table and the table's drawing, using the procedure given in the Add an Index to a Table topic:

Adding an index to a table:

Double-click on the table to open it.
Choose Edit - Schema.
In the Schema dialog click the Add Identity button.
Press the Save Changes button.

New Data Source Dialog and Controls

In the main menu, choose File - Create - New Data Source. The dropdown menu provides a list of favorites to choose from as well as a More... option.

Choose More... to launch the New Data Source dialog. Choose Web Server: osm in the Type box.

Name	Name for the new data source, "Data Source" by default. Specify a more memorable name as desired. If we forget the origin of a data source we can hover the mouse over the data source name in the Project pane and a tool tip will provide connection information.
Type	Choose Web Server: osm in the Type box to connect to a OSM server over the web. The same dataport is used to connect to both raster and vector data served by OSM servers.
Source	A connection string to the web server OSM data. The default is a URL that connects to the main OpenStreetMap server for vector data. This may be a simple URL, or a very lengthy, complex URL/connection string that embeds parameters such as keys that grant access or other parameters. The connection string can also be entered using the Web Login dialog launched by the [...] browse button.
Browse button	Click to launch the Web Login dialog, to allow use of a login and password plus use of a proxy server if desired. The Web Login dialog is also handy for providing a Test button that can be used to test the connection.
Open as read-only	Open the data source read-only. Has no effect with OSM servers since they are read-only in any event.
Cache data	Cache data downloaded from the server while the project is open. Provides better interactive performance and greater flexibility with read-only data sources.
Save cached data between sessions	Save the cached data for the next time this project is opened, within the .map project itself in a Cache sub-folder in the System Data hierarchy. Caution: checking this box can result in very large .map files when the results of browsing very large data from web servers are all saved. However, having such data cached in the .map is handy for offline browsing of the project.
Create Data Source	Create the new data source in the project pane and close the dialog.
Edit Query	Launch the Command Window loaded with a query that creates the data source using the given settings. A great way to learn how to use SQL to create data sources.
Cancel	Exit the dialog without doing anything.

Pressing the browse button next to the Source box launches the Web Login dialog, to specify a server and connection characteristics.

Server	The connection string for the server. This may be a simple URL, or a very lengthy, complex URL/connection string that embeds parameters such as keys that grant access or other parameters
Use login and password	Check this box for servers that require logging in with a login name and a password, providing the required name and password in the Login and Password boxes.
Use proxy server	Check this box when connecting through a proxy server. The Proxy, Login, and Password boxes allow specifying the connection string to the proxy server as well as the login name and password required to use the proxy server.
User agent	Identifies what application (Manifold) is asking for a connection. Some web servers want to know what client software is connecting, for compatibility or for business reasons. The default string optimizes compatibility (Mozilla is very generic) while also identifying Manifold Release 9 as the client. Users can adjust the string as necessary to comply with any special server requirements.
API key	Provide a key that authorizes use of an API when connecting to a proprietary data source that requires such a key. This option is disabled for server dataports that do not use it.
Application key	A secondary application key or authentication code for those servers, such as here.com (also known as wego.here.com) that may require it. This option is disabled for server dataports that do not use it.
Timeout	Specify a time in milliseconds to wait for connecting to the specified server. Use 0 for the default timeout, or specify whatever is the desired time to wait before giving up on the server.
Test	Press the Test button to try the connection using the specified parameters. If successful, a Connection established information dialog will pop open.

Notes

Problems connecting - Check the Log Window to see what is going on behind the scenes if an attempted connection does not work. The problem is usually a wrong connection string, failure to provide required credentials such as an key string that authorizes access, wrong choice of protocol (the server uses OSM and the user picks something else), an incredibly slow server, a server that is offline or a server that is wrongly configured and which is not correctly using the protocol it claims to use.

Visit the Manifold community forum and talk out difficulties with other users. Make sure to post full information on what you are doing, the connection URL you used, all details of how you tried to connect (including all settings in the data source dialog), what happened, and what the Log Window reported. If other users cannot help you, spending a tech support incident will produce an authoritative analysis of the issue.

Try the URL in a browser - Checking the URL by launching it in a browser can reveal many problems with the URL or with the web server. If a URL does not work in a Manifold web server dataport, try exactly the same URL in a browser. If a browser cannot connect to the URL, the Manifold web server dataport will not be able to connect to it either. If a browser cannot connect to that URL, that indicates the problem is the URL or the web server. For example, the web server might be offline. Or, for example, If the browser connects to a page other than the actual web service endpoint, such as, to a web page that lists various options for web servers, that shows the URL is not a URL for a web server but a URL to some other sort of web page.

Connection problems are often caused by incorrect URLs. There might be a typographical error in the URL or the URL might not be an endpoint to a functioning server but instead a URL to some other web page. The server responding to the URL may have geographic restrictions (surprisingly common) that does not respond to connections from IP addresses that are thought to be in a canceled country. Trying the URL in a browser will fail in such cases. Web servers may also have other restrictions, such as only allowing connections from white listed IP addresses, from paying clients, or from those clients that use a special security scheme.

User-unfriendly server configurations - Correctly configured web servers may sometimes make user unfriendly configuration choices, for example, only starting to show data when the view is zoomed in to some specific area. That is a perfectly legal configuration, but it may catch users by surprise if they open a web served layer and it is blank, because they have not yet zoomed sufficiently far into a particular region. If a web server layer is blank, add it to a map with a "known good" background layer, such as Bing Streets, and then zoom into the area of interest that is likely to be covered. OSM data usually will not appear until we are zoomed very far into a data set, such as only a smaller portion of a city.

Must Have Internet - It seems obvious that to connect to a web server we must have a live, functioning Internet connection. What might not be obvious is that sometimes we may have only a partial Internet connection, without realizing we do not have a full Internet connection. A classic example is using a browser, such as Opera, which may ignore failed IPv6 DNS resolution but work OK using IPv4 DNS resolution. Without IPv6 DNS resolution we can browse Internet using Opera but we might not be able to connect to Microsoft, Google, or some other web servers.

Manifold's web server connections, just like the latest Google Chrome and Microsoft Edge browsers, require full IPv6 DNS resolution. That normally happens automatically using whatever IPv6 DNS servers are assigned by our Internet service provider, but at times IPv4 DNS continues to work OK while the IPv6 DNS servers assigned by our ISP do not work. If we are using Opera, and not Edge or some other browser that uses IPv6 DNS resolution, we might not notice that our ISP-provided IPv6 DNS resolution is not functioning. We might only notice the problem when trying to use an image server or other web server within Manifold and it does not work.

If we cannot get a display from a web server, we can check the Manifold Log Window. If the log reports that a connection cannot be made, in Windows network dialogs for our Internet connection we can try setting the IPv6 DNS servers to some "known good" DNS servers. A good choice is using Google's public DNS service, using the following addresses for primary and backup DNS server addresses:

2001:4860:4860::8888
2001:4860:4860::8844