KML Servers

KML is an XML-based format with a three-letter file name extension of .kml originally used for annotations in Google Earth displays. KMZ is exactly the same format compressed using "zip" compression with a three-letter file name extension of .kmz. Manifold automatically decompresses KMZ to get the resulting KML. In this documentation we use KML as a term to mean either KML or KMZ. KML can be used for points, lines or areas with typically two text attributes, a Name field and a Description field, per object. Manifold can connect to KML files on our computer or, using the Web Server: kmlserver choice in the New Data Source dialog, Manifold can connect to a KML file hosted by a web server. See the KML, KMZ Google topic for connections to KML files on our local computer.

Caution: KML is a dangerous format for privacy, because KML files can contain links that trigger downloads from external web servers. KML files that contain links to other files are called recursive KML files. Triggering such links can allow third parties to track your activity, to discover the IP address of your computer, and may allow artificial intelligences to determine your identity and to track your activities online. Google loves that stuff, but you might not.

Please read the KML, KMZ Google topic for important information about KML format.

Connecting to a Web Served KML

We connect to a web served .kml using the File - Create - New Data Source dialog.

Connect to a web served kml:

Choose File - Create - New Data Source from the main menu.
In the dropdown menu, choose More... to launch the New Data Source dialog.
Choose Web Server: kmlserver as the Type.
In the Source box, enter the URL to the file. Use the [...] browse button to launch the Web Login dialog for more controls and options.
Check the Merge layers box if desired.
Increase Link depth to follow links more than one additional file deep, if the file on the server in turn links to other files.
Press Create Data Source.
Expand the new data source hierarchy to see the contents of the web served KML.

.kml drawings will often include "folders," that is, layers. Each layer will be imported into Manifold as a separate drawing so that they can appear within a map as layers. A map component containing all the layers will also be created. Some .kml files may contain a bewildering number of layers that are best organized within a single drawing. Checking the Merge layers option box in the New Data Source dialog will create a single drawing that contains all such layers

Importing a Web Served KML

A quick way to import a .kml file into a project is to use File - Import, and a quick way to link to a .kml file is to use File - Link. For better control over the resulting import, many Manifold users will import a .kml using File - Create - New Data Source, since the New Data Source dialog provides a Merge layers option.

Importing a web served KML is the same as connecting to a web served KML, plus the additional step of copying contents from within the new data source and pasting them into the local part of the project.

To import from kml format:

Choose File - Create - New Data Source from the main menu.
In the dropdown menu, choose More... to launch the New Data Source dialog.
Choose Web Server: kmlserver as the Type.
In the Source box, enter the URL to the file. Use the [...] browse button to launch the Web Login dialog for more controls and options.
Check the Merge layers box if desired.
Increase Link depth to follow links more than one additional file deep, if the file on the server in turn links to other files.
Press Create Data Source.
Expand the new data source hierarchy, highlight the linked components within, and choose Copy.
Click into a blank part of the project outside of the data source hierarchy and choose Paste.

Shortcut: Instead of copying and pasting from within the new data source into the local part of the project, we can just drag and drop components into the local part of the project. That has the same effect as Copy and Paste when dragging and dropping from a read-only data source into the local part of the project.

Once we copy components from within the data source into the local part of the project we no longer need the data source, and we can delete it.

New Data Source Dialog and Controls

In the main menu, choose File - Create - New Data Source. The dropdown menu provides a list of favorites to choose from as well as a More... option.

Choose More... to launch the New Data Source dialog. Choose File: kml in the Type box.

Name	Name for the new data source, "Data Source" by default. Specify a more memorable name as desired.
Type	Choose Webserver: kmlserver to connect to a .kml or a .kmz file on a web server.
Source	A connection string or path to the .kml file. The connection string can also be entered using the Web Login dialog launched by the [...] browse button.
Browse button	Click to launch the Web Login dialog, to allow use of a login and password plus use of a proxy server if desired. The Web Login dialog is also handy for providing a Test button that can be used to test the connection.
Open as read-only	Open the data source read-only. Not checked by default so that if we want to open a .kml file read/write we can do so. .kml format will lock out other users if we connect to a .kml file read/write. Therefore, if we would like to allow other users to be able to simultaneously connect to this same .kml file and we do not need to write to it, we can check this box. We can also check the read-only box if we would like to protect against accidental changes to the .kml file
Cache data	Create and utilize an accessory .MAPCACHE file for improved performance. Checked by default.
Save cached data between sessions	Save the cached data for the next time this project is opened, within a Cache subfolder within the System Data hierarchy.
Merge layers	Bring data from all folders within the .kml into a single drawing and table. Add a field for each record in the table giving the name of the original folder. Most Manifold users will check this option in the case of KML files that contain many folders, since it is usually easier to manipulate data within a single drawing in tasks such as styling the data than it is to work with a large number of layers (often the case with .kml files).
Link depth	The depth of links for recursion. Default is 1 , which means to follow any links to just one level deep. To disallow following any links, set this to a value of 0. Set the Link depth setting to greater numbers to follow links to more levels. For example, set Link Depth to 2 to follow links two levels deep, as in the case of a KML file that contains a URL to another KML, which in turn contains a link to a JPEG file on a different web server. Caution: The default setting of 1 allows the KML file to launch an external link one level deep. This is great for convenience, but it may be a risk to privacy.
Create Data Source	Create the new data source in the project pane and close the dialog.
Edit Query	Launch the Command Window loaded with a query that creates the data source using the given settings. A great way to learn how to use SQL to create data sources.
Cancel	Exit the dialog without doing anything.

Web Login Dialog

The Web Login dialog allows specifying a server and connection characteristics.

The above dialog provides the following controls:

Server	The connection string to get to the KML file on the server. This may be a simple URL or a very lengthy URL/connection string that embeds parameters such as keys that grant access or other parameters
Use login and password	Check this box for servers that require logging in with a login name and a password, providing the required name and password in the Login and Password boxes.
Use proxy server	Check this box when connecting through a proxy server. The Proxy, Login, and Password boxes allow specifying the connection string to the proxy server as well as the login name and password required to use the proxy server.
User agent	Identifies what application (Manifold) is asking for a connection. Some web servers want to know what client software is connecting, for compatibility or for business reasons. The default string optimizes compatibility (Mozilla is very generic) while also identifying Manifold Release 9 as the client. Users can adjust the string as necessary to comply with any special server requirements.
API key	Provide a key that authorizes use of an API when connecting to a proprietary data source that requires such a key. This option is disabled for server dataports that do not use it.
Application key	A secondary application key or authentication code for those servers that may require it. This option is disabled for server dataports that do not use it.
Timeout	Specify a time in milliseconds to wait for connecting to the specified server. Use 0 for the default timeout or specify whatever is the desired time to wait before giving up on the server.
Test	Press the Test button to try the connection using the specify parameters. If successful, a Connection established information dialog will pop open.

Example

We will connect to a KML file located on a USGS server that gives the locations of recent earthquakes. This is the same USGS served file that was used by the recursive KML in the KML, KMZ Google topic's example. The link to the file is:

https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/2.5_month_depth.kml

The URL shows the file is located on the earthquake.usgs.gov server. That server will, of course, log the IP address of our computer when Manifold or any other package connects, but if we trust USGS not to track our activity we probably will not care about that.

We choose File - Create - New Data Source and then in the New Data Source dialog we provide a memorable name for the data source, Web Served Recent Earthquakes. We choose Webserver: kmlserver for the Type, and then we Paste the URL string into the Source box.

We always want to get the latest data, so we will not check the Save cached data between sessions box. We do not expect this KML file will contain any onward links, but we leave the Link depth value set to 1 to allow one further reference, if the KML on the web server contains a URL.

Press Create Data Source.

A new data source appears in the project, named as we specified. When we expand the data source we see the contents linked from the KML file on the USGS web server.

Following the example in the KML, KMZ Google topic we can create a local map in our project that takes data from the linked components within the Web Served Recent Earthquakes data source.

Notes

Privacy risks from KML Files that contain links - KML files that contain links can be a risk for privacy. Nothing about the name or extension of a KML file that contains links warns users that it will open a connection across the Internet to some other server. Although it stands to reason that connecting to a KML hosted on a web server using the Web Server: kmlserver option is fair warning that we are connecting across Internet to a different computer, we might not realize that the KML which is hosted on that server might in turn contain a link to yet another server. See the discussion in the Notes to the KML, KMZ Google topic.

Problems connecting - Check the Log Window to see what is going on behind the scenes if an attempted connection does not work. The problem is usually a wrong connection string or URL, failure to provide required credentials such as an key string that authorizes access, wrong choice of protocol (the server uses WMS and the user picks something else), the server is not a FeatureServer or ImageServer or Mapserver, an incredibly slow server, a server that is offline or a server that is wrongly configured and which is not correctly using the protocol it claims to use.

Visit the Manifold community forum and talk out difficulties with other users. Make sure to post full information on what you are doing, the connection URL you used, all details of how you tried to connect (including all settings in the data source dialog), what happened, and what the Log Window reported. If other users cannot help you, spending a tech support incident will produce an authoritative analysis of the issue.

Try the URL in a browser - Checking the URL by launching it in a browser can reveal many problems with the URL or with the web server. If a URL does not work in a Manifold web server dataport, try exactly the same URL in a browser. If a browser cannot connect to the URL, the Manifold web server dataport will not be able to connect to it either. If a browser cannot connect to that URL, that indicates the problem is the URL or the web server. For example, the web server might be offline. Or, for example, If the browser connects to a page other than the actual web service endpoint, such as, to a web page that lists various options for web servers, that shows the URL is not a URL for a web server but a URL to some other sort of web page.

Connection problems are often caused by incorrect URLs. There might be a typographical error in the URL or the URL might not be an endpoint to a functioning server but instead a URL to some other web page. The server responding to the URL may have geographic restrictions (surprisingly common) that does not respond to connections from IP addresses that are thought to be in a canceled country. Trying the URL in a browser will fail in such cases. Web servers may also have other restrictions, such as only allowing connections from white listed IP addresses, from paying clients, or from those clients that use a special security scheme.