GeoJSON Servers

GeoJSON is a text-based format written in JavaScript Object Notation (JSON) that is sometimes used for interchange of vector data, drawings.   Manifold can import and link GeoJSON files.  Manifold can also export drawings to GeoJSON files.  

 

In addition to working directly with GeoJSON files, Manifold's GeoJSON server dataport can connect to data provided by web servers in GeoJSON format.   This topic shows how.   A related topic, GeoJSON, shows how to work with local files in GeoJSON format.   Note that GeoJSONL is a related, but different format from GeoJSON.  

 

Please read the GeoJSON topic for important information about GeoJSON format and Manifold.   See also the CSV Servers file for tips that apply to using CSV servers, GeoJSON servers, and JSON servers.

 

We can bring data from GeoJSON servers into Manifold using two, related techniques:

 

 

 

 

Connecting to a GeoJSON server:

 

  1. Choose File - Create - New Data Source in the main menu, or right-click in the Project pane and choose New Data Source.

  2. In the dropdown menu choose More... to launch the New Data Source dialog.

  3. Choose Web Server: geojsonserver in the Type box.

  4. Enter the URL for the GeoJSON data in the Source box.

  5. Default settings for other options will usually work for most GeoJSON data.   Press Create Data Source.

  6. A new data source appears in the project.  Open it to see the drawing and table that is linked in from the GeoJSON server.

 

Creating a local copy of the GeoJSON drawing and table:

 

  1. Ctrl-click on the drawing and on the table linked in from the GeoJSON to highlight them.

  2. Press Ctrl-C or click the Copy button in the Project pane toolbar.

  3. Click anywhere in the main, Manifold .map part of the project outside of the new data source's hierarchy.

  4. Press Ctrl-V or click the Paste button in the Project pane toolbar.

 

Drawings and tables that are linked from GeoJSON data are neither selectable nor editable since they do not have any indexes.  We cannot add an index to a linked GeoJSON table since GeoJSON format does not support indexes.   We can, however, add a key field and index to a local table that has been copied and pasted from a linked GeoJSON table.  Such local tables are fully capable, native Manifold tables.   We can easily add a key field and index to them to enable editing and selection in both the table and the table's drawing, using the procedure given in the Add an Index to a Table topic.

 

Adding an index to a table:

 

  1. Double-click on the table to open it.

  2. Choose Edit - Schema.

  3. In the Schema dialog click the Add Identity button.

  4. Press the Save Changes button.

 

New Data Source Dialog and Controls

In the main menu, choose File - Create - New Data Source.   The dropdown menu provides a list of favorites to choose from as well as a More... option.  

 

 

Choose More... to launch the New Data Source dialog.   Choose Web Server: geojsonserver in the Type box.

 

 

Name

Name for the new data source, "Data Source" by default.  Specify a more memorable name as desired.   If we forget the origin of a data source we can hover the mouse over the data source name in the Project pane and a tool tip will provide connection information.

Type

Choose Web Server: geojsonserver in the Type box to connect to a GeoJSON format source over the web.  Choosing the GeoJSON server type when connecting to a file over the web will use the GeoJSON server dataport regardless of the file extension, either .geojson, or .json, or other extension.

Source

A connection string to the web server GeoJSON data. This may be a simple URL as in the example above, which is just a URL to a file on the server, or a very lengthy, complex  URL/connection string that embeds parameters such as keys that grant access or other parameters.  The connection string can also be entered using the Web Login dialog launched by the [...] browse button.

 Browse button

Click to launch the Web Login dialog, to allow use of a login and password plus use of a proxy server if desired.   The Web Login dialog is also handy for providing a Test button that can be used to test the connection.

Open as read-only

Open the data source read-only.  Has no effect with GeoJSON servers since they are read-only in any event.

Cache data

Cache data downloaded from the server while the project is open.  Provides better interactive performance and greater flexibility with read-only data sources.

Save cached data between sessions

Save the cached data for the next time this project is opened, within the .map project itself in a Cache sub-folder in the System Data hierarchy.   Caution: checking this box can result in very large .map files when the results of browsing very large data from web servers are all saved.  However, having such data cached in the .map is handy for offline browsing of the project.

Create Data Source

Create the new data source in the project pane and close the dialog.

Edit Query

Launch the Command Window loaded with a query that creates the data source using the given settings.  A great way to learn how to use SQL to create data sources.

Cancel

 Exit the dialog without doing anything.

 

 

 Pressing the browse button next to the Source box launches the Web Login dialog, to specify a server and connection characteristics.

 

 

Server

The connection string for the server.  This may be a simple URL as in the example above, which is just a URL to a file on the server, or a very lengthy, complex  URL/connection string that embeds parameters such as keys that grant access or other parameters

Use login and password

Check this box for servers that require logging in with a login name and a password, providing the required name and password in the Login and Password boxes.

Use proxy server

Check this box when connecting through a proxy server.   The Proxy, Login, and Password boxes allow specifying the connection string to the proxy server as well as the login name and password required to use the proxy server.

User agent

Identifies what application (Manifold) is asking for a connection.  Some web servers want to know what client software is connecting, for compatibility or for business reasons.  The default string optimizes compatibility (Mozilla is very generic) while also identifying Manifold Release 9 as the client.  Users can adjust the string as necessary to comply with any special server requirements.

API key

Provide a key that authorizes use of an API when connecting to a proprietary data source that requires such a key.  Not used with GeoJSON servers.

Application key

A secondary application key or authentication code for those servers, such as here.com (also known as wego.here.com) that may require it.   Not used with GeoJSON servers.

Timeout

Specify a time in milliseconds to wait for connecting to the specified server.  Use 0 for the default timeout, or specify whatever is the desired time to wait before giving up on the server.

Test

Press the Test button to try the connection using the specified parameters.   If successful, a Connection established information dialog will pop open.

 

Example: Connect to a GeoJSON File through the Web

Web servers providing GeoJSON data may create the data on the fly or they may simply serve a GeoJSON file stored on the server.  A very common use of GeoJSON servers is to access data published as GeoJSON files and placed on web server for each access by anyone.    

 

For example, if we have a website called www.manifold.net and we want to make a GeoJSON file available to Manifold users through the GeoJSON server dataport, all we need do is to place that file somewhere on our site where it can be accessed with a URL.   Suppose we have a file called DistrictofColumbia.json that provides building footprints in the District of Columbia from the Microsoft GeoJSON building footprints data set.  This is the same example file used in the GeoJSON topic.  We could put it into a folder in our site called ~/files and the URL to connect to it would be:

 

http://www.manifold.net/files/DistrictofColumbia.json

 

The link above is a live link, so we should not just click it if we do not want 18 megabytes of GeoJSON text to appear in our browser.    To bring the data into Manifold. we can use that link to create a new data source.

 

We choose File - Create - New Data Source.  

 

 

Choose More... to launch the New Data Source dialog.

 

 

We use Data Source  as the Name.  We choose Web Server: geojsonserver as the Type.   In the Source box we enter the URL connection string:

 

http://www.manifold.net/files/DistrictofColumbia.json

 

We uncheck Save cached data between sessions and leave other options at their default settings.  Press Create Data Source.

 

 

A new data source appears in the project pane.  We expand it to see it contains a drawing called DistrictofColumbia Drawing.     We will not see the drawing until data comes down from the server, which can take seconds, or even tens of seconds, with slower connections or slower servers.   GeoJSON is also a very slow format, because it is plain text, so larger GeoJSON files can take seemingly forever to fetch from servers.

 

 

We double-click open the drawing and see it contains the same drawing as imported in the example in the GeoJSON topic.  The drawing is read-only, since it is still resident in the GeoJSON file up in the manifold.net server.   However, as a Data Source with the data on the server, if there are any changes in the GeoJSON file placed on the manifold.net website, for example, if there were edits to the building footprints in the file, the drawing in the data source in our project would be updated with fresh data from the server when we refresh it.   

 

 

As convenient as it may be to leave the data on the server, in most cases, a better way to use the data is to Ctrl-click the drawing and table in the data source, press Ctrl-C for Copy, and then click into the main part of the project (outside the Data Source) and press Ctrl-V for Paste.   That copies the drawing and table into the project.   We can then open the table, add an index, and the drawing will be read / write.  We can then do with it whatever we want, such as Style it,etc.

Notes

Problems connecting - Check the Log Window to see what is going on behind the scenes if an attempted connection does not work.     The problem is usually a wrong connection string or URL, failure to provide required credentials such as an key string that authorizes access, wrong choice of protocol (the server uses WMS and the user picks something else), the server is not a FeatureServer or ImageServer or Mapserver, an incredibly slow server, a server that is offline or a server that is wrongly configured and which is not correctly using the protocol it claims to use.  

 

Visit the Manifold community forum and talk out difficulties with other users.  Make sure to post full information on what you are doing, the connection URL you used, all details of how you tried to connect (including all settings in the data source dialog), what happened, and what the Log Window reported.    If other users cannot help you, spending a tech support incident will produce an authoritative analysis of the issue.

 

Try the URL in a browser - Checking the URL by launching it in a browser can reveal many problems with the URL or with the web server.  If a URL does not work in a Manifold web server dataport, try exactly the same URL in a browser.  If a browser cannot connect to the URL, the Manifold web server dataport will not be able to connect to it either.  If a browser cannot connect to that URL, that indicates the problem is the URL or the web server.   For example, the web server might be offline.  Or, for example, If the browser connects to a page other than the actual web service endpoint, such as, to a web page that lists various options for web servers, that shows the URL is not a URL for a web server but a URL to some other sort of web page.   

 

Connection problems are often caused by incorrect URLs.  There might be a typographical error in the URL or the URL might not be an endpoint to a functioning server but instead a URL to some other web page.  The server responding to the URL may have geographic restrictions (surprisingly common) that does not respond to connections from IP addresses that are thought to be in a canceled country.  Trying the URL in a browser will fail in such cases.  Web servers may also have other restrictions, such as only allowing connections from white listed IP addresses, from paying clients, or from those clients that use a special security scheme.  

 

See Also

Schema

 

File - Create - New Data Source

 

CSV Servers

 

GeoJSON

 

GeoJSONL

 

JSON

 

JSON Servers

 

JSONL

 

Example: Import CSV and Create a Drawing - Import a CSV file by creating a New Data Source, copying and pasting the table conveyed by the CSV, prepping the table for use as a geocoded table and then creating a drawing.  

 

Example: Create a Drawing Dynamically from a Geocoded Table - A more general, cooler way to create a drawing from a geocoded table, without adding any fields to the table.   We create a small query that generates geometry on the fly, and then we create a drawing from the query.   This is how more advanced users often do it, using a technique that is perfect for creating drawings from tables in remote DBMS packages or read-only files.