This topic is a companion and introductory topic to the Example: Spectacular Images and Data from Web Servers topic, a must-see topic that provides a gallery of views available from web based data sources. See File - Create - New Data Source for instructions on connecting to many different types of web servers, including custom web server connection strings.
In this topic we show a specific, step by step example of how to use Image Servers. We will show every step, even the small steps. That requires a long sequence of illustrations to show what is done in a few seconds and a few clicks of the mouse in real life.
To fit into this documentation, illustrations show a small Manifold desktop, with only a few panes, docked to the right side. In real life we use a much larger Manifold desktop, and more panes would be turned on, with panes docked to the left or to the right, or undocked, as we prefer. Right-click a pane's tab to change where it is docked. Manifold will remember that new arrangement for our next session.
There are two ways to add web served layers to a project:
Using the New Map dialog is how many users add image servers to a project, because usually we want to use an image server as a base layer in a map anyway, and most often we want to use one of the image servers in the Favorites list.
Starting with a new, blank project, we choose File - Create - New Map.
In the New Map dialog, we choose the image server we want as a Base layer. The pull-down menu for the Base layer box will be loaded with all of the image servers in our Favorite Data Sources list, with the last used image server offered by default. We choose the Bing streets image server. If our project had other layers, such as drawings, images, or labels, that could be added to the map, those would appear in the large list pane (empty in the illustration above, since the project is new and as yet empty) as choices. See the Maps topic for other options in the dialog.
Press Create Map.
Manifold creates a new Bing imageserver data source and a map that uses it as a layer. We double-click the map open to see Bing in action.
The Counts pane in the Status bar shows zero records selected out of over 366 billion records. That is because total size of the data set from which the image in view is served by Bing is hundreds of billions of tiles, with each tile being a record. Between image pyramids (intermediate levels) used by Bing and on-demand streaming of tiles, only a very small number of tiles need to be fetched from that huge data set. But still, the total number of records in the data set really is that big.
As convenient as the shortcut of using New Map may be, we might want to add an image server to our project that is not one of the favorites, or by not also creating a map. We might also want to add web served layers that use a web serving technology, such as TMS or WMS, that is different from image servers like Bing or Google. That is easy to do using Create - New Data Source.
We begin with a new, blank project.
To create an image server we want using Create - New Data Source, we have two choices:
Use a Favorite - If the image server is in our Favorites list, we can choose it with a single click.
Choose any image server - If the image server is not in our Favorites list, we can choose it from the very long list of options, configuring a choice two work with almost any image server.
We choose File - Create - New Data Source.
In the resulting dropdown menu, Manifold provides several image servers in the default Favorites list, including a Bing streets server. We can choose it with a single click if we like. Done!
For the sake of this tutorial, however, let us pretend we do not want to use Bing or one of the other Favorites. Here is how to choose any image server through the full process, whether or not it is in our Favorites list.
We click File - Create - New Data Source, and then we choose More... to launch the full New Data Source dialog.
We choose the name Wikimapia for the data source. The Type box in the New Data Source dialog is loaded with a very long list of file types, databases and other possible data sources, including a number of different web server types. We choose the Web Server: imageserver type and then from the alphabetic list of image servers, a list beginning with Bing Maps Hybrid, we will choose the image server we want.
We will choose the Wikimapia Street Map image server. Image servers on the list vary over time in different Manifold builds,, as different providers come and go. At the time of this writing, Wikimapia provided a free image server.
We have assigned a Name of Wikimapia so when we see this data source in the project pane we have a more descriptive name to tell us what the data source is.
We review our choices and press Create Data Source. A new data source appears in the project, named Wikimapia.
We click on the + icon to the left of the data source to expand the data source. As we do, Manifold displays the table of tiles fetched from the data source along with the image created to display the tiles.
We double-click on the image server image to open it. It initially appears zoomed to fit.
At a zoomed out view that shows the entire world few details are visible. We zoom into Europe by right-clicking and dragging to draw a zoom box.
As we zoom further in detail appears, such as borders for countries. Depending on the image server we use, we can have an entire world full of detail down to the smallest streets at our fingertips. We can zoom into a street corner in Manila or visit Times Square in New York. We can see the smallest road in Africa or choose a path to walk through Beijing.
However, for the time being we will switch gears, close the street map image by clicking the X in the corner, and add another image server, this one providing satellite photography, to our project.
In Manifold, a map is a component that shows multiple layers. Those layers can be different images or different drawings. We will use a map to show a satellite photography image server layer together in the same map as the streets image server layer.
To do that we first need to create a map. We will first create an empty map and then we will drag and drop layers into it.
Important: The Wikimapia data source is a read-only data source. The hierarchy within the yellow database cylinder of Wikimapia comes from Wikimapia's server. We cannot write to Wikimapia's server or create new components within that server.
Therefore, if we want to create something new, such as a table or a map, we must create that within the read/write portion of the project that is local and is not hosted within Wikimapia's server. When creating new data sources, if we use the File menu we should first click outside the Bing data source to make sure the focus is on the local part of the project outside of the data source hierarchy. When creating a data source using the Right-click context menu, we must Right-click outside the data source hierarchy.
We Right-click in a blank part of the main region of the Project pane, outside of the Wikimapia data source hierarchy.
We choose Create - New Map.
In the New Map dialog we use the default name of Map. For the Base layer, we choose (none) since we will use the Wikimapia image server already created. We leave the coordinate system as Pseudo-Mercator, since that is a good match to the Pseudo-Mercator coordinate system used by Bing image servers.
Press Create Map to create the new map. In the project we double-click on the new map to open it. It appears as a blank component since no layers have as yet been added to it.
We add the Wikimapia image server image to the map by dragging and dropping it into the map.
The layer appears, by default zoomed to fit since it is the only layer in the map. If it did not appear, we could click Zoom to Fit for the new layer to appear in the map. Completely blank maps have no scale chosen so when a new layer is added to them the display may need to adjust for an initial view.
As before we see the Wikimapia image server image. But now, instead of seeing it in its own image window we see it opened as a layer in the new map that we created.
We will now create another image server data source and add that as a second layer to the map.
To do that we will Right-click anywhere in blank region of the project and in the resulting context menu we will choose Create - New Data Source. The image server we would like to use is one of the default Favorites.
We choose Bing Maps Satellite, the Microsoft Bing image server web server that provides seamless overhead satellite photograph views of the Earth.
In the Project pane we open the Bing Satellite data source and then we can add the image to our map by dragging and dropping it into the map.
A new tab for the image appears in the tab strip at the bottom of the map and the image fills in, showing a satellite view of the entire world.
Because the image as a layer is opaque it hides the streets image below it. We can turn the layer off by double-clicking it so we can see the layer below. Alternatively, we could drag the satellite layer to the right of, that is, below, the streets layer.
The point of adding a satellite photo layer above a streets layer is so we can use the streets layer to more conveniently navigate to a location of interest. Most people can't find their way around the world based entirely on satellite photos so using conventional street maps is a much easier way to get around.
When our view into a map is panned and zoomed that view applies to all of the layers in the map whether they are visible or not. We can navigate to a particular location using the streets layer and then we can turn on the satellite photo layer to see what we want to see.
To zoom into the map we right-click and drag to create a zoom box. The map pans and zooms into the rectangle we draw to show a closer view of Europe.
The Wikipedia image server automatically adds more detail to the display based on zoom level, so as we zoom further into the view country borders will appear.
Using right-click and drag we will continue zooming further and further into the view, into the region marked with a magenta arrow in the illustration above, until we see a view of the central part of Rome, where the old forum and the Colosseum are located.
At this point we have zoomed far enough so we can turn on the satellite image layer and zoom further based on what we see in the satellite photography. We turn on the satellite image layer by double-clicking the tab for that layer.
Perfect! Now we have an aerial view of the old Forum, the Colosseum, the Palatine Hill and, running diagonally at lower left, the Circus Maximus. We will right-click and drag to zoom further into the view to take a closer look at the Colosseum.
There, we've just zoomed in from orbit to see the Colosseum in Rome, and the above is not even the maximum resolution we can see. There is even more detail if we zoom closer.
We can see the full resolution of the tiles the Bing image server is sending by choosing View - Zoom to Native. That will adjust the zoom level so that each pixel in the highest resolution tile available takes up one pixel on our monitors. Depending on the number of pixels per inch on the monitor a native zoom will usually show the sharpest visual display of an image.
Or, we can simply keep zooming in using right-click and drag to zoom box.
In major cities, Bing's satellite resolution is usually so good that we can easily make out individual people walking about. In the image above, it looks like the satellite snapped the image used early in the morning before the Colosseum was open for tourists.
By zooming in further we can sometimes see a more informative display than native resolution. Zooming further into the image does not add information which is not there at native scale, but it can make it easier for human sight to see details that might be difficult to make out at native resolution.
In the illustration above we have zoomed further into the view beyond native zoom resolution. Each pixel in the highest resolution tile is now shown using multiple pixels on the monitor. Instead of each pixel being the tiniest possible dot on the monitor, each pixel now appears as a small square.
The image is not as clear: it is definitely fuzzier, but it may be easier to understand what we see in the ruins.
We now will create a new, blank drawing in the project. We
will do that using a coordinate system (projection) for the drawing that
is the same used by the image server layers and thus, by the map. For another example of tracing over an image server
layer, see the Example:
Trace an Area in a Map over an Image Background topic.
Tech Tip: Manifold will re-project components on the fly if they appear as a layer in a map and the coordinate system they use is different than the coordinate system used by the map. For most components, such as most drawings, that process will be so fast it might not even be noticed. But for exceptionally large components even a fast re-projection may cause a noticeable pause or take even longer. We can avoid such pauses by ensuring all components which appear as layers in a map use the same coordinate system as the map. If we don't care what coordinate system an image or drawing uses, we may as well use whatever the map uses.
At any time, we can see what coordinate system any layer in a map uses. For example, to see what coordinate system the Bing satellite image uses we first click on the Bing Maps Satellite Image tab in the map to make sure it is the active layer.
We then click on the Info pane tab to switch to the Info pane, which by default will show the Component tab.
In the Component tab we see that Bing uses the Pseudo-Mercator
EPSG:3857 coordinate system, the
classic pseudo-Mercator coordinate system favored by most web servers.
We click the Project
tab to switch back to the Project pane.
Tech Tip: There are various ways of specifying coordinate systems when working with spatial data. Manifold basically supports all of them. Perhaps the very best way of specifying a coordinate system is to use an EPSG code, because EPSG codes are exact and unambiguous. Coordinate systems are often specified using both names with parameters plus the equivalent EPSG code. Whenever we see an EPSG code we should use that as the least effort and highest reliability approach. In addition, the Pseudo-Mercator and Latitude / Longitude coordinate systems are so frequently used that they are default options in the Favorites list for coordinate systems.
Next, we create a new, blank drawing. We Right-click onto white space in the Project pane.
In the context menu we choose Create - New Drawing.
The New Drawing dialog opens using the Pseudo Mercator coordinate system by default, exactly what we want. We press the Create Drawing button to create the new drawing. In fact, what we do is create a new table plus a new drawing to provide a visual interface to whatever data ends up being in the table.
We add the new drawing to the map by dragging and dropping the new drawing into the map.
New layers in a map are added just to the left, that is just above, whatever was the active layer. If we added the new tab somewhere other than the left-most position we can click and drag the layer to the left. Nothing appears in the map since the new drawing so far is completely blank, with no objects in it. It is just a completely empty, totally transparent layer.
We Click the layer to make it the active layer. It probably already is the active layer, since we have just dropped it into the map, but clicking on a layer to ensure it is the active layer is a good habit to get into just before editing, to ensure the new objects that are being created are added to the intended layer.
In the illustration above, before we choose the Create Area command, we have panned the view to move the Colosseum up slightly in the view, to get the entire outline of the former structure into view.
When the drawing layer is active, controls such as Create Area that can be used only when a drawing has the focus will become enabled. Clicking on the mode button, we choose Create Area from the drop-down menu. We can now create area objects in the drawing.
We click at every position where we want a coordinate placed to define the shape of the area we are creating. With the first click the Values tab of the Info pane automatically pops to the foreground. As we click at each location a blue coordinate box appears and the outline of the area shown by a blue line grows. The blue line will extend from the first coordinate placed to whatever is the current position of the mouse cursor, to show what the area outline will be if we click to put a coordinate at that spot.
We keep clicking until we have marked out the perimeter of the Colosseum with coordinate locations that will define the new area we want to create.
When we have clicked the last coordinate we intend, we can right-click anywhere in the drawing to choose to either commit the edit or to abandon it. We could also click the Add Record button. The blue line extending to that right-click location does not matter as only the last coordinate marked will count should we choose to commit the edit.
So far, the clicking of coordinates and resulting blue outline has just been a preview. To commit the changes and create the area, in the context menu that pops open we click on Save Changes. If we change our minds we can abandon the edits by clicking Undo Changes.
The result is a new area object created in the drawing. We have created this area by tracing over what we saw in the photographic satellite image layer below. In the illustration above we have just clicked onto the mode button to go back to Default navigation mode.
If we did a slightly sloppy job
clicking to form the area, that is no big deal. We can Alt-click the area to pick it for
review in the Values
tab of the Info pane, and then
click any vertex or segment to
enable it in Move Coordinates
editing mode. We can then move the vertices (coordinates)
we clicked as desired to even out the area, or use other editing tools
as desired. See the Example:
Change the Shape of Areas topic. See also the Example:
Trace an Area in a Map over an Image Background topic.
We can Choose View - Panes - Style to open the Style pane, to format the drawing however we like, for example, using a hatched pattern for areas where part of the pattern is transparent, to allow the image below to be seen.
Digital maps, that is drawings, are often created by tracing over aerial photography. Manifold makes that especially easy by automatically maintaining geographic synchronization between layers in a map and by providing a seemingly endless range of image servers and other web-based overhead image sources that we can use to get imagery for virtually any location on Earth.
Digitizing many structures, such as all of the buildings in a large city like Rome, will take a long time if only one person is working. All the same, it is amazing how fast digitization can go. In a small to mid-sized town, zooming and panning using a satellite photo layer as a background one person can usually easily mark the locations of all, say, fire hydrants in only one day.
In remote areas where digital maps showing dirt roads or trails may be poor quality it is usually very easy to create lines that trace over paths that are visible in image server photography. Such drawings of lines can then become digital maps for the region.
Be careful using cache - When we created new data sources for our image server images manually we checked the Cache data box in the New Data Source dialog, but we made sure the Save cached data between sessions box was not checked:
Creating an image server data source with the Cache data option checked will cache all tiles received from the server. Checking the Save cached data between sessions box will save those tiles within the Project's System Data hierarchy. That can grow the .map file to very large sizes when it is saved, possibly hundreds of megabytes if we browse a lot and thus cause many tiles to be downloaded.
Web connections are often fast in many parts of the world but even so fetching tiles from an image server is not as fast as fetching them from local storage on local disk. By caching tiles that we have seen as we pan and zoom within the images served by the image server we can provide faster performance, very much faster in the case of slower Internet connections, if we view a scene more than once.
For example, if we view a scene and then zoom in to part of it, zoom back out and then zoom into another part of it, when we zoom back to views we have already seen if the tiles for that view are cached the system does not need to fetch them from the image server over the web but instead it can fetch them much more quickly from cached storage. If we save the project and then open it later, the cached tiles will still be in the project so any views, such as a zoom to fit followed by zooms into areas of interest, are likely to be viewable using tiles already in cache.
This effect is also useful for offline use if we check the Save cached data between sessions box when creating an imageserver data source, we browse the resulting image layer and then we save the project. Suppose then we move the project to a computer not connected to the Internet, for example, a notebook computer used at a remote site where cellular phone Internet is not available: we can still browse the views in the project, and those tiles which had been downloaded into cache will still be there and will appear, even though we have no Internet connection.
Improved rendering performance, and enabling offline use by using cache is wonderful, of course, but it comes at the cost of storing tiles within cache tables that are saved within the project. As we zoom into views and pan to different views the number of tiles we have seen can grow to where the data stored in cache can be hundreds of megabytes.
If we have plenty of free space on disk, terabytes, we might not care about that at all. But if space is limited or if we want to send a project to someone else without also sending all of the cached tiles we should delete the cache.
Deleting cached tiles - To delete cached data for image servers we drill down into the System Data hierarchy and then delete the tables storing the tiles.
Click onto the + icon next to the System Data folder to open it.
Within the System Data folder we click open the Cache folder. We see there are two tables there, one table for the street image tiles served by the Bing streets image server and one table for the satellite image tile served by the Bing satellite image server.
With the Cache folder highlighted we click the Delete button or simply press the Delete key on the keyboard.
The Cache folder and the two tables it contains are also deleted. We can delete tables which cache tiles at any time without worries of de-synchronizing what is displayed. The system adjusts automatically.
Collections of Web Servers - Check the Examples page on the Manifold web site. for pre-packaged project files in Release 9 / Viewer .map format that Manifold publishes which contain collections of dozens of popular web servers data sources. Two favorites from the Examples page:
Important: Some web servers, notably TMS servers, use embedded command tokens. Use Manifold's custom setting to connect to those. See the File - Create - New Data Source topic for how to do that.
Imageserver or Image Server? - Manifold uses two styles for image server terminology, with and without a space character, using the two styles as synonyms without any special meaning attached to one or the other. For historical reasons going back to prior Manifold releases the term imageserver tends to be used in programmatic contexts. For discussions in ordinary text the term image server tends to be used.
File - Create - New Data Source - an absolutely key topic for connecting to many thousands of different web servers.
Example: Spectacular Images and Data from Web Servers - A must see topic providing a gallery of views illustrating how Manifold can use web servers such as image servers and other free resources to provide a seemingly endless selection of spectacular background maps, satellite images and GIS data with nearly zero effort.
Example: An Imageserver Tutorial - An extensive tutorial showing step by step how to add new data sources that are image servers, how to show them as layers in a map, how to create a new drawing that matches the projection of the map and how to trace over what is seen in an imageserver layer to create an area object in the drawing.
Example: Vector Layers from an ArcGIS REST Feature Server - Visit an ESRI web site, copy a URL, and then use that URL to connect to an ArcGIS REST web server that shows petroleum fields in Kansas, getting the data as a vector drawing layer. Style the layer as if it were local. ESRI refers to ArcGIS REST servers that provide vector data as feature servers.
Example: Connect to a WFS Server for State Government Data - Gathering our courage, we connect to a WFS server that provides 1200 vector layers, run by the state of Massachusetts. We open a layer showing airports and then scrape the vector data into our own local storage.
Example: Connect to a WMS Server for National Map Layers - Visit the National Map services web page, copy a URL for a shaded relief layer from USGS, and then use Style to enhance that shaded relief data for combination with other layers and really spectacular effects.
Example: Connect to a Custom Server for Cadastral Data - We connect to a custom image server that provides cadastral information originally from the French national cartographic agency, IGN. We create a map and use the Style pane to re-style the web served image on the fly into a more usable form.
Example: Connect to a Custom OpenRailwayMap Server - We connect to a custom server that provides an OpenRailwayMap view of railroads worldwide, showing railway, tram, and subway infrastructure based on OpenStreetMap data. Our first try at creating a data source does not work. After consulting the Log Window we try again with a slight adjustment and our second try works.
Example: Connect to an OSM Vector Server - We connect to an OSM Server that provides a vector layer containing points and lines in the OpenStreetMap database. We then show how to scrape (copy) data from the OpenStreetMap server into local storage. We extract building footprints from the local copy.
Example: Raster Layers from an ArcGIS REST Image Server - Visit an ESRI web site, copy a URL, and then use that URL to connect to an ArcGIS REST web server that provide a raster layer showing a mosaic of aerial photographs near Portland, Oregon.
Example: Create a New Data Source from a Web Server - Many web servers are image servers, providing image. tiles for a geographic view to browsers or to applications. Typical image servers are Bing, Google Maps, Yandex and many others. Image servers can provide street maps, overhead satellite imagery, combinations of streets and satellite imagery and other data as well. This topic shows how to create a new data source that is an image server. Using image servers is one of the most popular Manifold features.
Example: Trace an Area in a Map over an Image Background - In a map with a drawing layer above an image layer (served dynamically by an image server), create an area object in the drawing by tracing over the outlines of something seen in the image layer below
Example: Style Applied to an Image Server Image - Because the Style dialog simply changes the way an image is displayed and not the data, it can operate on read-only data served by various web servers such as WMS REST servers. In this example we look at every detail of creating a data source using a WMS REST server and then manipulating the appearance of the display with Style. We will connect to a WMS server that provides LiDAR data in various forms, including as terrain elevation.