Manifold builds can be either Cutting Edge builds or Base builds.
Cutting Edge builds for Manifold Release 9 and Manifold Viewer introduce new features for community feedback, and end in a short number, such as 9.0.178.6. Cutting Edge builds usually time-out after the next Cutting Edge build is issued.
This User Manual tracks the latest Cutting Edge build. Cutting Edge builds have proven to be so stable that almost all Manifold users switch to the latest Cutting Edge build when it is issued. Cutting Edge builds use portable installations to facilitate frequent updates.
Each new Cutting Edge build is announced in the Cutting Edge section of the user Forum with download links plus a list of changes, new features and bug fixes.
Base builds for Manifold Release 9 and Manifold Viewer are issued every few months for those who do not want to keep up with the latest Cutting Edge build. Base builds are called Manifold System builds and are numbered to end in a three-digit number, for example, Manifold System 9.0.179. Base builds do not include the latest features and bug fixes, which always appear first in the more frequent Cutting Edge builds.
Base builds usually are significantly older versions than discussed in the latest online User Manual. Base builds for Release 9 and Viewer run forever and never time out, although it is an epic mistake not to keep up with more recent base builds. Base builds are provided as Windows Installer installations with an optional portable installation as well.
Later builds will include all improvements and advances from previous builds. Install the latest build, and you get it all.
Base builds: Launch Manifold and choose Help - About. Manifold will reach out through Internet to check for a new Base build version. If we do not have an Internet connection the system cannot check for updates. The system does not automatically check for updates.
Cutting Edge builds: See the Cutting Edge section in the forum for the latest cutting edge build. New builds usually include bug fixes, which are not listed below. See the announcement in the forum for each build for a list of bug fixes in that build.
To learn how to install and run Manifold, see the Installations topic.
Run the latest, Cutting Edge version of Release 9 or Viewer. Do not worry about running Cutting Edge builds. Learn how to use portable installations so you can run Cutting Edge builds. Newer versions of 9 or Viewer are free to download.
If 9 has been activated on a machine, no new activation will be required to uninstall an older build and to then install a newer build.
New builds incorporate all that has gone before. No need to install an older build before downloading and using the latest build. No need to install an "official" build before downloading and using the latest Cutting Edge build.
This Changes and Additions topic always lists key changes in the latest build. Scan recent changes to get started while updates to documentation are prepared and published.
Topics and screen shots are updated frequently online. If there is a puzzling mismatch between a topic and the latest build, check back in a few days to see if the topic has been updated. For added discussion on new builds, visit the georeference.org user forum. To see discussion for a particular build, enter the build number in the filter box at the top of the forum page.
Build 181.0 is a Base Build that is recommended for all users. It implements a huge new GetFeatureInfo request capability that opens the door to WMS based click-to-get-data websites and OGC tools. It incorporates all improvements from the 9.0.180.x series of builds. Build 181.0 also introduces a new policy to remove expiration dates for Cutting Edge builds. That prevents websites from timing out when Cutting Edge builds are used for production sites, and it positions builds for planned developmental projects.
Base Build - Manifold Base builds incorporate all improvements from the previous few months of Cutting Edge builds. While Cutting Edge builds can include experimental facilities, Base builds are fully supported product configurations that define the standard product going forward. New Cutting Edge builds will add features and bug fixes to the current Base build for those users who want to work with the latest and greatest. After a few months a new Base build is issued that incorporates changes and the cycle starts again. Base builds are also published as full Windows Installer installations in addition to portable installations.
GetFeatureInfo - A GetFeatureInfo request in a WMS URL launches a query to fetch data for whatever object is under the specified cursor position in the specified WMS bounding box and image size. It provides a key OGC capability to fetch data when users click on a spot of interest in a web site or WMS-using tool.
Automatic Integration with Server - Manifold's GetFeatureInfo capability is automatically integrated with Manifold Server's WMS and HTTP serving capability. Publish a map from your project with Server and automatically you get GetFeatureInfo capability.
Automatic Defaults - Following Server's WYSIWYG (What You See Is What You Get) design, GetFeatureInfo capability exactly matches the interactive Web user interface shared by Server for HTTP pages. Configure your maps just once using the Layers pane in the Manifold desktop, arranging the layers in the stack you want, clicking any layers to set as pickable for data display or not, and that works for both the regular HTTP web pages Server creates and also for GetFeatureInfo querying through WMS. It all works automatically with just a few clicks in the desktop authoring environment.
Automatic Layers - No need to specify QUERY_LAYERS and LAYERS parameters: Server takes care of that automatically. A map can have dozens of layers and the same precedence and pickable status used for the desktop and regular Web pages also applies in WMS GetFeatureInfo requests. That simplifies URL strings for WMS and reduces errors from wrongly-specified layer names.
JSON Output - Server's GetFeatureInfo output uses INFO_FORMAT=application/json to generate neatly formatted data using JSON format. JSON has emerged as a modern format for web sites, being easily consumed by web text and data styling engines, and also easily imported into many data tools.
Easy to Test - URL strings to connect to Server using WMS to get feature info can be entered into browsers, to easily test URLs.
No Expiration - Starting with build 180.0, all future Cutting Edge builds will no longer expire 90 days after issue. That allows developers to keep using Cutting Edge builds for as long as they want, to compare new builds and to test experimental facilities that will be issued in future builds. Cutting Edge builds have proven so reliable that many users deploy them in production web servers, so removing expiration dates also prevents web sites from halting if admins do not remember to update a server to newer builds.
Build 180.8 adds WMS server capability to Manifold Server's HTTP server capability. The support is automatic and does not need any additional configuration.
The HTTP server now supports OGC WMS 1.3.0, generating a WMS page.
WMS support is automatic and does not need any additional configuration. To connect to the WMS server, add /wms to the end of the URL for the web page served by the HTTP server for the WMS URL. For example, if Server is serving an HTTP page to http://127.0.0.1:8080, the WMS URL will be http://127.0.0.1:8080/wms.
The published component is served to WMS as a single layer. Future builds will likely expose multiple layers to allow turning them on and off via the WMS API.
If the published component's coordinate system is an EPSG coordinate system, the coordinate system of the WMS layer wil be that coordinate system. If the published component's coordinate system is not an EPSG coordinate system, the coordinate system of the WMS layer will be the default pseudo-Mercator coordinate system, with the published component's coordinate system reprojected on the fly into pseudo_Mercator.
Images produced by the WMS interface are always PNG with transparent background. Future builds will allow specifying a solid background with desired background color via the WMS API.
Rendering images for WMS does not render virtual layers such as North arrows, scale bars, legends or the Layers pane background color. This is done to support WMS clients (first and foremost 9 itself) that cache rendered WMS tiles and stitch final screens from cached data.
REST Endpoints - New HTTP server endpoint:
/wms - OGC WMS entry point.
The number of choices in drop-down lists has been increased from 1000 to 4000. The increase primarily was made to support extremely long lists of field names when working with tables with huge numbers (over 1000) of fields.
(Fix) Drop-down lists for fields or components no longer fail to limit the number of choices to the maximum limit.
(Fix) When rendering legends in Server generated web pages, the Javascript control (web UI) no longer ignores changes to on/off layer states. Previously, turning layers on/off in the layers button of the web UI toolbar would turn layers on/off but if a legend was shown the legend would not be updated to turn those layers on/off.
(Fix) Attempting to filter or to join a table with an RTREE index on x, y, or tile fields and no BTREE indexes no longer sometimes fails, returning no records.
Build 180.7 adds conveniences to the desktop user interface as well as more information in the web user interface. Access keys in Windows provide keyboard shortcuts to launching commands from menus. Those shortcuts can now be used whether access keys are displayed or not.
Access keys are also turned on for automatic menus that are generated in Manifold dialogs and panes, for example, in the lists of fields that can be picked when choosing a field in the Transform pane. That makes it easy to pick a field in the Transform pane field picker dropdown: click on the picker box and enter the first letter in the field name desired. That will jump to the fields that start with that letter, and then the up/down arrow keys will move between the fields to pick exactly the field desired.
In the web user interface, if a description has been entered for a location or a layer, that description will automatically appear just before the location or layer entry in the web UI's menu tools for picking a location or turning a layer on and off.
Menus always use access keys for menu commands, even if access keys are hidden.
Automatically generated menu commands have access keys set to the first alphanumeric character in the command name. This allows quickly selecting, for example, a field name in a long drop-down list of many fields by typing the first character. If there are multiple names that start with the same character, using an up or down arrow will move the cursor between them.
(Fix) Parsing command names no longer interprets && as an access key (&& denotes a single & character. Previously, that was erroneously interpreted as an access key set to &).
The Info pane allows pinging the picked record in a map window. Pinging shows a sequence of circles with decreasing radius centered at the record. Pinging helps locate the record on a big and / or busy screen.
The Info pane allows pinging the picked record in a layout window.
The Javascript control (web UI) shows descriptions for locations.
The Javascript control (web UI) shows descriptions for layers.
Build 180.6 fixes quirks in the prior experimental build to create a robust cutting Edge build suitable for production use. It also expands the Info Tool in the Manifold Server web user interface to launch email and URL text values as hyperlinks in a new browser window. The Info tool has also gained a dropdown list control that allows specifying the layer from which Info tool clicks show field data.
The Info tool in the Javascript control (web UI) shows text fields that have been styled as email or URL styles as hyperlinks, launching the link when clicked in a new browser tab.
The Info tool's pane provides a dropdown list that allows specifying the layer from which an Info tool click picks data. The available choices are limited to layers that can be turned on or off that also can be picked. The dropdown list provides a list of layers, plus an auto choice (the default) that picks data from the topmost visible layer. If only one layer in the displayed map is visibule and can be picked, the dropdown list box does not appear.
REST Endpoints - New HTTP server endpoint:
/info - Now supports an optional layer=xxx parameter, which specifies the ID of the layer to pick data from, as returned in the layers=[...] configuration section.
(Fix) The Info tool in the Javascript control (web UI) no longer picks wrong objects on a non-square map, that is, where the cx and cy pixel extents are different.
(Fix) The Info tool now checks the click location against object geometries instead of object bounding boxes, which eliminates inaccurate clicks for complex geometries.
(Fix) In the web UI for touchscreen devices such as telephones and other mobile devices, touching a scrollbar area in lists for the Locations tool, Layers tool, or Info Tool no longer sometimes misinterprets the touch as a command to recenter the map.
(Fix) In a desktop session, opening a map window for a persistent or virtual map with a layer that references an unknown table no longer shows an error. Instead, the layer is ignored silently.
(Fix) Exporting a component bound to a query to GPKG / SQLITE no longer fails.
Build 180.5 is an experimental, cutting Edge build that adds an Info tool to the web server user interface. The Info tool reports attributes (field values) for a clicked object.
The Javascript control (web UI) used by the HTTP server includes a new Info tool. Clicking the Info tool changes the cursor into a cross cursor. Clicking on the map with the Info tool active takes one of the objects under the cursor and shows attribute field values for that object. Retrieving field values from the server is performed asynchronously without reloading the page.
The Info tool picks the topmost layer that has a Layers pane pick mode other than "no pick" and which is currently visible, excluding layers that are currently hidden, for example, dynamically hidden by min / max scale settings, explicitly hidden by the layers button in layers in the web UI toolbar, or implicitly hidden in the map and not appearing in the layers button list.
The Info tool shows all fields in the order specified by the schema except binary fields, which are not reported.
The Info tool formats field values according to the styles set in the table window.
The Info tool works on touch screens.
REST Endpoints - New HTTP server endpoints:
/info - Returns data for the Info tool as JSON, supporting the following parameters: bounds, size and layermod with the same meanings as in the /map endpoint, center=XX,YY specifying click coordinates in pixels (the default is the center of the screen), and radius=RR specifies search radius in pixels (the default is 6, same as on the desktop).
Build 180.4 is a cutting Edge build that adds new web server capabilities, upgrades use of optional GDAL file import capabilities, adds useful capabilities when working with Z values, and provides fixes.
Zoom on Cursor Position - Zooming using the mouse wheel in the Javascript control (web UI) centers on the location under the initial cursor position, the same as in the desktop Manifold UI.
URL base - The HTTP server supports a new -urlbase:xxx command-line option to specify the URL base. The default is /. The option is useful when the HTTP server is used together with a general-purpose web server.
REST Endpoints - New HTTP server endpoints:
/version - Returns server version and server start date and time in plain text.
Compatibility with New GDAL Builds - The GDAL dataport now supports GDAL 3.4.x, GDAL 3.5.x and 64-bit raster data, GDAL 3.6.x, and GDAL 3.7.x and signed 8-bit raster data.
Raster GDB - GDAL 3.7.x allows reading raster geodatabases.
Retain Z Values - The GeomNormalize, GeomToShapes, and GeomToShapesEsri query functions retain Z values instead of discarding them.
SHP Export with Z values - Exporting areas with Z values to SHP no longer discards Z values.
(Fix) Zooming using the mouse wheel in the Javascript control (web UI) disables panning until zooming finishes.
(Fix) Exporting DBF / SHP no longer fails on large negative integer values (-1 billion or more negative).
(Fix) Reading vector data using GDAL no longer sometimes misapplies the rectangle in the spatial filter if it is specified. That might have happened when showing data coming from a live GDAL data source on a map.
(Fix) Exporting areas with Z values to GDB no longer fails.
Build 180.3 is a cutting Edge build that extends Manifold Server capabilities for HTTP map serving, including automatic adjustment of the default web page user interface for mobile devices, the ability to turn layers off and on, support for locations, and the ability to set display image size.
The ODBC driver supports extra connection options required by newer versions of Excel 365.
Set web map size - The HTTP server allows -cx:xxx and -cy:xxx command line options to specify the image size in pixels. The maximum allowed value for each option is 4096. If neither CX nor CY is provided, the image size is 800 x 800 pixels. If both CX and CY are provided, the image size is CX by CY. If only CX or CY is provided, the image size is CX by CX or CY by CY, providing a quick way to set square image sizes by specifying only one parameter.
The HTTP server better handles invalid requests with attached content. (Previously they were being rejected later than they could have been.)
Attempting to create a second instance or to reinitialize an existing instance of the Javascript control provided by the HTTP server does nothing. There can currently be only one instance of the Manifold Javascript control per web page. This limitation will be removed in the future.
The HTTP server provides safeties if the image size is specified to be very small: If the size of the Javascript control (the web UI) provided by the HTTP server is less than 200 x 200 pixels, the control reduces to a static image and the toolbar will be turned off for lack of space. Panning and zooming using the cursor also will be turned off for lack of a way to get back to the original extent (also, because the image might be too small to do panning and zooming in general.)
The HTTP server shows the description of the served component below its name. When providing the description for a map, make sure to edit the description of the map component instead of the description of the active layer.
The Javascript control (web UI) provided by the HTTP server puts the toolbar directly onto the top edge of the map without any gaps, to better use the available space.
Cache priorities - The HTTP server uses the following priorities for the image cache: full view is prioritized over any other view, and views showing default layers are prioritized over views with some layers turned on or off.
Adapt to mobile devices - The Javascript control (default web page) provided by the HTTP server adjusts the UI for devices with a touch screen such as phones: toolbar buttons are made bigger to make them easier to tap with a finger, hover effects for toolbar buttons are turned off, and panning and zooming the map using the cursor is turned off. Tapping or clicking on the map re-centers it to that spot.
Locations Toolbar Button - The HTTP server includes available locations in the configuration data for the Javascript control (web UI). The Javascript control shows locations provided via the configuration data in a drop down menu and allows zooming to those locations.
Turn Layers On and Off - The HTTP server includes layers that can be turned on or off in the configuration data for the Javascript control (web UI) . The Javascript control layers toolbar button shows layers provided via the configuration data in a drop down pane with check boxes and allows turning them on or off and then saving changes using an Apply button. The default list of layers that be turned on or off is composed according to the following rules:
Hide layer groups - If the served component is a map, top level layers can be turned on or off but layers inside layer groups cannot. The distinction allows maps to have both layers that can be controlled via the web UI and layers that cannot be turned on and off via the web UI.
Last layer stays -If the served component is not a map, its only layer cannot be turned on or off.
REST Endpoints - Changes to HTTP server endpoints:
/config - A new endpoint: returns the configuration data for the served component as plain text, this is useful as a reference when setting up a web site.
/map - Supports a new parameter: layermod=hXXsYY... to turn layers on or off, the value of the parameter is a string of hXX and sYY commands where hXX forces a layer with an ID equal to XX to be hidden, and sYY forces a layer with an ID equal to YY to be shown unless it is oherwise hidden due to the current scale being outside of the layer's min-max scale range. h is a mnemonic for hide and s is a mnemonic for show. Layer IDs are found in the configuration data for the component.
Build 180.2 is a cutting Edge build that introduces many optimizations and improvements for Manifold Server when used as a TCP server to share data to Manifold clients or as an HTTP Server for Internet map serving to any browser. Rendering enhancements increase performance for desktop use as well as map serving performance. A new setting for resampling allows zooming in to see individual pixel boundaries in rasters, a useful feature for pixel based editing.
New caching capabilities for web server sources often used as backgrounds in maps served to the web by Server can result in dramatically faster performance, especially when many users are browsing maps on a Server-enabled web site.
A dramatic increase in the number of connections allowed, even in the limited map server built into inexpensive Manifold Universal licenses, allows sophisticated use of many connections from web browsers for an enhanced user interface. Command buttons in the default web interface have been reworked to be more compact and easier to use.
The Style pane Options tab allows specifying the resample method for an image. Available choices are bilinear (the default) or nearest neighbor. Choosing bilinear results in "square pixels" when zoomed far into the map, to show individual pixel positions. See the Style: Resample topic for illustrations.
Rendering a map with image layers that are completely offscreen performs faster. This enhances performance in desktop use as well as in map serving when maps have large image layers, by eliminating any rendering overhead for layers that are not visible in the current view.
HTTP maps served now put the manifold.net watermark into the middle bottom. This keeps corner positions free in the display for use by legend, North arrow or scale bar.
Web data sources tunneled by either TCP or HTTP servers now may use persistent cache if their parent is a database such as PostgreSQL, SQLite or similar. Web data sources such as image servers might be created either directly within .map files or within database data sources like PostgreSQL. The Save cached data between sessions option when creating a web data source can be used to let web data sources save all downloaded data in their parent data source so that future requests for the same data do not have to re-request it. That parent data source can be a data source such as a PostgreSQL or other database.
When projects are used by TCP servers or HTTP servers, web data sources created in the .map file will not be able to save cached tiles in the project because TCP servers and HTTP servers open the .map file in read-only mode. However, if the web data source has been created within an intermediate data source that is a read/write database like PostgreSQL or some other database, the web data source will now be able to write to the cache in that database. This significantly improves the performance of web data sources, albeit at the cost of the storage space required for cached web data source tiles.
The manifoldsrv.exe executable (used for TCP servers and HTTP servers) ignores the amount of memory to use for system cache specified as a user option in Tools - Options. Instead, it allows specifying the amount of memory to use for cache via a new -memory:xxx command-line parameter, specifying the amount of memory to use in gigabytes. The default is 4 GB. (Servers ignore the value specified in Tools - Options because server instances frequently run under specialized user accounts, and accessing user options for these accounts might be difficult or impossible.)
The startup database operated by a TCP server or HTTP server has been made available for scripts. Scripts can access the startup database using Application.GetDatabaseRoot().
TCP servers allow controlling server scripts via a new -scripts:xxx command-line parameter. Available choices:
-scripts:on - Allows running scripts (currently applies to calling script functions in queries, computed fields and constraints, in the future will also apply to running script components).
-scripts:off - Disallows running scripts (the default).
The query engine always runs queries from data sources for TCP servers on the server. See comments below.
Buttons in the HTTP map user interface use PNG icons and also show tooltips.
Buttons in the HTTP map user interface have been grouped into a toolbar placed at the center of the top edge of the map. The toolbar can be collapsed and expanded.
The default number of connections for HTTP servers has been increased from 20 to 100, to account for web browsers creating multiple connections to the server for the same client. The maximum number of connections for an HTTP server for a Universal edition of Manifold has also increased from 20 to 100.
Pages served by HTTP servers now declare compliance with HTML 5. They also now have a viewport meta tag and a UTF-8 charset meta tag.
The HTTP server caches rendered map images for performance.
The amount of memory to use for cache can be decreased below 4 GB, down to 1 GB. The default is still 4 GB.
Click the illustration at right for a full resolution view of the new default user interface for web maps served by Server.
As is well known, Manifold allows accessing data from data sources other than the main .map file within a query. An example:
--SQL9
SELECT * FROM [server]::[data];
The above will return records from the table named data on the data source named server.
If data is a query, however, the logic is more complicated. All data sources may run queries using the Manifold query engine, with Manifold syntax. Some data sources, for example, databases like PostgreSQL or SQL Server, may also run queries using their own query engine, with their own syntax.
If the data source for server has its own query engine, the system checks whether data should be run using the server's native query engine or using the Manifold query engine. (The system does that by inspecting the text of the query and looking for a $manifold$ directive.)
If data is a native query, the system dispatches it to the server to run it. If data is a Manifold query, the system runs it on the Manifold client, since the server does not know how to run a Manifold query (only the Manifold client does).
The same logic also applies to:
--SQL9
EXECUTE [[ SELECT * FROM [data]; ]] ON [server];
In the above, it looks like the query asks the server to run SELECT * FROM [data]; - that is superficially true, but to be precise, the system is asking the data source. The data source then follows the same logic as above: it checks whether data is a native query or a Manifold query, and then either asks the server to run it if it is not a Manifold query, or if it is a Manifold query, runs it on the Manifold client. That makes sense if one assumes that only Manifold clients know how to run Manifold queries written in Manifold's implementation of SQL.
All of the above is generally desired. But if the data source is Manifold Server running as a TCP server, then both of the above example queries ran on the server even if the query was a Manifold query. That is because, unlike other types of server data sources, Manifold Server can run Manifold queries. The general logic used by data sources to decide where a query should be run has been updated in build 180.2 to incorporate the exception that Manifold Server can run Manifold queries.
Before the enhancement in build 180.2, Manifold Server data sources already were running queries on the server in many circumstances. For example, if the served .map file had a query component, connecting to the server and then opening and running that query would run it on the server. Connecting to the server with a Manifold client and then opening a command window for the server and running statements in that command window would also run those statements on the server.
But opening a command window for the main .map file and running one of the above two example queries, or queries similar to those, would run the query on the client because of the previous logic that a data source should always run a Manifold query on the Manifold client. That logic has now been updated in build 180.2 so attempting to run queries on a Manifold Server data source will now always run them on the
Build 180.1 is a cutting Edge build that introduces web server (HTTP) / map server capabilities in Manifold Server within Manifold Release 9 Universal edition and Manifold Release 9 Server edition. This allows Manifold users to create a GIS-enabled web page without any need for either a web server like Internet Information Server or Apache, or a map server. The web server and map server are now built in parts of Manifold. If you have a static IP address reachable from Internet, you can create GIS-enabled web pages. No programming required.
Build 180.1 is a first release of HTTP serving in Server with a focus on keeping it simple. Coming up in future builds will be many additional features, such as many extensions to the map UI with new REST endpoints, support for WMS and other protocols, and other features.
The HTTP server can be launched and run as a service from the Tools - Manage Services command in Manifold Release 9 Server edition or from a command prompt in either Manifold Release 9 Server edition or Manifold Release 9 Universal edition.
The Server executable, manifoldsrv.exe, launched from a command prompt allows specifying the server type using a -type:xxx command line option. Specifying -type:http runs the HTTP server and specifying -type:tcp runs the TCP server (the default).
Manifold Release 9 Universal edition is limited to a single server instance regardless of its type. For example, it can run a single TCP server or a single HTTP server, but not both simultaneously. Manifold Release 9 Server edition provides an unlimited number of Server instances per server machine, allowing a mix of TCP and HTTP servers running simultaneously.
The HTTP server serves a single map, drawing, image, or labels component. The name of the served component is specified using the -component:xxx command line option. The default component name used is Main if no component name is explicitly specified.
The HTTP server implements HTTP/1.1. Supported methods are GET and HEAD. All text is returned as UTF-8. All images are returned as PNG.
The HTTP server accepts multiple HTTP requests queued to the same connection without waiting until any prior requests are processed.
The HTTP server does not use the -workers:xxx command line option. Threads used by the HTTP server are organized differently from threads used by the TCP server: the number of threads for the HTTP server is managed automatically.
The HTTP server does not use the -user:xxx and -password:xxx command line options. Future builds might support them for HTTP authentication, but that is not a priority as it is generally better to use client-side authentication.
The HTTP server defaults to port 80. Using the default port allows any browser to connect by just using the IP address as the URL, for example, http://192.168.1.99. You can use other ports by specifiying the -port:xxx command line option, for example, -port:8080 to use port 8080. Using the non-default port will require use of a URL that specifies the port, such as http://192.168.1.99:8080. The TCP server continues to default to port 9099.
The Server executable, manifoldsrv.exe, shows the server type (HTTP or TCP) in the window caption when running in a console window.
Map images rendered by the HTTP server have a manifold.net watermark in the right bottom corner. In future builds this will be moved to the center bottom position to avoid conflicting with use of the right bottom corner for scale bars, legends, or North arrows.
The web page served by the HTTP server has been organized so the map UI (user interface) is inside a Javascript control and the page itself merely creates and initializes that control. This allows putting the map served by the HTTP server into an existing web site with minimal changes. For details see the comments for web developers below.
The web page served by the HTTP server shows the name of the served component in the page title above the map.
The map UI used by the HTTP server allows panning the map using the mouse cursor with a click and drag motion. Pressing Escape while panning cancels the operation.
The map UI used by the HTTP server allows zooming the map using the mouse wheel. Multiple rapid zooms, that is, multiple clicks, using the wheel are combined together with the map being updated after the wheel has not been scrolled for 1 second.
The map UI used by the HTTP server shows buttons in the left top corner of the map for zoom in, zoom out, and zoom to fit. The extent used by zoom to fit ignores layers for web images such as Google or Bing unless all other layers in the map are empty.
Launching Manifold in the context of an external application via ODBC, COM or .NET no longer places any restrictions onto the COM threading model of the calling thread. Previously, the calling thread must have been using COM STA, otherwise attempting to launch Manifold would fail.
It is easy to put the map served by the HTTP server into an existing web site. The HTML code to do so looks like this:
<script src="webscript.js"></script>
<script>
window.onload = function () {
let config = {
bounds: { xmin: xxx, ymin: xxx, xmax: xxx, ymax: xxx, },
cx: 800,
cy: 800,
urlbase: "/",
};
manifold.createMap(document.querySelector("#map"), config);
};
</script>
...
<div id="map"></div>
The above can be put into an existing web site served by some other web server, like Microsoft Internet Information Server (IIS) or Apache. The URL for the script and the urlbase config parameter will have to be adjusted to point to an appropriate instance of the Manifold HTTP server.
When the web page loads, the script defines the manifold object and the call to manifold.createMap will create all necessary HTML elements for the map UI in the specified div, style them and wire them to events. The content of the div will then display the map served by the Manifold HTTP server and allow operating it. Future extensions to the map UI will only affect the config section in the inserted code.
Build 180 is a base build that is recommended for all users. It provides a number of conveniences requested by the user community and incorporates all improvements from the 9.0.179.x series of builds.
The Layers pane shows none overlap mode as 'none', not as an empty string. This helps distinguish label layers with none overlap mode from non-label layers.
Re-projecting a component into a new component, for example, when re-projecting an image, copies style into the new component.
Re-projecting a component into a new component selects the new component in the Project pane.
The Style pane saves in style data the method used to produce intervals for thematic formatting, thus remembering the method.
Dragging and dropping a single .mxb file into the Project pane opens it instead of importing the contents, the same as dragging and dropping a single .map file into the project.
Map windows allow centering the view to specified coordinates, automatically panning the view, using the new Center command in the context menu for the Position status bar pane. The center coodinates can be specified as:
latitude / longitude, or
pixels / stored coordinates, or
projected coordinates.
Layout windows also allow centering the view for the active layout frame to specified coordinates, automatically panning the view, using the Center command for the Position status bar pane. If the layout is read-only, the command is not shown.
(Fix) Rendering a layout frame with italic text aligned left or right no longer sometimes slightly clips the first or last letter.
Build 179.4 is a technical build that updates third party libraries, improves label performance, and introduces some user conveniences from the wishlist.
Automatic table names: Entering the name of a new, destination drawing or image in the Transform pane automatically will generate a corresponding name for the new table. The name of the new table will be adjusted as required by the database within which it is created. For example, if the destination database does not allow names that contain spaces, spaces will be replaced by underscore characters.
Reading coordinate system data from an IMG ERDAS file preserves the datum name.
Exporting data to TIFF produces TIFFW (world file) and PRJ files. Manifold always writes TIFF with GeoTIFF tags, so the world file and PRJ files are redundant when using GIS packages that understand GeoTIFF format. Adding them results in an export ensemble of files that preserve projection information when using packages that do not understand GeoTIFF format.
Label layers support a new overlap mode: none, for no overlap clipping. In this mode, the system always renders all labels, even if they overlap. This is a great way to check if desired labels have been clipped. Rendering layers with a huge number of labels in this mode may take noticeably longer than it does in other modes since the system is forced to render every label.
Rendering label layers with overlap mode set to none only stores click data for the first million labels. This avoids wasting large amounts of resources on data that is of very limited use.
Rendering label layers in high resolution with overlap mode set to layer or map, for example, during during printing, performs faster.
VCREDIST (Microsoft Visual C++ runtime) has been updated to 14.34.31938.0. This is a minor update with several fixes.
LIBWEBP has been updated to 1.3.0. This is a minor update with several fixes, particularly for use from multiple threads.
The MRSID SDK has been updated to 9.5.5.5244. This adds support for generation 4 SID files.
SQLITE has been updated to 3.41.0. This adds several performance improvements for queries. The update also adds support for canceling long operations, a capability that might be used in future builds for canceling long operations when using SQLite.
Build 179.2 introduces new capabilities for controlling the display of contents within layout frames by setting scale using new capabilities for setting scale as absolute values, by pixel sizes, or by the size of the view within the frame. The new scale controls also work for map, drawing, image, and labels windows.
The Scale status bar pane supports new modes:
absolute - Show scale as an absolute value, for example, 1:24000. This is the default mode.
pixel size - Show scale as pixel size in coordinate system units, for example, 5 m / px.
view size - Show scale as view size in coordinate system units, for example, 500 x 300 m.
The Scale dialog launched from the Scale status bar pane's context menu allows specifying the scale either as an absolute value, a pixel size or a view size.
The Position status bar pane reports the coordinates under the cursor for any active layout frame that shows a component (map, drawing, image, labels).
The Scale status bar pane reports the scale for any active layout frame that shows a component. Reporting the scale of a layout frame in pixel size mode reports the number of coordinate system units per millimeter. If the layout that contains the frame is writable, the scale can also be changed.
(Fix) Commands switching between different modes for the Position status bar pane show their descriptions in the status bar.
(Fix) Reading data from a TIFF file no longer sometimes incorrectly places pixels in the last row of tiles, pushing some pixels down and making a horizontal strip of pixels invisible.
Illustration at right: The scale readout shows the size of the view in the Map window, 205 meters by 161.982 meters. Right click it to show the scale in meters per pixel like 0.529 m / px, or in absolute scale like 1:2000.
Build 179.2 is a quick update that incorporates community requests for the new Save as Image capability introduced in the prior build, 179.1. See the 5 Minute Tutorial - DPI in Save as Image video and the 5 Minute Tutorial - Custom Views in Save as Image video for two essential Save as Image capabilities added in 179.2. See also the new Edit - Save as Image topic.
The Save as Image dialog shows the approximate size of the produced image in bytes as a reference. This helps gauge whether the produced image will be the right size various uses, such as fitting into an email as an attachment, small enough to print, big enough for the desired level of detail, and so on.
The Save as Image dialog allows specifying the render DPI (dots per inch). The default is 96, which is the default screen DPI on Windows. The render DPI is used to scale styles for vector shapes and text. Rendering web images such as Bing retrieves image data as if the DPI was at most 96, to render vector shapes and text in retrieved data at a size similar to other layers.
The Save as Image dialog in full view mode sets the render area to include all layers that are turned on, instead of all layers whether they are turned on or off. This avoids producing images with empty space near borders.
The Save as Image dialog includes a new view named custom view. The extent of the custom view can be edited by pressing an Edit button which opens the Edit View dialog. The Edit View dialog allows entering the minimum and maximum X and Y coordinates of the view as projected coordinates, in units used by the coordinate system with local offsets removed. The Edit View dialog also allows setting the extent of the view to that of the current view or to the extent of the full view, or to the extent of a specific data-containing layer in a drop down list (layers that contain no data are not listed in the dialog). By default, the extent of the custom view is set to that of the current view.
TileRenderXxx query functions include a new parameter to specify the render DPI. The render DPI must be greater than zero.
New GeomCoordX, GeomCoordY, and GeomCoordZ functions: These take a geom and a coordinate index, and return the X, Y, or Z value of the specified coordinate.
(Fix) Reading a TIFF file reads local offsets from tiepoints even if the file contains no pixel scales.
(Fix) The TileRenderXxx query functions used by Save as Image correctly restore RGB values for semi-transparent pixels. This gets rid of multiple rendering issues over transparent background: semi-transparent layers looking darker than they should, light halos for drawing objects or labels having dark edges, etc.
(Fix) Saving an image using the Save as Image dialog in full view mode on a drawing whose bounding box has zero width or height (usually happens when the drawing contains a single point or a single horizontal or vertical line) no longer produces no tiles.
Build 179.1 is a cutting edge build that introduces technology for high performance creation of raster images that capture visual displays in map windows (including drawing, image, and labels windows). The new technology can be used from a user friendly Edit - Save as Image command, or from SQL functions. Capabilities delivered in this build are just the visible tip of extensive technology inside the system that provides a foundation for fast generation of rendered views for use in upcoming new features, such as web serving.
See the First Look - Save Map Views as Images video for a quick introduction to new Save as Image capabilities.
Map windows (map, drawing, image, and label windows) support a new Edit - Save as Image command which allows rendering the contents of the window into an image of the specified size. The image is created in the data source of the map. The default name for the image is <window name> Image. Available modes:
centered view at current scale - Renders data from the center of the current view at the current scale out to the given extents
current view - Renders data within the rectangular area shown in the window into an image of the specified number of pixels horizontally and vertically. Increasing the number of pixels increases the resolution and thus detail.
full view - Renders data in the rectangular area that covers all layers except base maps (the same area that Zoom to Fit would show) with the desired level of detail.
The centered view at current scale mode allows editing both the X and Y size of the produced image. The current view and full view modes only allow editing the X size, with Y size automatically being set to be proportional to the dimensions of the render area. The maximum size of the produced image by either X or Y is 25 million pixels (this is not a technical limitation, but a sanity check).
A Current Window button in centered view at current scale and current view modes allows setting the size of the produced image to that of the current window.
A Current Scale button in full view mode allows setting the size of the produced image to cover all layers at the current scale.
Big images are rendered in parts. Rendering an image with multiple parts adjusts label placement logic to avoid part boundaries, to protect from rendering artifacts. Labels placed on a part boundary try moving to alternative locations. If that fails, for example due to overlap control, the label is suppressed.
Rendering an image tracks progress and reports the name of the currently rendered layer. This allows seeing which layers take a long time to render. Usually, the layers that are slowest to render are web images such as Bing, since those images have to be loaded through the web. Rendering can be canceled at any moment. When rendering is canceled, the system waits for in progress database and web requests to either complete or fail.
The Save as Image dialog includes a Render virtual layers option to render virtual layers, that is, the Background, Grid, Legend, North Arrow and Scale Bar virtual layers in the Layers pane for a map window. If the option is turned on, the system renders any virtual layers that are currently shown in the window. If the option is turned off, the system ignores the on/off settings for virtual layers, with the background of the produced image imade transparent regardless if the Background virtual layer is off or on. By default, the Render virtual layers option is off.
The Save as Image dialog includes an Edit Query button that composes the rendering query that implements the dialog's settings and allows customizing or saving the query.
The Save as Image dialog shows the name of the data source that the new image will be created in if that data source is not the currently opened MAP file.
After the rendering process completes the Save as Image dialog selects the created image in the Project pane.
TileRender - Takes a component, the rectangle to render, the render size in pixels, the tile size in pixels, and a boolean value that controls whether to render virtual layers, and then renders data in the specified rectangle and returns the result as a table of BGRA tiles. The component must be a map or a component that can participate in a map as a layer. The rectangle is specified in the coordinate system of the component. If the proportions of the rectangle do not match the render size, the rectangle is increased to avoid vertical or horizontal distortion. For example, attempting to render a 7000 x 8000 rectangle into a 200 x 200 image will increase the rectangle to 8000 x 8000 to match the proportions of the result. The coordinate system of the result is shifted to the left bottom corner of the render rectangle. If virtual layers are rendered, their parameters are taken from the component's properties. If virtual layers are not rendered, the background will be transparent and grid, legend, north arrow and scale bar virtual layers will be turned off.
TileRenderWindow - Same as TileRender, but instead taking the name of a component, takes the name of a window and renders data from that window. If the window shows a virtual map, the parameters for virtual layers are taken from the window.
TileRenderSingle - Takes a component, the rectangle to render, the render size in pixels, and a boolean value that controls whether to render virtual layers, and then renders data in the specified rectangle and returns the result as a BGRA tile of the specified size. The component must be a map or a component that can participate in a map as a layer. The rectangle is specified in the coordinate system of the component. If the proportions of the rectangle do not match the render size, the rectangle is increased to avoid vertical or horizontal distortion. If virtual layers are rendered, their parameters are taken from the component's properties. If virtual layers are not rendered, the background will be transparent, and grid, legend, north arrow and scale bar virtual layers will be turned off.
TileRenderSingleWindow - Same as TileRenderSingle, but instead of taking the name of a component, takes the name of a window and renders data from that window. If the window shows a virtual map, the parameters for virtual layers are taken from the window.
TileRender and TileRenderWindow are optimized for rendering big images, such rendering a map into a static image using the Save as Image command. TileRenderSingle and TileRenderSingleWindow are optimized for rendering small images, such as rendering a single tile of a map for a web request.
All TileRenderXxx query functions require the render rectangle to be non-empty. When computing the render rectangle from component data, be alert for cases where the rendered component is empty. The Save as Image dialog sets the render rectangle to [ 0, 0, 1, 1 ] in such cases so that an empty component still renders and produces an image.
(Fix) Editing the legend style for a layout frame no longer resets the Position parameter in the Legend dialog to center horizontally and center vertically, losing changes made previously.
(Fix) The Merge dialog generates a name for the new component so that it is unique in the data source of the map. Previously, the dialog always generated a name that was unique in the currently opened MAP file even if the data source of the map was different.
(Fix) The CoordSystemOverride query function no longer fails to apply overrides to an incomplete coordinate system, for example, such as an empty string.
Build 179 is a new Base build, incorporating all improvements in the 9.0.178.x series of builds. It is recommended for all users. The main changes since the prior cutting edge build, 9.0.178.6, is the provision of quick choices for which map layers are incorporated into the legend, as well as quick and easy customization of formatting for different legend entries. See the First Look - Legends in Maps and Layouts video for a quick introduction to new legend capabilities.
The Legend dialog allows specifying the vertical position of the legend within a legend layout frame. The default is center.
Layout frames with text that is too long to fit into a frame trim the legend text to the first words that fit.
The Layers pane filter button allows showing and editing Legend Mode for map layers:
full - always legend the layer.
short - always legend the layer, reducing thematic formatting to a single sample using the main title, such as the name of the layer.
auto full - legend the layer if it is currently visible, that is, not turned off explicitly, not in a folder that is turned off explicitly, and not hidden due to the current scale. This is the default.
auto short - legend the layer if it is currently visible, reducing thematic formatting to a single sample using the main title, such as the name of the layer.
none - never legend the layer.
The Legend dialog allows specifying separate font and text size for different types of legend items:
text - header items for thematic formatting.
single - sample items for non-thematic formatting, which typically appear in a single legend entries.
multiple - sample items for thematic formatting, which typically appear in multiple legend entries.
The Legend dialog allows specifying the text alignment for each type of legend item. Defaults are: text items use right alignment, single items use right alignment, and multiple items use left alignment.
Opening a MAP file created by Manifold 8 converts legend, north arrow, and scale bar layout frames.
Dropping a component into a layout copies the background color of the component into the created layout frame. Creating a virtual layout for a component using File - Print Preview does the same.
Dropping a component into a layout or creating a virtual layout for a component using File - Print Preview copies grid parameters into the created layout frame.
VCREDIST (Visual C++ runtime) has been updated to 14.34.31931.0.
LASZIP has been updated to 3.5.0. This fixes a small memory leak when reading v3 LAZ files.
SQLITE has been updated to 3.40.1. This fixes several obscure bugs and adds some performance optimizations.
Cutting edge build 178.6 is a large build with many new features and improvements for legends, including a dynamic Legends virtual layer for map windows. The build also includes significant improvements in automatic label placement and overlap resolution and improvements in Style. Build 178.6 is the last cutting edge build in the current series focused on the cartography features and the last build for 2022. It will be the basis for the next base build early in 2023.
The Style dialog for labels includes new options for placing labels near points:
label - either one label per coordinate or one label per record. Specifies how many labels to place for points with multiple coordinates (multi-points). The default is one label per coordinate,
move to avoid overlap - if turned on, allows trying multiple label positions in case of overlaps. The default is off.
Placing labels near lines in one per branch or one per record modes tries multiple label positions in case of overlaps. Positions near centers of visible branches are given preference.
In Style pane buttons, dropdown lists for sizes highlight choices that map to whole pixels at 96 DPI. 96 DPI is the default screen resolution. Using sizes that map to whole pixels at that DPI, for example, setting point size to 4.5 points and stroke size to 0.75 points, helps make vector shapes look crisper.
Dropdown lists of sizes have been adjusted to include more choices that map to whole pixels at 96 DPI.
Editing area, line, point, or text style shown in the Style pane using the Style dialog allows applying changes via an Apply button to the edited style without closing the dialog.
Using Tally for unique values in the Style pane keeps formatting for existing values.
Using Tally for numeric intervals in the Style pane copies formatting from closest values. For example, going from two breaks of 10 = red, 20 = green to four breaks of 8, 13, 18, and 23 will produce intervals and styles of 8 = red, 13 = red, 18 = green, 23 = green.
The North arrow and scale bar in a map window may overlap each other and are only auto-hidden when the window is too small to fit them (with margins).
The Layers pane for a map window provides a virtual layer for a Legend, similar to how the Layers pane provides virtual layers for a North arrow and a scale bar. The layer can be turned on or off. The default is off. Double-clicking the Legend layer edits its parameters using the Legend dialog.
Legend parameters:
foreground color - Default is black,
background color -Default is white,
style - Legend style, there currently is only one style. More styles will be added in the future,
stroke - Stroke width for the border. Default is 1 pt,
text font - Font for text. Default is Tahoma,
text size - Size of font for text. Default is 8 pt,
position - Horizontal and vertical position relative to the map window. Default is left bottom,
margin - Horizontal and vertical margin within the map. Default is 8 x 8 pt,
size - Horizontal size. Default is 180 pt. The vertical size varies with the number of items and is computed automatically,
padding - Padding between the border and the legend items. Default is 3 pt.
The Legend dialog includes a preview of the legend style, which updates after changes to the parameter values.
The Legend dialog includes an Apply button which allows applying changes to the legend style without closing the dialog.
Legends ignore images with no thematic formatting.
Legends ignore drawings and labels with no records.
Legends for drawings with a spatial index provided by Manifold ignore formatting for geometry types that do not exist in the drawing.
Legends with too many items render the first items that fit into the legend frame (instead of increasing the frame size offscreen). Legends with no items do not render at all (instead of rendering an empty frame).
Legends no longer have separate items for component names. If a component uses thematic formatting, the component name is shown next to the field name that controls the formatting using Component : Field style. If a component does not use thematic formatting, the component name is shown next to constant formatting using the component name instead of, for example, 'Points'. This makes legends shorter and easier to read.
Layouts support legend frames. To add a legend frame, use the new Create Legend cursor mode. Keyboard shortcut: Shift-E.
Legend layout frames use data from the next component frame below the legend in the display stack, or, if there are no such frames, from the topmost component frame. This is similar to how north arrow and scale bar layout frames derive their context.
The Info pane for a legend layout frame shows the legend style and allows editing it.
The context menu for a legend layout frame in the Layers pane includes a new Split into Samples command which splits the legend into individual frames for formatting samples and adds them to the layout. The new frames are put into a separate folder on the same level as the legend frame, right above it in the display order. The original legend frame is not deleted, but is hidden. The old View - Mode - Create Legend command used to create similar legends from individual frames for formatting samples has been removed.
(Fix) A failed read or write of JPEG format no longer sometimes reports a generic 'can't read / write data' message instead of a specific error.
(Fix) Selecting north arrow and scale bar virtual layers for a map window in the Layers pane and turning them on or off together no longer sometimes copies the style for the north arrow to the scale bar or the reverse.
Cutting edge build 178.5 provides label overlap management between layers in maps, as well as some technical improvements to multi-threaded rendering of layouts.
The filter button in the Layers pane includes a new Overlap Mode command which shows the overlap mode for label layers and allows changing the overlap mode.
Available overlap modes:
layer - labels in the layer may not overlap each other, but may overlap labels from other layers. This is the default mode.
map - labels in the layer may not overlap each other and may not overlap labels from other layers whose overlap mode is also set to map. Labels from lower layers are given priority over labels from higher layers.
Overlap modes work in map windows, in layout windows, and during printing.
Using map overlap mode only makes sense for multiple label layers. Editing a label layer set to map overlap or turning that layer on or off will re-render other label layers set to map overlap above it. Rendering label layers set to map overlap decreases parallelism as such layers must be rendered in sequence. However, in practice, this does not necessarily decrease rendering performance, as the final number of labels to render also tends to decreases.
Layout windows limit the number of rendering threads for a map frame. This significantly improves performance when rendering maps that contains many layers: instead of trying to render all layers at the same time, the window now starts rendering a few layers and only starts rendering new layers after previous layers finish rendering. This rendering plan was originally introduced for map windows, so now layout windows use the same strategy.
Layout windows limit the number of rendering threads across multiple frames. This helps render layouts with multiple maps.
Layout windows dynamically increase the number of rendering threads if the window becomes active and decrease the number of threads if the window becomes inactive. Map windows also dynamically increase/decrease rendering threads the same way.
The dynamic adjustment of rendering threads has been tuned to work more smoothly for systems with many CPUs.
(Fix) Reading TIFF files with data stored in strips with heights that are not powers of 2 no longer sometimes fails.
(Fix) Performing an EXECUTE [[ ... ]] on a SQL Server data source with a command that returns multiple results no longer returns the number of affected rows instead of the first result table.
Cutting edge build 178.4 provides a variety of conveniences and user interface improvements for grids, North arrows, scale bars, Style, and layouts. It also provides new query functions to work with interactive selections from SQL, a command line option to help manage serial numbers, and important updates to third party software used by Manifold, notably a migration to the latest Visual C++ environment from Microsoft.
Preview controls for styles for areas, lines, points, and text use a single preview background color, which persists per Manifold session. The preview color can be changed using the color picker box in the right top corner of the preview panel. This makes it easier to set a desired background color just once without having to set it for each use of style. Previously each control started with the background color of the map which could then be changed interactively, with the changes only persisting until the dialog was closed.
The default preview background color has been set to light blue. The new default makes it easier to see Style effects using white color.
Preview controls for Style dialogs for grids, north arrows, and scale bars use the new preview background color. The preview color can be changed using the color picker box in the right top corner of the preview panel.
The North Arrow and Scale Bar dialogs use a better caption for position options: the Align caption has been renamed to Position.
The Scale Bar dialog allows specifying the desired scale bar length in scale bar units. If the space allocated for the scale bar is too small to show the desired length, the length is computed automatically from the visual size of the scale bar, rounding down to the first two significant digits (same as in the previous builds). The default is to always compute the length automatically.
The Grid, North Arrow, and Scale Bar dialogs include a new Apply button which allows applying changes to the map without closing the dialog, which makes interactive tinkering fast and easy. The button is currently only shown when the dialog is used to edit the style of a virtual layer in the Layers pane for a map. In the future, the Apply button might also be used when the dialog is used to edit the style of a layout frame in the Info pane.
Snapping to a grid in a map window now snaps to the virtual grid layer. Snapping does not require the virtual grid layer to be turned on. Snapping to a grid always uses the base grid step for the virtual grid layer. The Snap Parameters dialog no longer contains means to specify the grid step and grid unit, as there is no longer a separate grid used for snapping.
The Layers pane has new filters for layout frames: North Arrows, Scale Bars, and Samples. The Text Frames filter has been renamed to Text.
When showing data for a layout frame, the Info Pane's Update Record button has been renamed to Update Frame.
The Create Area Sample command for layouts has a new icon.
The Create North Arrow and Create Scale Bar commands for layouts have new icons. The cursor modes for those commands have new cursors as well.
The manifold.exe executable supports a new -showserial command line option:
manifold -showserial
That launches a Manifold session that in the Log window pane shows the first 8 characters of the serial number. The first characters of the serial number are not written into the log file and are only shown in the Log window. This option allows administrators to see what serial number has been used on a given machine. The new command line option works for all editions of Manifold, SQL for ArcGIS Pro, and Release 8, but not for Manifold Viewer (which does not use serial numbers).
The Microsoft Visual C++ (also called MSVC or simply VC) environment used has been updated to 14.34.31933. The VC runtime has been updated to 14.32.31332.0. This adds many minor performance improvements and makes Manifold modules use the same version of the runtime as the latest modules from Microsoft and other major vendors, saving resources that would otherwise be spent on keeping multiple versions of the runtime in memory.
NVIDIA CUDA has been updated to 11.8. GPGPU modules for sm_50 and sm_60 architectures now use slightly fewer GPGPU resources.
The ECW SDK has been updated to 5.5.0.2268. This fixes a couple of minor SDK bugs related to the reading of ECW and JPEG2K images.
Google LIBWEBP has been updated to 1.2.4. This improves the visual quality of compressed WEBP images and fixes several Google bugs related to transparency, etc.
SQLITE has been updated to 3.39.4. This adds support for RIGHT and FULL outer joins (SQLITE did not have them until 3.39) as well as several minor performance improvements. This also fixes a number of SQLite bugs, most notably with join optimizations and with the UPSERT statement.
New query function: SelectionReplace. Takes a table to select records in, a table with keys to select and a flag whether records with the passed keys should be selected or unselected. The keys table should contain fields from the result table of the SelectionKeys function, with compatible types. The keys table may contain other fields as well, which will be ignored if present. The returned value for SelectionReplace is the number of unique keys fetched from the keys table. Since some of the keys might not exist in the table, replacing the selection using this function marks the selection as not knowing the exact number of selected records. To determine the exact number of selected records, click the ? button in the Info pane.
New query function: SelectionReplaceWindow. Same as SelectionReplace, but also takes a window name and a layer name, which allows this function to replace the selection for a dynamic layer based on a query.
In addition to manipulating selections in tables that have been opened, the SelectionReplace and SelectionReplaceWindow query functions will automatically create a selection even within a table that has never been opened as long as it is static (that is, not the result table of a query) and it belongs to a database opened in the Manifold user interface or one of its child databases (that is, not to an arbitrary database opened by a script).
To use the following examples, create a new .map file, open the mfd_meta table, open a new command window, then dock the new command window below the mfd_meta table (right-click the tab for the command window and select Dock Down) so that you can run the example queries and immediately see what they do.
--SQL9
-- example 1
VALUE @a FLOAT64 =
SelectionReplace(mfd_meta, (VALUES (2) AS (mfd_id)), TRUE);
-- example 2
VALUE @a FLOAT64 =
SelectionReplace(mfd_meta, (VALUES (2), (3) AS (mfd_id)), TRUE);
-- example 3
VALUE @a FLOAT64 =
SelectionReplace(mfd_meta, (VALUES (2), (3) AS (mfd_id)), FALSE);
-- example 4
VALUE @a FLOAT64 =
SelectionReplace(mfd_meta, (SELECT * FROM mfd_meta WHERE name='mfd_root'), TRUE);
(Fix) The automatically computed length of a scale bar in a layout frame may no longer round to different values depending on the zoom or print resolution (was caused by incorrect handling of rounding errors).
(Fix) A lattitude / longitude grid for a component in a non-lattitude/longitude coordinate system no longer sometimes uses the wrong unit for the grid step if the grid unit is set to auto.
(Fix) Using the Selection query function to return unselected records in a table that was never opened in a window now correctly returns a full table instead of an empty table.
(Fix) Using the SelectionWindow query function to return unselected records for a table in a window or layer that no longer exists now correctly returns a full table instead of an empty table.
Cutting edge build 178.3 extends North arrow and scale bar capability to layouts. North arrow and scale bar frames can now be added to layouts, with automatic context for those frames taken from component frames in the layout. Grids have also been extended with more options. North arrows, scale bars, and grids have been integrated into the Info pane.
Layouts support North arrow frames. To add a North arrow frame, use the new Create North Arrow cursor mode (keyboard shortcut: Shift-N).
Layouts support scale bar frames. To add a scale bar frame, use the new Create Scale Bar cursor mode (keyboard shortcut: Shift-S).
North arrow frames and scale bar layout frames use data from the topmost component frame below the frame in the display stack, or, if there are no such frames, from the topmost component frame in the layout.
A North arrow layout frame with auto bearing turned on checks if the center of the north arrow shape is inside the component frame used by the North arrow. If the center of the North arrow shape is inside the component frame, the bearing is computed for the geographic location at that center. Otherwise, the bearing is computed for the geographic center of the component frame.
Grids are available for component frames that appear in layouts. Alt-click the component frame, and in the Info pane's Style tab a Grid check box will allow turning on a grid. A preview panel in the Info pane shows the grid style and allows editing it. Click the [...] browse button to launch the Grid dialog to configure the grid. Note: In the current build, if the grid step is allowed to change, the layout window might use different grid steps at different zoom levels. Printing the layout, however, will always use the same step. In future builds, the layout window will also set the grid step to whatever step will be used during printing.
If a map has a grid turned on for the map, dropping that map into a layout as a component frame will not show that same grid in the layout. Instead, Alt-click the component frame, and in the Info pane's Style tab a Grid check box will allow turning on a grid. Using a different grid in layout makes it possible to show a grid that is styled and configured to look good in the layout for printing, while a different grid can be used in the map for interactive work on the computer desktop.
The Grid dialog allows specifying grid step by X and Y. The default is a single step in X and Y using whatever is the coordinate system unit for the map. Grid cells never become smaller than the specified step.
The Grid dialog allows specifying whether the grid step is increased during rendering. The default is on, to allow increasing the grid step. If the option is turned off, the grid step is locked to the specified step and grid cells never become larger than the specified step. If the visible size of grid cells becomes too small the grid is not rendered. The spacing parameter has been removed and has been replaced with internal logic to automatically manage spacing.
Rendering a grid for a latitude / longitude system restricts parallels to the -90 to 90 range. If the Use latitude / longitude option is turned on, rendering a grid also restricts meridians to the -180 to 180 range, otherwise meridians are not restricted.
The Grid dialog allows specifying the unit of measure for the grid step. The default is to use whatever is the unit of measure for the coordinate system in use.
The Layers pane shows icons for grid, north arrow, and scale bar layers in a map.
The Info pane for a North arrow layout frame or a scale bar layout frame shows the North arrow or scale bar style and allows editing it.
Cutting edge build 177.8_2 introduces virtual layers that provide North arrows, scale bars, and grids in maps. See the First Look - North Arrows video for a quick introduction.
The Layers pane for a map window shows virtual layers for a north arrow, scale bar and grid. Virtual layers can be turned on and off (by default they are turned off). The position of the virtual layers in the display stack cannot be changed. Double-clicking a virtual layer edits its parameters using a dialog.
The state and parameters of the virtual layers are saved in the map component. If the window is not a map component but a drawing, image, or labels window, the virtual layers still work, but changes to their state and parameters are lost when the window is closed. To save these changes, save the window as a map using Edit - Save as Map.
Dialogs that edit parameters for north arrows, scale bars, and grids include a preview that immediately updates after any changes to parameter values.
Configure a North arrow by double-clicking the North Arrow virtual layer. The North Arrow dialog allows setting the following parameters:
(Foreground color) - Foreground and stroke color for shapes and text. Default: black.
(Background color) - Fill color and the color for halos. Some north arrow shapes may use colors that are mixes of foreground and background colors. Default: white.
(Shapes gallery) - Choose a North arrow shape from a gallery of popular shapes. Default: compass. Hovering the mouse over a shape will show the name of that shape in a tooltip.
Stroke - Stroke width for the shape. Default: 1 pt.
Text - Check to show text (captions) near north arrow shapes for NESW directions. Some shapes can show captions in all four directions, some only two directions or one direction. Default: off (do not show text).
NESW - Four boxes for text to use for each of the four directions. Default: empty strings.
(Text font) - Font for text. Fully configurable for typeface, effects, bold, etc. Default: Tahoma.
(Text size) - Font size for text. Default: 8 pt.
Align - Horizontal and vertical position of the North arrow shape relative to the map. Default: right top.
Margin - Horizontal and vertical margins between the North arrow shape and the map's edges. Default: 8 pt margins.
Size - Size of North arrow shape. Default: 96 pt.
Halo - Width of background color halo for shape and text. Default: 1pt. Setting Halo to 0 turns off the halo.
Bearing - The direction of the North arrow, if the Auto box is not checked. Default: 0 (North up).
Auto - When checked (the default), compute the North arrow's bearing automatically. Bearings are first computed using the center of the north arrow shape, and then, if that fails because the north arrow is outside of the coordinate system domain, based on the center of the map.
Configure a scale bar by double-clicking the Scale Bar virtual layer. The Scale Bar dialog allows setting the following parameters:
(Foreground color) - Foreground and stroke color for shapes and text. Default: black.
(Background color) - Fill color. Default: white.
(Shapes gallery) - Choose a scale bar shape from a gallery of popular shapes. Default: rect. Hovering the mouse over a shape will show the name of that shape in a tooltip.
Stroke - Stroke width for the shape. Default: 1 pt.
Text - Check to show text (a caption) with the number of units. Default: on (show text).
(Text font) - Font for text. Fully configurable for typeface, effects, bold, etc. Default: Tahoma.
(Text size) - Font size for text. Default: 8 pt.
Align - Horizontal and vertical position of the scale bar shape relative to the map. Default: right bottom.
Margin - Horizontal and vertical margins between the scale bar shape and the map's edges. Default: 8 pt margins.
Size - Approximate desired size of the scale bar shape in horizontal and vertical dimensions. The actual size is computed based on the units and current scale. Default: 144 pt x 8 pt.
Halo - Width of halo for shape and text. Default: 1pt. Setting Halo to 0 turns off the halo.
Unit - Unit of measure shown in the text caption. Default: coordinate system unit for the map window.
The length of the scale bar is computed automatically based on the specified size, unit and the current scale. The width of the scale bar is slightly reduced so that the reported length is rounded to no more than two significant digits.
Configure a grid by double-clicking the Grid virtual layer. The Grid dialog allows setting the following parameters:
(Foreground color) - Stroke color for grid styles. Default: black.
(Background color) - Fill color. Default: white.
(Styles gallery) - Choose a grid style from the gallery. Default: line. Hovering the mouse over a style will show the name of that style in a tooltip.
Dashes - Dash pattern for lines, using the same conventions as the Dashes parameter for line styles. Default: 3,3 for a 3pt dash and 3 pt space.
Stroke - Stroke width for line and point styles. Default: 0.75 pt.
Point size - Size for point styles. Default: 5 pt,
Halo - Width in points of halo for line and point styles. Default: 0. Setting Halo to 0 turns off the halo.
Spacing - Minimum size of a grid cell. The actual size is computed based on the scale of the map and the units used (either native map units or latitude / longitude). Default: 72 pt. The minimum allowed value is 36 pt.
Use latitude / longitude - Check to create a latitude / longitude grid. Default: off, meaning to use the projection of the map.
The grid step is selected automatically based on the specified spacing and the current scale. The grid step in the units used (either the projection of the map or latitude / longitude) is rounded up to the closest power of 10 multiplied by 1, 2 or 5, resulting in grid steps such as 100, 200, 500, 1000, and so on.
Connections to SQL Server try using specific versions of the MSOLEDBSQL driver if the generic version fails to start due to being misconfigured.
Renamed in the Scales dialog: closest scale to Minimum scale, farthest scale to Maximum scale. The new names help prompt which of the two numbers should be greater.
Exporting a data source to a .map file or to an .mxb file writes tables with an invalid schema as tables with a single default field and no records. Tables with invalid schemas can occur when linking from data sources where a table schema cannot be read due to insufficient permissions for the user role used.
(Fix) Going back and forward between views in a map window no longer fails to update the scale readout in the status bar.
(Fix) The table window showing a table from a MAP file no longer suggests unnecessary refreshes to data after changes to values in non-key fields. Tables from MAP files show such changes immediately without any need to refresh.
Cutting edge build 177.8 provides a variety of user interface conveniences, including the ability to Copy from tables and to Paste into third-party applications like Excel and Notepad.
There is a new style for text fields: c = command line. The table window and the Info pane show command line text values in blue highlight color and allow running them using the context menu. This allows putting Windows command lines into a field, which can then be executed by right-clicking a cell and choosing Run Command.
Refreshing data in the table window either manually or automatically, for example, after deleting records or changing the table schema, keeps filters and orders for fields that did not change.
The table window sorts fields by name in the View - Order submenu.
The table window showing records for a table from a data source other than MAP marks records that have been deleted in that window as deleted. Previously, deleting records from such a table only suggested refreshing data using the bar at the bottom of the window. Now the window also shows records that it tried to delete.
The status bar includes a new Counts pane (a section in the status bar) that shows the total number of records and the number of selected records in the active component. This shows the number of selected records at all times without needing to switch to the Info pane.
The record counts shown in the Counts status bar pane can be refreshed by right-clicking the section and choosing Refresh in the context menu. That is useful when working with data sources that are stored outside the .map project where the record count might change independently of any work done in the Manifold session.
The Info pane and the Counts status bar pane show the total number of frames and the number of selected frames for a layout.
Copying records from a table window now copies them to the Windows clipboard in both binary format suitable for pasting back into Manifold and in text format suitable for pasting into third-party applications like Excel or Notepad. Copy records by selecting the desired records and then choosing the Edit-Copy menu command. Copying records in text format copies only visible fields. It copies fields in the order they appear in the table window, and formats field values according to field styles. The visible order of records is not preserved. The amount of copied text data is limited to 32 million characters because many third-party applications can only handle limited amounts of text coming from the clipboard. If there are more records than fit into the limit, extra records past the limit do not appear in text data and only appear in binary data (for pasting into Manifold).
The context menu on column heads (right-click on a column head) now includes a Copy command which copies field values in that column from selected records to the clipboard. Copying field values copies them in text format only. The copied values are formatted according to the field style. Empty values are omitted.
(Fix) The Edit - Select Inverse command for a layout window adjusts the final selection for groups in the Layers pane. (Previously the selection could produce a selected group with an unselected child.)
(Fix) Selecting frames using the mouse cursor in a layout window adjusts the final selection for groups in the Layers pane. (Previously the selection could produce a selected group with an unselected child.)
Build 177.8 is a new Base build, incorporating all improvements in the 9.0.177.x series of builds. It is recommended for all users. The main changes since the prior cutting edge build, 9.0.177.4, are a new system for specifying zoom ranges in maps and also for rendering to a fixed scale. Base builds starting with 9.0.178 for Release 9 and 8.0.34 for Release 8 are now exclusively 64-bit.
Layers pane readouts for opacity, min / max scale,, and field width have been aligned left. The color picker sample box for the virtual background layer has been made wider to be similar to color pickers in the Style pane.
A new, single Min / Max Scale mode readout in the Layers pane replaces the former Closest (Min) Scale and Farthest (Max) Scale modes for map layers. The new Min / Max Scale mode readout shows both scale settings together, to make it easier to compare zoom scales between different layers.
Editing zoom scales is now done using a new Scales dialog: double-click into a Min / Max Scale cell in the Layers pane to launch the dialog. The dialog shows the current scale for reference and allows setting the closes, minimum scale, the farthest, maximum scale, and also a new Render scale value. The dialog allows direct entry of any scale value and also provides a pull-down list that includes even scale values in factors of ten (1, 10, 100, 1000, 10000, etc.) as well as uneven scales used as limits in other layers. That makes it easy to match scales used in other layers.
The Scales dialog shows scales with thousand separators for readability, and accepts scales entered with thousand separators by the user.
Applying scale limits to a layer keeps the layer visible at a zoom scale exactly equal to the closest scale, but hides it at a zoom scale exactly equal to the farthest scale. This allows showing exactly one layer out of a sequence of layers with matching scale boundaries, for example, +10k, 10k-50k, 50k+.
The Layers pane using the Scales dialog allows specifying a Render scale value for map layers that show vector data (drawings and labels). Render scale is applied during rendering in windows and during printing. When a render scale value is specified for a layer, the vector objects and labels in that layer are rendered as if the scale were fixed at that value, so zooming in will make the labels and objects bigger, while zooming out will make them smaller.
Rendering labels and objects with render scale will not render anything if the specified render scale is too small (10 times smaller than the current scale) or too big (100 times bigger than the current scale).
Layout windows render drawings and labels at true scale: zooming in and out will change the visual size of drawing objects and labels. Label placement algorithms may still produce slightly different results on screen vs. on page, or for different zooms on screen, due to different tolerances.
Editing a text style for Labels allows specifying whether the system should prefer placing labels following a line with left / right offset above or below the line. The default is above.
Editing a text style for Labels allows specifying a smooth tolerance factor to be applied to a line for labels following lines. The default is 50% of the font height.
(Fix) Labels following lines no longer sometimes apply bend spacing a little too early.
(Fix) Applying bend spacing to labels following nearly straight lines no longer sometimes (rarely) packs label glyphs into the same location.
(Fix) Placing labels following a line with left / right offset in repeat mode always tries to place labels along all left or all right parts of a branch, instead of stopping after successfully placing labels along one such part.
Build 177.4 provides many controls for line labels and label overlap spacing. It also adds an export control to Manifold Commander and two fixes.
See the updated Style: Labels topic for examples of new label style controls in action.
Editing a text style allows specifying how many labels to place onto a line:
one per branch - (default) The system will try to place a single label for each visible branch of each record. This applies to visible branches, that is, the visible parts of branches, not physical branches. For example, if a physical branch goes off screen and then returns back on screen, it will produce multiple visible parts and the system will try to put a label onto each such part.
one per record - The system will try to place a single label for each record.
repeat - the system will try to place as many labels as will fit into each visible branch of each record given the specified spacing between labels on the same branch. Spacing can be specified either in absolute units, for example, 40, meaning 40 points, or in relative units for example, 800%, the default, meaning 8 times the font size.
Editing a text style allows specifying how to place labels onto a line:
curve - (default) Bend label text to follow the shape of the line.
straight - Place labels without any bends in the direction of the line in the region of the center of the text.
straight horizontal - Place labels without any bends. aligned horizontally.
straight perpendicular - Place labels without any bends aligned perpendicular to the direction of the line in the region of the center of the text.
Placing labels onto lines preprocesses the line metric to make it smoother.
Placing labels onto lines in one per branch or one per record modes prefers positions near the visual center of the line.
Placing labels onto lines produces a tighter and more accurate overlap shape. This improves the accuracy of clicking into labels. This also allows showing more labels in that labels that are close to each other start conflicting later than they did with less accurate overlap shapes.
Editing a text style allows specifying overlap spacing, which controls how close individual labels can be to each other. The default is 1 pt.
Editing a text style allows specifying bend spacing to apply to labels following lines. With bend spacing, placing text onto a line adds small amounts of space between individual letters at each bend to improve readability. The amount of added space depends on the bend angle: sharper turns get more space. Bend spacing defines the maximum amount of space to add to a single bend. The default is 50%, adding up to half of the font size at each bend.
There is a new -export:xxx command for Commander. The command exports the specified component in the input file to the output file. The specified component can be in a nested data source. When using this command the input file is always opened in read-only mode and the value of the -open:xxx option is ignored. For example:
manifoldcmd "c:\world.map" -export:"Main Map" -out:"c:\world_map.gdb"
Sample positions and shapes for layout frames have been localized.
Reading MIF format allows duplicate field names and automatically renames duplicates to make all field names unique, thus automatically dealing with problems that can happen from encoding issues in MIF format files.
(Fix) Writing MIF removes unsupported characters from field names.
(Fix) Clicking a label no longer sometimes picks a wrong label or does nothing.
Build 177.3 adds new capabilities to Manifold Commander, improves labels along lines, provides a vertical text alignment style in layouts, adds new dataports, and provides fixes.
The -runscriptfile:xxx command for Commander works in the context of a data file. The command will open the specified data file and then run the script from the specified script file.
There is a new -runqueryfile:xxx command for Commander. The command is similar to -queryrun:xxx, but the text of the query is taken from the specified file. That allows writing new queries to do workflow using SQL without opening Manifold interactively, including parametrized queries.
There is a new -runquerytext:xxx command for Commander. The command is similar to -queryrun:xxx, but the text of the query is taken directly from the command line. For example:
manifoldcmd data.map -runquerytext:"SELECT * FROM blocks WHERE state='CA'" -out:out.jsonl
Editing a text style allows specifying an offset number for labels following lines. The default is 0 for placing a label for each visible branch directly on top of the line. If the offset is not 0, the system will try to place a label for each visible branch on each side of the line until it succeeds on one side or the other. For example, if the system first succeeds in placing a label on the left side of a particular branch, it will not try to place a label on the right side of that branch.
Editing a text style allows specifying the vertical alignment of the text. Vertical alignment is used when rendering text in layout frames.
Many progress messages have been simplified from mmm (xxx) format to mmm: xxx format for readability and consistency.
Printing geometry to GeoJSON no longer prints optional bounding box data. This reduces output size, particularly for points. In the future, if there emerges a need to include bounding box data, an explicit option will be added to do so.
There is a new dataport for JSONL files. JSONL files are line-delimited JSON, a simple format designed to be friendly to streaming data. A JSONL file contains a single table. Reading a JSONL file parses it dynamically. Each line must be 2 GB or less of text, but the file itself can be any size.
There is a new option to export tables to JSONL files. Binary fields are ignored. The export tracks progress and can be canceled.
There is a new dataport for GEOJSONL files. GEOJSONL files are line-delimited JSON, similar to JSONL, but with GeoJSON data. A GEOJSONL file contains a single drawing. Reading a GEOJSONL file parses it dynamically. Each line must be 2 GB or less of text, but the file itself can be any size. The Geom and Geom_x field names are reserved: properties with these names are ignored.
There is a new option to export drawings to GEOJSONL files. Binary fields except for the geometry field used by the drawing are ignored. Exported geometry is automatically converted to Latitude Longitude WGS84, curves are linearized, and areas are normalized to OGC rules. The export tracks progress and can be canceled.
(Fix) Table windows for a tables, queries, and command windows, for .map files served by Manifold Server fetch and show the number of records specified in the Initial number of records to show (non-MAP) option in the Tools - Options dialog, instead of automatically showing all records.
(Fix) Table windows for tables stored in the .map project no longer autorefresh data after changes to a big table if there is a filter applied and the number of records that pass the filter is small.
(Fix) Writing a TIFF file with FLOAT64 values correctly writes missing pixels as NaN.
(Fix) Writing a TIFF file with INT8U values and a palette no longer fails to write the palette.
(Fix) Writing a JSON file escapes component and field names.
(Fix) Writing a GEOJSON file escapes component and field names.
(Fix) Writing a DBF file no longer sometimes miswrites boolean values.
(Fix) Tracking progress when writing DBF or SHP files no longer sometimes counts every written record twice.
Build 177.2 is a historic build in that it drops support for 32-bit versions of Manifold projects. All Manifold products are now 64-bit and require 64-bit Windows operating systems.
Build 177.2 expands the display viewport size in table windows to 2 billion records. That is absurdly larger than humans can comprehend, but together with Manifold speed plus reasonable safety measures now built in, expanding the number of records that can be interactively displayed in a table window greatly expands the reach of interactive procedures to very large tables.
Build 177.2 supports ArcGIS 3.x for SQL for ArcGIS Pro, and also adds features to various dataports and provides bug fixes.
Release 8, Release 9, Manifold Viewer, and SQL for ArcGIS Pro no longer include 32-bit modules. Users who require 32-bit versions should use build 9.0.177 for Release 9 and build 8.0.33 for Release8, which are base builds that include 32-bit modules and which never expire.
Manifold 9 products no longer use the Bin or Bin64 folders. DLL and EXE modules are placed directly within the install folder. The Shared folder has been renamed to Extras.
SQL for ArcGIS Pro no longer includesan Open SQL 32-bit button. The 'Open SQL 64-bit' button is renamed to 'Open SQL'.
Discussion
Manifold was one of the first companies to ship 64-bit products. Despite being one of the first to move to 64-bit, for over 20 years Manifold has maintained both 64-bit and 32-bit versions of all Manifold products.
Over time most, if not all, Manifold users naturally moved to primarily use the 64-bit versions of the products. At the same time, maintining obsolete 32-bit code has become a limiting factor for adding new features. 32-bit only limits often impact new, memory-intensive features, for example, when field statistics are computed for the Style pane, limits must be applied so that 32-bit code does not run out of memory. For some time, features have not been available in 32-bit mode due to lack of support from third parties. For example, CUDA is 64-bit only, and ArcGIS Pro for which Manifold provides an add-in is 64-bit only as well. Outdated versions of clients for database must be used because they are the last available versions to support 32-bit mode.
In 2022, the limits and costs of continuing to provide 32-bit versions is no longer worth the decreasing value of that compatibility. Manifold is therefore going exclusively 64-bit from build 177.2 onward. That will make it more efficient to add new features, and will remove limits on how those new features must be designed. That will make many things simpler for new users. The first example of such a new feature is providing 2 billion record displays in table windows, which 64-bit operation makes possible. Going exclusively 64-bit also reduces the size of the Manifold download to less than 31 megabytes.
There are several areas users may be relying on 32-bit code directly or indirectly:
If you are currently relying on 32-bit code in any way, you don't have to switch immediately. The move to 64-bit only can be gradual. You can use the new 64-bit only builds of Release 8 and Release 9 as portable installs in parallel with the last base builds that still include 32-bit code. Eventually, you may be able to transition to using 64-bit builds all of the time, with only occasional use of legacy portable installs that have 32-bit code.
The maximum number of records that can be shown in a table window is now 2 billion.
The table window can either show all records in a table (up to 2 billion) or show a first sample of records. Showing a first sample of records instead of all records is useful when working with external data sources like PostgreSQL, or, say, Manifold Server: not fetching all records immediately protects against wasting server resources and network traffic needlessly. Tables from .map files show all records by default, while tables from other data sources show a first sample of records. This also applies to the result tables of queries, including ad-hoc queries in command windows. The "show a sample from non-.map data sources" is not absolutely bulletproof. For example, a query stored in a .map file may access records in a table stored on SQL Server. When the user runs such a query, the table window will try to fetch all records in the result table, because the query itself is stored in a .map file, even though the records come from SQL Server. In cases like this, the responsibility for not overloading the server for no good reason is on whoever writes or runs the query.
The Tools -Options dialog includes an Initial number of records to show (non-.map) option to specify the size of a first sample of records fetched for non-.map data sources. The available values are: auto (default), all, or a specific number of records. The auto value is currently set to 5000 records. The previous default was as high as 50000 records because it applied to tables in .map files as well. 5000 records is much friendlier to database servers. If 5000 records is too low for the data sources you are working with, you can always change the option to a higher value in the Options dialog.
The Info pane shows the number of fetched records in the table window. If there are more records available, the Info pane also shows a (+) notation. If the table window is showing a first sample of records instead of all records, the Info pane also shows a button to fetch all records. Once a table window has been told to fetch all records for the displayed table, it will keep fetching all records for all further operations that refresh or replace the table.
The Info pane shows the number of displayed records in the table window. That number will be the same as the number of fetched records that pass any filter.
The Info pane shows the Component tab for a command window with a table.
The table window detects changes to displayed data from a non-.map data source and shows an action bar below the record list with a prompt to refresh data manually. To refresh data, use the View - Refresh Data command or click the prompt in the action bar.
A table window displaying data from a .map file does not automatically refresh data after changes to the data if the number of records in the table exceeds 64k. Instead, the window shows an action bar with a prompt to refresh data manually. For example, if a drawing's table is open and the table shows 100000 records, and in the drawing some of the object are deleted, the table will not automatically refresh. Instead it will show an action bar with a prompt to refresh.
A table window showing over 64k records detects deleted records and shows them in gray background color, with an X icon on the record handle. For example, deleting objects in a drawing window that has 100000 objects when the drawing's table is open will show empty gray rows in the table for the deleted objects. Pressing the refresh prompt in the action bar will refresh the table so those empty rows will be deleted. This provides fast means to delete objects in maps without fully refreshing tables, while still showing which records have been deleted.
The context menu for the fill record in the table window no longer includes a Stop Reading Records command.
The table window always applies filters only to the fetched records. The Filter Fetched Records Only command has been removed. Now that table windows can show up to 2 billion records, always using only the fetched records is no longer a limitation in the vast majority of cases. Tables bigger than 2 billion records can still be filtered using a query composed with the Filter using Query command.
Table windows always apply sort order only to the fetched records. If the table window shows a first sample of records instead of all records and you want to sort the entire table, fetch all records first. The maximum allowed size of field values used for sort ordering is 16 GB.
Watch the new 10 Minute Tutorial - Very Big Table Windows video to see 1.295 million records displayed in a table window. Fast!
SQL for ArcGIS Pro tries to locate manifold.exe using a dialog if it cannot be found automatically. The location of manifold.exe is then saved in an environment variable for future reference.
SQL for ArcGIS Pro supports ArcGIS 3.x. The install packages include two versions of the add-in: a version for ArcGIS 2.x in the ArcGis2 folder and a version for 3.x in the ArcGis3 folder. The EXE and MSI install packages detect the installed version of ArcGIS and register the corresponding version of the add-in.
Failed attempts to connect to a data source are recorded. Subsequent attempts to connect to the same data source are throttled. The delay starts at 1 second and gradually increases to 15 seconds. Errors for failed attempts to connect to a data source are logged with repeat errors (same error message) omitted.
Exporting ECW or JPEG2K tracks progress and can be canceled.
Exporting ECW or JPEG2K aligns reads to tile boundaries for better performance.
Exporting ECW or JPEG2K automatically adjusts internal buffer sizes when exporting very big images for performance (and, in very rare cases, to allow the export to succeed where it previously was failing due to exceeding one of the internal limits on temporary data).
Reading XYZ with a non-space delimiter allows whitespace both before and after the delimiter.
Reading XYZ with integer values that do not fit into INT32 converts all values to FLOAT64.
Reading XYZ skips NaN values. Records with such values have no business being in the file, but they might technically be there and some software packages apparently produce files with them.
Reading XYZ tracks progress and can be canceled.
Reading XYZ supports files bigger than 4 GB.
Reading XYZ protects from tiny drift in XY values to handle regular grids more reliably.
Reading XYZ / NC performs significantly faster, 4-5 times faster, and even faster with big files.
Writing XYZ performs significantly faster, 5-6 times faster, and even faster with big files.
(Fix) Exporting the result of a query with computed fields to MDB, SQLITE, or similar formats no longer fails.
(Fix) Printing geometry to WKT prints points with Z values as POINT Z or MULTIPOINT Z instead of POINT orMULTIPOINT.
(Fix) Printing geometry to WKT no longer ignores Z values for areas.
(Fix) Reading FLT with a malformed HDR may no longer erroneously set pixel scales to zero.
(Fix) Attempting to use components from a data source that fails to connect, for example, because it is referencing a file that does not exist, no longer disrupt the user interface, such as map windows, layout windows, or panes.
(Fix) Reading XYZ no longer ignores the first record if the file starts with UTF8 BOM.
(Fix) Reading NC creates an RTREE index on tiles.
(Fix) Writing XYZ prints all values with full precision. Previously, values that were very small or very large could lose some of the digits.
Build 177.1 introduces in Universal and Server editions a new version of Manifold: Manifold Commander is a console application version of Manifold that provides the full range of Manifold query and scripting capabilities from a command line, making it easy to automate almost any GIS, DBMS, or ETL job.
Commander is provided as a manifoldcmd.exe executable file within the Manifold installation hierarchy. The manifoldcmd.exe executable file launches from a command line such as in a Windows Command Prompt window, or in a .bat batch file, or from a Windows PowerShell script and so on. Commander can run either queries or scripts from within a specified .map file, and Commander also can run scripts from a specified script file stored outside a Manifold project. When Commander is used in conjunction with Windows Task Scheduler, you can launch command lines to have Manifold automatically do work on a regular schedule, such as updating a CSV file every night that is served via a CSV server.
Build 177.1 also extends interactive selection capabilities, makes it easy to switch licenses, and provides bug fixes as well as improvements in the MapInfo MIF dataport.
Install packages for Manifold Release 9 (not Manifold Viewer and not SQL for ArcGIS Pro) include a new executable: manifoldcmd.exe, called Manifold Commander in these release notes. Commander is a console application that can be launched from a command line to automate common tasks.
Commander is limited to Universal and Server editions. When launched on a machine activated with Professional edition, nothing happens.
Commander log files use a -cmd postfix, to distinguish them from log files for other Manifold applications.
Commander can run queries within .map files, and can run scripts either within .map files or within script files that contain the desired script text.
Syntax:
manifoldcmd <command> [<options>] [<file>]
Commands:
-runquery:xxx - Open the file and run the specified query component. The component can be in a nested data source.
Supported options: -logfilter:xxx, -logfolder:xxx, -open:xxx, -out:xxx
-runscript:xxx - Open the file and run the specified script component. The component can be in a nested data source.
Supported options: -logfilter:xxx, -logfolder:xxx, and -open:xxx
-runscriptfile:xxx - Run the specified script file.
Supported options: -logfilter:xxx and -logfolder:xxx
Options:
-logfilter:min - Skip dates and prefixes when logging to console
-logfilter:minscript - Skip dates and prefixes, skip non-error non-script messages when logging to console, useful to limit output for further processing with command-line tools
-logfilter:none - Log full data to console (default)
-logfolder:xxx - Folder for log files
-open:readonly - Open file in read-only mode (default)
-open:readwrite - Open file in read-write mode
-open:readwritesave - Open file in read-write mode, save file after operation completes, this includes saving all nested data sources
-out:xxx - Output file, for example, a CSV or JSON. The type is determined by the file extension, .csv or .json, used.
Examples:
A .map project called Mexico.map contains a query called MexQuery that from a linked table generates a result table of all provinces in Mexico with a population greater than 3000000. To run the query and save the result to a CSV file called Mex.csv:
manifoldcmd -runquery:MexQuery -out:"C:\Projects\Mex.csv" "C:\Projects\Mexico.map"
Save the result to a SQLite file called Mex.sqlite:
manifoldcmd -runquery:MexQuery -out:"C:\Projects\Mex.sqlite" "C:\Projects\Mexico.map"
Save the result to a Manifold .map file called Mex.map:
manifoldcmd -runquery:MexQuery -out:"C:\Projects\Mex.map" "C:\Projects\Mexico.map"
Save the result to an Esri file geodatabase within a folder hierarchy C:\Projects\Mex\gdb.gdb\ and create the Mex and gdb.gdb sub-folders:
manifoldcmd -runquery:MexQuery -out:"C:\Projects\Mex\gdb" "C:\Projects\Mexico.map"
Double-quotes around arguments like the names of files are not necessary if there are no spaces in the argument. If the output file already exists, the above will overwrite it with the new version.
The Info pane shows the number of selected records in a table window (this includes table windows opened for query components, but not command windows for ad-hoc queries) or for the current layer in a map window, if that number is known.
There are two scenarios in which the number of selected records is unknown:
The total number of records in a table is unknown and the selection is inverted. This happens with streaming tables. Prime examples are the result tables of queries: such tables compute their data as they are being read so that you can start working with the returned records without waiting until all records are computed. An inverted selection stores keys of the unselected records, so while the system knows how many records are unselected, it cannot compute how many are selected.
The total number of records in a table is known, but the table has changed in a way that could affect the selection after the last selection operation. Examples of changes that can affect the selection are adding or removing records, or changing values in the key fields.
If the total number of records is unknown, the number of records is shown as a question mark, ?. If the number of selected records is unknown, the number of selected records s shown as a question mark, ?. If either of these numbers is unknown, the Info pane shows a Refresh button next to them. Pressing the Refresh button will scan the table and update numbers that are currently unknown. The scan tracks progress and can be canceled.
Manifold Server reports the total number of records in a served table, if the table knows the total number of records.
Combining the results of a selection template with an existing selection performs faster.
The Help - About dialog includes a new Switch License button that allows switching the current license to a different serial number, for example, to upgrade to a different edition of Manifold. The dialog prompts the user to restart the application after the license is switched.
Writing MapInfo MIF format now exports xN and UUID fields as CHAR data type.
(Fix) The Clear Filter command in a table window no longer fails to uncheck the selection filter button in the toolbar.
(Fix) Copying a table with mixed geometry into a database (eg, PostgreSQL) using copy and paste or drag and drop no longer sometimes fails because the new geometry field is wrongly limited to a specific geometry subtype.
(Fix) The Spatial select template that selects data based on a spatial overlay no longer fails to clear the selection when the overlay is empty and the selection mode is replace or intersect.
(Fix) The result tables of SelectionXxx query functions no longer sometimes fail to return data.
(Fix) Reading MapInfo MIF format no longer sometimes ignores the last object.
(Fix) Reading MapInfo MIF format correctly parses DATE, DATETIME, and TIME values.
(Fix) Writing MapInfo MIF format exports INT16U as INTEGER data type (was SMALLINT) and INT32U as FLOAT data type (was INTEGER), to avoid potential data loss.
Build 177 is a new base build, incorporating all improvements in the 9.0.176.x series of builds. It is recommended for all users. The main change since the prior cutting edge build, 9.0.176.7, is the expansion from a single Release 9 Universal edition into three new licensing levels: Professional, Universal, and Server. The three different editions are distinguished by the availability and configuration of Manifold Server. Over time, it is likely that other differentiations will emerge, with higher end features in the higher end editions.
Includes all features and functions of the former Manifold 9.00 Universal license, except Manifold Server.
Manifold 9.00 Professional can connect to Manifold Server as a client and Professional can create project files for sharing by Server installations, but Professional edition does not include Manifold Server to serve data outbound.
Includes all features and functions of the former Manifold 9.00 Universal license, and also includes a limited version of Manifold Server.
Manifold Server in Universal edition is limited to one Server instance on a server machine, it must be launched from a Command Prompt window, and the Server instance is limited to 20 connections.
A Manifold 9.00 Universal license includes simultaneous interactive use of Manifold 9 on the server machine, which makes it easy to configure project files that are served.
Manifold 9.00 Universal is designed to support sharing files via Manifold Server in smaller workgroups, where each user will be running a Universal license that allows running a Server instance as well as an unlimited number of interactive sessions on the user's machine. When most users in the workgroup are hosting Server instances, the total number of Server instances makes it easy for the workgroup to share many files.
Includes all features and functions of the former Manifold 9.00 Universal license, and also includes a unlimited version of Manifold Server.
The unlimited version of Server has no limit to the number of connections, no limit to the number of Server instances on a server machine, and the ability to launch and to manage Server instances as Windows services.
A Manifold 9.00 Server license includes simultaneous interactive use of Manifold 9 on the server machine, which makes it easy to configure project files that are served and also to launch and to manage Server instances as Windows services using the Manage Services dialog.
Manifold 9.00 Server is perfect for serving data from read-only projects to larger workgroups or, with more powerful server hardware, to entire organizations. The ability to launch many instances makes it easy to organize access to different communities of users based on access URLs and credentials. An unlimited number of connections is perfect for taking advantage of cost-efficient, many core CPUs that can provide very many threads on server machines at low cost.
Pressing F1 in the Tools - Manage Services dialog opens the Manifold Server topic in the default web browser.
There is a new style for text fields in tables: file = absolute path to a file. When text fields are formatted using that style, table windows and the Info pane will highlight text in blue color and will show a new Open File command in the context menu when a cell is clicked. If a specific value is a well-formed path, but the file referenced by it does not exist, the Open File command is disabled. The Open File command opens the file using whatever is the default Windows handler for that file type. That makes it easy to have fields in tables that contain file names that are images, PDF files, saved RDP files, or anything else that Windows can open, and to be able to open those files instantly from within Manifold.
Build 176.7 contains small improvements to the user interface and a set of additions to Manifold Server. Server can now run as a Windows service. The next build will likely be 9.0.177, a new base build.
Watch the new 10 Minute Tutorial - Manage Services video to see the new Manage Services dialog in action.
Illustration: The new Service dialog makes it easy to create new Server instances on a server machine, without having to launch a Command Prompt window and configure new instances using command lines.
Server binds to the network interface in exclusive mode for security purposes.
Server supports a new -logfolder:xxx command-line option to redirect logs.
When launched in a Command Prompt window, Server sets the console title to Server - addr:port - filename. This hides sensitive information like usernames and passwords and it also shows connection information even if that was not spelled out in the command line.
Server supports a new -service command-line option to run as a service. There can be multiple Server services running on the same machine. If a particular service is configured to run under one of the non-interactive system accounts, such as SYSTEM, running it requires installing the Manifold license for all users. The default location for log files for a service is in the common application data folder, C:\ProgramData\Manifold\v9.0.
There is a new Tools - Manage Services command that lists Server services on the system. For each service, the command shows its name, path to the served file, network address and port, start type (auto, manual, or disabled), status (starting, running, paused, stopping, or stopped) and process ID. This command is not available in Viewer or in SQL for ArcGIS.
The Manage Services dialog allows double-clicking a service in the list of services to show service parameters: service name, path to the Server executable, path to the served file, network address, network port, start type, additional command-line options.
The Manage Services dialog allows refreshing the list of services.
The Manage Services dialog allows adding a new service, editing the parameters of an existing service and deleting selected services. Adding, editing and deleting services requires administrator rights. If the current user has no administrator rights, the dialog disables relevant functionality and shows a prompt to run the program as administrator.
Adding a new service in the Manage Services dialog suggests a network address (the first registered network address) to use and a network port (the first unused port between all listed Server services, starting with the default 9099 and going up) to use. For convenience, it provides a list of all registered and loopback network addresses to select from.
The Manage Services dialog allows starting and stopping selected services. Starting and stopping services requires administrator rights.
Tech Tips: When running Manifold Server on a machine, launch Manifold with administrator rights: right-click the manifold.exe file and choose Run as Administrator. When Manifold launches, in Help - About, press the Install button to install the license for all users if it has not yet been installed for all users. To manage Server services, always launch Manifold with administrator rights.
Illustration: The new Manage Services dialog makes it a lot easier to manage Manifold Server instances running as services, without needing to hassle with Windows services dialogs. Click the New toolbar button to add a new Server instance. Ctrl-click a row to select it, and then press start, stop, or delete buttons as desired. Double-click a row to change settings like auto or manual launch on system reboot.
There is a new View - Panes - Fit to Screen command that puts all undocked panes and windows onto the screen. If there are multiple monitors, the command chooses the nearest monitor for each pane or window and puts the pane or window onto that monitor. The command helps locating panes and windows that have been accidentally moved nearly offscreen using the cursor or ended up being completely offscreen due to changes in monitors (may happen with remote desktop connections, among other scenarios).
The Register pane shows automatically placed control points with a distinctive icon. Any editing done to an autoplaced control point removes the autoplaced flag.
Alt-clicking a control point in the Register pane pans the map window to that control point.
Creating a new RTREE index for tiles suggests using fields starting with x and y for XY values.
Editing a traverse allows entering a traverse command for the insert coordinate. This allows entering geometry data as a sequence of traverse commands, with visual feedback at every step.
(Fix) The TileGeomOverlayCount query function on longer allows specifying a non-numeric field as an argument. Either specify either no field by passing an empty string, or specify a numeric field.
Build 176.6 introduces dramatically faster spatial overlay functions and faster LiDAR performance with LAZ files. Big jobs that previously took over two and a half hours now can be done in six minutes.
GeomOverlayXxx (non-topology) query functions can be canceled in the initial data collection phase.
GeomOverlayXxx (non-topology) query functions optimize cases where a big source table has a small overlap area with an overlay table, as long as the source table has a spatial index.
GeomOverlayXxx (non-topology) query functions optimize cases where a source table has a small overlap area with a big overlay table, as long as the overlay table has a spatial index and the source table can determine its bounds without being read in full (this usually means that the source table also has a spatial index and is static, that is, not a query).
GeomOverlayXxx (non-topology, non-filter) query functions now only return pairs of records for the actual overlay incidents. Previously they were also returning a single record for each record in the source table which had no corresponding records from the overlay table, similarly to a LEFT JOIN. We changed the behavior to be similar to an INNER JOIN, because otherwise the functions have to return at least as many records as there are in the source table, which is frequently undesired and just makes processing the result slower. If there is a need to obtain records from the source table that have no overlay incidents, that can be done separately after.
GeomOverlayXxxFilter query functions perform proportionally to the number of records that have overlay incidents. Previously, fetching all records of the overlay filter required going through all records of the source table. This and other changes to GeomOverlayXxx functions apply to various UI tools that use them, eg, they apply to the spatial overlay select templates and to the Join dialog when it transfers data between drawings.
SelectionXxx query functions perform proportionally to the number of selected records. Previously, fetching all selected records of a table required going through all records of that table. The change applies to various UI tools that work with selections, eg, copying a few selected records from a large table now performs much faster. The only exception is that selections that have been inverted continue to perform proportionally to the size of the table, not to the size of the selection.
Opening a LAZ file with a COPC index validates index data and turns off the index if its data is invalid.
Reading a LAZ file no longer progressively caches uncompressed data on disk and instead uses multiple reading heads. This significantly improves performance.
Performing a spatial search on a LAS or LAZ file with a COPC or .mapindxp index works faster.
The above improvements significantly improve the performance of spatial overlays when input data sets are not similar in size or do not significantly overlap. That is a very common situation since often a smaller data set, such area objects for a particular town or parcel, is used to extract data of interest from a very much larger data set, which might cover an entire county or province. The exact performance gains depend on the overlap size, and can easily be 5x, 10x or more. For LAZ, the improvements are even bigger. A test example used a LAZ file 375 million points with a COPC index and fetched 3.5 million points covered by an area in a different drawing using GeomOverlayTouchingFilter. The time to complete the job in prior builds was two hours and 38 minutes (9230 seconds). Using build 176.6 the time dropped to only six minutes (363 seconds), 25 times faster. If the area was even smaller, extracting fewer points from the LAZ file, the task would have taken even less time.
(Fix) Selecting records in a table with an identity index on text fields with external collates (for example, as in an external database) no longer sometimes fails to work.
(Fix) Writing a BIL, FLT, GRD, IMGERDAS, or TIFF file with pixel scale expressed in units other than meters no longer writes wrong local offsets to the accompanying .MAPMETA accessory file.
(Fix) Rendering a LAZ file with a COPC index no longer sometimes prunes search too aggressively.
Build 176.5 expands the license activation system to add an option for enabling installation of a license for all users on a machine at once. That simplifies use for some organizations, for server use, and for IMS use in Release 8, which uses the same activation system.
Build 176.5 also introduces a new, integrated, parallel Download Manager, Manifold's new, high speed, parallel system for downloading tiles from web servers such as image servers, TMS servers, WMTS servers and ArcGIS REST servers. The download manager is a built-in feature within Manifold that works automatically in background. There is no need to install, configure, or administer any modules or other software. Just use Manifold the way you always do and web servers seem to be much faster and more responsive.
This build also provides conveniences such as persistent data types in the Schema dialog remembering the last-used type, Drag and Ctrl-Shift-Drop to link data sources read-only (which facilitates creating projects for Manifold Server and other multiuser settings), a new email style in tables for rapid emailing, and a Suggest Locations toolbar button in the Register pane for faster georeferencing. A variety of other features and a few fixes are also in the new build.
Manifold licenses optionally can be installed (activated) for all users on the same system. The About dialog shows whether the license in use has been installed for all users, and it allows installing or removing the license for all users. Installing or removing the license for all users requires administrative privileges and will prompt the user for administrative credentials.
The Activate dialog includes an option to install (activate) a Manifold license for all users on the same system. The option requires administrative privileges and will prompt the user for administrative credentials.
Manifold licenses installed for all users on the same system work for restricted user accounts for web applications managed by IIS. This allows using Manifold 8 IMS in web applications served by Windows 10+ IIS without any extra configuration for the application pools.
The download manager caches downloaded data in an application-wide cache which is only cleared when the running application, such as desktop Manifold, Manifold Server, or a third-party application using Manifold through the scripting API, is closed. That allows the system to reuse downloaded data between multiple dataports connecting to the same server even across different MAP files.
The download manager allows the user interface to cancel outstanding downloads in the middle of those downloads without waiting for the network. This prevents freezes when a map window that shows a web image is getting panned or scrolled or closed in cases when requests for tiles being downloaded could take a long time to determine if they will complete or fail.
The image server dataport uses the new application-wide download manager.
The TMS dataport downloads tiles using the new download manager.
The WMTS dataport downloads tiles using the new download manager.
The ArcGIS REST dataport downloads tiles using the new download manager. Note that ArcGIS REST servers can be configured to serve some images as screens rather than tiles. Such servers and images will work, but without the benefits of parallel downloads and reliable cancels provided by the download manager.
The Field dialog in the Schema dialog saves and restores the last used data type when creating a new field. This allows quickly adding multiple fields of the same type, for example, quickly adding a series of fields that are all text fields when altering a table's schema.
Drag and Ctrl-Shift-Drop - Dragging and dropping files into the Project pane with both the Ctrl and Shift buttons pressed will link those files in read-only mode. That makes it easy to create projects with read-only data sources, a great convenience when creating projects that will be used simultaneously by other users or served by Manifold Server.
The Project pane shows disconnected data sources that are linked in read-only mode using a lock icon. That makes it easy to see which data sources are read-only even before they are used.
Tables now provide a new style for text fields that are email addresses. Email addresses are shown using the same color as URLs. Right-clicking an email address shows a new Send Email command in the context menu, which automatically launches your mail client, such as Outlook, with the email address loaded in a new email message. That makes it very easy to compose and send an email to the right-clicked address.
New query function: StringEmailValid - takes a string and returns true if the string is a valid email address. The validation is permissive and should only be used to filter out strings that are certainly not email addresses.
New query function: StringUrlValid - takes a string and returns true if the string is a valid URL.
The Manifold ODBC driver allows connecting to a new temporary database using DRIVER=...;DBTYPE=TEMP; in the connection string.
The SQL Server dataport supports DATETIMEOFFSET and DATETIME2 data types. DATETIMEOFFSET values are automatically converted to the timezone for the current user on both read and write. Reading a date with a timezone converts data to the timezone for the current user. Entering a date without a timezone will use the timezone for the current user.
(Fix) Importing a TIFF file no longer sometimes fails to create an RTREE index on x-y-tile fields (a regression).
(Fix) Alt-clicking a tile in an image in a linked TIFF file no longer sometimes fails to show pixel values.
Build 176.4 introduces Manifold Server, Manifold's new, high speed, fully parallel, multiuser spatial database. Server provides very powerful new technology for sharing vast amounts of data from Manifold .map project files, enabling many users at once to use data in projects served by Server. Server is a built in part of Manifold Release 9 Universal edition, build 176.4 or more recent. Build 176.4 also includes new features for the Manifold ODBC driver, improvements to ArcGIS REST, GDB, ECW and TIFF dataports, as well as fixes.
Manifold Server, called Server for short, is a read-only spatial database server that shares the contents of a Manifold project to one or more Manifold clients. Clients are regular Manifold desktop sessions, or even free. Because .map projects can contain nested links to other .map projects, the project served by Server can be the top level entry point to a hierarchy that shares the contents of thousands of .map projects, containing many terabytes of data.
Manifold Server is automatically installed when installing Release 9 Universal. Server can run on a machine at the same time as a regular Manifold session, and multiple Server instances can run at the same time on the same machine, with each Server instance sharing out a different .map file.
See details in the new Manifold Server topic. See also the First Look - Manifold Server video.
Easy to Launch, Load, Administer - Server is the easiest to administer multiuser database server around. Simply launch Server in a command line with the Manifold project file containing data to use. No need to mess around with loaders. Change the data in the database? Stop the server and then launch it again with an edited project, or an entirely different project. Share the contents of the database with someone else? Send the .map project file and you're done. Same with backups - easy!
Many Clients and Many Servers - Many Manifold client sessions can run on the same machine, with each such session connecting to many different Server instances on that same machine or on other machines. Many Server instances can run on the same server machine, with each Server instance using a different port and/or IP address, and each Server instance can serve many Manifold clients from other machines or from the same machine.
Parallel Connections - Manifold client sessions connect to Server using asynchronous, multithreaded, parallel connections for significantly better performance than narrow technology like ODBC. Many threads are balanced on the fly to handle both large and small threads.
Server-Side Rendering Pipeline - When rendering big layers most of the work is setting up the rendering task. Server prepares layers for rendering server-side with full parallelism with only the final stages of rendering happening in the client. That allows very large data in Server to be browsed by clients almost as fast as if it were local. Server-side rendering eliminates the need to send tens or hundreds of gigabytes through the network connection to a client for rendering. You can pan and zoom through a 100 GB vector project through a Server connection almost as fast as if it were local.
Server-Side Execution - Queries in projects served by Server are executed on the server machine, sending only results to clients. Queries can be executed within the Server, within the client, or in a mix of both. Executing queries within Server avoids sending large amounts of data through network connections.
Distributed Parallel Computation - Manifold clients can connect to many different Server installations at the same time, pulling layers from those different Server installations to use together in maps. If a map includes three different layers from three different Server installations on three different machines, when that map is panned and zoomed each of the three different layers will be prepared for rendering and otherwise serviced by the Server machine sharing that layer, so that the computational resources of three different machines will be utilized. Instead of parallel processing on a single computer handling all three layers, parallel processing on four different machines, the three Server machines plus the client machine, will be used to speed up the rendering pipeline for the map. That can result in visibly faster response with big layers.
Database / Data Source Tunneling - Tunneling allows users to reach through Server to utilize data sources within Server. If a project served by Server contains data sources that are databases, such as SQL Server, Oracle, PostgreSQL and others, a Manifold session connecting to Server can open those data sources in the Server hierarchy and reach down into them to use their contents. Other data source content, such as linked files, linked Manifold .map projects, linked Esri geodatabases, and other data sources is also automatically tunneled. Web sources like Bing or Google image servers or ArcGIS REST servers reachable through Internet is not tunneled, but instead automatically set up for direct connections from the Manifold client session to the web source. In both cases layers and data sources served through Server can simply be used without users worrying about the details of whether they are tunneled or not.
Servers Nested within Servers - Projects served by Server can contain Manifold Server data sources, with nested Server data sources automatically setting up pass-through tunneling. For example, a project served by Server on machine A might include Server data sources using Server installations on machines B and C. A Manifold client connecting to Server A will see the Server B and Server C data sources within that project and will be able to open them and use the data and links they contain. Server A will automatically arrange pass-through tunneling to Servers B and C, which will in turn automatically arrange pass-through tunneling to the data sources they share.
Automatic Version Negotiation - Different build levels can be used for a Server installation and for Manifold clients connecting to that Server installation. When the connection is formed the different versions will negotiate to use a common feature set despite different build levels.
Easy User Interface - Both Server and client user interfaces have been kept as simple as possible. Server launches in a simple command line, with a few simple configuration options. Connections to Server within Manifold clients use the same New Data Source interface used for other databases and data sources.
Copy / Paste Server Data Source - Server data sources in a Manifold project are just like any other data source. They can be saved as Favorites for one-click addition to any new projects, they can be copied from one Manifold session and pasted into a different Manifold session, and they are saved as part of the project whenever a Manifold client project is saved, with automatic reconnection when the saved project is opened.
Secure Access - Server can be launched with a user login and password required, to control access when accessible from public Internet.
manifoldsrv [<options>] <file>
The file is required and can be of any type supported by 9. If no file is provided, Server prints the command line syntax and exits. If the file is provided, Server opens it and starts serving connections to it.
-addr:xxx -- network interface to listen on, the default is the primary network interface.
-port:xxx -- port to listen on, the default is 9099, currently unassigned by IANA.
-user:xxx -- user required to connect, the default is none.
-password:xxx -- password required to connect, the default is none, ignored if user required to connect is not specified.
-connections:xxx -- maximum number of connections, the default is 20, use 0 to remove the limit.
-workers:xxx -- number of worker threads to process requests, the default is 4, the number can vary between 1 and 128.
Build 176.3 provides bug fixes, a variety of much-requested features, and significant improvements to printing layouts. It dramatically improves PostgreSQL / PostGIS insert speeds, adds a Fill transform for raster images, and provides a Selection button on the toolbar to quickly toggle filtering to showing only selected records in tables. Layouts can now be printed with automatic, on the fly scaling of any web image layers to provide intended scale, and printing a layout now supports printing images of any size to any resolution.
Illustrations: Applying the Fill transform to a space shuttle SRTM terrain elevation raster with missing pixels (left) by filling in using nearest pixel values gets a really smooth and natural looking result (right). It's really hard to see where pixels were filled. One click does the trick.
Features
Selection filter commands for table windows have been reduced to a single Selected command in context menus. The command switches between showing all records or showing only selected records. In addition to context menus, the command also appears as a new Selected mode button in the main toolbar. Click the Selected button to toggle showing all records or only selected records.
Printing a layout on the fly scales any web image layers to their intended viewing resolution of 96 DPI, to make labels, lines and other features visible as intended.
Printing a layout now allows printing images of any size with any resolution.
New tile transform: Fill. Fills tile pixels in the specified channel using one of three options:
missing pixels - fills missing pixels with a specified value,
missing pixels from nearest - fills missing pixels with the nearest value in a specified radius,
visible pixels - fills visible pixels with a specified value.
Batch inserting records into PostgreSQL now uses the COPY command so long as all of target fields have types with a compatible binary representation. If some of the target fields have types with an incompatible or unknown binary representation (as may happen for fields of new types added to PostgreSQL in the future), the operation falls back to using the INSERT command in batches. Performance benefits are twofold:
COPY in general takes much less time than INSERT. Eight to ten times faster record inserts are common.
COPY is much friendlier to slow connections, where it saves even more time due to significantly reduced round trips.
Illustrations: A screenshot of a PDF created by a layout that includes a Bing Streets image server layer using the old system at left, where scaling to print resolution results in too-small lines and labels. At right the new system automatically scales to screen resolution in the PDF, so when the PDF is printed or viewed we see what we would see in a browser showing the same image server.
Fixes
(Fix) Filter and Slope tile transforms no longer always use channel 0, ignoring the specified source channel..
(Fix) The main application window no longer tries to paint a welcome prompt when windows are horizontally split.
(Fix) Reading an OSM or O5M file no longer sometimes fails and creates no components.
(Fix) Reading a PBF file with less than 65,536 coordinates no longer sometimes fails.
(Fix) Reading a PBF file no longer fails to create areas whose boundaries reference lines with attributes, processing only sequences of coordinates with no attributes attached.
(Fix) Reading a PBF file no longer sometimes loses branches for areas with more than one branch.
(Fix) Reading a PBF file no longer incorrectly classifies some closed lines as areas and vice versa.
Build 176.2 provides a new optimization in the query engine for dramatic performance increases for IN <table> expressions (both for IN and NOT IN), plus new SQL functions for attributes and powerful new Sieve functions for rasters. New transforms provide point-and-click access to Sieve and to random value generation functions. A new set of Select functions provide easy selection based on uniqueness criteria. A user-friendly addition to the Register pane pre-loads entered control points to make workflow faster and easier.
Faster Queries
The query engine optimizes IN <table> expressions (both for IN and NOT IN) on a single value for all data types, for cases where the table has no index on the searched field.
The query engine optimizes IN <table> expressions (both for IN and NOT IN) on a tuple (multiple values of any data type), both for cases where the table has no index on the searched fields and also for cases where it has an index on those fields by making more efficient use of the index.
As is usual for optimizations that start building and using indexes instead of doing full scans, the speed ups are dramatic, running over 800 times faster than before. For example, consider a test query:
VALUE @count INT32 = 1000;
DROP TABLE t;
CREATE TABLE t (mfd_id INT64, INDEX mfd_id_x BTREE (mfd_id), v INT32);
INSERT INTO t (v) TABLE CALL ValueSequence(1, @count, 1);
INSERT INTO t (v) TABLE CALL ValueSequence(1, @count, 40);
DELETE FROM t WHERE mfd_id NOT IN (SELECT First(mfd_id) FROM t GROUP BY v);
Following are typical results for various numbers of records set by the @count global variable, with "old" being before build 176.2 and "new" using build 176.2:
For 20000 records, the optimization introduced in 176.2 reduces the time required from 482 seconds to about half a second, running well over 800 times faster.
Illustrations: Applying the Sieve transform to a land use and land cover classified raster (left) using a Minimum area of 100 to get the resulting, simplified raster (right). Pixel values throughout the raster are adjusted so regions of very small pixels are merged into larger neighboring regions, with the pixels in the former smaller regions given the values of the larger neighboring region into which they have been merged. Sieve functions can be used for many purposes, such as de-noising and simplifying rasters.
New query functions: TileSieve and TileSievePar. Perform a sieve operation on an image channel. Pixels with the same value are used to form areas. Areas smaller than the specified threshold are merged into their neighbors and get assigned the value of the neighbor they are merging into. The result always is a new image. Parameters:
source image.
source channel. Use zero based numbering, from 0 to the total number of channels - 1.
quantization level. Setting this to 1 tells the system that pixels with values of 1.1, 1.2 and 1.7 are similar enough to each other and should be put into the same area if they are neighbors, but a pixel with the value of 2.0 should be put into a different area.
area threshold in pixels. Areas smaller than this value will be merged into larger neighboring areas, if possible. Areas equal to or larger than this value will annex any smaller neighbors that are less than the area threshold.
method used to determine which neighbor to merge into when there are multiple candidates: 1 (default) = merge into the neighbor with the biggest area, or 2 = merge into the neighbor with the biggest common boundary,
thread configuration. (TileSievePar only).
New query functions: TileSieveStep and TileSieveStepPar. Perform a single step of a sieve operation on an image channel. This can be much faster than performing a full sieve using TileSieve or TileSievePar until all areas below the threshold are merged into larger areas. Performing a single step instead of running the operation until there are no more areas to merge is sometimes useful as an intermediate step in bigger computations.
New tile transform: Sieve. Performs a sieve operation on an image channel. Runs until there are no more areas to merge.
New query aggregate function: Percentile. Returns a value for the specified percentile. Percentile 0 returns the minimum value, percentile 100 returns the maximum value, and percentile 50 returns the median value. Works for values of any type.
New query function: StringStart. Reduces a string to the first N characters. If the specified number of characters is negative, the entire string is returned, consistent with the behavior of StringSubstringLen.
New query function: StringEnd. Reduces a string to the last N characters. If the specified number of characters is negative, the entire string is returned, consistent with the behavior of StringSubstringLen.
New numeric transform: Random. Produces random values between the specified minimum and maximum values, with an option to return only whole numbers (on by default).
New UUID transform: Random. Produces random UUID values.
New numeric select: Unique. Selects values based on how many times they occur in a table:
duplicate - for each duplicate value, selects all such records except one,
non-unique - selects records for values that occur more than once,
unique - selects records for values that occur only once.
New numeric vector select: Unique. Similar to numeric select.
New binary select: Unique. Similar to numeric select.
New date select: Unique. Similar to numeric select with an extra option to Ignore time (default: on).
New text select: Unique. Similar to numeric select with extra options: Ignore case (default: on), Ignore whitespace at start and end (default: on).
New UUID select: Unique. Similar to numeric select.
New numeric select: Statistic. Selects values based on how large or small they are relative to other values in the table:
maximum - selects records with the maximum value,
minimum - selects records with the minimum value,
median - selects records with the median value,
top - selects the specified number of records with the largest values,
bottom - selects the specified number of records with the smallest values,
top percent - selects records with values larger than the specified percentile, with an option to include or exclude the bounding percentage.
bottom percent - selects records with values smaller than the specified percentile, with an option to include or exclude the bounding percentage.
New numeric vector select: Statistic. Similar to numeric select Statistic.
New date select: Statistic. Similar to numeric select Statistic with an option to Ignore time (default: on).
New text select: Statistic. Similar to numeric select Statistic with options to Ignore case (default: on) and to Ignore whitespace at start and end (default: on).
Layouts support minimum and maximum visibility scale limits if specified for map layers.
Switching the source window in the Register pane maps already entered control points to source control points by name, to preserve as much of the already entered data as possible.
Build 176.1 introduces visibility scale limits (also known as scale ranges or zoom ranges) for layers in maps using the Layers pane. The build also provides improvements in Esri Geodatabase (GDB), Shapefiles (SHP), DBF, and Google KML or KMZ dataports, facilitating interchange with Google products.
Visibility Scale Limits
The Layers pane Filter button allows showing and editing visibility scale limits (zoom ranges) for map layers on a per layer basis using two new commands: Show Farthest (Max) Scale and Show Closest (Min) Scale. Maximum and minimum scale limits automatically turn layers off and on as the view in the map window zooms in and out in scale. Zooming a map window closer in than the minimum limit hides the layer. Zooming a map farther out beyond the maximum limit also hides the layer. Zooming a map into the scale range between the maximum and minimum limits will display the layer. Layers that are hidden do not participate in interactive operations such as editing, picking, selecting, or snapping. Layout windows do not currently honor layer scale ranges, neither on screen nor during printing. Scale ranges in the layout pane for both viewing on screen and printing will be added in the next build.
For readability, the Layers pane automatically shortens the listing of max and min scales for map layers to only the first two digits as in 1:nn plus k for thousands, m for millions, b for billions, and then for very large values, two digits plus exponential notation. Examples: 1:40k for a scale of 1:40000 or 1:30m for a scale of 1:30000000.
Scale limits are exposed as simple JSON text for a layer in the map's properties, making it easy to programmatically set and change visibility scale ranges.
The new capability makes it easy to create map displays that automatically show or hide layers depending on the zoom level (scale) of the map view. We can automatically turn on more detailed layers as we zoom into a map, or show different content depending on the zoom level.
Exporting data to Esri file geodatabase GDB format automatically truncates text values to 4096 characters, to protect against accidental insertions of very long text values that might impact file geodatabase performance.
Exporting data to Esri file geodatabase GDB format that resulted in the truncation of text values reports the names of fields containing truncated values as well as the total number of truncated values in the log window.
Exporting data to shapefile SHP format automatically truncates text values to the maximum allowed length and reports truncations in the log.
Exporting data to DBF format automatically truncates text values to the maximum allowed length and reports truncations in the log.
Exporting data to Google KML or KMZ format exports values of fields for drawing records as allowed by KML 2.2. Binary fields such as geoms, tiles, or varbinary data types are not exported. Name and Description fields are no longer special-cased and are exported as regular fields.
To let Google products show labels bound to areas, lines, or multipoints, exporting data to Google KML or KMZ format will export labels bound to an area as bound to a point located at the inner centroid of the area, labels bound to a line as bound to a point in the middle of the line at one half the length of the line, and labels bound to a multipoint as bound to the first point of the multipoint. Labels bound to points will continue to be bound to the point location.
Reading data from a Google KML or KMZ file reads field values.
Build 176 is a new base build, incorporating all improvements in the 9.0.175.x series of builds. It is recommended for all users. Changes since 9.0.175.7 include a 32-bit mode launch option for SQL for ArcGIS Pro, Refresh within ArcGIS Pro, mobile geodatabase support, and GPKG and SQLite layers within ArcGIS Pro support. Also added for Advanced moved are new options to specify system colors, a new dataport for Google Sheets, and enhancements to the layers pane.
Additions and Changes
The Tools - Options dialog has a new Colors tab which allows changing system colors. Changes to colors are only applied after a restart. Colors that can be modified include:
Selection - selected data,
Selection + preview - selected data with selection preview on top, used in table window,
Preview and paint progress - preview data or changed data or layer tab chevron for layers that are being painted,
Cache progress - layer tab chevron for layers that are being cached,
Invalid value - Font color used for an invalid collation, unassigned coordinate system, etc.
Unknown value - Font color used for a NULL value or an unrecognized choice.
Reading data from a LAZ file supports COPC (cloud optimized point cloud) spatial indexes. See copc.io for details and example data.
There is a new Web Server: documentserver dataport for document servers that allows accessing spreadsheet data stored on Google Docs. The dataport takes an URL for a spreadsheet and shows it as one or more tables. The exposed data is read-only.
The data source for a document server can be refreshed.
The table coming from a document server can be refreshed.
A table can be exported as a new SQLITE database.
Reading data from a SQLITE database supports an SRID of -1 that used by Spatialite for an unqualified coordinate system.
Table values for a field formatted as a list of choices that do not correspond to any of the specified choices are shown in grayish Unknown Value color instead of with a # prefix.
The Style dialogs with choices for table fields have a toolbar with the Delete button which deletes selected choices. More toolbar buttons will be added in the future.
The Style dialog for table fields that are numeric or vector values no longer shows a language picker button when using choices.
The Layers pane supports the following new commands for table fields:
Show Align - shows and edits field alignment,
Show Language - shows and edits field language,
Show Style - shows and edits field style, specifying a new style for multiple fields only changes the style for fields whose type is compatible with that of the active field.
SQL for ArcGIS Pro
There is a new button to open SQL in 32-bit mode: Open SQL 32-bit. The former Open SQL button has been renamed to: Open SQL 64-bit.
There is a new Refresh button that refreshes tables and table definitions for all layers in all maps, to pick up changes made outside of ArcGIS (eg, in the SQL for ArcGIS add-in).
The Open SQL buttons support the following types of layers besides GDB: GDBMOBILE, GPKG, SQLITE.
The Open SQL buttons pick up layers from all maps instead of just the active map.
Fixes
(Fix) Opening a LAZ file no longer sometimes fails for files with variable-size chunks.
Build 175.6 concentrates on small adjustments, library updates, and fixes in preparation for wrapping up the current series of cutting edge builds and issuing a new base build.
Additions and Changes
The KoBo dataport automatically refreshes opened tables every 5 minutes. This feature utilizes table versions published by KoBo. Table versions do not advance for absolutely all types of changes to the underlying data, but they do advance for the most common type of changes, such as adding new records, as well as for many of the less common types, such as adding or deleting fields.
The Side box caption in the Style dialog used to format numeric table values has been renamed to Positive degrees and now uses a combo box with choices for: combo box with choices for: (none), E, e, N, n, S, s, W, and w.
Definitions for choice styles interpret value; sequences as value=value;, that is, as a choice whose display name coincides with its value. This helps shorten the definitions for choice styles designed to simply limit values for a field to a specific subset without changing the way these values are displayed.
The StringFormatAny query function supports choice styles for text values. The original text value is trimmed before matching any of the choices. The Style dialog for text values allows specifying a choice style.
Sample data in the Style dialog for vector values have been adjusted to be simpler to read.
Lists of choices in the Style dialogs have Value and Show as captions to make it clearer which column is doing what.
Editing a value formatted with a choice style sorts choices in the dropdown menu so that numbers appear before non-numbers and are sorted by a numeric, rather than a text, value. A choice is treated as a number if it starts with a number. An example of order given a variety of choices mixing text and numbers: -9999 (invalid), 1, 2, 2.5, 3 (standard), 4, 10, other. The value other has no numbers in it so it comes last. A text value like 3 (standard) which starts with a number comes between 2.5 and 4 in how the list of choices is shown.
The About dialog reports Windows 11 as Windows 11 vvvv, Windows 10 as Windows 10 vvvv and Windows Server 2016+ as Windows Server vvvv, with vvvv being a Windows 10 style version label such as 21H1.
The image server for Yandex Maps Street Map has been hidden, since at the present time it has been discontinued, apparently with a new version coming.
Third Party Updates
CUDA modules have been recompiled using CUDA 11.5.1. This adds minor performance improvements.
The WEBP dataport has been updated to use LIBWEBP 1.2.1. This fixes a couple of minor bugs in raster encoding.
The internal version of SQLITE used for all types of SQLITE data, including GPKG, has been updated to 3.37.0. This adds support for STRICT tables, improves the performance of some types of queries, and fixes a couple of bugs.
The C runtime (CRT) has been updated to 14.30.30704.0. The portable ZIP package published for the cutting edge build does not include the new version of the CRT, but the MSI / EXE packages for the next base build will include it. One of the notable changes in the new CRT is that the x64 flavor now includes native modules for ARM64 architecture. You can download and install the new version of the CRT from Microsoft. You do not, however, have to do that right away and can wait for the next base build, which will install it automatically. The current cutting edge build for 9 supports both the new version and the previous version of the CRT seamlessly.
Fixes
(Fix) The URL style for text values recognizes URLs with file: and other non-web schemas.
(Fix) Replacing a text fragment using a regular expression no longer fails to ignore case when matching class character sequences such as [a-c] or [abc] if asked to ignore case.
(Fix) The LAS library dataport no longer sometimes returns incomplete results when working with more than around 500 files. This problem arose from a series of issues occurring because the library was keeping too many files opened simultaneously for prolonged periods of time, hitting various limits within itself and forcing other parts of the system into hitting these limits as well. Work on a fix began by reworking the code to limit the number of files kept open at any given moment. Effort continued to not to lose any performance when files above the limit had to be closed and reopened. In the end, thanks to a some algorithmic changes, instead of not losing too much performance, the system managed to gain a little performance.
(Fix) Computing a Voronoi diagram no longer sometimes fails to completely build shapes for some points in near-collinear point configurations, for example, a series of points following a reasonably smooth line in short steps.
(Fix) Exporting labels to DXF no longer fails if label text is defined using a text pattern.
(Fix) Exporting labels to KML no longer fails if label text is defined using a text pattern.
Build 175.6 adds even more formatting options for field values, including the ability to launch URLs from table fields in browsers. Styles also include the ability to set angular measurements, such as latitude and longitude values in degrees, in different formats. A new style also allows choice fields in tables. Styles set in tables are now automatically inherited by labels. Language options have been extended to include over 600 languages, supporting the full set of languages in Windows 10.
Build 175.6 also adds a new dataport for KoBoToolbox, a widely used open source technology for data acquisition using mobile clients or a web browser. The data can be hosted on a server provided by KoBoToolbox (there are generous free options), or on your own system with KoBoToolbox. The new capability makes it easy to acquire data in the field using survey forms and to instantly make that data available within a Manifold project.
The next build is expected to fill missing pieces for features added during the 175.x build series, add various small items based on user feedback, fix outstanding bugs, and go public as the next base build, 176.0.
Styles
Field values shown in labels are formatted according to the field style specified in the table window.
GEOMWKB values can show type, coordinate count and branch count data if requested by style.
The collation picker shows invalid collations in red.
The language combos in the Style dialogs for field values have been replaced with language pickers. A language picker shows a menu with favorite languages plus a command to select a different language using a dialog, which includes a filter box to make it easy to find a desired language-country combination, plus a command to edit favorite languages. The neutral language is the default favorite. See the new Styling Table Fields topic for an example of choosing a language and also adding a favorite language.
The list of supported languages has been extended to cover all languages supported by Windows 10. This increases the number of languages supported in table field styles from over 200 to over 600. Whew!
Field values shown in the Info pane are now automatically formatted according to the style set for each field.
There is a Style dialog for text fields and a new style for text values: u = URL. If a text field is styled as a URL, valid URL values are shown as hyperlinks, both in a table window and in the Info pane.
The context menu for cells in table windows or the Info pane for a text field styled as a URL value includes a new Open URL command which opens the URL in a web browser.
There is a new style for numeric values that can be used with the StringFormatNumber SQL function: a or A = angle, which is used to choose different formats for angular measures, such as latitude and longitude values in degrees. The style is specified as a string that concatenates a or A with optional letter codes for <subformat>, <side>, and <decimals>.
The <subformat>letter specified the overall format, and can be d or D = decimal degrees, m or M = degrees and decimal minutes, or s or S = degrees, minutes and decimal seconds. If the <subformat> letter code is empty or unrecognized, it is mapped to s is used for degrees, minutes and decimal seconds. The
<side> letter code specifies which hemisphere is represented with positive numbers, with values in the other hemisphere being negative numbers. There are both N and S options as well as E and W options so that North or South latitudes can be shown as positive/negative numbers, and so that East or West longitudes can be shown as positive negative numbers. The <side> letter code can be n, N, e, E, s, S, w, or W, with the case of the code choosing whether the appended N, E, S, or W character is upper or lower case in the styled result. The usual choices for conventional positive/negative number assignments is choosing N for the <side> code for latitude numbers and E for the <side> code for longitude numbers. If the side letter code is unrecognized, no side code is used.
The <decimal> value specifies the number of decimal digits for the smallest term. If no <decimal> value is specified, the default number of decimals for the language will be used. For example, a style argument of 'adN2' will format a number as an angle (a), using a general format of decimal degrees (d), with positive numbers having an N appended and negative numbers an S appended to indicate North or South latitudes (N), with the decimal degrees limited to two decimal digits (2).
A typical use might be StringFormatNumber([Latitude], 'adN2', '') which would result in a numeric value of -6.0363 being formatted as the string 6.04º S.
The Style dialog for formatting numeric table fields lists ad, am, and as style variants, corresponding to angle formats using decimal degrees, degrees and decimal minutes, and degrees, minutes, and decimal seconds. The style dialog allows specifying the number of decimals and the side, corresponding to settings used for the StringFormatNumber function.
Vector values formatted using the a or A style alternate the side between individual n, N, e, E, s, S, w, or W values to support the typical use of vectors to provide longitude and latitude values within the vector.
If the <side> code is specified and it is specified as e or E (easting), the first value is formatted as e and the second as n (northing). This corresponds to typical X,Y vector longitude, latitude ordering and the convention that eastern hemisphere longitudes and northern hemisphere latitudes are positive.
When n or N is specified as a <side> code for vector values, the first value is formatted as n and the second as e. That corresponds to Y,X vector latitude, longitude ordering while retaining the convention that eastern hemisphere longitudes and northern hemisphere latitudes are positive.
When w or W is specified as a <side> code for vector values, the first value is formatted as w and the second as s. This corresponds to typical X,Y vector longitude, latitude ordering but using a backwards arrangement from the usual convention by representing eastern hemisphere longitudes and northern hemisphere latitudes as negative, with western hemisphere longitudes and southern hemisphere latitudes as positive.
When s or S is specified as a <side> code for vector values, the first value is formatted as s and the second as w. This corresponds to a double backwards arrangement of Y,X vector latitude, longitude ordering while using positive numbers for western hemisphere longitudes and southern hemisphere latitudes.
If there are three values in the vector, the third value is formatted as plain floating-point (not an angle). That is a typical arrangement for a three component vector showing longitude, latitude, and Z (height).
If there are four values in total, the third value is formatted the same as the first value, and the fourth value is formatted the same as the second value. That is a typical arrangement for a vector of four values that represent two point locations, such as the beginning and end of a straight line segment.
Choice fields: a fixed set of choices for numeric fields can be specified in a list the Choice tab of the Style dialog for table fields. The list in the Choice tab is a lookup list of lookup key and value pairs, with the left column in the list being a lookup key that is an integer number, with the right column being the value specified by that key. Entering values in cells for that field will call up a drop down list of the available choice options. Choice options in the Choice tab list can be edited: add a new choice by editing cells in the last, * row of the list, or edit the key or value for any existing choice. One or more choices in the list can be deleted by selecting them with a Ctrl-click or other of the usual selection moves, and then pressing Delete. If a numeric value stored in a table does not correspond to any of the choice keys, it is shown prefixed with a # character. Upcoming builds are expected to extend this feature to support text fields and support choices stored in tables.
Table Editing
Editing a boolean value in a table window or in the Info pane shows a menu with choices for true and false.
Editing a numeric value formatted as a choice in a table window or in the Info pane shows a menu with available choices.
Editing a vector value formatted as a choice in a table window or in the Info pane shows a menu with available choices, with a separate submenu for each vector component.
Data Collection with KoBo
There is a new dataport for KOBO toolbox. The dataport supports the following servers: Kobo Everyone - the default, Kobo Humanitarian - a special server for humanitarian organizations where such organizations can store big data sets for free, and Custom - your own server. To connect, create a user account and enter your API key in the dialog. The dataport allows accessing all assets stored for the account, in read-only mode. Future builds are expected to allow writes as well.
The URL for a KOBO data source can be set to that for a specific asset. This is convenient when the number of assets on the account is large.
The KOBO data source can be refreshed, to pick up changes to the list of assets.
The table on a KOBO data source can be refreshed, to pick up changes to records or fields of a specific asset.
The KOBO dataport translates image / audio / video / file sentinels to URLs.\
See new topics for KoBo and data collection: Collecting Data in the Field, KoBo Servers, Example: Create and Use a Survey Form, Example: Connect to Data Collected in the Field.
See the video versions of new example topics in the 10 Minute Tutorial - Create and Use a Survey Form and 10 Minute Tutorial - Connect to Data Collected in the Field videos.
Fixes
(Fix) The PostgreSQL dataport no longer fails to determine the schema for a materialized view on PostgreSQL 12+.
Build 175.5 is a large build with many new features. It utilizes new formatting functions introduced in the prior build to provide point-and-click formatting of data displayed in tables with a very wide range of preset and custom formatting options for different data types. Upcoming builds will automatically apply formatting specified in tables to labels and other settings.
Build 175.5 also introduces new formatting functions, new utility functions, and improvements in the Schema dialog.
Schema
The Add Field, Add Index, Add Constraint, and Add Identity commands in the Schema dialog focus the last added schema item.
The Schema dialog includes a filter box to filter schema items using their name. Filtering by name automatically disables Add Field, Add Index, Add Constraint, and Add Identity commands to prevent newly added items from being immediately hidden due to not passing the filter. Editing an existing item and changing its name lets the item stay visible even if the new name no longer matches the filter.
The Schema dialog includes a filter button with a drop down menu to filter schema items using their type: All, Fields, Indexes, or Constraints. Filtering by type automatically disables Add Field, Add Index, Add Constraint, and Add Identity commands for items that will not pass the filter. For example, when setting the filter to only show fields, we can still add new fields, but we cannot add indexes and constraints until we change the filter to show indexes and constraints.
The Move to Top, Move Up, Move Down, and Move to Bottom commands in the Schema dialog have been removed. These commands could only reorder new schema items and such reordering only made sense for computed fields referencing each other. This was a case where the commands were trying to help those who already knew what they were doing at the cost of confusing everybody else, so they have been removed. Future builds will be able to handle computed fields in arbitrary order, provided there are no circular references.
Table Formatting
Table windows allow changing the alignment of field values using Align Left, Align Center, and Align Right commands in the context menu for a field column header.
The default size for binary fields in a table window has been forced to be constant (instead of adjusting to the system font).
Table windows allow changing the format of field values using a new Style command in the field column header context menu. Fields of all data types except text support formatting, with support for text fields likely to be added in future builds. The Style command launches a dialog with controls depending on the field data type:
binary: select style from a list of available styles, with optional language specification.
boolean: select style.
datetime: select a standard style or compose a custom style from format parts, with optional language specification.
geom: select style, with optional language specification.
number: select style, specify the number of digits and decimals, with optional language specification.
tile: select style, with optional language specification.
uuid: select style.
numeric vector: select style, specify the number of digits and decimals, with optional language specification.
All styles and style parts show what kind of output will be produced in the dialog. The language is specified using an editable drop down box which allows either typing the desired value directly or picking it from several default choices, such as neutral language and languages for available localizations.
Note: There are two cases where table windows ignore field styles:
Editing a cell always uses the default style. Otherwise it is too easy to lose data. For example, suppose that a numeric field is set to show two decimal digits. This shows 3.14159 as 3.14. If we start editing the value and the edit box opens with 3.14, pressing Enter will accept 3.14 and will change the value from 3.14159 to 3.14, a possible mistake. To prevent such errors, when we start editing the value, the edit box opens with 3.14159, showing all available digits.
Filtering records by right-clicking a cell and using commands in the context menu always reports the value of the clicked cell using the default style. As with editing, applying a style would miscommunicate what the filter is going to do.
Formatting Functions
The StringFormatNumber query function now uses a language-specific decimal separator for the default format.
The StringFormatVector query function now uses a language-specific list separator for the default format.
New query function: StringFormat. The function formats an arbitrary value according to its type. GEOMWKB values are currently reported as binary to avoid conversion, similarly to previous builds. Future builds will change that to report GEOMWKB values as geometry.
New query function: StringFormatBinary. The function formats an arbitrary value using one of the following formats:
b (default) = <varbinary> -- with the type name
B = <varbinary, XXX b> -- with the number of bytes
New query function: StringFormatGeom. The function formats a geometry value using one of the following formats:
b = <geom>
B = <geom, XXX b> -- with the number of bytes
t (default) = <geom, line> -- with the type of geometry value
T = <geom, line, 200 c / 3> -- with the number of coordinates and the number of branches after a slash; if the number of branches is 1, it is not shown
The StringFormatGeom query function detects geoms with Z values and reports them as areaz, linez, or pointz.
New query function: StringFormatTile. The function formats a tile value using one of the following formats:
b = <tile>
B = <tile, XXX b> -- with the number of bytes
t (default) = <tile, 100 x 100 x 3> -- with width, height and number of channels; if the number of channels is 1, it is not shown
T = <tile, 100 x 100, float64x3> -- with width, height and pixel type, including the number of channels
Other Query Functions
New query function: StringDateTime. The function takes a string and attempts to parse it into a date time value according to the specified language.
New query function: Random. The function produces a random floating-point value between 0 and 1, with 1 excluded.
New query function: RandomInt. The function takes an integer limit and produces a random integer value between 0 and limit-1.
New query functions: RandomRef and RandomIntRef. The functions produce random values similarly to Random and RandomInt, but take an extra argument of an arbitrary type to use as a dependency.
New query function: RandomHash. The function takes an arbitrary value, composes its 64-bit hash using the Fowler-Noll-Vo FNV1a hash function, and turns that hash into a pseudo-random floating-point value between 0 and 1, with 1 excluded. The purpose of this function is to produce a value that is both uniformly random across the domain of argument values, and coincides for calls where the argument value is exactly the same.
New query function: RandomHashInt. The function takes an arbitrary value for the hash and an integer limit, composes a 64-bit hash using the Fowler-Noll-Vo FNV1a hash function, and turns that hash into a pseudo-random integer value between 0 and limit-1.
New query function: DataHash. The function takes an arbitrary value and composes its 32-bit hash using the Fowler-Noll-Vo FNV1a hash function.
Miscellaneous
The Clip transform automatically removes records with NULL geometry when producing a new table.
Error reports produced by the GDB dataport and dataports for databases like SQL Server have been extended to cite the names of relevant fields and indexes.
Fixes
(Fix) The LAS dataport no longer sometimes returns incomplete data for a spatial search.
(Fix) The GDB dataport marks computed fields for length and area so that they do not prevent copy and paste of tables into a .map project. Instead, they get converted to non-computed fields.
(Fix) The table window no longer hides Unicode characters, such as the euro sign and several other characters, that are not recognized as printable in the default CRT locales.
Build 175.4 begins adding formatting options for field values. This build adds query functions that format values of various types. Subsequent builds will allow applying this formatting to fields in tables and queries, and automatically carry it over to labels. This build can be used immediately to format values in labels by using expressions in the label text pattern.
Labels
The New Labels dialog has been reworked to include a field combo for an existing drawing ;that allows selecting a field or a special (pattern) choice. Selecting a field will create labels with text taken from that field. This allows keeping workflow as simple as possible in typical, simple scenarios. Selecting (pattern) allows editing the text pattern directly. Creating labels for a drawing tries to select a field which makes the most sense to display in labels. The priority order is: a text field > a non-text field that does not have a unique index built on it (to deprioritize fields like MFD_ID or SHAPEID) > a non-text field that has a unique index built on it.
The Components tab of the Info pane no longer shows a 'Text' field for labels, since labels now use text patterns.
When a label is being added or is picked for editing in a labels component, the Info pane automatically determines fields that are referenced in the label text pattern, and shows a distinctive icon in the row handle for each referenced field in the Values tab. That makes it easy to see which fields are used in the label. Starting to add a new label or starting to edit an existing label also automatically puts the cursor onto the value of the first such field.
Query Functions and Format Patterns
Format patterns for all new functions are roughly compatible with .NET even though the functions themselves are implemented using native Windows calls. The system does not use .NET directly because there are cases where this would waste too much resources. The system does not currently use ICU, because it is only available on Windows 10 and there must be reasonable fallback behavior for non-Windows 10 systems. In the future Manifold likely allow formatting calls to use ICU, if it is available.
New query function: StringFormatDateTime. The function takes a datetime value to format, a format pattern, a language, and returns a formatted string. The language can be an empty string (neutral) or a language code like en-US or de-DE. The format pattern can be either a single letter that defines the whole format, or a combination of format parts.
Single-letter format patterns - Using an example value of 25 March 2020 13:45:00 and an example language of en-US, single-letter format patterns and their result are:
d = short date: 3/25/2020
D = long date: Wednesday, March 25, 2020
f = long date + short time: Wednesday, March 25, 2020 1:45 PM
F = long date + long time: Wednesday, March 25, 2020 1:45:00 PM
g = short date + short time: 3/25/2020 1:45 PM
G = short date + long time: 3/25/2020 1:45:00 PM
m = short month and day: 25 Mar
M = long month and day: 25 March
r or R = RFC1123. The language does not matter and is always neutral Wed, 25 Mar 2020 13:45:00 GMT
s = sortable. The language does not matter: 2020-03-25T13:45:00
t = short time: 1:45 PM
T = long time: 1:45:00 PM
u = universal sortable, the language does not matter: 2020-03-25 13:45:00Z
y or Y = year and month: March 2020
An empty format string is interpreted as G = short date + long time.
Illustrations: Simple format changes allow creating many different labels and strings from a text field and a datetime field.
If the format string differs from any of the single-letter formats and is not empty, it is interpreted as a combination of the following format parts. Two letter variations add a leading zero for one-digit numeric values and use two characters for text values, for example, 01 instead of 1, and PM instead of P.
d or dd = day: 25
ddd = short week day: Wed
dddd = long week day: Wednesday
g or gg = era: A.D.
h or hh = hour on a 12-hour scale: 1
H or HH = hour on a 24-hour scale: 13
m or mm = minute: 45
M or MM = month as a number: 3
MMM = short month: Mar
MMMM = long month: March
s or ss = second: 0
t or tt = time marker: PM
y or yy = short year: 20
yyy or yyyy or yyyyy = long year, accepts up to five letters because some languages allow up to five: 2020
Any character or any character sequence different from the above patterns, for example, x or hhh, is copied without any changes.
Example pattern: d-MMM-yy h:mm result: 25-Mar-20 1:45
The default conversion of a datetime value to a string now uses leading zeros for month, day and hour to be the same as the output of an empty format for neutral language.
Illustrations: Specifying a language-country code automatically sets the format and language used. The exact same datetime value shown using D format with language set to English using en-US, German using de-DE, Cherokee using chr-US, and Indian Kashmiri using ks-IN shown above.
New query function: StringFormatDurationDays. The function takes a number of days to format, a format pattern, a language, and returns a formatted string. The format pattern is a combination of the following format parts:
d = days
f or ff or fff or ffff or ... (up to nine f characters) = fractions of a second
h or hh or H or HH = hours
m or mm = minutes
s or ss = seconds
Format parts must be ordered from bigger to smaller, that is, days before hours, hours before minutes, and so on.
An empty format string is interpreted as s = seconds.
Examples:
? StringFormatDurationDays(1.133, '', '') -- 97891 (seconds)
? StringFormatDurationDays(1.133, 'd:hh', '') -- 1:03
? StringFormatDurationDays(1.133, 'h:mm:ss', '') -- 27:11:31
New query function: StringFormatDurationSeconds. Same as StringFormatDurationDays, but the value to format is in seconds.
New query function: StringFormatNumber. The function takes a numeric value to format, a format pattern, a language, and returns a formatted string. The format pattern consists of a single-letter format and an optional integer number specifying the desired number of digits or decimal digits, depending on the format.
Using an example value of 123456.7890123 and an example language of en-US, single-letter format patterns and their result are:
c or C = currency, with an optional number of decimals: $123,456.79
d or D = decimal integer, with an optional total number of digits and optional leading zero. The language does not matter: 123457, or, for 'd08': 00123457
e or E = exponential, with an optional number of decimals. The default number of decimals is 6 regardless of the language: 1.234568e+05
f or F = fractional, with an optional number of decimals: 123456.79
n or N = fractional + digit groups, with an optional number of decimals: 123,456.79
x or X = hexadecimal integer, with an optional total number of digits and optional leading zero. The language does not matter: 1e241
If the format string is empty or invalid, the number is formatted as the shortest form that preserves all significant digits. The language does not matter: 123456.7890123
Using upper or lower case e or E or x or X specifies the case of the resulting exponent character or hexadecimal letters.
New query function: StringFormatVector. The function takes a vector value to format, a format pattern, a language, and returns a formatted string. The format pattern is the same as for StringFormatNumber. The formatted string consists of strings for the individual vector values separated using a list separator for the specified language, and enclosed in square brackets, for example, [ 2.34, 5.67 ]
New query function: StringFormatUuid. The function takes an UUID value to format, a format pattern, and returns a formatted string. The format can be either x or X. The use of upper or lower case controls the case controls the case of hexadecimal letters in the result. The default is x.
New query function: StringFormatBoolean. The function takes a boolean value to format, a format pattern, and returns a formatted string. The format can be either b or B to output TRUE as 1 and FALSE as 0, or t or T for output as true or false, with the case of the result controlled by the case of the pattern letter. The default is t.
For all format functions, using an empty format string and an empty language string produces the same string as the default conversion to a string. For example, StringFormatXxx(..., '', '') produces the same result as CAST (... AS NVARCHAR).
Other Query Functions
New query function: DateTimeCurrent. The function returns the current date and time.
New query functions: DateTimeCurrentRef and UuidMakeNewRef. Both functions take a single argument of an arbitrary type. DateTimeCurrentRef then returns the current date. UuidMakeNewRef generates and returns a new UUID. Neither function uses the value of the argument. The purpose of having an argument is to tell the query engine that it should not cache the result returned by the function between calls where the argument value might change. This is useful in multiple ways.
For example, the function allows forcing a new UUID value for each record in a SELECT. Consider the following examples:
--SQL9
SELECT [name], UuidMakeNew() AS [uuid] FROM [cities table];
The result of the above query is that all records ill have the same UUID. Because the query engine sees that the call to UuidMakeNew() is exactly the same for each record, it calls the function once, caches the result, and then copies the result into each record.
--SQL9
SELECT [name], UuidMakeNewRef([name]) AS [uuid] FROM [cities table];
In the above query, each record will have a new UUID generated. Because the calls to UuidMakeNewRef(...) are different between records, the query engine sees that and dutifully calls the function anew for each record.
The following allows creating a computed field with the current date, which will update itself after some of the fields are changed:
--SQL9
ALTER TABLE [cities table] (
ADD [last_edited] DATETIME AS [[
DateTimeCurrentRef([name] & [state] & CAST([population] AS NVARCHAR))
]]
;
The query takes a table named cities table and adds a computed datetime field named last_edited. Adding a field populates it with the current date. If we then pick some record and change the value of one of the fields mentioned in the expression, such as name, state or population, the computed field in that record will be recomputed and set to the now-current date.
Build 175.3 reworks labels to use a text pattern instead of a text field. The new system applies to all use of labels, such as storage, rendering, printing, and so on. A text pattern is much more flexible than a text field: it can reference more than one field, and it also allows easy changes to label texts without changing the table (tables can be very big, or may not have permissions to be altered). Future builds will allow formatting field values in the pattern, and will include various improvements for rendering labels. Support for GDB is gradually being extended, and future builds will include several updates specific to SQL for ArcGIS.
Illustration above: Labels can include expressions that use almost any SQL function and feature, such as CASE. The Style pane makes it easy to edit existing label text.
Labels
Labels have been reworked to compose text from zero or more fields instead of always using a single field. The text used for labels now is specified as a pattern that may reference fields using their names within square [ ] brackets, similar to Release 8. The plain text parts of the text pattern may contain characters escaped with a backslash \ character, for example, \[, \], \\, and so on, as well as Unicode sequences beginning with a backslash \ character in the form \uxxxx. For example, \u00a9 is a copyright sign. Unknown fields are rendered in the resulting label as three asterisk *** characters.
Label text may contain expressions within double square [[ ]] brackets. Expressions can be almost any SQL expressions, including use of SQL functions and complex constructions using CASE and similar. Note that the first character of the expression following the initial [[ characters must be different from a left square bracket [ character, or otherwise the expression will be interpreted as a field name. For example, [[[a] will be interpreted as a field named [[a, the same as how that string would be interpreted in a query. To immediately refer to a field name in an expression, use a space character after the opening double square [[ brackets, for example, [[ [length]/100 ]]. White space within expressions, that is, within double square [[ ]] brackets does not matter, so we can format expressions as we like using white space for legibility. Expressions used in label text may not reference database components, the same as with Expression transform templates. Invalid expressions are rendered in the resulting label as three asterisk *** characters.
The Style pane allows editing label text for label components using a specialized dialog with a query builder.
Rendering labels optimizes the rendering of complex whitespace. Whitespace in the label text pattern outside of expressions is used in the resulting labels.
Migrating from Manifold 8 labels that are bound to a drawing no longer creates a computed field with label text. Instead, it just uses the label text pattern without any changes. If (not likely to happen frequently) the Release 8 label text pattern includes double square [[ ]] bracket text sequences, these sequences should be escaped manually to make the label text exactly what it was in Manifold 8. If need be, an automatic feature to escape such sequences might be added in future builds.
The New Labels dialog allows editing the label text pattern when creating labels for an existing drawing.
Other
The Info pane remembers the last used view options for the Coordinates tab on a per-window basis (map window / table window). For example, if last we used traverse commands in a window, the next time the Coordinates pane will still be in Show Traverse mode.
Attempting to write into a GDB a text value that is too long for a field now shows an error message with the name of the field, the maximum length allowed by the field and the actual length of the value.
Creating a new text field in a GDB sets the maximum length of that field to 4096 characters , an increase from the previous maximum of 255.
Special fields in a GDB that cannot be edited, for example, Length and Area, are marked as read-only in Release 9 and in SQL for ArcGIS Pro.
Fixes
(Fix) Attempting to open a damaged GDB no longer tries to read its structure, which previously would report annoyingly many errors.
Check out the new video providing a tutorial introduction to creating labels:
5 Minute Tutorial - Labels from Fields and Expressions - A quick look at the new label system in Manifold Release 9 that makes it easy to create labels from fields and expressions. Create label text from multiple fields and also using expressions, taking advantage of any of the hundreds of SQL functions provided in Manifold. A new addition to the Style pane for labels makes it really easy to edit label text, including a full expression builder. Works in the free Manifold Viewer, too!
Build 175.2 provides an extensive new system for interactive editing of traverses, including automatic adaptation using the compass rule to close traverses, all part of enhancements to vector editing and the Info pane. It also includes improvements and fixes for various dataports. Even if we are not interested in traverses, the new capability for live traverse editing allows us to create lines and areas parametrically by specifying the direction and distance of line segments, a very useful capability in general purpose vector editing.
Illustration at right: Edit traverses interactively in the info pane, using any of the formats supported for Esri-style traverses.
Traverses / Vector Editing / Info Pane
When a line or area object is being created or picked for editing in a map window, the Coordinates tab in the Info pane allows editing traverse coordinates for vector values. Double-click into any row in the Coordinates pane to edit that row, using Esri-style traverse format for the traverse coordinate commands in each row. Traverses specify coordinates relative to what has gone before, so editing a traverse coordinate within a branch (that is, not at the end of the branch) repositions all coordinates that follow it in the branch. Editing the first coordinate of a closed area branch alters both the first and last coordinates of that branch. Editing the last coordinate of a closed area branch does nothing. Edited angles and distances are accepted in the same format that they are displayed in the Coordinates tab row, that is, Angle and Distance or Direction and Distance.
We can "grow" a branch for a line or area object using traverse commands by editing the new traverse row for each new coordinate. In the map window, enter Insert Coordinates mode (which will already be in action when creating a new line or area) and click the approximate spot for the next coordinate. That will add a traverse row in the Coordinates tab. Next, double-click into that row and edit the traverse command for that segment as desired. Press Enter. The new segment/coordinate will be added as specified, and the mouse cursor in the map window will still be in Insert Coordinates mode, which will enable us to rapidly click the approximate next coordinate so that next, new traverse command can also be edited, and so on.
We are not limited to inserting and editing new traverse rows only at the end of a branch. If we like we can, in the usual way, in the map window we can enter Insert Coordinates mode within a branch, to add a coordinate within a segment. In Insert Coordinates mode we can click at the approximate location where we want to add the coordinate. That will add a traverse row in the Coordinates tab. Next, double-click into that row and edit the traverse specification for that new segment as desired. Press Enter. The new segment/coordinate will be added as specified, with all subsequent segments in the branch being repositioned. Since all the rest of the segments must be moved, the mouse cursor will drop out of Insert Coordinates mode.
When in Insert Coordinates mode, we are not limited only to growing the current branch by adding new coordinates to the end of it, and we are not limited only to inserting new coordinates within the current branch being edited. After entering Insert Coordinates mode for a branch, we might notice an error in some other branch of the same object. We can correct that error by double-clicking into a traverse row for that other branch, making any changes desired, and then pressing Enter. All subsequent coordinates in that other branch will be repositioned, but since none of the coordinates need be repositioned in the branch where we entered Insert Coordinates mode, the mouse cursor will still be in Insert Coordinates mode, allowing us to continue on with our inserts despite having taken a moment to edit a coordinate in a different branch.
The Coordinates tab uses Esri-style traverse format. Traverses can thus contain only circle arcs in addition to straight line segments. However, line and area objects in Manifold can also contain ellipse arcs and spline arcs in addition to circle arcs and straight line segments. When we edit a traverse row for an existing object in the Info pane Coordinates tab the system automatically will convert straight line segments, circle arc, ellipse arc, and spline arcs into straight line segments and circle arcs, depending on the traverse command entered.
Context menus when creating new lines and areas, or when editing existing lines and areas to continue the branch, have a new command intended to support interactive creation and editing of traverses: End Current Branch + Close Gap. The command attempts to close the current branch using the compass rule for adjusting traverses and then ends the branch. If the branch contains less than 4 coordinates, the branch is ended but the gap is not closed. Curved segments are adjusted using approximate lengths. The compass rule adjusts all angles and lengths of traverse segments assuming that the errors which resulted in the traverse not being closed are distributed through all the angles and lengths in the traverse.
The Info pane allows copying field names from the list in the Values tab. Right-click on a field name and choose Copy.
The Info pane allows copying XYZ values and traverse commands from the list in the Coordinates tab. Right-click a coordinate number or a traverse row and choose Copy.
The Info pane allows copying the Coordinates tab list in plain text format, using Manifold coordinates format for XYZ coordinates and Esri traverse format for traverse rows. Right-click onto any row and choose Copy Coordinate List. See the Info Pane: Coordinates topic for format definitions.
Dataports
When set to use cache, the ArcGIS REST dataport compresses stored IDs of vector objects.
The ArcGIS REST dataport provides record counts for tables.
The GDAL dataport has added support for GDAL 3.1.x and 3.3.x.
Caches for web images now supports tables with multiple tile fields, for example, for ECWP.
The ERDAS dataport now parses datum for coordinate systems marked as LOCAL.
The GDAL, JPEG2000, ECW, and SID dataports check channel descriptions for a 4-channel image and set the image to either RGB or RGBA (was previously always set to RGBA).
The JPEG2000 dataport parses coordinate system data stored as EPSG codes.
Fixes
(Fix) Loading a malformed traverse file rejects any circle arcs that have no previous coordinates, instead of producing garbage.
(Fix) Using a web image or a similar dataport with a cache database other than a MAP file no longer sometimes leaks memory.
(Fix) Rendering SID data no longer sometimes produces black pixels in areas close to the image boundary.
(Fix) Rendering JPEG2000 data no longer sometimes produces black pixels in random locations (occurs only on specific zoom levels).
(Fix) The ECW dataport no longer misreads transparency data.
Check out the new video providing a tutorial introduction to editing traverses:
First Look - Edit Traverses - Traverses specify borderlines for areas or lines by describing the segments followed to draw the figure, beginning at a starting point and then going in a given direction for a given distance, then the going in a given direction for the specified distance for the next segment and so on. Traverses are used in surveying for parcel descriptions and in general purpose GIS for drawing figures parametrically, as in drawing a line at a given angle for a given distance.
Manifold Release 9 has the rare ability to edit traverses on the fly, either when interactively creating a new traverse, or to edit existing traverses imported from parcel or other traverse descriptions. The video shows how easy it is to edit traverses in Release 9, including how with one click you can automatically adjust a traverse so it closes precisely, using the mathematically sophisticated "compass rule" to automatically adjust angles and distances throughout the traverse. All this works in the free Manifold Viewer, too!
Build 175.1 is a cutting edge build that contains a big set of fixes and improvements for databases, better error diagnostics for operations on data in MAP files, and a rework of query operations for tile masks.
Databases, Error Diagnostics, and Query Operations
Error messages for table design operations in MAP files have been extended to include the names of fields, the names of constraints, and the names of indexes relevant to the error. For example: Cannot rename field 'length', referenced by computed field 'length-km'.
Error messages for table record operations in MAP files have been extended to include the names of fields, the names of constraints, and the names of indexes relevant to the error. For example: Cannot set values of one or more fields, rejected by index 'name-x'. For example: Invalid value of key field 'x', null not allowed.
The file filter for GDB has been changed from *.gdb;gdb to just gdb.
The file filter for GDBMOBILE has been renamed from GEODATABASE Files to GDB Mobile Files.
Computing a bounding box for geometry data on SQLITE with SPATIALITE or ESRI ST_GEOMETRY extensions uses ST_MinX, ST_MinY, ST_MaxX, and ST_MaxY functions for much better performance.
When creating a new table on DB2, Jet, MySQL, Oracle, PostgreSQL, and SQL Server, names that are longer than allowed by the server automatically will be reduced to the name length allowed by the DBMS.
Data sources (like web servers) set to cache data in the parent database (when the Save cached data between sessions option has been checked in the in the File - Create - New Data Source dialog) check the capabilities of the parent database and proceed without cache if the database does not support all features required for caching. For example, Manifold .map is accepted, and databases like SQL Server are accepted with a couple of minor limitations. DB2 is rejected for vector caches because the DB2 default SRID requires X and Y values to be non-negative (engineering work is underway to find a workaround for that DB2 limitation). GDB and similar data sources that look like databases but have many limitations are rejected for cache.
Data sources (like web servers) set to cache data in the parent database use separate cache tables for each database schema. Previously, the system tried to use cache tables in any accessible schema: that could create situations where different users with limited permissions could create their own cache tables (OK, no problem) and then users with extensive permissions could end up having to choose between multiple available cache tables, having to make the choice basically at random (not OK). The change makes things much more predictable. Whichever schema you connect to by default now is used for the cache.
Data sources (like web servers) set to cache data in the parent database will continue working without cache when cache tables cannot be created or are inaccessible.
All databases implement uniqueness requirements implied by BTREExxx indexes using UNIQUE indexes instead of UNIQUE constraints. The change has been made because using UNIQUE constraints could backfire in some situations. Using UNIQUE within indexes matches more closely semantics currently in use by the system. BTREExxx indexes created by older versions of 9 will continue to work as before.
The TileMaskExtract SQL function has been renamed to TileMask.
The TileMask query function now produces a numeric tile with 0/1 values (was producing a tile with boolean values).
The TileMaskReplace query function now accepts mask as a numeric tile (was accepting a tile with boolean values). Pixels whose mask is either missing or 0 are made invisible in the result. Other pixels are made visible. Pixels that were previously invisible but have become visible are set to 0.
The NOT, AND, OR, and XOR operators for tiles now accept numeric tiles with 0 or any other values: 0 is interpreted as FALSE and any other value is interpreted as TRUE. The operators produce a numeric tile with 0 or 1 values.
The comparison operators for number/tile, tile/number, and tile/tile combinations now produce a numeric tile with 0 or 1 values.
The Compare query function for number/tile, tile/number, and tile/tile combinations now produces a numeric tile with -1, 0, or 1 values.
The TileCompare query function has been removed, since the Compare query function now does everything the former TileCompare function did and more.
The BETWEEN operator for tiles, that is, used in cases where at least one of the operands is a tile and operands that are not tiles are numbers, now produces a numeric tile with 0 or 1 values.
Fixes
(Fix) ALTER TABLE no longer can rename a schema item of the wrong type. Previously, attempting to rename, for example, an index did not check whether the schema contained an index with the specified name; instead it just checked that the schema contained an item with the specified name. The operation could then rename an item of a different type, for example, a field. The fix now causes such an attempted rename to fail early.
(Fix) Creating a new non-primary index on SQLITE using a non-unique name automatically adjusts the name of the index to make it unique.
(Fix) Creating an identity field on GPKG no longer forces the identity field to be autoincrementing.
(Fix) Creating an RTREE index on GPKG fails if the index references a field other than the primary key field (such an index referencing a non-primary key field violates the GPKG spec).
(Fix) Data sources set to cache data in the parent database (for example, Bing Maps or some other web image server with the Saved cached data between session option turned on) no longer sometimes fail to work if the parent database is not a MAP file.
(Fix) Creating a new computed field or constraint no longer sometimes accepts an expression with extra tokens at the end. For example, b+100- (with a minus sign at the end) previously was accepted as b+100 (with no minus sign). The system now complains about the extra minus sign - character at the end.
Build 9.0.175 is a Base build. This build or more recent is a must have build for most users to replace all prior builds, for all Release 9 and Manifold Viewer users. Build 9.0.175 provides numerous improvements with substantial changes to many topics in documentation, with illustrations now totalling over 12,300 full color illustrations. For detailed lists of changes, see the build notes in the Changes and Additions topic for builds 9.0.174.1 though 9.0.174.4.
Starting with Release 9 build 9.0.175 and Release 8 build 8.0.31, Manifold® products use a new activation system. See the discussion in the New Activation web page.
Important: If you do not want to switch over to the new activation system, do not install and activate Release 9 build 9.0.175 or Release 8 build 8.0.31. Activating a newer build using a legacy serial number will automatically upgrade that serial number to the new activation system. That is a one way upgrade: Once upgraded, a serial number cannot be degraded back to legacy status.
While the new system provides great convenience for most people, some licensees may prefer to stay with the old activation system. If desired, licensees can stay with the old activation system by not installing new builds. To stay with the old system, continue using Release 9 build 9.0.174 and Release 8 build 8.0.30. See the Manifold Activation topic for more information.
Highlights of build 9.0.175 can be grouped into several main sections:
Labels: Numerous additions to labels, such as curved line labels to follow line segments, both on screen and when printed, along with numerous style features such as curved Unicode labels, mixing multiple languages, mixing right to left and left to right languages, multiple lines in labels, shadows, and much more.
Coordinate Systems: Thousands of updates for EPSG 10.19, new Natural Earth projections.
Dataports: Upgrades to GDAL, GDB, SQLITE / GPKG, Esri mobile geodatabases, LAS/LAZ LiDAR dataports, LAS library, and more.
User Interface: Drag and drop from Windows Explorer to import or to link files and data sources into the Project pane, ODBC configuration from About, better copy and paste into GDB and similar data sources.
SQL: Use of double quotes as delimiters, new functions such as GeomWithin, GeomOverlayWithin / GeomOverlayWithinFilter, faster GeomOverlay functions, dramatically faster unindexed joins, specification of field properties when creating tables using SELECT INTO and much more.
For details, see the listings below for builds 9.0.174.1 through 9.0.174.4. Improvements integrated into build 9.0.175 since build 9.0.174.4 include:
URLs used for Manifold sites in the product and in install packages have been switched from HTTP to HTTPS.
Build 174.4 provides several significant improvements for the query engine and SQL functions, which can provide dramatic speed increases in certain cases. Other new features provide improvements in ESRI file geodatabase (GDB) connectivity and LAS / LAZ LiDAR operations.
LAS / LAZ
Support for LAS / LAZ has been updated to use LASzip version 3.4.3. This fixes several bugs in encoding and decoding.
(Fix) Reading LAS files with extra field values no longer sometimes fails.
Esri File Geodatabase (GDB)
Esri file geodatabase (GDB) 64-bit modules have been updated to the version 1.5.2.308 SDK that was issued by Esri last week. The new version of the Esri SDK does not contain 32-bit modules, so they continue to be at level 1.5.1.253. Esri's new update fixes a few issues which rarely occur.
The GDB dataport attempts to work around an Esri bug that frequently occurs in geodatabases created by recent versions of ArcGIS Pro. The bug makes a newly created table fail to register completely. The dataport examines the state of the geodatabase and, if the table has been created and is operable, reports that the table has been created and marks it in the metadata to prompt execution of special workaround logic within Manifold that compensates for the table not being registered completely in the geodatabase.
The GDB dataport no longer reports indexes on tables returned by file GDB modules as BTREExxx, since Esri's indexes can only be efficiently used to search for individual records, with other operations being unoptimized. This lets the query engine optimize around not having these indexes instead of being tricked into using them with subpar results. Built-in Esri indexes on the OBJECTID field continue to be reported as BTREE, with all operations optimized internally by the dataport. Using Manifold optimizations can result in dramatically faster operations than using limited Esri indexes.
Exporting a drawing to GDB no longer creates a feature data set. Creating a feature data set for each drawing, for example, when exporting a map with multiple layers, is not necessary and often is not desired by the user. Manifold now only creates a feature class and leaves adding any desired feature data sets up to the user.
Query Engine
The query engine allows enclosing identifiers in double quotes " characters, in addition to square bracket [ ] characters and back tick ` characters. Double quote characters were already allowed as delimiters in the ODBC driver, because double quotes are the default for many tools. This added capability extends the use of double quote " characters as delimiters for all queries, such as those run interactively or through scripts. This helps interoperability when adapting SQL copied from other systems.
The query engine optimizes unindexed INNER / LEFT / RIGHT joins with a condition such as: f(a) = b.x (where a and b are tables and x is a field. That is a very common condition, so the optimization has a big effect. The performance gains depend on the number of records in the involved tables, but are typically really big, similar to the performance difference between indexed and unindexed joins. This optimization helps a lot because while it is always advisable to build appropriate indexes on fields that are used in queries, that is not always possible or desirable. For example, a table we might want to index may reside in a data source that does not support indexes, or we might not have permissions to alter the data source by adding an index. Adding an index to such a table would mean making a copy of it and indexing that, which can be both time consuming and inconvenient.
The query engine writes into createdname and creatednamequoted pragmas the final name of a table created by SELECT INTO, for use by further statements that might want to create components linked to the created table. This is similar to what is done for CREATE.
The query engine allows specifying properties for the fields of a table created by SELECT INTO, for example, using the syntax SELECT ... INTO t (PROPERTY ..., ...) ...; This allows using SELECT INTO with data sources like GDB or GPKG which require additional configuration for geometry fields at the time the table is created. Typical properties specified on creation of tables in such data sources are the coordinate system, the geometry type, whether the geometry is 2D or 3D, and so on. For example, when creating a table within an Esri file GDB if we want to store areas in that table we must specify the FieldGeomType.geom property to have a value of area:
--SQL9
SELECT [area], [code], [name], [geom]
INTO [gdb]::[states] (
PROPERTY 'FieldGeomType.geom' 'area' -- the default would be 'point'
) FROM [states];
SQL Functions
The GeomWithin query function optimizes tests for zero distance.
There are new GeomOverlayWithin and GeomOverlayWithinPar query functions that take a primary drawing, an overlay drawing, a distance in the coordinate system of the primary drawing, and return all pairs of geoms in the primary drawing and the overlay drawing that are within the specified distance.
There are new GeomOverlayWithinFilter and GeomOverlayWithinFilterPar query functions that take a primary drawing, an overlay drawing, a distance in the coordinate system of the primary drawing, and return all geoms in the primary drawing for which there is at least one geom within the specified distance in the overlay drawing.
The GeomOverlayXxx (non-topology) query functions skip normalizing geoms whenever possible for performance. This frequently produces a significant performance gain, particularly for cases when the overlaid drawings have many overlaps.
Build 174.3 provides improvements for LiDAR data use in the LAS and LAS library dataports as well as various system improvements. Several other development branches (for example, with various improvements to labels) have been maintained during recent builds: changes from these branches will appear in the near future.
Welcome text in the gray background area for the Manifold desktop can be turned off in Tools - Options.
Layout windows supports keyboard shortcuts to set cursor modes:
Default = Shift-Space,
Create Text = Shift-T,
Create Area Sample = Shift-A,
Create Color Sample = Shift-C,
Create Line Sample = Shift-L,
Create Point Sample = Shift-P,
Create Text Sample = Shift-X.
The LAS dataport exposes data within built in and custom LAS headers as Info and InfoVLR tables.
The LAS library dataport exposes data within built in and custom LAS headers from individual files as Info and InfoVLR tables. Both tables contain an additional field for the file path.
Processing a folder of LAS files using the Index LiDAR Files tool caches header data in MAPMETA files. MAPMETA files composed by older versions of the tool that contain no header data are accepted and updated on the fly.
Exporting data to LAS or LAZ format sets the name of the generating software package to Manifold System followed by the build number.
Activation dialogs use a short version of product names in the title, and use the product name in text prompts. This helps make it clear which product is being activated on a system with multiple Manifold products installed.
MSI install packages use product-specific banner images.
There are two new projections: Natural Earth and Natural Earth II. These are modern projections designed to better portray the entire Earth.
There is a new command line switch to install the ODBC driver or update it to the desired location. Running the command requires administrator privileges:
manifold.exe -odbcupdate
There is a new command line switch to remove the ODBC driver. Running the command requires administrator privileges:
manifold.exe -odbcdelete
The About dialog allows updating or removing the ODBC driver from a non-administrator account. Attempting to update or remove the ODBC driver from a non-administrator account asks the system to perform an action with administrator privileges, which in turn prompts the user for administrator credentials using a system screen.
Copying and pasting a table with geometry data from a MAP file into a data source that does not allow mixing geometry types within a single table (for example, GDB) automatically sets the geometry type of pasted data to that used by most records.
(Fix) Cutting edge builds of Manifold Viewer are now correctly reported as '... Beta' builds.
Illustrations above: The Natural Earth (at left) and Natural Earth II (at right) projections. Natural Earth II is a more rounded projection, with smaller polar regions.
Build 174.2 is a quick build to enhance label capabilities, along with some database and UI improvements and miscellaneous fixes.
Labels
Rendering curved line labels renders halos.
Placing a line label prioritizes placing the label near a straight line segment that is long enough to fit it. If there are multiple such segments, the system uses the segment closest to the center of the line.
Placing a line label avoids placing the label onto a sequence of segments with high curvature (large angles between segments), to improve legibility. The maximum bend angle is edited in the Label Style dialog (in the Position tab, accessed from the More... choice at the bottom of the pull down menu for the total style button). The default maximum bend angle is 60 degrees. Setting the maximum bend angle to 0 places labels near straight line segments only.
Labels with multiple lines of text, with a box, or with an icon are always placed near points or near straight line segments.
Reduced graphics mode supports labels with multiple lines of text.
Rendering labels with multiple lines of text uses the alignment specified in the in the Label Style dialog (in the Position tab, accessed from the More... choice at the bottom of the pull down menu for the total style button). Available choices are: left (default), center, right.
Other
EPSG coordinate systems that switched bases (5 cases) and coordinate transforms that switched source or target systems (3 cases) in the latest version of the EPSG database have been adjusted to coincide with their EPSG definitions.
The gray background area of the main Manifold desktop window shows a dimmed Manifold logo and dimmed text with tips for new users.
The context menu for a component in the Project pane includes a new Open command, which opens the component.
Creating a primary key on an existing text field in PostgreSQL always uses the collation of the field. This became a requirement in PostgreSQL 13. Secondary indexes continue to be able to use collations different from those of the fields.
Creating an index on an existing text field in MySQL or SQL Server always uses the collation of the field and does not attempt to change it. Previously, adding an index on a field that was not part of any other index attempted to change the collation of the field to that specified for the index. The system no longer does that and instead shows, for a likely edit, the collation of a field directly in the user interface.
Fixes
(Fix) Importing data automatically skips tables that cannot be accessed, for example, due to permissions. This fixes the regression with MDB import in the previous build.
(Fix) Opening a table that cannot be accessed, for example, due to permissions, no longer creates a fetch record (row with a small "more" down triangle at end of tables) in an otherwise empty table window.
(Fix) Double-clicking a word in a text window no longer sometimes fails to select it if the text has more than 64k characters.
(Fix) Nested include directives in queries resolve from the current data source instead of from the data source of the topmost query. (This allows composing a query with includes, then including that query from within a different data source with nested includes resolving correctly.
(Fix) References to components in included queries now resolve from the current data source instead of from the data source of the topmost query. This allows composing a query which references a script function, then including that query from within a different data source with the script reference resolving correctly.
Build 174.1 introduces rendering improvements and provides a starting implementation of curved line labels, including improvements in Unicode / multi-language labeling for all labels. The build also provides significant improvements in importing and linking files, such as drag and drop files from Windows Explorer to import or link, as well as upgrades to EPSG. Upcoming builds will expand on labels, add new features for vector editing, and add various small improvements throughout the system.
Rendering and Labels
Label placement on lines only considers those parts of line branches that appear on screen. (Previously, in cases where some branch parts were invisible a label might have been placed partly off screen.)
Label placement on lines has been reworked to place labels at the center of each visible part of a line. Labels can follow multiple consecutive segments. (Previously, labels were placed on line segments that were long enough to fit the label, a too restrictive constraint.)
Curved labels, that is, labels following multiple line segments, are rendered with the following supported / not supported label features:
Unicode features, including multiple code points collapsing into a single glyph, a single code point expanding into multiple glyphs, all types of whitespace, etc - supported,
Mixing multiple languages in the same label, including switching fonts - supported,
Right to left text, mixing left to right and right to left fragments - supported,
Multiple lines in label text: Normally curved text uses a single line, but if curvature is not high, the results with multiple text lines are reasonably legible), - supported,
Font face, font weight, font italics - supported,
Shadows - supported,
Underline, strikeout - not supported,
Halos - pending (not currently supported but will be added),
Icons - pending (not currently supported but will be added),
Underline - not supported (continue to be supported for single line segment labels),
Strikeout - not supported (continue to be supported for single line segment labels),
Boxes - not supported (continue to be supported for single line segment labels),
Curved labels are supported for printing, with optimizations for high resolution printing and large print areas.
Labels following multiple segments are not supported in reduced graphics mode.
Label placement on lines slightly simplifies line metrics, to avoid label text distortion from overly close line following. Upcoming builds will allow turning this feature off or specifying tolerances. Upcoming builds also will allow line label placement relative to the line, such as above or below the line.
Drag and Drop / Import and Link
Dragging and dropping files from Windows Explorer into the Project pane imports files, and dragging and Shift-dropping links those files. The cursor automatically changes to show import or link when pressing the Shift key during the drag.
Dropping files into a child data source confirms the action. If the confirmation is turned off, it can be turned back on using the Confirm dropping files into child data source option in Tools - Options.
Dropping files into a child data source that has not been opened yet or is read-only shows an error.
Dropping a single MAP or MXB file opens the file as long as the user is not trying to link it (by holding Shift).
See the 10 Minute Tutorial - Drag and Drop video for examples.
Importing or linking a VRT file (*.VRT) automatically invokes GDAL / OGR. GDAL must be installed for this facility. A VRT file is a text file that specifies which GDAL / OGR driver to use and with which parameters, similar to DSN for ODBC or UDL for OLE DB.
The Link command logs the names of processed files, same as Import.
Other
The Edit dialog for editing text can be resized. Tab size for text in the dialog has been reduced to 4 characters from 8.
EPSG coordinate systems have been updated to EPSG 10.19. Rare exceptions are a few cases where an EPSG coordinate system was redefined to change its base coordinate system (!). Those changes have not yet been accepted, but will be accepted in upcoming builds in the near future.
A new coordinate system type has been added: Lambert Conformal Conic (Single Parallel, B).
Activation dialogs have been renamed to include Manifold 9 in the caption. This allows distinguishing them from Release 8 activation dialogs, in cases where a user may have both 8 and 9 installed on a machine and has launched a yet-to-be-activated Release 8 installation.
The GDAL / OGR dataport supports GDAL 3.2.x (GDAL302.DLL).
Writing a coordinate system to WKT or WKT2 (for example, using a Transform template) writes authority codes for EPSG systems. Parsing a coordinate system from WKT or WKT2 reads authority codes for EPSG systems: if the definition of the parsed system matches that of the referenced EPSG system, the system will add the EPSG code to the parsed system, to allow using transforms for that system specified in the EPSG system, for example, transforms based on grid files.
The SQLITE and GPKG dataports support deleting table fields.
The SQLITE and GPKG dataports support renaming table fields.
The SQLITE and GPKG dataports support ESRI ST_GEOMETRY extension for all operations if it is installed, in both 32-bit and 64-bit modes.
Refreshing a table within a database automatically reloads table metadata from the database to catch changes done in other sessions, for example, in a different instance of Manifold or other client connected to the same database.
Attempting to open a MAP or MXB file from the list of recently opened files removes the file from the list if the attempt to open fails.
Migrating a compressed image from a MAP file created by Manifold Release 8 puts the compressed image into a new ECW / JP2 file with a stable name, to avoid creating a new such file on each migration.
Migrating an image from a MAP file created by Manifold Release 8 no longer creates a Level field in the image table, to avoid making the image unusable in many transforms.
Fixes
(Fix) The Erase command in a map window when keeping the inner part no longer fails to erase data far from the edited geometry.
(Fix) Matching a bracket in a text edit control using Ctrl-] or Ctrl-Shift-] no longer fails to scroll the caret into view if the match is offscreen.
(Fix) Parsing a coordinate system from WKT or WKT2 no longer misparses systems with different units for XY and Z.
(Fix) The application window no longer fails to show non-ANSI characters in the caption.
(Fix) Opening a MAP or MXB file with non-ANSI characters in the path no longer fails.
Build 9.0.174 is an Official build. This build or more recent is a must have build to replace all prior builds, for all Release 9 and Manifold Viewer users. Build 9.0.174 provides hundreds of improvements with substantial changes to hundreds of topics in documentation and illustrations now totalling over 12,100 full color illustrations. The online user manual now uses Google search within the manual. For detailed lists of changes, see the build notes in the Changes and Additions topic for builds 9.0.173.1 though 9.0.173.6.
A preliminary description of 9.0.174 follows. Highlights can be grouped into several main sections:
Registration: There is a new Register pane which allows georegistering (georeferencing) raster or vector data using control points. Control points can be entered using the cursor or by typing coordinates directly. Control points can be saved to a component and loaded back. There are multiple registration methods available: numeric methods of varying orders, including methods with restrictions and projective generalization, thin-plate spline, and triangulation. The pane can show registration errors for individual control points, as well as the total registration error. Reported errors are updated dynamically as control points are added, deleted, edited, or turned on and off. The results of the registration can be previewed.
Editing: There are new commands for editing vector data. Clip trims the edited geometry to objects in the specified layer, keeping parts that are either inside or outside of the objects. Erase deletes parts of objects in the specified layer that are either inside or outside of the edited geometry. Merge combines the edited object with other objects in the active layer, merging field values according to user-specified transfer rules. Split cuts objects in the specified layer into parts using the edited geometry as a cutter line, producing new objects and splitting field values according to user-specified transfer rules.
Analysis and Queries: Gravity (IDW) interpolation allows specifying interpolation order. There is a new interpolation method for thin-plate spline. There is a new interpolation method for natural neighbors. There is a new Spatial select template which allows selecting geometry using a spatial criteria. There are new query functions for registration and interpolation, including a parallel variant for gravity (IDW) interpolation, and a new function to filter pixel values in a tile. The DROP statement tolerates missing components.
Dataports: Image libraries load faster, and report the number of files matching the mask for diagnostics. LAS / LAZ and LAS libraries expose shifts and scales for coordinate values in metadata. LAS libraries load faster, and can be refreshed with automatic updates to the coordinate system and similar data. Exporting data to LAS / LAZ embeds the coordinate system into the file, always produces LAS 1.4 with the record type determined from the available fields, and does not require any fields apart from geometry. PostgreSQL and Jet databases support boolean fields. Multiple databases support UUID fields either via a native database type or as a fixed-length string. MySQL databases support unsigned integer fields.
Components: CUDA has been updated to 11.2.2, GPGPU modules have been recompiled. ECW has been updated to 5.5.0.2034 for several minor fixes. WEBP has been updated to 1.2.0 for several minor fixes and better compression quality. SQLITE has been updated to 3.35.3 for new query clauses and functions as well as improvements to the query planner. ICU has been reworked to use the built-in Windows 10 implementation, expanding support for non-English languages and reducing install size. V8 Javascript engine has been removed with V8 scripts mapped to JScript.NET, reducing install size.
Build 173.6 provides a big batch of updates to external libraries, major enhancements for vector editing and the Select pane, major updates for LAS and LAS libraries for LiDAR data, updates for databases, improvements in queries, and bug fixes. The general direction for builds after that will be editing and labels. Build 173.6 is a candidate for the next official build.
External Libraries
Vector Editing and Select Pane
LAS/LAZ and LAS/LAZ Library Dataports for LiDAR
Databases
Queries
Fixes
This build contains various improvements and bug fixes all over the system, getting ready for an upcoming build, probably the next build, as an official release. The general direction for builds after that will be editing and labels.
Important: Runtime Libraries Updated
Build 173.5 has switched to the new version of Microsoft's Visual C++ compiler and new versions of Microsoft Visual C++ runtime libraries. Since cutting edge builds use portable installs, using build 173.5 or subsequent portable installations requires upgrading Visual C++ runtime libraries to the new versions.
Download the new versions from Microsoft's latest supported Visual C++ downloads page.
Changes
Error values reported in the Register panes are scaled to the unit of the target coordinate system unit. Previously, if the coordinate system had local scales set to, say, 5 feet by 5 feet, for example, for an image, the error values were in pixels, that is, "5 foot units". Now they are scaled to plain feet.
When the Register Pane's list display style is set to Show Errors, a total error (RMS) readout at the bottom of the pane will report the total RMS registration error, together with the unit of measure.
The Info pane reports pixel size for an image.
Map windows shows a snap reticule in Edit Control Points mode when snapping is turned on.
Map windows provide a context menu in Edit Control Points mode that allows editing snap options.
The DROP statement in queries has been adjusted to do nothing and to return 0 in the result table if the dropped component does not exist. If the dropped component exists, but is of a different type than that specified in the statement, for example, when writing DROP TABLE but citing the name of a drawing, the statement fails with an error.
Map windows provide a the new Edit Coordinates command in the context menu for a picked object in Values mode. The command switches the object into coordinates editing mode, also switching the Info pane to the Coordinates tab to make coordinates editable in the pane. This is a helper command for beginners: Experienced users will simply click a vertex or segment in a picked object to switch into editing mode, or, if the mouse is over the Info pane, will simply click the Coordinates tab.
Attempting to register an image using triangulation now works even if control points cover only part of the image.
Map windows show and allows editing the coordinates of a path in Draw Path mode (formerly tracker mode) in the Coordinates tab of the Info pane. See an example in Measurements topic. The coordinates of the path are in the coordinate system of the active layer. For example, if the active layer is an image, coordinates will be pixel coordinates in that image.
The layout window automatically repaints after changes to layer states maps shown in layout frames. Changes to layer states includes changes such as adding or removing layers, turning layers on and off, reordering layers, and so on. Automatic repaints are not currently triggered by changes to the data inside layers. For example, after adding a new object to a drawing, the map showing the drawing will update, but the layout showing the map will not update. Automatic repaints in layouts on data changes will likely be added in future builds. For now, pan the layout screen slightly or turn the map frame off and back on to trigger a redraw.
Exporting a drawing to LAS or LAZ embeds the coordinate system for the drawing into the produced file. If the coordinate system info is WKT or WKT2, it is used as is. Otherwise it is converted to WKT.
Linking a LAS or LAZ file exposes shifts and scales for X, Y, and Z fields as properties in the linked drawing's table. Exporting to LAS or LAZ format uses this data to produce a file with the exact same shifts and scales as the original. Important: The above feature was introduced in build 173.5. When linking to LAS or LAZ files previously used in Manifold builds before 173.5, delete any .mapmeta file in the same folder that were created by any builds before 173.5. That will prevent the old .mapmeta file from limiting the properties created to only use used in builds before 173.5.
Table windows now provide a new Export Selected command in the Edit main menu. Choose Edit - Export Selected to export selected records to a file.
Fixes
(Fix) Exporting to LAS now honors the specified extension and writes either LAZ if the extension is .laz or LAS if the extension is not .laz. If the extension is not specified, the export writes LAS. (Previously, exports were forcing forcing the extension to .laz and exporting LAZ.
Recommended New Topics
5 Minute Tutorial - Editing with Clip - The new Clip interactive editing command makes it easy to create areas that automatically follow boundaries of existing polygons, to create complex new areas. It's also easy to create new lines that are clipped to complex boundaries. We see how in this short video.
We also see how to combine the Clip command with the new Merge editing command to edit polygons "in place" in whatever data source we want, including ESRI geodatabases, PostGIS, data sources, GPKG, shapefiles or any of hundreds of other data sources.
Newsflash - Merge, Clip, and Split - See a three minute, video providing a first look at how to use fast and easy, new editing commands in Manifold Release 9.
5 Minute Tutorial - LAS Point Statistics by Area - Save thousands by using Manifold Release 9: The "LAS Point Statistics by Area" tool in Spatial Analyst for ArcMap and ArcGIS Pro adds attribute fields to polygon features for LiDAR points that fall within those polygons, adding the minimum, maximum, and mean Z values in each polygon, as well as the point count of LiDAR points in each polygon, and also the standard deviation of Z values in each polygon. The video shows how to do the same thing in just a few clicks in Manifold's point-and-click Join dialog. No need to buy Spatial Analyst. Manifold includes the Join dialog and thousands of other capabilities, all for only $95. Save thousands of dollars and enjoy easier workflow too!
5 Minute Tutorial - Scrape an ESRI Feature Server - Working with web-based ESRI feature servers? If you want your own copy all the vector features (points, polylines, polygons) those servers provide that's easy to do with Manifold Release 9. This video shows how to connect to a typical ESRI feature server, how to drill down and open the feature layer desired, how to copy all the features and to paste them into the local project. It's all a simple matter of point and click, copy and paste.
5 Minute Tutorial - Create a Multiband Landsat Image - Landsat data provides images in individual bands, such as Bands 4, 5, and 6, for visual red, near infrared and shortwave infrared. Creating large, multiband Landsat images using many Landsat tiles can involve combining dozens of individual band images. Manifold Release 9 is perfect for the job, because it can handle many large rasters effortlessly, merging them instantly, and then combining bands into false color RGB multiband images. Manifold does all that with easy to use, point and click dialogs. No need for programming. This fast-paced video shows how in under five minutes of workflow, including importing the original Landsat data downloaded from USGS. All this works in the free Manifold Viewer too.
This build contains extensions to geometry editing and raster interpolation.
Editing
Map windows support several new commands in the context menu used for editing vector geometry:
Suppose we would like to draw a single polygonal area that automatically fills in the regions between building footprint polygons in a neighborhood.
Choose Create Area and in the usual way draw a polygonal area that outlines the neighborhood. Right-click and choose Clip from the pop-up edit menu, and choose OK. Instantly, the provisional area outlines are automatically redrawn to follow the boundaries of building polygons within. We can further edit those if we like, or proceed with a single click to create the new area.
Commit the edit (a single click or Ctrl-Edit) and Manifold auto-completes the new polygonal area to fill in the neighborhood, with open regions that exactly fit the building polygons. We can Cut that new area and Paste it into a new layer to see it without the building polygons, to see the fantastic complexity of the created polygon, created in seconds.
Interpolation
Other
Many new videos have appeared to cover the spectacular new georeferencing features:
5 Minute Tutorial - Georegistration - In just five minutes we learn how to georegister (georeference) a vector drawing with an unknown coordinate system to a known-good map. Georegistration is a key capability that allows us to cast raster images and vector drawings into geographic context, so they can be used as GIS layers in maps. We can georegister aerial photos and drone photos, scan paper maps and georegister those for use in GIS, we can georegister CAD drawings, and we can rescue vector drawings and raster images that once had coordinate systems but were published in formats that failed to preserve coordinate systems. Super! Works in the free Viewer, too.
5 Minute Tutorial - Georegister a Drone Photo - See the fast and easy way to georeference drone photos for use in GIS and online web mapping: Learn how to georegister (georeference) a drone photo to line up with Google imagery for full GIS use and for use within Google Maps and other web mapping applications. This video uses exactly the same drone photo used in ESRI's ArcUser example of how to georeference a drone photo in ArcGIS Pro. The difference is that using Manifold is faster and easier.
5 minute Tutorial - Georeference Many CAD Layers - Georeferencing CAD layers is a common task in any GIS. Manifold makes it a lot easier with fast, simple workflow that avoids extra effort and lets us recycle what we've already done. This video shows how we can add a few control points just once and then georeference an entire stack of CAD layers imported from a DWG without adding more control points or repeating any work.
Georeference a Scanned Paper Map - In only five minutes of actual work we use Manifold to georeference a 157 MB scanned paper map so it can be used as a layer in GIS. The scanned map is a historic map showing Davy Crockett National Forest in Texas, downloaded from the Library of Congress website.
Georegistration - Save and Load Control Points - Georegistration (georeferencing) in Manifold uses control points to match features visible in the raster image or vector drawing to be georeferenced with corresponding features visible in a known-good reference. This video shows how with a single click we can save or load control points. Because Manifold saves control points as ordinary vector drawings, we can take advantage of that to make mass changes to control points if we like. In the video we use two versions of a scanned map, one with a gap in the middle and in the other where the gap is closed. Control points that were placed in the version with a gap can be easily adjusted, dozens at a time, for use in the other version, saving a lot of repeated work.
Georeference a Historic Map using a List of Cities - Georeference a scanned paper map downloaded from the Library of Congress that shows slave populations in Southern States in the 1860 census. The scanned map shows locations of cities, which we will use as control points. We create a drawing to quickly mark the locations of cities in the scanned image. Next, we download a modern map of cities in the US and their locations. We can use the list of cities in the modern map as a source of control points for the target, saving us from having to enter them manually. Manifold automatically matches names, ignoring those that are not needed, from the modern map during the georegistration process.
Georeference a Whole World Image - See how to georeference an image scraped from the web that shows the geology of continents as they were 200 million years ago.
We mark four control points in the image, then we roughly mark four corresponding control in a Manifold map using Bing as a background layer. In the Register pane we edit the coordinates of those control points to be even +/- 90 and +/- 180 degrees, and then we press Register. Done! The video also shows how we can import and georeference a second image similar to the first, without needing to add any control points, just re-cycling the ones we created before.
This build provides incremental improvements to registration, including implementing user requests from the past month of using the Register pane.
See the updated Register Pane topic for discussion and illustration of new features listed below. Next: improvements to geometry editing, and various quality of life features.
The Register pane allows selecting fields to show in the control point list, using a new menu button next to the toolbar:
Show Marks - (the default) Show the control point name, a coordinate mark (icon), and an off/on toggle box.
Show Coordinates - Show the control point name, X coordinate, Y coordinate, and an off/on toggle box. X and Y coordinates can be edited.
Show Errors - Show the control point name, the error for that control point, and an off/on toggle box. Applies when the Register pane is showing control points for a target component. Any changes to control points (adding or deleting control points, changing their coordinates, toggling them on or off) as well as changes to the registration method automatically update error values.
The Register pane supports several new registration methods:
order 1, shift + uniform scale + rotate - An affine registration with no shear and with scale forced to be the same scale in both X and Y.
order 1, shift + uniform scale - As above, but without any rotation.
order 1, projective - A generalization of affine, called projective in ESRI products.
The Register pane allows specifying rendering options for images:
forward, average - The image is composed by traversing the source image, converting source pixels to the target and computing a weighted average of the results. This is analogous to the former direct method in the Reproject Component dialog. The Pixel divisions parameter controls quality: higher values produce results of higher quality, but are more computationally expensive.
forward, nearest - As above but with nearest neighbor instead of averaging. This is a new option developed to allow registering images with values that cannot be averaged. It allows using any registration method, even if it is not invertible.
inverse, bilinear - The image is composed by traversing the target image, finding corresponding source pixels and using bilinear interpolation on their values. This is analogous to the former standard method in the Reproject Component dialog.
inverse, bicubic - As above but with bicubic interpolation.
inverse, nearest - As above but with nearest neighbor, used when the image stores values like classification codes that cannot be averaged.
Inverse options are only supported for registration methods that are invertible: order 1, affine and other order 1 variations.
The Reproject Component dialog for images has been reworked to specify the same rendering options for images as the Register pane.
The default number of pixel divisions used for the forward, average rendering option in both the Register pane and the Reproject Component dialog has been set to 2. This is a better default trade-off in terms of quality versus processing speed than the former value of 1.
The CoordConverterMakeWarpNumeric query function has been reworked to take a conversion type instead of conversion order as the second parameter, with the following values:
0 = auto (currently maps to: order 1, affine),
1 = order 1, affine,
2 = order 1, projective,
3 = order 1, shift + uniform scale,
4 = order 1, shift + uniform scale + rotate,
5 = order 1.5, affine + cross-product,
6 = order 2,
7 = order 3.
The CoordConvertTileSetDirect and CoordConvertTileSetDirectPar query functions allow setting the number of pixel divisions to 0 to compose tiles using nearest neighbor instead of weighted averages.
The CoordConvertTileSetDirect and CoordConvertTileSetDirectPar query functions limit the number of pixel divisions to 8 to prevent against erroneously passing an unreasonably high value and spending a lot of time performing unnecessary computations.
This build contains improvements to registration with significant enhancements to the Register pane. Next: incremental improvements to registration, improvements to geometry editing, and various quality of life features.
New Georegistration help topic, with extensive additions to the Register pane topic.
There is a new registration method, thin-plate spline, which can be used with both drawings and images.
The (no source) caption in the Register pane has been renamed to (current window)
Illustration: Georegistering a scanned, paper Elliott map of the Gettysburg battlefield using 37 control points and the thin-plate spline method. OpenStreetMap is used as the background layer in the target window.
When control points are selected in the Register pane, the map window highlights selected control points as well using dark blue color.
The map window allows selecting individual control points using Ctrl-click. Shift-Ctrl-click deselects. We can click either the location of a control point or the label for that control point.
The map window allows selecting multiple control points using Ctrl-drag. Ctrl-drag selects all control points within the drawn box. Shift-Ctrl-drag deselects all control points within the box. Ctrl-drag and Shift-Ctrl-drag boxes ignore labels and only use control point locations. Control points selected in the map window are selected in the Register pane as well.
The default names of control points have been changed to using a shorter name stem of P xxx. The first control point is named P 1, instead of just P.
Control points for a target window in the Register pane can be turned off and on using check boxes in the Register pane, like check boxes in the Layers pane. By default, all control points are turned on. Turning a control point off hides it in the map window and will not use it during registration. Changing the location of a control point in the map window turns it back on if it was turned off.
Illustration: The thin-plate spline method matches control points exactly. Once the Elliott map image (original at left) has been georegistered (at right), we can use it with endless other GIS layers, such as a terrain elevation layer created from one-meter resolution LiDAR data, as seen above. Partial opacity in the georegistered Elliott layer blends with the terrain elevation layer to provide hill shaded relief and better understanding of the Elliott map information. Features have been labeled in a third layer.
The Register pane includes a toolbar with a Delete command. In prior builds we could delete control points selected in the control point list by using the Delete key. Now we can also click the Delete toolbar button with no need to consider keyboard focus.
The map window allows selecting multiple control points using Ctrl-drag. Ctrl-drag selects all control points within the drawn box. Shift-Ctrl-drag deselects all control points within the box. Ctrl-drag and Shift-Ctrl-drag boxes ignore labels and only use control point locations. Control points selected in the map window are selected in the Register pane as well.
The default names of control points have been changed to using a shorter name stem of P xxx. The first control point is named P 1, instead of just P.
Control points for a target window in the Register pane can be turned off and on using check boxes in the Register pane, like check boxes in the Layers pane. By default, all control points are turned on. Turning a control point off hides it in the map window and will not use it during registration. Changing the location of a control point in the map window turns it back on if it was turned off.
The Register pane has a new Saved Control Points toolbar button, which allows saving current control points. Saving control points creates a new drawing and a new table in the main part of the map project. The new table includes an identity field and index, a geometry field and index, and a name field. Control points for a target window with no locations provided are not saved. The created drawing is marked as containing control points (via the FieldNameRegister property of the drawing) and is selected in the Project pane. The FieldNameRegister property of the saved control points drawing specifies the name of the field in the drawing's table which contains the names of control points. The coordinate system of the saved control points drawing is the same as that of the window for which it was created, either the source window or the target window.
The Register pane allows loading current control points from previously saved control points drawings using the pull down menu for the Saved Control Points toolbar button. Drawings with a FieldNameRegister properties will appear in the top part of the pull down menu list as possible choices. Records with empty geometry or with geometry other than single points are ignored, so areas, lines, and multipoints are ignored. Records with empty names are ignored. Records with repeating names are ignored after the first such record encountered (the choice is random). The total number of loaded control points is limited to 1024, with only the first 1024 control points loaded and any additional control points ignored. This allows using drawings containing points as saved control points drawings, even if there are more than 1024 points in the drawing.
Illustration: The region of PIckett's Charge, an uphill Confederate infantry charge over open fields against Union artillery. The Elliott map original (at left) shows the locations of battlefield graves in the week after the battle, with each small line a Confederate grave, and each crossed line a Union grave, graphically illustrating staggering Confederate casualties. The georegistered image (at right) appears in partial opacity above a Google Satellite layer to show the location of burials relative to modern development. Although most battlefield burials were moved to memorial cemeteries, some graves remain where soldiers from either side fell.
We can use any drawing containing points as a saved control points drawing, as long as it has a text field giving the name of points, a FieldNameRegister property specifying the name of that text field in the drawing's table, and a usable coordinate system. For source windows, the coordinate system of the drawing to be used as a saved control points drawing must be the same as the coordinate system of source window in which it will be used. Target windows are already correctly georegistered, so drawings that will be used as saved control points drawings for target windows can be in any coordinate system: when loaded, the control points will be converted into the coordinate system of the target map window.
Numeric registration (order 1 through order 3) no longer attempts to solve a warp when there are too few control points. Instead, it quickly fails safe. Previously, the process would use one of the possible solutions, which is not desired.
Numeric registration solves a warp exactly if possible. Previously, the process used a least-squares fit. That lost some accuracy in cases when the warp could have been solved exactly. The process has been extended to achieve an exact solution whenever possible.
New query function: CoordConverterMakeWarpThinPlateSpline - Takes a line geom containing warp coordinates and returns a coordinate converter object that performs the warp.
(Fix) RTREE indexes in MAP files no longer reduce single-point geoms with XYZ data to XY in output for spatial searches. The fix does not apply to already existing indexes, but it does apply to new records inserted into such indexes.
(Fix) RTREE indexes in MAP files no longer reduce multi-point geoms which are repeats of a single coordinate to single-point geoms in output for spatial searches. The fix does not apply to already existing indexes, but it does apply to new records inserted into such indexes.
This build is dedicated to Georegistration. Georegistration is the process of adjusting a raster image or vector drawing to match the coordinate system and conformation of a known-good reference component, such as a Bing or Google web served map, or other image or drawing that we know has a correct coordinate system, is located in the correct geographic location, and shows geographically accurate features. Georegistration is also called georeferencing in ESRI and other GIS products. Manifold uses georegistration and georeferencing as synonyms.
See the 5 Minute Tutorial - Georegistration video for a quick start on georegistration, as well as the step by step Example: Georegister a Vector Drawing and Example: Georegister a Drone Photo topics.
There is a new Register pane, seen at right. Keyboard shortcuts for activating panes have been adjusted to: Ctrl-1 = Project, Ctrl-2 = Info, Ctrl-3 = Layers, Ctrl-4 = Register, Ctrl-5 = Select, Ctrl-6 = Style, Ctrl-7 = Transform.
The registration process uses two windows: one window for the component being registered (source) and another window for the desired, known-good location (target). Each window has its own set of control points. Control points have names, which are incrementally numbered.
Initially all windows work as source windows.
With the focus on a window, choosing a source window in the Register box in the he Register pane will make the context window a target window, allowing it to work as a known-good registration target.
When a source window is chosen in the Register box, the control points in the context, target window are synchronized with control points in the source window.
A target window cannot be a source window for some other target window, but a source window can be chosen in the Register box for more than one target window. If a source window is closed, all target windows for which it has been the source window return to being source windows.
The Register pane shows a list of control points for the active window. Each control point has a unique name and an optional location, shown using a + icon on the right. For a source window, each control point always has a location. For a target window, a specific control point may or may not have a location.
Map windows (including drawing and image windows) supports a new cursor mode for editing control points: Edit Control Points. The mode uses a distinctive cursor. Switching to Edit Control Points mode shows control points in the context window, and also pops open the Register pane if it was hidden.
For a source window, clicking in the map window places a new control point. The new control point is assigned a unique, numerically iterated name.
For a target window, clicking in the map window assigns the location for the control point that has the focus in the Register pane. If the control point already had a location assigned, the location will be changed to the new position that was clicked. After the click, if there are control points with no assigned locations, the focus moves to the first control point in the list that does not have an assigned location.
Painting control points in the map window automatically moves the names around points to try to stay on screen.
The Register pane allows editing names of control points for a source window. If the newly entered name is not unique, it is forced to be unique by adding a numeric postfix. The names of control points in a target window are automatically adjusted to be the same as those in the source window.
The Register pane allows selecting control points. Selected control points can be deleted using the Delete key. For a source window, deleting control points removes them from the set. For a target window, deleting control points clears their target locations.
After both the source and target control point sets are placed, the Register pane for a target window allows registering any layer from the source window to the target locations. The list of layers omits labels and includes only drawings and images. The Register button performs the registration, creating a new drawing or image component and selecting it in the Project pane.
The Preview button previews the results of the registration.
The Edit Query button shows a registration query in the Command Window. Running the registration query will have the same effect as pressing the Register button would.
Registering a component allows specifying the registration method in the Fit box. Available methods are:
order 1 (affine)
order 1.5 (affine + cross-product)
order 2 (square)
order 3 (cubic)
triangulation
The default registration method is order 1 (affine). Avoid using order 2 (square) and order 3 (cubic) with small numbers of control points. For those methods use at least 5 or six control points. Using triangulation will only register data covered by the convex hull of the control points. We can use a set of control points that does not cover a whole component for a drawing, but control points must cover an entire image when using triangulation.
Registering a drawing sets the coordinate system of the new drawing to that of the target window with local scales set to 1 and local offsets set to 0.
Registering an image sets the coordinate system of the new image to that of the target window with local scales adjusted to produce about the same number of pixels as in the original image and with local offsets set to the left-bottom corner of the image. The pixels in the new image are forced to be square.
Registering a component in the Register pane shows the time it took to perform the operation at the bottom of the pane. This helps confirm that pressing the Register button did have an effect when the operation ended up being too fast and the progress dialog never appeared. Registering a component also logs operation time in the log window.
New query function: CoordConverterMakeWarpNumeric - Takes a line geom containing warp coordinates and a numeric order for the warp, and returns a coordinate converter object that performs the warp. Each branch of the geom is interpreted to contain the source location as the first coordinate and the target location as the second coordinate. The numeric order is rounded to one of the following values: 1, 1.5, 2, 3.
New query function: CoordConverterMakeWarpTriangulation - Takes a line geom containing warp coordinates and returns a coordinate converter object that performs the warp.
(Fix) The minimum height of the template list in the Select and Transform panes has been increased. (Could become too small if a pane was undocked and reduced in size.)
Planned additions for upcoming builds: save / load control points, provide more control over registration parameters, and report registration error for each control point.
Build 9.0.173 is an Official build. This build or more recent is a must have build to replace all prior builds, for all Release 9 and Manifold Viewer users. Build 9.0.173 provides many hundreds of improvements with substantial changes to hundreds of topics in documentation and updates to over 9000 illustrations. For detailed lists of changes, see individual build notes below for builds 9.0.172.1 though 9.0.172.7.
Highlights can be grouped into several main sections:
Panes and Windows
Individual panes have been taken out of the Contents pane and are now standalone can be shown simultaneously and independently.
Windows and panes can be docked, moved between top and bottom rows, undocked or closed using the context menu.
Panes can also be moved between left and right side docking.
Floating windows and panes snap to each other when moved or resized.
New Info Pane:
The former Component and Record panes have been merged into the Info pane.
The pane shows various new data for components: dimensions for an image, language for a script, page size for a layout, editable descriptions for all components, and more.
Editing a record in a table shows record data into the Info pane: geometry values can be expanded to show coordinates and tile values can be expanded to show pixels in a separate tab. A new Related tab allows searching for records in a secondary table related to the picked record.
Illustration: Creating a buffer zone using a formula for the distance, with kilometers as units of measure, and then saving the result to a new geometry field in the table with a drawing automatically created to visualize that new geometry.
New Select and Transform Panes
The Select and Transform panes have been redesigned to use a new workflow that provides better use of screen space, greater persistence and the ability to more rapidly, iteratively reapply operations with changes in parameters and targets.
Both panes remember their state on a per-window basis, allowing switching between windows while retaining Select and Transform settings for each window.
Both panes show a list of last-used templates, and allow pinning last used templates for use between sessions, and to allow quick repeats of more popular commands.
The Transform pane now easily redirect results to the desired destination, including creating new components on the fly with easier editing of names directly in the Transform pane.
Many transforms that previously required creating new components now allow putting the results into a new field as an option.
Transforms provide broader control over the amount of parallel CPU and parallel GPU computation resources used by a template.
Enhancements include easy picks from a vast range of units and not just the base unit used in a coordinate system.
There are numerous new templates and extensions to existing templates.
Queries generated for templates are shorter, with queries for transforms having clearly separated setup and repeatable parts, allowing rapid customizing and recycling of queries.
Previews
A new system for Select and Transform pane previews in table windows and visual windows provides greater accuracy and speed with larger data, with better visual presentations for data science and "what if" visualizations.
Separate query generation paths are used for preview queries, with new supporting machinery in the query engine for increased speed. Previews using full data sets are now often faster than previous previews only limited, partial data.
Previewing the results of a select or transform in a table now uses a separate preview field that can be dragged in position as desired for rapid comparisons, resized, or temporarily hidden.
Previewing the results of a select or transform in a map uses a separate preview layer, which can be resized to cover the full width of the window, or shown in split screen mode with the display partially previewed and partially without preview, with a draggable splitter line to allow sizing preview/no preview display as desired.
A single click can toggle 100%, 75%, 50% or 25% opacity for previews to compare proposed results with see-through background data.
All data for the preview layer is computed with full accuracy at the beginning rather than dynamically on zoom or pan, using multiple threads, ensuring very fast subsequent display.
Transforms that previously lacked previews now have them.
Previews persist on a per window basis, allowing multiple simultaneous previews in different windows.
Previews persist through other operations, allowing pan and zoom, adding new layers, turning layers off and on and much more without losing a preview.
Illustration: Dynamic viewshed (visibility zone) preview showing regions (blue) visible from the selected parcel. 75% opacity used for the preview with a split screen showing preview only to the left of the vertical blue splitter line. Synthetic terrain from SRTM data with satellite imagery backfilled into missing SRTM pixels. The selected parcel is in a small valley in the hills but still gets a view of the sea.
Queries
There are hundreds of new query functions.
Many raster functions have been extended to allow operating on a specific channel of a multi-channel image.
Many vector functions for 3D data have been extended to allow choice of a field giving Z data.
Dataports (File Formats and Data Sources)
Dataports for databases detect disconnects and automatically attempt to reconnect. Dataports for databases optimize batch operations for faster performance, automatically breaking big transactions into manageable portions to avoid failures.
GPKG always utilizes built-in support for SQLite.
Exporting a GPKG performs faster and produces files conforming to the latest version of the GPKG spec.
Reading a TIFF recognizes pyramid data stored within the file.
Geocoding servers and Reverse Geocoding Servers of different types are now available, merged together with URLs for individual services specified separately.
Geocoding servers for Bing / Google / Here / MapBox / MapQuest / Yandex support search services that allow locating features near a lat/lon location with a filter.
Linking an ArcGIS REST server supports vector (feature) data as well as raster data.
Various web dataports have been strengthened to work around server errors.
Infrastructure
The limit on temporary data used internally in 64-bit Manifold has been increased to 1 TB.
Spatial indexes for tiles have been extended to support a number of operations which help the UI: computing bounds limited to visible pixels, computing approximate statistics, etc.
Spatial indexes for tiles are supported in .map files and in file dataports that store data in MAPCACHE files.
Virtual drawings
Operations on geometry can use any table with a geometry field. The geometry field does not have to be part of a spatial index, although having such an index will frequently help performance.
Query functions that operate on geometry can accept either a physical drawing component stored in a database, or a virtual drawing created on a geometry field (using the ComponentFieldDrawing function) of a table or of a query component.
Virtual drawings support all functions available for regular components: for example, reporting coordinate systems or the name of the underlying component.
Both physical and virtual drawings can be limited to using only selected records with the result being accepted as a drawing by query functions.
Virtual images
A query function that operates on tiles can accept either a physical image component stored in a database, or a virtual image created on a tile field (using the ComponentFieldImage function) of a table or of a query component.
Virtual images support all functions available for regular components.
Both physical and virtual images can be limited to using only selected records but the result will not be accepted as an image by query functions and will only work as a plain table with no spatial context. Planes are to extend selections in images to be per-pixel and after that is accomplished, the result of limiting an image to only selected pixels also will work as an image.
Numerous UI Improvements
Enhanced Layers pane functionality with new commands and capabilities.
Layouts can now zoom a frame to the location of a different opened window, or to a saved Location.
Layout frames can save a Location from that frame.
Editing text windows like comments, locations, queries or scripts automatically saves changes every 5 seconds or on switch to a different window.
New images created by transforms use an automatic style computed from pyramid statistics, with the lowest available pixel values being set to black, the highest to white and the rest to a shade between black and white.
Automatic style applies to all images which have a spatial index and which do not have an explicit style set, which is helpful for images created by user queries and scripts.
Updates since 172.7
The default number of neighbors for Kriging / gravity parts of the Interpolate transform has been changed to 10 (Voronoi neighbors) from 0.
(Fix) The value of the radius parameter for Kriging / gravity parts of the Interpolate transform has been adjusted for output resolution.
Kriging computes model parameters in units of the output coordinate system, to make the model resolution-neutral.
Computing the automatic radius for Kriging has been adjusted to better represent the model. Previously, computing the automatic radius might have produced a value that was somewhat too small, as a result of being reduced too aggressively based on the number of neighbors. Logic has been adjusted to match logic used in ESRI products.
Previews for transform operations in a table window show only values for selected records when the Transform selection only box has been checked.
For Select and Transform pane previews, the context menu for the preview caption bar in map windows now provides commands to set the opacity of the preview results:to 100% Opacity, 75%, 50%, or 25%, allowing a view of layers below the preview results.
The minimum width of a field header in a table window required to show icons, like used to show the target of a preview or to show sort order, has been reduced to enable showing icons in as narrow a field as possible.
(Fix) The LIKE operator in queries correctly handles non-alphanumeric characters in the search pattern.
Illustration: Reduced scale image from the Example: Flooded Roads topic. Click the image for a larger, full-size illustration, one of the over eleven thousand screen shots that illustrate the User Manual for Release 9 and Viewer.
User Manual:
The Release 9 User Manual now includes 1007 topics with 11,480 illustrations.
8,550 illustrations are new or updated since the prior Official build.
738 topics have been added, extensively rewritten, or updated to show new workflow and capabilities.
Hundreds of new illustrated examples, large and small, have been added within hundreds of topics.
Following is a selection of new User Manual topics added since the prior official build, including a few renamed topics with substantial re-writes (only a few of many substantially rewritten topics are included below):
This build expands the preview feature for Select and Transform panes by adding support for previews in map windows. Map windows include all visual windows, such as opening drawings, labels, or images in their own windows.
See the 5 Minute Tutorial - Previews video.
Previews
In a table window, the Context menu on the preview field's column header has renamed the former Values command to Results.
Clicking on the preview field column header in table windows or the blue preview caption bar at the top of map windows shows or hides results, either values in the preview field or visual preview results in map windows. (Previously, in table windows the click had to be on the up / down icon).
The Select and Transform panes can show previews of operations that produce vector or raster data in map windows.
Previews appears on top of all other layers. Vector data previews in map windows are shown using blue and light blue colors. Raster data previews are shown using shades from blue (smallest value) to white (biggest value). Labels previews show previews of actions on the relevant label anchor objects.
Previews show a blue caption bar at the top of map windows. The preview caption bar shows the name of the operation being previewed. Right-clicking a preview caption shows a context menu with preview commands.
The Hide Preview command in the preview caption context menu removes the preview. Press the Preview button again to recompute and display the preview.
The Results command shows or hides preview data. Clicking on the preview caption does the same. This is a quick way to toggle the preview visual display without hiding the preview and then having to recompute the preview.
The Left and Right commands in the preview caption context menu limit preview data to one side of the window, so that one side of the screen shows map layers plus the preview while the other side shows just map layers. Drag the splitter line left or right to resize the portion of the view that shows the preview. The Full command extends preview data to cover the full width of the window.
Map windows automatically remove the preview if one or more of the referenced layers is removed from the map or refreshed.
Map windows compute all preview data when the preview is launched, immediately after the user presses the Preview button in either the Select or Transform pane. Computing data for the preview tracks progress and can be canceled. Further display of the preview does not involve any computations. (Previously, previews were computing data on the fly. This had a number of issues, including, but not limited to: a - rendering was slowed down significantly, b - preview results computed on partial data shown on the screen were misleading for many transforms, c - canceling a preview or closing the window could take a long time.) Computing preview data for a map window always uses multiple CPUs and GPUs, ignoring the limits imposed in the Transform pane.
The map window renders preview data so that turning layers on and off or altering their data does not require re-rendering the preview, and toggling the preview on and off does not require re-rendering layers. Threads rendering layers and the preview cooperate for better utilization of rendering resources and faster visual feedback.
Illustration above: Right click on the blue preview caption bar and choose Left or Right to generate a Split screen visual preview in a map window, showing the visible area from the selected point using the Viewshed transform. We can drag the splitter line left or right to expand or contract the width of the preview display.
The first in a two-build series introducing new preview capabilities for Select and Transform panes. This build provides contains the core capabilities for table windows, including changes to panes, separate query generation paths for preview queries, new supporting machinery in the query engine for the preview queries and more. The second build in the series will add refinements for table windows and extend previews to map windows.
Previews
Select and Transform panes include a Preview button to show a preview of the operation. In the current build, previews are limited to table windows. The next build will provide previews in map (drawing, image, etc.) windows as well. See the previews discussions in the Select Pane and Transform Pane topics for a quick introduction.
Previews for a transform operation in a table window show a virtual preview column in blue preview color on top of table columns. The name of the preview column is set to the name of the transform template. Previews values are computed dynamically as the table is scrolled. Preview columns cannot be scrolled out horizontally and are always visible.
Previews for a select operation in a table window show a virtual preview column on top of table columns, but reduce the preview column to its caption, with records in the table changing background color as follows:
white = the record is currently unselected and will stay unselected,
red = the record is currently selected and will become unselected,
blue = the record is currently unselected and will become selected,
violet (red + blue) = the record is currently selected and will stay selected,
gray = the selection value is being computed.
The name of the preview column for selections is set to the name of the select template. Previews values are computed dynamically as the table is scrolled
Preview columns can be hidden using a Hide Preview command in the context menu popped open by right-clicking on the preview column head.
Preview columns for a transform can be temporarily collapsed to the caption (column head only) or expanded back to caption and values using the Values command in the context menu (works as a toggle), or by clicking on the up/down arrow icon on the right side of the caption.
Preview columns can be resized by dragging either of the borders. Dragging the border of a preview column snaps it to a border between regular fields, with reasonable tolerance.
Preview columns can be dragged left or right in the table using either the caption or the column body with values. Dropping the preview column onto a regular column makes the preview column start at the left border of the target column. Dropping the preview column close to the right border of the rightmost table column makes the preview column start at that border.
Previews for a transform operation that saves results into an existing field will mark that destination field with a large dot icon in the column head for that field.
Previews for a transform operation that produces geometry, tile, or binary values in a table window reduce the preview column to its caption and show a tooltip reporting that the preview is unavailable. Operations that produce geometry or tile values should be previewed in a map window. Operations that produce binary values are not previewed because checking whether a particular binary value is intended is impractical without a highly specialized UI.
Previewing the result of a select or transform operation and then changing the parameters for the operation in the relevant pane does not update the preview automatically. To update the preview, click the Preview button again. Previews for select and transform operations override each other. For example, clicking the Preview button in the Transform pane will show a preview in a table window. Clicking the Preview button in the Select pane will hide the transform preview and will show the Select preview. Going back and forth between pressing Preview in the Transform and Select panes will alternate between Transform and Select previews. Performing a select or transform operation by clicking the Select button or Transform button applies the operation and removes the preview from the active window.
Switching between windows does not remove previews. It is possible to have multiple previews in multiple different windows shown simultaneously.
Other
Renamed Split transform template parameters: cuts -> parts, Cut with: -> Split with:, Cut with selection only -> Split with selection only, coordinates with step -> line coordinates. (Relevant Transform topics have been updated.)
Renamed query function: GeomToCuts -> GeomToParts.
Renamed query functions: GeomSplitToConvex and GeomSplitToConvexPar -> GeomToConvex and GeomSplitToConvexPar.
New query function: GeomPartLine - computes part of a line between two specified distances. If the begin distance is larger than the end distance, the returned part goes in the direction opposite to that of the line.
New query function: GeomToPartsLineSequence - splits a line into parts between specified distances. If the begin distance is larger than the end distance, the returned parts go in the direction opposite to that of the line. The result table includes minimum and maximum distance for each part.
The Split transform template for geometry values allows splitting lines into line parts.
For compatibility with other packages, the rules for world file extensions have been changed to: (a) take the original extension and add W, (b) take the first letter, take the last letter and add W. See the Projections topic.
The Reshape : rotate transform operation rotates geometry values clockwise. The GeomRotate query function keeps rotating geometry values counterclockwise, with the the template merely changing the sign of the parameter value.
The Copy transform template for geometry values support copying a rectangle to an x4 field.
The ArcGIS REST dataport supports ArcGIS REST servers with vector data. Downloaded vector data can be cached in the parent data source via the 'Save cached data between sessions' option. For an example, see the server that powers the dashboard for COVID-19 data from Johns Hopkins University, at https://services1.arcgis.com/0MSEUqKaxRlEPj5g/arcgis/rest/services/ncov_cases/FeatureServer Also see a step by step example in the Example: Vector Layers from an ArcGIS REST Feature Server topic.
The GDB dataport attaches EPSG codes to coordinate systems with equivalent WKID codes, to allow using grid-based and other EPSG coordinate transforms when re-projecting components with these systems.
The WMS dataport detects server errors that are not flagged as errors by the server and reports them in the log window. For example, some WMS servers announce the use of a specific format for tiles but then fail to use that format or will send no tile, failing to announce (as the standard requires) an error in sending no tile. Manifold will log such server errors, which can be very useful to debug server errors or when informing webmasters their server is misconfigured.
The WMS dataport performs additional filtering of layer coordinate systems to better work with servers that are misconfigured or fail to conform to the WMS standard. For example some WMS servers report bounding box data for a layer in a coordinate system that the server does not support, but then will advise in layer information the use of a different coordinate system that is supported. Manifold patiently tries various possibilities to work around such server errors, to enable more use of misconfigured servers.
Dataports for SQL Server, Oracle, DB2, and MDB wrap batch insert, batch delete, and batch update operations into transactions for robustness and performance. Dataports for other databases were already doing this, with this addition providing this feature for all dataports for databases.
Dataports for all databases automatically break big transactions for batch operations into manageable portions. Without this feature, most databases will fail with an overflow error when the amount of data to be committed exceeds a certain limit. The limit usually is too small for GIS work, often being in the range of 15,000 to 20,000 records or 50 MB to 100 MB of record data. Those values are far too low for GIS work so Manifold protects against overflows.
The SQL Server dataport supports the newest ODBC driver for SQL Server available from Microsoft, v17. A notable addition to support the new Microsoft ODBC driver is that the dataport recognizes custom types for geometry, geography, and XML fields reported by the driver.
The MDB dataport automatically corrects names of newly inserted tables to fit into the limits imposed by MDB, for example, no leading spaces, no double quotations, no periods, and so on.
(Fix) Computing viewsheds for all observers using visible area from all, or visible level from all no longer sometimes fails to handle tiles without observers.
Another large build, continuing implementation of the new user interface for Select and Transform panes. The build including major new transforms, such as the new geometric Split : cuts transform, which splits objects in one layer by using lines or areas in another layer. Includes major new capabilities in geocoding servers, and updates to key libraries used internally.
Select Templates
(Geometry fields) Search - supports a new part that is useful for ranking geometry values by size: radius, the radius of the minimum enclosing circle of the geometry.
(Text field) Search - supports new parts related to URLs: url scheme, url host, url port, url user, url password, url path, and url extra parts.
Transform Templates
(Geometry fields) Overlay - performs a topology overlay on a pair of drawings using identity, intersect, union, or update. The result drawing includes fields from the source drawing (transferred using the original field names) as well as fields from the overlay drawing (transferred using original names as modified based on the specified pattern, by default Overlay {name}). Both drawings can be restricted to the selection.
(Tile fields) Viewshed - computes viewsheds using numerous options: visible area from all, visible area from any, visible count, visible level from all, or visible level from any. Z values for view sources can be taken from a numeric field or directly from geometry values (the default). If Z values are taken from a numeric field, they can be computed relative to the raster height (on by default). There are options to specify refraction (the default is 0.13), minimum angle ( default -90 degrees), maximum angle (default 90 degrees), radius (default is 0 = no limit), and whether or not to use datum curvature (the default is not to use). View sources can be restricted to the selection.
(Geometry fields) Split - supports a new split into operation: cuts, which splits geometry values into parts (called cuts) using areas or lines in a different drawing. Both drawings can be restricted to the selection.
(Geometry fields) Copy - supports a new part: radius, reporting the radius of the minimum enclosing circle, useful for ranking geometry values by size.
(Geometry fields) Compose - composes a geometry value from coordinate numbers: point, point with z, circle, rectangle, segment, and triangle.
(Date fields) Compose - composes a date value from year, month, and day.
(Date fields) Arithmetic - performs arithmetic datetime operation: add time, subtract time, time after, and time before. The add time and subtract time operations add or subtract the specified amount of time units (day, week, hour, minute, second, or millisecond) to the original date and return a new date value. The time after and time before operations measure the difference between the original date and the second date and return a numeric value in time units.
(Numeric vector fields) Compose - composes a vector value: new or rearrange values. The new operation composes a new numeric vector from numeric values. The rearrange values operation rearranges values within an existing numeric vector. Setting a specific value to (none) sets it to 0.
(Tile fields) Compose - composes a tile value: new or rearrange channels. The new operation composes a new tile from numeric values or single-channel tiles. The rearrange channels operation rearranges channel values in the existing tile. Setting a specific channel to (none) sets it to 0.
(Text fields) Compose - composes a text value: geojson, gml, wkt, url, or url from base and relative. A geojson text field, for example, can be composed from a geometry field and a coordinate system name.
(Text fields) Copy - supports new parts: json named value, json array value, gml geometry, gml coordinate system, geojson geometry, geojson coordinate system, wkt geometry, url scheme, url host, url port, url user, url password, url path, and url extra parts.
(Numeric fields) Limit - limits a numeric value: limit both, limit minimum, and limit maximum.
(Tile fields) Limit - limits values of a specific channel in a tile: limit both, limit minimum, and limit maximum.
Queries
GeomOverlayTopologyXxx functions for union and update overlays allow the source drawing to be empty, and will compose the result from the overlay drawing.
GeomOverlayTopologyXxx functions for identity, union, and update overlays allow the overlay drawing to be empty, and will compose the result from the source drawing.
TileViewshedMakeXxx functions take an additional parameter for the channel to use.
TileViewshedXxx functions take an additional parameter for the Z field to use. If the name of the Z field is a blank string, the functions use Z data stored in geometry values.
TileViewshedTilesXxx functions return an indexed table to allow joins.
New functions: TileViewshedArea and TileViewshedAreaPar - compute visible area for a single observer.
New function: GeomToCuts - splits a geometry value into cuts with an area or a line.
TileFlowDirAccumXxx functions take an additional parameter for the channel to use.
New function: GeomRadius - computes the minimum enclosing circle for a geometry value and returns its radius.
New function: GeomRadiusGeo - maps a lat/lon geometry value to meters using the scale near its center, computes the minimum enclosing circle and returns its radius.
New function: CoordMeasureRadius - computes the radius of a geometry value using the specified measure.
New function: StringJsonArrayValue - parses a JSON array and returns the value with the specified index, as a string.
New constants: INT8MIN, INT8MAX, UINT8MAX, INT16MIN, INT16MAX, UINT16MAX, INT32MIN, INT32MAX, UINT32MAX, FLOAT32MIN, FLOAT32MAX, FLOAT64MIN, and FLOAT64MAX. These constants provide minimum and maximum values for various numeric types. There are no constants for INT64, UINT64 because computations in queries are performed predominantly in FLOAT64, which is not big enough to store such constants.
Other
TITANic CUDA - CUDA support has been updated to CUDA 11, the GPGPU.DAT file includes modules compiled for compute capability 6.0 (TITAN and upper-level GeForce GTX, see CUDA GPUs).
Latest and Greatest - SQLite has been updated to 3.32.3, V8 to 8.4.371.23, ICU to 67.1.0.0, and Visual C++ runtime libraries have been updated to 14.27.29016.0.
All in One - Geocoding servers of different types for the same provider have been merged together. Creating a data source for a geocoding server now allows editing URLs for all services supported by the server.
Be Here Now - The Here geocoding server has been updated to support the latest version of the Here API (v7).
Geocoding Search Services - Geocoding servers for Bing, Google, Here, MapBox, MapQuest, and Yandex support search services (Search, Search Circle, and Search Rectangle) that allow locating features near a lat/lon location with a filter string. Search services are typically used to limit results to restaurants, ATMs or something similar, with exact syntax depending on the server used.
That YX Thing - Reading GDB data using coordinate systems with no axes specified will force axes order explicitly to XY, even if the corresponding EPSG coordinate system is YX. GDBs almost always contain XY data despite an EPSG code that says YX data, so default use of XY saves users from this common error found in GDBs. See commentary in the That YX Thing essay.
Enhanced legibility - Parameter reports in template queries are now automatically processed to report date values without #...#, numeric vector values without CAST, UUID values without CAST, expressions without extra parens, field names without [...], channel choices without translation ( using channel X instead of X), and result choices without translation ( using (same field) instead of the field name).
A huge new build, setting records for the number of improvements and additions. Check out the rewritten Select Pane and Transform Pane topics, as well as the new Select Reference and Transform Reference topics for major user interface changes to the Select and Transform panes.
The new build contains numerous breaking changes to queries, mostly due to extensions in function arguments. The full list of function changes is in the Queries section below.
Select and Transform
UI redesign - The Select and Transform panes have been redesigned. Both panes use a two-step workflow:
In the first step, we pick a layer and a field. The pane then shows a list of templates available for the field. The list can be filtered using the filter box. Double-clicking a template (or repeat-clicking or focusing and pressing Enter or focusing and clicking the Edit Parameters button at the bottom of the pane) picks the template and proceeds to the second step.
In the second step, we specify parameter values for the picked template and then click the Select or Transform button at the bottom of the pane to perform the operation. The name of the picked template is shown at the top. To go back to the list of templates, click the Up button at the top corner of the pane. The layer and field used to pick the template can be changed without going back to the list of templates, with the picked template remaining the same and the parameter values automatically adjusting to the new choices.
Dynamic updates - The Select and Transform panes automatically update themselves after changes to components. For example, if we open a table, then open the Select pane and pick a template, then add a new field to the table, the pane will adjust and start offering the new field in lists for parameter values. If we pick the field for use as a parameter value and then delete it from the table, the pane will adjust again and switch the parameter value referencing the now deleted field to use a constant value or a different field, depending on the parameter. Similarly, if we open a map and start adding or removing layers, the panes will automatically adjust to all changes. Unlike before, changing the active layer has no effect on the panes, adding a new layer has a minimal effect (the new layer becomes available for use but the picked layer does not switch to the new layer), and deleting a layer only has an effect if that layer was the one picked.
Context saved - Switching between component windows automatically saves and restores the state of the Select and Transform panes for each window. For example, we can open a drawing, start preparing a transform, then decide to take a look at the records in a different table, opening that table in a new window, and when we return to the drawing, the Transform pane will be in the exact same state as we left it. We can have several alternative transforms or selects being prepared for different windows and switch between them freely. Performing a select or transform does not clear the relevant pane and leaves it in the state where the operation can be repeated with or without modifications.
Time readout- Since many operations complete nearly immediately, after clicking the Select or Transform button, we might wonder if we clicked the button. To show that we did indeed click the button and that the operation has already finished, the pane shows the time it took for the operation to complete next to the clicked button. After 10 seconds, the time readout disappears, ready to appear with a new readout the next time we click the button.
Expanded templates - Templates have been expanded with each new template frequently combining as operations many formerly separate templates. This, together with not resetting the panes after performing a select or transform, allows longer uninterrupted workflow using the same pane display, with greater persistence of parameter values, less need to re-enter parameter values, and reducing round trips up to the top level templates list to pick a different template.
Most recent templates saved - Each performed select or transform is saved for the duration of the Manifold session and the most recent operations are shown in the list of templates in the relevant pane. Picking a saved operation picks the template used for the operation and sets the values of the non-field parameters to the ones used during the operation. (Saving and restoring the values of all parameters including those that can be set to fields will likely be added as well.) Names of recent operations shown in the list include the values of key parameters. The total number of saved operations is not limited; however. the panes only list the last ten most recent operations that can be applied to the picked field. Performing the same select or transform multiple times does not create multiple saved operations (to avoid flooding the list), and instead updates a single saved operation putting it as most recent.
Pinned templates - A saved select or transform can be pinned for persistent use between Manifold sessions by clicking the pin icon in the right column of the list. Pinned operations are listed first, with non-pinned saved operations following, and then with templates listed last. Each group is sorted alphabetically.
Tooltips - The list of templates shows a tooltip for each template. The text of the tooltip shortly describes more details of the template, such as representative examples of the operations the template can do.
Filter box - The filter box searches the name of the template, the tooltip, and the available values for key parameters. This allows, for example, typing in '>' in order to quickly find the select template that selects values greater than specified, or typing 'aspect' to quickly find the transform template that computes aspect, even if the top level names of the relevant templates are more general.
Time logged - Performing a select or transform logs operation time in the log window (previously, this was only done for transforms). The log message includes the names of the used layer and field, the name of the template, and the values of key parameters.
Parameter metadata commented in queries - Queries composed for select and transform templates list used parameter values in the opening comment section. Queries for transforms that create new fields or components separate the part of the query that performs one-time setup from the rest of the query, to make it easy to run the query once and then run only non-setup parts repeatedly, if desired. VALUE statements are used to compartmentalize global values and to help make queries more legible.
Expression dialog - Expressions used for parameter values are edited in a separate Expression dialog, using a query builder similar to that in the Command Window to help compose the expression. The Expression dialog automatically checks the syntax of the entered expression and diverts to an error message if there is an error. The dialog also automatically checks the type of the entered expression and diverts to an error message if the type is incompatible with the expected type. The expected type for the expression is shown in the left bottom corner of the dialog. Expression code shown in the parameter control and used in the template is automatically compressed: comments and unnecessary whitespace are removed, multiple lines are squeezed into a single line, etc. Expressions that consist of multiple terms are also enclosed in parens to enforce computation order in the query text.
Action setting for combining selections - Result options for a select are now listed in a separate Action parameter (see illustrations near the end of the Select Pane topic.
replace selection - (the default) Replace any prior selection with the resulting selection.
add to selection - Add the resulting selection to any prior selection.
intersect with selection - Select only those records that were previously selected and which were selected by the template.
invert with selection - An "anti-intersect": select all records except those records in the intersection of the
subtract from selection - Subtract the resulting selection from any prior selection.
Result destinations for a transform can be either:
Same Field - perform the transform on the picked field and put the result back into it.
<an existing field> - perform the transform on the picked field and put the result into a different field of a compatible type that already exists. Tile fields are considered to be compatible when they have the same tile size and the same number of channels.
New Field - perform the transform on the picked field and put the result into a new field of a compatible type. The name and type of the new field are specified in the pane. For a tile field, the type is that of the channel values.
New Table - perform the transform on the picked field and put the result back into it, copying data into a different table. The name of the new table is specified in the pane.
New Drawing or Image - If the pane is set to create a new field or a new table and the resulting field is geometry or tile, a new drawing or image is also created with the name specified in the pane.
Read only fields guarded - Attempting to perform a transform on a computed field or on a built-in identity field (MFD_ID) disables the 'same field' result option. Attempting to perform a transform on a table that cannot alter its schema (common for result tables of queries) also disables the 'new field' result option.
Choosing names - The default name for a new field is set to a blank string to force entering it before running the transform. After the transform is run, the field name is left unaltered so that repeating the transform without any changes quickly fails with a 'name already in use' error and does nothing, alerting the user to alter the name or to choose a different Result destination. Similarly, the default names for new components are set to blank strings to force entering them, and after the transform is run the component names are left unaltered so that repeating the transform without any changes quickly fails, alerting the user to change names or to choose a different destination.
Autogenerated identity fields in databases - Creating a new table for a transform automatically creates an autogenerated identity field and a unique index on that field. If the target database does not support autogenerated fields, the transform fails. Manifold supports autogenerated fields on the vast majority of databases that Manifold can write to. The only major databases left out are those based on Microsoft Jet: Access and Excel files. But for Access and Excel files, it is better to copy the data into a .map file, perform the transform in the .map, then copy the result back into the Access or Excel file anyway. Performing the transform through the .map instead of directly on the data stored in the Access or Excel file limits the time the Jet database is being written to, which helps ensure its integrity, particularly when the database is being open from a network share, as happens often.
Clean transfers to new tables - If a transform results in a geometry field, the new table excludes (does not copy over) all other geometry fields and creates a spatial index on the kept geometry field. If the resulting field is tile, the new table excludes (does not copy over) all other tile fields and creates a spatial index on the kept tile field. Manifold does not copy over all geometry and tile fields except the resulting field using the assumption that when a table contains multiple such fields, they are intermediate results created by prior transforms. When starting a new table, this starts a new chain of transforms from the desired point and the new table is set to include all non-spatial attributes, but not the intermediate results of prior transforms. If some or all of the intermediate results are desired, first make a copy of the table and then run the transform on the new table updating an existing field.
Spatial indices required - If the target database does not support spatial indexes required for the resulting field, the transform fails safe. This limits transforms creating new tables with tiles to .map files. This may change in the future. Transforms creating new tables with geometry can run on .map files and on most databases.
Identity index required - Both selects and transforms require that the original table has an identity index (easy add with one click in the Schema dialog, if missing). Transforms on tile fields that depend on tile placement (that is, which must place tiles relative to each other for some reason, for example, to read the values of neighbor pixels) also require that the table has a spatial index on the picked field.
Automatic tolerance extended - Operations on drawings no longer specify a tolerance value except for a few special cases where this value materially defines the operation, for example, normalize. The tolerance value passed to query functions is set to 0, for automatic tolerance. In the future, this parameter likely will be removed from most functions altogether, so users need not consider it at all.
Direct operations on tables throughout - All operations in queries can be performed directly on tables without any other components involved. For the technical details, see the Components and Queries section below.
Parallelization controls - There is a new Resources parameter for transforms that specifies parallel computation resources available for the transform. The available options are:
all CPU cores, all GPU cores - the default: parallel CPU and parallel GPU may be used.
all CPU cores - parallel CPU may be used but parallel GPU cannot be used.
one CPU core, all GPU cores - only one CPU thread is allowed, but parallel GPU can be used.
one CPU core - only one CPU thread is allowed and parallel GPU cannot be used.
Each transform decides which of the allowed resources to use based on the nature of computations, including the specified parameter values. For example, if there are multiple CPU threads are available, but the computations are too small to benefit from that, a transform may elect to use a single thread to make better use of the resources.
Restrict to selected records expanded - Most transforms, specifically, transforms on non-tile fields and transforms on tile fields which do not depend on tile placement, allow restricting the transform to selected records. Transforms that involve secondary components allow restricting data used from those components to the selected records in those components as well.
Protections against NULLs - Many transforms protect against unexpected results when encountering NULL values. For example, attempting to append a text value to a text field using Concatenate treats NULL values as empty strings, in order to keep the field value unchanged instead of turning it into a NULL. Sometimes such a protection is undesirable and whether to have it or not depends on the typical use of the transform. Current protections have been added based on engineering judgement. Anyone finding some of the protections that have been added undesirable, or finding cases where such protections are absent but are required, should please send in a note.
Pass through of unaffected geometry - Geometry transforms that only make sense for a particular geometry type keep geometry values of other types unchanged whenever this makes sense. For example, reversing lines will keep areas and points unchanged instead of turning them into NULLs.
Uneven X and Y scales are OK - Geometry transforms that operate on distances also automatically compensate for uneven X and Y scales. Previously, if a coordinate system of a geometry field had different scales by X and Y, creating a buffer would create a circle in the coordinate system of the drawing which would become an ellipse if the scales for X and Y were made the same. Same for other distance computations. Now the transforms make the X and Y scales even prior to computing the buffer and then force the computed buffer back to the scales used in the coordinate system. This makes the results of computations independent of the scales used in the coordinate system, which is much more reasonable.
Autoselect in Project pane - Creating a new component using a transform automatically selects that component in the Project pane.
Select Templates
(All field types) Expression - allows specifying a boolean expression and selects all records for which the expression is TRUE.
(All field types) Null - select NULL or non-NULL values.
(Binary fields) Search - select values with the number of bytes less, less or equal, equal, not equal, greater or equal, greater, or between the specified values.
(Boolean fields) Search - select values equal, or not equal to the specified value.
(Date fields) Search - select values using date and time, date without time, year, year day, month, week, week day, day, hour, minute, second, or millisecond, comparing to the specified value or values (for between). For week numbers and week days, there is an option to select the day on which the week starts.
(Geometry fields) Search - select values using type, number of branches, number of coordinates, number of curves, area, bearing, length, or minimum or maximum x or y or z, comparing to the specified value or values. Examples using Search: select all areas between 10 and 20 square miles. Select all objects with curvilinear segments.
Units of measure - Select template parameters using angular values, such as bearings, allow choice of units. Standard angular units are Arc Minute, Arc Second, Degree, and Radian. Parameters using linear or square units allow choice of units. Standard linear or square units are any of the very long list of linear units Manifold supports. When units compatible with the coordinate system of a drawing are used in a template, if the drawing uses angular coordinate units, such as degrees used in Latitude / Longitude projections, any of the standard angular units may be specified. If the drawing uses linear coordinate units, such as meters used in Pseudo Mercator, any of the standard linear units available in Manifold may be used.
Vincenty formulae throughout - Area, bearing, or length for lat/lon data are computed using high precision geodetic (over the ellipsoid) Vincenty formulae. For metric data, there is a Measure option to compute in projected coordinates (auto) or to transform to lat/lon and compute using geodetic formulae (geodetic).
(Numeric fields) Search - select values less, less or equal, equal, not equal, greater or equal, greater, or between the specified values. There is an option to round the field value before comparisons, using to nearest (round), to zero (trunc), down (floor), or up (ceil) , which is useful for selecting all values nearly equal to the specified values.
(Numeric vector fields) Search - select values using the vector value, or the value of the nth component of the vector value (component number 0 to 3, depending on the type), comparing the vector value in the given field to the specified value or values. For individual vector component values, there is an option to round the value before comparisons, using to nearest (round), to zero (trunc), down (floor), or up (ceil).
(Text fields) Search - select values using the text or the number of characters in the text, comparing to the specified value or values. Text searches can use contains, starts with, ends with, contains regular expression, matches regular expression, matches pattern (like), or sounds like (soundex). Text searches allow specifying the collation, the default is 'neutral, nocase' for case-insensitive searches. All collations are supported. Text searches with regular expression allow ignoring case. All text searches support an option to trim the field value at the end, the start, or both the end and start before the search.
(Tile fields) Search - select values using number of pixels, number of missing pixels, number of channels, x or y size, or channel statistic (average, minimum, maximum, std deviation, std deviation pop, sum, variance, variance pop), comparing to the specified value or values. Channel statistics allow specifying the channel number in multichannel images. Example using Search: select all tiles with pixels in channel 0 greater than 1000.
(Uuid fields) Search - select values equal, or not equal to the specified value. The search value can be copied and pasted from a table cell.
Transform Templates
Flexible Result destination - The results of any transform template can be placed into any of the allowed result destinations described in the Select and Transform section above, that is, saved in place, saved to a different, existing field in the same table, saved to a new field in the same table with automatic creation of a new drawing or image, or saved to a new table with automatic creation of a new drawing or image. Saves into the same field or into existing or new fields in the same table are available only with transforms that create the same number of records as the source table.
Units of measure - Transform template parameters using angular values, such as bearings, allow choice of units. Standard angular units are Arc Minute, Arc Second, Degree, and Radian. Parameters using linear or square units allow choice of units. Standard linear or square units are any of the very long list of linear units Manifold supports. When units compatible with the coordinate system of a drawing are used in a template, if the drawing uses angular coordinate units, such as degrees used in Latitude / Longitude projections, any of the standard angular units may be specified. If the drawing uses linear coordinate units, such as meters used in Pseudo Mercator, any of the standard linear units available in Manifold may be used.
Vincenty formulae throughout - Measurements such as area, bearing, or length for lat/lon data are computed using high precision geodetic (over the ellipsoid) Vincenty formulae. For metric data, there is a Measure option to compute in projected coordinates (auto) or to transform to lat/lon and compute using geodetic formulae (geodetic).
(All field types) Expression - allows specifying an expression evaluating to the data type compatible with the target field, and filling the field or other result destination with the result.
(Binary fields) Copy - copies binary data, or the number of bytes into the desired result destination.
(Boolean fields) Copy - copies the boolean value into the desired result destination.
(Boolean fields) Logic - performs a logical operation: not, and, or, or xor and saves into the desired result destination.
(Date fields) Copy - copies date and time, date without time, year, year day, month, week, week day, day, hour, minute, second, or millisecond into the desired result destination. For week numbers and week days, there is an option to select the day on which the week starts.
(Geometry fields) Buffer - computes a inner buffer or outer buffer with the specified distance (positive numbers = outer buffer, negative numbers = inner buffer) and saves the computed buffer geometry into the desired result destination. The distance is specified in units compatible with the coordinate system of the field, which can be angular units (If the field has no coordinate system attached, the default pseudo-Mercator coordinate system is assumed, as always.)
(Geometry fields) Center - computes a centroid using circle, inner (area), or weight (area) and saves into the desired result destination.
(Geometry fields) Clean - performs maintenance operations on object metrics: normalize metric, remove curves, convert curves to lines, remove z, fill z, or fill missing z and saves into the desired result destination. Fill missing z has no effect on geometry values which already have a Z value. Convert curves to lines allows specifying the maximum number of coordinates generated per curve (the curve limit, parameter, 50 by default) and the tolerance value (default is 0 = automatic tolerance). Normalize metric also allows specifying the tolerance value. The tolerance value is specified in units compatible with the coordinate system of the field, either angular or linear. That allows specifying a tolerance in familiar units, such as feet or inches, even if the coordinate system uses some other unit, such as meters.
(Geometry fields) Clip - Inner clip or outer clip geometry with areas from a secondary drawing and saves into the desired result destination. If there is no secondary drawing available (for example, we are working from a table window or from a map that contains only one drawing layer), the clip with parameter is empty and the transform will fail safe. There is an option to limit the clip drawing to only the selected records. There is an option to keep either the inner (checked, the default) or the outer part of the clip.
(Geometry fields) Convert - converts geometry to a different object type: area, line, or point.
(Geometry fields) Copy - copies geometry, branch, coordinate x or y or z or xy or xyz, type, number of branches, number of coordinates, number of curves, area, bearing, length, or minimum or maximum x or y or z, and saves into the desired Result destination using the appropriate data type. The Measure option allows controlling computations of area, bearing, or length. Area and length allow specifying a metric unit, while bearing allows specifying an angular unit. Branch allows specifying the branch number using zero-based counting. Coordinate allows specifying the coordinate number using zero-based counting, and the counting direction (count from start or end).
(Geometry fields) Enclose - computes enclosing geometry: rectangle, rectangle with rotation, circle, or convex hull and saves into the desired result destination.
(Geometry fields) Interpolate - interpolates geometry values from a drawing and saves into a new image. Available interpolation options: triangulation, triangulation with segments, gravity, Kriging, Kriging with median polish, or Kriging with regression. Z values can be taken from a numeric field or directly from geometry values (the default). There is an option to specify resolution of the produced image, in units compatible with the coordinate system of the geometry field. For triangulation with segments, there is an option to remove flat areas using DEST (on by default). For gravity and Kriging, there are options to specify margins (the default is 0), radius (the default is 0 = auto), number of neighbors (the default is 0 = use Voronoi neighbors), step (the default is 0 = auto), model (auto (default), circular, exponential, gaussian, linear, power, rational, or spherical) and regression (auto (default), linear, or quadratic). The result is always a new table and a new image.
(Geometry fields) Merge - merges geometry values together into: area (dissolve), center, center inner (area), center weight (area), circle, convex hull, line, points, rectangle, or rectangle with rotation and saves into a new drawing. There is an option to specify a group field (none by default = group all geometry into a single value). The result is always a new table and a new drawing. Values in numeric fields are aggregated using Sum, values in non-numeric fields are aggregated using FirstNonNull (see the descriptions of added query functions). It is always possible to use a different aggregate or to use more aggregates by using the Join dialog after the transform.
(Geometry fields) Reshape - applies a simple transform to geometry values: shift, scale, unscale, rotate, reverse, flip horizontal, flip vertical, smooth, or snap to grid and saves into the desired result destination. Rotation angle is specified in angular units. Shift amounts and grid steps are specified in units compatible with the coordinate system of the field. Smooth uses a tolerance value specified in units compatible with the coordinate system of the field.
(Geometry fields) Split - splits geometry values into: shapes, shapes (ESRI non-OGC), branches, convex parts, coordinates, coordinates with step, segments, or triangles and saves into a new drawing. The two shapes options differ in normalization rules for areas: the shapes option uses OGC rules and the shapes (ESRI non-OGC) option uses rules required by some ESRI products that are different from those mandated by OGC. The coordinates with step option is used to put equally spaced coordinates onto lines, and allows specifying the starting, ending and step distances in units compatible with the coordinate system of the drawing. To cover an entire line, set the end distance to an unreachable value, for example, , 1000000 meters and the computations will handle that efficiently. The result is always a new table and a new drawing.
(Geometry fields) Topology - perform maintenance operations on topology and save into a new drawing. The only available option is currently: clean (generalize)., which is equivalent to the normalize topology transform in prior builds. More options likely will be added in the future for correcting specific types of issues. The result is always a new table and a new drawing. The number of returned records currently coincides with the number of original records, so the current transform could technically be made to update an existing table. That option is not offered because the transform likely will be adjusted to split records as necessary, which would require putting the result into a new table.
(Geometry fields) Trace - traces bounded areas from existing geometry values and saves into a new drawing. This was previously called Bounded Areas. The name was changed because other options likely will be added in future builds. The result is always a new table and a new drawing.
(Geometry fields) Triangulate - triangulates points and produces: areas, or lines and saves into a new drawing. The result is always a new table and a new drawing.
(Geometry fields) Voronoi - computes a Voronoi diagram and produces: areas, lines, or points and saves into the desired result destination. There is an option to specify margin in units compatible with the coordinate system of the drawing.
(Numeric fields) Arithmetic - performs an arithmetic operation: add, subtract, subtract from, multiply, divide, divide integral, divide remainder, square, square root, cube, cube root, power, reciprocal, absolute value, sign, sign invert, exponent, exponent binary, exponent decimal, logarithm, logarithm binary, logarithm decimal, error, error complement, gamma, or gamma logarithm and saves into the desired result destination.
(Numeric fields) Bit Logic - performs a bit logic operation: not, and, or, or xor and saves into the desired result destination. Bit logic operations are 32-bit, with all bits higher than 32 bits being lost.
(Numeric fields) Copy - copies numeric values and saves into the desired result destination.
(Numeric fields) Hyperbolic - computes a hyperbolic function: hyperbolic sine, hyperbolic cosine, hyperbolic tangent, hyperbolic arc sine, hyperbolic arc cosine, or hyperbolic arc tangent and saves into the desired result destination. There is an option to specify the angular unit of measure to use.
(Numeric fields) Round - rounds numeric values: to nearest (round), to zero (trunc), down (floor), or up (ceil) and saves into the desired result destination. There is an option to round to decimal position values other than 1: 10, 100, etc, or 0.1, 0.01, etc.
(Numeric fields) Special - computes a special function: bessel 1st kind (jn) or bessel 2nd kind (yn), using the specified order and saves into the desired result destination.
(Numeric fields) Trigonometric - computes a trigonometric function: sine, cosine, tangent, arc sine, arc cosine, or arc tangent and saves into the desired result destination. There is an option to specify the angular unit of measure to use.
(Numeric vector fields) Copy - copies the vector value, or the value of the nth component of the vector value (component number 0 to 3, depending on the type), and saves into the desired result destination.
(Text fields) Case - converts the case of text values to: lower, title, or upper case and saves into the desired result destination. Title case follows Microsoft rules. There is an option to specify the collation, the default being neutral, nocase.
(Text fields) Concatenate - concatenates text values adding a new value either at the end or the start and saves into the desired result destination. There is an option to specify a separator string (empty by default), which is added in between the concatenated strings if both concatenated values are not empty. Example: add a value to a comma-separated sequence of values, adding a ', ' (a comma character and a space character) sequence as a separator if the sequence is not empty. See the discussion in the Transform - Text topic.
(Text fields) Copy - copies text strings, or the number of characters in the text string, and saves into the desired result destination.
(Text fields) Encrypt - encrypts or decrypts text values using specified password and saves into the desired result destination.
(Text fields) Pad - pads text values, repeatedly adding the specified characters at the end or start until the text value has the specified length and saves into the desired result destination.
(Text fields) Reduce - reduces text values to: start, end, cut, cut to end, or regular expression occurrence and saves into the desired result destination. For start or end, specify the number of characters to keep. For cut to end, specify the starting character. For cut, specify the starting character as well as the number of characters to keep. For regular expression occurrence, specify the regular expression and the occurrence, using zero-based counting. For regular expression occurrence there is also an option to ignore case.
(Text fields) Replace - replaces: text, text occurrence, regular expression, or Select Templates by Data Type and saves into the desired result destination. For text replaces, there is an option to specify the collation, the default being neutral, nocase. For regular expression replaces, there is an option to ignore case.
(Text fields) Reverse - reverses text values and saves into the desired result destination.
(Text fields) Trim - trims text values at: start and end, start, or end and saves into the desired result destination. The trimmed characters can be: whitespace, whitespace and custom, or custom, with specification of custom characters to trim.
(Tile fields) Arithmetic - performs an arithmetic operation on the specified channel: add, subtract, subtract from, multiply, divide, divide integral, divide remainder, square, square root, cube, cube root, power, reciprocal, absolute value, sign, sign invert, exponent, exponent binary, exponent decimal, logarithm, logarithm binary, logarithm decimal, error, error complement, gamma, or gamma logarithm and saves into the desired result destination. If the tile field contains multiple channels, the result must to be written into a different field.
(Tile fields) Contour - computes contours on the specified channel and outputs into a new drawing, as either areas or lines. The Full Range button automatically sets the start and end values for the height range to the approximate range computed for the image. There is an option to round start and end values to the step (on by default). For start = 112, end = 390, step = 100, without rounding the contours will be produced for 112, 212, 312. With rounding, the contours will be produced for 200 and 300. There is an option to split the produced geometry into shapes (on by default). The result is always a new table and a new drawing.
(Tile fields) Copy - copies pixels, channel, number of pixels, number of missing pixels, number of channels, x or y size, or channel statistic (average, minimum, maximum, std deviation, std deviation pop, sum, variance, variance pop), and saves into the desired result destination. When copying a single channel, the transform can put the result into a specific channel of any tile field of the same size.
(Tile fields) Filter - filters the specified channel using: average, blur, blur direction, blur gaussian, count, diversity, diversity index, edges, edges direction, major, maximum, median, minimum, sharpen, std deviation, std deviation pop, sum, variance, or variance pop and saves into the desired result destination. All filters allow specifying the radius (the default is 1). Many filters also allow specifying shape: square, circle, diamond, or cross. Some filters also allow specifying the center value. Direction filters allow specifying the direction angle. The result of the transform does not have to be put into a new table or even into a new field. If the tile field contains a single channel, the result can be put back into it. This applies to several other transforms which in prior builds could only put the result into new components.
(Tile fields) Hyperbolic - computes a hyperbolic function on the specified channel: hyperbolic sine, hyperbolic cosine, hyperbolic tangent, hyperbolic arc sine, hyperbolic arc cosine, or hyperbolic arc tangent and saves into the desired result destination. There is an option to specify the angular unit of measure to use.
(Tile fields) Path - computes closest paths on the specified channel of an image from or to specified points in a drawing using: Euclidean, cost, or cost for slope distance metric and saves into the desired result destination. Euclidean paths allow using barriers (off by default). Cost paths allow forcing each pixel to have the same cost; otherwise, the cost is taken from the channel value. Cost for slope paths compute costs based on slopes: the only allowed slopes are those between the specified low slope and high slope values. Costs are specified for low slope, flat slope, and high slope. Costs for intermediate slopes are interpolated linearly. Slope paths are directional: since costs for slopes depend on which direction the path is traversed, there is an option to reverse path, which finds paths not from the specified points, but rather towards them. The points from which to compute paths are specified in a drawing, with an option to use only selected points. The output of the transform for each result pixel can be a: value, direction, or distance. For value, the pixels in the produced image contain the numeric identifier of the closest point. That identifier can be taken from a field or be assigned sequentially (the default).
(Tile fields) Round - rounds values in the specified channel: to nearest (round), to zero (trunc), down (floor), or up (ceil) and saves into the desired result destination. There is an option to round to decimal position values other than 1: 10, 100, etc, or 0.1, 0.01, etc.
(Tile fields) Slope - computes a slope or a similar function on the specified channel: slope, aspect, gaussian curvature, mean curvature, plan curvature, or profile curvature and saves into the desired result destination. All functions allow specifying the radius (the default is 1). Slope and aspect allow specifying the unit: angular units for aspect and angular unit plus grade unit for slope.
(Tile fields) Special - computes a special function on the specified channel: bessel 1st kind (jn) or bessel 2nd kind (yn), using the specified order and saves into the desired result destination.
(Tile fields) Trace - traces areas marked out by similar pixels on all channels of the tile field and saves into a new drawing. There is an option for the similarity level using a default of 10, with which value being best to use depending heavily on the data used. There is an option to split the output into shapes (on by default). The result is always a new table and a new drawing.
(Tile fields) Trigonometric - computes a trigonometric function on the specified channel: sine, cosine, tangent, arc sine, arc cosine, or arc tangent and saves into the desired result destination. There is an option to specify the angular unit of measure to use.
(Tile fields) Watershed - computes watersheds on the specified channel: lines, downstream lines, upstream lines, areas, areas for sinks, or upstream areas and saves into a new drawing. There is an option to specify a channel giving in each pixel the precipitation amount, with the default being no channel, thus assuming all pixels have equal precipitation. Downstream and upstream computations require specifying starting points in a drawing. There is an option to use only selected points in the starting points drawing. There is an option to keep overlaps for downstream and upstream objects that result. Options allow specifying minimum flow value to limit the number of created objects (the default is 100). The result is always a new table and a new drawing.
(Tile fields) Watershed Prepare - prepares the specified channel for computing watersheds by filling sinks, and saves into the desired result destination. There are options to specify fill height (the default is 0 = no limit) and fill flow (the default is 0 = no limit).
(Uuid fields) Copy - copies uuid values and saves into the desired result destination.
Components in Queries
Tables rule - Previously, the system sometimes relied on drawings and images to describe the structure of spatial data in tables. Starting with this build, the system instead relies on indexes and treats spatial data in tables as self-describing, with drawings and images used only for visual display purposes.
Virtual drawings - Operations on geometry can use any table with a geometry field. The geometry field does not have to be part of a spatial index, although having such an index will frequently help performance. A query function that operates on geometry can accept either a physical drawing component stored in a database, or a virtual drawing created on a geometry field (using the ComponentFieldDrawing function) of a table or of a query component. A virtual drawing supports all functions available for regular components: for example, it can report its coordinate system or the name of the underlying component. Both physical and virtual drawings can be limited to using only selected records with the result being accepted as a drawing by query functions.
Virtual images - Operations on tiles use a table with a tile field, a pair of X and Y fields, and a spatial index connecting these fields together. Images on some data sources, for example, web images, use a slightly different structure with a non-spatial index on X and Y fields, sometimes also including a level field, instead of the spatial index. Such images have to be converted to images with a spatial index prior to the operations. A query function that operates on tiles can accept either a physical image component stored in a database, or a virtual image created on a tile field (using the ComponentFieldImage function) of a table or of a query component . A virtual image supports all functions available for regular components. Both physical and virtual images can be limited to using only selected records but the result will not be accepted as an image by query functions and will only work as a plain table with no spatial context. Plans are to extend selections in images to be per-pixel and after that is accomplished, the result of limiting an image to only selected pixels also will work as an image.
Rects are display frames only - The rectangle value stored in an image component is used solely for display purposes and is ignored for both selects and transforms. Previously, some selects and transforms ignored the rectangle and others used it. Starting with this build, the system treats the rectangle as a strictly visual display frame that limits what part of the data is shown on the screen, similar to how a table window can limit which records are shown in the list by filtering field values, and ignore it for selects and transforms. Operations that do not depend on tile placement and which treat pixels independently from each other may operate either on all or on selected records. Operations that depend on tile placement always operate on all records. After per-pixel selections are implemented, all operations on images will be able to operate on only selected pixels.
Expert commentary: There is a potential issue with ignoring image rectangles for data analysis. Since some transforms previously used the rectangle value to limit queried data, it might have been possible for an image to contain garbage outside of the rectangle without this being noticed. It is also possible that some of the older dataports created such images during the import. Preliminary checks on archived test files that were created with old builds have found no such images existing de-facto, but it is possible that they exist. If any such images exist in the wild, running a transform on it will read data outside of the image rectangle that might not have been read in prior builds, and the results of the current transform may differ from the results from previous builds, likely to the worse. To understand whether an image contains data in pixels outside of the image rectangle, one can force the image rectangle to the extent of visible pixels. This currently requires using a query (see the changes to the ComponentBounds function). Upcoming builds will allow changing the image rectangle to cover all visible pixels via the UI. Upcoming builds likely will also allow cropping the image to the current rectangle, throwing away all data outside of the rectangle. Any instances indicating that a particular dataport or a UI tool may create an image with data in pixels outside of the image rectangle should be reported as a bug, to allow investigation and an immediate fix.
Storage
Spatial indexes enhanced - Spatial indexes for tiles have been extended to support a number of operations which help the UI: computing bounds limited to visible pixels, computing approximate statistics, etc. Spatial indexes for tiles are supported in .map files and in file dataports that store data in MAPCACHE files.
Queries
BETWEEN has been changed to return FALSE if the lower bound is bigger than the higher bound, similarly to other databases.
(Fix) Tuple comparisons no longer sometimes incorrectly handle non-constant NULL values. (Example 1: (1/0, 1) BETWEEN (1, 1) AND (2, 2) is now NULL, was FALSE. Example 2: (1/0, 1) < (1, 1) is now NULL, was FALSE.)
(Fix) Tuple equal and not equal comparisons no longer sometimes incorrectly stop on constant NULL value. (Example 1: (NULL, 2) = (2, 1) is now FALSE, was NULL. Example 2: (NULL, 2) <> (2, 1) is now TRUE, was NULL.)
New function: Bound - bounds a numeric value to the specified range. If the strict parameter is TRUE, the minimum value has to be lower than or equal to the maximum value, otherwise the function returns NULL. If the strict parameter is FALSE, the minimum value can be greater than the maximum value.
New function: CoordUnitScale - takes a definition of the coordinate system unit, parses it and returns unit scale as a number.
New function: CoordSystemScaleXYZ - composes XYZ scales for a coordinate system. If the coordinate system is metric, the values are in meters. If the coordinate system is lat/lon and the forceMeters parameter is FALSE, the XY values are in degrees and the Z value is in meters. If the coordinate system is lat/lon and the forceMeters parameter is TRUE, the XY values are converted to meters based on the scale at the Equator.
New function: CoordSystemScaleXYZBounds - composes XYZ scales for a coordinate system and the specified bounds. The function behaves similarly to CoordSystemScaleXYZ except the lat/lon scale is converted to meters based on the scale at the center of the bounds.
ComponentCoordSystem always forces the coordinate system for an image to XY axes.
ComponentCoordSystemScaleXY has been removed, use ComponentCoordSystem and then CoordSystemScaleXY,
ComponentCoordSystemScaleXYZ has been removed, use ComponentCoordSystem and then CoordSystemScaleXYZ, Bounds.
ComponentCoordSystemAutoXY has been removed, use ComponentCoordSystem which now does what ComponentCoordSystemAutoXY was doing before.
New function: ComponentFieldDrawing - creates a virtual drawing for a geometry field in the specified component.
New function: ComponentFieldImage - creates a virtual image for a tile field in the specified component. The tile field has to be part of a spatial index.
ComponentBounds has been extended to compute bounds for virtual components and to limit bounds for an image to visible pixels. If the forceCompute parameter is FALSE, the function returns bounds stored in a spatial index or the rectangle stored in the metadata for a physical image. All select and transform operations use rectangles from virtual images that include data from all records, but retrieving the rectangle stored in the metadata is still useful, for example, for creating a copy of an image. If the forceCompute parameter is TRUE, the function computes bounds from the field values, and for a tile field it also restricts bounds to visible pixels. If the forceCompute parameter is TRUE and the tile field belongs to a web image known to be very big, the function ignores the parameter, thus protecting against inadvertent attempts to compute bounds by downloading all tiles from Bing or Google to compute bounds by examining tile values.
New function: ComponentFieldBounds - computes bounds of a geometry or tile field in the specified component.
New function: ComponentFieldCoordSystem - returns the coordinate system for a geometry or tile field in the specified component.
Selection and SelectionWindow ignore attempts to create a selection on a selection (were prevously stacking).
New function: GeocodeSearchMatches - searches for geocoding matches for an address with a filter (for example, 'hotel') near a lat/lon location with or without a radius in meters.
New function: GeocodeSearchRectMatches - searches for geocoding matches for an address with a filter in a lat/lon rectangle.
New function: GeocodeSearchSupported - checks whether a geocoding data source supports GeocodeSearchXxx functions above.
New functions: GeomBoundsX, GeomBoundsY, GeomBoundsZ - compute minimum and maximum X, Y, Z for a geometry value as an x2 value.
New function: GeomCurveCount - returns the number of curve segments in a geometry value. Takes a curve type: a negative value or 0 = count any curvilinear segment, positive value = count curvilinear segments of a specific type.
GeomSnapToGrid collapses equal coordinates together.
TileContourXxx functions take an additional parameter for the channel to use.
TileInterpolateXxx functions take an additional parameter for the Z field to use. If the name of the Z field is a blank string, the functions use Z data stored in the geometry values.
TileInterpolateXxx functions take an additional parameter for the XY scales to use for the produced pixels, expressed in degrees or meters depending on the coordinate system of the image. The XY shifts are assumed to be 0.
TileDistanceMakeXxx functions take an additional parameter for the channel to use.
TileDistanceTilesClosestXxx functions take an additional parameter for the source field to use. If the name of the source field is a blank string, the functions use sequential numbers, starting from zero.
TileDistanceTilesXxx functions return an indexed table to allow joins.
TileFillSinksXxx functions take an additional parameter for the channel to use.
TileFillSinksXxx functions return an indexed table to allow joins.
TileWatershedMakeXxx functions take additional parameters for the height and water channels to use. If the water channel is a negative value, all pixels are assumed to have equal precipitation.
New function: TileFilterDefDiamond - creates a diamond filter shape (mid-way between cross and circle).
New function: TileSize - returns the dimensions of a tile as an x2 value.
New function: TileValueCountMissing - returns the number of missing pixels in a tile.
New function: TileUpdateFieldPyramids - updates pyramids for a tile field in the specified component. If the tile field is not part of a spatial index, the call does nothing.
New function: TileUpdatePyramidsAll - updates pyramids for all tile fields in the specified component.
StringPadEnd and StringPadStart treat negative length as no padding.
StringConcat takes a separator parameter and concatenates two strings through the separator as long as both are not empty.
StringSubstring and StringSubstringLen treat negative start as start of the string and negative length as full length of the string.
New function: VectorCross - computes a cross product of x3 values.
New function: VectorDot - computes a dot product of x2, x3, x4 values.
New aggregate function: FirstNonNull - returns the first non-NULL value. (First may return a NULL even if the field contains non-NULL values.)
New aggregate function: LastNonNull - returns the last non-NULL value. (Last may return a NULL even if the field contains non-NULL values.)
Images
Using a single channel within multichannel images - Operations that work with a single channel allow specifying the channel number and can be used with multi-channel images.
Bounds set to extents - New images created by transforms have their bounds automatically set to the extents of visible pixels (not tiles).
Automatic image style - New images created by transforms use an automatic style computed from pyramid statistics, with the lowest available pixel values being set to black, the highest to white and the rest to a shade between black and white. The automatic style applies to all images which have a spatial index and which do not have an explicit style set, which is helpful for images created by user queries and scripts. Automatic style for images with UINT8 channel values is special-cased to always map 0 to black and 255 to white for compatibility reasons and because this is the desired behavior in the majority of cases.
More auto in autocontrast - The autocontrast options in the Style pane for images limit the computed contrast range to the actual range of pixel values within the image for better image quality. The options have been reordered from smallest to biggest adjustments so that options most likely to be used appear first.
Other
Faster MySQL - The MySQL dataport wraps batch operations into transactions for performance and safe rollback in case an operation fails mid-way.
Merge areas hidden - The Join dialog hides the merge areas transfer to avoid a confusing choice between merge areas and union areas.
FirstNonNull in Joins - The Join dialog uses the FirstNonNull aggregate function for copy and convert transfers, to return a non-NULL value whenever possible.
Kriging cancellation - Computing Kriging parameters can be canceled. Previously, attempting to cancel had to wait until the parameters are computed in full.
GDB dataport - Significant improvements in the Manifold dataport for ESRI GDB geodatabases:
Export to GDB no longer splits geoms with and without Z and instead adds zero Z to geoms that do not have it.
The GDB dataport better handles renames and deletes of components, and better handles changes to table schemas. (Various operations could sometimes fail in prior builds.)
The GDB dataport supports refreshing data from the Project pane. This is useful for picking up changes to the geodatabase made outside of Manifold.
The GDB dataport supports autogenerated fields (OBJECTID).
The GDB dataport supports non-NULL constraints.
The GDB dataport translates systems with WKID values matching the correspondent EPSG code to EPSG:xxx, thus enabling coordinate transforms in Manifold. Small differences between the definition of the coordinate system used by the database and the definition of the EPSG code used by Manifold are put into coordinate system overrides.
See the following topics for preliminary documentation on build 9.0.172.4:
Introduction to the Select pane. A similar topic is the Select Reference topic. |
|
Using expressions in the Select pane, either using the Expression template and as expressions within parameter boxes. |
|
Using expressions in the Select pane. (mostly duplicates the Select Pane Expressions topic). |
|
Introduction to the Transform pane. A similar topic is the Transform Reference topic. |
|
Using expressions in the Transform pane |
Select templates for variable length binary fields, of type varbinary. |
|
Select templates for Boolean fields, of type boolean. |
|
Select templates for date and time fields, of type datetime. |
|
Select templates for geometry fields, of type geom, geommfd, or geomwkb. |
|
Select templates for numeric fields, of type float32, float64, int8, int16, int32, int64, uint8, uint16, uint32, or uint6. |
|
Select templates for numeric vectors, that is fields with ordered sets of multiple numbers, of x2 type float32x2, float64x2, int8x2, int16x2, int32x2, int64x2, uint8x2, uint16x2, uint32x2, uint6x2, of x3 type float32x3, float64x3, int8x3, int16x3, int32x3, int64x3, uint8x3, uint16x3, uint32x3, uint6x3, or of x4 type float32x4, float64x4, int8x4, int16x4, int32x4, int64x4, uint8x4, uint16x4, uint32x4, uint6x4. |
|
Select templates for variable length text fields, of type nvarchar or varchar. |
|
Select templates for raster data fields, of type tile. |
|
Select templates for Universally Unique IDentifier (UUID) fields, of type uuid. |
Transforms for variable length binary fields, of type varbinary. |
|
Transforms for Boolean fields, of type boolean. |
|
Transforms for date and time fields, of type datetime. |
|
Transforms for geometry fields, of type geom, geommfd, or geomwkb. |
|
Transforms for numeric fields, of type float32, float64, int8, int16, int32, int64, uint8, uint16, uint32, or uint6. |
|
Transforms for numeric vectors, that is fields with ordered sets of multiple numbers, of x2 type float32x2, float64x2, int8x2, int16x2, int32x2, int64x2, uint8x2, uint16x2, uint32x2, uint6x2, of x3 type float32x3, float64x3, int8x3, int16x3, int32x3, int64x3, uint8x3, uint16x3, uint32x3, uint6x3, or of x4 type float32x4, float64x4, int8x4, int16x4, int32x4, int64x4, uint8x4, uint16x4, uint32x4, uint6x4. |
|
Transforms for variable length text fields, of type nvarchar or varchar. |
|
Transforms for raster data fields, of type tile. |
|
Transforms for Universally Unique IDentifier (UUID) fields, of type uuid. |
A technical build delivered out of sequence to fix bugs introduced in the TIFF dataport by the previous build. In addition, several new features have been added, including enhanced TIFF performance, GPKG improvements, faster loads for database dataports, and expansion of temporary data that can be used in a Manifold session to 1 terabyte in 64-bit mode.
(Fix) Reading a TIFF with tiled data no longer sometimes shows no images. (The regression that is the main reason for this out-of-band build.)
(Fix) Reading a TIFF no longer sometimes fails to mask pixels with no data values for int8, int8u, and int16 data types.
(Fix) Reading a TIFF no longer sometimes inaccurately decodes CMYK data.
Linking a TIFF supports pyramids stored within the file. This is a new feature that allows rendering a big TIFF without spending any time on preparing data for intermediate levels, if the file already has that data prepared and stored internally. There is a limitation in that the pyramids are only supported if both the main image and all sub-images within the file are tiled, but that is how such files are usually organized.
Dataports for databases skip loading support for personal geodatabases for technologies that cannot have them, for faster starts.
Dataports for databases skip loading support for SDE geodatabases for technologies that cannot have them, for faster starts.
(Fix) The color picker tool no longer picks the color from a wrong pixel on screens with DPI scaling.
The limit on the amount of temporary data used in a Manifold session in 64-bit mode has been increased to 1 TB.
The internal implementation of SQLITE has been updated to 3.32.2, adding a couple of new SQL functions.
(Fix) Dataports for web servers no longer sometimes fail to connect via HTTPS for the first time: Tthe first HTTPS web request no longer sometimes fails due to SSL / TLS policy being applied at the wrong time.
Exporting a map to GPKG shows a progress bar and reports export speed.
Reading a GPKG exposes an additional virtual table for each image. The virtual table has a fixed structure and includes both a BTREE index on the x-y field and an RTREE index on the x-y-tile field. This allows both rendering intermediate levels stored in the database as well as Alt-clicking into image tiles to see pixel values. The virtual table is read-only. The physical table storing image data continues to be accessible and writable.
Importing data into a data source other than MAP no longer stops if some of the indexes fail to create.
Pasting an image table with an RTREE index on the x-y-tile field automatically generates data for intermediate levels.
Extensive work on panes and the user interface continues, as well as improved input / output features for databases and GPKG. The Component and Record panes have been consolidated into a new Info pane that combines functions of both and introduces significant new features, such as extending former Record pane (now the Info pane) functionality to tables, with optional searching of related tables for lookups to picked records, like an on the fly, per-record join. Layout frames can now be zoomed to a location.
Panes and User Interface
The Record pane has been renamed the Info pane.
The former Component pane now is a Component tab in the Info pane. When the Info pane has no picked records, the Component tab is the only tab shown. When the Info pane has a picked record, the Component tab is shown last.
The Component tab in the Info pane separates data for a map and the active map layer using a separator instead of a text caption.
The Info pane shows '(no index)' for a drawing / image / labels with no spatial index.
The Info pane shows the text field used for labels.
The Info pane shows the language for a script.
The Info pane shows the number of lines for a comments / location / query / script text.
The Info pane shows the page size for a layout.
The Info pane shows the number of fields for a drawing / image / labels / table.
The Info pane shows the dimensions for an image.
The Info pane shows and allows editing the description for any component.
The Project pane reduces the description of a component to a single line (the first line) in the tooltip. (Previously it was trying to show the entire description.)
Editing entire records in a table window has been reworked to use both the table window and the Info pane. Alt-clicking a record cell or a record handle puts the record into the Info pane and starts editing it. Any changes to the record values made in the table window show in the Info pane and vice versa. Attempting to edit a cell for a new record automatically starts a new record and puts it into the Info pane as well. If the table is read-only, Alt-clicking a record cell or a record header puts the record into the Info pane for viewing. Alt-clicking empty white space in a table window clears the Info pane.
The Info pane allows undoing changes to a single value using an Undo Changes command in the context menu.
Changing a value in the Info pane and then changing it back to the old value is interpreted as undoing changes to the value.
Pressing Escape in a table window clears the Info pane if it contains no uncommitted changes, similar to maps.
The Info pane for a table window can move between the fetched records. The Zoom command has been renamed to Go, to make sense for a table. The icon has been changed as well. Future builds will likely allow moving beyond fetched records.
Reordering records in a table window keeps the picked record.
Filtering records in a table window keeps the picked record unless it does not pass the filter or the filter is applied to the whole table instead of to the fetched records.
The Info pane for a table record allows viewing coordinates for a geometry value using a View Coordinates command in the context menu.
The Info pane for a table record allows viewing pixels for a tile value using a View Pixels command in the context menu.
The context menu for a record handle in a table window includes an Edit Record command which puts the record into the Info pane. (This is an alternative to Alt-click.)
The Info pane allows searching for records in a secondary table that are related to the picked record. To show such records, pick a record, switch to the new Related tab, then click the Setup button in the tab toolbar to specify the relation. The button will open the Related dialog which allows selecting the source table and a pair of key fields, similar to how the Join dialog works. After the dialog is closed the system will search for records in the source table with the value of the key field equal to that in the record picked in the Info pane, and show the first 10 found records in the Info pane. The Refresh button in the tab toolbar repeats the search. The relation parameters are saved in the component window so that they can be used multiple times.
Searching for related records in the Info pane uses a BTREExxx index on the key field if the source table has one.
Searching for related records in the Info pane tracks progress and allows canceling. If the search is canceled, it returns however many records were collected prior to the cancel.
Specifying the source table and search criteria for the related records in the Info pane allows specifying fields to show for the related records.
Switching the Info pane to a different record automatically refreshes the related records if the Info pane is shown and the Related tab is active.
Other UI Features
The layout window allows zooming a layout frame for a map to a location, either stored in a Location component or to the location (viewport) of a different opened window. To zoom to a location, double-click the frame to make pan and zoom commands apply to the frame rather than the layout, then use the location commands in the View menu or in the toolbar.
The layout window allows saving the location for a layout frame for a map. This is useful for zooming a different window to the same location as used by a layout frame.
Editing a comments, location, query, or script automatically saves changes every 5 seconds. Switching to a different component window saves changes immediately. Changes are not saved on every keystroke because that does not work well when a component is stored on a remote database and the connection is slow. Saving every 5 seconds is a big improvement in usability for such connections. If saving every five seconds is reported to be still too frequent in some cases, future builds could allow either setting the save frequency explicitly or auto-adjusting the frequency of saves based on connection performance.
Comments, location, query, and script components no longer support the New Window command, to avoid conflicts when saving between multiple windows.
Migrating a MAP file created by Release 8 merges component notes into descriptions.
The command window for a script can be saved as a script using the Save as Script command.
(Fix) Resizing the Properties dialog avoids visual artifacts near the bottom. The minimum size of the dialog has been increased to avoid clipping controls.
(Fix) Using the color picker on a multi-monitor system no longer sometimes uses an inappropriate color format for the clicked monitor, thus picking a wrong color.
(Fix) Editing a record in a filtered or reordered table window no longer sometimes shows edits as applying to a wrong record.
Input / output
Dataports for databases detect disconnects and automatically attempt to reconnect once. If the attempt to reconnect fails, the Project pane shows the data source as disconnected. Most database drivers automatically close connections after they are not used for some time, to conserve resources. Previously this was putting the data source into a state where it could not return any data and the only method to get it working again was to recreate it or to close and reopen the MAP file. Now, such cases automatically are detected and reconnected seamlessly.
The Project pane allows reconnecting to a data source using a Reconnect command in the context menu. The reconnection is forced, that is, the existing connection, even if it is alive, is shut down and a new connection is established.
Refreshing a data source using the Refresh command from the context menu in the Project pane automatically reconnects to the data source if the connection has been lost.
(Fix) Reading a TIFF no longer sometimes creates extra images for files with pyramids.
(Fix) Exporting a GPKG writes application ID and user version according to the GPKG spec, to mark the file for third-party clients. Manifold supports the latest available version of the GPKG spec. At the present time, the latest version of QGIS appears to only partially support this latest version, and will notify the user of that limitation when opening a GPKG written by Manifold.
Connecting to a GPKG no longer loads spatial extensions like Spatialite and instead always uses the built-in version of SQLite.
Exporting an image to a GPKG performs significantly faster. If the image has intermediate levels, they are exported as well, otherwise only the export only writes the base level.
Creating a geometry field in a GPKG table set the field type to the appropriate flavor of geometry instead of to 'binary', to conform to the GPKG spec and to allow creating spatial views.
Attempting to write to a geometry field in a GPKG table rejects values of disallowed geometry types if the field has type restrictions. Previous versions of the GPKG spec relied on triggers to reject such values. The latest version of the spec requires filtering without triggers, so Manifold does that. This is a good change, much welcome.
(Fix) Linking an ArcGIS REST server no longer sometimes leaks memory.
This build introduces a major user interface change, promoting all panes to standalone, independent operation, with all panes now becoming independent panes just like the Project and Layers panes. This is the first step in a process of improvements to panes overall that will occur over the next few builds. In addition to new commands for facilitating docking and undocking with windows and panes, many other small user interface improvements provide enhanced quality of life. New query functions and transforms support manipulation of and computation on DateTime data types.
Panes and User Interface
The Component, Record, Select, Style, and Transform panes have been taken out of the Contents pane and can be shown and hidden individually. The Contents pane has been removed.
Panes that provide previews cooperate between themselves so that a component window only shows the last submitted preview. (Example: If you open a table window and start using the Select pane, the window will show the preview for that pane. If you then switch to the Transform pane while the Select pane is still on screen, the window will start showing the preview for the Transform pane. If you switch back to the Select pane and change the template or alter a parameter, the window will switch back to showing the preview for the Select pane again. We are currently reworking the previews, they will function a little differently than they do now.)
View - Pane commands for panes have been reworked to always show and activate panes, and never to hide them. Each command has a shortcut, with Ctrl-1 assigned to the Project pane and Ctrl-2 through Ctrl-7 assigned to all other panes. To hide a pane, use the Close X button in the tab strip.
(Fix) Closing an inactive floating component window no longer sometimes changes the active component window.
Pressing Esc (Escape) in a map window clears (unpicks) the picked record shown in the Record pane, if there are no uncommitted changes.
Pressing Esc (Escape) in a layout window clears (unpicks) the picked frame shown in the Record pane if there are no uncommitted changes. Previously, pressing Escape unpicked the frame even if there were uncommitted changes. This change makes the behavior safer and the same as in map windows.
Editing a frame in a layout window allows committing or canceling changes using Save Changes and Undo Changes commands in the context menu.
(Fix) A repeat click on an opacity value for the current layer in the Layers pane starts editing the value. (Previously, one had to click the layer name or press F2 to start editing the value, clicking the value itself did not work.)
(Fix) Alt-clicking the name of the current layer in the Layers pane no longer starts editing the opacity value. Instead, it activates the layer.
Alt-clicking the row header for a layer in the Layers pane activates the layer.
Alt-clicking the name of a layer in the Layers pane activates the layer.
Ctrl-clicking the row header (or Shift-clicking for a swath select) for a record in a table window selects the record without entering or leaving edit mode.
The New Script dialog shows the type of each script engine in the Language box as (COM), (.NET), or (V8).
Right-clicking the name tab for a component window or a pane and right-clicking the name caption bar for a floating component window or pane shows a context menu with dock commands. Component windows support the following commands: Dock, Dock Down, Dock Up, Undock, and Close. , with dock commands appearing as the situation warrants. For example, if there is only one component window open there is no Dock Down option, which appears when there are two or more docked windows. Panes support all these commands plus Dock Left and Dock Right, which move the pane between left and right sides. Future builds will also allow rearranging component windows and panes using drag and drop.
The default pane layout has been changed to use both left and right sides of the main application window with Project and Layers panes shown on the left and Component, Record, Select, Style, and Transform panes shown on the right. The dock state of all panes is saved between sessions. The default layout is not final and can easily be changed based on user feedback on using both left and right sides out of the box. If the previous layout is preferred, it is easy to switch back to that by moving all panes from the left side to the right: the system will remember that layout for future sessions.
The main application window cannot be resized smaller than a certain reasonable size.
Floating component windows and floating panes cannot be resized smaller than a certain reasonable size.
Floating component windows and floating panes can snap to each other, both for resizing and for positioning. Previously, floating component windows could only snap to other floating component windows, and floating panes could only snap to other floating panes.
Floating component windows and floating panes no longer show a close button in the right corner. To close a floawing window or pane, right-click the caption (title bar) and choose Close. The active floating component window can also be closed using the Close command in the Window menu, keyboard shortcut Ctrl-F4.
(Fix) Middle-clicking or right-clicking the caption of a floating component window no longer fails to make the component window active.
(Fix) Resizing the Select and Transform panes no longer sometimes produces visual artifacts near the bottom.
Other
(Fix) The Merge Images dialog computes shifts for the target image using shifts in the selected projection as a base. (Previously, the dialog was using 0 as a base.)
The Merge Images dialog fails safe when none of the source images can convert to the projection of the target image. (Previously, in that case the dialog was creating a 0x0 image.)
The PBF dataport imports faster and indexes tables slightly differently. See the new SQL Example: Extract Airport Runways from an OpenStreetMap PBF topic for an example.
New Functions and Templates
New functions and corresponding templates have been added for DateTime types and strings that are URLs:
DateTimeMake query function and Compose Date transform to compose a date from year, month, and day. A new DateTimeMakeFull query function composes a date from year, month, day, hour, minute, second, and millisecond.
DateTimeClearTime query function and Clear Time transform to clear the time part of a date.
DateTimeYearLeap query function and Date Year Leap transform to determine whether the year of a date is a leap year.
DateTimeYearDay query function and Date Year Day transform to return the day number for a date counting from the start of the year, with January 1 being day 1.
DateTimeWeek query function and Date Week transform to return the week number for a date counting from the start of the year, with the week containing January 1 being week 1. A parameter specifies the start day for weeks: 0 means that weeks start on Sunday, 1 on Monday, and so on.
DateTimeWeekDay query function and Date Week Day transform to return the day number for a date counting from the start of the week. A parameter to specifies the starting day for weeks: 0 means that weeks start on Sunday, 1 on Monday, and so on.
DateTimeAddDays query function and Date Shift transform to shift a date by the specified number of days. The number of days can be positive, to shift to later dates, or negative, to shift to earlier dates. The number of days can also have a fractional part, to shift by hours or minutes, etc.
DateTimeDifferenceDays query function and Date Difference transform to find the difference between two dates in days. The returned difference can be positive or negative, and can have a fractional part.
StringUrlxxx query functions and URL xxx transforms that parse a string as an URL and extract various parsed parts:
StringUrlScheme query function and URL Scheme transform.
StringUrlHost query function and URL Host transform.
StringUrlPort query function and URL Port transform.
StringUrlUser query function and URL User transform.
StringUrlPassword query function and URL Password transform.
StringUrlPath query function and URL Path transform.
StringUrlExtra query function and URL Extra transform.
StringUrlMake query function query function and Compose URL transform to compose a URL from parts.
StringUrlAppend query function query function and Append URL transform to append a relative URL to an absolute URL.
Build 9.0.172 is an Official build. This build or more recent is a must have build to replace all prior builds, for all Release 9 and Manifold Viewer users. Manifold System 9.0.172 delivers numerous enhancements released in the preceding series of Cutting Edge builds. For detailed lists of changes, see individual build notes below for builds 9.0.171.1 though 9.0.171.4.
Highlights in 9.0.172 can be grouped into six major sections:
Projecting Images
Images can now be reprojected using a choice of interpolation types: nearest neighbor / bilinear / bicubic.
A new projection method, direct sub-pixel, provides higher accuracy and better quality.
GPGPU
GPU parallelism in Manifold and Viewer is no longer tied to a particular version of CUDA and can use CUDA devices starting with compute capability 2.0 (Fermi).
GPGPU modules are loaded from a GPGPU.DAT file built into 64-bit installation packages for Manifold and Manifold Viewer.
The system automatically chooses the best module version for GPU devices in use.
Image above: Scene from the Knock Out Pixels Using Join and Alpha video, using the Join dialog to create cutouts of images based on area objects, that can be overlaid in maps on other layers.
Join
A new Join dialog transfers data between components in 5 different join and spatial join scenarios: table to table, drawing to drawing, drawing to image, image to drawing, or image to image.
Each scenario supports transferring data to existing fields or channels or to newly created fields or channels, with a variety of transfer methods available for each transferred item.
Each scenario supports transferring data to or from queries or components built on queries.
A join operation set up in the Join dialog can be saved to an update query which can then be used to customize the operation or to repeat the operation, for example, to capture changes in data.
Queries and Transforms
There are over 50 new query functions, including GPGPU variants.
Multiple functions have been extended to take additional parameters, enabling extended behavior.
Vector transforms such as Shift, Scale, Rotate and underlying query functions have been extended to support 2D curves. New transforms have been added, for example, to reverse the direction of lines, and for linear referencing, placing sequences of coordinates with the specified step and distance range onto lines.
Dataports
Reading a PBF file performs significantly faster.
Reading a KML / KMZ file recognizes more variants of geometry data and more variants of URL links.
Reading a LAS / LAZ file or a LAS library exposes scaled XYZ coordinate values as separate fields.
GPKG is now built-in: Connecting to an SQLite / GPKG file no longer requires an external version of SQLITE3.DLL and can use an internal SQLite implementation built into Manifold.
Documentation and Videos
Many new topics and videos have been added. See the Recent Topics section below for a list.
Specific enhancements since build 9.0.171.4 include:
LAS / LAS library dataports include scaled XYZ values in addition to unscaled ones in tables.
Opening an MDB or XLS file collapses log messages generated by attempting to use different versions of Microsoft Jet and reports only the finally used version on success or the list of attempted versions on failure.
Direct sub-pixel re-projection clamps and rounds produced pixel values if they are integer, for smoother results.
(Fix) Projecting an image with automatic local offsets no longer sometimes generates tiles with X or Y = -1 due to rounding errors in computing the dimensions of the resulting image. (The error did not affect values of projected pixels.)
(Fix) Reading a DEM file no longer sometimes shifts the coordinate system of the produced image one pixel down.
The map window can now show a map layer even if some of the screen locations fail to project to the coordinate system of the layer. For example, a Bing Maps image in a Latitude / Longitude map will no longer disappear when zooming far out.
Connecting to a SQLite database / GPKG file now uses an internal implementation of SQLite if an external version of SQLITE3.DLL is not available. The internal implementation of SQLite includes support for RTREE indexes, JSON, and GeoPoly. The internal implementation of SQLite only supports SQLite databases with GPKG-style spatial data, SQLite databases with ESRI ST_GEOMETRY data or with SpatiaLite registry tables require using external SQLITE3.DLL plus the required extension DLL, as before.
Coord SQL Functions - Functions that begin with Coord and manipulate coordinate systems.
Other SQL Functions - Non-aggregate functions that do not begin with Coord, Geom, String or Tile.
This build provides dramatic new capabilities in the Join dialog, adding raster to raster joins, vector to raster joins and additional general Join capabilities. Joins into rasters allow different coordinate systems between the target, original image and the source, joined image or drawing, with automatic reprojection to match coordinate systems for the join. Joins can move data from any channel in the source image to any channel in the target image, including creating new channels in the image, if desired. The Join paradigm is used in a general purpose way to accomplish raster combinations and vector to raster transformations that in other systems require a patchwork of special purpose tools. Other additions include a roster of significant new raster functions and utility functions, and improvements in parallelization with many threads.
Raster to Raster Joins (image -> image)
The Join dialog allows joining data from one image into another. Raster to raster joins are, also called raster overlays.
The only join condition available is matches. The join operation dynamically projects the source, joined image into the coordinate system of the target, original image and transfers data between matching pixels.
The list of fields contains a separate item for each channel in the target, original image. Any channel in the target, original image can receive data from any channel in the source, joined image. If the pixel type of the source image is the same as the pixel type of the target image, the transfer method is set to copy. If the pixel types are different, the transfer method is set to convert.
The Add button in the toolbar allows adding new image channels, up to a limit of 4 channels per image. Until the Join is run, newly added channels can be deleted.
The Clear image pixels option specifies whether to clear the target, original image before transferring data, or to transfer data on top of existing data (the default).
The Resize image option specifies whether to resize the target image before transferring data. During the resize, the coordinate system, including the pixel size, stay the same, but the image rect changes. If the Clear image pixels option is off, the image can only be resized larger. If the Clear image pixels option is on, the image can be resized either larger or smaller.
The update query generated for joining data between images includes the code to clear / not clear image pixels and to resize / not resize the image, so that these operations are repeated dynamically as part of the update when the update query is run.
Joining an image to itself is allowed and can be used to remap channel values. The query generated for this case is significantly simpler than update queries for joins between different images.
Vector to Raster Joins (drawing -> image)
The Join dialog allows joining data from a drawing into an image. Vector to raster joins are also called vector to raster overlays. See the Spatial Join Example: Transfer Drawing Attributes into Image Pixels topic.
The only join condition available is contained in. The join operation dynamically converts the drawing into a raster in the coordinate system of the image and transfers data between the images.
The list of fields contains a separate item for each channel in the target image. Any channel in the image can receive data from a field in the drawing. Available transfer methods are: count (the default), min, max, and sum.
The Add button in the toolbar allows adding new image channels, up to a limit of 4 channels per image. Until the Join is run, newly added channels can be deleted.
Clear image pixels and Resize image options work the same way for drawing to image joins as they do for image to image joins.
See many new Join Examples topics.
Join (general)
Selecting a transfer method for a field that is not yet being transferred in the Join dialog tries to use a source field with the same name as the target field.
Adding new fields or new image channels prior to performing the join specified in the Join dialog is done in a background thread that tracks progress and can be canceled.
Using the Join dialog to join data into a drawing or into an image layer in a map automatically clears the Record pane of any picked object or tile, discarding all changes, and clears field statistics used by the Style pane. (Previously both panes would show outdated data until the map window was closed and reopened.
The Join dialog logs the time spent performing the join.
Functions and Templates
There are new TileValueAvg, TileValueCount, TileValueMax, TileValueMin, TileValueStDev, TileValueStDevPop, TileValueSum, TileValueVar, and TileValueVarPop query functions that compute statistics for a single tile. All channels except the first channel are ignored, similar to other numeric functions for tiles.
The Line Coordinate transform template has been reworked to return a geometry value instead of an x2 value.
The Coordinate transform template has been reworked to return a geometry value. The former version of the template that returns an x2 value has been renamed to Coordinate XY. The Coordinate with Z transform template that returns an x3 value has been renamed to Coordinate XYZ.
There is a new TileFillBaseLevel query function that takes an image and adds all missing records with NULL tiles for tiles within the image rect.
There is a new TileClear query function that takes a tile and makes all pixels in it invisible.
There is a new TileChannelsConcatFill query function that takes two tiles and concatenates their channels. Unlike TileChannelsConcat, which only makes a pixel in the resulting tile visible if it is visible in both of the producing tiles, TileChannelsConcatFill makes a pixel in the resulting tile visible if it is visible in any of the producing tiles, filling missing channel values with zeros.
There is a new TileChannelCopy query function that copies values of a single channel between two tiles.
There is a new GeomInflateRect query function that inflates or deflates a rect (an x4 value) by the specified size (an x2 value: positive values inflate the rect and negative values deflate).
There is a new GeomInflateRectTileSize query function that inflates a rect (an x4 value) to cover whole tiles of the specified size (an x2 value, values required to be positive).
There are new TileGeomOverlayCount, TileGeomOverlayMax, TileGeomOverlayMin, and TileGeomOverlaySum query functions with parallel variants that take an image, a drawing and a field name and produce an image with the same parameters as specified, filled with data from the drawing. The referenced field has to exist and have a numeric type. For TileGeomOverlayCount, the field name can be an empty string.
Tables returned by the CoordConvertTileSet family of query functions operate slightly faster, particularly on systems with many cores.
There is a new ComponentBounds query function that computes the bounds of a component. The function takes a boolean flag controlling whether to compute bounds for a component that is not automatically maintaining bounds by reading component records. Example components that automatically maintain their bounds: images, drawings and labels in a MAP file based on a table with an RTREE index on the needed geometry field. Example components that do not automatically maintain their bounds: drawings and labels based on queries.
Other
(Fix) Projecting an image using the direct sub-pixel method no longer produces incorrect results if tile XY values in the resulting image can be negative, as might happen when local offsets are not set to Auto.
The Collation dialog applies the filter to the language code in addition to the language name.
Projecting an image using multiple threads limits the number of used threads to the number of CPUs * 2, to protect against trying to create too many threads when used programmatically.
(Fix) Aggregating pixel values under a geom no longer sometimes (extremely rarely) returns a non-NULL value for a geom that is outside of the image.
Vector to Raster Performance Compared to 8
In Manifold 8, we copy a drawing and paste it as a surface, setting field = ID, type = int32, and pixel size = desired pixel size.
In Manifold 9, we take the same drawing, create a blank image with int32 pixels, the desired pixel size, and the same coordinate system as the drawing. We then join the drawing into the image, setting channel 0 = max ID, clear image pixels = on, and resize image = on.
Test 1: 32 areas -> 3,000 x 2,000 pixels
8: 8.500 sec
9: 0.234 sec
Test 2: 527,000 areas -> 10,000 x 10,000 pixels
8: 341.250 sec (+130 sec if you count the time to perform Copy and then open the Paste dialog)
9: 34.278 sec
Further improvements are likely, as on many data sets the rasterization is so fast that using multiple threads currently does not gain much. That could be improved by tuning multi-threading workloads dynamically.
A technical build containing numerous functions, including new vector-related functionality required for upcoming new features such as georegistration, functions which have been planned for a while, and some special requests from users.
Functions and Templates
There is a new GeomScaleCentered SQL function that scales a geom around the specified center location. The Scale transform templatehas been adjusted to use that function and to scale around the centroid.
There is a new GeomShift SQL function which calls GeomScaleShift with scales equal to 1, only applying a shift. There is a new GeomScale SQL function which calls GeomScaleShift with shifts equal to zero, only applying a scale. There is also a new GeomScaleRev function which calls GeomScaleShift with scales equal to 1/scale, unscaling the geom scaled with a previous call to GeomScale back.
There is a new GeomFlip SQL function that flips a geom horizontally and / or vertically, and a new Flip transform template that calls it.
The GeomFlip, GeomRotate, GeomScale, GeomScaleRev, GeomScaleCentered, GeomScaleShift, and GeomShift functions preserve curves in 2d geoms and preserve Z values but not curves for 3d geoms. Caveat: attempting to apply uneven XY scales to a circular arc will currently not convert it to an ellipse arc, and will instead compose a new circular arc that passes through the scaled ends and the scaled control point. Future builds will address that with a slight rework of curves in geoms.
There is a new GeomReverseLine SQL function that reverses a line, and a new Reverse Line transform template that calls it. Reversing a line preserves curves and Z values.
There is a new GeomIsLineClosed SQL function that reports whether each branch of a line geom is closed (has the starting coordinate equal to the ending coordinate), and a new Lines, Closed select template that call it.
There is a new ComponentCoordSystemAutoXY SQL function that returns the coordinate system for a component and forces the coordinate system for an image to use XY axes. This reflects the fact that for an image, because it is stored in a special way, the axes of a coordinate system are ignored and are always interpreted as XY / XYH. That also allows passing the returned coordinate system without any further modifications to the coordinate converter object. Queries generated by UI tools have been modified to use the new function instead of ComponentCoordSystem.
The CoordConvertTile SQL function allows specifying interpolation type: 0 = nearest neighbor (the only former option), 1 = bilinear, 2 = bicubic. If the interpolation type is set to a negative value (auto), the function will use nearest neighbor, for compatibility with previous builds and due to the principle of least surprise.
The CoordConvertTileSet and CoordConvertTileSetPar SQL functions have been renamed to CoordConvertTileSetDirect and CoordConvertTileSetDirectPar. There are new CoordConvertTileSet and CoordConvertTileSet Par SQL functions that use the standard projection method, allow specifying interpolation type and return the projected image as a table. The result tables returned by the CoordConvertTileSetXxx functions (all four) support a BTREE index on XY fields, which allows efficiently using them in a join directly without having to copy the returned data into a different table.
A technical note on performance of SQL functions for projecting images: for the standard projection method there are thus two ways to perform the projection with multiple threads: Either use CoordConvertTile in a SELECT with THREADS, or use CoordConvertTileSetPar. CoordConvertTileSetPar should generally be able to arbitrate threads better. If the result of the projection will be used as a temporary table in a join, the use of CoordConvertTileSetPar is thus preferable. However, if the result of the projection will simply be used to populate a persistent table, it is better to use CoordConvertTile because this will avoid creating a copy of the projected image in memory.
There is a new GeomCoordLine SQL function that takes a line and a distance and returns the coordinate on the line at the specified distance from its start. The function considers all line branches: if the first branch ends before the specified distance, the function moves on to check the second branch and so on, until the distance is reached. If the distance is negative or bigger than the total length of all line branches, the function returns a NULL value. There is a new Line Coordinate transform template (available when used on a field of x2 type - future builds may perhaps rework it so that it is used with a geom field instead) that uses the function.
There is a new GeomToCoordsLineSequence SQL function that takes a line and returns a sequence of coordinates at the specified distances on the line. The distances are specified as begin distance, end distance and step, similar to ValueSequence. The step can be positive or negative. For example, for begin = 0, end = 5, step = 1, the function will return coordinates at 0, 1, 2, 3, 4, 5, and for begin = 5, end = 0, step = -1, the function will return coordinates at 5, 4, 3, 2, 1, 0. One or both of begin and end can be outside of the line. For example, for begin = -2, end = 50, step = 10, the function will return coordinates at 8, 18, 28, 38, 48. The coordinates are returned as a table with two fields: distance from the start (Value) and coordinate (XY). There is a new Decompose to Line Coordinates transform template that uses the function.
There is a new CoordSystemScaleXYToUnit SQL function which takes a coordinate system and a unit name and returns the XY scale required to scale the geom in the coordinate system to the specified unit. If the specified coordinate system and unit are incompatible (for example, if, the coordinate system is metric, but the unit name is 'degree'), the function returns a NULL value.
Discussion: The idea behind CoordSystemScaleXYToUnit is to use it together with GeomScale or GeomScaleRev instead of more complex coordinate converter functions, to perform measurements in the desired units when the math required is minimal. For example, if we want to find a location on a line 2 miles from the start of the line, we can call CoordSystemScaleXYToUnit and pass mi to get the scale required to convert the line coordinates into miles, call GeomScale with that scale on a line, call GeomCoordLine and pass 2 as a distance, then convert the returned x2 value to a geom using GeomMakePoint, and finally call GeomScaleRev with the exact same scale as before to convert the created geom to the coordinate system of the line.
Other
(Fix) The TileGeomXxx SQL functions used in the Join dialog for raster to vector joins no longer fail to produce values for some of the lines.
Trying to use a transform that needs to know the bounds of a map layer on a layer based on a query that does not know them will auto-computes the bounds (instead of silently failing).
The Join dialog copies tile reduce / tile size / tile type properties for newly created tile fields.
The Reproject Component dialog for an image allows specifying interpolation type: nearest neighbor (the only former option), bilinear (the default since this build), and bicubic.
Numerous topics and illustrations have been updated, with documentation as of build 9.0.171.2 consisting of over 9600 illustrations and 680 topics. New topics in recent builds include:
Show / Rename / Delete Related
Tools - Edit Localization File
Example: Resize an Image using Merge - We can change the size of an image while maintaining georegistration by using the Merge Images command. This example shows how to take an image that is 3,038 x 4,334 pixels in size, using approximately 36 meter pixels, and to create a re-sampled image that is 1,115 x 1,590 pixels in size, using 100 meter pixels.
Example: Use Edit - Join to Map a Pandemic - Creating a data source for a CSV web server, we acquire current data on cases and deaths worldwide for the Covid-19 pandemic. We then use the point and click Edit - Join dialog to rapidly join that data, automatically aggregated as we desire, into a world map for visualization.
Example: Using the Select Pane Expression Tab - Writing snippets of SQL into the Expression tab to make selections.
Significant extensions to Edit - Join allowing spatial joins between drawings in a map and between drawings and images in a map. This makes it very easy to join drawings and images using a point and click dialog instead of as formerly requiring use of transforms and SQL. See the updated Edit - Join topic for initial examples of the new capabilities below.
Join (drawing -> drawing)
The Join dialog allows joining data from one drawing to another, a join dialog way to do spatial overlays.
To perform a join between drawings, open a map containing both drawings as layers, click the drawing layer, and then launch Edit - Join.
The Edit -Join dialog for drawings is the same as used for joining data between tables, however, the list of source components is limited to map layers. This both extends the range of possible joins in that we can use the dialog to join components from different data sources, and simplifies it in that we do not see all drawings from the data source of the target drawing.
Joining data between drawings allows selecting a join condition from the list of the usual spatial operations: adjacent to, contains, contained in, intersects, and touches, the default.
Joining data between drawings is always x:N, that is, a single target record gets values from zero or more source records. There is no need for uniqueness which sometimes exists with joins between tables. Consequently, transfer rules are always aggregates such as average, min, max, sum and so on.
Joining data between drawings uses multi-threaded GeomOverlayXxx function variants for automatic CPU parallelism.
The update query, if saved, is created in the data source of the map to allow joining data between any pair of layers. (If the update query were created in the data source of the target drawing, the source layer might be unreachable.)
Join (image -> drawing)
The Join dialog allows joining data from an image to a drawing. This is an extended version of the transfer heights feature in 8, and formerly requiring SQL in 9, as in the Example: Transfer DEM Terrain Heights to Areas in a Drawing topic.
To perform a join from an image to a drawing, open a map containing both components as layers, click the drawing layer, and then launch Edit - Join. The Join dialog will allow joining data from either a drawing or an image.
The only available join condition is contains, but the name of the condition is saved in a separate localization token from the same sondition used with joins from one drawing to another, so that it can be localized differently.
When joining data from an image, we select a channel instead of a field. Joining data from an image to a drawing is always x:N, that is, a single target geom gets values from zero or more source pixels. Consequently, transfer rules are always aggregates. The list of available aggregates is similar to that available for numeric fields, except the copy rule, which for an aggregate means 'use any value out of multiple values from multiple matching records', is replaced with sample, which does the same thing, just faster.
Joining data from an image to a drawing uses multiple threads through a nested SELECT with THREADS for automatic CPU parallelism.
The update query, if saved, is created in the data source of the map to allow joining data between any pair of layers.
Functions
The TileFilter and TileFilterSum query functions produce missing pixels if the filter shape covers no valid pixels (formerly were producing pixels with the value set to 0). This is consistent with other TileFilter functions.
There are new query functions for tile filters: TileFilterStDev, StDevPop, Var, VarPop, and Count. Each function has a GPGPU variant. Each function has a corresponding transform.
The TileGeomAvg, Diversity, DiversityIndex, Major, Max, Median, Min, and Sum query functions take a channel number as a parameter, to work with a single channel in a multi-channel image.
There are new query functions for geom tile statistics: TileGeomStDev, StDevPop, Var, and VarPop. The functions take a channel number as a parameter, to work with a single channel in a multi-channel image.
There is a new query function for capturing a sample pixel value under a geom: TileGeomSample. This is the fastest way to capture data for a geom from an image. The function takes a channel number as a parameter, to work with a single channel in a multi-channel image. (We might add functions which take multi-channel samples in the future.)
The GeomConvexHull query function takes a boolean parameter which controls the behavior of the function when the produced convex hull is a single point or a straight line segment. If the parameter is false (former behavior), the function returns NULL. If the parameter is true (optional new behavior), the function returns an area with coincident coordinates.
Miscellaneous
(Fix) The Stack Vertical command in layout window no longer stacks frames in bottom up order instead of top down order.
The GPGPU.DAT file is updated.
The ValueFieldChannelF1 localization string has been replaced with ValueFieldChannel to force unique channel names even if the localized version of the string messes up and omits the {0} escape sequence for the channel number.
The Join dialog can be used on a persistent query, if the result table is writable. This allows joining data to, for example, a table alias (TABLE [datasource]::[table]) or a selection in an existing component (TABLE CALL Selection([component], true)). We cannot join data to an ad-hoc query shown in a Command Window because that query has no persistent name and, as such, cannot be addressed from an update query.
The Join dialog allows adding multiple fields at once using the Fields command (shows a dialog) in the dropdown menu for the Add toolbar button. The number of fields listed in the dropdown menu is limited to 20.
The Join dialog automatically copies coordinate systems for the new geom and tile fields from their specified source fields.
The Center and Convex Hull transfer rules now work for all types of geometry (were working for lines and areas)
Highly recommended for all. This new build introduces a new dialog providing the massive power of JOINs with point-and-click ease, the Edit - Join dialog - no SQL required. The build also saves GPGPU users from being stranded by newer NVIDIA drivers that support only the latest GPUs. It introduces new infrastructure for very high accuracy reprojection of images, with the ability to convert coordinate systems with accuracy more detailed than the size of a single pixel. The new infrastructure also provides required functionality for upcoming image georeferencing capabilities. PBF functionality continues to be improved, with the ultimate goal of being able to work on all of OpenStreetMap, literally all roads and OSM data for the entire planet, on an average desktop computer. Other features include new and improved commands for layouts, formats, and more.
Sub-Pixel Image Reprojection
The Reproject Component dialog for images includes an option to specify the reprojection method. Available methods are:
standard (fast) - the traditional reprojection method used in 8 and used in 9 for display, the default,
direct sub-pixel (more accurate) - an improved method which takes longer, but produces a better quality reprojected image.
The direct sub-pixel reprojection method allows specifying the number of pixel subdivisions, 1 being the default and enough for most cases. Increasing the number of subdivisions improves image quality (with diminishing returns) at the cost of processing time.
New query functions support sub-pixel reprojection:
CoordConvertTileSet - Takes a coordinate converter, an image in the source coordinate system, a rectangle in the target coordinate system and the number of pixel subdivisions, and returns a table with image tiles projected to the target coordinate system.
CoordConvertTileSetPar - A parallel variant of CoordConvertTileSet.
The above two functions might be renamed in the near future, so names can be reused for similar functions using the standard reprojection method.
GPGPU
The GPGPU interface is no longer tied to a particular version of CUDA. It can use CUDA devices starting with compute capability 2.0, that is, Fermi GPUs.
GPGPU modules are now loaded from a GPGPU.DAT file that is included within install packages for 64-bit versions of Manifold and Manifold Viewer. The GPGPU.DAT file contains multiple versions of each module supporting different compute capability generations. The GPGPU interface automatically loads modules for the latest version that can be used on the CUDA device present in the system. If the system has multiple CUDA devices, the GPGPU interface loads modules of the latest version that can be used on all devices, to enforce consistency and avoid recompilations. Currently, GPGPU.DAT includes modules for compute capability 2.0 (Fermi), 3.0 (Kepler) and 5.0 (Maxwell). Performance differences between versions are mostly small, with later versions performing very slightly better than earlier ones.
Edit - Join Dialog
The new Edit - Join dialog alters a table, called the original table, by adding new fields, which are populated with data taken from another table or query, called the joined table. It provides a point and click way to use the power of JOINs without writing any SQL. The original table also may be called the target table, and the table providing data also may be called the source table.
The original table, is shown on the left and cannot be changed. The joined table providing data for the join is shown on the right. The joined table can be a table or a query in the data source of the current component. (Tables or queries from other data sources can be used by creating a query within the current component that passes them on.)
Three boxes below the original and joined tables specify the join criteria, with a key field from the original table to use, the join method to use, and a key field from the joined table. The only join method available for tables is equal, with fields chosen from each table having common values that can be used to match between tables. As upcoming builds expand the Join dialog to allow joins between drawings and other components, additional join methods (for example, spatial relationships such as contained) will be added.
For both tables, the dialog tries to guess the field to use as a key field. For the original table, the dialog tries to use a field (other than mfd_id) that has a BTREE / BTREENULL index, with a data type preference for numbers, then text, then everything else, and a name preference for field names with id in them ('..._id' = '... id' > '...id' > everything else). For the joined table, the dialog uses a similar logic, but first tries to use a field with the same name as in the original table.
The central panel of the dialog lists fields in the original table. Pressing the Add button in the toolbar will add a new field to the list, to be populated from the source table. The list has three columns (not marked by vertical lines, but still there) which provide the name of a field in the original table, a transfer method, and the name of a field in the joined table that provides data. Fields in the original table that participate in unique indexes (BTREE / BTREENULL indices, including the implied system index on mfd_id fields in .map files), fields used as join criteria, and computed fields will be shown in the list with a gray background, and are read-only. Other fields in the original table will be shown with white background, and can be populated by data from the providing table, using the transfer method picked in the middle column and the source field in the last column. To set an original field to be populated from a field in the joined table, edit (double-click, or click and press Enter or F2) the empty space for either the transfer method or the joined field. Editing the transfer method lists all transfer methods which can be used with the original field and with any of the joined fields. Editing the joined field lists either all fields in the joined table which can be used with the selected transfer method or all fields in the joined table which can be used with any transfer method, if there is no transfer method as yet selected. Editing both the transfer method and the joined field allows selecting '(none)', excluding the field from the join.
If the join criteria are such that for each record in the original table, there is at most one record in the joined table, the list of transfer methods is limited to copy (used when original field and joined field data types are exactly the same) and convert (used when data types are different but an automatic conversion can be done).
If the join criteria are such that for each record in the original table, there might be multiple records in the joined table, the list of transfer methods includes all transfer methods available for aggregated fields in the Transform pane, such numeric aggregates, spatial aggregates, string aggregates, etc.
If the original table allows adding new fields, the Add button will be enabled in the dialog's toolbar. Clicking the Add button shows a drop-down menu of fields from the joined table. Selecting a joined table field from the menu adds a new field to the list of original table fields in the dialog, sets the transfer method for the added field to copy. and chooses the selected joined table field as the source field for populating the new field to be added to the original table. The name of the new field can be edited. The data type of the new original table field is not shown but is set to be the same as the data type of the source field. New fields are shown with blue preview color background and have a distinctive icon in the record header. If we change our minds, unwanted new fields can be selected with a Ctrl-click and deleted using the Delete button in the toolbar.
The list of fields can be filtered by name using the filter box, to show only those fields containing the text given in the filter box. The filter matches either original table field names or joined table field names, if any are in the list.
To perform a join, click the Join Component button at the bottom of the dialog. That will copy data from the joined table to fields in the original table. Any changes to data in the joined table made after the join will not automatically appear in the target table. We can save the join operation as a query using the Save update query option: that creates a query that performs all JOIN operations required to duplicate what the Join Component button did given current settings in the dialog. We can run that query to repeat the join operation whenever desired. Since each saved update query is just a query, we can have as many such queries in our project as we want. We can also customize what those queries do by editing the SQL in them.
The Edit - Join command is enabled when the focus is on a writable table window. It is disabled for queries, for system tables such as mfd_meta or mfd_root or mfd__srid and similar, and for read-only tables such as tables that do not have an index (and thus cannot be updated) or tables on read-only data sources.
Edit - Join works only on tables within the same data source, for example, all tables within the main body of a .map project, but not within linked data sources. We can join data from a table or query in a different data source by using an intermediate query. Create a query within the same data source as the original table containing query text such as TABLE [datasource]::[table] that cites the desired component within the foreign data source. That works as an alias for the table in the foreign data source, and it will be visible in the Join dialog's pull down menu of tables and queries we can use as the joined table.
With the focus on the Edit - Join dialog, pressing F1 launches a quick reference guide to the dialog. The initial quick reference guide is simple, but it is expected to grow. Quick reference guides for other dialogs as well are likely to appear in future builds.
PBF / OpenStreetMap
(Fix) Reading PBF no longer sometimes skips blocks of lines and areas, a problem that appeared occasionally with exceptionally large PBF files.
Reading PBF splits PBF data tags into multiple tables based on the tag name: a table for tags starting with addr, a table for tags starting with building, a table for tags starting with source, and a table for all other tags. This change allows importing whole-world PBFs covering the entire planet without running into the current limit of 2 billion records per table. After the 2 billion record per table limit is removed in upcoming builds, the PBF dataport again will read all tabs into a single table.
Reading PBF has been optimized to perform significantly faster, particularly on huge files.
A note on PBF work: Manifold's objective is to enable import of the entire OpenStreetMap dataset for the entire planet, for practical work on desktop computers, a realistic objective given the speed and capacity of Radian technology used within Manifold. Removing roadblocks from a per table record limit, as cited above, is a step toward that goal, along with successful removal of two or three similar roadblocks elsewhere. Whole-planet dataset work is now focusing on performance. In the past, reading a PBF with OSM data for the entire planet formerly took multiple days, before running into limits and failing. Improvements now allow the process to run successfully, but it still requires almost a full day to complete, due to the very large amounts - over a terabyte - of intermediate data processed when decoding large and complex PBF data structures. Future builds will reduce the time required for that. Insights gained from dealing with very large, whole-planet, vector data sets will result in performance improvements in many other large vector data settings.
Other
Stack Horizontal and Stack Vertical commands for layout windows now support stacking frames of varying width or height, instead of stacking evenly using the only the width or height of the active frame.
New Align Center Horizontal and Align Center Vertical commands for layout windows align layout frames to a common center.
(Fix) Switching the cursor mode in a layout window no longer fails to update the cursor mode icon on the toolbar.
Various favorites dialogs require selecting new favorites to add them to the list of current favorites. Adding new favorites to the list of current favorites does not remove them from the dialog.
ECW support has been updated to the latest version of the ECW SDK, incorporating fixes to reading ECW and JPEG2K files.
WEBP support has been updated, incorporating fixes to reading and writing WEBP files.
The V8 Javascript engine has been updated, incorporating improvements to Javascript compilation and execution, including reductions in the amount of used memory.
(Fix) Attempting to export an XLS file no longer sometimes fails in the creation of the initial XLS file.
Reading KML recognizes more variants of geometry data, including all types published by Google in KML samples.
Reading KML recognizes more variants of URL links, such as NetworkLink, etc.
Build 9.0.171 is an Official build. This build or more recent is a must have build to replace all prior builds, for all Release 9 and Manifold Viewer users. Manifold System 9.0.171 delivers hundreds of enhancements released in the preceding series of Cutting Edge builds (see individual Cutting Edge release notes for details).
Highlights in 9.0.171 can be grouped into eight major sections:
Multiple raster operations now use data structures optimized for heavily multi-core systems, for significantly better performance.
Many new raster transforms for Euclidean distances, path distances, and slope path distances groups with direction, distance and source Z transforms within each group
Projections:
Reprojecting data can use NADCON5 transforms
Grid files for many standard NADCON5 transforms found in EPSG have been included in the grids.dat file in the downloads page, providing hundreds of transformation grids for NADCON5, NADCON, HARN, and NTv2.
Layouts and Legends:
Layout frames are rendered with anti-aliasing and other advanced features.
The new Insert Legend command automatically creates a legend for one or more components, producing layout frames for legend entries that can be manipulated individually or as a group.
Layout frames for legend entries can be further edited in the Record pane.
Many new commands automatically align, stack and resize multiple layout frames at once for quick editing.
Moving multiple layout frames with the cursor uses an improved version of snap that is much more intelligent.
The New Layout dialog allows choosing paper format and orientation.
The new Page Setup dialog allows switching to a different paper format / orientation and allows using a custom page size.
Extended and Improved User Interface:
A new series of smart commands allow renaming related components all at once, automatically updating references and names in related components. Show Related and Delete Related commands make management of related components really easy.
The New Map dialog has been reworked to include use of a base image server layer, like Google or Bing, to remember past choices and to allow picks from all potential layers in the data source. That allows much faster and easier creation of map.
The Layers pane has new controls for fast work with layers, like making a layer active in a map.
Double-clicking text in an edit control uses smart select, for example, selecting a clicked word and matching bracket characters, faster and easier editing.
Clicking an active cell in a grid automatically opens it for editing, for more intuitive editing, like in Excel.
LiDAR:
Spatial indexes for LAS / LAZ files perform much faster.
A new librarylas dataport can link an entire folder hierarchy of LAS / LAZ files as a single LAS library data source with a drawing and table showing data from all files merged together, automatically managing different schemas and projections.
LAS library data sources can dynamically update with a refresh, to incorporate additions, deletions or changes in files that participate in the library.
A new Index LiDAR Files command in the Tools menu allows building indexes in advance for a folder hierarchy of LAS or LAZ files, for use with LAS/LAZ or librarylas dataports.
Tools building point cloud spatial indexes can now build indexes for multiple files in parallel.
LAS libraries detect LAS / LAZ files without an index, showing a new icon in the Project pane, and providing a context menu command to quickly build indices.
Dataports:
Exporting a GPKG file publishes coordinate system data in WKT format for third-party applications.
Reading an ECW file supports 16-bit values without reducing them to 8-bit, and supports multi-channel images.
Reading a KML file can download referenced data from web resources, recursively with a custom recursion limit.
A new web KML dataport allows linking KML data from a web resource.
Reading a NITF file supports reading embedded JPEG2000 data.
Localizations:
A new Edit Localization Files command in the Tools menu provides a sophisticated editing system for creating, editing, and updating localization files.
The Tools - Options dialog allows specifying the localization to use, allowing non-standard names to facilitate testing new localization files being translated.
For detailed lists of changes, see documentation or threads for individual 9.0.170.x builds in listings below.
Specific enhancements since build 9.0.170.5 include:
New features wrapped into the 9.0.171 official build primarily revolve around a sophisticated new editing console, the Edit Localization File dialog, that supports the creation, editing and updating of localization files.
Edit Localization File dialog
The Tools - Edit Localization File command launches a new editing console for creating, editing and updating localization files used in translations. The dialog opens with three columns, two of which are read only and show a list of all tags with localizable strings and a list of the default English string for each tag. A third read/write column in white allows specifying a translation for each tag.
The [...] browse button at the top of the dialog opens an existing localization file and loads it into the rightmost column, automatically matching tags in the localization file with the list of tags and English defaults in the leftmost two columns. The opened localization file can be UTF16, UTF8 or ANSI. New translation files can be created from scratch, by editing the blank cells in the rightmost column to add translations for tags, and then pressing the Save button.
Double-click a cell in the third column to open it for editing, which launches the Edit Localization String dialog. Clicking a cell pressing Enter or F2 also opens it for editing.
The Save button saves changes made in the dialog into the opened localization file. If there is no opened file, the dialog asks what file to save the data to. The saved localization file is always UTF8 with BOM.
The Edit Localization String dialog used to edit the translation for a string shows the Tag and the English default. Command buttons below provide one click options for choices that translators frequently make:
Use Default sets the translation to the (default) English text.
Use Existing sets the translation to an already-specified translation for a different tag with the same default English text. If there is only one translation variant found for all tags,, the button sets the translation to that variant. If there are multiple variants, the button shows a drop-down menu with choices, showing a translation for each. This allows quickly reusing translations for similarly named controls or commands.
Use Unmatched sets the translation to that used for a string attached to an unknown tag. Unknown tags happen when loading a localization file that was created for past builds and may include tags no longer found in current Manifold builds, or renamed in current builds. This feature allows quickly reusing a translation made in the past for the new name of a tag. If the file contains only one string attached to an unknown tag, the button sets the translation to that string. If the file contains multiple strings attached to unknown tags, the button shows a drop-down menu with choices, showing a tag and translation for each. If the number of strings attached to unknown tags is big, the list of choices is limited to the 20 choices that best match the edited tag.
Loading a file in the Edit Localization File dialog detects strings attached to unknown tags and saves all such strings at the end of the file after a '###' separator. (This sometimes happens when tags themselves get renamed between the builds - the change keeps the translation attached to an old tag in the file instead of throwing it out.)
The Edit Localization File dialog can be resized.
The list of strings can be filtered using a text fragment in the filter box. Filtering keeps strings which contain the text fragment in either the tag, the default value, or the translated value.
Pressing the filter icon next to the filter box can filter the list of strings to: all strings (All), strings with a translation (Localized), strings with a translation that coincides with the default English text (Localized as Default), strings without a translation (Unlocalized), strings changed since the last save operation (Edited), strings unchanged since the last save operation (Unchanged).
The dialog reports the number of shown strings and the total number of strings in the readout on the left bottom.
The dialog includes an option to save unlocalized strings as comments. The option is on by default. This produces files useful for localization in third-party editors outside of Manifold.
More Localization Features
(Fix) Loading localization data no longer throws an error if the last line of the localization file does not contain end of line characters.
Loading localization data supports files with Unix-style line ends.
The Options dialog allows specifying a localization to use. The choices are:
(auto) - use localization files matching the UI language for the user specified in the OS (previous behavior and the default),
(none) - ignore localization files and stick to the built-in English strings, or
a specific language - The dialog automatically lists languages in files that are named in the form ui. ... .txt, found in the same folder as manifold.exe.
Switching a language in the Options dialog only takes effect after a restart - or in a new instance of Manifold launched after the Options dialog has been closed.
Other
A new Same Format command for a layout applies the active frame's format to all selected layout frames, making it easy to format many frames in the same way. This is especially useful with legends.
A new Resize to Selected command for a layout resizes and moves the active layout frame to fit selected frames. This is typically used to resize and move a base legend frame so it wraps around legend frames that have been altered and moved about.
The formatting sample in a sample (legend) layout frame can cover the entire frame if the frame text is empty. This allows creating legend lines with multiple formatting samples.
A new Margins command for a layout toggles virtual page and frame border guidelines in the layout window. This provides a final look at the layout without virtual borders used in editing getting in the way.
Reading PBF data performs faster and uses less memory.
A more prominent icon is used for any data source that needs user intervention to capture additional data not yet usable, for example, a database with a Release 8 Enterprise storage that contains components not yet migrated to regular tables, or a LAS library that contains files not yet indexed.
All multiline edit controls support smart double-click select, as well as select all (Ctrl-A) and bracket matching (Ctrl-] and Ctrl-Shift-]).
Many improvements have been added to layouts and legends, plus significantly faster indexing of LiDAR files through a) parallelism and b) background operation.
Invoking the Zoom to Selection command for a layer tab corresponding to a group of layers zooms the map window to the combined extents of the selection in all layers in the group.
The Layers pane allows zooming to one or more map layers using Center / Zoom / Zoom to Selection commands in the context menu.
The Layers pane allows resizing one or more (to all selected) table fields using Best Fit / Best Fit Title commands in the context menu.
The main application window tracks changes to the opened MAP file and includes '*' in the caption if there are unsaved changes. Changes to data in linked data sources do not count as changes to the opened MAP file: For example, attempting to close a MAP file with no changes to its own data but with unsaved changes to data in linked data sources will still prompt to save changes.
Web CSV, web KML, and KML data sources log each downloaded web resource.
Reading PBF data reports record type as an int8 instead of as a string, for better performance.
Layouts and Legends
Samples in sample layout frames have been made square by default. If the height of the frame exceeds its width, the sample height will be reduced to be square: the width of the sample will always be at least the height.
The Record pane allows specifying the sample shape for an area, line, or point sample layout frame, as used with vector layers.
The Record pane allows specifying the sample shape for a color sample layout frame, as used with raster layers. . If the frame is set to show transparent color, the sample will not be painted.
The Record pane allows the specifying sample text, by default Abc, for a text sample layout frame, as used with labels layers.
The Record pane allows specifying the sample width, either as a value in points or as a percentage of height, by default 100%, and the position left (the default) or right within the frame for a sample layout frame.
The Page Setup dialog for a layout has been reworked. The new dialog allows using either a standard paper format or a custom one. The width and height of the page, as well as page margins, are specified in using millimeters, inches, or points, with the last used unit of measure being remembered.
The Page Setup dialog automatically rounds layout measurements: millimeters are rounded to 1 decimal position, inches to 2 decimal positions, and points to 1 decimal position.
The New Layout dialog allows specifying paper format and orientation and whether or not to use margins.
The last used paper format, orientation, and margins options in the New Layout dialog are saved between sessions.
Snapping layout frames to each other during moves and resizes behaves much smarter. For example, moving always preserves the original size of the frame, resizing preserves edges that were not dragged, and If there are multiple snap options the best one is chosen.
Layout windows allow aligning selected layout frames to a specific edge of the active frame (pick with an alt-click) using Align Left, Align Right, Align Top, and Align Bottom commands in the Edit - Align menu. Commands also are replicated in the main toolbar in the new Alignment button.
Layout windows allow resizing selected layout frames to the same size as the active frame using Same Width and Same Height commands in the Edit - Align menu and in the toolbar Alignment button.
The layout window allows stacking selected layout frames around the active frame using Stack Horizontal and Stack Vertical commands in the Edit - Align menu and in the toolbar Alignment button. The stack order will be the same as the Z order shown in the Layers pane. Depending on its Z order within the display stack, the active frame can therefore end up at the top or left, of at the bottom or right, or somewhere in the middle of the stack.
LiDAR
There is a new Index LiDAR Files command in the Tools menu that allows scanning a folder of LAS / LAZ files and building indexes for them. The Index LiDAR Files command is available even if there is no opened MAP file. The Index LiDAR Files command is intended to allow background indexing of LiDAR files while we continue work on other things. To use the command, launch a Manifold session and launch the command. Choose the folder (and thus subfolders) containing LiDAR files and allow that Manifold session to index them all. We can continue working in other Manifold sessions while the session running the Index LiDAR Files command does its work. Indexing LiDAR files takes time, so if they are indexed in background ahead of when they are needed, they can be quickly linked into Manifold projects when they are needed, either using the LAS dataport or the librarylas dataport.
The Index LiDAR Files command opens a dialog which allows specifying the folder to scan. After the folder is specified, the dialog scans that folder with all its subfolders and lists all LAS / LAZ files found. Each found file shows a status, which can be:
no index - No .MAPINDEXP file with a Manifold LiDAR spatial index.
no metadata - No .MAPMETA file with Manifold coordinate system info and record counts
the number of points - Reporting the number of points in the LAS or LAZ file.
Selecting one or more files and pressing the Index button creates both spatial indexes and the metadata files for all selected files. The indexing process reports progress and can be canceled.
The Index LiDAR Files dialog includes an option to index multiple files simultaneously. The maximum number of files indexed in parallel is 8 for 64-bit mode and 2 for 32-bit mode. That provides reasonable parallelism for indexing speed while leaving reasonable resources for continued work in other Manifold sessions or other applications.
Linking a LAS / LAZ file with no previous spatial index built and then opening its table will show table records without waiting until the index is built.
Linking a LAS / LAZ file with no spatial index built and opening its drawing will show that the spatial index is being built by underlining the layer tab in black. After the index is built, the map window refreshes automatically.
The LAS library dataport no longer builds spatial indexes automatically. Instead, the library creates a list of files without the index and allows building spatial indexes on them using the new Index Excluded Files command in the Project pane context menu. Indexing files for the LAS library always uses multiple threads. The process of indexing reports its progress and can be canceled, with partial results accepted by the library.
The LAS library reports the total number of files included in the library and the total number of files excluded from it (due to having no indexes) in the log window. If the library data source contains one or more excluded files, the Project pane shows the data source using a distinctive icon.
The coordinate system for LAS is always forced to XY axes. Experience shows many files specifying an YX system with the intent of X in the file still remaining X for analysis and display puposes, etc., but no files specifying an YX system with the intent of actually reversing X and Y stored in the file. In a word, it is That YX Thing again. If LiDAR files claiming YX coordinate order and actually using YX coordinate order ever turn up, Manifold will adjust the option to allow a choice, as some other formats do.
(Fix) The LAS library reads an edited coordinate system for a LAS file from its accompanying .MAPCACHE.
ALAS library data source can be refreshed using a Refresh command in the Project pane context menu. That allows the data source to pick up and incorporate new files put into the folder on which the library data source operates. Refreshing a library data source may change the schema of the exposed table, because different versions of LAS files expose different fields, and because LAS files may contain custom fields.
A massive new build with numerous quality of life UI improvements, improvements in layouts including a first step in legends, faster algorithms in geoprocessing, new capabilities in dataports including KML and LiDAR, and significant improvements in Path geoprocessing.
A big change: Maps are assigned the projection with which they are created, using the New Map dialog. They no longer take their initial projection from the first component dropped into the map. See the newly updated Maps and File - Create - New Map topics.
The filter button in the New Map dialog has been renamed from Selected to Picked.
(Fix) The New Map dialog no longer gets stuck in showing no layers once all layers are unpicked and filter is set to show only picked layers.
The New Map dialog allows filtering layers by name using a filter box.
The New Map dialog allows adding a base layer using one of the Favorite image servers. Adding a base layer creates a new data source for the specified image server or reuses an existing data source if it already exists.
The New Map dialog saves the last used base layer between sessions.
Google image servers in the default favorite data sources have been switched from v2 to v1, to work without an API key.
Dropping layers into an empty map no longer sets the projection of the map to that of the first layer.
(Fix) The Component pane for a map window shows the name and projection of the map if the map is empty (was showing nothing).
Double-clicking text in a comments / query / script / log window selects:
whitespace (a sequence of space / tab and similar characters), or
a word (a sequence of non-whitespace characters) enclosed in one of the recognized bracket pairs such as (), [], {}, ##, or
an identifier (a sequence of letters / numbers / underscores), or
an ellipsis (a sequence of repeating punctuation characters like ----- or //////).
See the Editing Queries, Scripts and Comments topic.
Opening a map zooms to the extents of all layers, excluding layers for image servers.
The list of styles for areas includes a style for solid fill with a resizable border: clicking area size changes the border stroke width.
Clicking the active cell in a grid starts editing its value (repeat-click).
(Fix) Editing the value of a grid cell adjusts the height and position of the edit control to avoid clipping its top edge.
(Fix) The Find and Replace dialogs for a table window only list fields currently visible in the window (attempting to use hidden fields was doing nothing).
(Fix) The Best Fit / Best Fit Title commands in a table window correctly handle very long strings.
The Replace dialog for a comments / query / script window allows replacing all occurrences of text using the Replace All button.
The layout window paints all frames with anti-aliasing and other advanced features of the Normal graphics engine. (Previously, component frames were painted using whatever graphics engine was specified in the Options dialog, but text frames were always painted using the Basic graphics engine due to performance requirements.)
The Record pane separates coordinates for a layout frame into a separate Position tab. In addition to left / right / top / bottom coordinates, the Position tab also shows width and height, and switches between editing any two of the left+bottom / width+height / right+top combinations, with the third being computed automatically using radio buttons.
The Record pane saves the last used unit, inches, mm, or points, for layout frame coordinates between sessions. The idea is that it usually is set once and then forgotten about it for most layouts.
Applying changes to a layout frame in the Record pane using the Update Record button or picking a different layout frame in a layout window using Alt-click preserves the active tab (Style or Position).
Layouts may include frames for color / area / line / point / text samples as used with legends. Sample frames show a formatting sample and an optional text. The width of the formatting sample is computed automatically from the frame height but can not exceed half of frame width. The layout window allows inserting sample frames, moving and deleting them, and editing their content in the Record pane.
The layout window allows generating a legend for a component shown in the layout using the Create Legend command, in the pull down menu for the cursor mode and also in the View - Mode submenu. The command shows a New Legend dialog which allows picking a component for which the legend will be created. If the component is a map, the dialog allows picking specific layers to include in the legend, and setting their order. By default, hidden layers are excluded and the order is the same as in the map. Creating a legend generates one or more sample or text frames. Tech tip: Legends in this build are a first step that allow use of the basic machinery. Expect many UI enhancements, and many new capabilities in the builds ahead. Stay tuned for new topics and check out the First Look at Legends video for a quick introduction.
For a drawing, the legend includes separate sections for areas, lines and points, which are automatically simplified if the Ignore unused geometry types option is checked. If a section uses no thematic formatting, it includes a single sample frame with default formatting. If a section uses thematic formatting on a single field, it includes sample frames showing the field values and the formatting assigned to these values. If a section uses thematic formatting on multiple fields (multi-variate formatting) or on a single field but with different value splits, it includes sample frames for each thematic split.
For a labels component, the legend includes a single section for all labels, with the same rules as for drawings.
For an image with a palette applied to a channel, the legend includes sample frames showing the field values and the colors assigned to these values. If an image uses RGBA channels and channel min/max ranges are the same, the legend includes a sample frame for black color and a sample frame for white color, showing the min/max values. If an image uses RGBA channels and channel min/max ranges are different, the legend includes sample frames for red, green and blue colors showing the min/max values for each.
The Record pane allows editing the border color and border size for a layout frame. The default is no border: Border color is set to transparent. The border can also be hidden by setting border size to 0 pt.
The Record pane allows editing the padding for a layout frame. The default is no padding, 0 pt. In sample frames, padding between the sample and the text is only applied once.
Creating a virtual layout for a map window using the Print Preview command captures the current center and scale of the window and zooms the layout to the same area.
The New Legend dialog allows specifying a caption for the legend. If the caption is set to an empty string, it is not created.
The New Legend dialog allows creating a base frame for the legend. The base frame is a text frame with no text, used for its solid background (white) and border (black, 0.5 pt), put underneath all other legend frames.
Layout frames created for a legend are put into a separate folder with subfolders for component sections, to allow quick operations with the entire legend or its parts. Tech tip: Organize your Manifold desktop so both the Layers pane and the Contents pane are always visible, perhaps Ctrl-clicking the layers pane to dock it in a second row below the Contents pane. Selecting in the Layers pane and right clicking to make a desired legend frame active before moving it or modifying it with Record pane style controls is a key part of fast workflow when rearranging legend frames.
The New Legend dialog includes an Ignore unused geometry types option, which checks whether drawings actually have objects of a specific geometry type, and whether labels have any objects at all, before including them into the legend. If a drawing cannot determine the geometry types of objects it contains, for example, if it is based on a query, or if it comes from a data source other than a MAP file or a file which stores its spatial index in a MAPCACHE file, etc, it ignores the option.
The layout window paints the active layout frame using light blue preview color, and shades the content of selected layout frames using a semi-transparent red selection color. These colors are not used if the graphics engine is set to Basic.
The active layout frame uses 8 drag handles (corners and edges) instead of 4 (corners) to avoid accidentally resizing the frame when trying to move it but clicking close to the edges.
The Fill Sinks transform uses a new, faster algorithm to fill sinks when there is no limit on fill flow, and an even faster variation of that algorithm when there also is no limit on fill height. No limit on fill heigth is indicated by setting the fill height parameter to 0. This is a change from prior builds which were interpreting a fill height parameter of 0 as a parameter error. A Quick look at performance:
9.0.170.0 (4500x4500 raster, fill height of 100000 that fills everything), Fill Sinks: 571.896 sec
9.0.170.4 (same raster, fill height of 0 that fills everything), Fill Sinks: 11.464 sec
The Closest Path Xxx transforms run on Latitude / Longitude rasters keep XY scales in degrees, instead of converting them to meters using approximate scale at the center of the raster. This interprets costs in the same manner as ESRI products.
The Closest Slope Path Xxx transforms include an option to switch path direction, called Start from source. If the option is turned off (the default), paths are computed from drawing objects to pixels. Otherwise, paths are computed from pixels to drawing objects. The direction matters because a downward slope path is "shorter" (that is, lower cost) than the reverse upward path.
The Closest Euclidean Xxx transforms include an option to use invisible pixels as barriers. Manifold added this to provide an equivalent to a big recent addition to comparable ESRI analysis tools. See the new Euclidean Barriers video to see the transforms in action.
(Fix) Exporting data to GPKG writes projections.
(Fix) Exporting data to GPKG no longer sometimes puts 'no such table: sqlite_sequence' into the log window.
Reading KML supports downloading data recursively referenced by URLs in the KML. The recursion limit is specified via a link depth option in the New Data Source dialog. For example, if a KML references a different KML that references a JPEG, to download it all, the link depth option has to be set to 2. By default, when using File - Create - New Data Source the link depth is assumed to be 1 = download just one level. When using the File - Link dialog the link depth is assumed to be 0 = download nothing. See the KML, KMZ Google topic for details.
A new dataport has been added for web KML, to link KML data from web resources.
(Fix) Reading a LAS library no longer sometimes fails to read projection data.
LAS / LAS library dataports build their spatial index in background, to allow doing other things in the same Manifold session while the index is building. Previously, attempts to access LAS components waited until the index was built, which could take significant time, particularly for a LAS library, and the operation could not be canceled. Tech tip: If we open a LAS drawing while the spatial index is still building, the map window will report that there is no spatial index and will suggest building a temporary one. Do not do that, as trying to build a temp index takes longer than what is already in process. Wait until the dataport finishes building its own persistent index. Upcoming builds will let the map window know that it should not suggest anything in such cases.
The LAS library detects LAS and LAZ files with the same name and ignores the LAZ file.
Reading a LAS / LAZ file reads projection from any accompanying PRJ / PAM XML files.
(Fix) Pasting or inserting a new drawing / labels component which references a non-existent table component into a GDB no longer fails.
Reading a NITF file supports reading embedded JPEG2000 data.
This build continues quality of life additions, with many user interface improvements. Analytics introduce directional path computations on slopes for Closest... rasters. Build components have been updated, including compiler, GPGPU library, V8, and ECW and MrSID modules.
All build components have been updated to the newest versions available:
Manifold is now built using a new version of the MSVC compiler, with usual tuning of numerous optimization choices resulting in slight general increases in performance, with some functions benefiting more than others. Most importantly: no regressions.
GPGPU code has been recompiled using CUDA 10.2. The next version of CUDA issued by NVIDIA may no longer support devices of compute capability 3.0 (Kepler), so GPGPU capability in Manifold may require more recent GPUs.
ECW code has been updated to use the latest ECW modules, fixing a couple of minor bugs during reads.
MRSID code has been updated to use the latest MRSID modules, fixing a couple of minor bugs as well.
V8 has been updated to the latest version used by Chrome, providing noticeable savings memory used.
Renamed transforms and parameter:
Closest Euclidean, Xxx -> Closest Euclidean Xxx.
Closest Path, Xxx -> Closest Path Xxx,
Use equal weights -> Use equal costs.
(Fix) Closest Path transforms treat zero costs as invalid. (Zero costs can be thought of as infinite). Negative costs already were invalid.
(Fix) Tile distance / viewshed / watershed computations correctly handle images with YX coordinate systems.
TileDistanceMake and TileDistanceMakePar query functions have been split by distance type. The new TileDistanceMakeWeighted and TileDistanceMakeWeightedPar functions create a distance object for path distances. The remaining TileDistanceMake and TileDistanceMakePar functions create a distance object for Euclidean distances. See the SQL Functions topic.
New Closest Slope Path transforms and their parameters (see the SQL Functions topic):
Closest Slope Path Direction, Closest Slope Path Distance, Closest Slope Path Source - Compute path distance generating costs from raster slopes, where going downhill has lower costs.
Min slope, Max slope - Control minimum and maximum value for the allowed slopes. Slopes equal to or lower than minimum, or equal to or higher than maximum are considered to not be traversable and will be avoided by the path. The defaults for Min slope and Max slope are -90 and 90 (note that neither value is reachable).
Min cost, Flat cost, Max cost - Specify costs for minimum slope, zero or flat slope, and maximum slope. The defaults for Min cost, Flat cost, and Max cost are 0, 1, and 2 respectively. The default values weight paths going downhill as having lower costs.
New query functions: TileDistanceMakeSlope and TileDistanceMakeSlopePar - These functions power the Closest Slope Path transforms.
Transform templates operating on table windows for tile fields in tables automatically determine the number of channels in tile values from metadata and filter available transforms accordingly. As a side effect, this eliminates the double Limit Xxx transform listings for tile fields, leaving only the versions of those transforms that operate on pixel values, and not on whole tile values taken as binary data.
The Rename Related dialog is smarter at finding and adjusting similar names. For example, numeric postfixes are ignored so that Mexico 2 and Mexico Table 2 will be renamed together.
Map windows allows switching out of cursor modes for inserting new geometry on an Alt-click (pick) or Ctrl-click (select) if there is no current geometry being edited, for example, right after adding an object.
Circle and ellipse arcs are rendered more accurately, leveraging improvements in linearization code and increasing the maximum number of intermediate coordinates allowed per curve segment accordingly. The result is smoother curves at no additional cost to rendering. The improvement applies to exports and transforms as well.
(Fix) The map window no longer fails to show the snap reticule for the tracker tool when it is just starting its line.
Delete Related and Rename Related dialogs select the clicked component on startup.
Turning off background for a map in the Layers pane preserves the background color, no longer resetting background color to black.
Map windows allow deselecting any selections in all layers (including hidden ones) using the new Edit - Select None Map command.
The Layers pane allows right-clicking a map layer and making it active using the new Active command in the context menu. Making the layer active from the Layers pane will both switch the active tab in the map window and switch the active component within that tab if the tab hosts a group of layers. If the layer is already active, the Active command will show this using an icon.
The Layers pane allows right-clicking a layout frame and making it active. The frame has to be visible.
The Layers pane allows right-clicking a table field and making it active. The field has to be visible.
Alt-clicking the name of a map layer, layout frame, or table field in the Layers pane makes it active in the window.
The table window allows right-clicking a field header and showing the field in the Layers pane using the Show in Layers command.
The table window allows right-clicking a field header and sorting records using values in that field using the new Order Ascending and Order Descending commands. If the current order is already based on the clicked field, the commands will indicate that using icons. If the current order is based on multiple fields, neither of the icons will be engaged: That disallows choosing one of the commands to alter the order and make it based on the clicked field alone.
The table window no longer shows the Use as Identity command for binary fields, since binary fields cannot be used as identity.
The table window allows right-clicking a field header and resizing the field to fit its values using the Best Fit and Best Fit Title commands. Using Best Fit commands on binary fields does not retrieve field values and instead uses width that is wide enough to fit most reasonable values of the relevant type.
The New Map dialog has been reworked to always display a list of components in the data source in which the map will be created. The toolbar above the list of components includes a button that switches between showing all available components and only those components that have been picked for the map. If the dialog is launched on a single component, it starts in a mode that shows all available components, and the context component is picked for the map. If the dialog is started on multiple components, it starts in a mode that shows only the components picked for the map and with the context components all picked. The dialog will be expanded in future builds with additional features.
Exporting labels to DXF writes a text height tag to make exported files readable in various versions of AutoCAD.
(Fix) Reading an NTF NITF file with an embedded JPEG2000 no longer fails with an error. JPEG2000 data within NTF NITF files currently remains inaccessible; work continues to enable seamless reading, but for now at least there is no error.
Reading ECW reads 16-bit values in raw format. Previously, these values were mapped to 8-bit RGB.
Reading ECW supports multi-channel images in ECW v3.
A build focused on quality of life user interface features that make it easier to rename, delete and identify components that can be related to each other in projects with many hundreds of components and complex relationships, such as multiple different drawings taking data from different fields from within the same table, or created on the fly based on queries involving very many tables. The new build also includes significant new raster analytic capabilities for generating rasters with Euclidean or shortest paths to points, new features to maintain libraries of LiDAR files and faster LiDAR data indexes.
Project pane context menus include new commands for working with related components:
The Show Related context menu command pops open a dialog showing a list of all components related to the clicked component. The dialog shows how components are related. Future builds may add commands to the dialog as well. See the Show / Rename / Delete Related topic.
The idea of related components is to find all components that will need to change if a specific component changes, which often involve asymmetric relationships between components. For example, deleting a drawing does not mean the table from which the drawing takes data must be deleted (some other drawing might also take data from that table), but deleting or renaming a table means that all drawings which take data from that table will be affected. Related components include:
For a table or query: all drawings and images based on that table or query, all labels based on drawings, all maps that include drawings / images / labels as layers, all layouts that include any of the mentioned components as frames.
For a drawing: all labels based on that drawing, all maps that include them as layers, all layouts that include them or maps that include them as frames.
For an image or labels: all maps that include them as layers, all layouts that include them or maps that include them in frames.
For other components: all layouts that include them as frames.
Finding related components using Show Related works across all opened data sources. If the list of related components includes components from data sources other than the opened MAP file, the components are sorted by data source with the innermost data source appearing first (the reverse of the order in the Project pane).
Clicking a component in the Show Related dialog list shows components that reference it directly using a distinctive, chain icon to the right of the component. For example, clicking a table will mark all drawings that reference the table with the chain icon.
The Delete Related context menu command pops open a dialog showing a list of all components related to the clicked component, and allows picking components to delete by checking a box (all boxes checked by default). The list of related components includes only those components which cannot function without other components on the list. For example, maps are excluded because deleting a drawing will at most delete a map layer and the map will continue to function. Marking a component for deletion marks all components that depend on it for deletion as well. Unmarking a component unmarks all components that it depends on.
The Rename Related context menu command pops open a dialog showing a list of all components related to the clicked component, and allows picking components to rename by checking a box (all boxes checked by default). The list of related components includes only those components which cannot function without other components on the list, similar to the Delete Related dialog. Marking a component for rename allows specifying the new name using F2, or double-clicking the item, or pressing Enter. Entering a new name automatically adjusts the names of other components marked for renaming, to provide a preview of what will happen. Unmarking a component resets the name to the current name. Press Rename for the renaming to take effect or press Close to abandon renaming.
The Delete Related and Rename Related commands detect read-only data sources and disallow deleting or renaming components on read-only data sources.
Important: deleting or renaming components using a slow double-click in the regular way in the Project pane continues to work as it did before, with no effect on related components. In the future, manual renaming may become somewhat smarter; however, not at the cost of losing the ability, for example, to manually rename a component from Cities to Cities Temp and then rename Cities Backup to Cities without making any other changes to any other components.
Component windows no longer close when their main component is renamed, but instead they adjust to the new name. All types of renames are supported: those made in the UI as well as in queries or scripts. When renaming components in data sources for databases, this feature requires support from the database that hosts the component; however, all Manifold dataports for various databases have been adjusted to provide such support. Renames are not tracked for components other than the main component of a window. For example, if we open a drawing (which creates a virtual map, for the purposes of enabling a drawing window to host multiple layers) and then we drop a second component into that window, the renaming on the fly logic will detect and adjust, keeping the window open, if the original drawing is renamed, but will not adjust on the fly when the second component that was added to that window is renamed. Future builds will extend this feature to track renames of all components. In the meantime, to track renaming of all layers, save the drawing window (a virtual map) into a real map using Edit - Save as Map.
There are new query functions to create rasters with Euclidean or shortest paths to points. See the SQL Functions topic:
TileDistanceMake / TileDistanceMakePar - take a raster and the distance type, and return a distance buffer that can be used in further calls. The distance type can be 0 (Euclidean distance, the default), 1 (shortest paths, avoiding missing pixels), or 2 (shortest paths avoiding missing pixels and using values in valid pixels as a distance / travel cost). TileDistanceMakePar is a parallel variant.
TileDistanceTilesClosest / TileDistanceTilesClosestPar - take a distance buffer and a drawing with points, and return a raster with each pixel set to the Z of the closest point. (Use the Set Geom Z transform to set Zs to the desired value.) TileDistanceTilesClosestPar is a parallel variant.
TileDistanceTilesDirection / TileDistanceTilesDirectionPar - take a distance buffer, a drawing with points and the direction type, and return a raster with each pixel set to the direction of the shortest path from the pixel to the closest point. The direction type can be 0 (direction to the closest point, in degrees between 0 and 360, Euclidean distances only, the default), 1 (direction to the next pixel on the shortest path to the closest point, in degrees between 0 and 360, path distances only), 2 (direction to the next pixel on the shortest path to the closest point, a quadrant value between 0 and 7, -1 for the pixel coincident with the closest point, path distances only), 3 (direction to the next pixel on the shortest path to the closest point, a quadrant value between 1 and 8, 0 for pixel coincident with the closest point, path distances only).
TileDistanceTilesLength / TileDistanceTilesLengthPar - take a distance buffer and a drawing with points, and return a raster with each pixel set to the distance from the pixel to the closest point.
Future builds are expected to extend these functions to allow working with lines and areas, or to start from a raster with pixels pre-filled with the IDs of source objects.
Corresponding transforms to create rasters with Euclidean or shortest paths to points. See the Transform - Tiles and Raster Paths topics:
Closest Euclidean Direction - produces a raster with Euclidean directions, in degrees.
Closest Euclidean Distance - produces a raster with Euclidean distances.
Closest Euclidean Source - produces a raster with Z values of closest points using Euclidean distances.
Closest Path Direction - produces a raster with path directions, in degrees. The 'Use equal costs' option switches between just routing around missing pixels and also using pixel values as path costs.
Closest Path Distance - produces a raster with path distances. The 'Use equal costs' option works as above.
Closest Path Source - produces a raster with Z values of closest points using path distances. The 'Use equal costs' option works as above.
Future builds are expected to extend these functions to allow working with lines and areas, or to start from a raster with pixels pre-filled with the IDs of source objects.
Resolving plateaus for watersheds has been adjusted to use A new algorithm for shortest paths, which is faster and which produces much more robust results. Previously, paths may have varied depending on the order of processing tiles, which is unpredictable for parallel execution.
There is a new laslibrary dataport for a folder of LAS / LAZ files. There is an option to recursively include files in subfolders. The dataport exposes a single composite drawing for all files and manages access to individual files dynamically. If individual LAS / LAZ files are of different detail level, the composite table exposes fields for the most verbose level and files at lower detail levels return NULL values for fields they do not have. If individual LAS / LAZ files use different coordinate systems, the library picks one of the coordinate systems at random and projects data from all files to that system dynamically.
Accessing each LAS / LAZ file in a LAS library for the first time captures metadata and stores it in JSON format near the file to avoid having to re-capture it later, for better performance.
Spatial indexes used by LAS / LAZ data have been adjusted to perform better, particularly if the file is part of a library.
Exporting a drawing to GPKG puts the coordinate system of the drawing in WKT format into the GPKG system table, where it can be accessed by third-party applications. Same for copying and pasting a drawing into a GPKG database.
Path calculations using rasters can compute solutions for an entire region at once. The illustration above shows least cost paths from each pixel to the nearest green point, where cost is computed on slope. The background raster is not terrain elevation, but a computed path distance surface where each color represents the distance through the lowest cost path to the green dot. Black borders show the regions of pixels that are "closest" by lowest slope cost distance to each green point. The bright green lines are watershed lines through the lowest cost distance surface, showing what paths should be taken for lowest cost travel to each dot. Illustration taken from the Raster Paths topic.
An infrastructure build introducing significantly enhanced performance, three times faster, on heavily multi-core systems, particularly for raster computations. Improvements make better use of many threads and better use of cache memory as well, and provide significant performance increases even with relatively small (four to eight) numbers of threads. These improvements are a first step, with significant increases in core count saturation numbers expected in the months ahead. Grid transformations have been extended by adding NADCON5, the new replacement for NADCON, together with numerous NADCON5 grid transformations added to the grids.dat file.
Significant speed improvements in Watersheds computations, especially with many cores. Watersheds algorithms were chosen for test cases for performance improvements because they already were very highly optimized. Previously, watersheds computations would tend to saturate manycore processors at 8 to 10 threads (a result of the exceptional efficiency of the algorithms), with little improvement and possibly slight decreases as thread counts increased. New infrastructure allows even highly optimized watershed algorithms to utilize 24 threads with significant speed increases. Improvements scale well to larger tasks, so running 24 threads can reduce what once was an eight hour parallel job to two and a half hours. Work is underway to move saturation even further, for numerous other raster and vector computations.
Prior to 170.1 (minutes:seconds):
4 threads: 10:30
8 threads: 7:00
16 threads: 6:55
24 threads: 7:50
170.1 (minutes:seconds):
4 threads: 6:40
8 threads: 3:20
16 threads: 2:40
24 threads: 2:20
Several implemented improvements benefit heavily from increasing the cache size in the Tools - Options dialog. For desktop systems with 16 GB or more RAM memory seeking top performance from manycore CPUs with single user sessions, the cache should generally be increased from the default 4 GB to half of the available physical memory, or slightly less than half.
Exporting an image with channels that have been remapped using the Style pane to TIFF produces data with remapped channels, so the exported image looks the same as the remapped image in Manifold.
(Fix) Attempting to test a connection to a MySQL database by pressing Test in the Database Login dialog actually attempts to connect to the database instead of merely validating the connection string.
(Fix) Attempting to test a connection to an Oracle database no longer logs an error into the log window if the attempt to connect fails.
The Tools - Options dialog includes a switch to control access keys: auto (default) / show / hide.
(Fix) Importing a SHP file with no associated coordinate system info no longer sets the coordinate system of the produced drawing to the default pseudo-Mercator system instead of leaving it unfilled.
New coordinate transform type: NADCON5. Built-in coordinate system data include EPSG transforms based on NADCON5. NADCON5 is the new version of NADCON with updated grids and an optional adjustment for altitude. See the Reproject Component topic.
The grids.dat file has been updated to include grid files for NADCON5 transforms.
The new version of grids.dat is backwards-compatible with the old version and can be downloaded from the Product Downloads page on the Manifold web site.
Reprojection dialogs collapse the names of grid files in conversion paths from 'xxxxx.las + xxxxx.los' to 'xxxxx.las / los' to remove redundant repetitions.
Reprojection dialogs allow selecting a conversion path when converting between systems which have equal parameters but different EPSG codes. EPSG data contains a fair number of such conversions.
Exporting data to various ESRI formats writes coordinate system as a PRJ file in addition to MAPMETA.
Build 9.0.170 is an Official build. This build or more recent is a must have build to replace all prior builds, for all Release 9 and Manifold Viewer users. Manifold System 9.0.170 delivers hundreds of enhancements released in the preceding series of Cutting Edge builds (see individual Cutting Edge release notes for details). A summary of highlights in 9.0.170 can be grouped into thirteen major sections:
Layers in the Layers pane can be grouped into folders.
Folders can be nested within other folders to arbitrary depth.
Layers in a group can be easily manipulated together: they can be selected, reordered, assigned a desired opacity, and so on.
Layer tabs in map windows use layer groups as well.
Layer tabs for groups report rendering and caching status for an entire group, allow zooming to an entire group, allow reordering entire groups using drag and drop, etc.
Groups have their own on / off status. A layer is considered to be visible only if it is turned on and all groups it is in are turned on as well.
Alt-clicking in a map window picks up to 10 overlapping objects under the clicked location.
Picked objects can then be cycled through in the Record pane.
Moving between objects in the Record pane automatically zooms to them.
The Record pane can be used to cycle between selected objects.
Alt-clicking an image tile picks it for the Record pane.
Clicking a pixel in a picked tile scrolls to that pixel to show pixel values, which can be edited.
Explicit editing modes for inserting, moving and adding coordinates.
Snap modes to snap to grid, to a bearing or to a relative bearing, with configurable snap parameters.
Snap to coordinates can use either the active layer or an arbitrary subset of visible layers.
Snap to coordinates automatically snaps to coordinates in the same object.
Dragging coordinates uses snap modes.
New commands allow converting line segments to circle, ellipse or spline curves and back.
New commands can start, close, re-start, delete or reverse branches.
Backspace deletes the last entered coordinate, and can be used repeatedly to delete multiple coordinates.
Many editing commands have keyboard shortcuts.
All commands with keyboard shortcuts show them in context menus for reference.
The entire editing infrastructure is available in the tracker tool, allowing tracker measurements over complex paths.
The Record pane Coordinate list can be switched to show coordinates as traverse commands dynamically, with interactive changes to coordinates immediately reflected in the commands.
The coordinate list can be loaded from, or saved to, a file, using either Manifold coordinates format or standard ESRI traverse format.
Clicking a traverse command allows changing it to a compatible command (for example, a direction-distance command that specifies a line segment can be switched to an angle-direction command and back).
The Traverse Parameters dialog specifies direction type and direction units used for traverse commands, including support for all ESRI standard options.
Built-in coordinate systems now include EPSG data up to EPSG 9.8.2, adding about 1000 new systems and transforms.
New supported projection: Equal Earth.
High precision reprojection with support for NADCON / NTv2 transforms, using grid files loaded from Manifold folders.
A grids.dat file with over 200 MB of compressed definitions of 170+ grids for many countries can be downloaded separately, to avoid bulking up the build for those who do not use grids.
Parsing PRJ files supports WKT2.
Coordinate systems can be copied and pasted.
Re-projection dialogs allow a choice of conversion path between systems if there are multiple alternatives available.
Re-projecting an image allows specifying explicit pixel scale, image origin, pixel type and pixel tile size, with automatic choices available. That provides fast and easy ability to resize / resample rasters, as shown in the Example: Change the Pixel Size of a Terrain Elevation Image topic.
Favorite coordinate systems can be set either to override or to ignore scales and offsets. Favorite systems that override scales and offsets are shown with a trailing '#' in the menus.
Visuals used in grids and various controls have been slightly adjusted for a flatter, more modern look.
Grids with toggles switch toggles using the traditional Space key or plain click (not double click).
Grids with folders support expanding and collapsing items.
Keys on the numeric keypad work for shortcuts.
Menus support access keys (eg, Alt-F-O for File - Open). Access keys are localizable for international language support.
Coordinate system menus and other menus using Favorites have a new, streamlined layout.
Clicks have been reorganized both to allow more options and to be simpler to remember.
Some UI features can use repeated clicks (Alt-click to pick an object in a map window, click a coordinate to start editing coordinates, click away from the object to stop editing coordinates, etc) for faster operation.
Compute astonishingly fast, fully parallel, upstream watershed areas or stream networks on large terrain elevation data. Illustrations from the Upstream Areas and Lines topic.
New transforms compute watershed areas, watershed lines (streams), and upstream or downstream watershed objects.
Created watersheds include data for linking watersheds into a hierarchy, including Shreve / Strahler orders, individual flow through a watershed and total flow through a watershed together with all watersheds that flow into it.
Watershed computations are much faster and have more options than in Release 8.
Watershed computations can use multiple threads.
A new transform allows filling sinks prior to performing watershed analysis.
New transforms provide viewshed / visibility computations, including computing areas visible from any or all specified observers, computing a visibility index image giving the number of observers visible from each pixel, or computing a visibility index for any or all observers.
Viewshed computations are much faster than Release 8.
Viewshed computations can use multiple threads.
Spectacularly fast, fully parallel, visible areas and visibility index transforms work so fast we can see previews in real time. Illustrations show visibility zones from multiple observers, with a preview of zones for all trimmed to a radius of six kilometers, as shown in the Viewsheds topic.
Code for ECW / JPEG2000 / Unicode collates / V8 has been moved into separate DLLs that are not loaded until they are needed, reducing startup time.
64-bit installs use the same data for Unicode collates in both 32-bit and 64-bit modes, reducing install size.
Viewer installs no longer include components of V8 that were previously part of the core, reducing install size.
Install packages use a new, more robust procedure to register the ODBC driver, which allows uninstalling the driver even if Manifold DLLs have been damaged.
The ODBC driver can also be installed or uninstalled from within the application, which is useful for portable installs.
A new "everything" zip package for portable installations bundles all options, including the grids.dat file, IronPython, and client software for MySQL, PostgreSQL, and SQLite into a single, fat zip package.
A new, specialized, high speed index for point cloud data in LAS / LAZ dramatically improves speed when linking LiDAR data files.
Linking a LAS / LAZ file builds a specialized spatial index optimized for fast thinning of point data for rendering, which is stored in a separate file.
Images linked from ECW / JPEG2K / SID files allow overriding the coordinate system, performing color adjustments via styles, etc.
A new dataport supports NASA PDS files.
A new dataport for generic JSON, parses JSON data that is not necessarily GeoJSON into tables.
New dataports for web CSV and web JSON allow linking CSV or JSON data from a web URL.
Dataports for databases allow extending tables with virtual computed fields, with the values of those fields computed dynamically in a Manifold session and not saved back into the database.
Oracle connections are much faster at bulk reads.
SQL Server connections can use the newest family of SQL Server drivers available from Microsoft (MSOLEDBSQL).
Working with GDB allows editing styles or coordinate systems of existing components and allows creating new components of all types except images, storing all such changes and additions in a specialized table in the geodatabase.
Working with GPKG checks whether the SQLite stack includes geometry parsing functions used by spatial indexes, and if not, provides its own implementation of these functions to allow inserting or altering geometry data without SpatiaLite.
The GDAL dataport supports GDAL 2.3, 2.4, and 3.0.
Dozens of new functions support composing coordinate systems and coordinate converters, Euclidean / geodetic measurements, computing watersheds including specialized functions to compute various subproducts, computing viewsheds, retrieving pixel values covered by geometry, and much more.
Performance has increased yet again, in scripts, in GPGPU code, in dataports, in contour transforms, in tracing transforms, etc.
The memory cache performs much better under heavy load.
The size of the memory cache can be increased from the default values in the Options dialog, which is useful on machines with a lot of RAM.
Renamed transform templates: Viewshed Area, Visible from All -> Visible Area, All Observers; Viewshed Area, Visible from Any -> Visible Area, Any Observer; Viewshed Image, Visible Count -> Visibility Index, Observer Count; Viewshed Image, Visibility of All -> Visibility Index, All Observers; Viewshed Image, Visibility of Any -> Visibility Index, Any Observer, renamed transform template parameters: Min camera -> Min V angle, Max camera -> Max V angle. See the Viewsheds topic.
Viewshed transform templates set the refraction parameter to 0.13 by default.
(Fix) Viewshed transform templates correctly handle images with non-square pixels. The radius parameter passed to the TileViewshedXxx query functions is now in degrees / meters instead of in pixels, to allow uneven scales.
(Fix) Various UI elements that do not use menu access keys no longer show them.
(Fix) Pressing an unambiguous access key for a submenu now opens the submenu instead of merely selecting it.
Initializing the Manifold object model from a third-party application turns off services that prefetch data from opened MAP files, to be less intrusive and give the application more control over the CPU / memory.
Renamed command for vector editing: Show Metes and Bounds -> Show Traverse. See the Traverses topic and the Example: Create Parcels from Traverse Files topic.
Reading a traverse file in ESRI format supports polar directions.
The Record pane coordinates / traverse option button has a Traverse parameters that allows specifying display parameters when showing a traverse: direction type (North azimuth / South azimuth / Quadrant bearing / Polar) and direction units (Degrees / Degrees-minutes-seconds / Radians / Gradians (gons)). See the Traverses topic.
The Record pane Coordinates tab in Traverse display mode allows specifying display parameters for clicked or selected coordinates using a context menu. Ends of straight line segments can be set to define segments using: angle + distance, or direction + distance. Ends of circle arcs can be set to define arcs using: angle + arc, or angle + chord, or angle + radius, or arc + chord, or arc + radius, or chord + radius. Ends of circle arcs can also be set to use one of the following direction options: chord (makes the arc non-tangent), or radial (makes the arc non-tangent), or tangent (makes the arc tangent unless the difference between the starting tangent of the arc and the tangent of the segment preceding the arc is too large). See the Traverses topic.
Map windows supports a new snap mode: Snap to Relative Bearing. Keyboard shortcut: R. The mode is similar to Snap to Bearing, but uses the bearing of the last entered segment as a base. Using the default 4 bearing divisions for snap parameters, setting Snap to Relative Bearing when drawing a line causes line segments drawn to be at right angles to each other.
Map windows allows switching to default cursor mode using Shift-Escape.
Layout windows allows switching to default cursor mode using Shift-Escape. Switching to default cursor mode no longer clears the Record pane (and no longer cancels changes made to the layout frame selected into it).
(Fix) Saving a non-map component as a map from a map window no longer sometimes forgets to save the coordinate system used in the window into the map.
(Fix) The Component pane correctly reports the coordinate system used by a map window as XY even if the coordinate system of the relevant component uses YX axes. (Map windows always renders X horizontal and Y vertical.)
(Fix) Painting and printing layouts correctly handles components with YX coordinate systems.
Renamed menu command for collations: Edit Collation -> More... Dropdown menus used by collation pickers put favorite collations at the top of the menu.
Renamed menu commands for coordinate systems: Change Coordinate System -> Reproject Component, Edit Base Coordinate System -> More..., Edit Coordinate System -> More..., Edit Metrics -> More..., Use Default Metrics -> Use Default Values. , Edit Favorites -> Favorites. Dropdown menus used by coordinate system pickers put favorite coordinate systems at the top of the menu.
The CoordConverterMake and CoordConverterMakePath query functions no longer fail when a coordinate converter cannot be created due to an impossible combination of parameters in either of the coordinate systems, or due to an invalid coordinate conversion path, or due to a path that references a missing or invalid grid file. Instead the functions return a NULL object. (Failing the function does not behave well in query constructs with multiple threads. The change allows to recover from a failure.)
The Reproject Component dialog has been made modal. (Previously, it allowed clicking into windows underneath it, switching to a different component, etc. This was a leftover from earlier builds when panes were modeless dialogs as well.)
The Reproject Component dialog has been reworked to display a coordinate conversion path using a specialized picker control instead of a list. If the source and target coordinate systems are the same, or if they are the same save for coordinate system metrics and / or axes, the conversion path is fixed (named 'default' plus a short description) and cannot be customized. Selecting a new target coordinate system preserves the selected conversion path if it remains available. Selecting a conversion path from the list of available paths will display path definitions similar to how definitions of coordinate systems are displayed.
(Fix) The Reproject Component dialog correctly handles images with coordinate systems set to YX axes. The coordinate system of a produced image is set to use XY axes.
The Reproject Component dialog for images allows direct editing of pixel scale (local scales). The Auto checkbox (on by default) sets pixel scale to values that produce an image with roughly the same number of pixels in X and Y dimensions. There are two adjustments that make the number of pixels with Auto not exactly the same as in the original image: first, the exact scales are rounded to two digits (a scale of 1.31923 ft becomes 1.3 ft), and second, if the rounded scales differ by less than 25%, they are made the same (equal to the smaller value) to make pixels square.
The Reproject Component dialog for images allows direct editing of image origin (local offsets). The Auto checkbox (on by default) sets image origin so that the coordinate of the pixel in the left bottom corner of the image is 0, 0.
The Reproject Component dialog for images allows customizing tile type and tile size for the new image. The Auto checkbox (on by default) sets tile type and tile size to coincide with those of the original image.
The Reproject Component dialog for images reports the dimensions for the image about to be created and the size of produced image data in bytes. The exact size of image data in the storage will vary due to a number of factors (such as compression, visible / invisible masks, intermediate levels, storage overhead), but the reported size is good for a ballpark estimate.
The Reproject Component dialog for images computes the bounds of a new image with greater accuracy (32x32 intermediate coordinates instead of 8x8).
The Reproject Component dialog checks all parameters prior to performing the reprojection, or generating a query that will attempt to perform the reprojection, and fails if the reprojection cannot succeed due to an impossible combination of parameters in the coordinate systems, etc. Attempting to produce an image with more than 16 TB of pixel data also fails.
(Fix) Clicking Edit Query in the Reproject Component dialog no longer opens a window with the generated query below the active component window.
The base coordinate system picker allows copying and pasting coordinate system definitions to or from the clipboard. SRID:xxx definitions that only make sense in the context of a specific database are converted to JSON definitions prior to the copying.
The coordinate system picker allows copying and pasting coordinate system definitions to or from the clipboard. SRID:xxx definitions that only make sense in the context of a specific database are converted to JSON definitions prior to the copying. There are two paste commands: Paste pastes everything, Paste without Metrics pastes everything except coordinate system metrics.
Parsing coordinate systems from WKT and WKT2 recognizes many additional keywords.
(Fix) Reading GDB puts drawings and labels into the same folder as their tables.
(Fix) Reading E00 no longer fails to read files with compressed rasters.
(Fix) Exporting an image to BIL supports multi-channel images with pixel values other than UINT8.
Reading XYZ image format adds a postfix to the name of the table instead of to the name of the image (Filename Image + Filename -> Filename + Filename Tiles).
New snap modes and vector editing commands provide very rapid editing, both GIS and CAD-style. Illustration from the Example: Edit Areas in a Layer to Align with Another Layer topic.
Numerous topics and illustrations have been updated, with documentation as of build 9.0.170 consisting of over 9100 illustrations and 663 topics. New topics in recent builds include:
Example: Change the Pixel Size of a Terrain Elevation Image - Use the Reproject Component dialog to change the pixel size of a terrain elevation image, reducing the total number of pixels used. This process is also called resampling.
Example: Create Parcels from Traverse Files - Traverse files using ESRI traverse file format are widely used by surveyors and government organizations in the US to define parcels and lines by describing a sequence of directions, distances and curves from a starting point. Manifold automatically handles both tangent and non-tangent curves in ESRI traverse file format as well as the full variety of options used to specify angles, distances and curves. This video shows how it's easy to create a parcel from a traverse file.
Example: Edit Areas in a Layer to Align with Another Layer - Editing areas in one layer so their boundaries align, either all or in part, with boundaries of areas in a different layer is a common task in GIS and CAD. For example, we might want area boundaries in a layer with different zoning areas for tax or regulatory purposes to be guided by the boundaries of administrative jurisdictions, such as the boundaries of cities, in a different layer. This example shows how, using fast and simple techniques.
Example: Add Vertices in the Middle of a Line being Created - During the creation of a new object we can go back and make corrections, additions and deletions to coordinates already marked. In this example we start creating a new line, and then notice we have skipped over some locations we wanted to click. We go back to add those vertices (coordinates), and then we continue with creating the line.
Example: Import AutoCAD DWG and Georeference - Neither AutoCAD DXF nor AutoCAD DWG format provide coordinate system information. This example shows a typical case where documentation provided on the web site from which we have downloaded a DWG allows us to quickly and simply georeference the imported drawing.
Example: Import AutoCAD DXF and Georeference - Neither AutoCAD DXF nor AutoCAD DWG format provide coordinate system information. This example shows a typical case where we import a DXF using an unknown coordinate system, but based on a lucky guess we accurately georeference the imported drawing.
Example: Create Terrain Elevation Raster from a NASA PDS Table - A long, multistep example showing many powerful features of Manifold in various steps. We import terrain elevation raster data from a NASA PDS archive and style the images with an elevation palette and hillshading. We notice one of the images was wrongly georegistered by NASA, so we import the original LiDAR point data from a table in the PDS archive. We transform the imported table from 0 to 360 degree longitudes into +/-180 longitudes, construct a geometry field, and then we create a drawing from the table that shows the LiDAR point data in the table. Next, we use Kriging to create a terrain elevation raster from the LiDAR point drawing, and then we style that and hillshade it so it can be exactly overlaid on a Bing satellite layer to enhance the satellite photography with enhanced terrain relief.
SQL Example: List Transformation Grids - In this example we use a few snippets of easy SQL to list NTv2 and NADCON transformation grids that are available within the grids.dat compressed collection of transformation grid files. Grid transformation files can be used when creating custom base coordinate systems, for NADCON / HARN / HPGN and NTv2 high accuracy transformations that use grid files.
The Changes and Additions list is continued in the Changes and Additions Volume 1 archival topic.