This topic provides the first volume of archived Changes and Additions. See the Changes and Additions topic for notes on more recent builds.
Yet another large build with over 1000 new EPSG systems and transforms, and support for very high precision coordinate system transformations using NADCON and NTv2, with over 170 national and regional transformation grids. New viewshed transform templates provide point-and-click access to viewshed functions.
Viewer and Manifold downloads have been reduced in size by approximately 20 MB, providing faster downloads and faster launches. Viewer is now only 47 MB and 64 bit Manifold only 60 MB.
Built-in coordinate system data now include EPSG data up to EPSG 9.8.2. Compared to the previous EPSG 8.9 data, EPSG 9.8.2 adds about 1000 systems and transforms, deprecates a number of previous systems, and includes other updates.
New coordinate system type: Equal Earth. Built-in coordinate system data include EPSG systems based on the new Equal Earth projection.
New coordinate transform type: NADCON. Built-in coordinate system data include EPSG transforms based on NADCON. NADCON transforms use grid files, typically named XXX.LAS and XXX.LOS. Using a NADCON transform searches for the required grid files in the Manifold folders (~\SHARED, etc). If one or both of the required files is not found or otherwise cannot be loaded (due to no permissions to read, corrupted data, or similar problem), the transform logs the name of the file that failed to load and returns NULL values for the coordinates.
New coordinate transform type: NTv2. Built-in coordinate system data include all EPSG transforms based on NTv2. NTv2 transforms use a single grid file, usually named XXX.GSB. If the file fails to load, the transform logs the name of the file and returns NULL values for the coordinates.
The Base Coordinate System dialog allows setting the default transform from the edited base lat/lon system to WGS84 to NADCON or NTv2, and specifying the names of grid files to use. Browsing for a grid file allows picking a file outside of the Manifold folder; however, the transform will only use the name of the file and will always try to load the file only from Manifold folders, to avoid tying the definition of the coordinate system to the machine.
Parsing coordinate systems supports WKT2. Printing coordinate systems supports WKT2 through the new CoordSystemWkt2 query function.
Coordinate system conversions support prime meridians. The Coordinate Base dialog shows prime meridians for base lat/lon systems and allows editing them for custom base lat/lon systems.
Loading built-in coordinate system data looks for deprecated transforms and never uses them as default transforms to WGS84. Deprecated transforms come from EPSG. Transforms are normally deprecated either because previously submitted data for a transform was incorrect, or because there is a newer transform which is better than the old one. In the latter case the deprecated transform is usually OK; however, because many transforms are deprecated due to errors, the system never uses a deprecated transform as the default.
Converting coordinates between coordinate systems better optimizes scales and shifts, reducing errors.
Coordinate transforms that use grid files can load grid files from a compressed GRIDS.DAT file. The GRIDS.DAT file must be at the top of the ~\SHARED folder, (that is, it must be at ...\SHARED\GRIDS.DAT, similar to how ICUDTL.DAT must be at ...\SHARED\ICUDTL.DAT. The file contains 170+ grids referenced by EPSG, including traditional NADCON, HARN NADCON, and NTv2 grids for Australia, Austria, Brazil, Canada, France, Germany, Ireland, Japan, Netherlands, New Zealand, Portugal, Spain, Switzerland and the UK. The GRIDS.DAT file may be downloaded from the Product Downloads page on the Manifold web site.
Grid files loaded by coordinate transforms are cached. The cache is limited to 16 MB in 32-bit Manifold and unlimited in 64-bit Manifold.
Changing the coordinate system of a component displays available transform paths and allows picking a specific transform to use. For each transform path, the dialog displays the name of the transform, the names of referenced grid files if the transform uses grid files, and the reported accuracy if it is available. Deprecated transforms are shown with distinctive icons. (Transform paths are available when converting between EPSG systems. Several conversions with multiple transform path, any direction: EPSG:4312 <-> EPSG:4258, EPSG:4204 <-> EPSG:4326, EPSG:4267 <-> EPSG:4269.)
Built-in coordinate system data has been optimized to load faster.
Legacy coordinate system data used in Manifold 8 superceded by GRIDS.DAT has been removed, reducing the size of installation packages noticeably.
New query functions:
CoordSystemMatchEpsg - takes a coordinate system definition and returns the corresponding EPSG code. The EPSG code is taken from the name. The definition of the referenced EPSG system is then checked against coordinate system parameters without local scales and offsets.
CoordSystemMatchEpsgBase - takes a coordinate system definition and returns the EPSG code corresponding to its base lat/lon coordinate system. The EPSG code is taken from the name of the base. The definition of the referenced EPSG system is then checked against datum parameters.
CoordTransformChoices - takes a pair of coordinate systems and returns a table of transforms that can be used to convert coordinates between them.
CoordConverterMakePath - takes a pair of coordinate system and a conversion path hint, and returns the coordinate converter object that can be used to convert coordinates between the systems. The conversion path hint can be either 'EPSG:xxx' with the EPSG code of the transform to use in the path, or 'GRID:xxx' with the name of the grid file of the transform to use in the path. For a list of transforms that can be used to convert coordinates between the specified systems, use CoordTransformChoices.
New transform templates:
Viewshed Area, Visible from All - takes an image, a drawing with observers, and visibility parameters, and computes an area visible from all observers.
Viewshed Area, Visible from Any - as above, but computes an area visible from any observer.
Viewshed Image, Visibility of All - takes an image, a drawing with observers, and visibility parameters, and computes an image with a visibility index from all observers. The visibility index is a positive value if a pixel is visible, and a negative value if it is not visible. The magnitude of the value conveys how visible or invisible the pixel is.
Viewshed Image, Visibility of Any - as above, but computes an image with a visibility index from any observer.
Viewshed Image, Visible Count - as above, but computes an image with the number of visible observers for each pixel.
Connecting to SQL Server supports the old SQL Server driver named SQLOLEDB. This allows working with SQL Server on systems which do not have the newer driver from the SQL Server native client: SQLNCLI10 / SQLNCLI11.
Connecting to SQL Server supports the newest SQL Server driver, named MSOLEDBSQL. This driver appeared fairly recently as a result of Microsoft un-deprecating OLE DB, and is recommended for all modern uses of SQL Server. The current latest version of the driver is available from: https://www.microsoft.com/en-us/download/details.aspx?id=56730
(Fix) Connecting to SQL Server via ADO .NET no longer sometimes fails due to a connection string version unsupported by the driver.
The Edit Schema dialog only allows creating indexes on computed fields in MAP files, not on other data sources.
The main menu and context menus allow using access keys (Alt-F-O for File - Open, etc). The names of all commands have been altered to specify access keys. If a menu includes multiple commands with the same access key, pressing it cycles between these commands without invoking them. Access keys for commands with plain key shortcuts without any modifiers (for example, M = Move Coordinates) coincide with those shortcuts.
The main menu and context menus honor the global system setting for showing access keys: only show access keys if the menu was activated using the keyboard and hide access keys if the menu was activated using the cursor (the default) or always show access keys.
Metes and bounds data in the Record pane are processed with higher tolerance for angles, to avoid turning tangent curves to non-tangent prematurely.
(Fix) Reading GDB data no longer sometimes produces NULL geoms. Exporting GDB no longer sometimes fails.
(Fix) Importing GDB with a table that contains multi-patch geometry no longer fails to import the table and stops the import.
Another large build with many new features for editing. Major feature additions include metes and bounds and viewsheds. Links that appear in the list below will jump to topics that have been updated. Stay tuned for additional updated topics and new videos! These notes will now include bug fixes as well as new features.
A significant user interface
change: Pressing Escape in a map window no longer clears the picked
/ edited record and no longer clears the cursor mode. Instead, use Ctrl-Backspace
or right-click into the map and choose Undo Changes. Using Escape
made it too easy to lose changes to geometry / values.
Yay! GPKG is back, after a one-build vacation. (Fix) Exporting data to GPKG no longer sometimes fails, due to changes in the previous build.
(Fix) The tracker tool no longer ignores curves, and instead linearizes them with a fixed number of intermediate coordinates per curve part.
The context menu for the tracker tool allows copying the last clicked location. The Copy Location command uses lat/lon in the datum of the current layer. The Copy Location (Projected) command uses the coordinate system of the current layer without local scales and local offsets, the same as the 'Projected' coordinate readout in the status bar.
Snap in a map window is turned off by default.
New edit commands for lines and areas in a map window:
Add Mid Coordinate - Appears in the context menu when a segment in an object being edited is right-clicked. Adds a new coordinate in the middle of the right-clicked segment,
Add Spline Coordinate - Appears in the context menu when the active (has been previously clicked) control coordinate for a spline in an object being edited is right-clicked. Adds a new spline coordinate in the spline part following the right-clicked coordinate.
(Fix) Switching the edit mode in a map window from Insert Coordinates to Move Coordinates or Move Coordinates + Split no longer stays in Insert Coordinates if the focus coordinate has been moved away from the insert coordinate in the coordinate list in the Record pane.
(Fix) Deleting the start or the end of a closed area branch no longer does nothing.
Deletion logic has been optimized when editing geometry in a map window to delete coordinates:
Deleting coordinates will automatically delete ellipse arcs that became incomplete.
Deleting coordinates for a line or an area deletes finished branches that became incomplete.
Deleting coordinates no longer always deletes the insert coordinate, switching away from Insert Coordinates mode. Instead, the location of the insert coordinate is checked after the deletion and if it remains valid, the insert coordinate is kept.
Deleting coordinates no longer always resets the focus coordinate to coordinate zero. If the focus coordinate remains valid after the deletion, the focus coordinate is kept. Otherwise, the focus coordinate is set to the closest (in list order) remaining coordinate in the same branch as the deleted focus coordinate.
Deleting coordinates no longer always closes the last branch. If the last branch is opened, it is kept opened.
New edit command for points, lines and areas in a map window: Delete Coordinate. Appears in the context menu for a coordinate or spline coordinate, and deletes the clicked coordinate. If the clicked coordinate is selected, the command deletes all selected coordinates.
(Fix) Deleting all coordinates of an edited point, line or area in a map window no longer prevents new coordinates from being inserted. Deleting everything and creating new geometry from scratch now works.
In a map window, using Shift-click to attempt ending a branch of an edited area using Shift-click does nothing if the branch contains less than 3 coordinates. Previously, the branch was ended with coordinates padded to 3 coordinates plus a closing coordinate.
In a map window, using the End Current Branch command to attempt ending a branch of an edited line or area pads the branch to 2 coordinates for a line and 3 coordinates plus a closing coordinate for an area.
New edit command for points, lines and areas in a map window: Delete Last Coordinate. Inserting coordinates into the last branch of an edited point, line or area in a map window allows using the Delete Last Coordinate command to delete the last inserted coordinate. The command can be used repeatedly until the branch is deleted entirely. Keyboard shortcut: Backspace.
Switching to Insert Coordinates mode when editing a multipoint in a map window always moves the insert coordinate to the end of the coordinate list. Multipoint branches are preserved, but inserts into the middle of a multipoint branch are not allowed. A better idea: avoid using multipoints.
(Fix) Moving the Record pane to a record with no geometry (a NULL) no longer paints a bogus bounds rectangle near the left top corner of the window.
New edit command for points, lines and areas in a map window: Delete Branch (appears in the context menu for a coordinate, curve coordinate or segment) deletes the clicked branch.
New edit command for lines in a map window: Reverse Branch Direction. Appears in the context menu for a coordinate, curve coordinate or segment, and reverses the direction of the clicked branch. Curve arcs are reversed as well.
New edit command for lines in a map window: Continue Branch. Appears in the context menu for a coordinate (right-click the coordinate) that starts or ends a branch. Starts inserting coordinates into the clicked branch from the clicked end. The branch is moved to the end of the coordinate list, and is reversed as necessary.
New edit command for the tracker tool in a map window: Clear Tracker. Clears tracker coordinates. Keyboard shortcut: Ctrl-Backspace, same as Undo Changes for a record.
The Record pane allows switching the coordinate list for a line or area to show metes and bounds instead of XY / XYZ coordinate values. The metes and bounds are shown in the format used by ESRI traverse files.
The Record pane allows saving the coordinate list to a text file. If the coordinate list shows XY / XYZ coordinates, the file uses a simple format specific to Manifold that closely matches what is displayed in the list and preserves all data, If the coordinate list shows metes and bounds, the file uses ESRI traverse files format and preserves XY values (not Z) and circle arcs (not ellipse or spline arcs). Short descriptions of both formats follow.
The Record pane allows loading the coordinate list from a text file. Both the Manifold format and the ESRI format are supported, with the format being detected automatically. No need to switch to coordinates or to metes and bounds prior to loading data. Future builds will provide user interfaces for easy metes and bounds editing.
A quick description of the Manifold format for storing coordinates lists:
The first line contains P for point, L for line, A for area.
The following lines contain coordinates:
C <x> <y> [<z>] - coordinate.
CC <x> <y> [<z>] - circle arc coordinate.
CE <x> <y> [<z>] - ellipse arc coordinate.
CS <x> <y> [<z>] - spline coordinate.
E <x> <y> [<z>] - coordinate that ends a branch.
A quick description of the ESRI format:
The first two lines specifies the direction type format: quadrant bearing, north azimuth, or south azimuth. Polar format is not currently supported.
The second line specifies direction units: decimal degrees, degrees-minutes-seconds, radians, or gradians.
The following lines contain coordinates:
SP <x> <y> - starting coordinate.
EP <x> <y> - ending coordinate.
DD <direction> <distance> - coordinate in specified direction.
AD <angle> <distance> - coordinate in specified direction relative to the current direction.
TC <circle arc parameters> - circle arc tangent to the current direction. The arc parameters are any pair from: central angle / arc length / chord length / radius plus whether the arc turns left or right.
NC <circle arc parameters> - circle arc not necessarily tangent to the current direction. The arc parameters are those for the tangent arc plus one extra direction, from: tangent / radial / chord.
See also ESRI documentation for traverse file format.
For both formats:
Case is not important. Whitespace, apart from line ends, is not important. Line ends can be either CRLF, or LF or CR. Text encoding is detected automatically: supported encodings are: ANSI, UTF8, UTF16 BE, UTF16 LE. The file can contain empty lines, which are ignored. The file can contain comment lines with comments starting with #. Comment lines are also ignored.
The above applies during loading. When Manifold writes coordinates, the system uses single spaces, uppercase letters, no comments, etc. This follows the 'be conservative in what you produce and liberal in what you accept' robustness principle.
Loading a coordinate list validates all data after loading and removes invalid curves or branches. Loading a coordinate list from an ESRI format file remembers the format used for direction type and direction units as well as the format used for each coordinate, and keeps them during editing. Future builds will allow changing the format for direction type and direction units using the user interface. Future builds will also provide a command that allows closing a branch by distributing a closing error between branch coordinates.
The coordinate limit in the GeomLinearize query function applies per curve part instead of per curve, so that spline curves with many control points produce more intermediate coordinates.
The View - Panes - Log Window command now has an icon.
(Fix) DELETE / UPDATE / SELECT INTO / GROUP / SPLIT query constructs no longer sometimes stop the query if the source table contains no records.
(Fix) The status bar no longer sometimes fails to turn on the busy indicator during long operations that run in background, for example, creating a temporary spatial index for a drawing or an image.
Viewsheds can now be computed using new query functions. See expanded comments in the SQL Functions topic.
TileViewshedMake takes an image and global viewshed creation parameters, and returns a viewshed buffer. Global viewshed creation parameters are: a refraction coefficient (0 allowed), and an option to use datum curvature. TileViewshedMakePar is a parallel variant.
TileViewshedAreaAll takes a viewshed buffer, a drawing with XYZ points, viewshed creation parameters, and returns an area visible from all points in the drawing. Viewshed creation parameters are: an option of whether Z values for points in the drawing are relative to heights in the image (true) or are absolute heights (false), the maximum visible radius (a negative value or zero means there is no limit), and minimum and maximum angles of the camera in degrees (-90 / 90 means that there are no limits). TileViewshedAreaAllPar is a parallel variant.
TileViewshedAreaAny takes the same parameters as TileViewshedAreaAll and returns an area visible from any point in the drawing. TileViewshedAreaAnyPar is a parallel variant.
TileViewshedTilesCount takes a viewshed buffer, a drawing with XYZ points, viewshed creation parameters, and returns a table with tiles with pixel values containing the number of points visible from the pixel. TileViewshedTilesCountPar is a parallel variant.
TileViewshedTilesLevelAll takes a viewshed buffer, a drawing with XYZ points, viewshed creation parameters, and returns a table with tiles with pixel values containing the visibility level of the pixel from all points. If the visibility level of the pixel is positive or zero, the pixel is visible from all points. If the visibility level of the pixel is negative, it contains the difference between the height at which all points become visible and the height of the pixel. The bigger the magnitude of the value, the more the pixel has to be raised to become visible from all points. TileViewshedTilesLevelAllPar is a parallel variant.
TileViewshedTilesLevelAny takes the same parameters as TileViewshedTilesLevelAll and returns a table with tiles with pixel values containing the visibility level of the pixel from any point. TileViewshedTilesLevelAnyPar is a parallel variant.
TileViewshedTraceLine takes a viewshed buffer, begin and end XYZ coordinates, and viewshed creation parameters, and returns the visibility level of the end coordinate from the begin coordinate.
Future builds will provide transform templates for viewshed functions.
SHP and other dataports read coordinate system data from PAM XML files (.AUX.XML). PAM XML files may encode coordinate system data in many different ways. The system supports all ways encountered to date by Manifold, but this is a moving target and new ways will likely be encountered. If a PAM XML file is encountered with coordinate system data not loaded by Manifold, please send in a suggestion to support it.
Reading data from PDS reads inventory data.
Reading data from PDS supports floating-point values with non-Intel byte order.
(Fix) the IMG ERDAS dataport no longer reads Albers conical equal area projection as Lambert conformal conic.
(Fix) Reading a coordinate system from WKT data no longer incorrectly reads several variants of Hotine oblique Mercator systems.
(Fix) Exporting data to MDB no longer fails if the file path contains spaces.
(Fix) Reading data from XLS no longer fails to locate an MFD_META$ sheet.
A large build with many new features for editing. Links in the list below jump to topics that have been updated. For example, see the updated Measurement topic for illustrations of using new editing capabilities. Stay tuned for additional updated topics and new videos!
A major user interface
change: As before, an Alt-click
onto an object in the active layer in a map picks that object as
the context object in the Record pane Values tab, and has no effect on
objects in layers other than the active layer A Shift-Alt-click
onto any visible object in a map, in any layer that allows picks,
picks that object as the context object in the Record pane and makes the
layer it is in the active layer. Clicking onto any segment or coordinate
handle of a picked object switches to the Coordinate tab.
Oops: Build 9.0.169.8 breaks GPKG export (will be fixed in the next build). Launch 9.0.169.7 to export to GPKG.
Map windows disallow editing, picking and selecting records in hidden layers (layers that are turned off for display). That makes sense, since we cannot see the objects in hidden layer.
Map windows display the focused pixel in a picked tile. Clicking a pixel in a picked tile moves focus to that pixel and scrolls it into view in the list of pixels in the Record pane.
The Layers pane in the dropdown menu on the filter button includes an option to display field type (instead of the default, field width) for table fields.
The Layers pane in the dropdown menu on the filter button includes an option to display pick mode (instead of the default, opacity) for map layers. The default is to allow picks.
Alt-click in a map window tries to pick a record from the active layer. Shift-Alt-click tries to pick a record from all layers that allow picks, in top down order.
Alt-click in a map window has been reworked to never clear the cursor mode. Using Alt-click in default cursor mode picks records. Using Alt-click in other cursor modes (for example, measurement mode) works like a click would, with the Alt modifier being ignored.
Alt-clicking a record that is already picked in a map window no longer re-picks the record, abandoning all changes made to it. Alt-clicking a stack of records keeps the picked record if it is in the stack.
Ctrl-click and Ctrl-drag in a map window have been reworked to never clear the cursor mode or the edited record. Using Ctrl-click and Ctrl-drag in default cursor mode with no edited record selects records.
In edit mode, using Ctrl-click or Ctrl-drag with an object with editable geometry selects coordinates.
Clicking a coordinate, segment, or pixel of a picked record in a map window switches the Record pane to the Coordinates / Pixels tab if it is not already in that tab. Clicking some distance away from all coordinates or pixels switches the Record pane to the Values tab. This allows starting or stopping the process of editing coordinates of a picked record: Alt-click to pick the record, click a coordinate handle or segment between handles to make coordinate handles writable, do some editing, click "far" away from the record to make coordinate handles readonly again, etc. This applies when editing existing objects. When creating new objects, switch between the Values tab and Coordinates tab of the Record pane by clicking the tabs.
Clicking a segment of a picked record in a map window sets the focus coordinate to the beginning of the segment.
The context menu for an edited record in a map window no longer includes commands to commit or cancel changes if the record is read-only.
Editing geometry in a map window allows switching the edit mode between Insert Coordinates (keyboard shortcut: I) and Move Coordinates (keyboard shortcut: M) using the context menu. The Insert key formerly used to create / move / delete the insert point in the map window or in the list of coordinates in the Record pane, which behaved somewhat similarly to the new edit modes, has been deprecated and no longer has any effect.
New edit mode in a map window: Move Coordinates + Split (keyboard shortcut: P). While in this mode, clicking a coordinate handle sets focus to that coordinate, clicking a segment inserts a new coordinate at the clicked location and sets focus to that coordinate, and dragging a coordinate handle moves that coordinate plus all other selected coordinates if the coordinate is selected, or all coordinates if the Shift key is on (similar to Move Coordinates).
Editing line or area geometry in a map window adds new commands to the context menu that work with branches:
Begin New Branch,
Continue Last Branch (shown if there is an unfinished branch),
End Current Branch (shown if there is an unfinished branch that is currently being added to).
Clicking a segment of an edited line or area in a map window allows changing the segment type using new commands in the context menu:
Convert to Line,
Convert to Circle,
Convert to Ellipse,
Convert to Spline.
To convert a curved segment back to a linear segment, right-click a coordinate handle for one of the curved segment's control points.
The Layers pane in the dropdown menu on the filter button includes an option to display snap mode (instead of the default, opacity) for map layers. The default is to allow snaps.
When editing geometry in a map window a new context menu (right-click when editing) allows switching between snapping to the active layer and snapping to all eligible layers (the default is to snap to the active layer). As before, toggling snap on and off using the spacebar works. The new context menu (right-click when editing) also allows setting snap mode.
See the new Snap Modes topic.
Editing geometry in a map window allows snapping while dragging coordinates. Keyboard shortcuts for snap modes work during dragging as well.
New snap mode in a map window: Snap to Grid (keyboard shortcut: G). There is a new Snap Parameters command, found in the main menu, in the toolbar selector for the cursor mode, and in the context menu for an edited record, that allows specifying grid size and grid unit. The default grid step is 10x10 units of the coordinate system, so a coordinate system that uses meters will have a default grid step of 10 meters. Snap parameters can be edited outside of an editing session. The system does not save snap parameters to a component's properties, but likely will in future builds.
New snap mode in a map window: Snap to Bearing (keyboard shortcut: B). The Snap Parameters command specifies the number of bearing divisions. The default is 4 which means that there are 4 fixed directions: 0, 90, 180, 270 degrees. Snap to Bearing mode is frequently called "an ortho mode" where only horizontals or verticals are allowed. Specify 8 bearing divisions to get snaps to 45 degrees, and specify more bearing divisions for divisions into smaller angles.
Editing geometry of an existing record in a map window no longer snaps to the old version of the geometry. Instead, editing snaps to the edited coordinates. Snapping to coordinates of an edited object uses a smaller snap distance than for other objects (50 pixels instead of 150) and is disabled if the edited object is a point. This is done to help avoid creating walkbacks, etc.
New editing and snapping capabilities work for the Tracker tool as well as when creating or editing lines.
Selecting records using Ctrl-drag in a map window no longer slightly extends the selection rectangle to make a tiny selection rectangle work as a Ctrl-click. Instead, Ctrl-click continues to select data within some distance of the click, but Ctrl-drag is allowed to be as thin as possible. The window only starts tracking the rectangle if the cursor is dragged far enough from the initial click point.
The Table.SearchBatchRTreeThin method in the object model accepts a new boolean parameter to return data either touching or completely within search rectangle.
There is a new dataport for JSON data that is not GeoJSON. Top-level structures in JSON data that look like lists of records are exposed as tables. Other top-level structures are exposed as comments. *.JSON files are remapped from GeoJSON to JSON.
There is a new dataport for web JSON data.
Web CSV / JSON / GeoJSON dataports allow refreshing tables to re-read them from the web.
The web CSV dataport uses a more sophisticated content type check (a blacklist instead of a whitelist) to allow custom content types.
The Oracle dataport optimizes bulk reads. Reading big tables in 9 should be at least on par with 8.
The GDB dataport supports storing MFD_META properties inside the geodatabase. This allows overriding coordinate systems, setting up styles, etc.
The GDB dataport allows creating, and also renaming and deleting, components of all types, storing component definitions inside the geodatabase.
The GDAL dataport adds support for GDAL 2.3, 2.4, and 3.0, in addition to GDAL versions 2.0.x, 2.1.x, and 2.2.x previously supported.
The MySQL dataport no longer creates virtual drawings for tables with geometry fields.
The GPKG dataport provides its own versions of spatial functions required to enable writes to spatial indexes when SpatiaLite or similar extensions that normally provide these functions cannot be loaded. This allows operating GPKG databases in full read-write mode with just the SQLITE3.DLL.
The LAS / LAZ dataport optimizes spatial queries to perform significantly faster (4x or so).
The LAS / LAZ dataport uses significantly less memory when creating spatial indexes.
Dataports for databases allow tables to have virtual computed fields. This allows having, say, a field with virtual geoms composed from X and Y values stored in the database. Computed fields on databases cannot refer to other computed fields and cannot participate in indexes.
The web image dataport uses new URLs for Yandex maps.
There is a new dataport for PDS files, currently supporting tables and images. *.XML files are mapped to PDS. *.XML files were not mapped to any particular dataport before this build, even though many formats use them. There is no better extension to map to PDS, however, so the system first maps *.XML to PDS and the remaps to other dataports later as part of automatic detection where it would make sense to do so.
The API doc has been updated to include the latest changes to the object model. Topics for several existing calls have been extended. There is a new VBScript example for ValueSet.CreateLocked, provided for completeness since the performance gain from using ValueSet.CreateLocked for VBScript is minimal.
Highlights include new query functions to access individual pixels within tiles, improvements to linked LAS/LAZ indexes that implement optimized spatial searches standard within .map projects, a key improvement to folders with layers, and improvements in the object model.
Folders in the Layers pane are toggled independently from layers. A layer is considered to be visible when it is turned on and all folders it is in are also turned on. See the updated Layers pane topic.
The drop-down menu for favorite coordinate systems displays coordinate systems which override metrics with a trailing '#' symbol. See the updated Favorite Coordinate Systems topic.
Shift-clicking (instead of Ctrl-Shift-clicking) a field header in a table window no longer sorts records. Supported clicks after the change are Ctrl-click = sort the table on the clicked field, replacing previous sort order; Shift-Ctrl-click = add the clicked field as a sub sort to any existing sort order.
Invoking View - Zoom to Native in a map window for an image adjusts the zoom center to avoid smoothing image pixels. Resizing a window might require a re-adjustment by a repeated Zoom to Native command..
Invoking View - Zoom to Native in a map window for a drawing, labels or map adjusts the zoom center to make entered coordinates whole numbers (the adjustment is opposite to the one needed to avoid smoothing pixels for images). Resizing a window might need a re-adjustment. See the View - Zoom... topic for an example.
New query functions:
TileGeomToValues - takes an image and a geom, returns a table with all pixels under geom. Fields: X, Y, Value.
TileGeomToValuesX2 / X3 / X4 - variants of TileGeomToValues for images with multiple channels, which return pixel values as an x2, x3, or x4.
See the SQL Example: Acquire Pixel Values within Areas topic.
The LAS dataport optimizes spatial searches without thinning with big rects and returns records progressively, instead of first collecting all records to return.
Snapping to coordinates in a map window performs faster on data sources that support data thinning, namely MAP and LAS.
The object model supports expression contexts:
Schema.Constraint has a new property: ExpressionContext (string, read / write).
Schema.Field has a new property: ExpressionContext (string, read / write).
Schema.AddConstraint has a new optional parameter for expression context (string).
Schema.AddFieldComputed has a new optional parameter for expression context (string).
The object model supports searching for records via an r-tree index with thinning. Table has a new method: SearchBatchRTreeThin which takes the name of an r-tree index, search keys (ValueSet with a rect for the indexed geometry field), array of fields to output, X and Y size of a grid to use for thinning (eg, window size), and a boolean flag allowing (true) or disallowing (false) to reduce the number of coordinates in returned geometry values. Thinning is currently implemented by r-tree indexes in MAP and LAS files. If an index does not support thinning, SearchBatchRTreeThin works as SearchBatchRTree. Changes to the API doc reflecting the additions are coming.
Cutting edge builds expire on the first day of the third month following the build month. For example, a cutting edge build issued mid-August will continue working during August, September and October, and will expire on the 1st of November. One month or less before the expiration date, starting a cutting edge build will open the log window with a warning that the build will soon expire. After the expiration date, attempting to start a cutting edge build will fail with a message the build has expired and advising the download of a newer build.
The ODBC driver now uses a 4-component version number instead of a 3-component version. This removes an old limitation from a time before Manifold fully supported ODBC 3.x completely. The ODBC driver now is assigned the exact same version number as the product.
This build continues enhancing the folder system for layers, adding folders to map layer tabs. It also implements significant internal improvements for performance utilizing memory, and introduces significant changes in various user interfaces for better ease of use and efficiency. The Layers pane is now a standalone pane, like the Project and Contents panes. Watershed query functions have been reworked to a more flexible create / use pattern and watershed performance has increased.
A major user interface
change: Windows and panes are now undocked and docked using a Shift-click instead of an Alt-click.
The Layers pane shows on/off toggle icons for folders which contain layers, allowing one click on/off of layers within a folder. If a folder contains both hidden and shown layers, this is shown using a 'mixed state' icon. See the updated Layers Pane topic.
The New Map dialog layers list toggles layers on/off using Space or click instead of Enter or double-click, to match use in the Layers pane and similar.
The Merge Drawings, Merge Images, and Merge Labels dialog layers lists toggle layers on/off using Space or click instead of Enter or double-click, to match use in the Layers pane and similar.
Double-clicking a folder in the list of layers in the Layers pane edits its name. The edit box is positioned to start exactly where the folder name starts, unless the list is too narrow for that.
Map windows show tabs for top-level components or folders. Top-level folders that have no components are skipped. Tabs for folders choose one of their layers to be the current layer, for visual operations such as for adding new objects or for selecting objects using Ctrl-drag, and for panes such as the Style pane. If a folder for a tab contains more than one component, the layer context menu for the tab includes a Switch to submenu that allows picking a different layer in the folder to be the current layer.
Tabs for folders are considered to be visible if any of the layers corresponding to the tab are visible. If any of the tab layers are being cached or painted, the tab displays a chevron of the appropriate color. If any of the tab layers have messages, the tab displays an icon.
Dropping layers from the Project pane into a map window drops them into the top level position just before the active tab.
Deleting from the map a layer tab for a folder by using the context menu Delete from Map choice deletes from the map all layers and folders within the folder. An exception is a case of map windows that are multi-layered drawing, image or labels windows, where the base component cannot be deleted. For example, if we open a drawing and add layers to that drawing window, we cannot delete the original drawing layer from that drawing window.
Zooming to a layer tab for a folder using the context menu Zoom command will zoom to the extent of all layers corresponding to the tab.
Toggling a layer tab for a folder using the context menu or a double-click shows or hides all layers within the folder.
The Layers pane allows filtering the layers list using a filter button, similar to the filter button in the Project pane. Specified filters are kept for the duration of the session. This allows filtering long lists of layers to only drawings, or only images, or only labels.
The Layers pane has been moved out of the Contents pane. This allows having both the Layers pane and one of the panes in the Contents pane visible at the same time, in addition to the Project pane.
Keyboard shortcuts for individual panes in the Contents pane have been remapped: Ctrl-1 = Component (as before), Ctrl-2 = Record, Ctrl-3 = Select, Ctrl-4 = Style, Ctrl-5 = Transform.
The Create Label tool used in layout windows has been renamed to Create Text.
The context menu for a layer tab in a map window includes a new Show in Layers command, which opens the Layers pane and moves the layers list cursor to that tab's layer. If the layer is in a folder that is collapsed, the command moves the cursor onto the folder.
Alt-clicking a layer tab in a map window shows the current tab layer in the Layers pane, switching to the Layers pane if need be.
Floating (undocking) and re-docking a window tab is now done via Shift-clicking instead of Alt-clicking. This frees Alt-clicks for other things.
Favorite coordinate systems allow specifying whether applying the system over an old system should keep or override coordinate system metrics. The default is to keep metrics. This allows creating Favorites which either keep or override metrics, as desired.
Query functions for watersheds have been reworked into a two-stage create / use pattern. First, we must create a watershed object using one of the new functions. The created watershed object can then be passed to other functions which compute geometry. Specifically:
TileWatershedMake and TileWatershedMakePar - take an image with heights and create a watershed object. TileWatershedMakePar is a parallel variant.
TileWatershedMakeDir and TileWatershedMakeDirPar - take an image with directions and create a watershed object. TileWatershedMakeDirPar is a parallel variant.
TileWatershed... - all other watershed functions have been reworked to take the watershed object instead of the image.
The change allows using pre-computed directions for computing all watershed geometry without creating a duplicate of each function. It also allows re-using the same watershed object when computing geometry in different variants.
TileWatershedMakeXxx query functions allow creating watersheds with weights (precipitation) for each pixel. To use weights, pass a two-channel image (with either heights or directions in the first channel and weights in the second channel) and pass TRUE for the useWeights parameter.
Computing watershed geometry performs much faster due to multiple optimizations, especially on big images that do not fit into memory cache, and even more so on big images that do not fit into physical memory.
The TileWatershedLinesDownstream and TileWatershedLinesDownstreamPar query functions no longer return total flow for computed lines. This is no longer computed due to changes in algorithms.
Template parameters for Fill Sinks have been renamed: Min delta has been changed to Fill height, and Min flow has been changed to Fill flow.
The TileMakeNew query function has been renamed to TileMake, for consistency with all other XxxMake functions. The only function now using the naming style of XxxMakeNew is UuidMakeNew, which uses New because it generates a new UUID.
The memory cache performs much better under heavy load. This improvement includes multiple optimizations which apply when the amount of RAM available to the memory cache is not big enough to hold all data to be analyzed and some data has to be spilled to disk.
Deleting big tables within MAP files performs much better when the memory cache is overflown. Discarding temporary tables used by analysis performs enormously faster when the memory cache is overflown. Big tables stored in MAP files will now delete 2-3x faster than before, and temporary tables created for a session now take basically no time to delete, at no cost to other operations.
The LAS / LAZ dataport uses less space for a spatial index and creates it faster. Spatial index files are named using the pattern xxx.LAS.MAPINDEXP or xxx.LAZ.MAPINDEXP.
This build provides additional watersheds functionality, improves the groups user interface by replacing leading layer groups with folders for groups, improves multilayer map window rendering logic, and introduces a specialized point index to allow fast operations with big LiDAR data.
Watershed Areas and Watershed Lines transform templates automatically create a unique index on stream ID and a non-unique index on target ID, for further analysis.
Computing directions on plateaus for watersheds performs significantly faster.
Tables returned by TileWatershedDownstreamLines, TileWatershedDownstreamLinesPar, TileWatershedAreasUpstream, TileWatershedAreasUpstreamPar, TileWatershedUpstreamLines, and TileWatershedUpstreamLinesPar query functions include the starting and ending coordinates used to extract downstream and upstream geometry.
TileWatershedAreasUpstream, TileWatershedAreasUpstreamPar, TileWatershedLinesDownstream, TileWatershedLinesDownstreamPar, TileWatershedLinesUpstream, TileWatershedLinesUpstreamPar query functions take an additional parameter for keeping or removing overlaps.
Reading a CSV file allows date fields to use multiple different languages, for example, one field can US English while an other field can use German.
Keyboard shortcuts treat number keys on the numeric keypad the same as number keys on the keyboard's number row.
Double-clicking an expand or toggle icon in the Layers pane no longer starts editing layer alpha or field size, and instead simply is treated as a click.
In the Style pane Options tab for images, the note on shading requiring a style based on a single channel has been moved from the shading check box to a separate caption below the shading controls.
Field headers and record handles in grids have been made slightly flatter, for a cleaner look.
In the display stack for windows, table fields, layout frames, and map layers that have invalid or missing Z order info are placed last in the display stack (at the bottom or rightmost in tables) rather than first (the top, or leftmost).
Layer groups in tables, layouts, maps have been reworked to use folders instead of leading layers. Group and Ungroup commands have been removed. The new New Folder command creates a new folder. The new folder is created in the same place in the folder hierarchy as the current layer. The new Move to Folder command allows moving selected layers to any unselected folders, with the target folder specified using a dropdown menu. Layers moved into the folder are placed after all existing layers in the target folder. Folder names can be edited. Groups with leading layers created by prior builds are automatically dissolved. Upcoming builds will allow drag-and-drop of layers between folders.
The Move to Folder dropdown menu in the Layers pane includes a Move to Root choice that allows moving selected layers out of all folders and into the root folder.
Moving layers into a folder in the Layers pane automatically expands the folder if it is collapsed.
Pressing Insert with the focus in the layer list in the Layers pane invokes the New Folder command.
The map window no longer allows multiple layers to reference the same component. When multiple layers reference the same component, all but the first one are rendered invalid and do nothing. Invalid layers will still show up in the Layers pane and in layer tabs, so they can be deleted.
The map window better distinguishes between changes to underlying data and no longer repaints after changes that cannot modify the screen: changes to data in hidden layers no longer force a repaint, changes in Z order of a mix of hidden and visible layers that do not change the Z order of visible layers no longer force a repaint, changes to alpha of hidden layers no longer force a repaint, etc.
The map window better tracks changes to metadata of layer components and automatically reloads layers, when necessary, after changes to referenced components (for example, to a table) or fields (for example, a geom).
Dropping new components into a map window creates new layers in the same folder as the active layer.
Moving layers using layer tabs in a map window changes the folder of the moved layer to that of the target layer. Upcoming builds will use folders more in the layer tab strip.
The LAS / LAZ dataport creates a specialized spatial index for points, which allows quick display of big point sets as drawings. See the updated LAS, LAZ LiDAR topic.
The specialized index is only created for a linked file in a writable location. Importing a file will create a standard RTREE index, which will perform slower than the specialized index used with linked LAS / LAZ files.
The specialized index creates a hierarchical data structure, stored as a .MAPINDEXP file, that is then used to service spatial queries.
During a spatial query performed for display, the data structure thins the points in the query area and returns a limited number of the best representatives, with the limit on the number of sample points displayed depending on the display window size.
The size of a .MAPINDEXP file will be significantly smaller than the size of an uncompressed LAS file, but usually will be bigger than the size of a compressed LAZ file.
The .MAPINDEXP file is versioned so that future builds can choose to reuse files created by this build or overwrite them with new versions.
Future builds will add the specialized index for point sets within the MAP file as well, as soon as it stabilizes.
The LAS / LAZ dataport reports the time to create the index (could be long, but this is only done once per file).
This build fills out watersheds functionality, provides bug fixes, and improves CSV and web CSV dataports.
New query functions:
TileFillSinks and TileFillSinksPar - takes a 1-channel image, a fill height value, and a minimum flow value, and returns an image with filled sinks. TileFillSinksPar is a parallel variant, taking thread configuration as an extra parameter.
If both fill height and minimum flow are negative or zero, the image is left unchanged.
If fill height is positive and minimum flow is negative or zero, sinks are filled based solely on height.
If fill height is negative or zero and minimum flow is positive, sinks are filled based solely on flow = area.
If both fill height and minimum flow are positive, sinks are filled based on both height and flow.
TileFlowDirAccum and TileFlowDirAccumPar - takes a 1-channel image, computes flow direction and accumulation, and returns the result as a dual image using a table with fields for X, Y, FlowDir, and FlowAccum. Flow direction is returned as an INT8 data type, where, 0 corresponds to 'up', and 1-2-3-4-5-6-7 proceed clockwise. Flow accumulation is returned as a FLOAT64 data type. TileFlowDirAccumPar is a parallel variant, taking thread configuration as an extra parameter.
TileWatershedLinesDownstream and TileWatershedLinesDownstreamPar - takes a 1-channel image and a drawing of points, and returns downstream watershed lines for each point. TileWatershedLinesDownstreamPar is a parallel variant, taking thread configuration as an extra parameter.
TileWatershedLineDownstream and TileWatershedLineDownstreamPar - takes a 1-channel image and a coordinate, and returns a downstream watershed line for the coordinate. TileWatershedLineUpstreamPar is a parallel variant, taking thread configuration as an extra parameter.
New transforms:
Watershed Prepare: Filling Sinks
Watershed Areas Upstream
Watershed Lines Downstream
Watershed Lines Upstream
TileWatershedAreasSinks and TileWatershedAreasSinksPar query functions take an additional boolean parameter that specifies whether to keep (true) or discard (false) areas that flow out of the image.
ECWP, web CSV, and similar web dataports which produce a single image, drawing or table compose component names from the URL object name. Extensions are cut so that data.php becomes data, and parameters are cut so that data?... becomes data, with other similar clean up of URLs to form names.
The CSV dataport uses all UI languages as listed in Help - About when recognizing date values. For now, the language determined to be the best match globally is used for dates in all fields. That will change in future builds to logic that allows different fields to use different languages.
This build provides numerous small improvements to the GUI for faster and easier editing, plus significant enhancements to Watersheds. Feedback on the prior build has been implemented in the Layers pane to support groups planned for the next build, which will feature folders instead of leading layers.
Clicking the toggle icon for a layer in the Layers pane toggles the layer (and all selected layers if the clicked layer is selected) on and off.
Updating image style from the Style pane accepts changes made both to image palette / channel map and to shading. Shading controls are enabled even if the image is rendered using a channel map with shading not applied.
Switching cursor mode in a layout window clears active layout frames.
Alt-click, Alt-drag, Ctrl-click, Ctrl-drag reset cursor mode in layout window. (Inserting a new layout frame and then Alt-clicking an existing frame will activate the existing frame instead of inserting one more new frame. Same for Ctrl-clicking and Ctrl-dragging which will start selecting frames.)
Alt-click, Alt-drag, Ctrl-click, Ctrl-drag reset cursor mode in a map window. (The behavior is the same as for layout frames. In addition, Alt-clicking a record in drawing / image / labels, then Alt-clicking a different record will put the new record into the Record pane. Same for Ctrl-clicking and Ctrl-dragging which will clear the Record pane and start selecting records.)
Dragging the first coordinate of an area branch in a map window will automatically move the matching last coordinate of the branch.
Dragging coordinates of geometry values in a map window only allows dragging the focused coordinate. To drag a coordinate other than focused, first click it to make it focused. This protects against accidental drags and makes drags more predictable.
Rendering coordinates of an edited geometry value in a map window always puts the focused coordinate on top of all other coordinates.
Rendering the outline of a geometry value in a map window uses a halo for the line / area outline, with a thick border and halo for the focused coordinate.
Rendering the outline of a geometry value in a map window paints a box around the whole value if it is too small at the current zoom, to make it more visible. The additional box is only painted for existing records when coordinates are not being edited, that is, when the Coordinates tab in the Record pane is not active.
Snap reticules in a map window are rendered with a thin halo, for better visibility.
The Record pane disables moving to the previous / next record if it is editing a new object.
Moving to the previous / next record in a layout or map window from the Record pane pans / zooms the window to the new record. If the record is already completely visible (although possibly small), the view is not altered. If the record is small enough to fit into the window at current scale, the window is panned to include the record. If the record is too big to fit into the window at current scale, the window is zoomed to the record.
Alt-clicking in a drawing / labels layer in a map window picks up to 10 records under the clicked location. The first picked record is put into the Record pane, other picked records can be moved to using Move to Next. That is, if there are multiple picked records, the virtual record order used by the Record pane is altered to put the picked records first and unpicked records next. This makes it much easier to edit objects that are covered by other objects in the same layer.
Moving to the previous / next record in a map window from the Record pane uses a background thread to (show the progress dialog and) allow canceling the move if it takes too long. (This is useful for slow data sources and for components made on queries.)
The Record pane allows moving only to selected records using a toolbar button. (Turning the mode on keeps the current record unchanged, but moving to the previous / next record will only move to selected records. If there are no selected records in the specified direction, the current record stays unchanged.)
The Record pane shows and allows editing pixel values in tiles. Pixel values are shown as X - Y - value (possibly multi-channel) - toggle that indicates whether the pixel is present or missing. Pixel values can be selected, values can be edited and toggled, editing or toggling a selected pixel value alters all selected pixel values. Changed pixel values are highlighted. (Saving changes to the edited tile will currently require updating intermediate levels manually in the map window - we will perhaps automatically update them if they were updated before the changes.)
The Record pane shows field names in the list of field values as readonly.
Composing contours for many heights (thousands+) performs much faster.
Composing traces for many values (thousands+) performs much faster. The limit for the number of unique values has been removed.
New query functions:
TileWatershedAreasSinks and TileWatershedAreasSinksPar - takes an image, and returns watershed areas for all sinks (no minimum flow parameter necessary). TileWatershedAreasSinksPar is the parallel variant, taking thread configuration as an extra parameter.
TileWatershedAreasUpstream and TileWatershedAreasUpstreamPar - takes an image and a drawing of points, and returns upstream watershed areas for each point. TileWatershedAreasUpstream is the parallel variant, taking thread configuration as an extra parameter.
TileWatershedAreaUpstream and TileWatershedAreaUpstreamPar - takes an image and a coordinate, and returns upstream watershed area for the coordinate. TileWatershedAreaUpstreamPar is the parallel variant, taking thread configuration as an extra parameter.
TileWatershedLinesUpstream and TileWatershedLinesUpstreamPar - takes an image, a drawing of points and the minimum flow value, and returns upstream watershed lines for each point. TileWatershedLinesUpstreamPar is the parallel variant, taking thread configuration as an extra parameter.
TileWatershedLineUpstream and TileWatershedLineUpstreamPar - takes an image, a coordinate and the minimum flow value, and returns upstream watershed line for the coordinate. TileWatershedLineUpstreamPar is the parallel variant, taking thread configuration as an extra parameter.
New transform: Watershed Area, Sinks - computes watershed areas for all sinks. See the Watersheds topic for examples, including use of upstream query functions.
Exporting a TIFF or ERDAS IMG skips writing data for intermediate levels if the image is too big (intermediate levels will spend too much space on disk) or too small (intermediate levels are unnecessary because they are easy to compute on the fly).
There is a new dataport for web CSV, similar to web GeoJSON, to allow easy linking of web URLs with CSV data.
The Options dialog allows specifying a value for the cache size. The default is (auto) and that can be overriden with an explicit value in gigabytes. The minimum allowed value is 4 GB, the default, and the maximum allowed value is 64 GB, however the dialog only lists values up to the amount of installed physical memory. The option only applies to 64-bit mode Manifold operation 32-bit Manifold ignores it. Any changes to the cache size are only applied after restarting the application. This setting allows increasing cache size in 64-bit mode, which can help performance of some operations. Increasing the cache size can also hurt performance depending on the scenario. Generally, it makes sense to keep the cache size no bigger than half of the installed physical memory, perhaps slightly lower than that when regularly running multiple instances of Manifold. Whenever in doubt about the effect on performance, use (auto), which is conservative and safe.
This build provides new, parallel, Watersheds capabilities that are more sophisticated and dramatically faster than prior Manifold generations, while also handling immensely larger data. Other highlights include new infrastructure for coordinate system queries and extensive new facilities for measurement, allowing a choice of Euclidean or geodetic measurement. Work with grouping in the Layers pane continues to more forward as well.
The Layers pane uses a single column for the layers list. Pressing Enter or double-clicking a layer starts editing it. Pressing Space toggles the active layer as well as all other selected layers if the active layer is selected.
Selecting layers in the Layers pane follows group hierarchy. Selecting or unselecting a layer with children selects or unselects all children. Unselecting a layer with parents unselects all parents. That preserves the invariant rule that if a layer is selected, all its children are selected as well.
Moving layers in the Layers pane follows group hierarchy. Moved layers never change their group level, layers never move between groups.
Grouping layers in the Layers pane preserves group selection. Actions that create groups with a selected parent and unselected children are denied.
The Layers pane includes a filter box that allows filtering layers using parts of their names. Applying a filter disables moving and grouping commands, since their effect could be invisible. Deleting, editing, and toggling selected layers can affect layers hidden by the filter, similar to how selection works in tables. Selecting or unselecting layers can affect layers hidden by the filter as well. For example, selecting a layer will select all its children, including those that are hidden.
The Layers pane allows expanding and collapsing layer groups. The Plus + key or the Right Arrow expand. The Minus - key or the Left Arrow collapse. Switching between windows preserves expanded or collapsed states of layers per-window. Moving or grouping commands automatically expand groups that contain selected layers, to make the effect of commands visible.
Clicking the + or - icon of a layer in the Layers pane expands or collapses the layer.
Command windows mark their log messages with (Command : CREATE ...) or (Command : [datasource]::[datasource] : CREATE ...) instead of with (ad-hoc) to help distinguish commands from each other.
Parsing coordinate system from WKT (PRJ) has been extended to recognize a number of new coordinate systems.
New query functions for coordinate systems:
ComponentCoordSystemScaleXY - takes a component object and returns an x2 value for local scales for X and Y.
CoordSystemScaleXY - takes a coordinate system definition and returns an x2 value for local scales for X and Y.
CoordSystemBase - takes a coordinate system definition and returns the base lat/lon coordinate system.
CoordSystemForceXY - takes a coordinate system definition and returns the same coordinate system with axes forced to XY order. If the original coordinate system definition is an EPSG or SRID code, the new coordinate system is the same code with overrides.
CoordSystemPlain - takes a coordinate system definition and returns the same coordinate system with local offsets set to 0 and local scales set to 1.
CoordSystemWkt - takes a coordinate system definition and a strict boolean flag and returns the WKT (PRJ) representation. If the coordinate system cannot be represented as WKT, the function returns a NULL value. If the strict flag is false, the function maps coordinate systems that cannot be represented as WKT to close variants. For example, the Lambert conformal conic variant for Michigan is mapped to Lambert conformal conic.
New query functions for measurements, low-level:
GeomAreaGeo - takes a geometry value and values for major axis, eccentricity, tolerance and returns geodetic area. The geometry value must be an area in lat/lon, with X (lon) and Y (lat) expressed in degrees. The returned value is in squared units of the major axis (typically, meters, for a returned value in square meters).
GeomBearing - takes a geometry value and returns the projected bearing. The geometry value must be a line with a single branch. The bearing is computed from the first to last coordinate of the line. The returned value is in degrees.
GeomBearingGeo - takes a geometry value and values for major axis, eccentricity and returns the geodetic bearing, computed using Vincenty's formulae. The geometry value must be a line with a single branch in lat/lon, with X (lon) and Y (lat) expressed in degrees. The bearing is computed from the first to last coordinate of the line. The returned value is in degrees.
GeomBearingPoint - takes a pair of point (x2) values and returns bearing. The returned value is in degrees.
GeomBearingPointGeo - takes a pair of point (x2) values and values for major axis, eccentricity and returns geodetic bearing, computed using Vincenty's formulae. The point values must be in lat/lon, with X (lon) and Y (lat) expressed in degrees. The returned value is in degrees.
GeomDistancePointGeo - takes a pair of point (x2) values and values for major axis, eccentricity and returns geodetic distance, computed using Vincenty's formulae. The point values must be in lat/lon, with X (lon) and Y (lat) expressed in degrees. The returned value is in the units of the major axis.
GeomLengthGeo - takes a geometry value and values for major axis, eccentricity, tolerance and returns geodetic length, computed using Vincenty's formulae. The geometry value must be an area or a line in lat/lon, with X (lon) and Y (lat) expressed in degrees. The returned value is in the units of the major axis.
New query functions for measurements, high-level:
CoordMeasureMake - takes a coordinate system definition, a unit name and a geodetic boolean flag and returns a measure object (similar to coordinate converter) that can be used to perform measurements.
If the coordinate system is lat/lon and the unit is lat/lon, measurements are Euclidean, performed in the specified lat/lon coordinate system.
If the coordinate system is lat/lon and the unit is metric, measurements are geodetic.
If the coordinate system is metric and the unit is lat/lon, measurements are Euclidean, performed in the base lat/lon coordinate system of the specified coordinate system. The geometry values are converted to lat/lon as part of the process.
If the coordinate system is metric, the unit is metric, and the geodetic flag is false, measurements are Euclidean, performed in the specified metric coordinate system.
If the coordinate system is metric, the unit is metric, and the geodetic flag is true, measurements are geodetic. The geometry values are converted to lat/lon as part of the process.
CoordMeasureArea - takes a measure object and a geometry value and returns area. The geometry value must be an area in the coordinate system of the measure object. The returned value is in the squared units of the measure object.
CoordMeasureBearing - takes a measure object and a geometry value and returns bearing. The geometry value must be a line with a single branch in the coordinate system of the measure object, the bearing is computed from the first to last coordinate of the line. The returned value is in degrees.
CoordMeasureBearingPoint - takes a measure object and a pair of point (x2) values and returns bearing. The point values must be in the coordinate system of the measure object. The returned value is in degrees.
CoordMeasureDistancePoint - takes a measure object and a pair of point (x2) values and returns distance. The point values must be in the coordinate system of the measure object. The returned value is in the units of the measure object.
CoordMeasureLength - takes a measure object and a geometry value and returns length. The geometry value must be an area or a line in the coordinate system of the measure object. The returned value is in the units of the measure object.
Removed query functions: GeomScaleToSystem, GeomScaleToSystemCff, GeomScaleToUnits, GeomScaleToUnitsCff. These functions have been superceded by functions in the CoordMeasureXxx family.
New query functions for watersheds:
TileWatershedAreas and TileWatershedAreasPar - take an image and a number for minimum flow, return a table of watershed areas. The XxxPar variant takes an additional parameter for thread configuration. The image must be single-channel.
TileWatershedLines and TileWatershedLinesPar - take an image and a number for minimum flow, return a table of watershed lines (streams). The XxxPar variant takes an additional parameter for thread configuration. The image must be single-channel.
Watershed functions return a table with the geometry values as well as values for stream ID (int64), target stream ID (int64), Shreve order (float64), Strahler order (float64), flow (float64), and total flow with children (float64).
Watersheds are computed using a new algorithm that is radically different from the algorithm used by 8. Due to differences in algorithms, watersheds computed by 8 and 9 are similar, but frequently not equal. Most of the differences come from plateaus, where the behavior of streams is wide open to interpretation. The algorithm used in 9 is much faster than that in 8 and can handle vastly more data. Multiple threads help more with more data
New transform templates: Watershed Areas and Watershed Lines. (Transforms and functions to fill sinks to specified level / trace upstream lines are coming.)
Some performance examples for the new watershed algorithm:
Example 1, 1200 x 1200 raster, computing streams (lines)
8: 75.000 sec
9, single thread: 3.024 sec
9, multiple threads: 1.727 sec
Example 2, 1200 x 1200 raster, computing watersheds (areas)
8: 175.000 sec
9, single thread: 3.843 sec
9, multiple threads: 2.705 sec
A Cutting Edge build introducing group logic in layers, with extensive structural improvements for better installations, enhanced GPGPU, and even faster startup. GPGPU now requires Kepler generation or later GPU.
The Layers pane allows grouping layers for maps, grouping frames for layouts and grouping fields for tables. See the updated Layers Pane topic for examples.
Each layer group has a leading, parent layer. Layers following the parent can be grouped with it by using the new Group button in the Layers pane toolbar
Grouping indents child layers one step to the right
Groups can be nested within further indent levels.
Ungrouping (moving one indent to the left) is done using the new Ungroup button in the Layers pane toolbar
Selecting a group's parent layer selects all its children. This allows quickly manipulating the entire group to show / hide, etc.
Reordering, deleting, adding layers, etc., auto-adjusts group info.
The Layers pane has new keyboard shortcuts for groups: Ctrl-Right arrow = Group, Ctrl-Left arrow = Ungroup.
Structural improvements decrease the size of installs and conserve resources at runtime. More importantly, they significantly decrease startup time when launching Manifold or Viewer, usually cutting it in half, for noticeably faster startup:
Code for ECW / JPEG2000 has been moved into a separate DLL and is not loaded until it is needed.
Code for external Unicode collates has been moved into a separate DLL and is not loaded until it is needed.
Data for external Unicode collates now loads from a single location (~\extras\icudtl.dat) in both 32-bit and 64-bit modes.
Code for V8 has been moved into a separate DLL and is not loaded until it is needed.
Viewer installs exclude most V8 code (not needed because Viewer disallows running scripts).
V8 scripts run faster. V8 objects exposed by Manifold do less locking and use less memory. Work proceeds on API documentation for the object model for V8. It is similar to the object model for .NET, but tailored specifically for use from Javascript.
Installation packages register the Manifold ODBC driver without running code in Manifold binaries. This helps solve the most frequent problem with uninstalls, when the 32-bit executable gets blocked due to a false positive in a third-party antivirus tool, and thus cannot run as part of the uninstall.
The Help - About dialog shows the install state of the Manifold ODBC driver, 32-bit or 64-bit, or regular or cutting edge. Possible report types:
Not installed - the ODBC driver is not installed
Installed - the ODBC driver is installed into the same location from which Manifold was launched
Installed with mismatching options - the ODBC driver is installed into the same location, but the install is either partial or damaged or out-of-date. The driver has to be reinstalled to work properly
Installed in different location - the ODBC driver is installed but into a different location, for example a portable install in a different folder.
The Help - About dialog for Viewer does not report ODBC driver install state, since Viewer does not include the Manifold ODBC driver.
Installation and un-installation of the Manifold ODBC driver is now accomplished from the Help - About dialog (Administrator privileges required). See the updated Help - About topic. Command line installation / un-installation of the Manifold ODBC driver is no longer supported.
GPGPU code requires CUDA devices of compute capability 3.0+, that is, Kepler or higher. Cards with Kepler started appearing in 2012. All GPGPU modules have been adjusted to use Kepler features. See the discussion in the updated GPGPU topic.
GPGPU code can use much more memory, allowing for bigger tiles and longer sequences of operations for enhanced speed. This is new code not specific to the new requirement for Kepler or higher.
All Kriging implementations (regular Kriging, median polish, and regression Kriging) compute model parameters on GPGPU for significantly faster operation.
The schema field dialog shows and allows specifying type restrictions for geometry fields (area / line / point) for databases that support it.
The schema field dialog hides bounds and Z option for geometry fields that are not native to the database. These are GEOMMFD and GEOMWKB, both stored as binary data opaque to the database.
The schema index dialog disallows creating spatial indexes on geometry fields that are not native to the database.
Linking ECW, JPEG2K, or SID images allows editing the coordinate system, style and other metadata for the linked images.
Exporting an image to PNG or WEBP automatically creates an RGB, an RGBA, a grayscale or a palette image depending on the number of channels in the tile and the settings for the image in the Style pane (three illustrations below).
Exporting a big image to ERDAS IMG or TIFF skips producing data for intermediate level 1, producing produces intermediate levels starting from level 2, to save space.
Reading GML, GeoJSON, KML and similar data preserves all coordinates in multi-type geometry values. Geometry values declared as multi-type but with all parts having the same underlying type (all areas, all lines, or all points) preserve that type. Geometry values that mix parts of different types are converted to either areas, if there are no lines and the value is a mix of areas and points, or lines (all other cases).
This build or more recent
is a must have build to replace
all prior builds, for all Release 9 and Manifold Viewer users. 9.0.169
provides truly superior rendering improvements over the prior Official
build. Build 9.0.169 is an Official build. Manifold System 9.0.169
delivers hundreds of enhancements released in the preceding series of
Cutting Edge builds. See individual Cutting Edge releases for details.
A summary of highlights from 9.0.168.1 through 9.0.168.12
can be grouped into eight major sections:
Better controls and dialogs:
Transform and Select panes use parameter pickers with easy switches between fields, values, and expressions, units for numeric parameters, and more.
Copying and pasting components automatically adjusts references across data sources.
New transfer rules for transforms based on new aggregates.
Totally new Schema dialog with separate dialogs for fields, indexes, and constraints, with a separate dialog for building expressions used in computed fields and constraints.
Schemas allow easy, one-click addition of key mfd_id field and index to enable editing and selection, plus inline editing of coordinate system info when adding geoms, easy creation of image tile fields and indexes and much more.
Radically improved vector graphics and Style:
Color pickers provide a huge selection of standard colors, plus filtering of colors by name, and color picker tool to copy colors found anywhere on a monitor.
Style pickers provide huge lists of built-in styles.
Custom styles provide many separately controlled options for areas, lines, points, and labels.
Explicit control over area borders, area inner / outer zones, line left / right sides, line begin / end caps, line dash patterns, halos, shadows, point and label boxes.
Use SVG paths for points symbols, and bitmaps for point symbols and area fills.
Numerous rendering optimizations for accelerated graphics. Reduced graphics mode has been trimmed to basic rendering features in order to be very fast and very robust.
All style information has been streamlined into portable JSON strings that can be safely copied and pasted as well as parsed and synthesized programmatically.
A staggeringly diverse range of visual effects can now be created in Manifold, with examples of bitmap use as seen above from the Example: Use Repeating Images to Fill Areas topic.
Smoother, faster rendering with less flicker:
Rendering of small data finishes faster due to reduced waits.
Rendering of large data reduces blinking by smarter scheduling of screen updates.
Web images are quickly filled with reduced resolution data with the display detailed with increased resolution data as it comes in from servers.
Re-rendering maps after changes to data (a layer is turned on or off, an object is selected, etc) coordinates with rendering threads to reduce blinking.
Rendering maps with multiple layers makes much better use of system resources, producing results faster.
Significantly extended raster transforms, with many more GPGPU optimizations:
GPGPU has been enabled for Viewer. Viewer now runs both CPU parallel and GPU parallel.
Aspect, Slope, numerous linear filters all run on GPGPU.
Better versions of all linear filters: Blur, Direction Blur, Gaussian Blur, Edges, etc.
New Curvature: Gaussian, mean, plan, profile (GPGPU enabled).
Median filters (GPGPU enabled).
Many new, common numeric aggregates running on pixel vicinity (GPGPU enabled).
Fill Missing / Fill Missing Nearest transforms to fill missing pixels.
Many new functions and enhancements for working with Tile Filters, have opened the door to dramatically improved image processing using filters, as shown in the SQL Example: Process RGB Images using Matrix Filters topic.
An RGB image of a butterfly as processed by a multichannel RGB Prewitt edge detection filter. Open a larger version of the above image.
Many improvements and additions in the query engine:
Big optimizations for searches on BTREExxx indexes on remote databases.
Radical improvements in GPGPU code generation, better use of device memory and big savings in data transfers.
Functions to create read-only or read-write (with writethrough) table caches.
Functions to print data (like tiles) to JSON and parse them back.
Functions to create shapes for linear filters.
Far faster numeric aggregates and new numeric aggregates: Diversity, DiversityIndex, Major. INLINE Sum(a, b, c) syntax for aggregates. See the INLINE topic.
Function to compute bounded areas.
Functions to compute raster aggregates for pixels under geoms (same as Transfer Heights in 8, but many times faster).
Regression Kriging.
Many improvements for databases:
Native reading and writing of SQL Server geometry.
Reworked discovery of database objects to better support databases with many tables and views.
Support for TinyPoint data in GPKG.
ADO.NET connections adopt to database and recognize geometry and other data types on SQL Server, Oracle, etc.
MySQL connections support geometry features added in MySQL 8.
Databases cooperate with the Schema dialog: allow specifying non-NULL field constraints, allow or disallow creating autogenerated fields, ask to specify whether a new geometry field will contain Z data if the database requires that info in advance, highlight indexes used as primary keys, etc.
Extended support for Unicode and Collations:
New options for built-in collations. See the new Collations topic.
Collations specific to databases, available collations listed in the new MFD_COLLATE table.
Collations based on UCA (the Unicode standard currently being adopted by many databases).
Specialized pickers and dialogs for collations.
Fail-safe loading of collations that depend on external data.
Many improvements in file / web dataports:
Manifold now uses, and auto-produces during export, intermediate levels stored as RRD, which enables "instant on" viewing of very large images linked from formats that utilize RRD sidecar files.
Many dataports no longer reorder channels for RGB data and no longer invert values in the alpha channel, putting all necessary adjustments into dynamic style data.
Export to ERDAS IMG.
CSV better autodetects separators and delimiters.
SHP allows working with files bigger than 2 GB. Exporting data larger than 2 GB to SHP creates multiple SHP files.
SHP export automatically resizes text fields to actual length and writes text values as UTF8.
TIFF recognizes GDAL metadata, reads mask images, reads Z offset and scale if available.
New supported formats: ICO, HEIF.
The ArcGIS REST dataport has been extended to support more types of servers.
Illustration, above: The Richat Structure in Africa, a blended display using Space Shuttle SRTM terrain elevation data colored and hill shaded, blended with Bing satellite imagery with increased contrast, and enhanced by 7x7 matrix calculation of Curvature, Profile as discussed in the Example: Enhance Terrain with Curvatures topic. The diameter of the view is approximately 42 kilometers.
A Cutting Edge build with numerous internal improvements to rendering. This build represents a new generation in display quality, as rendering improvements are truly wonderful.
"Blinking" in map and other windows has been virtually eliminated, with dramatically smoother rendering.
Optimizations provide visibly faster rendering in many cases, especially with large and complex maps, images and drawings.
Numerous optimizations eliminate redraws in associated windows for better speed. For example, alterations in a table window would previously call a redraw in an associated map window that was open no longer do so.
Numerous optimizations in rendering of displays from web sources.
New query functions: TileInterpolateKrigingRegression, and TileInterpolateKrigingRegressionPar - These functions perform regresssion kriging. The parameters are those used for regular kriging plus one extra parameter for a choice of regression model, which can be auto, linear, or quadratic, specified as a numeric value.
There is a new Interpolate, Kriging with Regression transform that utilized the above query functions.
The ArcGIS REST dataport now supports servers with an exportImage command, adding more ArcGIS Rest servers that can be utilized.
Reading TIFF files better detects images with 4 channels where the 4th channel is not alpha.
Reading DBF better detects fields with floating-point values (DBF fields are 'numeric' with no separation into integer vs floating-point, whether a field is integer or floating-point is determined at runtime).
The SHP dataport now reads multipatch geometry values in shapefiles. Opening a SHP file with multi-patch geometry values will disable writes to the geometry field.
This build provides a dramatically new Schema dialog with numerous enhancements for field and Index creation, including computed fields and fields with constraints. The new Schema dialog allows use of the same interface and commands, learned once and used in many settings, to manipulate table schemas both within Manifold and also within a variety of popular DBMS packages, providing options and settings that are automatically configured for the varying capabilities of different DBMS packages.
Expression tabs in the Select and Transform panes provide an improved and simplified vertical query builder that joins available fields and SQL operators / functions into a single list to make better use of screen space.
The New Field button in the Options tab of the Style pane that is used to add a new field to store style overrides has been changed to launch the Schema dialog. Altering a table schema keeps the Style pane in the Options tab.
The Schema dialog has been reworked to show fields, indexes and constraints using a grid control with a toolbar on top. Grid rows for fields display the field name, field type and any optional expression used to compute the field (computed fields only). Grid rows for indexes display the index name, index type and a list of fields used by the index. Grid rows for constraints display the constraint name, constraint type (boolean) and the expression used to compute the constraint. To edit an existing field, index or constraint, double-click it.
The Add toolbar command in the Schema dialog allows adding new fields, indexes or constraints. The command displays a drop-down menu with separate commands for adding a field, index or constraint. If the table schema is read-only, all commands are disabled. If the table is in a database and the database does not support adding indexes or constraints, relevant commands are disabled.
Shortcut: pressing Insert in the Schema grid starts adding a new field.
The Delete toolbar command in the Schema dialog allows deleting selected fields, indexes or constraints. If the table schema is read-only, the command is disabled. To select items, use Ctrl-click or Ctrl-Space or any other selection method available in table windows.
Shortcut: pressing Delete in the Schema grid deletes any selected items.
The Move to Top / Move Up / Move Down / Move to Bottom toolbar commands in the Schema dialog allow reordering selected fields, indexes or constraints. If the table schema is read-only, the commands are disabled. When editing the schema of an existing table, only new items about to be added to the schema can be reordered. Discussion: Reordering of schema items is important for fields and indexes. The order of fields determines which fields can be accessed by computed fields: a computed field can only reference fields that appear before it. The order of indexes determines which index is going to be used a primary key when creating a table on a database: if there are multiple candidates, the system uses the first one.
The Field dialog, called by Add - Field in the Schema dialog, displays Expression and Expression Context tabs for a computed field using two multiline edit controls. Expression text can be scrolled, selected and copied even if the expression cannot be edited. The Edit Expression button launches the Expression dialog to allow editing expression and expression context using a separate query builder for each. The expression is only allowed to use fields that appear before the edited field in the schema. The expression context is not allowed to use any fields (statements in the expression context are evaluated once per field, not once per record). If the table is in a database that does not support computed fields (only MAP files do), the Edit Expression button in the Field dialog will be disabled.
The Field dialog allows specifying coordinate systems for geom and tile fields.
The Field dialog allows specifying bounds for geom fields for tables in databases that require specification of bounds. Some databases need to know bounds in advance to allow creating a spatial index.
The Field dialog allows specifying a Z values option for geom fields for tables in databases that require specification of Z values. Some databases need to know whether a geom field will have XY or XYZ values in advance. If the database requests both Z values and bounds, bounds can. but are not required to, include the Z range, specified as [xmin, ymin, zmin, xmax, ymax, zmax].
The Field dialog allows specifying pixel type, tile dimensions and tile reduce mode for tile fields.
The Field dialog allows specifying an autogenerated option for integer fields if the table is in a database that supports autogenerated fields. Most databases support autogenerated fields. MAP files do not, but tables in MAP files can always use an MFD_ID field, which is a hybrid: MFD_ID fields will autoincrement for a new record if inserted records do not specify a value for MFD_ID. If a valid value for MFD_ID is supplied, it will be accepted. When creating a new table or adding new fields to an existing table, the option to create a field as autogenerated is treated as a request to the database. The database is free to ignore the request. For example, if we ask a database to create multiple autogenerated fields but that database allows having only a single autogenerated field per table, the database may create only a single autogenerated field and create all other fields as regular, non-autogenerated fields.
The Constraint dialog displays expression and expression context for a computed field similar to the Field dialog. The expression for a constraint can use all available fields. The expression context for a constraint is not allowed to use any fields.
The Constraint dialog for a table in a database allows specifying a not-null constraint for a field, as long as the database supports not-null constraints. Most databases do. Within MAP files, Manifold only allows specifying constraints as arbitrary expressions. However, not-null constraints are very common and very useful, so Manifold now recognizes them and allows them in databases that support not-null constraints.
The Index dialog displays index types as one of three choices: regular index (btree) - supported in MAP files and on all databases,'spatial index for geometry (rtree) - supported in MAP files and on most databases, and spatial index for tiles (rtree) - MAP files only. Creating a btree index provides explicit options to allow duplicate values and nulls, and allows specifying up to four fields. All fields used in an index must be different, and the order of fields matters, but specifying fields 2 and 3 while leaving field 1 blank will work. Creating a btree index on text fields also allows specifying the collation for each field. Creating an rtree index for geometry allows specifying the geometry field. Creating an rtree index for tiles allows specifying the X and Y fields, and the tile field, plus options for the tile field (pixel type, tile dimensions, tile reduce mode). Selecting a tile field automatically loads options specified for that field in metadata, including any changes to it made in the context of the Schema dialog that have not yet been saved.
When working with a schema for a table in a database, editing a collation for a text field in the Index dialog lists collations that are available in that database in a Database tab in addition to the Standard tab of the Collation dialog. The Database tab lists collations exposed in the mfd_collate table, and the Custom tab allows copy / pasting a collation definition as plain text.
Until the Save Changes button has been pressed, the Schema dialog highlights changed fields, indexes and constraints using light blue, preview, background color. New fields, indexes and constraints also display an 'added' + icon in the row handle. This is similar to how edits in tables are shown with blue background until the edit is committed.
The Schema dialog includes an Edit Query button. Press it to open the Command Window loaded with a query that will perform the same changes that would be made in the Schema by pressing the Save Changes button. Besides providing an education in how to use SQL to do what the Schema dialog does,using the Edit Query button allows modifying proposed changes beyond what is packaged within the Schema dialog. For example, to avoid overly-complex/long dialogs the Schema dialog allows up to four fields to participate in a new index. If we wanted five fields to participate when creating a new index, we could create a new index using four fields, launch Edit Query, change the SQL slightly to add a fifth field to be used by the index, and then run the query to create a new index using five fields.
The Schema dialog strips properties with caching directives for fields and indexes (commonly produced by copy and paste from data sources that use .MAPCACHE) on the first change to the schema.
Deleting fields in the Schema dialog automatically deletes (after a confirmation dialog) indexes and not-null constraints referencing those fields. When deleting a selected field referenced by an index, if the index is also selected, the dialog silently deletes both. If the index is not selected, the dialog raises a confirmation dialog before deleting the index. If the user confirms the deletion, the dialog deletes both the field and the index. If the users cancels the deletion, the dialog does not delete anything, and selects the index.
The Add Identity toolbar command in the Schema dialog allows adding an identity field (key field) and index. In a MAP file, the command adds a system mfd_id field and a system mfd_id_x index regardless of which indexes the table already contains. System fields and indexes are 'free' in that they do not add to space requirements and they do not harm the performance in any way. If the table already contains an mfd_id field and/or an mfd_id_x index, the Add Identity command selects them. When the table is In a database, the command checks if the table contains a btree index allowing no duplicates and no nulls. If so, the command selects that index and the field or fields it uses. If the table contains no btree index, the command add a new field and a new btree index on that field. If the database supports creating autogenerated fields, the field is created as an autogenerated field.
The Schema dialog displays a coordinate system icon for geom and tile fields. The icon is red if the coordinate system is not specified and black otherwise.
The Schema dialog displays an index icon for fields that participate in indexes. The icon is dark blue if the field is the 'main' field in an index. A 'main' field is a geom field or a tile field in an rtree index, or the single field in a btree index of any type. The icon is gray if the field is not the 'main' field in any index.
The Schema dialog displays a key icon for a primary key index in a database.
The Create Drawing, Create Image, Create Labels, and Create Table dialogs when run within a database data source context will create autogenerated identity fields within the tables they create if the database supports them.
Exporting to SHP autosizes text fields in the DBF component of shapefiles.
Reading DBF interprets all-space text values as NULLs.
Pasting or importing drawings to a GDB data source automatically splits geoms of different types into separate drawings, similar to how export to GDB works.
The Oracle dataport supports collations for text fields. Recognized collations are listed in the system mfd_collate table.
The DB2 dataport supports collations for text fields with some limits: We can use collations when reading, but we cannot create indexes with collations, due to the current limitations of the latest versions of DB2. DB2 is in the process of migrating to doing everything in Unicode and that process is not yet complete. Recognized collations are listed in the system mfd_collate table.
Creating a spatial index on an Oracle database uses SPATIAL_INDEX_V2 (instead of SPATIAL_INDEX) if it is available.
Oracle, PostgreSQL, SQL Server and other database dataports use native names for indexes. Previously in Release 9, most index names were synthesized dynamically from field names and the dataport was mapping back and forth. When creating a new index, if the desired index name is already in use, the dataport synthesizes a new name using a GUID. The desired name may already be in use because index names in MAP files are unique within a table, and index names on many DBMS packages must be unique within the database.
Numerous small and or technical changes, not listed in these notes, have been implemented for the latest versions of DBMS packages, including MySQL 8, Oracle 18, PostgreSQL 11.
Collations are a big deal with modern, international data. See the new Collations topic.
Collations for text fields in btree indexes are specified as a single string with all options embedded into it. For example, instead of ... COLLATE 'en-US' NOCASE NOACCENT ..., the collation is specified as ... COLLATE 'en-US, nocase, noaccent'. Available options: noaccent, nocase, nokanatype, nosymbols, nowidth, wordsort.
The default collation language of " (two single quote ' characters with nothing between) is specified as 'neutral'. The only option supported by 'neutral' is 'nocase', all other options are ignored. The short form of '' for the entire default collation is accepted, but 'neutral, nocase' must spell the collation language as 'neutral' and cannot omit it or use an empty string. See the COLLATE topic.
Btree indexes on ANSI text fields are limited to the 'neutral' language. Discussion: Why are indexes on ANSI text fields limited to 'neutral'? Because the definition of ANSI text in 9 is 'interpret single-byte characters using whatever is the current language on the current system'. If we move a MAP file from a German system to a Portuguese system, what ANSI characters map to changes: what was being displayed as 'a' can now become 'b'. Consequently, when building indexes on ANSI character data we cannot rely on the linguistic meaning of characters, because that changes, and can only rely on what does not change, that is, on character codes. That's what the 'neutral' language does.
The type system has been extended to support external collations for character data. Discussion: The main purpose of the change is to better support indexes on character data in databases. Databases very often use text values as unique identifiers, so being able to use indexes on text fields is very important for performance. That can be a problem, because indexes on text fields often use complex rules to tell whether text values are equal to each other and / or how they are ordered relative to each other. This happens because default settings on many databases are set to use a specific linguistic collation, whether English or not. Another problem is that although support for Unicode has been gradually growing, many databases are only fully starting to support Unicode. For many databases, even famous ones, many changes to fully support Unicode are currently in development branches and are not in released versions of databases. Before the current efforts to support Unicode, various databases often implemented linguistic collations for various languages using custom code. That requires multiple, complex, adaptive efforts by Manifold to utilize indexes on character data within different databases, that is, to allow using these indexes in queries and various internal operations. Manifold had to (a) extend Manifold's type system to allow plugging in arbitrary code for implementing collations, and (b) implement custom collations currently used by each database product, at least the more common and more popular DBMS products. The above change is for (a), changes for (b) follow below. Additional DBMS packages (Oracle, etc.) will be added in the next build.
The SQL Server dataport supports binary collations and collations based on Windows. This covers the vast majority of collations available on SQL Server.
The PostgreSQL dataport supports binary collations, collations based on Windows and collations based on ICU. This covers the vast majority of collations available on PostgreSQL, including the coming ICU.
The MySQL dataport supports binary collations, collations in the 'latin' family and collations based on ICU. This covers a sizeable portion of the most commonly used collations.
Database dataports list available external collations using the system mfd_collate table. Each collation has a unique name and a definition that can be used in a query. Unsupported collations have NULL definitions.
Loading a MAP file with indexes with external collations that cannot load no longer makes the entire MAP file fail to load, and instead makes the index inoperable (reads via the index fail) and the table read-only (writes fail). The index that failed to load reports the failure in the log window and can be dropped cleanly. If all indexes that failed to load for a particular table are dropped, the table becomes writable automatically. Discussion: Indexes with external collations that cannot load can happen when, for example, a MAP file created on a later build which supports more external collations will try to load into this build, or when Manifold deprecates a particular external collation, or when an external collation that depends on data stored outside of EXT.DLL will not be able to find it, etc.
The Collate query function has been replaced with CollateCode - takes a string with the collation definition and returns a numeric code. If the collation definition is external, the function returns NULL.
New query function: CollateCodeDef - takes a numeric code for a (non-external) collation and returns the collation definition string.
'Xxx, Intl' transforms use a new collation picker with a readout and a button with a drop-down menu on the right. The drop-down menu allows selecting a favorite collation, editing the list of favorite collations, or composing a custom collation using a dialog. The custom collation dialog displays the list of available languages (with a filter) and collation options.
The Style tab of the Record pane arranges style pickers similar to the Style pane and includes a separate picker for full style.
The query engine allows inlining a call to an arbitrary aggregate query function with a variable number of arguments using INLINE: INLINE Max(3, 4, 5) or INLINE Covar(1, 1, 2, 5, 3, 6, 4, 1). If the aggregate function takes more than one argument, the number of arguments passed to the inline call has to be a multiple of that. See the INLINE topic.
The StringJoinTokens aggregate query function uses a personalized separator for each passed value. (Previously, the function was always using the first passed separator value and ignoring all other values.)
Query builder descriptions for clauses and functions have been cleaned up to use the same terms as much as possible - descriptions now only uses '<query>' when a query component is meant, otherwise using'<table>', etc.
Renamed transform template: Maximum Value -> Limit Low (parameters - Value: / At least:). See the Know Your Limits video.
Renamed transform template: Minimum Value -> Limit High (parameters - Value: / At most:).
New transform template: Limit (parameters - Value: / At least: / At most:).
New transform templates for tiles: Limit / Limit Low / Limit High - limit pixel values from one or both sides.
Removed query functions that are no longer needed after INLINE: GeomMergeAreasPair, GeomMergeLinesPair, GeomMergePointsPair, GeomUnionAreasPair, GeomUnionRectsPair, ValueMax, ValueMin. Renamed query function: GeomIntersectLinesPair -> GeomIntersectLines.
The TileFill query function has been reworked to keep the original tile type and to accept xN values. (Previously, the function was forcing the tile to FLOAT64.)
New transform for single-channel images: Fill. See the Fill in Missing Pixels video.
New query function: TileFillMissing - fills invisible pixels in a tile with the specified value and makes them visible. Keeps the original tile type.
New transform for single-channel images: Fill Missing.
TileCombine query function has been renamed to TileFillMissingCopy and reworked to keep the original tile type (was forcing both tiles to FLOAT64) and do nothing if tile sizes do not match (was returning NULL).
New query function: TileFillMissingNearest - fills invisible pixels in a tile with values of first found visible pixels in the specified radius. Keeps the original tile type.
New transform for images: Fill Missing Nearest.
Exporting SHP writes character data to DBF as UTF-8 and produces a CPG sidecar file describing the encoding in DBF as UTF-8 for third-party products.
Exporting DBF (including as part of SHP) computes the maximum length of each text field, increases it to the nearest multiple of 8 to accommodate for future edits, and uses that as DBF field length. (DBF files cannot store fields longer than about 250 characters. Previously, all text fields were exported with maximum length: this was wasting a lot of space and making text field sizes snowball in third-party products.)
Reading TIFF reads mask images.
Reading TIFF reads Z offset / scale / unit data if available.
Reading TIFF recognizes GDAL metadata and no longer reports it as unrecognized. (The metadata itself is appended into a comments component - now with Unix line ends converted to Windows.)
Exporting images to BIL, FLT, XYZ and other formats applies Z offset / scale to pixel values.
Specifying style for image channels allows inverting data interpretation by setting the channel minimum value to be higher than the channel maximum value. For example, switching the bounds for an improperly interpreted alpha channel from min = 0, max = 255 to min = 255, max = 0 will invert the interpretation without doing anything to the stored pixel values. See the example in the Style: Images topic.
Reading TIFF no longer reorders channels in color images to BGR and no longer inverts A from 0 = fully transparent to 0 = fully opaque. Instead, all data necessary to properly display the image is put into its style.
Image servers now support a {-y} escape sequence in custom URLs, which reverses the Y direction within a level.
Importing SDTS data reads Z scale and offset. Exporting SDTS data writes Z scale and offset.
The Median query aggregate has been optimized to perform significantly faster. Performance increases depend on the data type and are highest for numeric values. Quick performance numbers for Median before and after optimization:
Median on float64: 15.580 sec before optimization -> 1.883 sec after optimization
Median on float64x2: 16.820 sec -> 3.114 sec
Median on nvarchar: 32.102 sec -> 21.943 sec
StDev, StDevPop, Var, VarPop query aggregates use much less memory. Corr, Covar, CovarPop query use memory more efficiently, with less pressure for cache.
New query aggregates: Diversity, DiversityIndex, Major - all numeric.
Diversity computes the total number of different values.
DiversityIndex computes a measure of diversity: 1 - sum(individualcount^2) / (totalcount^2). A diversity index of 0 means that all values are equal.
Major computes the most frequently occurring value. If all values are different, Major reports the lowest.
Some record number limitations removed:
Median, Diversity, DiversityIndex, Major are currently limited to 2 billion values, effectively limiting them to aggregations of sets of no more than 2 billion records.
Corr, Covar, CovarPop are currently limited to 1 billion pairs of values. Those limits will be removed in future builds.
StDev, StDevPop, Var, VarPop were also limited to 2 billion values in the past, but 168.9 has removed that limit for those aggregates.
New transfer rules for transforms like Overlay, Touching:
diversity,
diversity index,
major.
New query function: TileFilterDefCircle - takes a radius and a center value, creates a definition for a circular filter.
Renamed query functions:
TileFilterDefBlur -> TileFilterDefSquare ('square' better describes what the function does),
TileFilterDefBlurDirection -> TileFilterDefDirection,
TileFilterDefBlurGaussian -> TileFilterDefGaussian.
New transform: Average - a variant of Blur that uses a fixed center of 1. Provided for convenience, it might not be awfully clear that Blur with a center of 1 computes an average value.
New query functions, which take an input tile, a radius and a filter definition, and return a tile. For each input pixel, take values in the pixel vicinity corresponding to non-zero filter weights and compute maximum / minimum.
TileFilterMax,
TileFilterMin
New transforms:
Maximum,
Minimum.
New query functions: TileFilterSum - takes an input tile, a radius and a filter definition and returns a tile. For each input pixel, take values in the pixel vicinity and compute their weighted sum.
New transform: Sum.
There are GPGPU versions of TileFilterMax, TileFilterMin, TileFilterSum. Limitations: radius must be 8 or less.
New query functions, which takes an input tile, a radius and a filter definition, and return a tile. For each input pixel, take values in the pixel vicinity corresponding to non-zero filter weights and compute diversity, diversity index, major value.
TileFilterDiversity,
TileFilterDiversityIndex,
TileFilterMajor
New transforms:
Diversity,
Diversity Index,
Major.
There are GPGPU versions of TileFilterDiversity, TileFilterDiversityIndex, TileFilterMajor. Limitations: radius must be at most 8, the number of non-zero coefficients must be at most 32.
(Note on pixel window computations like TileFilterMax / TileFilterMajor and others: there is a slight difference from 8 in handling invisible pixels. The surface evaluator tool in 8 was producing a visible pixel if at least one pixel in the vicinity was visible, this was making some of the invisible pixels visible. In 9 we changed that so that invisible pixels stay invisible and visible pixels generally stay visible and only turn invisible if non-zero filter coefficients all fall onto invisible pixels - so, if filter center is non-zero, then visible pixels will always stay visible.)
Blur, Diversity, Diversity Index, Major, Maximum, Median, Minimum, Sum transforms for images specify filter shape using a dropdown control. Available shapes: square (default), circle, cross. Blur, Cross and Median, Cross transforms are removed (no longer needed).
New query functions, which take an image and a geom, and compute specified statistic on all pixels under the geom. The image must have a single channel. The geom must be in the coordinate system of the image.
TileGeomAvg,
TileGeomCount,
TileGeomDiversity,
TileGeomDiversityIndex,
TileGeomMajor,
TileGeomMax,
TileGeomMedian,
TileGeomMin,
TileGeomSum
Quick performance numbers comparing TileGeomAvg in 8 and 9:
Transferring heights from 10k x 10k pixels to 8142 small areas: 8 = 37.800 sec. 9 forced to use a single thread = 7.835 sec. 9 with 8 threads = 1.769 sec. The maximum difference in results between 8 and 9: 9.3e-11 with a height range of 100-1500. 9 computes slightly more accurately.
Transferring heights from 10k x 10k pixels to 1050 big areas: 8 = 74.050 sec. 9 forced to use a single thread = 7.256 sec. 9 with 8 threads = 1.680 sec. The maximum difference in results between 8 and 9: 4.7e-11 with a height range of 100-1500.
Current builds do not expose transforms for transferring heights from images to drawings. These will be added in upcoming builds. In the interim, a simple example query to transfer heights from images to drawings using SQL:
-- $manifold$
-- prepare to convert coordinates from drawing to image
VALUE @conv TABLE = CALL CoordConverterMake(
ComponentCoordSystem([image]),
ComponentCoordSystem([drawing])
);
-- set 'height' field in drawing to average height taken from image
UPDATE (
SELECT [mfd_id], [height],
TileGeomAvg([image], CoordConvert(@conv, [geom])) AS [computed]
FROM [drawing]
THREADS SystemCpuCount()
) SET [height] = [computed];
GPGPU parallel functionality has been enabled in Viewer, the same as in Release 9.
GPGPU parallel functionality has been restricted to only 64-bit Manifold or 64-bit Viewer sessions. Launching either Manifold or Viewer in 32-bit mode will allow CPU parallelism, but not GPU parallelism. This change has been pending for a long time as NVIDIA has gradually been reducing support for CUDA in 32-bit applications. The main appeal of GPGPU parallelism is performance: if someone desires GPGPU performance, they should be running 64-bit in any event.
GPGPU code produced by the query engine supports processing tiles with borders of up to 8 pixels. Calls to tile functions which use borders of more than 8 pixels automatically go to the CPU, and can freely intermix with calls going to the GPGPU.
GPGPU code produced by the query engine allows individual operations to store intermediate results on the GPGPU device without passing them to CPU and back. This feature enhances performance in many scenarios, such as chaining operations.
New query function: TileJson - takes a tile and prints it into a JSON: [ p1, p2, p3, ... ]. Pixels are ordered by Y and then by X, with row 0 being the first row. There are no extra separators between rows. If there are multiple channels, each pixel is printed as the value of channel 0, then the value of channel 1, etc, for example: [ ... c0, c1, c2, ... ]. Invisible pixels are printed as 'null' (an allowed keyword in JSON). If there are multiple channels, invisible pixels are printed as a sequence of 'null' values for each channel.
New query function: StringJsonTile - Takes a string, the number of pixels in X and Y, the number of channels and then parses the string into a tile. The format is the same as used by TileJson. The function also takes a boolean <strict> flag which defines what to do when the number of values in the string is less or more than the X * Y * channels - if <strict> is true, the function returns NULL, otherwise the function returns a completed tile ignoring the extra values in the string, or an incomplete tile padded with invisible pixels.
New query function: TileRemoveBorder - Removes a border of the specified size from a tile. If the border size is too big for the passed tile, returns NULL. If the border size is negative or zero, returns the unmodified tile. TileAspect, TileBlur, TileBlurDirection, TileBlurGaussian, TileEdges, TileEdgesDirection, TileSharpen, TileSlope no longer remove the border from the passed tile automatically; corresponding transform templates have been reworked to use TileRemoveBorder.
New query function: TileFilter - Takes an input tile, a radius and a tile with filter coefficients, and performs a linear filter. This is a generalization of TileBlur and similar functions, which imply specific ways to create the filter tile.
There is a GPGPU version of TileFilter. Limitations: the radius must be 8 or less. For radius greater than 8, the query engine will use parallel CPU.
Query functions for built-in filters have been reworked to produce filter definitions to pass to TileFilter. This automatically makes transforms for built-in filters use GPGPU. Also, we can now easily check built-in filter definitions by using TileJson or TileToValues.
TileBlur - Has been changed to TileFilterDefBlur. The 'power' parameter has been changed to 'center'. The 'center' parameter provides the value for the center pixel, which is forced to be at least 1. Higher values result in more contribution from the original pixel, resulting in less blur.
TileBlurDirection - Has been changed to TileFilterDefBlurDirection. The 'power' parameter has been changed to 'center'.
TileBlurGaussian - Has been changed to TileFilterDefBlurGaussian. The 'power' parameter has been changed to 'center'. The filter shape has been changed to use a decaying exponent.
TileEdges - Has been changed to TileFilterDefEdges. The 'power' parameter has been changed to 'center'. The filter shape has been changed to the laplacian of gaussian, normed to the center value.
TileEdgesDirection - Has been changed to TileFilterDefEdgesDirection. The 'power' parameter has been changed to 'center'.
TileSharpen - Has been changed to TileFilterDefSharpen. The 'power' parameter has been changed to 'center'.
New query function: TileFilterDefCross - Produces the definition for a cross-shaped filter of specified radius.
New transform: Blur, Cross - Performs a cross-shaped linear filter.
New query function: TileFilterMedian - Takes an input tile, a radius and a tile with filter coefficients, and performs a median filter. Pixels with non-zero filter coefficients participate in computing the median, whereas pixels with zero filter coefficients and invisible pixels do not.
New transform: Median - performs a square median filter.
New transform: Median, Cross - performs a cross-shaped median filter.
There is a GPGPU version of TileFilterMedian. Limitations: the radius must be 8 or less, and the number of non-zero coefficients must be 32 or less.
Coordinate systems store local offset and local scale values as well as units for Z. Z metrics are reported and edited as part of coordinate system metrics. This is a big addition that involves code in many places, from dataports to transforms. This change has been implemented now in advance of universal support for vertical datums, planned for the future. Even before that, Z metrics will be used for many things, for example, for shading.
New query function: ComponentCoordSystemScaleXYZ - Takes a component and returns a FLOAT64X3 value with metric scales for X, Y and Z. If the component is in lat/lon, the metric scales for X and Y are approximated: images use scales at the center latitude of the image, while drawings use scales at zero latitude.
The TileAspect and TileSlope query functions now take an additional parameter for XYZ scales.
New query functions: TileCurvGaussian, TileCurvMean, TileCurvPlan, TileCurvProfile - These functions take an input tile and a radius, and compute curvature of the specified type.
New transforms: Curvature, Gaussian / Mean / Plan / Profile, analogous to the TileCurveXXX query functions.
There are GPGPU versions of the TileAspect, TileSlope, TileCurvGaussian, TileCurvMean, TileCurvPlan, TileCurvProfile query functions. Limitations: radius must be 8 or less.
The TileMakeNew query function now creates big tiles significantly faster. If the requested tile size is too big, the function returns NULL instead of throwing an error.
The About dialog reports the release ID for Windows 10 and Windows Server 2016 in parentheses, for example, (1809).
WEBP files are read progressively. Canceling the process of reading a big file is more responsive.
There is a new dataport for HEIF (High Efficiency Image File) format files, which can both read HEIF files and write them. The dataport requires a system component currently provided by Microsoft as a free download for Windows 10 from the Microsoft Store.
The MySQL dataport now supports MySQL 8.0, adding numerous capabilities related to geometry, including support for coordinate systems.
The preview pane in the style dialog automatically sets its background color to that of the component window.
The style dialog allows altering grids of bitmaps appearing with the Images choice by right-clicking a bitmap or a blank space in the grid to pop open a context menu. The context menu includes commands to add new bitmaps, to delete all bitmaps, and to delete the clicked bitmap. The number of bitmaps in the grid is limited to 1024. Switching to an empty Images grid will automatically launch the Import dialog to add bitmaps.
The style dialog caches bitmaps used for point and area styles for the duration of the session. If the style dialog is launched on a style that already uses a bitmap, that bitmap is shown in a group named 'Current'. The Current group is read-only.
New point styles have been added for basic shapes: diamond, pentagon, hexagon, octagon, plus variants with inserts. The diamond symbol is different from the rectangle symbol in that the rectangle fills the entire size x size symbol box, while the corners of the diamond are placed on a circle with a radius of the symbol box size.
There are new boxes for basic shapes: diamond, pentagon, hexagon, octagon.
Point styles with inserts have been adjusted to allow the insert color to be transparent.
Point styles with inserts allow specifying the size of the insert and the insert type. The default insert size is 30%. The default insert type is auto = same as the main shape. Other choices are to specify a different shape for the insert, such as a circle, rectangle, rectangle rd, triangle, diamond, pentagon, hexagon, or octagon.
There are new point styles for basic shapes which use dividers sthat split the shape into two parts. One of the parts is filled using background color. The other part is filled using foreground color by default and the color can be overridden. All basic shapes come in combinations using three types of dividers: diagonal, horizontal, or vertical.
There are new point styles for flags with dividers.
Reducing the width of rounded rectangle shapes converts circle arcs in corners into ellipse arcs.
There are new point styles for arrows, including an arrow triangle, an arrow wing, and several others.
The style dialog for lines allows specifying symbols at the beginning of a line and at the end of the line, called a Begin cap and an End cap, respectively. The caps are applied to each line branch. Available cap styles include basic point shapes (centered on the begin / end coordinate of the line) as well as arrows (rendered with the tip end of the arrow on the begin / end coordinate of the line). Placing caps automatically adjusts the line metric near the begin / end coordinates to produce a correct arrow direction. If the line is too short, caps are not drawn.
There are new line styles using repeating symbols that are basic shapes: circle, rectangle, rectangle rd, triangle, diamond, pentagon, hexagon, and octagon. Repeats are specified as a pattern of <size>, <space>, ..., similar to the pattern used for line dashes. A negative size rotates relevant shapes 180 degrees. If the repeat pattern is empty, the style assumes '300%,300%', which is a sequence of 300% shapes with 300% spaces. There is an option to turn a base line on or off.
Left / right lines are oriented to proceed in the same direction as the line whenever possible. This synchronizes repeats on left / right lines to start at the same logical position on the line and makes the orientation of repeats on left / right lines way more predictable.
The query engine supports new options for ordering criteria on Unicode text fields (used in CREATE TABLE / ALTER TABLE as well as in ORDER):
KANATYPE / NOKANATYPE - use or ignore kana type for Asian languages. The default is KANATYPE (two instances of the same character in, say, hiragana and katakana, are considered to be different),
WIDTH / NOWIDTH - use or ignore character widths for Asian languages. The default is WIDTH (two instances of the same character which only vary in width are considered to be different),
WORDSORT / NOWORDSORT - ignore or use space between words. The default is NOWORDSORT (space between words matters).
Dataports for databases expose all text fields as NVARCHAR (Unicode) even if they are stored as VARCHAR. Attempting to create a VARCHAR field will create it as VARCHAR on the database, but the field will look like an NVARCHAR data type in Manifold. Discussion: The intent here is to preserve data because what VARCHAR means varies between data sources and converting between different meanings frequently loses data. Example: in Manifold VARCHAR means 'characters in the currently active codepage, whatever it is', and on databases VARCHAR usually means 'characters in the codepage associated with the field / table / database'. If we connect to a database, we can ask it to return data as VARCHAR, and it might return data in a German codepage. If we then try to use that data as VARCHAR in Manifold on an English system, the characters will be interpreted wrongly. That will affect both the display of characaters as well as comparisons and orderings. Manifold 8 allows setting a codepage for the field. However the end result is always just to convert the data in all codepages different from the default one to Unicode and then to handle data as Unicode. Release 9 goes one step further and moves the conversion to Unicode to be as close to the data as possible.
SQL Server indexes on text fields honor field collates. Discussion: Previously, all database dataports assumed that indexes on text fields sorted values using binary order plus / minus case. This works reasonably well for field values with English-only characters, but starts breaking when field values have non-English characters and when the query engine starts off-loading searches to the database, as Manifold to an increasing degree does more aggressively. All database dataports are now in the process of being reviewed to make sure that all indexes on text fields that are recognized and reported in the schema work without any limitations on what characters the text values may contain.
Exporting a drawing to SHP writes points with a single coordinate and points with multiple coordinates into separate files. Discussion: Early builds reduced multipoints to points, throwing the extra coordinates away. That lost data, so a better approach was needed. Different approaches were tried within what the SHP spec allows, with the previous cutting edge build marking all points as multipoints. All of those solutions were producing issues in third-party software, including in ESRI products. The current build therefore separates points and multipoints into different files. This at least realistically avoids all avoidable issues with third-party software that cannot handle multipoints: if a particular product cannot handle multipoints, the user will not be able to work with them, but at least all other objects will be available for use.
There is a new dataport to read images in ICO file and CUR file formats. Reading images in these formats is useful for loading bitmaps for styles. If an ICO or CUR file contains multiple images, the dataport loads them all. When used to load an Images grid for a style dialog, if the same image exists in multiple sizes the style dialog by default loads the biggest image with the highest number of colors.
JSON Style definitions have been reworked with style data for individual elements compartmentalized into sub-objects more closely reflecting the structure of the rendering pipeline. Old style definitions may not work. Size values are no longer split into Xxx / XxxPct properties depending on whether they are absolute or relative, and are instead stored as strings. Size values support 'x@' syntax, with '5@' meaning '5 points bigger than the original size' and '-1@' meaning '1 point smaller than the original size'.
The Style dialog has been reworked to split style data into tabs. Style elements on each tab are turned on or off and customized independently. Tabs for most style elements allow specifying custom colors or a custom size.
The Style dialog displays tooltips with symbol names for grids and allows filtering grids by typing a partial symbol name. The name for a font character is set to the numeric character code.
Point styles allow specifying the offset and angle for placing the point symbol relative to the point location. Label styles allow specifying offset and angle for label symbols. The offset is only applied for labels based on points.
Point styles apply halo and shadow separately to point symbols and to point boxes. Halo and shadow can be applied at the same time. Label styles apply halo and shadow separately to label text, to label icons and to label boxes. Line styles allow applying a halo to lines. Area styles allow applying a halo to borders.
Line styles for dashes and dots have been collapsed into a single style with a customizable dash pattern. The dash pattern is a sequence of number pairs separated by commas: dash width, space width, ... . Each value can be specified either in absolute points ('3' = 3 points) or in relative points ('3@' = 3 point more than the line size) or in percents ('300%' = 3 times the line size). Example: a pattern of 5,5 creates a line composed of a pattern that is a repeating sequence of a dash five points long followed by a space five points long.
Line styles specify dash cap as a choice between 'flat' (default), 'round', 'square' and 'triangle'. The 'flat' choice results in a flat end to the line at the terminal coordinate location. The 'square' choice is similar, but the line extends beyond the terminal coordinate location to the same distance as the line width. 'Round' and 'triangle' also extend beyond the terminal location but with round or triangle point ends.
Area styles allow specifying formatting for regions of specified width that extend from the area border into or outside of the area. The 'Inner' tab specifies formatting for the region that extends from the area border into the area. The 'Inner Border' tab specifies formatting for the border of that region that is inside the area. The 'Outer' tab specifies formatting for the region that extends from the area border outside the area. The 'Outer Border' tab specifies formatting for the border of that region that is outside the area. The distance to extend into or outside is specified on the first tab as 'Inner / outer offset'. The same distance applies in both directions. Note: Using inner / outer regions require significant computation and will typically increase rendering times by 50-200%, depending on the data and on other formatting options in use. Higher offset values usually will result in better performance.
Line styles allow specifying formatting for left and right offset lines. What is 'left' and what is 'right' depends on the orientation of the line. Left and right offset lines for a closed loop coincide with what would be inner and outer borders if the closed loop was treated as an area. If the original line crosses itself or goes in different directions from the same location or converges into the same location from different directions, left and right offset lines for a rendered shape may not agree with left and right offset lines computed on the original geometry. Geometry functions to compute offset lines on geometry values are planned for future builds.
Point styles can be bitmaps, which can be taken from almost any graphics format from which Manifold can import images, including all of the common formats such as .png, .bmp, jpg and so on. To specify a bitmap to use as a point style, in the Symbol or Style dialog, click the dropdown menu button above the point symbol grid, choose 'Images...', and then select one or more image files. The dialog will import all files selected and display them in the grid. The process of the import tracks its progress and can be canceled. Bitmap styles are limited to 128 x 128 pixels, but the import will read images up to 1024 x 1024 pixels and scale them down proportionally. Files bigger than 4 MB are ignored without being read. Image pixels can be completely or partially transparent. Choose a bitmap image from the grid and press OK. The image will be imported into the style and saved within the style string. No connection with the originating file is maintained.
Bitmap pixels for styles are encoded as lossless WEBP, greatly reducing size in images with many similar colors, and are stored in the style JSON string as a BASE64 string. Storing bitmap pixels directly within the style JSON allows freely transporting components using bitmap styles between systems since all bitmaps used by styles are saved within the project. There is no need to bundle the original image files, to maintain projects in the same relative path in relation to the original image files or other problems caused by storing bitmaps for styles outside of the project file.
The halo and shadow for bitmap styles are rendered as if the style was a square. Future builds may change this to follow the boundary of transparent/non-transparent pixels in the bitmap.
Area styles can be bitmaps, for example, such as bitmaps that show stone, cloth, wood or other textures, or pictorial images. Bitmaps for areas are rendered on top of the fill color. The fill color can be set to be transparent, if desired.
The reduced graphics mode rendering engine has been better separated from the normal graphics rendering engine. The reduced graphics rendering engine has been simplified to use a fixed point / line / area / label style and to use as few resources as possible. The purpose of reduced graphics mode is to produce at least some display in scenarios where normal graphics mode may be unavailable. Reduced mode therefore has been made as simple, fast and robust as possible to enable a display while graphics drivers are updated or other issues that may prevent normal graphics mode are resolved.
Style dropdowns now save the last used order and view mode.
The query engine better optimizes searches on BTREExxx indexes on tables from remote databases. The query engine passes additional data with the search request describing what specific filtering it wants to perform, and this helps the dataport for the remote database translate the request into a command that optimizes better on the database engine. The difference in many cases is significant. For example, before the change, requests on non-clustered indexes in SQL Server optimized much worse than requests on clustered indexes. After the change, they optimize well in both cases and usually optimize much better than before.
The SQL Server dataport treats all views as updatable. Unlike with other databases, there is no way to tell whether a SQL Server view is updatable or not. There is a semi-standard way to declare a view to be updatable or not, but SQL Server ignores such declarations and always marks the view as read-only even if the view is actually writable.
The SQL Server dataport no longer attempts to determine the SRID for each new GEOMETRY or GEOGRAPHY field it sees. Instead, it determines the SRID only when the table with the field is opened or is otherwise used, for example, to create a drawing. The dataport also no longer creates a virtual drawing for each new geometry field. These changes eliminate long waits in some cases on initial connections to SQL Server.
New query functions have been added to aid working with parts of tables coming from remote databases and other data sources:
TableCache - Takes a table and caches its data, allowing either reads or both reads and writes depending on the parameter. The result table has all indexes that were in the original table. Writes to the result table update both the original table and the cache.
TableCacheIndexGeoms - the same as TableCache and also adds an RTREE index on each geometry field.
Given the new functions and other improvements in the query engine mentioned above, we can
Write a SELECT ... WHERE x=y query in Manifold which hits the index on the database, and have that query run fast on the database. We can then...
Wrap the result into a write-through cache using TableCache or TableCacheIndexGeoms. We can then...
Display the result as a drawing and work with it, with any changes to the drawing being seamlessly transported through the cache into the database.
The ADO.NET dataport supports UUID and CHARVAR (ANSI text) values natively, without conversions.
The ADO.NET dataport supports GEOMETRY and GEOGRAPHY values for SQL Server connections.
The ADO.NET dataport supports SDO_GEOMETRY values for Oracle connections, with conversion through WKB.
The SQLite dataport forces datetime values to ISO strings.
The OLEDB dataport supports the LINK table type used by Access for tables linked from foreign data sources.
Copying and pasting components converts maps to the latest Release 9 storage format.
Copying and pasting components no longer stops if a table refuses to paste. Instead, it moves on to paste other components. Tables can refuse to paste, for example, if the target data source does not support fields of the desired type and the table has no other fields.
Copying and pasting components adjusts references to components in maps and layouts.
Copying and pasting components adjusts references to components across data sources, if pasted into the same instance of Manifold. Example: open a new MAP file, add a data source for Bing Maps, expand the data source for Bing Maps, and then copy the image from Bing Maps into the MAP file. The system will automatically adjust the new, pasted image's properties to refer to the table inside the Bing Maps data source with no need to do that manually.
Copying and pasting drawings to SQL Server and similar databases preserves the original bounding box of the spatial index, if it exists, or sets it to the bounding box of the data set.
Copying and pasting drawings to SQL Server and similar databases preserves the original SRID of the spatial index, if it exists.
Opening MAP files created by Manifold 8 migrates layout items for components and text labels.
Parsing WKT supports EMPTY geometry.
(Fix) Parsing WKT no longer fails to skip whitespace other than space (hex 0x20).
Reading data from a CSV file better autodetects which characters are used for the list separator, decimal separator and quote characters. In previous builds, autodetection could be fooled by first lines containing sequences such as "a;b;c;d;e",f. Such sequences could cause the dataport to choose a list separator character of ';' rather than ','.
Exporting an image to an ERDAS IMG file writes intermediate levels into an RRD sidecar file.
Exporting an image to a TIFF file writes intermediate levels into an RRD sidecar file. Reading data from a TIFF file automatically uses intermediate levels in an accompanying RRD file.
GPKG tables report the total number of records to the Component pane.
Reading a SHP file supports reading files higher 2 GB. Shapefile format allows going up to 4 GB, but many software packages stop working sooner.
Exporting a drawing to a SHP file automatically starts a new SHP file if the amount of exported data exceeds 2 GB.
An administrative build with no feature changes from 9.0.168.4. Extends the time-out period on the 9.0.168.4 Cutting Edge build, to compensate for November holidays in many countries.
The Area format sample in the style pane has been changed to a triangle, for better differentiation from color samples, and is no longer widened to occupy available space.
Controls in the style editing dialog have been rearranged to better use screen space. The preview has been moved to the left. Pickers for colors, sizes and rotations have been moved to right, below the preview. Grids showing format samples now have thin row handles.
The preview in the style editing dialog allows changing canvas (background) color by clicking the color picker button in the top right corner.
Buttons with format samples in the Style pane have been resized and rearranged to occupy less space while being more legible. Buttons for colors, sizes and rotations have been reduced, while buttons for symbols have been slightly increased.
The Style pane no longer allows selecting multiple format parameters simultaneously. Instead, it allows selecting the full area / line / point / label style button to use a field to set multiple format parameters. When this happens, the list of format values at the bottom displays and edits full formats (this allows for very quick thematic formatting). All commands used by the list of format values can be applied to full formats: Apply Palette sets fill color to colors from the palette and stroke color a shade darker, Interpolate interpolates fill color, stroke color, size and rotation simultaneously, Darken / Lighten / Grayscale adjust both fill color and stroke color, Reverse reverses everything.
Rendering vector data with thematic formatting optimizes for cases when all values for a format are the same. This frequently happens when thematic formatting is applied to full formats, for example, when colors change, but sizes stay the same.
Selecting a button with already established thematic formatting in the Style pane no longer automatically computes field statistics (how many records are in each interval / value). To compute statistics, click Refresh Statistics in the toolbar above the list of format values.
The Style pane for images now opens directly with relevant controls and image formatting, with no need to first click a pixel sample button.
The color picker dropdown menu automatically sorts colors by hue on startup, and includes a mode button to switch between sorting by name and sorting by hue.
The font picker dropdown menu includes choices for bold / italic / bold italic versions of the default font (Tahoma).
Symbol picker dropdown menus include numerous new symbols. Menus display symbol names, which can be used in the filter box to filter the display by symbol name.
Format picker dropdown menus include a scrollbar and support common scroll keys (up / down, pageup / pagedown, home / end).
Format picker dropdown menus display tooltips for format choices. The tooltip includes the name of the choice and its group.
The color picker dropdown menu shows hex codes (#RRGGBB) for color values on the right side in list mode as well as in the tooltip.
Point styles can use SVG paths. Normal graphics only.
There are hundreds of new point symbols based on SVG paths from Font Awesome.
There is a new halo exterior option for points and labels. Supported parameters: stroke width (default 3 pt). Works for both normal and reduced graphics.
Reading data for an ad-hoc query on SQL Server automatically recognizes geometry values and seamlessly converts them to Manifold geometry.
Reading data for an ad-hoc query on GPKG / Spatialite automatically recognizes geometry values and seamlessly converts them to Manifold geometry.
Reading Spatialite geometry supports the TinyPoint geometry subtype.
Connecting to SQL Server caches SRID values for geometry columns in tables and views in MFD_META. This allows not recomputing these values for each session.
This is an experimental build incorporating feedback on style controls. Further refinements to style controls will appear in upcoming builds.
Color, font and style samples in all controls in dialogs and panes use the current rendering engine specified in options.
The Style pane for drawings includes new style pickers for areas, lines and points, which allow setting or editing all parameters for a style simultaneously (colors, size, symbol, rotation if applicable).
The Style pane for labels includes a new style picker for labels, which allow setting or editing all parameters for a style simultaneously (colors, size, font, symbol).
Editing just the symbol for areas, lines, points or labels using a dialog hides controls for colors and similar parameters near the preview.
Picking a new value for a color, size, rotation, symbol or a full style uses a new dropdown window with choices arranged as a grid. The choices can be scrolled using the mouse wheel. A scrollbar will likely be added as well..
The dropdown windows for picking size or rotation allow entering the desired value directly at the bottom of the window.
The dropdown windows for picking color or full style allow switching between grid view (each row contains multiple choices) and list view (each row contains a single choice and shows its name).
The dropdown windows for picking color or full style allow filtering choices by name. Example: enter 'green' into the filter box in the dropdown window for color.
The dropdown window for picking color includes a new 'Color Picker...' button that picks color from an arbitrary location on a screen. Clicking the button paints the entire screen with a light shade of white and changes the cursor to an eyedropper to indicate that clicking on a screen location will pick its color instead of working normally. Pressing Escape cancels picking. Any location can be clicked, including Windows system toolbars, Windows desktop background images, pages inside browser windows, etc.
The size of the text in font samples has been increased from 8 pt to 10 pt, to make fonts easier to distinguish.
The dropdown window for picking color includes many new colors. Colors are separated into groups. The prior selection of colors from previous builds is in the first group. Groups are highlighted using alternating shades of gray.
The dropdown window for picking full style definitions for areas, lines, points and labels includes many new styles. These are popular starting combinations. They may be altered using the More button as well.
Editing a label style allows overriding colors used for label icon.
Editing background color in the Layers pane uses a new dropdown window for picking color, like the color picker window for Style.
There are several new symbols for points with inserts. Insert color has been set to coincide with stroke color by default, and can be overridden.
Format picker controls use flat borders. Format picker controls in Record and Style panes no longer reserve space for an icon on the right and now use the entire control area to paint the format sample. Format picker controls and template parameter picker controls use a darker shade of gray (the shade used by system controls on Windows 10) to better stand out as controls that can be clicked.
Web dataports allow specifying a user agent string. By default, the user agent string has been set to 'Mozilla/5.0 Manifold/9.0'. Having a non-empty user agent string allows working with servers that require it. Some US government servers, for example, require it for internal statistics / bug reporting.
The ERDAS IMG dataport allows exporting raster data.
The ERDAS IMG dataport automatically creates RRD pyramids when exporting raster data.
Exporting data as GeoJSON exports xN values as JSON arrays.
Web geocoding dataports use {...} syntax for URL parameters, similar to web image dataports.
Web geocoding dataports that support both direct and reverse geocoding have been split into two different dataport subtypes to allow customizing URLs for both geocoding methods. Previously, the user could only access the URL for direct geocoding, and could not adjust parameters - like input language - for reverse geocoding.
This is a highly experimental build intended to expose new capabilities with a simple set of new style controls, to gather feedback. Style controls will change in upcoming builds.
Graphics engine choices in Tools - Options have been renamed to Normal and Reduced. Reduced means no hardware acceleration. Many features for styles and fonts are not available in Reduced configuration.
Font definitions use JSON. Old font definitions have been mapped to JSON seamlessly.
Fonts allow weights other than bold, including light, semi-bold, ultra-bold, etc.
Fonts allow a new type of italic: oblique. Currently this can only be set in the JSON string. The system font dialog does not show this option. The system font dialog will be replaced with a Manifold dialog in upcoming builds.
Fonts allow specifying stretch, including normal, condensed, expanded, etc. This capability is set via JSON and will be in the UI when the Manifold font dialog is introduced.
Style definitions use JSON. Old style definitions have been mapped to JSON seamlessly. Old and new style definitions can be mixed freely as well. For example, if we open a pre-168.2 MAP file with thematic formatting for styles, and we change the style for one of the thematic choices in the style pane, the system will happily use the new JSON style for the edited choice and old style strings for all other choices.
Style pickers for areas, lines, points, labels allow specifying a custom style using a dialog. The dialog allows specifying parameters for a style and displays a preview (for the style only, within the dialog - not a control to change what goes into the map).
Editing a point style allows choosing styles from either a built-in list of Standard vector shapes or from a font. Instead of listing the names of installed fonts, the dialog provides a list of symbol fonts available on Windows systems, such as Wingdings, and an option to choose a custom font. All parameters chosen for a custom font are preserved.
Point styles for vector shapes allow specifying stroke width. Stroke width as well as all other numeric parameters for distances can be set to either an absolute value in points (for example, '5') or a percentage of the size (for example, '100%'). The default value is 1 pt.
Line styles with dashes allow specifying dash length (default 300%), dash space (default 100%) and whether dashes should use round or flat caps (default flat).
Line styles allow specifying an exterior line shape rendered below the line. The only exterior option for lines is currently is a solid line which is as wide as the line size plus specified padding (default 3 pt). By default there is no exterior.
Default line size has been changed from 0 (device-specific pixel) to 1 pt.
Area styles allow specifying the border style, by default a solid line, and border stroke width (default 1 pt). Border styles with dashes allow specifying the dash length, dash space and dash caps, with the same defaults as for lines.
Point styles allow specifying an exterior shape. Exterior options are: circle, rectangle, rounded rectangle. Parameters for exterior shapes are: stroke width (default 1 pt) and padding (default 3 pt).
Point styles allow specifying a special exterior shape, a vector shadow. Parameters for the shadow are: angle (default 135 degrees) and offset (default 3 pt). Rotating a point does not rotate the shadow. That is intentional, to ensure the shadow shape always falls below the shape at the given angle.
Area styles support rotation for the fill pattern. The Style pane and the Record pane include controls to specify area rotation similar to point rotation. Area patterns rendered from vector data continue to use antialiasing and other display features.
Area styles allow specifying the stroke width for the lines in the fill pattern (default 1 pt).
Label styles have been reworked into label exterior shape options, similar to points, and similarly parameterized.
Labels can display a point sign, like an icon, together with the text. The location of the sign is determined by the label style. Currently the only option is to have the sign on the left, with more options in progress. The point sign can be either a built-in vector shape, with a parameter for stroke width, or a font character.
All styles allow overriding colors for the exterior shape. For example, we can set the shadow for a blue-yellow symbol to be light gray, or we can set the border of a red-transparent area to be a darker shade of red. All overridden colors can be set to be transparent.
Label styles allow specifying the size of the point sign (default 100%) and the space between the point sign and the text (default 3 pt).
Point styles for built-in vector shapes allow specifying width. The width can be 100% or less. 50% results in a vector shape that squeezed to half the default width.
Editing a style allows specifying colors, size and rotation for the preview. The preview helps to choose a style: it does not drive what happens in the map.
There are multiple new styles for areas and points.
The Clip transform has been renamed to Clip Individual, the Clip All transform has been renamed to Clip.
Parameter pickers in Select and Transform panes have been made slightly less tall and cleaner, drop down arrows on the right have been removed.
Gravity and Kriging transforms allow interpolating data with Voronoi neighbors outside of the convex hull. Specifying a zero for the number of neighbors (the default) restricts output to the convex hull. Specifying a negative value produces data outside of the convex hull.
Voronoi transforms use a single margin parameter instead of inflate X and inflate Y parameters. The margin is applied to the extent of the original data, not to the extent of the original data plus Voronoi vertices as before This change prevents the result of Voronoi transforms from looking like a huge rect with all data in one of the corners. If preserving all Voronoi vertices is important, this can be achieved by specifying a very large margin. Specifying a negative value for the margin sets it to 1% of the maximum dimension of input data.
The IMG dataport supports more variants of IMG with RRD.
Launching Kriging with very little data no longer sometimes fails if it is impossible to compute model parameters. Instead, if there is too little data to set up the model, Kriging degrades to gravity interpolation.
The IMG ERDAS dataport can use data for intermediate levels in the accompanying RRD file when linking. Linking a big IMG ERDAS image with an RRD opens instantly if there are intermediate levels stored in the RRD.
Merging images whose coordinate systems match except for offsets is faster.
Exporting an image to TIFF writes values for invisible pixels as legal values set to the reserved void value specific to the pixel type, and advertises used void value for other applications. Visible pixels with values equal to the reserved void value are adjusted so they do not coincide with the reserved void value. This may lose some data, but better the pixel is adjusted slightly than an invisible pixel is made visible or vice versa. Void values for signed integer types are chosen to be the minimum allowed value. In the -128 to127 range for pixel values, -128 is the void value. A pixel with a valid value of -128, that is, not intended to be a void value pixel, will be adjusted to -127 on export. Void values for unsigned integer types are chosen to be the maximum allowed value. Void values for floating-point types are chosen to be the maximum allowed value.
The query function for Kriging with median polish allows specifying the processing step for median polish. The default value is 0, for auto.
Kriging transform templates produce a report of the parameters used, including resolved values for autocomputed parameters, and save the report into the description property of the new component.
Global (not per-record) parameters for transform templates are specified using the parameter picker control.
Numeric parameters for distance values in transform templates show the coordinate system unit and allow changing it. The coordinate system is taken from the target field. If the coordinate system uses uneven local scales for X and Y, the parameter is set to use 'native unit' by default, and the parameter value is used without scaling. Otherwise, the parameter is set to use the unit specified in the coordinate system.
The model parameter in Kriging transform templates is selected using a drop-down menu (instead of being entered as a numeric value).
New transform template: Clip - clips a geom with another geom, works between fields.
New transform template: Clip All - clips geoms with the union of areas in the specified drawing.
New query function: GeomBoundedAreas - computes bounded areas for a line geom.
New transform template: Bounded Areas - computes bounded areas for a drawing.
Kriging interpolation and gravity interpolation optimize searches for Voronoi neighbors and perform slightly faster.
WARNING: this update contains changes incompatible with prior updates:
Queries created in this update might not run on previous updates and vice versa, due to changes to query syntax made to accommodate new features.
This build will read MAP files created by prior versions of Manifold, but MAP files created by this build will not load on previous versions of Manifold, due to extensions to computed fields and constraints, and due to changes to rtree indexes for images.
Build 9.0.168 is an Official build. It incorporates all prior 9.0.167.x improvements as well as the following:
The Style pane, when used with a big image, no longer tries to compute statistics for the entire image; instead, it computes statistics from intermediate levels. This greatly improves the usability of the Style pane with big images without losing much in terms of accuracy.
New transform templates, and the SQL Functions they use, for interpolating vector data:
Interpolate, Gravity - uses TileInterpolateGravity,
Interpolate, Kriging - uses TileInterpolateKriging,
Interpolate, Kriging with Median Polish - uses TileInterpolateKrigingMedianPolish,
Interpolate, Triangulation - uses TileInterpolateTriangulation,
Interpolate, Triangulation with Segments - uses TileInterpolateTriangulationSeg.
All added templates allow specifying a margin, the width of which is expressed in drawing coordinate system units (by default set to 0), and allow specifying the resolution, expressed as a multiplier of drawing coordinate system units (by default set to 1).
All added templates allow using multiple threads both for preparing interpolation data (except for Gravity which does not benefit from multiple threads when preparing interpolation data) and for performing the actual interpolation.
TileInterpolateGravity and TileInterpolateKrigingXxx query functions interpret the meaning of any specified, positive radius number and an auto-computed (zero or negative) number of neighbors as "use all neighbors within the specified radius". This covers all possible combinations of radius with a specified number of neighbors, or radius with an auto-computed number of neighbors. Other combinations are interpreted as:
Radius positive and neighbors auto-computed = Use all neighbors within the specified radius.
Both radius and neighbors auto-computed = Voronoi neighbors.
A specific number of neighbors, but radius auto-computed = Radius set by the system according to the global interpolation picture.
A specific number of neighbors and a specific radius = Use specified number of neighbors and specified radius.
Following is a summary of breaking changes to query syntax in 9.0.168. Queries written in older versions which use features listed below must be adjusted to the new syntax to work in 9.0.168 and subsequent builds.
-- Old
EXECUTE WITH (p NVARCHAR = 'folder') [[
SELECT * FROM mfd_meta WHERE
StringCompareCollate(property, p, Collate('', false, false, false)) = 0;
]];
-- New
EXECUTE WITH (@p NVARCHAR = 'folder') [[
SELECT * FROM mfd_meta WHERE
StringCompareCollate(property, @p, Collate('', false, false, false)) = 0;
]];
-- Old
FUNCTION combine(p NVARCHAR, q NVARCHAR) NVARCHAR AS
p & ': ' & q
END;
SELECT combine(name, type) FROM mfd_root;
-- New
FUNCTION combine(@p NVARCHAR, @q NVARCHAR) NVARCHAR AS
@p & ': ' & @q
END;
SELECT combine(name, type) FROM mfd_root;
Previous forms of syntax that allowed using EXECUTE without leaving the scope of the current query have been removed and can no longer be used.
-- Old
EXECUTE CALL ValueSequence(1, 10, 1);
-- New (uses replacement feature)
TABLE CALL ValueSequence(1, 10, 1);
-- Old
CREATE TABLE temp (
mfd_id INT64, INDEX mfd_id_x BTREE (mfd_id),
geom GEOM,
geomarea FLOAT64 AS GeomArea(geom, 0)
);
INSERT INTO temp (geom)
VALUES (GeomMakeRect(VectorMakeX4(0, 0, 10, 10)));
-- New
CREATE TABLE temp (
mfd_id INT64, INDEX mfd_id_x BTREE (mfd_id),
geom GEOM,
geomarea FLOAT64 AS [[ GeomArea(geom, 0) ]]
);
INSERT INTO temp (geom)
VALUES (GeomMakeRect(VectorMakeX4(0, 0, 10, 10)));
For more details, see cutting edge threads in the forum for builds leading up to 9.0.168, for example, the thread for build 9.0.167.6. The query builder lists all of the above changes plus new statements (VALUE, TABLE, SCRIPT) plus new functions.
Reading WMS tiles second-guesses servers that returns non-image data as a response and checks if the returned data is an image. Some WMS servers are erroneously configured to mislabel returned image data as non-image data. This second-guess allows Manifold to interoperate with such wrongly configured servers, albeit at the cost of some processing overhead when these servers or correctly working servers return error information.
The CSV dataport exposes an option for handling multiline text values. By default, the option is turned off.
Interpolating a raster from contours using DEST resolves complex inter-contour configurations similar to 8 (but much faster).
New query function: TileInterpolateTriangulationSeg - Takes a drawing with geoms with Z values and interpolates them using constrained triangulation, keeping segments from lines and areas. There is a boolean parameter that controls whether to improve triangulation of contours using DEST (true = use DEST).
New query function: TileInterpolateTriangulationSegPar - a parallel variant of TileInterpolateTriangulationSeg.
New query function: TileInterpolateGravity - Takes the same parameters as kriging without the model and interpolates data using gravity formulae, similar to 8. (There is no parallel variant, because preparing interpolation data for gravity does not benefit from using multiple threads, although producing tiles does.)
The Help menu includes a new Context Help command bound to F1, which shows a quick reference guide for the current component window. If there are no opened components, the command shows the general keyboard and mouse reference.
Migrating views from MAP files created by 8 better scales views for lat/lon maps.
The context menus used by parameter pickers in the Select and Transform panes show icons for Value and Expression commands and show the current choice. The icon for Value depends on the parameter type. The icon for Expression has been changed from an 'operator' icon to a 'function' icon.
Parameter pickers in the Select and Transform panes clean xN values and produce VectorMakeXn(...) calls for the query.
The Vector Value transform has been split into Vector x2 Value, Vector x3 Value, Vector x4 Value. This disambiguates the type of parameter value and allows parameter pickers to clean it (previously, entered text was going into the query verbatim).
The Select and Transform panes specify component parameters using parameter pickers.
Transforms that use collate parameters embed the collate value as a plain numeric code instead of a fairly involved call to Collate(...).
Transforms that use regular expressions embed the case switch as a plain string instead of a CASE statement.
Many (the majority of) transforms specify the 'main' transform parameter, automatically setting it to the target field.
The Select and Transform panes use parameter pickers for numeric and text parameters, but can be set to be taken from a field. This includes parameters like buffer size or tolerance. By default, such parameters are still set to use fixed values like 1 or 0, but the user can quickly switch to use a field. Quoting entered strings is no longer necessary.
Renamed transforms: Enclosing Rect, Rotation Allowed -> Enclosing Rect with Rotation, Is NULL -> Null Values, Is not NULL -> Non-null Values. Renamed transform parameters: DX -> Delta X, DY -> Delta Y, Pattern (when used by Like) -> Pattern (%_).
Parameter pickers sort fields and components alphabetically. The Transform pane sorts target fields alphabetically.
The log window tracks attempts to load add-ins from DLL files.
Loading add-ins ignores DLL files without an accompanying .DLL.ADDIN file, which, for now, may be empty. This dramatically reduces the time to initialize the main menu on configurations with many DLL modules in ~\bin, ~\bin64 and ~\extras which do not include add-ins. Ignoring DLLs not related to add-ins, such as DLLs for database clients or scripting engines, makes for a visibly faster launch.
Saving the current location from a map window saves the name of the context component into the location as an Entity entry. Right-clicking a location in the Project pane that has an Entity entry offers a View command in the context menu, which automatically opens the Entity component and navigates to that location.
Creating a new location based on a window using the New Location dialog saves the name of the context component into the location as the Entity context component as long as it is reachable from the data source of the location. Right-clicking on that component and choosing View launches the Entity context component if it is closed and navigates to the specified location.
Opening MAP files created by 8 migrates component views into locations. The created locations are put into the same folder as the producing component.
The log window reports the location of the active log file on startup.
The tracker tool reports total length using units of the current coordinate system, except for lat/lon, which is reported in meters.
The tracker tool reduces reported values for bearing and length to three decimal digits. The 'L:' and 'B:' captions are removed.
Right-clicking the map window with the tracker tool active displays a context menu which allows copying the values for bearing and length (both values are copied with full precision, with total length to the last-clicked location).
The Select and Transform panes specify parameter values which can be set to fields using a new parameter picker control. The control allows switching between fields of the required type, and includes options to use value or expression. Text entered for value is automatically quoted according to the value type (there is no need to type '...' for string values, no need to prefix each backslash with another backslash, etc). Text entered for expression is left intact (can freely combine field names, literals, etc, as before).
The PostgreSQL dataport optimizes bulk inserts, deletes and updates.
The GPKG dataport optimizes bulk inserts.
Exporting vector data to GPKG adds spatial indexes after writing geometry, for performance.
Parsing coordinate systems from WKT (PRJ) better handles Winkel-Tripel, polar stereographic and multiple other system types.
Parsing coordinate systems from WKT (PRJ) recognizes coordinate axes.
The DWG dataport recognizes variants of DWG up to R2018 and can read significantly more data from modern variants.
The DWG dataport maps character data to Unicode using codepage data in the file.
The build contains changes incompatible with prior builds. Specifically:
Queries created in this build might not run on previous build and vice versa. Queries created on previous builds will work in this build as long as they do not use function or query parameters.
MAP or MXB files created by this build will not load on previous builds.
Query parameters use @ as a prefix.
Old: EXECUTE WITH (p INT32 = 1) [[ SELECT * FROM mfd_root WHERE mfd_id=p; ]]
New: EXECUTE WITH (@p INT32 = 1) [[ SELECT * FROM mfd_root WHERE mfd_id=@p; ]]
EXECUTE requires either query text in [[ ... ]] or the name of an existing query, forms that allow using EXECUTE without leaving the scope of the current query have been removed.
Function parameters use @ as a prefix.
Old: FUNCTION f(p INT32) INT32 AS p+1 END; VALUES (f(10)), (f(11));
New: FUNCTION f(@p INT32) INT32 AS @p+1 END; VALUES (f(10)), (f(11));
All references to parameters inside a function body are resolved to local parameters. If a function wants to use a query parameter passed via EXECUTE, that parameter has to be transferred explicitly, under the same or different name.
Field values passed to computed fields and constraints continue to not use @ as a prefix. This allows existing computed fields and constraints to work without changes.
Parameter names can contain spaces and special characters: @[parameter name] or @[john@sample.com]
The query parser allows whitespace between @ and parameter name and between @ and unescaped string literal: @ abc works, @ 'c:\windows\system32' works.
There is a new VALUE statement which declares global variables.
Example: VALUE @deg2rad FLOAT64 = Pi / 180;
Example: VALUE @angle FLOAT64 = 30;
Example: VALUE @anglerad FLOAT64 = @angle * @deg2rad;
In the above, a type is required. The type can be TABLE.
A global variable can be redefined and assigned a different type and value. When this happens, previous references to the global variable will continue to use its former type and value (similar to overriding functions).
There is a short form of VALUE that omits the expression: VALUE @name NVARCHAR;
The short form of VALUE takes an existing value and converts it to the specified type. If there is no value with the specified name defined, the short form defines it and sets it to a NULL value. The short form is useful for handling query parameters.
Names of query and function parameters are allowed to be reserved words, since using @ removes the ambiguity.
There is a new TABLE statement which takes a query expression and returns a table.
Old: EXECUTE CALL ValueSequence(0, 10, 1);
New: TABLE CALL ValueSequence(0, 10, 1);
[[ ... ]] embeddings can be nested. [[[ and longer sequences are ignored instead of being parsed as multiples of [[, and the same for ]]. This allows having queries with nested EXECUTE statements and other constructions that now use embeddings, as discussed below.
Loading a MAP file resolves expressions in computed fields and constraints in a way that allows successfully loading expressions from cross-referencing tables.
Loading a MAP file allows expressions in computed fields and constraints to fail to load. All existing data including data in computed fields with expressions that failed to load is fully and safely accessible. Computed fields and constraints with expressions that failed to load can be safely removed.
Computed fields allow specifying one or more statements to prepare the execution context for an expression which computes values. This allows expressions for computed fields to use functions and scripts. Each computed field uses its own execution context. Functions and values defined in the context of a computed field are inaccessible from other fields. The syntax for computed fields in ALTER TABLE / CREATE TABLE has been changed to use embeddings and to allow specifying statements to prepare the execution context using WITH.
Constraints allow specifying one or more statements to prepare the execution context for an expression which accepts or rejects records, similar to computed fields.
There is a new SCRIPT statement which defines an inline script inside the query. Inline scripts can then be referenced by FUNCTION.
Queries generated by all panes and dialogs have been adjusted to use the new syntax: @ for parameter names, EXECUTE CALL migrated to TABLE CALL, a number of queries that were using parameter-less functions now use VALUE, etc.
The query builder lists all changes to syntax.
A temporary build with an extended timelock to enable operation past June.
Adds capability to automatically read files named with a .json filename extension as part of the .geojson dataport. Microsoft has recently published vector footprints of over 125 million buildings in the US in GeoJSON format under the Open Data Commons Open Database License (ODbL). However, Microsoft's files all use the .json extension instead of .geojson. This addition makes it easy to use Microsoft's new data set without the need to manually rename files. The GeoJSON dataport's limitation to 2 GB of text currently disallows use of Microsoft's Texas and California files, both of which are larger than 2 GB as GeoJSON text. This limitation will be removed in upcoming builds to allow use of these and other GeoJSON files that exceed 2 GB. See the Example: Import GeoJSON / JSON File topic.
Provides new, experimental query functions for interpolating surfaces: performance is far from final and point-and-click templates are not yet added (still in testing). Information on functions is preliminary information that may have small adjustments in future builds.
TileInterpolateKriging - Takes a drawing and the interpolation parameters and returns an interpolation object that can be used to produce tiles using kriging. See SQL Example: Kriging
The heights are taken from Z values in the drawing geoms. Geoms without Z values are ignored. All geoms are converted to coordinates. Duplicate XY coordinates are ignored: if duplicate XY coordinates have different Z values, the function uses one of these values and ignores the other ones.
The interpolation parameters are: radius for neighbor searches (kriging interpolates heights from closest coordinates), number of neighbors to use, and interpolation model. If both the radius and the number of neighbors are zero or negative, kriging uses Voronoi neighbors, (however many of them there are. If both the radius and the number of neighbors are positive, kriging uses no more than the specified number of neighbors in the specified radius. The interpolation model can be set to a zero or negative value to autoselect the model, or to a specific value as follows: 1 = linear, 2 = circular, 3 = exponential, 4 = gaussian, 5 = power, 6 = rational, 7 = spherical.
TileInterpolateKrigingPar - a parallel variant of TileInterpolateKriging. Takes an additional parameter with the thread configuration.
TileInterpolateKrigingMedianPolish - Takes the same parameters as TileInterpolateKriging and returns an interpolation object that can be used to produce tiles using kriging with median-polish, additional processing aiming to improve the quality of the interpolation.
TileInterpolateKrigingMedianPolishPar - a parallel variant of TileInterpolateKrigingMedianPolish, takes an additional parameter with the thread configuration.
TileInterpolateTriangulation - Takes a drawing and returns an interpolation object that can be used to produce tiles using triangulation. As with kriging, all geoms are currently converted to coordinates. This is going to be changed,: constrained triangulation will be used for lines and areas will be either converted to boundaries or will be ignored.
TileInterpolateTriangulationPar - a parallel variant of TileInterpolateTriangulation. Takes an additional parameter with the thread configuration.
TileInterpolate - Takes an interpolation object created by one of the above functions, a rectangle, and creates a tile with interpolated pixel values.
New service query functions have been added.
CoordConvertRect - Takes a coordinate converter object, a rect value, and the optional number of intermediate divisions, and converts the rect to the desired coordinate system. If the number of intermediate divisions is positive, the function creates a grid of intermediate coordinates covering the rect, projects all these coordinates and uses the results to compute the shape of the resulting rect more accurately (useful when the projection is curvilinear).
ValueSequenceTileXY - Takes a rect value for the image, a tile size value, and a contained / touching boolean switch, and returns a table of XY indexes for tiles either completely within or with any part within the specified rect. The returned table includes the following fields: X - X coordinate of a tile, can be negative, Y - Y coordinate of a tile, can be negative, Rect - rect of a tile (it can be computed from X, Y and the tile size, but is provided regardless for convenience).
Matching brackets in text windows using Ctrl-] matches #...# delimiters used for dates in SQL.
The list of Standard coordinate systems no longer includes generic templates for coordinate systems like Orthographic which do not cover the entire Earth and need customization. (These templates come from Manifold 8 which used them as a starting point for creating a custom coordinate system. Leaving these templates in the list in the Standard tab was inviting the user to make the mistake of using them without further customization, so they have been removed. All components set to use these templates will still continue to work / import / export as they should: the change affects only the list of displayed systems.)
The toolbar for a layout window or a map window collapses cursor modes into a single button with a dropdown menu. The icon on the button is set to the current cursor mode. The tooltip for the button displays the description of the current cursor mode.
The toolbar for a map window includes a button for locations, with a dropdown menu. See Locations
Switching the cursor mode in a map window cancels all pending editing. For example, starting to enter coordinates for an area, then switching to tracker removes all entered coordinates and clears the Record pane.
The toolbar no longer includes a button for the Help - About command.
The toolbar for a layout window no longer includes a button for the File - Print command. Ctrl-P continues to invoke this command, plus the change makes the toolbar line up with the toolbar for a map window, for smoother visual switching of toolbars. The File - Print Preview menu command has an icon.
Right-clicking a location in the Project pane allows applying the location to the active window using the new 'View in Active Window' command.
Saving a virtual layout, location, map or query as a persistent component puts the new component into the current path for the target data source in the Project pane. This handles all cases where the new component goes to data source A but the Project pane has a completely different data source B selected, etc. It also allows clicking a folder in the project pane to put the focus there, so that any locations created will go straight into that folder.
Reading a TIFF interprets 3-channel floating-point images as RGB instead of BGR, 4-channel floating-point images as RGBA instead of BGRA, and sets the default value range to [0, 1].
Exporting data to a TIFF writes additional data for the projection into the file.
Exporting data to an ECW or a JPEG2K writes image offset and scale info into the image file in addition to the projection info in the accompanying file.
Reading a KML file reads GroundOverlay data. See the Example: Import KML with GroundOverlay Image topic.
Pressing Ctrl-] in a comments, query or script window moves the cursor to the matching bracket. Symmetric brackets like '' or "" are matched within the same line. Asymmetric brackets like <>, [], () or {} are matched across multiple lines.
Pressing Ctrl-Shift-] in a comments, query or script window moves the cursor to the matching bracket and selects both brackets and all characters between them.
Right-clicking a value of a selected record in a table window allows copying the cell value to all selected records via the new Copy to Selection command (shortcut: Ctrl-F2). Copying the value tracks progress and can be canceled. Total operation time is logged.
The table window disables Copy to Selection and Delete commands for values in a field that belongs to an index used by the window as a key. (If the index used as a key is built on a single field, which is the most common case, none of these commands can succeed anyway.)
Attempting to overwrite an existing file in the Export or Save dialog displays a confirmation dialog with an option to confirm the overwrite and a default option to save data to a file with a different name that is currently unused.
The accept buttons in Export, Import, Link, Open, Save, Select Folder dialogs have been renamed to match the performed action, and not just named Open or Save.
Deleting records in a layout, map or table window displays a confirmation dialog. The default button in the confirmation dialog is set to Cancel.
Deleting components in the Project pane displays a confirmation dialog. The dialog lists the names of the first few components and the total number of remaining components. This is particularly useful when deleting folders which might have many components inside.
The Tools - Options dialog allows turning confirmations on and off. Confirmation dialogs also include an option to turn the current confirmation off ('Never show this again').
The confirmation to exit the application uses the new confirmation dialog, and can be turned off from that dialog.
The Location status bar pane has been renamed to Position.
The map window reports current scale in the new status bar Scale pane. The scale is reported as an absolute value in either 1:xxx or xxx:1 format, with the number of decimals adjusted for readability. If the coordinate system used by the window is lat/lon, the scale is approximated using the latitude of the center of the data rect.
Right-clicking the Scale status bar pane allows setting the scale of the map window to a fixed or custom value.
There is a new component type: location. Locations are analogous to views in Manifold 8, but are way more flexible. A location is defined by its center in lat/lon, WGS84, and an absolute scale. A location can be stored on any data source that supports writes to MFD_META, for example, on any database.
The File - Create - Location command creates a new location. The New Location dialog allows entering the longitude and latitude of the center, and the scale. The dialog also lists all opened map windows and allows setting the location parameters to the current center and scale of a specific window. If the active window is a map window, the dialog selects it by default.
The map window lists all locations in the data source of the map in the View - Locations submenu. Selecting a location zooms the window to it. (If the coordinate system of the map window is such that the center of the location cannot be converted to it, selecting the location does nothing. This only happens with coordinate systems that hide hemispheres, like orthographic.)
The View - Locations submenu for a map window includes the Windows submenu which lists all other opened map windows. Selecting a window zooms to its current location.
The View - Locations submenu for a map window includes the Save Current Location command, which saves the current location into the data source of the map.
Opening a location by double-clicking it in the Project pane displays its definition as text (JSON) and allows editing it.
Locations can be copied and pasted.
Locations can be exported to MML or TXT.
Locations can be created, altered and dropped using a query.
The Project pane allows filtering the list of components to display only locations.
The Transform Options dialog that edits transfer rules for a transform now uses Manifold's grid control. Transfer rules are changed within the grid without extra controls at the bottom.
Transfer rules like average or count require numeric fields. Transfer rules for binary fields are limited to copy. The names of transfer rules are localized.
New transfer rules for boolean fields: boolean and, boolean or, and boolean xor.
New transfer rules for numeric integer fields: 'bit and', 'bit or', and 'bit xor'.
New transfer rule for text fields: 'join tokens'. Tokens are separated by a comma. To use a different separator, use Edit Query.
New transfer rules for geometry fields: 'center', 'convex hull', 'merge areas', 'merge lines', 'merge points', 'union areas', and 'union rects'.
The New Map dialog displays potential layers using Manifold's grid control. Layers from child data sources display the full path to the data source (instead of displaying the component name and the data source icon). If the dialog is started without any potential layers, the grid is hidden.
Starting the New Map dialog by right-clicking an existing map populates the list of layers from the map, preserving Z order.
If started on an existing map, the New Map dialog sets the initial coordinate system for the new map to that of the context map. If started on a single potential layer, the New Map dialog sets the initial coordinate system for the new map to that of the context layer. Right-clicking a layer in the list of potential layers allows choosing 'Use Coordinate System,' to set the coordinate system of the new map to that of the layer.
Map windows report the current mouse cursor location in a specialized pane in the status bar. Right-clicking the pane allows switching the report format. Available report formats are: 'Latitude / longitude' (the default), 'Latitude / longitude (deg-min)', 'Latitude / longitude (deg-min-sec)', 'Pixels / stored' (reports pixel coordinates for an image and unmodified coordinate values exactly as they are stored for a drawing), 'Projected' (reports coordinates in the units used by the coordinate system, applying local scales and offsets). The report format persists between sessions. Previous controls for the status bar in the Options dialog have been removed.
Reporting the current location in a map behaves much more smoothly, with immediate reaction to cursor movement.
Right-clicking the prompt area in the status bar allows turning individual status bar panes on and off. The Location pane reporting cursor location is the only pane currently available; more panes will appear in the future. The set up of the status bar persists between sessions.
The table window displays tooltips for long or multiline values.
The table window displays tooltips for field headers, listing the name and type of a field and the expression for a computed field.
Grid controls in all panes and dialogs (Layers pane, Record pane, Properties dialog, favorites dialogs, etc.) display tooltips for long or multiline values.
Typing the first few letters of a component name in the Project pane selects the component.
Typing the first few letters of an item in a query builder list selects it. In the list of tables, the initial '[' can be omitted (typing 'pro' will successfully select '[Provinces]').
New tracker tool for map windows.. The tracker reports the total length of the specified line and the bearing of the last line segment, both measured over the ellipsoid (computed using Vincenty's formulae). The tracker supports branches, switching insert mode on and off, and snaps.
'
Internal engineering build, not published as a Cutting Edge build for download.
Merge dialogs display an icon near each component indicating how component coordinate systems will be managed during the merge. A red icon means that the coordinate system of a component is incomplete and will be treated as the default pseudo-Mercator. A black icon means that component data will be reprojected and that the reprojection will be curvilinear and lossy. A gray icon means that component data will be shifted, scaled or have axes swapped, resulting in a high accuracy transformation. No icon for vector data means that component data will not be transformed in any way. No icon for raster data means that component data might be at most shifted with the original pixel values kept intact.
The merge dialog for images reports the pixel scale and the number of pixels for the resulting image.
The merge dialog for images by default un-checks layers for big images on web data sources like Bing.
Merge dialogs by default set the coordinate system of the resulting component to the map window's coordinate system.
There is native support for INP files (EPANET).
Writing a coordinate system to PRJ automatically cleans up new lines and similar characters in the names of coordinate systems.
Writing a coordinate system to PRJ better matches the names of datums and spheroids to their parameters. The list of known datums and spheroids for PRJ has been extended as well.
Reading geometry collection values automatically merges individual values of the same underlying type used in Manifold geometry (area, line or point - differences between subtypes like line and multiline are ignored) and returns the result of the merge, instead of returning the first non-empty individual value. The change applies to all formats which support geometry collection values, including WKB, GeoJSON, native geometry in database-specific formats, etc.
Reading geometry collection values with individual values of mixed underlying types automatically converts areas to lines and lines to points in order to return all coordinates. Example: reading a geometry collection with an area and several points will return a multipoint containing all coordinates of all individual values.
Exporting an image to BMP, ECW, JPEG, JPEG2K, PNG or TGA exports rendered pixel values according to the formatting set in the Style pane. Exporting an image to TIFF always exports original pixel values. (Previously, exports were writing the original pixel values if the file format supported the pixel type and rendered pixel values otherwise.)
The ODBC driver allows enclosing identifiers in double quotes, to work around a limitation in newer versions of QGIS.
There are three new commands to merge data in existing components of different types. All commands work on layers in an existing map. All commands support merging layers from different data sources.
The Edit - Merge - Merge Drawings command allows merging one or more drawings into a single component. The new drawing and the accompanying table are created in the data source of the map. The coordinate system of the new drawing is initially set to the coordinate system of the first layer and can be edited. Right-clicking a layer and selecting 'Use Coordinate System' sets the coordinate system of the new drawing to that of the layer.
If the data source for the new drawing is a MAP file, the drawing table includes the system MFD_ID field and MFD_ID_X index. The values of the MFD_ID field in the source drawings are ignored.
The 'Copy all fields' option controls whether to copy fields other than geometry (on for drawings by default). Fields of the same name and type from different layers are merged together. Fields of the same name but different type are still merged together as long as field types are compatible, with the type of the resulting field being made big enough to hold all values (eg, if field X in drawing A is INT16 and field X in drawing B is INT32, the system will create a single field X in the merged drawing and set its type to INT32).
The 'Save source component for each record' option allows saving the name of source layer into a separate field (on for drawings by default).
The 'Skip records with null geometry' option allows skipping records with null geometry values (off for drawings by default). Null geometry values might occur in the original data or they might be produced when projecting original data to the coordinate system of the new drawing.
The Edit - Merge - Merge Labels command allows merging one or more labels into a single component. The options are the same as for drawings, but 'Copy all fields' is off by default and 'Skip records with null geometry' is replaced with 'Skip records with null geometry or text' and is on by default.
The Edit - Merge - Merge Images command allows merging one or more images into a single component. The new image uses the exact pixel scale specified in the coordinate system (shown in coordinate system metrics readout in the coordinate system dialog). If source images use different pixel types, the new image uses the type big enough to hold all values. In particular, if source images use different number of channels, the new image uses the maximum number of channels. (We are going to better indicate such cases in the dialog, because they are frequently an error.)
Merging data from multiple images follows the Z order of layers specified in the dialog, with pixels from upper layers overwriting pixels from lower layers.
Merging data from images includes special provisions for cases where the coordinate system of the original image coincides with that of the new image up to pixel scale and only differs in offsets. In such cases, pixels in the original image are not reprojected and keep their exact values.
Merging data from images uses multiple threads if this is beneficial. (Currently, we are only using a single thread only if all source images are copying their pixels without reprojection. Whenever there is scaling or any other operations, we are using multiple threads.)
Merging images into a MAP file generates intermediate levels for the new image at the end of the merge.
Merging components of all types reports progress and allows canceling.
Geometry values with mixed 2d and 3d coordinates in GML, GeoJSON, TopoJSON are automatically converted to 3d with 2d coordinates padded with zeros. (Previously, such geometry values were rejected.)
Editing a custom coordinate system or a custom base coordinate system saves and restores parameter values when switching system or transform type.
False easting and false northing parameter values are displayed in coordinate system units. Switching units of a custom coordinate system scales false easting and false northing values to new units.
The coordinate system metrics picker supports a new 'Use Default Metrics' command which resets metrics to the default values for the coordinate system. If coordinate system metrics are already set to the default values, the command is displayed with a check icon.
Edit controls for coordinate system parameters include a button with a dropdown menu and support the new 'Use Default Value' command which resets the parameter to the default value. If the parameter is already set to the default value, the command is displayed with a check icon.
Coordinate system parameter edits support a new 'Unit Converter' command which allows converting the parameter value between units using a dialog.
Editing a coordinate system or a base coordinate system displays coordinate systems in the new grid control. This solves a couple of minor UI issues. For example, SRID coordinate systems marked as deprecated now correctly display warning icons without requiring that the system is selected first.
Composing the definition of a custom coordinate system skips parameters set to trivial default values, such as 1 or 0.
Coordinate system parameters for rotation factors are displayed in degrees and support the conversion of units.
Editing a custom coordinate system or a custom base coordinate system detects invalid parameter values and suggests correcting them before the dialog is closed.
Exporting vector data sets local scales to 1 coordinate system unit instead of to 1 meter or degree.
The Window - Windows dialog uses the new grid control and a toolbar. Selecting one or more windows in the list and pressing Delete closes them.
Right-clicking a layer tab in a map window and selecting the new 'Zoom to Selection' command zooms the window to the selection in the specified layer. If the layer contains no selected data the command does nothing.
Attempting to center or zoom a map layer built on a query automatically computes the extents of the layer and remembers them for future use in zoom commands, until the layer is refreshed or the window is closed.
Ctrl-clicking a layer tab in a map window zooms to the layer. Ctrl-Shift-clicking a layer tab zooms to the selection in the layer.
Comments, query and script windows support new Edit - Advanced - Make Lower Case / Make Upper Case commands. Both commands operate on the current selection. Shortcuts: Ctrl-U, Ctrl-Shift-U.
Comments, query and script windows support new Edit - Advanced - Increase Line Indent / Decrease Line Indent commands. If the window contains no selection, the commands are applied to the current line. If the window contains a selection, it is extended to cover complete lines. Shortcuts: Ctrl-T, Ctrl-Shift-T.
Query and script windows support new Edit - Advanced - Comment Lines / Uncomment Lines commands. If the window contains no selection, the commands are applied to the current line. If the window contains a selection, it is extended to cover complete lines. Shortcuts: Ctrl-K, Ctrl-Shift-K.
Reading a GXF file recognizes more variants of coordinate system data.
GeoJSON dataports (both file and web versions) support reading TopoJSON. Geometry collections are expanded automatically.
Build 166.4 updates all third-party components used within Manifold. The most important components:
Visual Studio: Manifold is now built using a major new version of the compiler. The new compiler includes new types of optimizations; although none of them have a major effect, they do help every now and then throughout the code. The new compiler also allows built-in mitigations against Spectre attacks. This build does not have this turned on, but future builds likely will. There is a new recommended CRT (Visual C Runtime, also referred to as the Visual C Redistributable): https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads The previous CRT should continue to work with this build, but the installer packages for the next public build will include the new CRT.
ECW SDK / MRSID SDK / FileGDB SDK: All three new SDK versions used include bug fixes from their providers.
GPGPU: Pre-built GPGPU modules are compiled using CUDA 9.1 which has several minor code generation optimizations compared to the previous version.
Editing a coordinate system automatically selects the starting system in the lists of known systems and puts the parameters of the system into the controls in the Custom tab. If the starting system is not found in any list, the dialog switches to the Custom tab.
Editing a base coordinate system automatically selects the starting system in the lists of known systems and puts the parameters of the system into the controls in the Custom tab. If the starting system is not found in any list, the dialog switches to the Custom tab.
Settings for local scales, local offsets and units of a coordinate system have been separated from other parameters into the new Metrics readout at the bottom of the dialog. The readout reports one value for the local scales if they coincide and two values if they don't. Values for the local offsets are reported unless they are zero. All values are reported in the units of the coordinate system. Clicking a button to the right of the Metrics readout shows a context menu with an Edit Metrics command, a dialog for editing Metrics values.
Switching between coordinate systems, in the same tab or in different tabs, in the coordinate system dialog keeps or replaces metrics depending on whether they coincide with the default metrics of the coordinate system, or if they contain custom values. Custom values might be read from a world file or entered manually, for example. If there are no custom values, selecting a new coordinate system will replace metrics with the values for that coordinate system. Those are usually based on a unit of 1 meter or 1 degree, but some national coordinate systems use values like 0.9 meter. If there are any custom values, metrics are kept unchanged.
Metrics for EPSG and SRID coordinate systems are applied as overrides. This allows keeping the coordinate system definition as a code for readability and for easier conversion.
The Coordinate System Metrics dialog allows using a custom unit scale. This is desired in cases where the definition of a coordinate system uses a slightly different scale for a foot or some other standard unit than is used in 9. If a coordinate system uses a custom unit scale, the metrics readout displays '*' after the name of the unit.
Editing a coordinate system or a base coordinate system moves the focus to the list of systems in the initial tab, unless the initial tab is Custom.
Deprecated 'Graphics (XXX DPI, YYY)' variants of Orthographic used by Release 8 have been removed from the list of standard coordinate systems.
The GPKG dataport returns schemas for views: that allows using views as a base for drawing or for images.
The SQLITE / GPKG dataports keep binary values in non-binary fields as binary. This is specific to SQLITE, which allows values of different underlying types within the same field.
Creating a data source for a DWG or DXF file allows merging layers using an option in the New Data Source dialog. When merging layers, the dataport creates a single table for all vector data and adds a field for the layer name.
Creating a data source for a KML file allows merging layers using an option in the New Data Source dialog. When merging layers, the dataport creates a single table for all vector data and adds a field for the layer path.
Invoking Help - Documentation opens the user manual instead of the page that contains a link to it.
Pressing F1 in a comment or script window opens the keyboard reference for that window.
Queries representing database views report their schemas. The Project pane allows viewing the schema of such a query by right-clicking it and selecting the Schema command.
Queries for the Manifold query engine report their schemas as long as the schema of the result table can be discovered without running the query. (Example 1: SELECT * FROM t WHERE ...; - the schema of the result table can be discovered without running the query. Example 2: CREATE TABLE t (...); SELECT * FROM t; - the schema of the result table cannot be discovered without running the query.)
The New Drawing dialog allows creating a new component based on a query, as long as the query can report its schema.
The New Image dialog allows creating a new component based on a query, as long as the query can report its schema.
The New Labels dialog allows creating a new component based on a drawing based on a query.
The New Drawing and New Image dialogs disallow adding spatial indexes to tables which do not support altering table schema (eg, read-only).
The New Drawing and New Image dialogs no longer list system tables (MFD_xxx).
Creating a new component on a data source which adjusts names of created components (commonly seen with databases which prepend a schema name) captures the final created name and correctly selects the created component in the Project pane.
The DB2 dataport reports schemas for materialized views.
The Oracle dataport treats materialized views as queries, not as tables.
The PostgreSQL dataport allows accessing materialized views and reports their schemas.
The SQL Server dataport tries to resolve the SRID of geometry values in a table to the system MFD_SRID table (composed based on a SQL Server view). If the SRID cannot be resolved, it is interpreted as an EPSG code, according to Microsoft guidelines and similar to how these codes are used in practice by Manifold 8 and other products.
Dataports for SQL Server and other databases map unrecognized field types to NVARCHAR. This allows working with partial data in tables that contain fields which the database driver cannot work with - usually due to a bug, a misconfiguration, or version skew.
Exporting vector data to DXF, GDB, MFD, MIF files (in addition to E00 and SHP which were done earlier) clears the values of the local offset and local shift values in the exported coordinate system and forces axis order to XY.
Linking an isolated GCDB DAT file for a single state no longer creates an 'API' comments component with a list of supported geocoding functions (because geocoding functions require linking the full GCDB data set, not a single file).
Sorting records in a table window requires Ctrl-clicking the field header instead of just clicking. This avoids inadvertently sorting columns when adjusting column widths. Shift-clicking the field header continues to add the field to the current sort order.
Rendering a label bound to an area tries to places the label into the middle of the visible portion of the area. Labels automatically reposition as the display is panned, to keep as many labels in view for partial areas as possible.
The advanced rendering engine supports rendering labels with shadows. The basic rendering engine scales label shadows during printing (were frequently invisible before).
The advanced rendering engine supports rendering labels with halos. The basic rendering engine scales label halos during printing (were frequently invisible before).
The advanced rendering engine supports strikeout and underline font options.
(Fix) This is a hotfix build for an issue introduced in 9.0.166.1: Reading data from a database like SQL Server or PostgreSQL could fail with an 'Unknown error' message. The circumstances were very specific and did not occur normally. For example, one of the requirements was fetching zero fields and using the request to probe for data. However, several UI tools could step on the error so the issue has been immediately fixed.
UI tools composing query text avoid escaping characters in names and string literals that do not need to be escaped.
The context menu in the Project pane has been extended to include Copy and Paste commands. Commands creating new components have been moved to the Create submenu.
Importing data inspects all created components and tries to select the main imported component in the Project pane (instead of the first one). The main imported component is determined using component type, in order: maps > map layers (drawings / images / labels) > tables > all other components.
The default file mask suggested for image libraries uses new {...} escape sequences for X and Y instead of legacy ones.
The Help menu includes a Documentation command which opens the documentation web page. The Online Portal command in the same menu has been renamed to Web Site.
Opening a MAP file created by Manifold 8 migrates images linked from image servers. Discontinued OSM servers are mapped to OSM base. Yahoo servers are mapped to HERE (the successor). Yandex servers are mapped to Yandex V2. Unrecognized custom servers are mapped to Bing.
Views on PostgreSQL and other databases allow specifying a field to use as identity. Doing this allows records in the view table to be selected, and, if the view supports that, edited or deleted. To specify a field to use as identity, right-click the field header and select 'Use as Identity'. The system will scan all values in the field to make sure there are no duplicates or NULLs and if the values are clean, the table will expose a BTREE index on the field. The information about the field used as identity is saved into the database, so attempting to use the table in future sessions of 9 will automatically expose a BTREE index on that field without further scans. The identity option can be cleared or moved to a different field. The view can have only one identity field specified in this way.
Using a view with the identity field specified via the 'Use as Identity' command in the table window monitors reads via the index. If the system detects that some of the values in the field specified to be identity are duplicates or NULLs, it automatically removes the identity option from the field and reports the removal in the log window.
Dataports for PostgreSQL and other databases detect writable views and allow writes to them.
Pressing F1 in a component window opens a help topic with the keyboard quick reference for that window. Pressing F1 in a command window chooses between the help topic for the query portion of the window and the help topic for the table portion of the window depending on the current focus.
The CREATE query statement puts the final inserted name of the created component into 'createdname' and 'creatednamequoted' pragma values. The 'createdname' value contains the plain name and the 'creatednamequoted' value contains the name in the '[...]' form for use in properties.
Dialogs creating new components link components together using their final inserted names to avoid creating invalid links on databases which prepend schema names, etc.
Reading DWG and DXF files creates a field for the block name. If an object belongs to a sequence of nested blocks, the system uses the name of the top-level block.
Reading DWG files hides map layers that are marked as turned off.
Exporting a drawing to SHP or E00 clears the values of the local offset and local shift values in the exported coordinate system and forces the axis order to XY. (We are working to extend this behavior to all vector exports and allow controlling it via an option.)
Captures all 9.0.165.x additions.
Decomposing areas for contours and traces includes optimizations for large geoms. Performance improvements depend on the data and are typically significant: the decompose phase is several times faster and sometimes several tens of times faster).
Building contour lines on huge rasters automatically splits created lines after 64 million coordinates if decomposing is allowed.
Reading a DXF file reads HATCH objects as areas and avoids creating separate objects for area boundaries.
Composing contours and tracing areas removes redundant coordinates on horizontal and vertical edges.
The TileContourLinesXxx, TileContourAreasXxx, TileTraceAreasXxx query functions include an additional decompose parameter to specify whether the resulting geometry has to be decomposed into shapes.
Contour Lines, Contour Areas, Trace Areas templates in the Transform pane set the decompose parameter on by default.
Decomposing geometry as part of building contours / traces uses a specialized algorithm which performs much faster than the regular algorithm used in the Decompose to Shapes transform. The difference is huge (15x+ and even higher after numerous optimizations). Decomposing geometry as part of building contours / traces helps avoid producing geometry which is too large to fit into 2 GB (~120 million coordinates), the limit for the size of a single geometry value.
The Similarity parameter for the Trace Areas template in the Transform pane has been set to 1 by default. 1 produces the same result as 0 on integer rasters, and using 0 on floating-point rasters is more often than not a user error.
Composing contours and tracing areas reports detailed progress and allows canceling operation in all phases, including decompose.
The image library dataport allows using escape sequences for X, Y, etc, in names of intermediate folders in addition to the filename. If the file pattern includes escape sequences in the names of intermediate folders, the value of the Recurse into subfolders option is ignored.
Composing contour areas or lines includes optimizations for empty or missing tiles.
Composing contour areas or lines includes optimizations for tiles with a small height range.
The performance increases from the above two optimizations depend on the data, but (a) there is nearly always an increase, and (b) usually, the increase is significant.
Composing contour areas or lines and tracing areas reports progress.
Previews for Geoms Adjacent, Geoms Contained, Geoms Containing, Geoms Intersecting, Geoms Touching, Normalized Geoms templates in the Select and Transform panes are more accurate.
The Select pane includes an 'Allow parallel execution' option (on by default) used for overlay templates.
The Select and Transform panes stop the preview prior to performing the operation. This avoids wasting resources if the preview is still running when the operation is commanded.
Connecting to a WMS server assumes that servers using old versions of the WMS standard (pre-1.1.1) ignore axes order for the coordinate system and always use XY.
Reading a DWG or DXF file better handles block inserts and performs faster.
Exporting a drawing to a DXF file exports areas as hatches instead of exporting area boundaries as lines.
There is support for contouring.
New query functions:
TileContourLine - Takes a 1-channel image and height, and returns the contour line for the specified height. If the specified height is lower or higher than any of the image pixels, the function returns a NULL value.
TileContourLines - Takes a 1-channel image and a set of heights specified as a table, and returns a table of contour lines with fields for: Geom, Value (height). The set of heights does not have to be sorted and may contain duplicates (ignored). The returned table contains a record for each unique height. Some of the geoms in the returned table might be NULL values.
TileContourArea - Takes a 1-channel image, minimum height, maximum height and returns the contour area for the specified height range. If none of the image pixels is in the specified height range, the function returns a NULL value.
TileContourAreas - Takes a 1-channel image and a set of heights specified as a table, and returns a table of contour areas with fields for: Geom, ValueMin, ValueMax. The set of heights does not have to be sorted and may contain duplicates (ignored). For N unique heights, the returned table contains N+1 records: area below the smallest height, N-1 areas between subsequent heights and area above the biggest height. Some of the geoms in the returned table might be NULL values. The returned areas cover the entire image.
TileContour...Par - parallel variants for each of the above functions.
New Transform templates:
Contour Lines - allows specifying minimum height, maximum height, height step, and produces contour lines.
Contour Areas - allows specifying minimum height, maximum height, height step, and produces contour areas.
(Note: we will allow setting the height range automatically using a Full Range button. For now, you can quickly determine the height range by switching to the Style pane, setting the style to RGBA channels, selecting the desired channel, then right-clicking it and selecting Full Range. You don't have to apply the style after.)
There is support for tracing (converting raster to vector).
New query functions:
TileTraceArea - Takes an image, a value to trace and a quantization factor, and returns the area covering pixels with the specified value. The quantization factor is used to extend the area to pixels with values close to the traced value. If the quantization factor is negative or zero, any differences in pixel values are considered significant and the returned area covers pixels with values exactly coinciding with the traced value. If the quantization factor is positive, pixel values are quantized using the following formula: pixelNew = floor(pixel / quant) * quant, pixels that quantize to the same value are considered to be the same and the returned area covers them all. (Example: with quant size of 100, pixel values between 0 and 99.9... are considered to be the same, pixel values between 100 and 199.9... are considered to be the same, etc.) If the image contains more than one channel, the trace value has to be an xN value of the appropriate size. The quantization factor remains a single numeric value that is applied to all channels simultaneously.
TileTraceAreas / TileTraceAreasX2 / TileTraceAreasX3 / TileTraceAreasX4 - Takes an image and a quantization factor, and returns a table of areas covering pixels with different values with fields for: Geom, Value. TileTraceAreas works with 1-channel images and returns traced values as FLOAT64. TileTraceAreasX2 works with 2-channel images and returns traced valueas as FLOAT64X2, etc.
(Note: the functions track the number of different pixel values they encounter and stop if that number gets too big - currently, the limit is 20,000. Having too many different pixel values usually means that the image has to be preprocessed or that the function has to be called with a higher value of the quantization factor to make the results usable.)
TileTrace...Par - parallel variants for each of the above functions.
New Transform templates:
Trace Areas - allows specifying quantization factor and produces areas covering pixels with different values.
Things we are going to improve or add:
Currently, both contouring and tracing ignore the 'Restrict to Selection' option. We will extend the transforms to use that option.
Both contouring and tracing create huge geoms with lots of branches. A common post-processing step is decomposing these huge geoms into smaller objects. While this is perfectly doable using transforms that we already have, there are optimizations that we can use specifically for data produced by contouring and tracing, so we might add an option to decompose data directly into contouring and tracing functions and transforms.
There are new query functions for spatial overlays that can be used as filters for the primary data set:
GeomOverlayAdjacentFilter takes a source drawing and an overlay drawing, finds all objects in the source drawing that are adjacent to any object in the overlay drawing, and filters the source drawing leaving only objects which are adjacent to at least one object in the overlay drawing. Unlike the result table of GeomOverlayAdjacent, the result table of GeomOverlayAdjacentFilter contains all fields from the source drawing and no fields from the overlay drawing. GeomOverlayAdjacentFilter also performs faster than GeomOverlayAdjacent.
GeomOverlayAdjacentFilterPar computes the same data as GeomOverlayAdjacentFilter but allows using more than one thread.
GeomOverlayContainedFilter / Par, GeomOverlayContainingFilter / Par, GeomOverlayIntersectingFilter / Par, GeomOverlayTouchingFilter / Par are analogous to GeomOverlayAdjacentFilter / Par, but use different spatial criteria.
The result tables returned by GeomOverlayXxxFilter / Par functions contain all indexes from the source table and are writable. It is possible to chain calls to these functions as well as calls to SelectionXxx functions. For example, the following query returns all states touching selected parcels: EXECUTE CALL GeomOverlayTouchingFilter(states, CALL Selection(parcels, TRUE), 0).
The result tables returned by GeomOverlayXxxFilter / Par and SelectionXxx query functions perform faster when used from multiple threads.
New templates in the Select pane: Geoms Adjacent, Geoms Contained, Geoms Containing, Geoms Intersecting, Geoms Touching. Each template selects data in a drawing using an overlay drawing and the specified criteria. Note that the meaning of Geoms Contained / Containing is reversed compared to transforms for Overlay Contained / Containing due to the wording.
New templates in the Transform pane: Geoms Adjacent, Geoms Contained, Geoms Containing, Geoms Intersecting, Geoms Touching. The templates are similar to those in the Select pane and add the objects that would have been selected as a new component, controlling transferred fields.
Queries generated by the Select pane for map layers refer to layer components instead of to their tables.
Queries generated by the Edit Query button in the Select pane ignore the select mode specified in the pane. The purpose of the query generated by the Edit Query in the Select pane is to show what the system does to apply a specific template and to produce a reasonable starting point for customizing the query further. The code for combining the output of the template detracts from that. It usually is better in any case to combine multiple selections differently, through saved selections.
The map window displays a preview for Geoms Adjacent, Geoms Contained, Geoms Containing, Geoms Intersecting, Geoms Touching templates from both the Select and Transform panes. For the transforms, objects that participate in the transform and have matches in the overlay drawing are shown using the regular blue preview color, and objects that participate in the transform but do not have any matches in the overlay drawing are shown using a lighter shade of the blue preview color. Currently, the accuracy of the preview for Geoms Adjacent and, rarer, for other new templates might be a bit off in that some of the matches might not be picked up and vice versa. This happens because previews operate on simplified versions of the objects. Will be fixed in upcoming builds.
Web dataports prioritize fetching tiles closer to the center of the requested area whenever possible. This produces a usable picture noticeably faster than fetching tiles in XY or YX order.
Creating a new data source for CSV allows forcing all fields to text, specifying whether the first line contains field names, specifying characters used for list delimiter, decimal separator and text qualifier. The defaults for all options are to autodetect their values from file content. It is possible to keep some options on autodetect and set other options to explicit values. If option values conflict, the dataport ignores the value for the less important option. For example, if the list separator is set to autodetect and scanning the file discovers that list separator is a comma, the dataport disallows using a comma for the decimal separator character.
Entering coordinates for new drawing objects or labels in a map window no longer requires the selected tab in the Record pane to be Coordinates. Changing coordinates for existing objects still requires the selected tab to be Coordinates.
Holding Shift while Alt-clicking a drawing object or a label in a map window sets the selected tab in the Record pane to Coordinates instead of Values.
The Record pane for a label edits label text as a regular field value in the Values tab instead of using a separate text box in the Style tab. The record header for the field displays a distinctive icon.
Starting a new label or starting to edit an existing label automatically selects the value of the text field in the Values tab in the Record pane.
(Fix) Hiding a key field used for the selection in a table window no longer makes the table window unable to select or unselect records and unable to highlight selected records.
(Fix) Editing a cell value in a table window and ending editing by clicking a different cell no longer applies the editing to the clicked cell instead of the one that was being edited.
Migrating a labels component bound to a drawing from a MAP file created by Manifold 8 avoids creating extra computed fields to hold label text if the labels component was using a single field with no other text.
Migrating components from a MAP file created by Manifold 8 re-creates the folder structure.
Migrating components from a MAP file created by Manifold 8 migrates themes. The theme hierarchy is flattened with each theme converted into a drawing with formatting changes throughout the hierarchy captured.
Format pickers in the Style tab of the Record pane change the cursor to a 'hand' when the values they are displaying can be changed.
Pasting records between tables allows mapping fields using the new Paste dialog. By default, fields are mapped using their names and types. The default mapping for any field can be changed by editing it in the list (double-click or press F2).
The Paste dialog allows resetting changed fields back to the default mappings using the Autoselect Field command in the drop-down menu.
Pasting records into a table avoids displaying the Paste dialog if the system thinks it succeeded at mapping all fields possible. (The criteria: if the target table contains no fields that receive no data from the source table and can be set to receive data from one of the fields in the source table that has not yet been used, the system declares success.) Holding Shift while invoking the paste operation (Ctrl-Shift-V) forces the dialog to appear regardless of how successful the system thinks it has been at mapping fields.
Pasting records into a table in a MAP file avoids pasting data into the MFD_ID field by default. The user can still paste data into MFD_ID by selecting the source field for it.
Pasting records into a table stops at the first record that fails to paste instead of trying to paste as many records as possible. (The latter was a temporary measure used because we had no control over the paste operation.)
Pasting records into a table requires at least one matching field. If there are no matching fields, no records are pasted. (Previously the paste operation would create records with NULL values.)
The map window supports copying and pasting data for drawings and labels. Pasting records between drawings and labels automatically maps fields for geometry and / or text to each other.
Pasting records into a table / drawing / labels automatically converts coordinate systems for geometry fields. The conversion is performed only if both the target and source fields have a valid coordinate system (displayed in black instead of red in the coordinate system pickers, eg, in the Component pane).
Copying and pasting records reports the number of processed records in addition to the operation time.
The dataport for Enterprise storages supports themes.
The dataport for Enterprise storages supports folders. In addition, all virtual components exposed by an Enterprise storage are put into a special folder.
(Fix) The MIF dataport and several other dataports no longer fail to apply unit scale to easting and northing.
(Fix) The KML dataport puts created labels components into folders.
The KML dataport creates a map for all layers.
The KML dataport reads description and snippet values.
Right-clicking a field item in the list of tables in the query builder allows using the new 'Insert Table and Name' command to insert '[table].[field]' into the query text.
The dropdown list of rotation angles includes an item for 360 degrees, to allow setting up styles with min=0 and max=360 rotations.
(Fix) Editing the last shown record in a table window no longer sometimes fails to update values shown for that record after sending changes to the table. (Most commonly seen on tables with a single record.)
(Fix) Resizing the Record pane with the Style tab active no longer sometimes leaves screen garbage.
The memory readout in the About dialog is reworded to 'Using XXX MB RAM (XXX MB max cache) ...' for clarity and has been moved to the bottom of the dialog.
The query engine allows specifying string literals without escapes using the @'...'. The @'...' string literal cannot contain embedded ' characters (to include them, use an expression: @'...' & '\'' & @'...'). The unescaped form is useful for entering regular expressions among other things.
(Fix) The SelectionKeys and SelectionKeysWindow query functions no longer fail to return a table (with no records) if there are no selection keys because the records are either all selected or all unselected.
Alt-clicking an object in a drawing stored in a MAP file prioritizes points to lines to areas.
The Select pane supports filtering templates.
The Transform pane supports filtering templates.
The Style pane allows specifying area size. The Record pane shows area size for areas and allows editing it if the drawing is using style overrides.
Migrating drawings from MAP files created by Manifold 8 translates formatting data for area sizes.
Migrating labels not bound to drawings from MAP files created by Manifold 8 adds the default MFD_ID field and MFD_ID_X index to the created table.
Migrating surfaces from MAP files created by Manifold 8 translates shading options.
The dataport for Enterprise storages supports migrating formatting data for drawings and labels (encoded differently than in MAP files created by 8).
The dataport for Enterprise storages migrates lightweight components like maps dynamically and ignores them when scanning for changes to storage (previously, changing a multiple user map in Manifold 8 was making the dataport flag the entire storage as needing re-migration in 9).
(Fix) The image server dataport no longer uses incorrect URLs for OSM servers.
There is a new dataport for MVT files (vector tiles - an intermediate format used by web servers).
The file filter for the 000 dataport includes '(S-57)'. The file filter for the NTF dataport includes '(OSGB)'.
Right-clicking the caption of the active subpane in the Contents pane brings up the drop-down menu, similarly to left-click.
The File - Print command used by layouts is bound to Ctrl-P.
The advanced rendering engine supports area patterns. Area patterns are automatically scaled during printing (no more invisibly thin lines). The elements of area patterns are rendered with antialiasing. (More coming in 9.0.165.x.)
(summary)
The new File - Print Preview command creates a virtual layout component for the current component that can be adjusted as needed and printed. The Edit - Save as Layout command allows saving virtual layout as a regular, persistent component for future use.
The map window allows inserting labels using visual tools similar to those used for drawings.
The map window allows selecting labels using visual tools and using Edit - Select All / Inverse / None commands.
Alt-clicking an existing label in a map puts it into the Record pane for editing. The Record pane allows easy access to label text as well as formatting.
The map window allows using the Select and Transform panes with labels and previews the results of their actions. The Transform pane automatically selects the field containing label text.
The table window uses the Layers pane to show, hide, reorder and resize fields. Same for command windows and for windows showing tables produced by queries. Last used field layout for a persistent table component is automatically saved in the table properties.
Right-clicking in the map window no longer commits changes to the currently edited record and instead displays a context menu with the choice between Cancel Changes and Commit Changes.
The Style pane allows formatting images. Formatting images uses faster and more precise statistics. There are protections from downloading too much data when computing statistics for web images. Multi-channel formatting is streamlined with means to set ranges for more than one channel at once, etc. Single-channel formatting is extended to use all classification methods available for vector data (such as 'equal count' or 'natural breaks') on pixel values.
The Style pane includes the new Options tab with controls for per-record styles (style overrides) and image shading.
There is a new dataport for GEOJSON files and a separate dataport for GEOJSON web servers. There is an export to GEOJSON which exports drawings.
There is an export to GDB which exports tables, drawings and maps. (Warning: attempting to export to an existing geodatabase will overwrite it. To add data to an existing geodatabase, link it to the project and then transfer data using copy and paste or using SQL.)
The Contents pane has been reworked to use less vertical space for pane captions. Individual panes within the Contents pane are accessed using a drop-down menu or using Ctrl-1/2/3/4/5/6 keyboard shortcuts.
Migrating labels bound to drawings from MAP files created by Manifold 8 translates expressions for label text.
Migrating drawings from MAP files created by Manifold 8 translates formatting data for colors, sizes, rotations and styles. Missing styles are mapped to the default style with the formatting rules preserved (if the format was set up to map values 1, 2, 3 to styles that are not available in 9, the format in 9 will continue mapping values 1, 2, 3 but the styles will be changed to the default style).
Migrating labels from MAP files created by Manifold 8 translated formatting data for colors, sizes, styles and font. Missing styles are mapped to the default style with the formatting rules preserved.
The TIFF dataport uses deflate compression by default. (We are planning to controls this and several other settings using a dialog.)
The system activity indicator in the status bar shows activity for all background tasks, including UI ones. The number of currently running background tasks is shown in the tooltip.
Setting the Select or Transform pane to 'no action' either stops the background task for preview or marks it as not counting for the system activity indicator, whichever is better according to the window that displays the preview.
(Fix) Selecting the item used to create a new saved selection in the Select pane no longer treats it as a valid saved selection for the Edit Query button and the preview.
The Contents pane has been reworked to display a single caption for the selected subpane at the top. Clicking the caption for the selected subpane displays a dropdown menu which allows switching to all other subpanes.
The commands switching to the subpanes in the Contents pane use Ctrl-1/2/3/4/5/6 keyboard shortcuts.
(Fix) The Record pane no longer shows the toolbar when there is no active record.
(Fix) The Layers pane no longer shows the toolbar and the layer list when there is no active window or when the component window does not support layers.
The Layers pane recognizes when the data it displays is read-only and disables controls and commands. Temporary layouts and temporary maps are always writable. Tables and queries always appear writable with changes to tables on read-only data sources being kept in the window and being discarded after the window is closed.
(Fix) The layout window allows altering temporary layouts even if the data source for the original component was read-only.
(Fix) Toggling a field in the MFD_META table on and off no longer sometimes makes field values in the toggled fields display as NULLs.
Trying to compute statistics for a big web image (more than 16 M pixels) for the Style pane computes statistics on a small top-level view of the image.
The Select pane supports selecting data in labels. The map window shows preview for the selection.
The Transform pane supports transforming data in labels. The map window shows preview for the transform.
The Transform pane automatically selects the field that contains label text when switched to a labels component.
The Layers pane lists fields for tables, including result tables of queries with or without backing query components. The fields can be reordered and toggled on and off. If all fields are toggled off, the pane shows the first field in the table schema (which presumably contains some form of record ID) to avoid having records with no fields shown. Recomputing a query shows all fields in the new table in order they appear in the schema.
The Layers pane displays field sizes for tables and queries and allows editing them. Field sizes are expressed in points. Setting the size of a field to zero resets it to the default size for the field type.
Reordering fields of a table component or hiding fields of a table component on and off automatically saves field order and field states to table properties. If table properties are read-only, changes are still allowed but are not saved and are lost after the window is closed. Refreshing the table using View - Refresh re-reads field order and field states from table properties.
The SDTS dataport composes style data for images.
The SDTS dataport stores data for multi-channel images in a single table. (Previously, the dataport was creating a separate table for each image.)
The Oracle dataport composes style data for Oracle georasters.
(Fix) The GPKG dataport no longer sometimes fails to connect to the database in read-only mode (for example, from Viewer).
The OSM and PBF dataports expose record versions when they are available.
Exporting a drawing to GDB puts the resulting data into the feature dataset.
Viewer can open MXB files. If the MXB file being opened has not been uncompressed into a persistent MAP file (eg, by 9), Viewer will uncompress it into a temporary MAP file. Since Viewer cannot save MAP files, closing the file will throw away all uncompressed data and attempting to open the MXB file later on will have to uncompress it again.
The name for the new MAP file shown in the caption of the main application window is localizable and no longer contains a counter: [Project1] -> [Project2] -> ... are now (New Project) by default.
The Style pane allows specifying shading options for an image, using the Options tab. Shading is only available for images colored using a palette built on a single channel.
The old Style command and the old Style dialog are removed.
The File - Print command notifies the printer driver about the paper size, format and orientation used by the layout. If the printer driver does not support the specified paper size, the command uses the default parameters for that printer. (Similar changes to the File - Page Setup command are on the way.)
The TIFF dataport recognizes several new tags with min / max channel values for image data and uses that info to style images.
The GDAL dataport uses min / max channel values to style images, when the GDAL driver makes them available.
The GDAL dataport splits images for HSL, CMYK and YCbCr data into multiple single-channel images.
The Style pane displays channels on style items / in the field combo / in the channel list using distinctive icon.
The context menu in the channel list in the Style pane allows setting the range of values for selected channels to autocontrast of varying degrees (spreads of 1 / 1.5 / 2 / 2.5 / 3 standard deviations).
The channel list in the Style pane allows setting the channels to ARGB / BGR / BGRA / RGB / RGBA / single channel presets using a toolbar button on top of the list.
The Overrides dialog used in the Style pane is merged into the pane. The choice for a new field is converted to a button that creates a new table field.
The definitions of datums used in PRJ files use more precise values for inverse flattening.
The TIFF dataport produces style data for single-channel images with valid statistics and for four-channel images with no special channels (previously, the fourth channel would have been interpreted by the rest of the system as the alpha channel by default).
The GEOJSON web dataport caches data (entire GEOJSON value) if caching is allowed.
The Style pane supports formatting images. The pane displays a single style item for color which can be set to either of the following:
(no field) - all pixels are using the same color, selected using the drop-down menu;
(use RGBA channels) - the color for an image pixel is going to be combined from BGRA channels edited separately, each channel can be set to use either a tile channel with min/max range mapped to the 0..255 range or a specific value from the 0..255 range;
Channel X - the color for an image pixel is going to be determined using a palette, the palette is edited in the list, editing options are similar to those for thematic formatting in drawings and labels.
(A few UI tools that we had before are currently missing, we will re-add them very soon.)
Formatting images in the Style pane computes precise statistics for 8-bit and 16-bit pixel values regardless of the image size. Computing statistics tracks progress and can be canceled. Statistics computed for an image are cached in the component window that displays the image to avoid repeat computations.
The Style pane avoids computing statistics for images stored on web servers that are too large. This is useful to avoid computing statistics for Bing Maps or similar base maps.
Items in the list of image channels in the Style pane can be selected. Editing a value for a selected channel applies the edit to all selected channels.
Right-clicking a selected item in the list of image channels in the Style pane allows setting the min/max range of values to the full range of values occurring in the image.
The 'Per-Record Style' button in the Style pane is renamed to 'Overrides'. Same for the 'Per-Record Style' dialog it invokes.
There is a dataport for GEOJSON files. The GEOJSON file has to contain a FeatureCollection object at the top. We might support more subformats in the future.
There is a dataport for GEOJSON web servers. The requirements for GEOJSON data returned by the server are the same as for GEOJSON files.
GEOJSON dataport supports exporting drawings.
Parsing big GEOJSON data performs significantly faster.
GDB dataport supports exporting tables, drawings and maps (exporting a map exports all drawing layers). Exported components are renamed according to GDB rules. WARNING: attempting to export a component to an existing geodatabase will overwrite the geodatabase, similarly how exporting to a SHP file will overwrite that SHP file. To add a component to an existing geodatabase, link the geodatabase and transfer the component using copy/paste.
Export to GDB tracks progress and can be canceled.
Export to GDB optimizes bulk writes to perform faster.
GPKG dataport exports coordinate system data for drawings (converts the definition of the coordinate system to WKT, creates a new SRID value for the WKT, and assigns the created SRID value to the table field).
GPKG dataport supports exporting a single table.
(Fix) Web dataports that are using per-session cache disable writes to MFD_META table. (Before the change it was possible to, say, format components on a web dataport only to lose all formatting after closing the MAP file because changes made to formatting were stored in per-session storage that has been discarded.)
The WFS dataport uses a faster request strategy.
Copying and pasting virtual components for multiuser components on Enterprise storages removes internal bookkeeping properties.
The map window allows selecting labels using Edit - Select All / Inverse / None commands.
The map window allows selecting labels using Ctrl-click (Ctrl-click adds the clicked label to the selection, Ctrl-Shift-click removes the clicked label from the selection) or Ctrl-drag (Ctrl-drag adds the labels within rect to the selection, Ctrl-Shift-drag removes the labels within rect to the selection, keeping Ctrl pressed at the end of the rect selects labels with any part inside of rect, releasing Ctrl at the end of the rect selects labels entirely inside of rect). Both Ctrl-click and Ctrl-drag operate only on the labels that are currently visible on the screen and ignore labels hidden due to overlaps.
The map window allows deleting selected labels using Edit - Delete.
Labels components use a new icon.
Generating a unique name for a component treats ending tokens that consist of a mix of numbers and non-numbers as part of the base name. (Copying and pasting a component named 'Doc b12' creates 'Doc b12 2' instead of 'Doc b 2'.)
The Per-Record Style dialog allows creating a new field to store style data via a '(new field)' choice. The choice is only available if the table supports adding new fields. The default name for the new field is 'Style' (localizable). If per-record formatting is turned off, but the table contains a text field named 'Style' (localizable), the dialog pre-selects it.
The drop-down menu for sizes includes choices for 48 / 60 / 72 pt. This is useful for layouts.
(Fix) The layout window no longer fails to clip text frames to their extents.
The query builder lists constants supported by the query engine.
Right-clicking in the map window no longer commits pending changes to the edited record and invokes a context menu with the Cancel Changes / Commit Changes items instead. (After long talks we decided that right-clicking to commit changes is, while fast and similar to 8, ultimately too dangerous, and that it is best to make right-clicking always a safe operation that does not alter data.)
Web dataports connecting to servers with a lot of components allow accessing components that have already been discovered before the discovery process completes. (Previously, this could have resulted in errors due to component dependencies being in the process of being discovered and the user had to wait until the server returns metadata for all its components.)
(Fix) Reading data from a TAB file no longer stops at objects with no geometry.
(Fix) Reading coordinate system data for a MIF or TAB file no longer swaps center latitude and center longitude parameters for several coordinate system types.
Map windows allow inserting new labels using visual tools similar to those used for drawings. (Labels bound to areas are currently displayed near the 'first' coordinate of the area. This will be changed.) Inserting a label bound to a point creates either a single label bound to a multi-point if the label is finished using Shift-right-click or Shift-Ctrl-Enter, or multiple labels bound to each of the entered points if the label is finished using right-click or Ctrl-Enter. The coordinates and field values of the new label are displayed in the Record pane. The text of the new label is displayed on the Style tab of the Record pane.
Alt-clicking a label in a map window puts it into the Record pane. The map window highlights the coordinates for the label and allows editing them as long as the Record pane has the Coordinates tab selected (similarly to drawings).
The Record pane allows moving from the active label to the previous or next label, and zooming to the active label. (Currently, the previous and next labels can be anywhere, similarly to previous and next drawing objects, and they might be hidden because of overlaps with other labels. We are considering changing the logic to move only between labels that are on the screen because they are guaranteed to be visible.)
The Record pane displays formatting values for the active label in the Style tab. If the labels component uses per-record formatting, the Record pane allows editing formatting values.
The Record pane disables editing the field value with the text for the active label in the Values tab (use the Style tab).
Inserting a new label sets label text to the default value (localizable in UI.TXT).
The Component pane displays the geometry field and index used for labels. (We will also display the text field that has been displayed before.)
(Fix) Exporting a table to MML no longer sometimes fails (usually happens with tables with WKB geometry values from databases).
(Fix) Virtual tables for multiuser components on Enterprise storages are made read-only. (Attempts to edit always failed, but the read-only nature of the data was not reflected in the table schema and so the table window / map window assumed the components were editable.)
(Fix) Removing a virtual component for a multiuser component on an Enterprise storage cleans all relevant metadata to avoid re-creating the virtual component on next connection.
The TIFF dataport ignores subimages that store intermediate levels for the 'main' image - as long as they are marked as such.
The Style pane allows composing intervals using natural breaks. The algorithm computing natural breaks has been improved to work for arbitrary amounts of data.
Computing natural breaks optimizes breaks globally. (This produces better results than the algorithm used by Manifold 8, at negligible cost.)
The File - Page Setup and File - Print commands are only displayed for layout windows.
The map window supports the File - Print Preview command, which creates a virtual layout component. If the map window displays a virtual map, customizations made to that virtual map are ignored (save the virtual map as a regular component first).
The layout window supports the Edit - Save as Layout command, which saves the virtual layout as a regular component in the opened MAP file. If the MAP file is read-only, the command is disabled. Saving the virtual layout as a regular component selects the created component in the Project pane.
(Fix) The SQL Server dataport no longer sometimes fails to work with tables that contain indexes created by the SQL Server Spatial Extender.
(Fix) The SQL Server dataport no longer fails to capture keys for newly inserted records for tables with triggers.
Dataports for SQL Server and other databases automatically downgrade written geometry values to Manifold 8 format if the target table has been created by Manifold 8.