LiDAR technology (Light Detection and Ranging) measures terrain elevations by shining a pulsed laser from a sensing platform onto a surface and measuring reflected pulses. Manifold imports or links LiDAR data from LAS or LAZ format files as points in a drawing, using whatever coordinate system is specified in the file. Manifold's LAS dataport can also Export the drawing's table to LAS or LAZ format files. A related capability is using the librarylas dataport to create a data source that generates a drawing from an entire folder, or hierarchy of folders, containing multiple LAS or LAZ files. See the librarylas topic for details.
Another related capability is using the Tools - Index LiDAR Files command to create indices in advance for LAS or LAZ files we intend to use. When using COPC LiDAR data, there is no need to create indices in advance since Manifold will automatically use the COPC index.
Please see the Importing and Linking topic for the difference between linking a file into a Manifold project and importing data from that file into a Manifold project. See the Third Party Release Levels topic for the latest versions of LAS or LAZ files supported.
We can link a file using either File - Link or File - Create - New Data Source. If we do not need any extra options offered by the New Data Source dialog, we simply use the File - Link dialog.
Think ahead: Use
the Tools
- Index LiDAR Files command
to build .mapindexp indices for LiDAR files as soon as
they are downloaded. That will save time when linking the
files, because the indices will already be built and files will link instantly.
When using COPC
LiDAR data, there is no need to create indices in advance since Manifold
will automatically use the COPC index.
To link a LAS or LAZ format file:
Choose File-Link from the main menu.
In the Link dialog browse to the folder containing data of interest.
Ensure the Save cache box is checked (the default).
Double-click the file ending in .las or .laz for the data of interest.
A new data source will be created.
If we have not created an index in advance using the Tools - Index LiDAR Files command, the first time we open a drawing in the new data source, the opening process will be slow, while a spatial index is built (a one-time job). When using COPC LiDAR data, there is no need to create indices in advance.
Opening the data source thereafter will be fast, as long as the accessory .mapindexp file accompanies the .las or .laz file.
Linking a LAS or LAZ file exposes coordinate system shifts and scales for X, Y, and Z fields as properties in the linked drawing's table. Shifts and scales to the geometry field. X, Y, and Z fields are float64 data types and report scaled coordinate values. Exporting to LAS or LAZ uses this data to produce a file with the exact same shifts and scales as the original.
Important: The above feature was introduced in
build 173.5 and updated
in build 173.6. When
linking to LAS or LAZ files previously used in Manifold builds before
173.6, delete any .mapmeta file
in the same folder that were created by any builds before 173.6.
That will prevent the old .mapmeta file
from limiting the properties created to only those used in builds before
173.6.
The Manifold LAS dataport provides the following capabilities:
Exposes LAS data as a table with fields set up according to the LAS record type used in the file.
Automatically creates a drawing that uses a high performance point cloud spatial index.
Automatically utilizes COPC features, such as built-in COPC spatial index. When using COPC LiDAR data, there is no need to create indices in advance.
Automatically works with LAZ files, which are a compressed version of LAS files.
Analyzes the layout of LAZ files and caches uncompressed data accordingly. This significantly improves performance.
Parses coordinate system info within LAS files, and automatically assigns the specified coordinate system.
Linking a LAS or LAZ file exposes shifts and scales for X, Y, Z fields in metadata. Right-click on the drawing in the Project pane and choose Properties.
The LAS/LAZ and LAS library dataports bind shifts and scales to the geometry field.
If the LAS files are accompanied by any PRJ or PAM XML files, the coordinate system those sidecar files specify will be used.
Exposes a classification table, whenever available within the data set.
Exposes any extra fields added via variable-length records.
Exposes data within any built-in and custom LAS headers as Info and InfoVLR tables.
Exports table data as a LAS or LAZ file.
Reports the file source ID even if it is zero.
We can export point data to LAS or LAZ format by opening the drawing or table containing LiDAR data and choosing File - Export, picking LAS Files (*.las;*.laz) as the file type. In the Export dialog, entering a File name with no extension or with a .las extension will write a LAS format file. Entering a File name with a .laz extension will write a compressed LAZ format file.
When selecting records in a table window, the Edit - Export Selected command in the main menu allows us to export the selected records to a file. Since LAS format is one of the formats that Export Selected can export to, this allows us to select points in either a LiDAR drawing or its table, and then from the table window for that drawing to choose Edit - Export Selected in the main menu to export those points to a new LAS file. In the Export dialog, choose LAS Files (*.las;*.laz) in the Save as type box.
Exporting to LAS or LAZ using File - Export works if the focus is on either the drawing or the drawing's table. Edit - Export Selected works only when the focus is on an open table that has some records selected. We can select LiDAR points/records either in the drawing or in the table, and then with the focus on the table choose Edit - Export Selected.
To export to LAS or LAZ, the table must have at least one geometry field.
For users familiar with LiDAR data and LAS and LAZ formats, here are a few quick tips:
Install large data storage - Fast, large, hard disks are cheap. A six TB hard disk that can store ten thousand uncompressed LiDAR files of typical, 600MB size in LAS format is easily affordable, under $120. A two TB SSD in M.2 form factor for direct connect to the latest, high-speed PCIe interfaces is also cheap, well under $200, and can store over 3,300 uncompressed LiDAR files. Get plenty of storage space.
Use LAS files, not LAZ files - LAS files are significantly faster than LAZ format because there is no delay from decompression. LAS files are usually ten times bigger than LAZ, but given the absurdly low cost of large data storage, who cares? LiDAR data is often published in LAZ format to reduce download sizes. Get a copy of the free LASzip package and use LASzip to decompress all LAZ files into LAS files as soon as you download them, so they'll be ready to use when you want. Don't save LiDAR data as LAZ files. Use LAS files for more speed.
Link LAS files, don't import them - Manifold builds a high performance spatial index when we link a LAS or LAZ file, but not when we import it. Future builds will add the high performance index for imported files as well, but until then it is faster to link a LAS or LAZ file into a project.
Create index files in advance - Use the Tools - Index LiDAR Files command to create indices in advance for LAS or LAZ files we intend to use. Thinking ahead can save waiting around: when we first download some LAS or LAZ files, launch a Manifold session and run the Tools - Index LiDAR Files command to create indices for the downloaded files. Leave it running in parallel while we continue doing work in other Manifold sessions at the same time, and for bigger collections of files, leave it running overnight. When indices have already been created for the LAS or LAZ files we intend to use, when we link them into our projects they will be ready for use very quickly with no need to wait around while indices are created. When using COPC LiDAR data, there is no need to create indices in advance since Manifold will automatically use the COPC index.
Nothing might happen at first... but just once - If we have neglected to use the Tools - Index LiDAR Files command to create indices in advance for LAS or LAZ files we intend to use, the very first time we open a drawing within a linked LAS or LAZ data source, the drawing window will pop open but we will see nothing. That is because Manifold is building the spatial index in background. When the spatial index is built, the drawing window will automatically populate with the LiDAR points. The index is stored in an accessory file next to the LAS or LAZ file, so any future uses of that file in Manifold will have no delays to build an index. The drawing will pop open and be populated right away. Tables for drawings linked from LAS or LAZ files do not need point cloud spatial indices. They open right away and are populated with records for the LiDAR points whether or not an index has been built.
LAS file format is an ASPRS (American Society for Photogrammetry and Remote Sensing) standard format for storing LiDAR data as points. It is a vector format, not a raster format. LAZ format is identical to LAS except for being compressed. This documentation uses the term LAS to mean both LAS and LAZ formats, except when addressing performance or size issues that are specific to either LAS or LAZ.
LAZ format is popular for publishing LiDAR data because LAZ files are usually many times smaller than LAS files. A 450 MB LAS file, for example, will often compress to a LAZ file that is well under 60 MB. The price we pay for smaller file sizes is that for general purpose GIS work LAZ format must be decompressed to recover the original LAS data, at least into memory cache or temporary working files. Decompression is reasonably fast, but nonetheless significant time is required, possibly tens of seconds per file, to decompress bigger LAZ files.
Therefore, for faster opening and initial use of LiDAR data in Manifold, it is quicker to use uncompressed LAS files than compressed LAZ files. To decompress LAZ files into LAS form, we can use the LASzip.exe utility to decompress many LAZ files into LAS format. Manifold will automatically decompress LAZ into LAS using an internal copy of the LASzip code, but if we will be working with many LAZ files it is often more convenient to bulk convert them all at once overnight into LAS using the LASzip.exe utility.
Spatial indices allow far faster work with spatial data, but spatial indices designed for general GIS work are not efficient with the very large numbers of individual point objects in a typical LiDAR data set. Manifold therefore builds a specialized index for point cloud data when linking to LAS or LAZ files individually or when using the librarylas dataport to create a data source linked from a folder containing LAS or LAZ files.
Point cloud spatial indices are required to display LiDAR data in drawing or map windows. If an index has not been built ahead of time for a LAS or LAZ file, when the drawing is first opened it will appear blank, with a black underline on the drawing's tab to indicate an index is being built. When the index is built, the drawing window will be populated with points. Point cloud spatial indices are not required for the tables of LiDAR data drawings. Those will pop open immediately and will be populated right away, while the index for the LiDAR file is being built.
The best way to build spatial indices for LAS or LAZ files is to run the Tools - Index LiDAR Files command to create indices in advance for LAS or LAZ files we intend to use. The Index LiDAR Files command runs in parallel for faster performance and can index multiple LAS or LAZ files simultaneously. It is good practise immediately upon downloading LAZ or LAZ files to launch the Index LiDAR Files command to index those files. That will create indices so when we link those LAS or LAZ files we will not have to wait around in real time while the system creates indices.
When using COPC LiDAR data, there is no need to create indices in advance since Manifold will automatically use the COPC index. Attempting to run the Tools - Index LiDAR Files command to create indices on COPC LiDAR LAZ files will report the files already have indices.
If we have not created indices in advance and we are not using COPC data,, when we link a LAS or LAZ file into our project, Manifold will automatically create an index for that file.
When no point cloud spatial index exists, the system builds the index is built the first time a LAS or LAZ file is linked into a Manifold project. That can take significant time, possibly minutes per file in the case of LAZ files (faster with LAS files), but the spatial index needs only to be created once. The point cloud spatial index that is created is stored in an accessory .mapindexp file in the same folder as the linked LAS or LAZ file. Manifold will automatically use that .mapindexp spatial index the next time the accompanying LAS or LAZ file is used in a project.
Whenever we copy to another location LAS or LAZ files that have been used with Manifold, we also should always copy along with them the accompanying .mapindexp (point cloud spatial index) and .mapmeta (coordinate system) accessory files Manifold has created, so those accessory files can be re-used for much faster launching of projects that use their LAS or LAZ files.
The specialized point cloud index is a recent addition to Manifold which has not yet been implemented for LAS or LAZ files that are imported into a Manifold project. That creates, for now, the unusual situation that Manifold can display and work with LiDAR data faster when linking a LAS or LAZ file into a project, than importing the file into the Manifold project. Usually, workflow is much faster when we import a file into Manifold; however, LiDAR point cloud data in LAS files, for now, is an exception to that general rule.
Upcoming Manifold builds will include the specialized point index within .map files as well, which will mean imported LiDAR data will be even faster than linked LiDAR files. However, for now we should link LiDAR files if we want maximum speed. A bonus of linking LiDAR data into a project is that the Manifold project stays small, typically only one megabyte in size, since all of the LiDAR data remains stored in the LAS file.
If a point cloud spatial index does not yet exist for a LAS or LAZ file, Manifold starts building a point cloud index when we first open either a linked LAS or LAZ drawing or the drawing's table. The moment we click open the table, or we click open the drawing or add the drawing as a layer to an opened map, Manifold will launch a background process to build the index. Building the index in background allows us to keep working, doing other things, while we wait for the index to be built. Although the drawing window will open, no points will be visible until the index is finished building.
If we use the free Viewer
to link in a LAS or LAZ file, Viewer
will also build a point cloud spatial index file for us. Super!
We can avoid waiting around for the index to be built by running the Tools - Index LiDAR Files command to build an index in advance. The Tools - Index LiDAR Files command works in the free Viewer as well.
A recent innovation for LiDAR point cloud storage is Cloud Optimized Point Cloud (COPC) format. COPC format stores LiDAR point cloud data in a special form of LAZ format that stores point data organized in a clustered octree. COPC data is automatically read by Manifold's LAZ dataport. COPC spatial indexes are very new and thus rarely encountered compared to LAZ files not in COPC form, but the idea is a solid one that Manifold Engineering expects will become more popular.
If a LAZ file already contains a COPC spatial index, Manifold will recognize that spatial index and will automatically use it. When working with LAZ files that are in COPC form there is no need to run the Tools - Index LiDAR Files command to build an index in advance. There is no harm done in attempting to run the Tools - Index LiDAR Files command to create indices on COPC LiDAR LAZ files: the tool will report the files already have indices.
COPC was designed in July–November 2021 by Andrew Bell, Howard Butler, and Connor Manning of Hobu, Inc. See the copc.io website.
Manifold will read any coordinate system information specified by LAS or LAZ files. In addition, Manifold also will get projection (coordinate system) information from any sidecar PRJ or PAM XML files that accompany the LAS or LAZ files. If an accompanying file specifies a different coordinate system than the LAS file specifies, the sidecar file wins, since sidecar PRJ or PAM XML files are often provided to override, that is to fix, erroneous coordinate information specified within the LAS.
When exporting to LAS or LAZ, Manifold will embed coordinate system into the produced file. If the coordinate system info is WKT or WKT2, it is used as is. Otherwise it is converted to WKT.
In Windows Explorer, we have a LiDAR file called pentagon.laz. A LAZ format file is a highly compressed form of a LiDAR LAS file.
In Manifold, we choose File - Link and we navigate to the desired file. The Save cache box is checked by default. We leave it checked to ensure that a .mapcache file is built, to save formatting and other useful info for the next time that we want to link the file.
We double-click on the pentagon.laz file to link it.
A new data source, called pentagon from the stem of the .laz file, appears in the Project pane. We click the + icon to expand the data source and to look into it, to see the drawing and table within.
We double-click drawings to open them. For a more interesting display, we first create a new data source using a Bing street maps image server as shown in the Example: An Imageserver Tutorial topic. We then create a map and drag and drop the Bing layer into the map, and then we drag and drop the pentagon Drawing into the map.
The first time we open the drawing in the data source, Manifold will build a .mapindexp spatial index file. That takes time, a minute or so to build the index, during which time the drawing's tab will show a black underline. The drawing will remain empty until the index is built, after which it will be automatically populated with points.
It takes time to build the index, but that is a one-time job that happens only the first time the .laz or .las file is linked into a Manifold project. Once the index is built, it is saved in an accessory .mapindexp file with the .laz or .las file, and the next time any Manifold session either opens this project or links that .laz or .las file, the existing .mapindexp file will be used and the open will be fast, with no need to build the index again.
Avoid waiting: Use the Tools
- Index LiDAR Files command
to build indices for LiDAR files in advance.
If we like, we can double-click the pentagon table within the data source. That will immediately populate, as tables do not need the point cloud spatial index to show their records.
When the index is built, the drawing will automatically be updated to show the LiDAR points. The drawing appears in default formatting as a mass of gray points. It is located in the right place since LAS/LAZ format almost always provides projection information.
What we see
is a summary
of the data set, a representative preview. This particular data
set contains over eight million points, but only a few thousand are shown
as a representative sample. LiDAR data sets can contain billions
of points, far more than the number of pixels in the region covered by
the LiDAR data, so Manifold automatically thins the points in the viewing
area and shows a limited number of the best representatives. The
number of points displayed depends on the display window size.
If we open the drawing's table, we see it is a read-only table (as shown by the gray background color): linked LAS data sets are read-only.
Switching to the Layers pane, we can see the many attributes each LiDAR point has in this data set.
Manifold automatically reports the scaled values for X, Y, and Z. The Z value is what is generally thought of as the "true" height, while the X and Y values are the coordinates in the native units in whatever coordinate system (projection) is used.
One reason Manifold provides a specialized, high-speed, spatial index for points in LiDAR data is to enable more efficient spatial queries against large LiDAR data sets, with possibly billions of points.
After import, if we use Windows Explorer to look at the folder from which we linked the pentagon.laz file, we see two new files have been created, a pentagon.laz.mapcache file and a pentagon.laz.mapindexp file.
The pentagon.laz.mapcache file is the standard .mapcache accessory file Manifold uses to facilitate work with linked files. It is shown above with zero size because we have not yet saved the project. Until we save the project, the .mapcache information is kept cached in memory.
The .mapindexp file contains a specialized spatial index for points that is created when linking LAS LiDAR point cloud files into a Manifold project. The .mapindexp file is then used to service spatial queries, including internal spatial queries such as those performed to display the point cloud within Manifold. The file is versioned so that future Manifold builds can choose to reuse files created by an earlier build or overwrite them with new versions. The .mapindexp file is only created for a linked LAS/LAZ file in a writable location.
The size of a .mapindexp file is significantly smaller than the size of an uncompressed LAS file on which it is built, but it is usually larger than the size of a compressed LAZ. In the example above, the .mapindexp file is over three times the size of the .laz file.
Even though the data source for the linked .laz file is read-only, we can still format the drawing, with Manifold automatically storing the formatting in the associated .mapcache file.
We use a thematic format for point fill color as seen above, using a square point symbol with a stroke size of .3 point and a symbol size of 8 points. We use the Color Brewer CB Spectral palette in reverse order. This particular LiDAR data set uses a Maryland State Plane projection which has Z values in meters.
Scanning data to acquire statistics when we press the Tally button typically takes 40 to 50 seconds on a data set this size, about 8 million points. Once the data set is scanned, they will be saved while the window is open with no need to recalculate statistics when trying out different intervals for thematic formatting.
Using the thematic format provides a somewhat better visual preview of the data set. The display shows LiDAR elevations for the Pentagon in the United States.
When images are re-styled, the visual results for the same palette may be slightly different as different sets of representative points are sampled by the data structure, based on the size of the viewing window.
When we zoom further into the data set, more points will appear, to provide a more detailed view as we zoom in.
More points will fill in as we zoom further into the data set, to the full resolution of the data set.
When we save the project the .mapcache file will be saved with full information to speed up access to the linked file in the future, either from this project or from any other Manifold project that will link to the same .laz file.
When we save the project, it will be very small, only one MB, since all of the LiDAR data remains stored in the .laz file and the spatial point cloud index data is stored in the .mapindexp file.
Viewing LiDAR as colored points in a vector drawing is neither efficient nor appealing, at least not when points are thinned to representative samples.. We can do better by interpolating the vector point layer into a terrain elevation raster, an image that uses height for the single channel in the image, using the Kriging transform template, as discussed in the Example: Vector to Raster using Kriging topic.
If we are
working with a linked LAS or
LAZ file, to interpolate or do other operations that create new components
we must be
working with a map in the main part of the project that has the linked
drawing as a layer. That puts the context for what we are
doing in the main part of the project, which is writeable, so we can create
new components with a Transform template. If we just
open a linked drawing we will be unable to use transforms to add a component,
because the context will be inside the linked data source, a LAS or LAZ
file, into which we cannot write new drawings or images.
See the discussion in the Portals to Other Worlds section of the Project Pane topic, on how the main part of a Project is writable while parts of the project that are inside data sources, like linked LAS or LAZ files, might not be.
With the focus on the map window, in the Transform pane we choose the pentagon layer and then we choose the Geom field. We double-click the Interpolate template to launch it.
In the Interpolate template, we choose Kriging as the Interpolation method. For the Z value from the pull down menu we choose the Z field. We leave the Margin, Radius, and Neighbors parameters at their default values of 0.
The LiDAR data is fairly detailed so a Resolution of 1 meter seems appropriate. If we would like to know more precisely just how detailed the data is, we can zoom far into the data to where individual points start spreading apart, and then measure the approximate spacing between LiDAR points, to see if the spacing between points is approximately one meter.
The pentagon.laz file saves LiDAR data
using a meter-based State Plane coordinate system, so that is the coordinate
system used by the drawing LiDAR files sometimes are
published using degree-based coordinate systems such as Latitude / Longitude,
in which case the Resolution and other parameters for Kriging transforms
will use units of degrees or radians. That can require some quick
use of the Windows Calculator accessory to calculate a sensible value
in fractions of degrees or radians to use for Resolution, given the latitude
of the data set.
For a Result destination, we must create a New Table, that is, a new component, with a new image being created for that automatically. We enter Pentagon Kriging as the name for the new image and an analogous name for the new image's table. We can use whatever names we like, but it makes sense to use names that remind us what those components are.
Press Transform.
Unlike many operations
in Manifold, interpolation on LiDAR data sets is fairly slow, often taking
minutes. Future builds will introduce new interpolation technology
for LiDAR that will dramatically increase interpolation speeds.
Manifold calculates for a minute or two, and the result is a new image called Pentagon Kriging in our Project pane.
We drag and drop the new Kriging image into the map. In the illustration above, we have double-clicked the drawing layer to turn it off.
The new image appears in interpolated grayscale color, with Manifold applying a best guess based on samples of what will work for grayscale values. The result is a grayscale image where higher heights are brighter. That is an interesting effect, but we can do better by applying a colorful palette.
With the focus on the Pentagon Kriging image layer, we use the Style pane.
The Style pane by default shows the same channel, Channel 0, used to drive the Red, Green, and Blue display outputs. Since the same value is used in all three display outputs, the result is RGB outputs that are all shades of gray.
We will click on the channel arrangement box to switch from (use RGBA channels) to Channel 0, so we can apply a palette based on Channel 0. The Style pane remembers that we last used 10 breaks, so that is how it sets up. We click the Palette picker button to choose a palette.
We apply a color palette, the CB Spectral palette in reverse order as seen above. Press Update Style.
That provides a more interesting display. We can make it even more interesting by adding shading.
Switching to the Options tab in the Style pane, we check the Use shading box, use a Z scale of 0.3. We press Update Style.
The use of shading provides a nice, 3D effect.
We can zoom further into the image to see the effect that a one meter resolution image generated from detailed LiDAR data can provide. The flat rectangle in the lower right of the window is the Metro-Pentagon station.
So far, this topic has shown how to link a LiDAR data file into a Manifold project. Linking takes advantage of the specialized .mapindexp point index Manifold generates for point data as found in LiDAR files.
We can, if we like, import a LAS file into a Manifold project, fetching the data from the file and bringing it into the Manifold project. Importing the data into a Manifold file has the plus of very durable .map format, but it does mean that when we save the project the resulting .map file will likely be hundreds of megabytes or a few gigabytes in size.
To import from LAS or LAZ format:
Choose File-Import from the main menu.
In the Import dialog browse to the folder containing data of interest.
Double-click the file ending in .las or .laz for the data of interest.
A table and a drawing will be created.
Using the same pentagon.laz file as in the linking example, we choose File - Import and import the file.
As before, we create a new data source using a Bing street maps image server as shown in the Example: An Imageserver Tutorial topic. We then create a map and drag and drop the Bing layer into the map, and then we drag and drop the pentagon Drawing into the map.
When importing LiDAR data, current Manifold editions create an rtree spatial index on the data and, unlike the data structure used for linking, do not thin any points. The entire data set is shown.
Opening the drawing's table, we see it is fully read/write, since the data is fully resident in the .map project and since an mfd_id field is created and an index is automatically built in the drawing's table when a drawing is imported.
Switching to the Layers pane we see all attributes have been imported as well.
As with the linked example, we can use the Style pane to assign a palette to color the elevation values in the drawing.
We use the Style pane to color the points using a palette, but slightly differently than in the linked example since we have a denser set of points being displayed, given the different simplification used for linked as opposed to imported data. We assign the same palette to coloring both the stroke color and fill color for points. We have used the CB Spectral palette in reversed order.
That provides a more sensible display than in the linked example, where point thinning lowered the apparent resolution.
Zooming in, we can see better detail.
Even though we have all points displayed when importing LiDAR data, we can generate a better visual impression by using Kriging to create a smooth, raster surface. It takes only a minute and a half to create a raster with Kriging, repeating the same steps used for Kriging interpolation in the linked file example.
With the focus on the map window, in the Transform pane we choose the pentagon Drawing layer and then we choose the Geom field. We double-click the Interpolate template to launch it.
In the Interpolate template, we choose Kriging as the Interpolation method. For the Z value from the pull down menu we choose the Z field. We leave the Margin, Radius, and Neighbors parameters at their default values of 0. We use a Resolution of 1 meter.
For a Result destination, we must create a New Table, that is, a new component, with a new image being created for that automatically. We enter pentagon Drawing Kriging as the name for the new image and an analogous name for the new image's table. We can use whatever names we like, but it makes sense to use names that remind us what those components are.
The default table name created automatically as we enter a name for the New image ends in ...Table. This topic uses a table name that ends in ...Tiles, which is how some users prefer to name tables for images. Others prefer to have all of their tables, both for drawings and images, end in ...Table. We can adjust the name automatically created for the table however we like.
Press Transform.
Manifold calculates for a minute or two, and the result is a new image called pentagon Drawing Kriging in our Project pane. We drag and drop the resulting image into our map.
To color the image with a palette, with the focus on the pentagon Drawing Kriging image tab in the map window, we use the Style pane.
As before, we switch to using Channel 0 only, and then we choose the CB Spectral palette, in reverse order.
In the Options tab, we turn on shading and we use a Z scale of 0.3.
Press Update Style.
The image again appears very detailed because we have used Kriging to create a raster surface with a pixel at every meter. The result is identical to the linked example, since the data is identical, the only difference being whether it is stored in the original LiDAR .laz file or stored, as a result of import, within the .map file.
A more detailed look at the same image is in the Gallery page of the Manifold website.
Selecting desired LiDAR data is easy. All of the point data is in the drawing's table: we simply select which points we want based on the values for various attributes. After all, it is just data in a table. We can use the Select pane or SQL queries however we like, including to change the data, for example, using the Transform pane or SQL.
The Manifold
LAS/LAZ dataport for linking allows us to select
records in tables even when linking
LAS or LAZ files, as well as when we have imported
the LAS or LAZ file. To transform the data, however, the data
must be writable, which means we must import the
data.
For example, the illustration above shows a few fields (most are hidden using the Layers pane to provide a simpler illustration) for the pentagon table that stores data for the pentagon Drawing. If we want to find all points with a classification field value of 2, we can use the Select pane Search template for numbers.
In the Search template we choose the classification field with equal (=) as the condition and a Value of 2. Press Select.
That selects all points with a classification of 2. We can scroll the table down to where there are some selected points, shown in red selection color.
The selected points are also shown in red selection color in the drawing as well.
If we would like to export the selected points to a new file, with the focus on the table we can choose Edit - Export Selected and then choose one of the supported file formats for export, such as LAS or LAZ.
If we would like to save those points into a new table, with the focus on the table we can choose Edit - Copy to copy the records and then in the Project pane choose Edit - Paste to paste the copied records as a new table. Note that a Copy copies all selected records, and not just those that are shown in the table window.
Another way to save the selected points into a new table is to write a query that does what the Select pane did, but which immediately selects into a new table, for example:
SELECT * INTO [pent_class_2]
FROM [pentagon]
WHERE [classification] = 2;
Having created a new table called pent_class_2 we can create a new drawing from that table. We can save the coordinate system used by the original pentagon Drawing as a Favorite, so when we create a drawing from pent_class_2 we can specify the coordinate system for the new drawing to use with a single click of a Favorite.
We can create a new drawing from the pent_class_2 table and drag and drop it into the map:
It turns out there are nearly 2.4 million points with a classification of 2 in the example pentagon LiDAR data set. We see them as points above, styled green, with the image created by Kriging interpolation as a layer underneath to provide context.
If we turn off the pent_class_2 Drawing layer we can guess what a classification of 2 represents in that data set.
The pent_class_2 points appear to be LiDAR returns in zones of bare ground level, colored in darkest green in the illustration above.
The query used to select the pent_class_2 points is a very simple query, but we can do a lot with such queries. For example, we could select into a new table all points with ScaledX and ScaledY values within a given range, to save a subset rectangle of the points. We can use the status bar to read out projected coordinates to know what range to use for a desired rectangle, or we could reproject into Latitude / Longitude so we could use decimal degree numbers for the X and Y.
If desired, we could add a drawing layer above our LiDAR points set, and in that drawing layer draw a rectangle or other figure, or use areas in that drawing layer, and then use the Join dialog to do a spatial join to transfer some marker attribute into the LiDAR data set and then select on that attribute. That would select all LiDAR points within the areas. We could then Copy those and Paste them to create a new table and a new drawing that only includes the LiDAR points within the areas in the other layer.
Regrettably, it is routine to encounter LAS/LAZ files that wrongly specify coordinate system axis ordering of YX when in fact the data uses XY axis order. Such errors are a classic case of wrongly specified coordinate systems discussed in the That YX Thing essay.
To date, no LAS/LAZ files have been encountered that specify YX axis ordering and which in truth really do use YX axis ordering. Therefore, the Manifold LAS, LAZ LiDAR and librarylas dataports as well as the Index LiDAR Files and Index Excluded Files commands all assume the specification of YX ordering is a mistake, and instead force use of XY axis ordering. If any genuinely YX ordered LAS files are ever found, Manifold will add an option to allow YX ordering.
Visualization - Manifold LiDAR features are evolving rapidly and are subject to change. Check out the latest additions in the Changes and Additions topic. Current facilities are aimed at analyzing and manipulating data, for example, combining data sets or creating subsets of data, generating raster representations of the point cloud using facilities such as Kriging interpolation and so on. The focus (for now) is not on automatic generation of visual renderings in 2D or 3D. Future builds are expected to add such visualization.
Thank you, LAStools! - LAZ file format uses lossless compression, implemented in the open source LASzip compression library, developed by Martin Isenburg, the creator of LAStools. Manifold uses the LASzip library to read/write LAZ files: see the Third Party Release Levels topic for which version is used. That helps preserve use of a single LAS / LAZ file format standard, without forking proprietary variations, like ESRI zLAS.
LAS and LAZ speeds compared - Consider an example using a LiDAR data set for part of the city of Portland, Maine, in the US. In uncompressed LAS format, the file is 458 MB. In compressed LAZ format, the file is only 58 MB. To compare the speed of LAS files with LAZ files, in Manifold we used the librarylas dataport to create a data source on a folder containing the LAS file, and then we created a data source on a folder containing the LAZ file. We dragged and dropped both resulting drawings into a map, with timings shown below in a Log Window report.
The Render lines depend upon how fast the data can be fetched from the source. In the case of the LAS file, no decompression is necessary so rendering happens in a fraction of a second. In the case of the LAZ file, that must first be decompressed into a working, LAS version in memory, so rendering takes over 14 seconds, many times longer. Most users will use cache (the default) for the data source, so the extra time rendering is a one-time cost for only the first time the drawing is used in that particular Manifold session. But when working with ensembles of LAS/LAZ files that may contain dozens or over a hundred files, the extra time to decompress LAZ at the beginning can add up. If we leave data in LAZ form and just link to it, then every time we open a project with those drawings, the first time we open the drawing we will have to wait for decompression into working format in memory before the LiDAR points appear in the drawing.
In both cases in the above example, we had previously run the Tools - Index LiDAR Files command to create a Manifold point cloud spatial index file for each LiDAR file. That provides as close as possible a comparison of the time cost of LASzip decompression between using the LAZ file and the LAS file. If this is our first time using a LAS or LAZ file in Manifold building a spatial index will go faster with a LAS file than with a LAZ file, since there will be no additional time required to decompress the LAZ file into working form in memory.
When linking LAZ files into a project, leave the Save cache box checked in the Link dialog. When creating data sources using LAZ files, leave the Cache data checkbox checked in the New Data Source dialog. Those options will ensure cache is used for faster access to working data once the LAZ file has been decompressed behind the scenes. To reduce the size of .map files, we can uncheck the Save cached data between sessions box in the New Data Source dialog. Leaving that box checked will save the cache from each session into the .map file, resulting in a larger .map file, which somewhat defeats the idea of keeping file sizes small by leaving data in compressed LAZ file format instead of decompressing it into larger LAS file format.
A good beginning - LiDAR point cloud capability in Manifold is a new feature that is just getting started, and getting steadily better with new Manifold builds. LiDAR users are encouraged to send in suggestions for any changes or additions desired. Stay tuned for many new features and improvements!
Example: Spectacular Images and Data from Web Servers
Example: An Imageserver Tutorial
Example: Vector to Raster using Kriging
Example: Create Terrain Elevation Raster from a NASA PDS Table - A long, multistep example showing many powerful features of Manifold in various steps. We import terrain elevation raster data from a NASA PDS archive and style the images with an elevation palette and hillshading. We notice one of the images was wrongly georegistered by NASA, so we import the original LiDAR point data from a table in the PDS archive. We transform the imported table from 0 to 360 degree longitudes into +/-180 longitudes, construct a geometry field, and then we create a drawing from the table that shows the LiDAR point data in the table. Next, we use Kriging to create a terrain elevation raster from the LiDAR point drawing, and then we style that and hillshade it so it can be exactly overlaid on a Bing satellite layer to enhance the satellite photography with enhanced terrain relief.