What is SQL for ArcGIS® Pro?
SQL for ArcGIS® Pro is a
The add-in enhances ArcGIS Pro with the world's most powerful desktop spatial SQL engine, Manifold's Radian® engine, to unleash the phenomenal power of SQL from your ArcGIS Pro desktop. Enjoy full-featured SQL, superior quality, and the amazing speed of parallel CPU and parallel GPU query execution, all supported with hundreds of SQL functions for every spatial need.
In Advanced mode the add-in runs spatial SQL with hundreds of file formats, databases, web servers, and other data sources outside of ArcGIS Pro. Ultra fast scratchpad storage in the add-in makes it easy to combine huge data from outside your ArcGIS Pro project with data inside the project.
Connect to almost any data imaginable, from enterprise DBMS servers to file databases to web servers to file formats. SQL for ArcGIS Pro handles an almost unlimited range of data types in tables, vector data, raster data, drawings, maps and images.
"This works brilliantly for converting 3D faces to a DEM. Ran so fast I thought it hadn't worked." - Forum post
Access, clean, prepare, blend, analyze and transform data from many different data sources at once, including Oracle, SQL Server, PostgreSQL, DB2 and many more using point-and-click templates that preview the result before you commit. Use the full power of parallel SQL, asking SQL for ArcGIS Pro to write queries for you or writing your own. Pull data from your enterprise database, manipulate it in place or push a blend of data into your file geodatabases or corporate database from other sources.
Apply the infinite power of automatically parallel SQL, either by writing straight SQL or in Advanced mode by launching the effortless, easy power of SQL for ArcGIS Pro point-and-click dialogs. SQL for ArcGIS Pro unleashes the full power of all the CPU cores in your system and automatically launches massively parallel computations using thousands of available GPU cores for supercomputer computational performance unmatched by any other desktop GIS or spatial engineering tool. Nothing else comes close.
SQL for ArcGIS® Pro is a product of Manifold Software Limited, the creators of Manifold® brand software products. Manifold is not associated with Esri®, the makers of ArcGIS® Pro.
How is SQL for ArcGIS Pro different from Release 9?
Manifold creates both Release 9 and SQL for ArcGIS Pro. Both are built on Manifold's revolutionary Radian® spatial engine. Release 9 is Manifold's new GIS, ETL, and spatial engineering package, which provides thousands of powerful GIS features for visualizing, editing, transforming, and analyzing GIS data of all kinds, including rasters, vectors, and attribute data. Release 9 includes an extensive user interface providing point and click dialogs, interactive editing, with full SQL and programming capability.
SQL for ArcGIS Pro is a superset of Release 9 that has been integrated as an add-in to Esri's ArcGIS Pro desktop GIS package to provide SQL for use with vector file geodatabase layers within the ArcGIS Pro project. SQL for ArcGIS Pro contains extensive additional internal programming and features not found within Release 9, which provide seamless ease of use to ArcGIS Pro users.
Key distinctions between SQL for ArcGIS Pro and Release 9:
ArcGIS Pro Required- SQL for ArcGIS Pro requires an ArcGIS Pro version 2.8 or more recent installation on the same system. SQL for ArcGIS Pro cannot launch on a machine that does not have ArcGIS Pro installed. Integration with ArcGIS Pro- SQL for ArcGIS Pro installs and launches as an add-in to ArcGIS Pro, automatically scanning the ArcGIS Pro project to identify and to connect to all file geodatabases that are used by that project. All vector layers and tables in those file geodatabases are automatically populated into the SQL for ArcGIS Pro query builder, ready for instant, point and click use when building SQL queries. SQL for ArcGIS Pro also includes specialized synchronization code to maintain synchronization between the ArcGIS Pro session and the SQL command console launched by the add-in. Release 9 has no such integraton and synchronization with ArcGIS Pro. Automatic Launch of SQL Interface- Release 9 launches with a blank project, with the user interface configured for an alternative approach to GIS that is significantly different from ArcGIS Pro. SQL for ArcGIS Pro launches with the user interface pre-configured for doing SQL with the ArcGIS Pro project, ready to go and already populated with all file geodatabases used in that project, together with all tables, vector layers, and attribute fields from those file geodatabases pre-loaded into the query builder. It's the fastest, easiest way to do SQL with file geodatabases in Arc. Focus on SQL with ArcGIS Pro- SQL for ArcGIS Pro is sold and supported from the sql4arc.com website, with a dedicated User Manual that is organized to present SQL use with ArcGIS Pro, with all other features tucked into an Advanced Mode section of the documentation. This makes it much easier for ArcGIS Pro users to take advantage of SQL without having to wade through many pages of unneeded documentation.
What is Advanced Mode in SQL for ArcGIS Pro?
SQL for ArcGIS Pro launches with an SQL command console populated with file geodatabases that are used in the ArcGIS Project, ready to do SQL with those file geodatbases. Advanced Mode in SQL for ArcGIS Pro means turning on additional controls and user interfaces in the SQL for ArcGIS Pro session that provide additional capabilities.
Although the primary mission of SQL for ArcGIS Pro is to provide SQL for vector layers from file geodatabases that have been added to the ArcGIS Pro project, the add-in also includes hundreds of other Manifold features that can be used in Advanced mode but which have not been closely integrated into ArcGIS Pro. For example, Advanced mode provides access to point and click dialogs that capture the power of SQL with no need to write queries, as well as sophisticated transform, selection, analytic, editing and visualization capabilities.
Over time, Manifold plans to custom tailor more and more of those other features to provide a better user experience within ArcGIS Pro. In the meantime, by using the add-in's Advanced mode, users can take advantage of the many Manifold features in addition to SQL as they are.
Advanced mode has four primary uses:
Independent Connections- Using advanced controls like the Project pane and File and other menus, SQL for ArcGIS Pro can independently connect to and work with data sources other than those file geodatabases that are used within the ArcGIS Pro project. In Advanced mode we can write SQL combines layers within the ArcGIS Pro project with data from sources not accessible to ArcGIS Pro. When using Advanced mode controls the add-in can work with a vast range of other data sources that can be imported or linked into the add-in's own scratchpad storage, such as shapefiles, rasters, and many other types of GIS data. Point and Click Dialogs- Writing SQL queries is wonderful for people who know SQL, but for many people learning SQL it really helps to have point and click dialogs that encapsulate the power of SQL into dialog driven templates. A simple use of Advanced mode is turning on the Project pane, to allow opening tables and other layers in their own windows, and then using the point and click Select pane and Transform pane to do sophisticated selections, transformations and analytics. Another popular point and click dialog is the Join dialog, for access to sophisticated JOINs without the need to write any SQL. Ultrafast internal data store- SQL for ArcGIS Pro can handle over a terabyte of data stored locally in SQL's own incredibly fast, fully parallel data store. That allows splitting very big, complex queries into parts where intermediate results are created within the add-in's own internal data store, with only the final result pushed via a SELECT INTO, INSERT, or UPDATE back out to much slower data stores, like FGDB. Splitting big queries into smaller parts is easier to understand and debug, and you can check intermediate results to confirm workflow. You can also create indexes on intermediate tables for faster performance, and run huge SQL operations entirely in hyperfast parallel data storage for maximum speed with bigger data. Expanded Functionality- Turning on more advanced controls provides full access to all Manifold Release 9 features that exist within the SQL for ArcGIS Pro add-in, basically expanding the add-in session to be a single session Release 9 GIS environment. Release 9 provides thousands of GIS, ETL, and spatial engineering features, including high performance visualization of vectors and rasters, editing, and many analytic and geoprocessing capabilities similar to those Esri capabilities that within ArcGIS Pro require installation of Spatial Analyst or other options. Although the focus of the add-in is doing SQL, intrepid users wishing to take advantage of the many additional features they get in Advanced mode can do so.
Manifold capabilities that are available using advanced controls within SQL for ArcGIS Pro are not integrated with ArcGIS Pro. They do, however, provide a useful way within the add-in of prepping data sets for use within ArcGIS Pro, or for visualizing what is about to happen in complex SQL expressions or queries. Advanced mode is also a great way of converting data from formats which perform poorly with ArcGIS Pro into better formats.
How Does Built In DBMS Help?
Classic GIS products like ArcGIS Pro tend to focus on presentation and visuals. But even ArcGIS Pro, powerful though it may be, is not a DBMS engine like Oracle, PostgreSQL, SQL Server, MySQL, or the Radian® engine inside SQL for ArcGIS Pro.
The state of the art in spatial work today is that to get the most out of your spatial data you need both worlds: a high end GIS with superb presentation and other capabilities like ArcGIS Pro, plus a spatial data engine like Oracle, PostgreSQL, or the Radian engine to provide capabilities like SQL to do sophisticated work with data. Adding SQL for ArcGIS Pro to your ArcGIS Pro installation combines the best of both worlds.
Going beyond DBMS sophistication, SQL for ArcGIS Pro implements all capabilities using full CPU and GPU parallelism, providing astonishing speed way beyond what legacy GIS can accomplish, often accomplishing in seconds what even a premium legacy GIS like ESRI's ArcGIS Pro cannot do in hours. ArcGIS Pro is a fine product, but it is not a fully parallel product like SQL for ArcGIS Pro. A non-parallel product like ArcGIS Pro will use only 1/16th of the power of a typical Core i7 computer. Fully parallel SQL for ArcGIS Pro will use all of the power of your computer to provide dramatically faster speed. When SQL for ArcGIS Pro unleashes the supercomputer power of GPU, SQL for ArcGIS Pro can run hundreds of times faster than non-parallel ESRI products. You need real parallel power to keep up with the demands of high performance data analytics, and to be a competent partner with modern, enterprise DBMS servers.
Use SQL for ArcGIS Pro to manipulate, to analyze and to transform your data in ways a legacy GIS cannot accomplish. That's what spatial data engineering is all about, a data-centric workflow that combines point-and-click spatial wizardry with DBMS muscle to get the most value out of your spatial data. Ultrafast and supersmart GIS is not only big fun, it provides the best possible platform for your success.
Use SQL for ArcGIS Pro standalone or together with other GIS packages in the workflow that works best for you: Present results, create wonderful cartographic displays, interoperate with your organization's installed GIS base, serve to the web or serve to an endless array of mobile gadgets. Working together with other platforms, the result is the success you want.
Is SQL for ArcGIS Pro Really New?
Yes. The Manifold technology within SQL for ArcGIS Pro has been created from the ground up as a single, unified, all-inclusive product. There is no need to buy anything extra or to play system integrator cobbling together third party packages. SQL for ArcGIS Pro includes everything you need in a single, hyper-modern package. SQL for ArcGIS Pro is not a fresh-looking GUI painted over older software. Instead, the Radian parallel database engine at the heart of SQL for ArcGIS Pro is totally new technology developed today using the latest advances in data science and massively parallel engineering. Neither Radian nor SQL for ArcGIS Pro has been cobbled together from third party code. Manifold's professional development team, with greater coding and algorithmic experience in massively parallel processing than any other spatial development team, created all of the Radian engine and SQL for ArcGIS Pro as a unified, powerful system providing "always on" parallel power for intense spatial and non-spatial data engineering.
Is SQL for ArcGIS Pro Reliable?
You bet. SQL for ArcGIS Pro is absolutely bulletproof, based as it is on the famously reliable and crashproof Radian engine. In years of beta testing followed by commercial use on four continents the Radian engine crashed just
Today, there is no known way to crash the system using native SQL for ArcGIS Pro features. That's a tribute to the proven, absolutely bulletproof quality of the Radian engine, which has been running for over six years, steadily evolving and improving every step of the way. SQL for ArcGIS Pro may be the first version for you, but the Radian engine that powers SQL for ArcGIS Pro is now in the ninth generation of technology with over 170 major releases and hundreds of minor builds on the way. You can launch SQL for ArcGIS Pro every day for years without encountering a flaw.
"I had the pleasure to be part of the beta... The program was so bulletproof reliable that I had used it for production too..." - Beta tester
Even better, when bugs are found they are immediately fixed, typically in days, not months. SQL for ArcGIS Pro's rapid cycle time using frequent Cutting Edge builds published to the user forum means that any bugs discovered can be eliminated in the very next build, usually within a few days.
SQL for ArcGIS Pro also includes fault tolerant technology for SQL for ArcGIS Pro file management to guard against Windows failures or hardware failures. SQL for ArcGIS Pro may well be bulletproof but it knows your Windows system and hardware could fail.
I have massive amounts of data in Oracle. Can SQL for ArcGIS Pro help?
Yes. SQL for ArcGIS Pro has been designed to play well with and to enhance whatever technology you currently use for spatial data. Storing your spatial data in Esri enterprise geodatabases? Cool. SQL for ArcGIS Pro will make it better. Using Oracle or SQL Server for your spatial data? SQL for ArcGIS Pro will bring a wealth of analytic power ease to increase the value of your investment while making you look like a hero for the effortless improvement in quality and productivity. Do you have to interoperate with a stack built on PostgreSQL and open source tools for users and the web? SQL for ArcGIS Pro will enhance that experience and make life easier for you. SQL for ArcGIS Pro is completely agnostic: whether you store your data in file geodatabases or within whatever other storage you use, total SQL access to that data is cheerfully there at your fingertips.
Is SQL for ArcGIS Pro a Database Server?
Desktop users might think of SQL for ArcGIS Pro as a personal DBMS, because of the local, high speed storage it provides, but IT users of enterprise products like Oracle or PostgreSQL would describe SQL for ArcGIS Pro as a very powerful client database engine. SQL for ArcGIS Pro has its own built-in, parallel database engine, the better to keep up with enterprise DBMS servers like Oracle to which SQL for ArcGIS Pro connects, but SQL for ArcGIS Pro is not intended as a replacement for an organization's database server. Instead, SQL for ArcGIS Pro enhances the value of existing servers.
SQL for ArcGIS Pro's value in IT comes mainly from exceptionally easy ability to connect to and integrate with brand-name DBMS servers, faster analysis using automatically parallel SQL, and unmatched sophistication and ease of use working with spatial data via SQL, whether the task is a massively complex spatial engineering query involving many functions and millions of objects, or just a simple Extract, Transform, and Load operation transferring data from one source to another. The add-in's SQL is genuinely superb with many advanced features such as the ability to define and use functions, but SQL for ArcGIS Pro does not provide transactions or some other features expected in a major database server like Oracle, although it will work in conjunction with such features when connected to a DBMS server that does use transactions.
On the desktop many users will employ local storage in SQL for ArcGIS Pro to enable faster, more flexible, more sophisticated queries. SQL for ArcGIS Pro's native storage provides high speed and huge capacity on the desktop, with pass-through nesting of project hierarchies that can provide easy access to a vast array of formats and petabytes of external storage through a single SQL for ArcGIS Pro project. Offloading the management of crushingly huge intermediate results from Pro to the add-in allows you to handle bigger data in complex operations that would not be reliable without the add-in.
Although ArcGIS Pro cannot handle a mix of object types (areas, lines, points, etc.) within the same layer, SQL for ArcGIS Pro has no problems with a mix of object types, easily connecting to a data source that contains such mixed object types. An SQL query can then on the fly extract separate object types and store those into new file geodatabase feature classes within the ArcGIS Pro project.
Is SQL for ArcGIS Pro a GPU DBMS?
Yes, as most people use the term. A GPU DBMS is a DBMS that uses many GPU cores in parallel to increase the speed of database operations such as the speed of an SQL query engine. SQL for ArcGIS Pro certainly does that, with the added distinction of providing a more complete and more sophisticated SQL than some other GPU DBMS products.
SQL for ArcGIS Pro also differs from some other GPU DBMS packages in that SQL for ArcGIS Pro is not only GPU parallel but also simultaneously runs manycore CPU parallel as well. SQL for ArcGIS Pro will automatically shift execution back and forth between massively parallel GPU and manycore parallel CPU. Depending on the number of CPU cores available, some operations can run faster when parallelized to manycore CPU than dispatched to GPU cores: SQL for ArcGIS Pro's on the fly parallel architecture optimizers choose what is the best mix of GPU parallelism or CPU parallelism or both to use for each task and subtask as it runs.
Some GPU DBMS products run exclusively as in-memory databases and thus require specialized hardware. SQL for ArcGIS Pro runs on standard Windows desktops, using parallelism to increase performance even when disk storage is involved, as it is in real life for most applications. SQL for ArcGIS Pro's mix of manycore parallel CPU plus massively parallel GPU helps achieve maximum speed in real world applications on standard systems.
Is SQL for ArcGIS Pro based on Hadoop or Spark?
No. SQL for ArcGIS Pro is entirely original work from the ground up. Systems like Hadoop and Apache Spark are wonderful software to which SQL for ArcGIS Pro is happy to connect in various settings, but SQL for ArcGIS Pro does not use either for parallel computations. To extract maximum performance on desktop systems SQL for ArcGIS Pro's internal architecture has to integrate manycore CPU parallelism and GPU parallelism with all other parts of the system, such as parallel data access and optimizations for very fast analytics, editing, transformations, and visualizations of big data that reach across the system.
Does SQL for ArcGIS Pro apply DAG or MapReduce?
DAG, given the generality of the term, Directed Acyclic Graphs. SQL for ArcGIS Pro parallelizes big jobs by splitting them into parts so different parts can be done by different computing nodes. Those nodes are connected to each other so they do indeed form a graph and can be analyzed as a graph. Those connections ("edges" in graph theoretic nomenclature) are not symmetric in that info getting passed in either direction over an edge is different: the graph is directed. Last, there are no loops in how SQL for ArcGIS Pro uses that graph so the graph is acyclic. Describing how SQL for ArcGIS Pro internally directs parallel computational flow as anything other than a directed acyclic graph or DAG is inaccurate. But it may not be as useful as a simple description in ordinary language:
SQL for ArcGIS Pro has a sophisticated query engine with a very high degree of modularity. Big tasks get split into small tasks, which get sent to different computation nodes. Some of those nodes, potentially many thousands of nodes, can be GPGPU cores, and those GPGPU cores can be different types of cores from different GPUs. Some of those nodes, also potentially thousands but more likely only a few dozen or few hundred, can be CPU cores. Computational nodes also can be external databases like PostgreSQL. The query engine can split the work between any and all such nodes so that they run in parallel. In addition to the query engine's fundamentally parallel SQL capabilities, SQL for ArcGIS Pro has a big roster of built-in SQL functions, both spatial and non-spatial, which all work within that parallel scheme. 9 also provides a rich set of facilities to add very involved custom functions for execution within SQL using scripts.
Manifold has very rich experience at graph theory: Manifold's very first commercial product over 25 years ago was the General Graph Facilities (GGF) library providing hundreds of programming functions for working with graphs.
Can SQL for ArcGIS Pro Parallelize ArcGIS Pro?
No. When launched, the SQL for ArcGIS Pro add-in executes in a different session from ArcGIS Pro. That allows the add-in to run fully parallel, but it does not change the execution status of ArcGIS Pro, which continues to run as a non-parallel application.
SQL for ArcGIS Pro uses Esri code to connect to file geodatabases, and that Esri code is not thread safe, which puts an upper limit on the performance SQL for ArcGIS Pro can achieve with reads and writes from file geodatabases. Operations nonetheless can go faster because the add-in uses sophisticated in-memory caching strategies as well as making extensive use of the add-in's own, native parallel data store for higher performance, which minimizes read/writes to relatively slower file geodatabase storage.
Analytics can also run much faster because SQL for ArcGIS Pro will launch those as fully parallel processes using all available threads, both CPU and on all GPU cores where it makes sense to utilize as a parallel execution node. Even though the read/write to file gdb is not parallel, the phenomenal speed of parallel analytics using all CPU and GPU cores can result in dramatic speedups compared to execution within ArcGIS Pro without using the add-in.
Does SQL for ArcGIS Pro Require GDAL?
No. SQL for ArcGIS Pro has hundreds of file formats, file databases, DBMS servers, web servers and interoperability standards built in using native, parallelized code, written entirely by Manifold engineering in tight integration with the rest of SQL for ArcGIS Pro, with no need to install or to use GDAL. SQL for ArcGIS Pro's vast collection of file formats and data sources covers all formats that might be encountered even in a lifetime of doing GIS, including all popular formats, and many rarely-met or exotic formats. Most Manifold users never install GDAL.
In addition to the hundreds of built-in formats and data sources, SQL for ArcGIS Pro also provides an optional capability to use GDAL, to connect to any format GDAL knows. For a partial list, see the Data Sources page. That capability adds a few more formats not built into SQL for ArcGIS Pro, primarily formats that are so rarely encountered it makes no sense to add them to SQL for ArcGIS Pro. Adding the hundreds of choices built into SQL for ArcGIS Pro plus all formats known to GDAL results in the widest connectivity range of any desktop GIS.
The option to add GDAL also provides the benefit of getting a "second opinion" on many formats that both SQL for ArcGIS Pro and GDAL know. Users have a better chance of getting more data from some rare, quirky or poorly defined formats with two different mechanisms to extract data. While GDAL is a great thing, having virtually all known formats and data sources already built into SQL for ArcGIS Pro provides much greater convenience, greater reliability, immediate turnaround for fixes on any bugs found in formats, and superior performance. At the same time, having GDAL as an optional second source provides access to even more formats. The result is the best of both worlds.
Is the price an annual rental or a fully-paid license fee?
The price is a fully-paid license. For current pricing see the Products page.
Buy Now via the Online Store
Buy SQL for ArcGIS® Pro on the Online Store. The store is open 24 hours / seven days a week / every day of the year. Orders are processed immediately with a serial number sent out by email in seconds. Enjoy the world's best desktop spatial SQL today!
Manifold® products deliver quality, performance and value in the world's most sophisticated, most modern and most powerful spatial products for SQL, GIS, ETL, DBMS, and Data Science. Total integration ensures ease of use, amazing speed, and unbeatably low cost of ownership. Tell your friends!