Collecting Data in the Field

Manifold makes it easy to collect data in the field via mobile devices or the web, even when disconnected from Internet, by providing tight integration with the KoBoToolbox ecosystem.   That provides a complete solution for creating, sharing, and analyzing surveys and data collected in the field using forms, including smart forms with skip logic, defaults, and support for multiple languages.   

 

Manifold leverages KoBoToolbox, Enketo, ODK (Open Data Kit), and W3C web standards to enable access to a very wide ecosystem of resources.

 

How to collect data in the field:

 

  1. Visit the KoBoToolbox site and sign up with a free account.  Note the API key for your account.
  2. Create a form for your project.
  3. Deploy the form.
  4. Install the KoBoCollect app on your Android phone, or use a web browser to connect to your project on non-Android devices.
  5. Collect data.
  6. In Manifold, create a data source using the Web Server: kobo choice, entering your API key in the dialog.  (Save as a Favorite Data Source for one-click connections in the future.)
  7. The data source will show all of your KoBo projects, with tables and drawings you can use.
  8. Update the data source with any new data by right-clicking it and choosing Refresh.

 

 

See a step by step example of creating KoBo forms and collecting data in the Example: Create and Use a Survey Form topic.   See a step by step example of connecting to KoBo servers and using data collected in the Example: Connect to Data Collected in the Field topic.

 

See the video versions of those topics in the 10 Minute Tutorial - Create and Use a Survey Form and 10 Minute Tutorial - Connect to Data Collected in the Field videos.

ODK Standards

ODK is an extensive open source ecosystem for data collection using forms that take advantage of W3C web standards like Xform using XLSform.  Using collection apps on mobile phones for completely offline data collection or web forms in any browser on any mobile or desktop device that is connected to the Internet, a very wide variety of data types, including locations, georegistered lines or georegistered areas, can be collected in the field with data automatically being synced to servers when a connection is found.

 

ODK technology is used by a very wide variety of open source and commercial packages, including Esri's Survey123 applications.  Many Esri XLSforms for Survey123 will immediately work for Manifold field data collection.   The nonprofit organization behind ODK hosts paid servers that, for a fee, provide server side storage of data acquired using ODK tools.

KoBoToolbox

KoBoToolbox is a free, highly refined, easier to use packaging of ODK technology that retains full compatibility with ODK standards.   KoBoToolbox provides a complete solution based on ODK that provides a very well integrated set of tools for creating forms, deploying forms to mobile and web devices, collecting data using custom Android KoBoCollect application or Enketo web forms in any browser, and automatically syncing and storing data on KoBo servers.  

 

KoBo provides three server options, all of which are free, to host data collected using KoBo: one option is for humanitarian organizations and provides free, unlimited storage for an unlimited number of projects.  Another option provides free server storage for everyone else and all other organizations, for up to 10,000 submissions per month with up to 5GB of data storage per month and unlimited number of projects per account.   The third option is for more advanced users, who can install KoBoToolbox on their own server or a local machine to operate their own KoBo data storage.    As a practical matter, collecting 10,000 data points and 5GB of data per month per account is far more than most organizations will ever need.

 

It is free and easy to sign up for a KoBo account and then all form building can be done online using KoBo's online form builder.   We can also modify forms using classic XLSforms to either create forms from scratch or to edit forms created with KoBo or ODK in Excel or other spreadsheet package, and then upload the XLS file to update our forms in KoBo.

Manifold Integration

Manifold uses KoBoToolbox for field data collection because it provides capabilities comparable to Esri's Survey123 using standards supported by a very wide ecosystem of commercial and nonprofit resources.  The only thing missing compared to Survey123 is "out of the box" tight integration with GIS.   

 

Manifold adds that missing capability by providing a KoBo web server data port that uses KoBo's extensive API to connect to your KoBo account on KoBo's servers, automatically linking within Manifold your various KoBo projects and the data collected in those projects.  Manifold automatically creates drawings when data collected includes geopoints (point locations), geotraces (lines), and geoshapes (areas).

 

Choosing KoBo for integration also takes advantage of very easy form building within KoBo with many features, as well as free storage of server side data for most Manifold users.  KoBoToolbox also provides many server-side features for analysis and presentation.

Benefits

From the KoBoToolbox web site:

 

Form Builder

 

 

Collect Data

 

 

Analyze and Manage Data

 

 

Resources

There are very many web sites that provide information, tutorials, forums, examples, and other resources for data acquisition using ODK based solutions like KoBo.  KoBo Toolbox is based on ODK, so browsing ODK can illuminate more obscure or advanced capabilities within KoBo.

 

 

 

 

 

 

 

 

 

 

 

 

Notes

Alternatives to Google - The KoBoCollect client app for Android telephones is normally downloaded from Google Play, but for that one fourth of the world's population in China and elsewhere subject to US sanctions that may block access to Google Play, generic Android versions of KoBoCollect may be downloaded from alternative sites such as Huawei's AppGallery.  One additional factor is that even the generic Android versions may be tied to Google's Play Services, which prevent the geotrace (collect lines) and geoshape (collect areas) widgets from working, allowing only collection of geopoint (point) location data.   A better alternative is to simply use a web browser on a smart phone to launch a web form, the same method used with Apple and other non-Android phones, to collect data using the web form automatically generated by the KoBoToolbox form builder.

 

Can data collection be done without an Internet connection?   Yes.  The KoBo toolset was designed for use in challenging, humanitarian response situations, where Internet access is often poor or nonexistent.  All data collection can take place offline, both with KoBoCollect and with web forms.   See this article in the KoBoToolbox documentation.

 

What does KoBo mean and how is it pronounced?  Kobo means transfer in the Acholi language, which is a Southern Luo dialect spoken by the Acholi people in northern Uganda.   The "o" is pronounced the same as the "o" in hello.   The KoBoToolbox organization capitalizes the "b" in kobo, so Manifold does that as well when referring to the organization or brand, with lower case "b" used otherwise.

 

What is Enketo?  ODK technology uses the Xform standard, which has been part of the W3C web specification for a long time.   However, none of the popular browsers include built-in support for Xform.  They rely instead on form engines, like Enketo, to process Xforms that define survey forms.  Enketo is hugely popular in both commercial and non-profit apps.  

 

Enketo Core is a library that contains Enketo's form engine. It is not an app by itself but a minimal building block that can be used to create Enketo-powered apps.   Enketo Express is one of the apps that is powered by Enketo Core.

 

Esri uses Enketo Core as the form engine that powers Esri's Survey123 for ArcGIS web application. Esri was a sponsor of Enketo's libraries and paid for some features and improvements in the past.   Enketo Express is an integral part of KoBoToolbox and KoBo has historically sponsored a large chunk of Enketo Express' development. It still provides code, feedback, emotional support, and funding for features and bug fixes.

 

What does Enketo mean and how is it pronounced? According to enketo.org: "Enketo means survey in Esperanto.  We are not sure how to properly pronounce it in Esperanto but you could say [\in-ˈke-tō]. The first "e" and second "e" should be pronounced the same because each letter has only one pronounciation in Esperanto."  Roger that.  Enketo is a cool thing however you pronounce it.

 

Videos

10 Minute Tutorial - Create and Use a Survey Form - The first of two videos showing how to collect data in the field with Manifold Release 9.  We create a new survey form using the KoBoToolbox ecosystem, the same ODK/Enketo based technology used by Esri's Survey123.  We create a form to collect information and locations for restaurants in Chartres, France.  We use the form to capture data for a restaurant, automatically syncing collected data into our Kobo server.  The next video shows how to connect Manifold and automatically harvest collected data.  

 

10 Minute Tutorial - Connect to Data Collected in the Field - The second of two videos showing how to collect data in the field with Manifold Release 9. The first video showed how to create a survey form and to use that to acquire data for restaurant locations. This video shows how easy it is to form a live connection to the KoBo server storing our survey data, automatically linking tables and drawings into our Manifold project.  We create a map of restaurants surveyd in Chartres, France, using a satellite layer for background.  We add labels and see how when we add more data using mobile or other devices, we can refresh to automatically capture those new locations in the Manifold map.   Works in the free Manifold Viewer, too!

 

See Also

File - Create - New Data Source

 

KoBo Servers

 

Example: Create and Use a Survey Form - Create a new survey form on the KoBoToolbox site, and then modify the form to provide a starting view for map widgets used to collect locations.  Use the form to collect data in the field, automatically syncing collected data into our KoBo account on the KoBo servers.

 

Example: Connect to Data Collected in the Field - Use the Web Server: kobo dataport to connect to KoBo servers, using your KoBo account's API key to automatically authenticate your connection to your KoBo projects on the server, with automatic linking of data from those servers as tables and drawings in the Manifold project.  Use KoBo data to create a map of restaurants surveyed in Chartres, France.   Add new data and refresh to see new locations appear in the Manifold map.