Google Online Mapping Tools: Planning Advanced Mapping

In TechSoup's companion article A Nonprofit's Introduction to Google's Online Mapping Tools, we provide an overview of the more basic features and uses of Google Maps and Google Earth, and offer guidance for organizations that are considering embarking on a mapping project.

Yet while Google Maps and Google Earth offer powerful, built-in capabilities that don't require advanced technical skill or programming ability, the type of map your nonprofit is envisioning may require advanced, customized functionality.

Below, we'll introduce you to two related toolsets Google has designed for programmers to accomplish just that. First, software and site developers can access Google's map-related services in their scripts by writing calls to the Google Maps APIs. Second, Google has extensively documented the KML file format that Google Earth, Google Maps, and other applications rely on for saving and sharing map content. These toolsets and their uses are described in more detail below.

Key Definitions for Advanced Mapping Projects

Mapping applications rely on a few technologies and protocols that you are unlikely to encounter in many other programming or development projects. Many articles and online discussion forums about digital maps assume that you have a basic familiarity with terms such as KML, mapping API, and mashup; below, we'll define these terms to help you understand how they relate to your project.

Geographic Data

Geographic data is any data (a row in a spreadsheet, a record in a database, or a paper record) that contains information about the physical location of the object or person it represents, regardless of how that location is specified. For example, your donor database probably specifies the street address, city, state, and ZIP code of each of your donors. Each donor record is a geographic data point, while the records as a whole make up geographic data or a geographic data set. All mapping projects rely on geographic data, whether they use Google services or other digital mapping tools.

KML

KML (or Keyhole Markup Language) is a file format used to store, distribute, and display geographic data for use in a virtual globe application (like Google Earth) or a Web mapping application (like Google Maps). KML files save structured metadata about geographic data using an XML schema. You can save geographic data in many other file formats (such as a text file or an Excel file), but KML is the format preferred by Google Maps and Google Earth. Not only is it easy to read relative to most file formats, experts on Google's mapping products tend to know more about KML than other file formats. Some people still associate KML files solely with Google Earth, since the standard was originally created for use with that application. However, it's now an open-file format standard maintained by the Open GeoSpatial Consortium, so KML files can be read by Microsoft Virtual Earth 3D, NASA World Wind, Live Search Maps, and many other mapping applications.

Google Earth and Google Maps both let you save your custom map content (placemarks, annotations, routes, and so on) as a KML file, and both will read the KML files made in the other program. You don't need any programming background to do this, and anyone can share a KML file with other users by posting it on their Web site or uploading it to sites popular with Google Earth fans (like the Google Earth Community and the other sites listed in our Further Resources section below). End users visiting your Web site won't see a map in their Web browser when they click on the link that to your KML file. Instead they'll see a File download window that gives them the option of saving the file or opening it in an application of their choosing. Therefore, the page on your Web sites that links to the KML file should include a brief explanation of Google Earth and a link to its download page, as well as an explanation of why you're sharing this KML file and what sort of map content it contains.

Though you can create or modify a KML file from the Google Earth interface the same way you'd create or modify a Word file from within Microsoft Word (File Save As), you can also edit a KML file directly in a text editor. You might need to do this in order to troubleshoot the position of a placemark or some other feature that isn't working the way you expected. The KML documentation is clear enough that in many cases you can take a KML file created by Google Maps or Google Earth, follow the instructions in the documentation, and edit the KML file in a text editor until it suits your needs. If you come across a term in the KML documentation that you're unfamiliar with, it's likely to be a concept related to XML (eXtensible Markup Language), which is KML's parent language. (To learn more about XML and KML, read w3schools.com's What Is XML? and watch Google's online video Quick and Dirty KML Creation.)

Mapping APIs

An API, or Application Programming Interface, is a set of services and functions that one software application makes available to other applications. When a programmer needs to develop a complicated function, he can often include a few lines of code in his program referencing another application and asking that application to do the hard work. That second application is often hosted by another organization and accessed over the Internet. And it's only possible if the programming team responsible for the second application wrote an API into their software and published documentation telling other developers how to access that API. A mapping API is an API that makes customized, dynamically generated maps available to other applications.

When the software developers at Google created Google Maps, they built several mapping APIs into it and made the documentation for them publicly available. When another programmer writes a Web application that requires some mapping functionality, she can add some commands telling that application that it should request a map from Google's server whenever a certain event occurs. For example, when Goodwill's developers created the Goodwill Locator, they added code to that Web page that causes their Web server to perform a series of actions whenever someone types in an address and clicks Submit. First, Goodwill's server sends a request to Goodwill's database asking for the exact location of all the nearby stores, donation centers, and drop-off facilities. The Web server then sends a second request (also known as an API call) to one of Google's mapping servers asking for a customized map. Among other things, this second request contains information about the size of the desired map, the zoom level (that is to say, the level of detail), the address specified by the end user, and the location of nearby Goodwill locations. Google's server sends the requested map back to Goodwill's Web server and the Goodwill Web server embeds the map inside a Web page and sends it back to the end user. The developers at Goodwill had to understand at least three different pieces of software: The Web browser, the database, and Google's mapping service. In other words, the developers had to know the specific programming commands that would elicit the responses they wanted from these applications.

Mashups

When Web application developers make use of databases and APIs in the way described above, the end product is often referred to as a mashup. A mashup takes data and software functionality from two or more sources and integrates it all in a single, seamless Web application. Usually one of these sources is an open, freely-available API such as the Google Maps API. The second data source might be another API, or a locally hosted database, or both. Google Maps mashups are mashups that combine mapping data from Google with dynamic data pulled from other sources, such as a database of restaurants and food banks (for example, the NOLA Food Map ), or a data source with information about toxic spills (like Planet Hazard ).

Questions to Consider When Planning a Digital Mapping Project

When planning an advanced mapping project - which may require large amounts of data and coordination between various players - it's important to assess your existing data and resources. This can not only help you best identify the mapping tool or tools you'll need to use to create your project, but also the ongoing costs and requirements you're likely to encounter.

• Do you have existing geographic data that's digitized, well-structured, and available in a widely accepted, standard format? A list of names and addresses on an event sign-up form qualifies as geographic data, but it's not digital data. The same list in an email document or a Word document is geographic data and it's also digital, but it's not well-structured. (When a computer reads an email message, it isn't clear where a person's name ends and where his or her address begins.) The same information stored in a donor database is probably well defined and well structured, but it might not be available in a widely recognized, standard format. Certain databases and applications store data in proprietary formats, making it hard to share that data with other applications. However, that happens less often than it used to. Most databases and spreadsheet applications nowadays can export data as a .csv file or respond to outside queries according to certain standard protocols (like ODBC ). Several of the tools and techniques described in this article rely on consistent, well-structured, standardized data. If your data isn't consistent and well-structured, you might be able to copy and paste the data into an application that supplies the needed structure (like a database or a spreadsheet application), though that can be tedious and time-consuming if you have a lot of data. If you don't have any geographic data at all, you can still start a mapping project, but you'll need to spend time up front thinking about how you'll collect the data, store it, and ensure its quality. For more information, see the section below on How to Create a Google Mapping Project without Existing Data.
• Do you fully understand the significance of your data set and your reasons for displaying it in a map? If your organization was given your data, or if it was collected by someone who no longer works with you, spend some time understanding it before proceeding any further. It's OK if you don't grasp the technical specifics, but you should know what the data describes.
• Do you have permission to use the data in the way you're planning? It may seem obvious, but it bears repeating that information that might be used to identify and locate individuals should be treated with care.
• Will your maps rely on dynamically generated, real-time data, or data that's fairly static? Dynamic data is data that changes frequently in response to updates and input from end users. Therefore, data queries and manipulations have to occur automatically. On the other hand, static data changes infrequently or not at all. If your map relies on demographic information that gets updated once a year, you can easily import that into Google Maps or Google Earth yourself. You don't need to write a program or script to automate the process.
• Will you present your data in Google Earth, Google Maps, or both? The main difference is that Google Earth projects are more likely to rely on the generation of KML files, while large Google Maps projects often rely on scripts that call the Google Maps APIs. (For more information on how the two applications differ, see A Nonprofit's Introduction to Google's Online Mapping Tools.)
• Are you using a large data set or a small data set? For projects with a small data set (30 service locations or fewer), you're often better off entering the data directly into Google Maps or Google Earth using the graphical interface to add placemarks to your map. If you need to share the map, you can save the content as a KML file. Google Maps also lets you link directly to your customized map. (For more information on this approach, see A Nonprofit's Introduction to Google's Online Mapping Tools. For the purposes of this article, we'll focus on projects that use large data sets or dynamic data sets.)
• What skill sets can you draw on? In this article, we concentrate on the technical and programming skills needed for advanced mapping project. However, for some projects, you'll also need to find experts in graphic design, Web design, or Geographic Information Systems to create and publish your map.

How to Plan a Google Mapping Project that Relies on Existing Sources of Data

Oftentimes, your mapping application will rely on geographic data that you're already collecting for some other purpose. In other words, you might have a spreadsheet with your service locations or a database with information about human rights abuses. In other cases, you can build a mapping application partially or entirely around data that some other organization is collecting and making available to you. Governments, universities, nonprofits, and businesses provide a huge array of geographic data sets, sometimes for a fee, but often for free. Geospatial One Stop and the Geography Network are online directories that describe and classify publicly available data sets.

In this section, we'll discuss ways to create mapping projects with existing data. First, we will describe the process of creating static maps with large data sets (i.e. maps where the underlying data changes infrequently or not at all). Then we will briefly describe the process of creating dynamic mapping applications that change based on user input or the latest information.

Strategies for Mapping Projects that Rely on Existing Data Sets

The development techniques and tools that you use for your mapping application will depend on a few key factors such as which application you're using and whether the maps you generate will change frequently or not. This table provides a brief summary of the strategies that we describe in greater length below.

	Projects with Dynamic Data Sets	Projects with Dynamic Data Sets
Projects that use Google Earth	Import your data set into Google Earth Pro and then save the resulting map as a KML file, or use one of the other file conversion tools described in the next section.	Develop a small application or script that generates KML files dynamically based on user input or database queries, as explained in the video Dynamic KML
Projects that use Google Maps	Import your data set into Google Earth Pro and then save the resulting map as a KML file.	Develop a small application or script that generates maps dynamically based on calls to the Google Maps APIs, end user input, and database queries.

Projects that Rely on Static Data

If you're already collecting data and it doesn't change very often, or the changes don't have to be displayed in real time, importing the data into Google Earth or Google Maps is often the easiest approach. Once you've imported the data set, make sure that all the map content appears where you expect it to, and then share the resulting map. With Google Maps, sharing usually consists of distributing a link to a Web page containing the map. With Google Earth, you'll usually distribute a link to a KML file that you've uploaded to your Web server.

Nonprofits frequently collect and store geographic data for a purpose (like donor tracking) other than use in a map or mapping application. Moreover, nonprofits often receive their geographic data from another organization (such as a government entity), or another department within their organization. In these cases, it's rare that the application that stores the data will be able to export in the KML file format. However, as long as the application or database supports a widely recognized file format (for example, the .csv format) there's a good chance that Google Earth Pro http://earth.google.com/enterprise/earth_pro.html (available for free to 501(c)3 nonprofits) can import that file and convert it to KML. In fact, Google Earth Pro recognizes 14 different data formats and can convert them all to KML. If Google Earth Pro doesn't recognize the format you've used to store your data, or if you don't have a license for Google Earth Pro, there are many other tools that turn existing geographic data sets into a KML file. In price these tools range from free to expensive. Some examples are:

• The Spreadsheet Mapper is a useful free tool that might help to speed up the creation of KML files by letting you paste large volumes of geographic data from an existing spreadsheet. It can also help you collaborate with others who want to help develop your map.
• GDAL/OGR  (Geospatial Data Abstraction Library) is a suite of free command-line tools for converting all sorts of data to different formats. It is difficult to use and not well documented, but it is very powerful.
• Zonum Solutions Software. Zonum makes a number of free applications, including Shp2kml, Kml2shp, and EPoint2GE, which convert .shp files to KML, KML to .shp, and Excel to KML.
• Microsoft's MapCruncher was designed to convert image files for Microsoft's Virtual Earth. When combined with CrunchUp2KML, it creates KML files. Both MapCruncher and CrunchUp2KML are free for non-commercial use.
• KML2KML has a number of capabilities, including conversion of GIS data to KML and optimization of KML files. It is very powerful and fairly easy to use. Different versions cost between $50 and $195.
• Arc2Earth. A plug-in for ArcGIS, it provides a powerful set of features for data conversion to and from GIS files, optimization of your outputs, and online storage of your data. Educational and nonprofit prices start at $152.

For more free tools, check out FreeGeographyTools and click on the KML, Google Earth, and Google Maps categories.

Projects that Rely on Dynamic KML Files

It's possible to create KML files dynamically by pulling changing, up-to-date information from a database, a Web form, or other data sources. To accomplish this, you will need a Web programmer who understands a scripting language such as Javascript or Perl. Your programmer will also need to understand your database and how to query it programmatically (that is to say, using a scripting language or programming language). It also helps a great deal if the programmer has a solid understanding of XML, KML, or both. XML (or eXtensible Markup Language), is KML's parent language, and knowledge of XML makes it easier to read the KML documentation and the KML files themselves.

Projects that Rely on the Google Maps APIs

To make use of the Google Maps API, you will need a Web programmer who knows Javascript or Flash. It's also likely that the programmer will need to understand database programming in general as well as the structure of the specific data sources that you're relying on for your project.

Google currently makes five map-related APIs available to Web-application developers. The Google Maps API is the most widely used of these APIs since it allows you to access most of Google Maps' functionality and embed the resulting maps in your Web site. Moveover, the Google Maps API is written in the Javascript programming language, and knowledge of Javascript is widespread among Web developers, meaning you won't have trouble finding skilled developers to create and maintain an application written in Javascript. The Google Maps API for Flash is an alternative to the Google Maps API for organizations with the necessary knowledge and resources. Adobe Flash is a set of tools and standards that lets you integrate multimedia content (such as audio, video, and animation) into Web applications, mobile phone applications, and desktop applications. Development of Flash content requires Adobe Flash Professional, which costs $699 retail, or $160 if you get it through TechSoup as part of the Adobe Creative Suite Design Premium.

• Google Maps API is a Javascript-based API, and it's the one most frequently used to create applications with Google Maps. It lets your Web application request maps in real time based on input from the end-user and data that you pull from other sources (for example, your own database detailing the number of free meals you've served, or a database of toxic spill information hosted by the federal government). Once Google's servers respond to your request, your own Web server will instantly and seamlessly embed the customized map into a Web page and pass it along to the end user. You can also use this API to embed Google StreetView  into your Web site. StreetView lets people look at panoramic photographs of a neighborhood taken from street level. Again, the Google Maps API will send the requested images to the end user in real time.
• Google Maps API for Flash is similar to the JavaScript API, but relies on the Adobe Flash Web browser plug-in. It is a bit more complicated and expensive to build with because it requires some Flash tools that Adobe puts out. It makes up for this by being a little more powerful because it can take advantage of the Flash Web plug-in. Most of the functions you'll find in the Google Maps API are also available in the Google Maps API for Flash.
• Static Maps API is a straightforward, lightweight programming tool for grabbing a simple image of a map, maybe with some location markers on it, and embedding it in your Web site. This is a non-interactive map, the equivalent of putting a picture on your site. End users can't grab the map and move it around, or zoom in and out, but his lack of interaction also means that the map is smaller and loads faster, an important consideration if your clients access the Internet over a slow connection.
• Google Mapplets API. Mapplets are small programs that run inside the Google Maps service itself. Though similar to normal map applications, they are stored on Google's servers, and end users access them from the Google Web site. With map applications built on the traditional Google Maps API, end users visit your Web site to view the maps, and the code resides on your Web server. After you or your developer finish coding a mapplet, upload it to the Mapplets directory. From there, anyone with a Google account can click “Add it to Maps” to add your mapplet to the version of Google Maps that they see when they're logged on. For example, this Gas Prices Mapplet overlays the location of nearby gas stations along with current gas prices. The Mapplets directory is fairly new and most end users are unaware it exists, so think about who you're trying to reach before you spend resources on this approach.
• Google Earth API Like Google Maps, Google Earth provides an API that lets you embed Google Earth into your Web site. The maps and images you pull from Google Earth will have the same rich, three-dimensional look and feel that they have when you view them in the Google Earth application itself. The JavaScript API gives you complete control over Earth. The plug-in can also load any KML files you built for the stand-alone Google Earth. Users will have to install a free plug-in for their Web browser in order to view the maps that you pull into your Web page with the Google Earth API. To program with the Google Earth API you need to understand Javascript and KML.

How to Plan a Google Mapping Project without Existing Data

If you have lots of existing geographic data, you're best off using a script or a conversion tool to automatically generate your map content, as outlined above. However, in some cases the geographic data you need has never been captured, or never captured in a consistent, standardized, computer-readable form.

If you're undertaking a project that will rely on large amounts of newly gathered geographic data, you need to talk with the project stakeholders and ask them questions about how they plan to collect and use the data. This type of large-scale project may involve the purchase, acquisition, or creation of a database application for data storage and retrieval. For small projects where the geographic data doesn't exist yet in a standardized, digital form, you can usually enter the data directly using the graphical interface of Google Earth or Google Maps, as described in A Nonprofit's Introduction to Google's Online Mapping Tools.

Further Resources

If you're a programmer or are interested in learning about map programming, check out the Google Maps API reference site, the Google Geo Developers blog, the Google Maps API Demo Gallery and the Google Earth API Demo Gallery

While the sites below aren't affiliated with Google, they provide technical tips, news, and innovative applications for advanced users of Google Earth and Google Maps. You don't need a programming background to understand most of the postings, but the discussions are primarily intended for advanced users of Google Maps and Google Earth.