Lab 3: Import and mapping of field data
- Retrieving your data from the GPS receiver
- Differential corrections of GPS data
- Display corrected data and export to a dBase file
- Apply Tree offsets in Excel
- Importing tree data into ArcGIS and visualization in Google Earth
- Creating plot points and visualization in Google Earth
- Creating plot lines
- To turn in
In this lab you will learn how to import your field data from the GPS receiver and perform differential corrections. You will then import these data into a Geographic Information System (GIS) and export the data for visualization in Google Earth.
2.0 Retrieving your data from the GPS receiver
- Connect your GPS unit to a lab computer with the USB cable.
- Open GPS Pathfinder Office 4.2 from the Start Menu. Start->Trimble->GPS Pathfinder Office->GPS Pathfinder Office
- When the "Select Project" window opens, chage the project folder to "C:\Workspace". Give the project a name, like "Geog315_Lab3_Group1".
- Data from your GPS can be transfered with the Data Transfer utility. Go to Utilities->Data Transfer. A new window should pop open and a connection should be established with the GPS receiver (It should say "GIS Datalogger on Windows Mobile " under Device).
- Be sure the "Receive" tab is selected. Then click the "Add" button and select "Data File". The data file(s) on the reciever are displayed. They have a ".ssf" file extension and are called "rover files" by Pathfinder Office. There may be more than one file depending on how many files you created during Lab 2. To be sure we get all of the information on the receiver, go ahead and select all files by holding down the CTRL key and clicking the files. Then click "Open".
- Click the Transfer All button to begin transfering the data from the receiver to your computer. Click Close when the transfer has finished.
- Click Close on the Data Transfer window to close the tool.
3.0. Differential corrections of GPS data
Now we will post-process the GPS files with differential corrections from a nearby base station.
- Go to the Differential Correction tool under Utilities. This opens a "wizard" to walk you through the process.
- Verify that your GPS receiver file(s) that you downloaded in Step 2.0 is listed in the "Selected Files" window of the Differential Correction window. If not, then find the file(s) by clicking the "+" button on the right. By default, files are being stored in the folder you set when starting up the program (e.g., C:\Workspace).
- Remove any files that you do not want to process with the "X" button on the right.
- Click Next to go to the next screen.
- Select the dafault option of "Automatic Carrier and Code Processing (Recommended)". Click Next.
- Leave all of the default settings on this screen ("Output corrected positions only", "Smart automatic rover filtering", and "Re-correct real-time positions", Click Next.
- Under Base Date, Base Provider Search, click the Select button to find a base station on the Internet.
- You should have a window that looks like the one below...
- There are choices for CORS and UNAVCO stations. The Sonoma State University base station is offline. Note that the distance from our receiver files is listed in the distance field.
- Select the CORS Petaluma base station (11 km distance). Click OK.
- "CORS, PETALUMAIRCN2004 (P198), CALIFORNIA" will appear in the Base Provider Search window.
- Under Reference Position, be sure that the option "Use reference position from base provider" is selected. Click Next.
- For Output Folder, select " Use the project folder". For Output Filename, select "Use original filename, overwritting any existing .cor file".
- Click Start. You shoudl see a window with output that looks similar to the follow:
Position: 38°15'35.54258"N, 122°36'26.80768"W, -4.51 m
Source: C:\Workspace\Base\CORS, PETALUMAIRCN2004 (P198), CALIFORNIA
Local time: 3/5/2010 8:59:46 AM to 3/5/2010 11:59:46 AM
Position: 38°15'35.52854"N, 122°36'26.76290"W, -4.09 m, 0.01 m Antenna height
Distance from base provider: 1.17m
Rover file: R030510A.SSF
Local time: 3/5/2010 9:31:00 AM to 3/5/2010 10:39:30 AM
100% total coverage
100% coverage by p19806410173.zip
Differential correction settings:
Use smart automatic filtering: On
Re-correct real-time positions: On
Output positions: Corrected only
Processing rover file, CLEVELAND 4.SSF ...
...to output file, C:\Workspace\R030510A.cor
No carrier processing performed as file has no carrier data
Corrected 0 positions
Selected 392 positions for post-processing
Corrected 392 positions
Differential Correction Summary:
1 file processed. In this file:
392 (100.0%) of 392 selected positions were code corrected by post-processing
0 (0.0%) of 0 selected positions were carrier corrected by post-processing
Estimated accuracies for 392 corrected positions are as follows:
Differential correction complete.
- Click Close. You have now performed post-processing differential corrections to your field GPS data!
- Note: Differentially-corrected rover files have the extension ."cor"
4.0. Display corrected data and export to a dBase file
- Open one of your uncorrected rover files (.ssf extension) by clicking Open. Browse to the C:\Workspace directory to find your uncorrected and corrected rover files.
- Select View->Map to turn on/off a map of your points. If the map screen opens but there are no points displayed, then something is wrong with your points file.
- Righ-click on a point feature and select "Tree Properties" ... or "Plot point Properties". You should see the attributes for a point. Your data will not look extactly like this screen shot.
- We will now export your rover file. Select Utilities->Export. Under Chose Export Setup, select "Sample dBASE Setup". This will export the data in a very generic, dbase format that we can open in Excel or other spreadsheet program.
- Click the Properties button, then select the Coordinate System tab. Select the "Use Export Coordinate System" radio button. Click the "Change" button.
- We're going to export our data in the UTM projection, Zone 10 with a WGS 84 datum. Select the appropriate parameters. Your window should look like the following screen shot.
- Now we will select which attributes to export with the points. Go to the Attributes tab. For "All Feature Types", check the boxes for: PDOP, Correction Status, Date Recorded, Time Recorded, Total Positions, Filtered Positions. Under "Point Features", select Horizontal Position and Point ID.
- On the Output tab, select the radio button for "Combine all input files and output to the project export folder". This will combine all points from your rover files into one file for export.
- In the Position Filter tab, be sure that "Uncorrected" is checked.
- Click OK to return to the main Export window.
- We will export both corrected and uncorrected points. Click the Browse button and select all of your .ssf and .cor file.
- Direct your output to "C:\Workspace\Export"
- Click OK to begin exporting the files. If there is a message with a warning about overwritting files, just click Yes. The output will be written into 2 files, Tree.dbf for tree points, and Plot__Po.dbf for plot points. Note that both corrected and uncorrected points are combined in the same file for trees and plots, respectively.
- There will be a Export Completed window with a report of the number of features exported. There should be several features exported. If it says that no features were exported, then there is a problem.
- Use Windows Explorer to make a new work folder in c:\Workspace and call it what you want -- e.g., field_methods_group5). Browse to the C:\Workspace\Export folder. Copy the Tree.dbf and Plot__Po.dbf to a working directory in your work folder in C:\Workspace.
5.0. Apply Tree offsets in Excel
We will now load your point data in Excel and apply the offsets from distance and azimuth measurements.
- Start Excel and open the Tree.dbf dBase file . You will have to change the file type to "all files" or ".dbf".
- Immediately do a File->Save As and save your file in your working directory as a Microsoft Office Excel Workbook (.xls) file.
- Select all the cells by clicking the box in the corner between Column A and Row 1. Then select Fomat->Columns->AutoFit Selection. This should resize all of the columns to the width of the data.
- After the Azimuth column, insert 4 empty columns by selecting the column to the right of the Azimuth column and then do a Right-click, then Insert four times.
- Name the columns Declination, Radians, New_X and New_Y. For the Declination column, type in the magnetic north declination, 14.633333 (degrees) for the tape measure points, where we used the analog, hand-held compass. For all other points (e.g., GPS under tree, GPS control - laser) just enter 0. That is, we only needed the azimuth declination set for the hand-held compass measurement.
- We will create a column that both adds the declination to the azimuth and converts the resulting degrees measurement into radians, which is needed by Excel in trigonomic functions. In the Radians column, first empty row, enter in the formula "=radians(", then click the Azimuth cell on the same row, click the "+" key, then click the Declination cell on the same row. Finalize the formula with the end ")". Copy this formula down to the other rows in the Radians column. See the screen shot below, or ask your instructor for help if needed.
- Now we will calculate the New X and Y coordinates with the Horizontal Distance and Radians data. This will create a new X and Y coordinate at the distance and azimuth from your starting GPS points. Since the distance for the GPS point under the tree is zero, the X and Y for that point will be the same. The New_X formula is UTM Easting + SIN(Radians) * Horizontal Distance. See the screen shot below. In this example, the formula is "=B2+(SIN(H2)*E2)". Copy this formula down to all other rows in the New_X column.
- The formula for the New_Y is similar to X, but you use a Cosine function and add the offset to the UTM Northing -- "=C2+(COS(H2)*E2)". Copy this formula down to all other rows in the New_Y column.
- We're done! We now have new UTM Easting (X) and Northing (Y) coordinates that are calculated from your GPS points, distance and azimuth measurements from the field. Save the Excel workbook and close Excel.
6.0. Importing tree data into ArcGIS and visualization in Google Earth
We will now create point layers in a common GIS file format, the Shapefile. We'll do this in the ArcGIS program by ESRI. This is the software used in our Intro and Advanced GIS courses and is considered the industry standard. Shapefiles can be loaded by many other types of GIS software. We will also learn how to export Shapefiles into a KMZ format that is used by Google Earth, a popular -- and free -- geovisualization tool (also called a geobrowser).
- Start by opening ArcMap from Start-> All Programs->ArcGIS->ArcMap
- We will load our Excel table worksheet into ArcMap. Click the add data button and browse to your working directory in C:\Workspace. Open the Excel file with your tree coordinates... worksheet should be called Tree$
- You should see your worksheet loaded in the Table of Contents (TOC) to the left, below the Date Frame called "Layers" -- see screenshot below. It is ok if you do not recognize the terms used to describe the parts of ArcGIS. If you take Geog 387, you will learn this!
- Now open the attributes for the table to verify that ArcMap is viewing your Excel table. Right-click on the Tree$ in the TOC and select Open. You should see the New_X and New_Y columns. Close the table when you are satifsified that you are viewing the proper table.
- Now we will make a geospatial layer out of your UTM coordinate data in the Excel table. Go to Tools->Add XY Data. Chose your Excel table (e.g., Tree$) for the table and then use the pop-up menus to chose New_X and New_Y for the X Field and Y Field, respectively.
- We'll now define the coordinate system. Recall that we exported the data from Pathfinder Office in the UTM projection, Zone 10 North, WGS84 datum. Click the Edit button, then Select, then browse to "Projected Coordinate Systems"->"UTM"->"WGS 1984"->"WGS 1984 UTM Zone 10N.prj". Click the Add button, then click OK.
- Unclick the box next to "Warn me if the layer will have restricted functionality. Your Add XY window should look like the following screenshot.
- Click OK. You should see points in the map display part of ArcMap. Notice that a layer appeared in the TOC called Tree$ Events with a circle symbol. This is a temporary layer with data being pulled directly from your Excel worksheet.
- We will now export the temporary layer to a permanent Shapefile format file. Right-click on the Tree$ Events and select Data->Export Data. Use the browse button to direct the output to your work directory and call the output Shapefile "trees_group#.shp".... where # is your group number. Leave the default parameters. See screenshot below. Click OK.
- Click Yes when asked if you want to add the layer to the map. You should see the Shapefile added to the TOC.
- Right-click on the Shapefile in the TOC (e.g., trees_group1.shp), then select Properties. This will bring up options to look at and change the Shapefile's properties. We will modify the symbology to show differentially-corrected (i.e.., post-processed) and uncorrect points as different colors. We will also turn on labels.
- Click on the Symbology tab. Then click on Categories. In the Values field pop-up menu, select Corr_Type. This is the attribute column that records whether the point was "uncorrected" or "postprocessed code". Click the "Add All Values" button.
- Double-click on the Postprocessed Code circle symbol to open the Symbol Selector window. Then pop open the Color palette and select the green color "Quetzal Green" -- the names of the colors appear when your cursor "hovers" over the color in the palette. Increase the symbol size to 6, then click OK. Repeat this step for "Uncorrected" but change the color to "Mars Red". See screenshot below.
- Unclick the box next to "<all other values>".
- Now click on the Labels tab of the Properties window. Click the box next to "Label features in this layer". Pop the label field to "Survey_met" (it may be selected already as the default). Leave all the other options in their default state and click OK.
- You should see 6 points in your map display -- pairs of points (uncorrected in red and postprocessed in green) for each survey method (1) GPS under tree, (2) GPS control - tape, and (3) GPS control - laser. An example of the GPS under tree points is shown below.
- We will now export these points into a "KMZ" format file (stands for Keyhole Markup Zipped) that can be loaded in Google Earth. In the ArcToolbox window, pop open the" 3D Analyst Tools", then pop open "Conversion", then "To KML", then double-click the "Layer to KML" tool. In the tool, chose your Shapefile as the layer to export. Then direct the output to your working directory and name the output file "trees_group#.kmz", again where # is your group number. Use an output scale of 100. See screenshot below. Click the OK button. The file is exported to a KMZ. When the tool has finished processing, click Close. We can now open the KMZ with your points in Google Earth!
- Save your ArcMap document with File->Save. Store the file (.mxd extension) in your working folder in c:\Workspace. Close ArcMap, File->Exit
- Open Google Earth under Start->All Programs->Google Earth->Google Earth.
- Open your tree point KMZ layer under File. You should see the layer under Places (left side panel). Google Earth will then zoom to your points. More information on Google Earth navigation tools can be found in the online Google Earth User Guide. Turn on the "Scale Legend" under View. Go to Tools->Options. Set Show Elevation to "Meters, Kilometers", Show Lat/Long to "Universal Transverse Mercator" and Labels/Icons Size to "Small".Click OK.
- Distances among points can be measured with the Ruler tool . In the example below, the distance between the 2 GPS under tree points (uncorrected vs. post-processed) is 1.07 meters.
User the Ruler to fill in the table with distances (in meters). DGPS points are "post-processed" (differential corrections).
||DGPS under tree
||DGPS - tape
|DGPS - tape
|DGPS - laser
What are the mean and standard deviation of these distances?
||GPS under tree
||GPS - tape
|GPS - tape
|GPS - laser
What are the mean and standard deviation of these distances?
Do differential correction help reduce discrepancies in final point location among the 3 methods? What is your evidence?
According to your map in Google Earth, which of the 3 methods (GPS under tree, GPS - tape, GPS - laser) put a point nearest the tree's trunk? Why do you think the method performed better than the other 2 methods?
In the example above, the GPS under the tree method had very poor accuracy. The points were not even under the crown of the tree. Why would taking a GPS point at the trunk of the tree have relatively poor performance? Why don't differential corrections help resolve the problem?
Zoom into your points with the Navigation tool. Export a JPG image of your Google Earth map under File->Save->Save Image. You will need to turn this in.
7.0 Creating plot points and visualization in Google Earth
- Open Excel and load your Plot__Po.dbf file. Repeat the steps that we took in the Tree portion of the lab to autofit the column widths.
- Verify that you have four points for your plot that have Corr_Type of "Uncorrected" and four points that have Corr_Type of "Postprocessed Code". For each set, the points should be numbered from 1 to 4 in the Point_numb column. Notice that the Horz_Prec (horizontal precision) is lower for the differentially-corrected points (postprocessed code). If you do not have a set of 8 points that looks similar to the screenshot below, then ask your instructor for help. You may have stray points in your file that need to be deleted before you continue.
- Save your file as an Excel workbook as plot_group#.xls, in your working folder in c:\Workspace (# is your group number).
- Open ArcMap. Load your plot_group#.xls Excel worksheet -- called Plot__Po$
- We'll now create points from the table, as we did with the tree points. This time we're going to just use the original UTM Easting and Northing recorded by the GPS receiver. Go to Tools->Add XY Data, then select your Plot__Po$ worksheet for the input table. Use the UTM Easting for X Field and UTM Northing for Y Field. As with the trees, select UTM Zone 10 WGS84 datum for the Coordinate System.
- Click OK. You should see 2 points for each corner of your plots. We'll now export this temporary layer to a Shapefile. Right-click on the Plot__Po$ Events layer in the TOC, then select Data->Export Data. Direct the output to your work folder, and call the Shapefile plot_group#.shp (e.g., plot_group1.shp). Accept all other defaults and click OK. You should now have a Shapefile layer in your TOC.
- As with trees, we can symbolize the points in your Shapefile -- post-processed as green and the uncorrected points as red. This is done by right-clicking on the plot Shapefile, then selecting Properties->Symbology tab. Click Categories, then use Corr_Type for the Value Field. Change the colors to green and red, 6 point symbol size. Uncheck the box next to "<all other values>". See screenshot below.
- Now go to the Labels tab. Check the box for "Label features in this layer". Make sure the Label field is Point_numb. Leave all the other parameters in their default states. Click OK. You should see your corner points for the plot with their labels.
- Now we'll export these points to a KMZ file so that we can view them in Google Earth. In the ArcToolbox window, pop open the" 3D Analyst Tools", then pop open "Conversion", then "To KML", then double-click the "Layer to KML" tool. In the tool, chose your plot Shapefile as the layer to export. Then direct the output to your working directory and name the output file "plot_group#.kmz", where # is your group number. Output scale is 100.
- Open the KMZ in Google Earth.
Zoom to your plots with the Navigation tool. Export a JPG image of your Google Earth map under File->Save->Save Image. You will need to turn this in.
In considering your map in Screenshot #2, did you find a big advantage in using differential corrections of corner GPS points? Explain your reasoning. Feel free to use the Ruler tool to provide a solid argument. Would you be confident in using a uncorrected or differentially-corrected GPS data to locate all 4 corners of your plot? Be sure to consider a situation with significant canopy cover.
8.0 Creating plot lines
In this last exercise, we will retrace the lines around your plot using the laser rangefinder and digital compass. You will start with your post-processed DGPS point #1. Distance and azimuth measurements will then be used to traverse from Pt1 -> Pt2 -> Pt3 -> Pt4 -> Pt1. It is likely that your final tranverse from Pt4 back to Pt1 will not land right on the original starting Pt1 due to accumulated error in your measurements. We'll see how close you get!
- Start by downloading this Excel template into your working directory --> plot_template.xls
- Open the template in Excel. Open your other Excel workbook with your 8 GPS points (e.g., plot_group#.xls).
- Copy the Easting and Northing coordinates for Point #1, post-processed into the upper-left of the template workbook (green area)
- Copy the horizontal distance and azimuth data for the 4 points into the middle green area. Your Excel template should pick up these data and compute coordinates in the yellow area at the bottom. See screenshot below.
- The template does all the trignometric equations to traverse from the first point through the sequence of points, using your distance and azimuth data. The math is the same as we used in the Tree exercise -- nothing special here. The template is to save time in creating a text file that is compatible with ArcGIS. The text file will be used to generate your lines using a tool in ArcToolbox.
- Open up Notepad from Start->All Programs->Accessories. This is a very basic, no frills text editor. It's all you need to create the line generate file. Copy the columns in the yellow area of the template, then Paste the data into Notepad. You should have a file that looks like this...
- The "1" at the top is the line's ID #. You could give it any number. The coordinate data are for the sequence of points, starting at Pt #1 and ending at Pt #1. The first "END" statement terminates the line, and the second "END" tells ArcGIS to end reading the file. FYI: You could have multiple lines in one of these generate files. They would just need different IDs and each would terminate with an "END". Save the text file in your working directory. Call it plot_line_generate_group#.txt
- Open ArcMap. Go to ArcToolbox->Coverage Tools->Conversion->To Coverage->Generate. This opens the Generate tool, which will read our text file and generate a line layer (it will be a "coverage" ... don't worry if you do not know what that is).
- Direct the input file to your text file, plot_line_generate_group#.txt. Save the coverage in your working directory, and call it plot_lines# (# is group number). Note: coverage names can't be more than 13 characters long. Leave feature type on Lines. See screenshot.
- Click OK. The tool should report that lines were generated successfully. Click Close. Add the coverage with the button. It will be in the plot_lines#, then click the arc layer. You should see the lines of your plot in your map display.
- Now we are going to convert the coverage layer into a Shapefile for convenience. Right-click on the line layer in the TOC and select Data->Export Data. Direct the output to your work directory, and call the Shapefile plot_lines_group#.shp
- Click OK. Double-click the line symbol for the Shapefile and change the symbology of the line to something of your choice. Make it bright, because we will overlay this in Google Earth. Below is an example with a red solid line and width 2. Click OK.
- The Shapefile's projection has not yet been defined as UTM, Zone 10 North, WGS84 datum. In ArcToolbox, go to Data Management Tools->Projections and Transformations->Define Projection. Chose your plot_lines_group#.shp file for your input dataset and click the button next to Coordinate System. This will allow you to Select the UTM projection as you did with the Tree points. Your tool should look like the screenshot below. Click OK to run the tool.
- The projection has been defined, and now we can export the lines to a KMZ for viewing in Google Earth. Go to ArcToolbox, "3D Analyst Tools", "Conversion", "To KML", "Layer to KML". In the tool, chose your plot_lines_group#.shp Shapefile as the layer to export. Then direct the output to your working directory and name the output file "plot_lines_group#.kmz". Output scale is 100. Click OK.
- Open the plot lines KMZ layer in Google Earth.
Zoom to your plot lines with the Navigation tool. Export a JPG image of your Google Earth map under File->Save->Save Image. You will need to turn this in.
Did your line connect back to Point #1 from Point #4? If not, how far off, in terms of meters, were the start and end of the lines? What were some sources of error that could have led to the plot not being completly square (10 x 10m)? How might you improve this method?
Of the 3 methods used to survey a plot (GPS of corners, DGPS of corners, control point-laser rangefinder) which do you think was most accurate, and why?
9.0 To turn in
- The question sheet, with typed answers (Word document)
- Screenshot #1
- Screenshot #2
- Screenshot #3
Submit electronic files via email to firstname.lastname@example.org, with the subject "Geog315, Lab 3, [your last name]".
Lab design by Matthew Clark, Dept of Geography and Global Studies, Sonoma State University
This page was last modified on March 18, 2010 by Matthew Clark