Using Web Coverage Service (WCS) Scripts to Retrieve Data
Researchers with sufficient programming experience can use a web service to obtain data from the Remote Sensing Information Gateway (RSIG) and Estuary Data Mapper (EDM) without having to use the RSIG or EDM applications.
End users who are interested in bulk data transfer may opt to use the web service directly, or incorporate it into an automated script.
What is WCS
Web Coverage Service (WCS) is an Open Geospatial Consortium (OGC) compliant way of requesting geospatial data over the internet.
When data is requested through RSIG or Estuary Data Mapper (EDM), a WCS call is made behind the scenes to the rsigserver, which then streams the requested data back via HTTPS with a 30-minute timeout.
You can use the WCS web service directly from the command line to obtain data without having to use the RSIG or EDM applications.
Using the command line requires a utility program for downloading information from the internet, such as Wget or cURL. These utilities are open source and freely available for Linux, Mac OS X, and Windows platforms.
A WCS call consists of a server, a service, mandatory parameters, and optional parameters. The optional parameters define the data variable being requested, time range, bounding box, and returned data format.
curl --silent --retry 0 -L --tcp-nodelay --max-time 0 'https://ofmpub.epa.gov/rsig/rsigserver?SERVICE=wcs&VERSION=1.0.0&REQUEST=GetCoverage&COVERAGE=cmaq.amad.conus.metdot3d.uwind,cmaq.amad.conus.metdot3d.vwind&TIME=2005-08-28T20:00:00Z/2005-08-29T01:59:59Z&BBOX=-90,30,-88,32,1,1&FORMAT=netcdf-ioapi' > sample.nc ; ncdump sample.nc | more
Consult the Log Files for WCS Queries
WCS calls made by RSIG and EDM are printed to a log file you can find in each application’s directory. The log file is a simple text file. You can inspect or copy the parameters to use as a starting point for constructing your own web service calls.
So, the easy and practical way to learn WCS queries is to simply run the application, retrieve data, and then read the resulting log file. Then, execute the same WCS queries using cURL or Wget, redirect the result to a file, and examine the file.
There are thousands of possible different WCS queries available with rsigserver (over 5,000 for RSIG alone), and that number increases as new data are added.
Consider the EDM WCS page and the RSIG WCS page as starting points for your own discoveries. Crafting WCS queries is a trial-and-error process and each type of data carries its own idiosyncrasies to either work with or work around.
If you need help crafting your queries or have questions about how RSIG and EDM use WCS commands, please contact us.
Performance Tips
- Ideally, a script should make repeated, sequential, synchronous WCS calls. This is to ensure that each request completes before the next one is started, rather than loading the server with many simultaneous requests. A simple loop with an embedded curl command is an easy way to accomplish this.
- The script should make several small requests rather than one large one, particularly in light of the 30-minute web service timeout and potentially large file sizes.
Example: Retrieving 60 days’ worth of CMAQ level one data.
- One WCS call using a 60 day time range, or
- Sixty separate calls each spanning only one day.
The second option is a better choice because each request can complete in a short amount of time. But the first option could fail due to the 30 minute timeout.
Scripting makes it easy to automate the WCS calls. Sample RSIG WCS scripts and EDM WCS scripts are available. You may need to modify the scripts to tailor them to your computer system.