Next: , Previous: Auxiliary Coordinates, Up: Common features


3.19 UDUnits Support

Availability: ncbo, ncea, ncecat, ncflint, ncks, ncpdq, ncra, ncrcat, ncwa
Short options: ‘-d dim,[min][,[max][,[stride]]]
Long options: ‘--dimension dim,[min][,[max][,[stride]]]’,
--dmn dim,[min][,[max][,[stride]]]
There is more than one way to hyperskin a cat. The UDUnits package provides a library which, if present, NCO uses to translate user-specified physical dimensions into the physical dimensions of data stored in netCDF files. Unidata provides UDUnits under the same terms as netCDF, so sites should install both. Compiling NCO with UDUnits support is currently optional but may become required in a future version of NCO.

Two examples suffice to demonstrate the power and convenience of UDUnits support. First, consider extraction of a variable containing non-record coordinates with physical dimensions stored in MKS units. In the following example, the user extracts all wavelengths in the visible portion of the spectrum in terms of the units very frequently used in visible spectroscopy, microns:

     % ncks -C -H -v wvl -d wvl,"0.4 micron","0.7 micron" in.nc
     wvl[0]=5e-07 meter

The hyperslab returns the correct values because the wvl variable is stored on disk with a length dimension that UDUnits recognizes in the units attribute. The automagical algorithm that implements this functionality is worth describing since understanding it helps one avoid some potential pitfalls. First, the user includes the physical units of the hyperslab dimensions she supplies, separated by a simple space from the numerical values of the hyperslab limits. She encloses each coordinate specifications in quotes so that the shell does not break the value-space-unit string into separate arguments before passing them to NCO. Double quotes ("foo") or single quotes ('foo') are equally valid for this purpose. Second, NCO recognizes that units translation is requested because each hyperslab argument contains text characters and non-initial spaces. Third, NCO determines whether the wvl is dimensioned with a coordinate variable that has a units attribute. In this case, wvl itself is a coordinate variable. The value of its units attribute is meter. Thus wvl passes this test so UDUnits conversion is attempted. If the coordinate associated with the variable does not contain a units attribute, then NCO aborts. Fourth, NCO passes the specified and desired dimension strings (microns are specified by the user, meters are required by NCO) to the UDUnits library. Fifth, the UDUnits library that these dimension are commensurate and it returns the appropriate linear scaling factors to convert from microns to meters to NCO. If the units are incommensurate (i.e., not expressible in the same fundamental MKS units), or are not listed in the UDUnits database, then NCO aborts since it cannot determine the user's intent. Finally, NCO uses the scaling information to convert the user-specified hyperslab limits into the same physical dimensions as those of the corresponding cooridinate variable on disk. At this point, NCO can perform a coordinate hyperslab using the same algorithm as if the user had specified the hyperslab without requesting units conversion.

The translation and dimensional innterpretation of time coordinates shows a more powerful, and probably more common, UDUnits application. In this example, the user prints all data between the eighth and ninth of December, 1999, from a variable whose time dimension is hours since the year 1900:

     % ncks -H -C -v time_udunits -d time_udunits,"1999-12-08 \
       12:00:0.0","1999-12-09 00:00:0.0",2 in.nc foo2.nc
     time_udunits[1]=876018 hours since 1900-01-01 00:00:0.0

Here, the user invokes the stride (see Stride) capability to obtain every other timeslice. This is possible because the UDUnits feature is additive, not exclusive—it works in conjunction with all other hyperslabbing (see Hyperslabs) options and in all operators which support hyperslabbing. The following example shows how one might average data in a time period spread across multiple input files

     ncra -d time,"1939-09-09 12:00:0.0","1945-05-08 00:00:0.0" \
       in1.nc in2.nc in3.nc out.nc

Note that there is no excess whitespace before or after the individual elements of the ‘-d’ argument. This is important since, as far as the shell knows, ‘-d’ takes only one command-line argument. Parsing this argument into its component dim,[min][,[max][,[stride]]] elements (see Hyperslabs) is the job of NCO. When unquoted whitespace is present between these elements, the shell passes NCO arugment fragments which will not parse as intended.

NCO implemented support for the UDUnits2 library with version 3.9.2 (August, 2007). The UDUnits2 package supports non-ASCII characters and logarithmic units. We are interested in user-feedback on these features, which are relatively un-tested with NCO.

The UDUnits package documentation describes the supported formats of time dimensions. Among the metadata conventions which adhere to these formats are the Climate and Forecast (CF) Conventions and the Cooperative Ocean/Atmosphere Research Data Service (COARDS) Conventions. The following ‘-d arguments’ extract the same data using commonly encountered time dimension formats:

     -d time,"1918-11-11 11:00:0.0","1939-09-09 00:00:0.0"

All of these formats include at least one dash - in a non-leading character position (a dash in a leading character position is a negative sign). NCO assumes that a non-leading dash in a limit string indicates that a UDUnits date conversion is requested.

As of NCO 4.0.0 some of calendar attributes as specified by the CF conventions are supported. The unsupported types default to mixed Gregorian/Julian as defined by UDUunits.

Supported types:
"365_day"/"no_leap", "360_day", "gregorian", "standard"
Unsupported types:
"366_day"/"all_leap","proleptic_gregorian","julian","none"

An Example: Consider the following netcdf variable

     variables:
       double lon_cal(lon_cal) ;
         lon_cal:long_name = "lon_cal" ;
         lon_cal:units = "days since 1964-2-28 0:0:0" ;
         lon_cal:calendar = "365_day" ;
     data:
       lon_cal = 1,2,3,4,5,6,7,8,9,10;
     
     So the command
     "ncks -v lon_cal -d lon_cal,'1964-3-1 0:00:0.0','1964-3-4 00:00:0.0' in.nc out.nc"
     Results in the hyperslab lon_cal=1,2,3,4
     
     

netCDF variables should always be stored with MKS (i.e., God's) units, so that application programs may assume MKS dimensions apply to all input variables. The UDUnits feature is intended to alleviate some of the NCO user's pain when handling MKS units. It connects users who think in human-friendly units (e.g., miles, millibars, days) to extract data which are always stored in God's units, MKS (e.g., meters, Pascals, seconds). The feature is not intended to encourage writers to store data in esoteric units (e.g., furlongs, pounds per square inch, fortnights).