Next: Intrinsic mathematical methods, Previous: GSL random number generation, Up: ncap2 netCDF Arithmetic Processor
See the ncap.in and ncap2.in scripts released with NCO for more complete demonstrations of ncap and ncap2 functionality, respectively (these scripts are available on-line at http://nco.sf.net/ncap.in and http://nco.sf.net/ncap2.in).
Define new attribute new for existing variable one as twice the existing attribute double_att of variable att_var:
ncap2 -s 'one@new=2*att_var@double_att' in.nc out.nc
Average variables of mixed types (result is of type double
):
ncap2 -s 'average=(var_float+var_double+var_int)/3' in.nc out.nc
Multiple commands may be given to ncap2 in three ways.
First, the commands may be placed in a script which is executed, e.g.,
tst.nco.
Second, the commands may be individually specified with multiple
‘-s’ arguments to the same ncap2 invocation.
Third, the commands may be chained together into a single ‘-s’
argument to ncap2.
Assuming the file tst.nco contains the commands
a=3;b=4;c=sqrt(a^2+b^2);
, then the following ncap2
invocations produce identical results:
ncap2 -v -S tst.nco in.nc out.nc ncap2 -v -s 'a=3' -s 'b=4' -s 'c=sqrt(a^2+b^2)' in.nc out.nc ncap2 -v -s 'a=3;b=4;c=sqrt(a^2+b^2)' in.nc out.nc
The second and third examples show that ncap2 does not require that a trailing semi-colon ‘;’ be placed at the end of a ‘-s’ argument, although a trailing semi-colon ‘;’ is always allowed. However, semi-colons are required to separate individual assignment statements chained together as a single ‘-s’ argument.
ncap2 may be used to “grow” dimensions, i.e., to increase
dimension sizes without altering existing data.
Say in.nc has ORO(lat,lon)
and the user wishes a new
file with new_ORO(new_lat,new_lon)
that contains zeros in the
undefined portions of the new grid.
defdim("new_lat",$lat.size+1); // Define new dimension sizes defdim("new_lon",$lon.size+1); new_ORO[$new_lat,$new_lon]=0.0f; // Initialize to zero new_ORO(0:$lat.size-1,0:$lon.size-1)=ORO; // Fill valid data
The commands to define new coordinate variables new_lat
and new_lon
in the output file follow a similar pattern.
One would might store these commands in a script grow.nco
and then execute the script with
ncap2 -v -S grow.nco in.nc out.nc
Imagine you wish to create a binary flag based on the value of
an array.
The flag should have value 1.0 where the array exceeds 1.0,
and value 0.0 elsewhere.
This example creates the binary flag ORO_flg
in out.nc
from the continuous array named ORO
in in.nc.
ncap2 -s 'ORO_flg=(ORO > 1.0)' in.nc out.nc
Suppose your task is to change all values of ORO
which
equal 2.0 to the new value 3.0:
ncap2 -s 'ORO_msk=(ORO==2.0);ORO=ORO_msk*3.0+!ORO_msk*ORO' in.nc out.nc
This creates and uses ORO_msk
to mask the subsequent arithmetic
operation.
Values of ORO
are only changed where ORO_msk
is true,
i.e., where ORO
equals 2.0
Using the where
statement the above code simplifies to :
ncap2 -s 'where(ORO==2.0) ORO=3.0;' in.nc foo.nc
This example uses ncap2 to compute the covariance of two variables. Let the variables u and v be the horizontal wind components. The covariance of u and v is defined as the time mean product of the deviations of u and v from their respective time means. Symbolically, the covariance
[u'v'] = [uv]-[u][v] where [x] denotes the time-average of x and x'
denotes the deviation from the time-mean.
The covariance tells us how much of the correlation of two signals
arises from the signal fluctuations versus the mean signals.
Sometimes this is called the eddy covariance.
We will store the covariance in the variable uprmvprm
.
ncwa -O -a time -v u,v in.nc foo.nc # Compute time mean of u,v ncrename -O -v u,uavg -v v,vavg foo.nc # Rename to avoid conflict ncks -A -v uavg,vavg foo.nc in.nc # Place time means with originals ncap2 -O -s 'uprmvprm=u*v-uavg*vavg' in.nc in.nc # Covariance ncra -O -v uprmvprm in.nc foo.nc # Time-mean covariance
The mathematically inclined will note that the same covariance would be obtained by replacing the step involving ncap2 with
ncap2 -O -s 'uprmvprm=(u-uavg)*(v-vavg)' foo.nc foo.nc # Covariance
As of NCO version 3.1.8 (December, 2006), ncap2 can compute averages, and thus covariances, by itself:
ncap2 -s 'uavg=u.avg($time);vavg=v.avg($time);uprmvprm=u*v-uavg*vavg' \ -s 'uprmvrpmavg=uprmvprm.avg($time)' in.nc foo.nc
We have not seen a simpler method to script and execute powerful arithmetic than ncap2.
ncap2 utilizes many meta-characters
(e.g., ‘$’, ‘?’, ‘;’, ‘()’, ‘[]’)
that can confuse the command-line shell if not quoted properly.
The issues are the same as those which arise in utilizing extended
regular expressions to subset variables (see Subsetting Variables).
The example above will fail with no quotes and with double quotes.
This is because shell globbing tries to interpolate the value of
$time
from the shell environment unless it is quoted:
ncap2 -s 'uavg=u.avg($time)' in.nc foo.nc # Correct (recommended) ncap2 -s uavg=u.avg('$time') in.nc foo.nc # Correct (and dangerous) ncap2 -s uavg=u.avg($time) in.nc foo.nc # Fails ($time = '') ncap2 -s "uavg=u.avg($time)" in.nc foo.nc # Fails ($time = '')
Without the single quotes, the shell replaces $time
with an
empty string.
The command ncap2 receives from the shell is
uavg=u.avg()
.
This causes ncap2 to average over all dimensions rather than
just the time dimension, and unintended consequence.
We recommend using single quotes to protect ncap2 command-line scripts from the shell, even when such protection is not strictly necessary. Expert users may violate this rule to exploit the ability to use shell variables in ncap2 command-line scripts (see CCSM Example). In such cases it may be necessary to use the shell backslash character ‘\’ to protect the ncap2 meta-character.
Whether a degenerate record dimension is desirable or undesirable
depends on the application.
Often a degenerate time dimension is useful, e.g., for
concatentating, but it may cause problems with arithmetic.
Such is the case in the above example, where the first step employs
ncwa rather than ncra for the time-averaging.
Of course the numerical results are the same with both operators.
The difference is that, unless ‘-b’ is specified, ncwa
writes no time dimension to the output file, while ncra
defaults to keeping time as a degenerate (size 1) dimension.
Appending u
and v
to the output file would cause
ncks to try to expand the degenerate time axis of uavg
and vavg
to the size of the non-degenerate time dimension
in the input file.
Thus the append (ncks -A) command would be undefined (and
should fail) in this case.
Equally important is the ‘-C’ argument
(see Subsetting Coordinate Variables) to ncwa to prevent
any scalar time variable from being written to the output file.
Knowing when to use ncwa -a time rather than the default
ncra for time-averaging takes, well, time.