Next: , Previous: ncatted netCDF Attribute Editor, Up: Operator Reference Manual


4.3 ncbo netCDF Binary Operator

SYNTAX

     ncbo [-3] [-4] [-6] [-A] [-C] [-c]
     [--cnk_dmn nm,sz] [--cnk_map map] [--cnk_plc plc] [--cnk_scl sz]
     [-D dbg] [-d dim,[min][,[max][,[stride]]] [-F] [-h]
     [-L dfl_lvl] [-l path] [-O] [-o file_3] [-p path] [-R] [-r]
     [-t thr_nbr] [-v var[,...]] [-X ...] [-x] [-y op_typ]
     file_1 file_2 [file_3]

DESCRIPTION

ncbo performs binary operations on variables in file_1 and the corresponding variables (those with the same name) in file_2 and stores the results in file_3. The binary operation operates on the entire files (modulo any excluded variables). See Missing Values, for treatment of missing values. One of the four standard arithmetic binary operations currently supported must be selected with the ‘-y op_typ’ switch (or long options ‘--op_typ’ or ‘--operation’). The valid binary operations for ncbo, their definitions, corresponding values of the op_typ key, and alternate invocations are:

Addition
Definition: file_3 = file_1 + file_2
Alternate invocation: ncadd
op_typ key values: ‘add’, ‘+’, ‘addition
Examples: ‘ncbo --op_typ=add 1.nc 2.nc 3.nc’, ‘ncadd 1.nc 2.nc 3.nc

Subtraction
Definition: file_3 = file_1 - file_2
Alternate invocations: ncdiff, ncsub, ncsubtract
op_typ key values: ‘sbt’, ‘-’, ‘dff’, ‘diff’, ‘sub’, ‘subtract’, ‘subtraction
Examples: ‘ncbo --op_typ=- 1.nc 2.nc 3.nc’, ‘ncdiff 1.nc 2.nc 3.nc

Multiplication
Definition: file_3 = file_1 * file_2
Alternate invocations: ncmult, ncmultiply
op_typ key values: ‘mlt’, ‘*’, ‘mult’, ‘multiply’, ‘multiplication
Examples: ‘ncbo --op_typ=mlt 1.nc 2.nc 3.nc’, ‘ncmult 1.nc 2.nc 3.nc

Division
Definition: file_3 = file_1 / file_2
Alternate invocation: ncdivide
op_typ key values: ‘dvd’, ‘/’, ‘divide’, ‘division
Examples: ‘ncbo --op_typ=/ 1.nc 2.nc 3.nc’, ‘ncdivide 1.nc 2.nc 3.nc
Care should be taken when using the shortest form of key values, i.e., ‘+’, ‘-’, ‘*’, and ‘/. Some of these single characters may have special meanings to the shell 1. Place these characters inside quotes to keep them from being interpreted (globbed) by the shell 2. For example, the following commands are equivalent
     ncbo --op_typ=* 1.nc 2.nc 3.nc # Dangerous (shell may try to glob)
     ncbo --op_typ='*' 1.nc 2.nc 3.nc # Safe ('*' protected from shell)
     ncbo --op_typ="*" 1.nc 2.nc 3.nc # Safe ('*' protected from shell)
     ncbo --op_typ=mlt 1.nc 2.nc 3.nc
     ncbo --op_typ=mult 1.nc 2.nc 3.nc
     ncbo --op_typ=multiply 1.nc 2.nc 3.nc
     ncbo --op_typ=multiplication 1.nc 2.nc 3.nc
     ncmult 1.nc 2.nc 3.nc # First do 'ln -s ncbo ncmult'
     ncmultiply 1.nc 2.nc 3.nc # First do 'ln -s ncbo ncmultiply'

No particular argument or invocation form is preferred. Users are encouraged to use the forms which are most intuitive to them.

Normally, ncbo will fail unless an operation type is specified with ‘-y’ (equivalent to ‘--op_typ’). You may create exceptions to this rule to suit your particular tastes, in conformance with your site's policy on symbolic links to executables (files of a different name point to the actual executable). For many years, ncdiff was the main binary file operator. As a result, many users prefer to continue invoking ncdiff rather than memorizing a new command (‘ncbo -y sbt’) which behaves identically to the original ncdiff command. However, from a software maintenance standpoint, maintaining a distinct executable for each binary operation (e.g., ncadd) is untenable, and a single executable, ncbo, is desirable. To maintain backward compatibility, therefore, NCO automatically creates a symbolic link from ncbo to ncdiff. Thus ncdiff is called an alternate invocation of ncbo. ncbo supports many additional alternate invocations which must be manually activated. Should users or system adminitrators decide to activate them, the procedure is simple. For example, to use ‘ncadd’ instead of ‘ncbo --op_typ=add’, simply create a symbolic link from ncbo to ncadd 3. The alternatate invocations supported for each operation type are listed above. Alternatively, users may always define ‘ncadd’ as an alias to ‘ncbo --op_typ=add4.

It is important to maintain portability in NCO scripts. Therefore we recommend that site-specfic invocations (e.g., ‘ncadd’) be used only in interactive sessions from the command-line. For scripts, we recommend using the full invocation (e.g., ‘ncbo --op_typ=add’). This ensures portability of scripts between users and sites.

ncbo operates (e.g., adds) variables in file_2 with the corresponding variables (those with the same name) in file_1 and stores the results in file_3. Variables in file_2 are broadcast to conform to the corresponding variable in file_1 if necessary, but the reverse is not true. Broadcasting a variable means creating data in non-existing dimensions from the data in existing dimensions. For example, a two dimensional variable in file_2 can be subtracted from a four, three, or two (but not one or zero) dimensional variable (of the same name) in file_1. This functionality allows the user to compute anomalies from the mean. Note that variables in file_1 are not broadcast to conform to the dimensions in file_2. In the future, we will broadcast variables in file_1, if necessary to conform to their counterparts in file_2. Thus, presently, the number of dimensions, or rank, of any processed variable in file_1 must be greater than or equal to the rank of the same variable in file_2. Furthermore, the size of all dimensions common to both file_1 and file_2 must be equal.

When computing anomalies from the mean it is often the case that file_2 was created by applying an averaging operator to a file with initially the same dimensions as file_1 (often file_1 itself). In these cases, creating file_2 with ncra rather than ncwa will cause the ncbo operation to fail. For concreteness say the record dimension in file_1 is time. If file_2 were created by averaging file_1 over the time dimension with the ncra operator rather than with the ncwa operator, then file_2 will have a time dimension of size 1 rather than having no time dimension at all 5. In this case the input files to ncbo, file_1 and file_2, will have unequally sized time dimensions which causes ncbo to fail. To prevent this from occuring, use ncwa to remove the time dimension from file_2. See the example below.

ncbo never operates on coordinate variables or variables of type NC_CHAR or NC_BYTE. This ensures that coordinates like (e.g., latitude and longitude) are physically meaningful in the output file, file_3. This behavior is hardcoded. ncbo applies special rules to some CF-defined (and/or NCAR CCSM or NCAR CCM fields) such as ORO. See CF Conventions for a complete description. Finally, we note that ncflint (see ncflint netCDF File Interpolator) is designed for file interpolation. As such, it also performs file subtraction, addition, multiplication, albeit in a more convoluted way than ncbo.

EXAMPLES

Say files 85_0112.nc and 86_0112.nc each contain 12 months of data. Compute the change in the monthly averages from 1985 to 1986:

     ncbo -op_typ=sub 86_0112.nc 85_0112.nc 86m85_0112.nc
     ncdiff 86_0112.nc 85_0112.nc 86m85_0112.nc

The following examples demonstrate the broadcasting feature of ncbo. Say we wish to compute the monthly anomalies of T from the yearly average of T for the year 1985. First we create the 1985 average from the monthly data, which is stored with the record dimension time.

     ncra 85_0112.nc 85.nc
     ncwa -O -a time 85.nc 85.nc

The second command, ncwa, gets rid of the time dimension of size 1 that ncra left in 85.nc. Now none of the variables in 85.nc has a time dimension. A quicker way to accomplish this is to use ncwa from the beginning:

     ncwa -a time 85_0112.nc 85.nc

We are now ready to use ncbo to compute the anomalies for 1985:

     ncdiff -v T 85_0112.nc 85.nc t_anm_85_0112.nc

Each of the 12 records in t_anm_85_0112.nc now contains the monthly deviation of T from the annual mean of T for each gridpoint.

Say we wish to compute the monthly gridpoint anomalies from the zonal annual mean. A zonal mean is a quantity that has been averaged over the longitudinal (or x) direction. First we use ncwa to average over longitudinal direction lon, creating 85_x.nc, the zonal mean of 85.nc. Then we use ncbo to subtract the zonal annual means from the monthly gridpoint data:

     ncwa -a lon 85.nc 85_x.nc
     ncdiff 85_0112.nc 85_x.nc tx_anm_85_0112.nc

This examples works assuming 85_0112.nc has dimensions time and lon, and that 85_x.nc has no time or lon dimension.

As a final example, say we have five years of monthly data (i.e., 60 months) stored in 8501_8912.nc and we wish to create a file which contains the twelve month seasonal cycle of the average monthly anomaly from the five-year mean of this data. The following method is just one permutation of many which will accomplish the same result. First use ncwa to create the five-year mean:

     ncwa -a time 8501_8912.nc 8589.nc

Next use ncbo to create a file containing the difference of each month's data from the five-year mean:

     ncbo 8501_8912.nc 8589.nc t_anm_8501_8912.nc

Now use ncks to group the five January anomalies together in one file, and use ncra to create the average anomaly for all five Januarys. These commands are embedded in a shell loop so they are repeated for all twelve months:

     for idx in {1..12}; do # Bash Shell (version 3.0+)
       idx=`printf "%02d" ${idx}` # Zero-pad to preserve order
       ncks -F -d time,${idx},,12 t_anm_8501_8912.nc foo.${idx}
       ncra foo.${idx} t_anm_8589_${idx}.nc
     done
     for idx in 01 02 03 04 05 06 07 08 09 10 11 12; do # Bourne Shell
       ncks -F -d time,${idx},,12 t_anm_8501_8912.nc foo.${idx}
       ncra foo.${idx} t_anm_8589_${idx}.nc
     done
     foreach idx (01 02 03 04 05 06 07 08 09 10 11 12) # C Shell
       ncks -F -d time,${idx},,12 t_anm_8501_8912.nc foo.${idx}
       ncra foo.${idx} t_anm_8589_${idx}.nc
     end

Note that ncra understands the stride argument so the two commands inside the loop may be combined into the single command

     ncra -F -d time,${idx},,12 t_anm_8501_8912.nc foo.${idx}

Finally, use ncrcat to concatenate the 12 average monthly anomaly files into one twelve-record file which contains the entire seasonal cycle of the monthly anomalies:

     ncrcat t_anm_8589_??.nc t_anm_8589_0112.nc


Footnotes

[1] A naked (i.e., unprotected or unquoted) ‘*’ is a wildcard character. A naked-’ may confuse the command line parser. A naked+’ and ‘/’ are relatively harmless.

[2] The widely used shell Bash correctly interprets all these special characters even when they are not quoted. That is, Bash does not prevent NCO from correctly interpreting the intended arithmetic operation when the following arguments are given (without quotes) to ncbo: ‘--op_typ=+’, ‘--op_typ=-’, ‘--op_typ=*’, and ‘--op_typ=/

[3] The command to do this is ‘ln -s -f ncbo ncadd

[4] The command to do this is ‘alias ncadd='ncbo --op_typ=add'

[5] This is because ncra collapses the record dimension to a size of 1 (making it a degenerate dimension), but does not remove it, while, unless ‘-b’ is given, ncwa removes all averaged dimensions. In other words, by default ncra changes variable size but not rank, while, ncwa changes both variable size and rank.