Availability: (ncap2), ncbo, ncea,
ncecat, ncflint, ncks, ncpdq,
ncra, ncrcat, ncwa Short options: ‘-v’, ‘-x’ Long options: ‘--variable’, ‘--exclude’ or ‘--xcl’ |
Variables explicitly specified for extraction with ‘-v var[,...]’ must be present in the input file or an error will result. Variables explicitly specified for exclusion with ‘-x -v var[,...]’ need not be present in the input file. Remember, if averaging or concatenating large files stresses your systems memory or disk resources, then the easiest solution is often to use the ‘-v’ option to retain only the most important variables (see Memory Requirements).
Due to its special capabilities, ncap2 interprets the ‘-v’ switch differently (see ncap2 netCDF Arithmetic Processor). For ncap2, the ‘-v’ switch takes no arguments and indicates that only user-defined variables should be output. ncap2 neither accepts nor understands the -x switch.
As of NCO 2.8.1 (August, 2003), variable name arguments of the ‘-v’ switch may contain extended regular expressions. As of NCO 3.9.6 (January, 2009), variable names arguments to ncatted may contain extended regular expressions. For example, ‘-v '^DST'’ selects all variables beginning with the string ‘DST’. Extended regular expressions are defined by the GNU egrep command. The meta-characters used to express pattern matching operations are ‘^$+?.*[]{}|’. If the regular expression pattern matches any part of a variable name then that variable is selected. This capability is called wildcarding, and is very useful for sub-setting large data files.
Because of its wide availability, NCO uses the POSIX
regular expression library regex
.
Regular expressions of arbitary complexity may be used.
Since netCDF variable names are relatively simple constructs, only a
few varieties of variable wildcards are likely to be useful.
For convenience, we define the most useful pattern matching operators
here:
Q
, Q01
–Q99
,
Q100
, QAA
–QZZ
, Q_H2O
, X_H2O
,
Q_CO2
, X_CO2
.
ncks -v 'Q.?' in.nc # Variables that contain Q ncks -v '^Q.?' in.nc # Variables that start with Q ncks -v '^Q+.?.' in.nc # Q, Q0--Q9, Q01--Q99, QAA--QZZ, etc. ncks -v '^Q..' in.nc # Q01--Q99, QAA--QZZ, etc. ncks -v '^Q[0-9][0-9]' in.nc # Q01--Q99, Q100 ncks -v '^Q[[:digit:]]{2}' in.nc # Q01--Q99 ncks -v 'H2O$' in.nc # Q_H2O, X_H2O ncks -v 'H2O$|CO2$' in.nc # Q_H2O, X_H2O, Q_CO2, X_CO2 ncks -v '^Q[0-9][0-9]$' in.nc # Q01--Q99 ncks -v '^Q[0-6][0-9]|7[0-3]' in.nc # Q01--Q73, Q100 ncks -v '(Q[0-6][0-9]|7[0-3])$' in.nc # Q01--Q73 ncks -v '^[a-z]_[a-z]{3}$' in.nc # Q_H2O, X_H2O, Q_CO2, X_CO2
Beware—two of the most frequently used repetition pattern matching operators, ‘*’ and ‘?’, are also valid pattern matching operators for filename expansion (globbing) at the shell-level. Confusingly, their meanings in extended regular expressions and in shell-level filename expansion are significantly different. In an extended regular expression, ‘*’ matches zero or more occurences of the preceding regular expression. Thus ‘Q*’ selects all variables, and ‘Q+.*’ selects all variables containing ‘Q’ (the ‘+’ ensures the preceding item matches at least once). To match zero or one occurence of the preceding regular expression, use ‘?’. Documentation for the UNIX egrep command details the extended regular expressions which NCO supports.
One must be careful to protect any special characters in the regular expression specification from being interpreted (globbed) by the shell. This is accomplish by enclosing special characters within single or double quotes
ncra -v Q?? in.nc out.nc # Error: Shell attempts to glob wildcards ncra -v '^Q+..' in.nc out.nc # Correct: NCO interprets wildcards ncra -v '^Q+..' in*.nc out.nc # Correct: NCO interprets, Shell globs
The final example shows that commands may use a combination of variable wildcarding and shell filename expansion (globbing). For globbing, ‘*’ and ‘?’ have nothing to do with the preceding regular expression! In shell-level filename expansion, ‘*’ matches any string, including the null string and ‘?’ matches any single character. Documentation for bash and csh describe the rules of filename expansion (globbing).