Joining independent files together along a record dimension is called
concatenation.
ncrcat is designed for concatenating record variables, while
ncecat is designed for concatenating fixed length variables.
Consider five files, 85.nc, 86.nc,
... 89.nc each containing a year's worth of data.
Say you wish to create from them a single file, 8589.nc
containing all the data, i.e., spanning all five years.
If the annual files make use of the same record variable, then
ncrcat will do the job nicely with, e.g.,
ncrcat 8?.nc 8589.nc
.
The number of records in the input files is arbitrary and can vary from
file to file.
See ncrcat netCDF Record Concatenator, for a complete description of
ncrcat.
However, suppose the annual files have no record variable, and thus
their data are all fixed length.
For example, the files may not be conceptually sequential, but rather
members of the same group, or ensemble.
Members of an ensemble may have no reason to contain a record dimension.
ncecat will create a new record dimension (named record
by default) with which to glue together the individual files into the
single ensemble file.
If ncecat is used on files which contain an existing record
dimension, that record dimension is converted to a fixed-length
dimension of the same name and a new record dimension (named
record
) is created.
Consider five realizations, 85a.nc, 85b.nc,
... 85e.nc of 1985 predictions from the same climate
model.
Then ncecat 85?.nc 85_ens.nc
glues the individual realizations
together into the single file, 85_ens.nc.
If an input variable was dimensioned [lat
,lon
], it will
have dimensions [record
,lat
,lon
] in the output file.
A restriction of ncecat is that the hyperslabs of the
processed variables must be the same from file to file.
Normally this means all the input files are the same size, and contain
data on different realizations of the same variables.
See ncecat netCDF Ensemble Concatenator, for a complete description
of ncecat.
ncpdq makes it possible to concatenate files along any
dimension, not just the record dimension.
First, use ncpdq to convert the dimension to be concatenated
(i.e., extended with data from other files) into the record dimension.
Second, use ncrcat to concatenate these files.
Finally, if desirable, use ncpdq to revert to the original
dimensionality.
As a concrete example, say that files x_01.nc, x_02.nc,
... x_10.nc contain time-evolving datasets from spatially
adjacent regions.
The time and spatial coordinates are time
and x
, respectively.
Initially the record dimension is time
.
Our goal is to create a single file that contains joins all the
spatially adjacent regions into one single time-evolving dataset.
for idx in 01 02 03 04 05 06 07 08 09 10; do # Bourne Shell ncpdq -a x,time x_${idx}.nc foo_${idx}.nc # Make x record dimension done ncrcat foo_??.nc out.nc # Concatenate along x ncpdq -a time,x out.nc out.nc # Revert to time as record dimension
Note that ncrcat will not concatenate fixed-length variables, whereas ncecat concatenates both fixed-length and record variables along a new record variable. To conserve system memory, use ncrcat where possible.