Availability: ncap2, ncbo, ncea,
ncecat, ncflint, ncks, ncpdq,
ncra, ncrcat, ncwa Short options: ‘-3’, ‘-4’ Long options: ‘--3’, ‘--4’, ‘--64bit’, ‘--fl_fmt’, ‘--netcdf4’ |
netCDF supports four types of files: CLASSIC
, 64BIT
,
NETCDF4
, and NETCDF4_CLASSIC
,
The CLASSIC
format is the traditional 32-bit offset written by
netCDF2 and netCDF3.
As of 2005, most netCDF datasets are in CLASSIC
format.
The 64BIT
format was added in Fall, 2004.
The NETCDF4
format uses HDF5 as the file storage layer.
The files are (usually) created, accessed, and manipulated using the
traditional netCDF3 API (with numerous extensions).
The NETCDF4_CLASSIC
format refers to netCDF4 files created with
the NC_CLASSIC_MODEL
mask.
Such files use HDF5 as the back-end storage format (unlike
netCDF3), though they incorporate only netCDF3 features.
Hence NETCDF4_CLASSIC
files are perfectly readable by
applications which use only the netCDF3 API and library.
NCO must be built with netCDF4 to write files in the new
NETCDF4
and NETCDF4_CLASSIC
formats, and to read files in
the new NETCDF4
format.
Users are advised to use the default CLASSIC
format or the
NETCDF4_CLASSIC
format until netCDF4 is more widespread.
Widespread support for NETCDF4
format files is not expected for
a few more years, 2010–2011, say.
If performance or coolness are issues, then use NETCDF4_CLASSIC
instead of CLASSIC
format files.
As mentioned above, all operators write use the input file format for
output files unless told otherwise.
Toggling the long option ‘--64bit’ switch (or its
key-value equivalent ‘--fl_fmt=64bit’) produces the
netCDF3 64-bit offset format named 64BIT
.
NCO must be built with netCDF 3.6 or higher to produce
a 64BIT
file.
Using the ‘-4’ switch (or its long option equivalents
‘--4’ or ‘--netcdf4’), or setting its key-value
equivalent ‘--fl_fmt=netcdf4’ produces a NETCDF4
file
(i.e., HDF).
Casual users are advised to use the default (netCDF3) CLASSIC
format until netCDF 3.6 and netCDF 4.0 are more widespread.
Conversely, operators given the ‘-3’ (or ‘--3’) switch
without arguments will (attempt to) produce netCDF3 CLASSIC
output, even from netCDF4 input files.
These examples demonstrate converting a file from any netCDF format into any other netCDF format (subject to limits of the format):
ncks --fl_fmt=classic in.nc foo_3c.nc # netCDF3 classic ncks --fl_fmt=64bit in.nc foo_364.nc # netCDF3 64bit ncks --fl_fmt=netcdf4_classic in.nc foo_4c.nc # netCDF4 classic ncks --fl_fmt=netcdf4 in.nc foo_4.nc # netCDF4 ncks -3 in.nc foo_3c.nc # netCDF3 classic ncks --3 in.nc foo_3c.nc # netCDF3 classic ncks -4 in.nc foo_4.nc # netCDF4 ncks --4 in.nc foo_4.nc # netCDF4 ncks --64 in.nc foo364.nc # netCDF3 64bit
Of course since most operators support these switches, the
“conversions” can be done at the output stage of arithmetic
or metadata processing rather than requiring a separate step.
Producing (netCDF3) CLASSIC
or 64BIT
files from
NETCDF4_CLASSIC
files will always work.
However, producing netCDF3 files from NETCDF4
files will only
work if the output files are not required to contain netCDF4-specific
features.
Note that NETCDF4
and NETCDF4_CLASSIC
are the same
binary format.
The latter simply causes a writing application to fail if it attempts to
write a NETCDF4
file that cannot be completely read by the
netCDF3 library.
Conversely, NETCDF4_CLASSIC
indicates to a reading application
that all of the file contents are readable with the netCDF3 library.
As of October, 2005, NCO writes no netCDF4-specific data
structures and so always succeeds at writing NETCDF4_CLASSIC
files.
There are at least three ways to discover the format of a netCDF file, i.e., whether it is a classic (32-bit offset) or newer 64-bit offset netCDF3 format, or is netCDF4 format. Each method returns the information using slightly different terminology that becomes easier to understand with practice.
First, examine the end of the first line of global metadata output by ‘ncks -M’:
% ncks -M foo_3c.nc Opened file foo_3c.nc: dimensions = 19, variables = 261, global atts. = 4, id = 65536, type = NC_FORMAT_CLASSIC % ncks -M foo_364.nc Opened file foo_364.nc: dimensions = 19, variables = 261, global atts. = 4, id = 65536, type = NC_FORMAT_64BIT % ncks -M foo_4c.nc Opened file foo_4c.nc: dimensions = 19, variables = 261, global atts. = 4, id = 65536, type = NC_FORMAT_NETCDF4_CLASSIC % ncks -M foo_4.nc Opened file foo_4.nc: dimensions = 19, variables = 261, global atts. = 4, id = 65536, type = NC_FORMAT_NETCDF4
This method requires a netCDF4-enabled NCO version 3.9.0+ (i.e., from 2007 or later).
Second, query the file with ‘ncdump -k’:
% ncdump -k foo_3.nc classic % ncdump -k foo_364.nc 64-bit-offset % ncdump -k foo_4c.nc netCDF-4 classic model % ncdump -k foo_4.nc netCDF-4
This method requires a netCDF4-enabled netCDF 3.6.2+ (i.e., from 2007 or later).
The third option uses the POSIX-standard od (octal dump) command:
% od -An -c -N4 foo_3c.nc C D F 001 % od -An -c -N4 foo_364.nc C D F 002 % od -An -c -N4 foo_4c.nc 211 H D F % od -An -c -N4 foo_4.nc 211 H D F
This option works without NCO and ncdump. Values of ‘C D F 001’ and ‘C D F 002’ indicate 32-bit (classic) and 64-bit netCDF3 formats, respectively, while values of ‘211 H D F’ indicate the newer netCDF4 file format.