Availability: ncap2, ncbo, ncea,
ncecat, ncflint, ncks, ncpdq,
ncra, ncrcat, ncwa Short options: ‘-L’ Long options: ‘--dfl_lvl’, ‘--deflate’ |
All NCO operators that define variables support
the netCDF4 feature of storing variables compressed with Lempel-Ziv
deflation.
The Lempel-Ziv algorithm is a lossless data compression technique.
Activate this deflation with the -L
dfl_lvl short option
(or with the same argument to the ‘--dfl_lvl’ or ‘--deflate’
long options).
Specify the deflation level dfl_lvl on a scale from
no deflation (dfl_lvl = 0) to maximum deflation
(dfl_lvl = 9).
Minimal deflation (dfl_lvl = 1) achieves considerable storage
compression with little time penalty.
Higher deflation levels require more time for compression.
File sizes resulting from minimal (dfl_lvl = 1) and maximal
(dfl_lvl = 9) deflation levels typically differ by a few
percent in size.
To compress an entire file using deflation, use
ncks -4 -L 0 in.nc out.nc # No deflation (fast, no time penalty) ncks -4 -L 1 in.nc out.nc # Minimal deflation (little time penalty) ncks -4 -L 9 in.nc out.nc # Maximal deflation (much slower)
Unscientific testing shows that deflation compresses typical climate datasets by 30-60%. Packing, a lossy compression technique available for all netCDF files (see Packed data), can easily compress files by 50%. Packed data may be deflated to squeeze datasets by about 80%:
ncks -4 -L 1 in.nc out.nc # Minimal deflation (~30-60% compression) ncks -4 -L 9 in.nc out.nc # Maximal deflation (~31-63% compression) ncpdq in.nc out.nc # Standard packing (~50% compression) ncpdq -4 -L 9 in.nc out.nc # Deflated packing (~80% compression)
ncks prints deflation parameters, if any, to screen (see ncks netCDF Kitchen Sink).