2.10 Performance
An overview of NCO capabilities as of about 2006 is in
Zender, C. S. (2008),
“Analysis of Self-describing Gridded Geoscience Data with netCDF Operators (NCO)”,
Environ. Modell. Softw., doi:10.1016/j.envsoft.2008.03.004.
This paper is also available at
http://dust.ess.uci.edu/ppr/ppr_Zen08_ems.pdf.
NCO performance and scaling for arithmetic operations is
described in
Zender, C. S., and H. J. Mangalam (2007),
“Scaling Properties of Common Statistical Operators for Gridded Datasets”,
Int. J. High Perform. Comput. Appl., 21(4), 485-498,
doi:10.1177/1094342007083802.
This paper is also available at
http://dust.ess.uci.edu/ppr/ppr_ZeM07_ijhpca.pdf.
It is helpful to be aware of the aspects of NCO design
that can limit its performance:
- No data buffering is performed during nc_get_var and
nc_put_var operations.
Hyperslabs too large too hold in core memory will suffer substantial
performance penalties because of this.
- Since coordinate variables are assumed to be monotonic, the search for
bracketing the user-specified limits should employ a quicker algorithm,
like bisection, than the two-sided incremental search currently
implemented.
- C_format, FORTRAN_format, signedness,
scale_format and add_offset attributes are ignored by
ncks when printing variables to screen.
- In the late 1990s it was discovered that some random access operations
on large files on certain architectures (e.g., UNICOS) were
much slower with NCO than with similar operations performed
using languages that bypass the netCDF interface (e.g., Yorick).
This may have been a penalty of unnecessary byte-swapping in the netCDF
interface.
It is unclear whether such problems exist in present day (2007)
netCDF/NCO environments, where unnecessary byte-swapping has
been reduced or eliminated.