Next: , Previous: Climate Model Paradigm, Up: Strategies


2.3 Temporary Output Files

NCO operators are designed to be reasonably fault tolerant, so that if there is a system failure or the user aborts the operation (e.g., with C-c), then no data are lost. The user-specified output-file is only created upon successful completion of the operation 1. This is accomplished by performing all operations in a temporary copy of output-file. The name of the temporary output file is constructed by appending .pid<process ID>.<operator name>.tmp to the user-specified output-file name. When the operator completes its task with no fatal errors, the temporary output file is moved to the user-specified output-file. Note the construction of a temporary output file uses more disk space than just overwriting existing files “in place” (because there may be two copies of the same file on disk until the NCO operation successfully concludes and the temporary output file overwrites the existing output-file). Also, note this feature increases the execution time of the operator by approximately the time it takes to copy the output-file. Finally, note this feature allows the output-file to be the same as the input-file without any danger of “overlap”.

Other safeguards exist to protect the user from inadvertently overwriting data. If the output-file specified for a command is a pre-existing file, then the operator will prompt the user whether to overwrite (erase) the existing output-file, attempt to append to it, or abort the operation. However, in processing large amounts of data, too many interactive questions slows productivity. Therefore NCO also implements two ways to override its own safety features, the ‘-O’ and ‘-A’ switches. Specifying ‘-O’ tells the operator to overwrite any existing output-file without prompting the user interactively. Specifying ‘-A’ tells the operator to attempt to append to any existing output-file without prompting the user interactively. These switches are useful in batch environments because they suppress interactive keyboard input.


Footnotes

[1] The ncrename operator is an exception to this rule. See ncrename netCDF Renamer.