Next Previous Contents

4. NoSQL Command Reference

4.1 addcol

Usage: nosql addcol newcolumn ... < table

The columns specified on the command line are appended to the right of the existing ones. The program does not check for duplicates.

This NoSQL operator reads a table from STDIN and writes a table to STDOUT.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql addcol Job < inputtable
    

Output on STDOUT:

                NAME    COUNT   TYP     AMT    Job
                ----    -----   ---     ---    ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

4.2 body

Usage: nosql body < table

Strips the header off the input table, and prints the data (body) part to STDOUT.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql body < inputtable
    

Output on STDOUT:

                Bush    44  A   133
                Hansen  44  A   23
                Jones   77  X   77
                Perry   77  B   244
                Hart    77  D   1111
                Holmes  65  D   1111
    

4.3 cat

Usage: nosql cat [-d|--date datestring] table

Prints a NoSQL table the way it was at a point in time, by using RCS if necessary. The table must have been subject to versioning by RCS for this to work. Please refer to co(1) for details about the acceptable date/time formats. If the time string contains spaces, then it must be quoted, following the usual Unix shell quoting/escaping rules. If option '-d' is not specified, then the latest available version of the table is checked out. Examples of time specifications which are acceptable for RCS are:

A fairly common specification is "1998/05/21 LT".

This operator reads a table from a file and prints the requested version of the same table to STDOUT. The output stream can then easily be used to overwrite the original table on disk with the the following sequence of commands:

      nosql lock table && nosql cat -d "1998/05/21 LT" table |
          nosql write -s -o table && nosql unlock table
    

This will effectively roll-back the table to the desired point in time (i.e. the way it was before 1998/05/21 in the above example).

Note: locking and unlocking the table is not strictly necessary, but it should always be done in a multi-user environment.

4.4 column

Usage: nosql column columnname ... < table

Selects columns by name (and order) and outputs a table with these columns. Can effectively select, order, add, delete, or duplicate columns. If no columns are specified, then nothing is printed to STDOUT. If a column name does not match any of the columns in table, a new column with that name is inserted in that location. If the input table contains duplicated columns, with the same name but different values, only the first (leftmost) one will be printed to STDOUT.

This NoSQL operator reads a table from STDIN and writes a table to STDOUT.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql column TYP Job NAME < inputtable
    

Output on STDOUT:

                TYP  Job  NAME
                ---  ---  ----
                A         Bush  
                A         Hansen
                X         Jones 
                B         Perry 
                D         Hart  
                D         Holmes
    

4.5 compute

Usage: nosql compute [-x|--debug] 'column = expression [; ...]' < table

Options:

-x|--debug

Debug option. Print the actual AWK program to STDERR before executing it.

Computes values for data fields based on arbitrary AWK statements using column names. Any characters that are special to the UNIX shell must be quoted. Please refer to the documentation and man pages of your AWK implementation for more details on valid expressions. Comments, i.e. unquoted # signs in the AWK program are not supported.

This operator reads a table via STDIN and writes a table via STDOUT.

If the input table contains duplicated columns, with the same name but different values, only the first (leftmost) one is taken into account. The output will still have the duplicates, but this time with equal column values in them. If no expression is specified, then nothing is printed to STDOUT.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql compute 'COUNT = COUNT*10' < inputtable
    

Output on STDOUT:

                NAME    COUNT   TYP AMT
                ----    -----   --- ---
                Bush    440     A   133
                Hansen  440     A   23
                Jones   770     X   77
                Perry   770     B   244
                Hart    770     D   1111
                Holmes  650     D   1111
    

Note: in the AWK programming language, patterns in regular expression matching can be enclosed either in slashes (/) or in double quotes ("). With the 'compute' operator you sould always use the second form, i.e. you should not write :

      nosql compute 'if(NAME ~ /^Hans.*/) COUNT=COUNT*10' < inputtable
    

as it may or may not work, depending on what is inside the /.../ block, but you rather have to use the quoted AWK form :

      nosql compute 'if(NAME ~ "^Hans.*") COUNT=COUNT*10' < inputtable
    

The same considerations apply to other operators that parse the AWK program before calling the AWK interpreter, namely 'row', 'fcompute' and possibly others.

4.6 dashline

Usage: nosql dashline < table

Prints to STDOUT only the dashline of the input table.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql dashline < inputtable
    

Output on STDOUT:

                ----    -----   ---     ---
    

4.7 datatype

Usage: nosql datatype [-x|--debug] < table

Displays the type and length of the data in each column of a table. Supported data types are:

S

String

I

Integer

F

Float

C

Currency

D

Date

T

Time

This NoSQL operator reads a table from STDIN and writes a table with the datatype information to STDOUT.

Example 1:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql datatype < inputtable
    

Output on STDOUT:

                Field   Width   Type  Description
                -----   -----   ----  -----------
                NAME    6       S     string
                COUNT   5       I     integer
                TYP     3       S     string
                AMT     4       I     integer
    

Example 2:

Input table inputtable:

        NAME  TIME      DATE        NUM    FLOAT       USD
        ----  ----      ----        ---    -----       ---
        Bush  19:02:05  1998.09.13  12345  2345.234    1,234.01
        Hobbs 19:22:05  98.09.23    -1234  12345.2345  0.56
    

Command:

      nosql datatype < inputtable
    

Output on STDOUT:

                Field   Width   Type    Description
                -----   -----   ----    -----------
                NAME    5       S       string
                TIME    8       T       time
                DATE    10      D       date
                NUM     5       I       integer
                FLOAT   10      F.4     float 4
                USD     8       C       currency
    

4.8 edit

Usage: nosql edit [-K|--key field] [-u|--unique] table

This utility calls an editor to allow the editing of a table. The form of the file to be edited is the 'list' format, which is usually much more comfortable to edit than the 'table' format. The default editor is specified by the environment variable EDITOR if set, otherwise the editor 'vi' is used. EDITOR cannot be set to an editor that forks and detaches from the current shell (like xedit, 'vim -g', and the like). If you want to keep your current EDITOR value and still be able to use 'nosql edit' then you can set EDITOR to a non-forking editor in your ~/.nosql.conf file. See section Operators for details on setting NoSQL environment variables.

If option '-K field' is specified, when the table is written back to disk it is kept sorted on field. If '-K field' is not specified, then the table is kept sorted on the first (leftmost) field by default.

If option '-u' is specified, then besides sorting on the key field the table is also kept unique on the same field. Rows that are duplicated on that field will be silently dropped. This option should be used with care.

Care should be taken when editing, to abide by the 'list' format structure rules. See section Data formats for more details.

The table may be an existing file, or it may be automatically checked out from RCS. In the latter case it will be checked back into RCS after the editing is complete. The default action is that if the table does not exist an attempt will be made to find the table under RCS. To start RCS versioning on a table for the first time, the following command can be used:

      ci [-u] table
    

From now on, edit will handle all interactions with RCS automatically.

4.9 fcompute

Usage: nosql fcompute [-x|--debug] cfile < table

Options:

-x|--debug

Debug option. Print the actual AWK program to STDERR before executing it.

This operator is functionally identical to the compute operator, but it reads the AWK statements from file cfile rather than from the command line. It can be used when the computing statements are too complex and it would be impractical to type them directly on the command line. The compute file may contain usual AWK comments, even inline ones. Regular expression matching in the AWK program file should follow the same rules described in the section about the compute operator.

4.10 field

Usage: nosql field from [to] < table

Takes a column number and outputs the corresponding column of the input table. If two numbers are specified, then outputs the corresponding range of columns. If the second number is lesser than the first, then all columns starting from the first column number to the last (rightmost) column are printed. If no columns or if a non-numeric or out-of-range column is specified for the first field, then nothing is printed to STDOUT. The same happens if a non-numeric column is specified for the second field.

This NoSQL operator reads a table from STDIN and writes a table to STDOUT.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql field 3 5 < inputtable
    

Output on STDOUT:

                           TYP     AMT
                           ---     ---
                           A       133
                           A       23
                           X       77
                           B       244
                           D       1111
                           D       1111
    

4.11 fieldsof

Usage: nosql fieldsof < table

This NoSQL operator reads a table from STDIN and writes the list of column names to STDOUT, one per line of output.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql fieldsof < inputtable
    

Output on STDOUT:

                NAME
                COUNT
                TYP
                AMT
    

4.12 fromRDB

Usage: nosql fromRDB [-doc|-o|--output file] < table

Options:

-doc|-o|--output file

File name where to save RDB comments, header information and column documentation fields.

Converts an RDB table to NoSQL format. RDB header comments are NOT preserved during the conversion, as NoSQL tables do not support them. However, RDB table comments, column types and column documentation fields can optionally be saved in a separate file during the conversion process. The file format is compatible with NoSQL 'maketable' template files. Note that if a file with that name already exists it is simply overwritten without notice.

This operator reads an RDB table via STDIN and produces the NoSQL version of the same table on STDOUT.

Example:

Input RDB table inputtable:

                NAME    COUNT   TYP     AMT
                6S      5N      3S      4N
                Bush    44      A       133
                Hansen  44      A       23
                Jones   77      X       77
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql fromRDB < inputtable
    

Output on STDOUT:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23
                Jones   77      X       77
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

4.13 header

Usage: nosql header < table

Strips the body off from a table and prints the header to STDOUT

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql header < inputtable
    

Output on STDOUT:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
    

4.14 headline

Usage: nosql headline < table

Prints to STDOUT the column name line of the table received via STDIN.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql headline < inputtable
    

Output on STDOUT:

                NAME    COUNT   TYP     AMT
    

4.15 help

Usage: nosql help

This operator displays a message that tells how to access the NoSQL online documentation.

4.16 index

Usage:

nosql index table column ...

or

nosql index -u|--update [index_file ... ]

The first form of usage of this utility generates an index file for the column(s) given, that refers to the specified table. An index file is actually another (smaller) table containing only the column(s) given plus a column for index information. An index file can be used by the operator 'search' to quickly locate rows of data in the referenced table.

Index files are named by appending an 'x' and the column name(s) (separated by a dot) to the base name of the table it refers to. For example an index file for the table "area.rdb" on column "strip" would be "area.x.strip". An index file that was also on column "depth" would be "area.x.strip.depth".

The second form of usage of this utility updates the index file(s) given. If no files are given all index files in the current directory are updated. An update of an index file is necessary when the table it refers to has been modified.

This utility writes or rewrites tables with defined names in the current directory. NOTE: index does not work on tables smaller than three records, but it is not meant for them anyway :-)

4.17 inscol

Usage: nosql inscol newcolumn ... < table

Inserts new empty columns in the leftmost position of a table. The columns specified on the command line are inserted to the left of the existing ones. The program does not check for duplicates.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql inscol Job < inputtable
    

Output on STDOUT:

                Job     NAME    COUNT   TYP     AMT
                ---     ----    -----   ---     ---
                        Bush    44      A       133
                        Hansen  44      A       23
                        Jones   77      X       77
                        Perry   77      B       244
                        Hart    77      D       1111
                        Holmes  65      D       1111
    

4.18 islist

Usage: nosql islist [-v|--verbose] < file

Options:

-v|--verbose

Makes the error reporting more verbose.

-e|--edit

Used by the 'edit' operator.

Checks that a file has a valid NoSQL 'list' format. This operator reads a file from STDIN and returns an exit code equal 0 if the file has a valid NoSQL 'list' structure, or 255 otherwise.

Example:

Input file inputfile:


                NAME    Bush
                COUNT   44
                TYP A
                AMT 133

                NAME    Hansen
                COUNT   44
                TYP A
                AMT 23

                NAME    Jones
                COUNT   77
                TYP X
                AMT 77

    

Command:

      nosql islist -v < inputtable
    

Message printed:

                list ok
    

and the code returned to the calling program will be 0.

4.19 istable

Usage: nosql istable [-v|--verbose] [-n|--no-header] < table

Checks that a file is a valid NoSQL table. This NoSQL operator reads a file from STDIN and returns an exit code equal 0 if the file is a valid table, or 255 otherwise. If option '-n' is specified (together with option '-v'), then any failing row numbers printed to STDERR will not include the table header.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql istable -v < inputtable
    

Message printed:

                table ok
    

and the code returned to the calling program will be 0.

4.20 join

Usage: nosql join [options] table1 table2

where table1 is the "primary" table, while table2 is the "secondary" one.

Options:

-j|--column column

Join on column from both files.

-1 column

Join on column from file 1.

-2 column

Join on column from file 2.

-a|--all n

In addition to the normal output, produce a line for each unpairable line in file n, where n is 1 or 2 ("Master/Detail" join).

-J|--suppress-join-column

Exclude the join column (leftmost column) from the output table.

-x|--debug

Debug. Print the join(1) program to STDERR before executing it.

Joins two tables on a common field, using the Unix join(1) program. The two tables must be sorted on the respective join fields for the operation to function correctly. Either one or the other, but not both, of the two input tables must be specified as '-', meaning STDIN. See join(1) for more details on the meaning of each option. If neither '-j' nor '-1|-2' are specified, then the two tables are joined on their respective first (leftmost) fields. If any of the two input tables contain duplicated columns, i.e. columns with the same name but possibly different values, only the first (leftmost) one is taken into account. The output will still have the duplicates, but this time with equal column values.

A natural join produces a new table that contains only rows from the input tables that match on the specified columns (key columns). A master-detail join produces a new table that contains all rows from the master table and those rows from the secondary table that match.

If different column names are specified, the name of the join column in the output table will be from table_1.

The order of columns in the output table will be: first the join column (unless '--suppress-join-column' is specified), then the other columns from table1, then the other columns from table2.

This NoSQL operator reads two tables, one from STDIN and the other one from a file, and writes a table to STDOUT.

If we have the table (named samplej) here:

      name     nr     typ     amt
      ----     --     ---     ---
      Bush     1      A       133
      Bush     2      A       134
      Hansen   3      A       143
      Hobbs    4      B       144
      Hobbs    5      B       144
      Jones    6      C       155
      Perry    7      D       244
      Perry    8      D       311
    

and the table (named samplej2) here:

      name     cnt    typ     amt
      ----     ---    ---     ---
      Hobbs    41     A       141
      Hobbs    42     BB      142
      Hobbs    51     BB      144
      Hobbs    43     CC      143
    

then the command to do a natural join of samplej and samplej2 on column name is either:

      nosql join -j name - samplej2 < samplej
    

or:

      nosql join -j name samplej - < samplej2
    

and the result is shown in Table 6.

                          Table 6

         Natural join of tables SAMPLEJ and SAMPLEJ2

      name    nr      typ     amt     cnt     typ     amt
      ----    --      ---     ---     ---     ---     ---
      Hobbs   4       B       144     41      A       141
      Hobbs   4       B       144     42      BB      142
      Hobbs   4       B       144     51      BB      144
      Hobbs   4       B       144     43      CC      143
      Hobbs   5       B       144     41      A       141
      Hobbs   5       B       144     42      BB      142
      Hobbs   5       B       144     51      BB      144
      Hobbs   5       B       144     43      CC      143
    

The command to do a "masterdetail" join of the same two tables on column name is either:

      nosql join -a 1 -j name - samplej2 < samplej
    

or:

      nosql join -a 1 -j name samplej - < samplej2
    

and the result is shown in Table 7.

                           Table 7

       Master-detail join of tables SAMPLEJ and SAMPLEJ2

      name    nr      typ     amt     cnt     typ     amt
      ----    --      ---     ---     ---     ---     ---
      Bush    1       A       133
      Bush    2       A       134
      Hansen  3       A       143
      Hobbs   4       B       144     41      A       141
      Hobbs   4       B       144     42      BB      142
      Hobbs   4       B       144     51      BB      144
      Hobbs   4       B       144     43      CC      143
      Hobbs   5       B       144     41      A       141
      Hobbs   5       B       144     42      BB      142
      Hobbs   5       B       144     51      BB      144
      Hobbs   5       B       144     43      CC      143
      Jones   6       C       155
      Perry   7       D       244
      Perry   8       D       311
    

Note that these two forms are equivalent:

      nosql join -j name ....
    

      nosql join -1 name -2 name ....
    

Refer to the man page join(1) for more information on how the Unix join utility works. See also section pull for a faster alternative to 'join', that can be used in some cases.

4.21 justify

Usage: nosql justify [-w|--width width] < table

Options:

-w|--width width

Truncate output columns to width characters.

Makes a table more readable by adding spaces around fields.

This NoSQL operator reads a table from STDIN and writes a table to STDOUT, left-justifying string columns and right-justifying numeric ones. Justification is performed by padding with spaces where needed. The output stream is still a valid NoSQL table, just with more blanks.

Example:

Input table inputtable:

          NAME  TIME       DATE        NUM     FLOAT       USD
          ----  ----       ----        ---     -----       ---
          Bush  19:02:05   1998.09.13  12345   2345.234    1,234.01
          Hobbs 19:22:05   98.09.23    -1234   12345.2345  0.56
    

Command:

      nosql justify < inputtable
    

Output on STDOUT:

      NAME        TIME         DATE     NUM        FLOAT        USD
      -----   --------   ----------   -----   ----------   --------
      Bush    19:02:05   1998.09.13   12345    2345.2340   1,234.01
      Hobbs   19:22:05     98.09.23   -1234   12345.2345       0.56
    

Here are a few examples of supported data types :

zxZZ123

String

12345

Integer

-12345

Integer

+12345

Integer

1234.5

Float1

-1234.5

Float1

+1234.567

Float3

19:02:05

Time (HH:MM:SS)

39:02:05

String (HH > 24)

1998.07.28

Date

98.07.28

Date (Year:MM:DD)

123498.07.28

Date

98.07.28

Date

98.07.99

String (DD > 31)

1,346.99

Currency

-1,346.99

Currency

1,955,346.99

Currency

3425,251.00

String (leading group of digits longer than 3)

425,251.0

String (Decimal part is incomplete).

0.56

Float or Currency, depending on context.

+0.56

Float or Currency, depending on context.

4.22 listtotable

Usage: nosql listtotable [-t|--tabsize size] < table

Options:

-t|--tabsize size

Tabulator size. Any TAB characters in the data portion of input fields are replaced by size blank spaces (default is 4).

Converts a file in "list" format into a table. Long data fields may be folded over multiple lines in the input file, provided that each continuation line begins with one or more spaces and/or tabs.

Example:

Input file inputlist:


                NAME    Bush
                COUNT   44
                TYP A
                AMT 133

                NAME    Hansen
                COUNT   44
                TYP A
                AMT 23

                NAME    Jones
                COUNT   77
                TYP X
                AMT 77

                NAME    Perry
                COUNT   77
                TYP B
                AMT 244

                NAME    Hart
                COUNT   77
                TYP D
                AMT 1111

                NAME    Holmes
                COUNT   65
                TYP D
                AMT 1111

    

Command:

      nosql listtotable < inputlist
    

Output on STDOUT:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

4.23 lock

Usage: nosql lock table [ table ... ]

Locks one or more tables by creating a lock (semaphore) file with the same name as the specified table, with the string ".LCK" appended. See section Data access control section further down in this manual for more details. This operator honours the shell variable NSQLOCKER, that can specify an alternative lockfile creation program to be used in place of the 'lock' script provided with NoSQL.

Example in the Bourne shell syntax:


    if nosql lock table ; then
       .... do some modifications to table ...
       nosql unlock table
    else
      echo "The table is already locked"
    fi
    

Of course all programs must abide by this locking scheme for the latter to be effective. Note that we must remove the lock explicitly with nosql unlock table afterwards, or the table will remain unavailable (i.e. it will remain 'locked'). See the 'unlock' operator further down in this manual.

4.24 maketable

Usage: nosql maketable < template

Builds a valid table header from a template file. Reads a template file from STDIN and prints the corresponding table header to STDOUT. A template file is very similar to a NoSQL 'list' file, except that it only contains column names, without the associated values, plus optional additional information like table documentation comments and column specific comments, i.e.:


      # These are lines of table documentation. They can be of any length,
      # and occur anywhere in the file, not just at the beginning.
      # Each line must start correctly, e.g with "#", surrounded by any
      # number of spaces and/or tabs.

      Name      Name of item
      Type      Type: 1,2,3,7,9,A,B,X
      Count     Number of items
      K         Constant modifier
      SS7       Special status for type 7
      Size      In Kilobytes
    
    

Column names (on the left-hand side) must be separated from the associated optional comments by tabs and/or spaces. Comments (anything to the right of a column name) can span over multiple lines, provided that the first non-blank character of each continuation line be a hash (#).

If we feed the above template file to 'maketable' with:

      nosql maketble < table.tpl
    

here is what we get on STDOUT:

                Name    Type    Count   K   SS7 Size
                ----    ----    -----   -   --- ----
    

The original file.tpl should be kept in the same directory as its associated table, for documentation.

Column names in the template file may optionally be preceeded by the column position number in the table header, i.e.:


      # These are lines of table documentation. They can be of any length,
      # and occur anywhere in the file, not just at the beginning.
      # Each line must start correctly, e.g with "#", surrounded by any
      # number of spaces and/or tabs.

      1  Name      Name of item
      2  Type      Type: 1,2,3,7,9,A,B,X
      3  Count     Number of items
      4  K         Constant modifier
      5  SS7       Special status for type 7
      6  Size      In Kilobytes
    
    

Field numbers, if present, must be separated from column names by one or more spaces and/or tabs.

4.25 merge

Usage: nosql merge [options] < old_table column ... merge_table

Options:

-a|--add

Add option. Add rows where the key column(s) match.

-d|--delete

Delete option. Delete rows where the key column(s) match and the data value in the delete column is equal to the delete string, "..DEL.." (without the quotes) by default.

-dSTG|--delete=STG

Like the delete option above but use 'STG' as the delete string.

-n|--strip-header

Strip header from output.

-r|--reverse

Reverse option. The tables are sorted in reverse order.

-sub|--subset

Subset case. The structure of merge_table is a subset of the structure of old_table.

-x|--debug

Debug option.

This operator merges and/or deletes rows of 'old_table' based on data values in 'merge_table' in the specified column(s). Both tables should be sorted on the specified column(s).

In the normal case, one or more rows in 'merge_table' either replace one or more existing rows in 'old_table' if the key column(s) match, or are inserted in order if the key column(s) do NOT match.

If the delete option is specified on the command line, one or more existing rows in 'old_table' will be deleted if there is a key column(s) match and the data in the delete column is equal to the delete string, "..DEL.." (without the quotes) by default. The delete column is the first non-key column in 'merge_table'.

Both tables should have similar data structures. The header for the new table is taken from 'merge_table', thus allowing a change of header information to be made.

This operator reads two tables, one from STDIN ad the other one from a file, and writes a table via STDOUT. Options may be abbreviated.

4.26 mjoin

Usage: nosql mjoin [options] < table_1 column[=column_2] ... table_2

Options:

-c|--cartesian

Do a cartesian (cross-product) join.

-m|--master-detail

Do a "Master/Detail" join. The table from STDIN is the master.

-n|--strip-header

Strip header from output.

Does a join of two tables on the (Key) column(s) specified. Unlikely the standard join, that operates on one single field per input table, mjoin can operate on multiple fields. The disadvantage is that mjoin is slower than join on large tables.

The default is a "natural" join, with optional "Master/Detail" or cartesian (cross-product) type joins.

Each item in the list of column name(s) specifys a key column, which may be different in the two tables, i.e. '=column_2', if given, refers to a name in table_2 that corresponds to 'column' in table_1. If '=column_2' is not given it means that the corresponding column name in both tables is the same.

If the key column names are different in the two tables, the name of the key columns in the output table will be from table_1.

Note that the two tables must be sorted on the key columns in order for a join operation to function correctly.

The order of columns in the output table will be: first the key columns, then the other columns from table_1, then the other columns from table_2.

This operator reads a table via STDIN and writes a table via STDOUT. Options may be abbreviated.

4.27 modeof

Usage: nosql modeof [-v|--verbose] file [ file ... ]

This operator takes a list of file names on the command line and prints the associated file permissions, in octal, to STDOUT. If '-v' is specified, then also the file name is printed to STDOUT, followed by the file permissions, with the two fields separated by a TAB character. If no files are specified then a usage message is printed to STDERR. If any of the files cannot be found, an error is printed to STDERR and a non zero exit code is returned.

Examples :

      nosql modeof file1 file2 file3 

      0755
      0600
      0666


      nosql modeof -v file1 file2 file3

      file1     0755
      file2     0600
      file3     0666
    

4.28 myprog

Usage: nosql myprog [pathmode] yourprogram [<] table

where yourprogram is any program you like, followed by its options and arguments, if any, and pathmode can be one of --path-first, --path-last, --path-asis (default=--path-first). The path option can be specified in short form, i.e. as -f, -l and -a respectively. See explanation below.

Runs a local program under NoSQL. The program to be run is expected to be in a directory pointed to by the shell variable NSQMYLIB if defined, or $NSQLIB/mylib by default. The possibility of running local programs under NoSQL can be useful in many circumstances. Due to the simplicity of the NoSQL table data format, operators from other similar packages can be used in place of or in addition to the NoSQL ones. Suppose you are looking for a functionality that is not provided by any of the available operators, but which can be performed by an operator from RDB, Starbase or /rdb (running /rdb operators requires that you purchase the associated software license from RSW), then you can copy that program to $NSQLIB/mylib and use it seemlessly from within NoSQL with :

      nosql myprog otherprogram ....
    

In alternative, you may write your own new operator and put it in $NSQLIB/mylib.

For example, suppose you want to run the Starbase column operator in place of the NoSQL one, because the former is an executable binary and sometimes executes faster. You can copy it to $NSQLIB/mylib and run it just like any other NoSQL program with:

      nosql myprog column ....
    

This operator sets the program execution PATH in a way such that $NSQLIB/mylib is searched first, followed by whatever was in your PATH before. This is because sometimes there are operators which name conflicts with system utilities. The Starbase 'column' program conflicts with the /usr/bin/column system program. Putting $NSQLIB/mylib first in PATH ensures that other Starbase operators that want to use their 'column' program, will not pick the system utility with the same name (which does a completely different function). If for some reason you rather want $NSQLIB/mylib to come last in PATH, you can use 'nosql myprog --path-last'. If you rather do not want that $NSQLIB/mylib appares in PATH at all, you can use 'nosql myprog --path-asis'.

4.29 not

Usage: nosql not command ...

Reverts the exit status of command. If the command (or operator) exit code is 0, it gets turned to 255. Conversely, if it was non-zero it becomes 0. This can be useful for those UNIX shells that do not understand the syntax 'if ! command'.

For instance, if you want to test whether a file is a valid NoSQL table you would normally do :


    if nosql istable < table ; then
      echo "The input file is a valid NoSQL table"
    else
      echo "The input file is NOT a NoSQL table"
    fi
    

You can revert the test in this way :


    if nosql not istable < table ; then
      echo "The input file is NOT a NoSQL table"
    else
      echo "The input file is a valid NoSQL table"
    fi
    

4.30 null

Usage: nosql null [options] column ...

Options:

-a|--all

Return 0 if all the specified columns are null (default).

-A|--any

Return 0 if any of the specified columns are null.

-b|--also-blank

Treat blanks as null.

-v|--verbose

Report null fields verbosely to STDERR.

Takes a NoSQL table on STDIN and checks the specified fields for null values. Returns 0 if the match is successful, or 255 otherwise. If any non-existent columns are specified on the command line, those columns will match the null value by definition, and the final result code will be either 0 or 255 depending on whether '--any' or '--all' (default) was specified. A column is considered to be null if all of its records are null.

4.31 pick

Usage: nosql pick [-R|--raw-input] [-x|--hex] table < offsetlist

This operator takes a list of byte-offsets on STDIN and prints on STDOUT the corresponding table records from the table specified as a command line argument. If option '-x' is given, then the offsets are expected to be base 16 numbers instead of the usual base 10 numbers. The offsetlist on STDIN can be built with the 'index' operator, and 'pick' expects it to be a one-column table, like this:

           lpos
           ----
           228
           1117
           518
           1225
    

The above format can be obtained from a NoSQL secondary index file quite simply with the command :

      nosql column lpos < index_table
    

The column name, 'lpos' in the example, can take any name, provided that the list has a valid NoSQL table format. If option '-R' is given, then the offsets are expected to be a blank- , tab- or newline-separated raw list of numbers on STDIN, i.e. they are not expected to to have a valid NoSQL table header.

Examples :

      nosql column lpos < index_table | nosql pick table
    

or

      echo "228 1117 518 1225" | nosql pick -R table
    

or even

      cat <<_EOF_ | nosql pick -R table
      228
      1117 518
      1225
      _EOF_
    

This NoSQL operator uses the fseek(3) C library function and is extremely fast. It can be used to pick selected rows from a large, indexed table very quickly.

See also sect. index and search.

4.32 pull

Usage: nosql pull table < keytable

This operator takes a list of table keys on STDIN and pulls out the corresponding records from table. It is quite fast and can be used in place of 'nosql join' (see sect. join) if we just need to get the foreign key values of a relation between two tables. The records are pulled from table according to the key values specified in the keytable received on STDIN. The key coulmn(s) of interest must be the first (leftmost) field in both table and keytable. The two tables do not need to be sorted on the key field for 'pull' to work. All 'pull' does is building the proper command for egrep(1), so its speed is that of the latter utility. Although egrep performs a linear scan of table, it has proven extremely fast, even for very large tables (several megabytes in size, on a P-II 233 MHz CPU). The 'pull' operator stores the whole egrep search pattern in a shell variable. The UNIX shell used will limit the size of the egrep pattern, which implies a limit on the number of key items in keytable.

Examples :

Suppose that table contains the following data :

      name     nr     typ     amt
      ----     --     ---     ---
      Bush     1      A       133
      Hansen   3      A       143
      Hobbs    5      B       144
      Jones    6      C       155
      Perry    8      D       311
    

while keytablecontains :

      key      other
      ----     -----
      Bush     AA
      Perry    123B
    

the command

      nosql pull < keytable table
    

will produce the folowing output :

      name     nr     typ     amt
      ----     --     ---     ---
      Bush     1      A       133
      Perry    8      D       311
    

Note how the key (leftmost) columns in table and keytable need not be the same. The data rows and the output column names will be those of table.

4.33 print

Usage: nosql print [options] < table

Options:

-b0|--best=0

By default, when a multi-line record of output for each row is necessary (due to the width of the current window or terminal) the program will try to fill space at the end of lines that would otherwise be wasted by moving some columns. This option prevents the moving of any columns.

-b[N]|--best[=N]

This option attempts a "best fit" by rearranging columns (widest columns first). If 'N' is given the first N columns of the first line will not be moved.

-B|--big-fields

Handle very large data fields, e.g. over 1000 chars. This option takes longer but it works for any size data fields.

-f|--fold

Fold long data fields into multi line data based on field width. May be used with the '-t' option to limit the field width. Only a single line record of output is produced with this option.

-iN|--indent=N

Indent size of N spaces on 2nd and later lines of a multi-line record of output. Default is 4 spaces.

-lN|--line-width=N

Line length of N chars for output. Default is the width of the current window or terminal.

-pN|--page-size=N

Page size is of N lines. Default is the height of the current window or terminal. A value of zero '-p0' will turn paging off.

-PX[stg]|--print=X[stg]

Page headings and settings for printing. A two line heading is put onto each page: page number, current date, and an optional string (stg). Sets page length (in lines) and line length (in chars) according to the value of 'X' as follows:

             X: P  page: 60  line:   80     (default font size)
             X: R  page: 47  line:  116     (rotated default)
             X: A  page: 51  line:  125     (rotated 10 point font)
             X: 8  page: 63  line:  144     (rotated  8 point font)
             X: 6  page: 82  line:  192     (rotated  6 point font)
             X: W  page: and line: from current window size.
        

Other desired page and/or line size options may be set after this in the option list.

-sK|--separator=K

Separator 'K' (which may be multi char) placed between columns. Default is two spaces.

-t[N]|--trunc[=N]

Truncate data to the defined width. If N is given the width of printed fields will be limited to N chars.

-w|--window

List as many columns as possible in single line records that will fit in the current window or terminal width.

-x|--debug

Debug mode.

This operator is used for quick and easy printing of a table, in a simple but useful form. Columns containing only numbers are right-justified with blanks, while anything else is left-justified.

The printing of each row of data will be on one line if possible, but when multiple lines are necessary the second and later lines are indented for readability. Also when multiple lines are necessary a simple space availability algorithm is used to minimize the number of lines printed for each row of data. This may result in the order of some data values being rearranged from their order in the table. The '-b0' option can override this algorithm and force the same printing order as in the table.

This NoSQL operator reads a table from STDIN and writes a formatted report on STDOUT. Options may be abbreviated.

As an example using the sample table from the DATA section (named sample), the command to view this table would be:

      nosql print < sample
    

which would produce the output shown in Table 8. The same command with a page heading for printing:

      nosql print -PP < sample
    

produces the output as shown in Table 9. Using a table (named sample4) that has long data values, shown in Table 10, the command to print the table using the truncate option is:

      nosql print -t < sample4
  
                         Table 8

          Printing table 'sample' using 'nosql print'

                NAME    COUNT  TYP     AMT
                ------  -----  ----  -----
                Bush       44  A       133
                Hansen     44  A        23
                Jones      77  X        77
                Perry      77  B       244
                Hart       77  D      1111
                Holmes     65  D      1111
  

                         Table 9

    Printing table 'sample' with page heading using 'nosql print'

             Page   1   Mon Dec  2 16:56:43 PST 1991

                    NAME    COUNT  TYP     AMT
                    ------  -----  ----  -----
                    Bush       44  A       133
                    Hansen     44  A        23
                    Jones      77  X        77
                    Perry      77  B       244
                    Hart       77  D      1111
                    Holmes     65  D      1111

                          Table 10

    Table with long data values (sample4) actual content

    name<T>type<T>contact<T>contents<N>
    10<T>4<T>21<T>20<N>
    Hansen<T>AAA<T>R. Starr at the UCLA & USC<T>Duplicate data under
    processing order number 55-7.<N>
    Hart<T>CCC<T>Hobbs/Emerson at RAND Corporation<T>85 files, 2 per
    day containing 12 and 24 hour reports.<N>
    Hobbs<T>EEE<T>Marshall at Universal AFB<T>Original PAF messages.
    Both sets are incomplete.<N>
    Bush<T>KKK<T>General USAF personnel<T>Duplicate ATO messages,
    incomplete.<N>
    Lender<T>RRR<T>Army base in Nevada<T>Nothing.<N>
    Emerson<T>UUU<T>Navy at Washington DC<T>More than we thought at
    first.<N>
        

which will produce output with the data values truncated to the defined column width as in Table 11. Using the same table with the fold option:

      nosql print --fold < sample4
    

produces output with the long data values 'folded' within their defined column widths as shown in Table 12. Note that each line is repeated until the entire data value for each column is completely shown. This makes this type of output variable length.

If you need a quick and easy way to look at the data in an table use the -window option. This option will cause 'print' to list as many columns as possible in single line records that will fit in the current window or terminal width. Note that you do not have to type the column names (or even know them) to use this option.

                           Table 11

   Printing table 'sample4' with 'nosql print --truncate' option

   name        type  contact                contents
   ----------  ----  ---------------------  --------------------
   Hansen      AAA   R. Starr at the UCLA   Duplicate data under
   Hart        CCC   Hobbs/Emerson at RAND  85 files, 2 per day
   Hobbs       EEE   Marshall at Universal  Original PAF message
   Bush        KKK   General USAF personne  Duplicate ATO messag
   Lender      RRR   Army base in Nevada    Nothing.
   Emerson     UUU   Navy at Washington DC  More than we thought
    


                           Table 12

     Printing table 'sample4' with 'nosql print --fold' option

   name        type  contact                contents
   ----------  ----  ---------------------  --------------------
   Hansen      AAA   R. Starr at the UCLA   Duplicate data under
                     & USC                  processing order 
                                            number 55-7.
   Hart        CCC   Hobbs/Emerson at RAND  85 files, 2 per day
                     Corporation            containing 12 and 24
                                            hour reports.
   Hobbs       EEE   Marshall at Universal  Original PAF
                     AFB                    messages.  Both sets
                                            are incomplete.
   Bush        KKK   General USAF           Duplicate ATO
                     personnel              messages,
                                            incomplete.
   Lender      RRR   Army base in Nevada    Nothing.
   Emerson     UUU   Navy at Washington DC  More than we thought
                                            at first.
    

It may be combined with the -t option to increase the number of columns of data shown on each line at the expense of some column width.

For example the command 'nosql print < d11c' on an 80 character wide window or terminal produces the following:

    name    count            type  amt    n1                        n3
    ------  ---------------  ----  -----  ------------------------  ------------
        n2                        n4                        n5
        ------------------------  ------------------------  ------------
        n6            n7
        ------------  ------------
    Bush    3                A     133    alpha22.307               117722
     baker                     DDBBx17                   other
        124567        8GGXXH17
    Hansen  39               A     23     beta222.307               117723
    charlie                   DDBBx18                   data
        1239870       GGXXH17
    Newton  8                E     8      gama22.333                117724
    dog                       DDBBx19                   exists
        1239870       GGXXH17
    Hobbs   42               B     144    delta3.3.118              117725
    echo                      DDBBx20                   here
        1239870       GGXXH17
    Hart    2                C     55     epslion33.118             117726
    foxtrot                   DDBBx21                   also
        1239870       GGXXH17
    

This is readable, but not very nice to look at, and even worse if there are more columns. The command

      nosql print -w < d11c
    

produces:

 name   count           type amt   n1                       n3
 ------ --------------- ---- ----- ------------------------ ------------
 Bush   3               A    133   alpha22.307              117722
 Hansen 39              A    23    beta222.307              117723
 Newton 8               E    8     gama22.333               117724
 Hobbs  42              B    144   delta3.3.118             117725
 Hart   2               C    55    epslion33.118            117726
    

Not all the data is listed, but the first few columns (sometimes the most important) are easier to view. The command

      nosql print -w -t6 < d11c
    

shows even more of the data, at the expense of some data width:

 name   count  type amt   n1     n2     n3     n4     n5     n6     n7
 ------ ------ ---- ----- ------ ------ ------ ------ ------ ------ ------
 Bush   3      A    133   alpha2 baker  117722 DDBBx1 other  124567 8GGXXH
 Hansen 39     A    23    beta22 charli 117723 DDBBx1 data   123987 GGXXH1
 Newton 8      E    8     gama22 dog    117724 DDBBx1 exists 123987 GGXXH1
 Hobbs  42     B    144   delta3 echo   117725 DDBBx2 here   123987 GGXXH1
 Hart   2      C    55    epslio foxtro 117726 DDBBx2 also   123987 GGXXH1
    

4.34 pwcrypt8

Usage: nosql pwcrypt8

Takes the environment variable PW_CLEAR and prints its encrypted equivalent to STDOUT, in the usual 13-byte format. Handles cleartext passwords of max. 8 characters.

For example, the command

      PW_CLEAR="MyPasswd" nosql pwcrypt8
    

will print something like 'vAmi1OYRFzYMQ' to STDOUT. If PW_CLEAR is unset, then nothing will be printed.

This operator can be useful to manage files with user IDs and passwords, stored in NoSQL format.

4.35 pwmatch8

Usage: nosql pwmatch8 [-v|--verbose]

Takes two environment variables, PW_CLEAR and PW_CRYPT, and checks whether the former is the plaintext version of the latter. Returns 0 if it is, or 255 if it is not.

Handles cleartext passwords of max. 8 characters, while PW_CRYPT must be in the usual 13-byte format. If either variables are longer they will be truncated to fit.

For example, the command

      PW_CLEAR="MyPasswd" PW_CRYPT="1tsMvBcaBkx0E" nosql pwmatch8
    

will return 0 if PW_CRYPT is the encrypted version of PW_CLEAR, or 255 otherwise.

This operator can be useful to manage files with user IDs and passwords, stored in NoSQL format.

4.36 record

Usage: nosql record from [to] < table

Takes a record number and outputs the corresponding row of the input table. If two numbers are specified, then outputs the corresponding range of records. If the second number is lesser than the first, then all records starting from the first record to the end of table are printed. If no record or if an out-of-range record is specified, then only the table header is printed.

This NoSQL operator reads a table from STDIN and writes a table to STDOUT.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql record 3 5 < inputtable
    

Output on STDOUT:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Jones   77      X       77
                Perry   77      B       244
                Hart    77      D       1111
    

4.37 rename

Usage: nosql rename oldname newname < table

Renames a specified column. This operator takes exactly two arguments: 'oldname' and 'newname'. If more arguments are specified, they are silently disregarded. If neither 'oldname' nor 'newname' are specified, then the input table is simply printed to STDOUT unchanged. The same applies if only one name is specified.

This NoSQL operator reads a table from STDIN and writes a table to STDOUT.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql rename COUNT NUMBER < inputtable
    

Output on STDOUT:

                NAME    NUMBER  TYP     AMT
                ----    ------  ---     ---
                Bush    44      A       133
                Hansen  44      A       23
                Jones   77      X       77
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

4.38 repair

Usage: nosql repair [-F|--filler string] < table

Appends missing trailing tabs to data rows of a table. Takes table rows that may lack trailing empty fields, and pads them with TAB characters up to the No. of fields in the header. Such broken tables are often the result of manipulating tables with common spreadsheet programs. If option '-F string' is specified, then padding is done using whatever specified by string instead of using NULL fields. This will make added fields easier to spot visually.

To check whether a table is broken, the NoSQL operator see (explained further down) may be used:

Input file brokentable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X
                Perry   77      B       244
                Hart    77      D
                Holmes  65      D       1111
    

There are missing values in column AMT, but this could be ok if the corresponding TAB characters are there. To check that out we do :

      nosql see < brokentable
    

and what we get on STDOUT is :

                NAME^ICOUNT^ITYP^IAMT$
                ----^I-----^I---^I---$
                Bush^I44^IA^I133$
                Hansen^I44^IA^I23$
                Jones^I77^IX$
                Perry^I77^IB^I244$
                Hart^I77^ID^I$
                Holmes^I65^ID^I1111$
    

the see operator represents TAB characters with '^I' and newlines with '$'. As we can see, there is a missing TAB at the end of line 3 (Jones), while at line 5 the missing data value is correctly replaced by a TAB (followed by the trailing newline). To repair the table we do :

      nosql repair < brokentable | nosql see
    

And what we now get on STDOUT is :

                NAME^ICOUNT^ITYP^IAMT$
                ----^I-----^I---^I---$
                Bush^I44^IA^I133$
                Hansen^I44^IA^I23$
                Jones^I77^IX^I$
                Perry^I77^IB^I244$
                Hart^I77^ID^I$
                Holmes^I65^ID^I1111$
    

where line 3 (Jones) is now fixed.

4.39 rmcol

Usage: nosql rmcol columnname ... < table

Removes the specified column(s) from a table. Nonexisting columns specified on the command line are silently ignored. If only invalid columns are specified, then nothing is deleted.

This NoSQL operator reads a table from STDIN and writes a table to STDOUT.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql rmcol TYP COUNT < inputtable
    

Output on STDOUT:

                NAME    AMT
                ----    ---
                Bush    133
                Hansen  23
                Jones   77
                Perry   244
                Hart    1111
                Holmes  1111
    

4.40 row

Usage: nosql row [-x|--debug]'selection expression' < table

Options:

-x|--debug

Debug option. Print the actual AWK program to STDERR before executing it.

Selects rows from the input table based on an arbitrary AWK expression using column names. Characters that are special to the UNIX shell must be quoted. The safest bet is to enclose the whole awk expression in single quotes, to prevent the shell from parsing it.

If the input table contains duplicated columns, with the same name but different values, only the first (leftmost) one is taken into account, but the output table will still have the duplicates.

This operator reads a table via STDIN and writes a table via STDOUT.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql row 'NAME=="Bush"' < inputtable
    

Output on STDOUT:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
    

Comments, i.e. unquoted # signs in the AWK expression are not supported. Pattern matching inside the expression should follow the same rules explained in the section about the compute operator.

4.41 search

Usage:

nosql search [options] inputtable < keytbl

or:

nosql search [options] -ind index_file [inputtable] < keytbl

Options:

-i|--index

Index file search.

-n|--strip-header

Strip header from output.

-p|--partial

Partial (initial) match. Applies to string type data only.

-s|--first-match

Only a single row match is needed.

-r|--reverse

Reverse sort option. Input table is sorted in reverse order.

-v|--no-stdout

Verify-only mode. If an item of info from keytbl is valid prints "ok", else an error message, on STDERR. NO new table is produced. Used by other processes for verification.

-x|--debug

Debug option.

This operator does a fast search of 'inputtable' (or index_file) using a binary search on a key of one or more columns. The table (or index_file) must be sorted on the key columns. Each column in the key may be of type string or type numeric (but be carefull with numeric data and exact matches). In the second form of usage for this operator if 'inputtable' is not given its name will be inferred from the name of index_file. For example if index_file is 'skb.x.typ' then the table name inferred will be 'skb.rdb'.

The column in the file 'keytbl' specifies both the key column name and the argument values to search for. File 'keytbl' is in the usual NoSQL table format.

Normally an argument value and a data field must compare exactly for a match to occur (exact match). If the paritial match otpion '-p' is selected, and if the argument value compares with the initial part of the data field it is considered a match. This applies to string type data only. Note that for numeric type data an exact match is always necessary.

Normally all rows that match will be written to the new table, in the same order as in the old table. If only a single row key match is appropriate some execution time can be saved by specifing the '-sgl' option.

This operator writes a table via STDOUT. Options may be abbreviated. Returns the number of non-finds at exit.

4.42 see

Usage: nosql see < table

TAB and newline characters that define a structure of a NoSQL table are normally not visible. Sometimes, especially for large tables, it is not easy to look for missing or extra tabs that make a table invalid. This operator is simply a call to the UNIX command 'cat -vte'. This command shows TAB characters as `^I' and newlines as '$', making it much easier to see what is wrong in the broken table.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql see < inputtable
    

Output on STDOUT:

                NAME^ICOUNT^ITYP^IAMT$
                ----^I-----^I---^I---$
                Bush^I44^IA^I133$
                Hansen^I44^IA^I23$
                Jones^I77^IX^I77$
                Perry^I77^IB^I244$
                Hart^I77^ID^I1111$
                Holmes^I65^ID^I1111$
    

4.43 setfirst

Usage: nosql setfirst [-S|--subsep string] columnname < table

Moves a specified column to the first position in table. Takes a specified column and moves it to the first (leftmost) position in table, while the column that was in the first position is exchanged with the former. If an invalid target column is specified, then the input table is printed unchanged to STDOUT. The same happens if no target column is specified. If option '-S string' is specified, then the column moved to the 1st position is then 'normalized' as a potential key column. Normalization consists of stripping leading and trailing blanks, and replacing multiple middle blanks between words with the sub-field separator specified by 'string'. This will usually be one single character, like a colon (:), a semicolon (;), a slash (/), etc., but it must not contain TAB nor newline characters, or the resulting output table will be broken. Characters that are special to the Unix shell must be quoted or escaped in 'string'.

This NoSQL operator reads a table from STDIN and writes a table to STDOUT.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql setfirst TYP < inputtable
    

Output on STDOUT:

                TYP     COUNT   NAME    AMT
                ---     -----   ----    ---
                A       44      Bush    133
                A       44      Hansen  23
                X       77      Jones   77
                B       77      Perry   244
                D       77      Hart    1111
                D       65      Holmes  1111
    

4.44 shelltotable

Usage: nosql shelltotable < varlist

Converts a list of shell variable assignments into a one-record table. Converts a list of shell variable assignments (VARIABLE=value), with multiple statements separated by newlines, into a NoSQL table.

This operator reads a newline separated list of VARIABLE=value pairs from STDIN and produces the corresponding table to STDOUT.

Example:

Command:

      env | grep ^H | nosql shelltotable
    

Output on STDOUT:

                HOSTNAME        HOSTTYPE        HOME        
                --------        --------        ------------
                tango           i486            /home/carlos
    

4.45 show

Usage: nosql show [c|w]

Shows NoSQL copyright terms (option c) and writer's information (option w), as suggested by the text of the GNU General Public License.

4.46 sort

Usage: nosql sort [options] [column] ... < table

Options:

-x|--debug

Debug option. Print the actual UNIX sort command line to STDERR.

Additional sort options that can be specified on the command line are '-cbdfinru'. See sort(1) for more info on what each option means.

Sorts a table on one or more columns, using the UNIX 'sort' utility. This operator reads a table via STDIN and writes a table via STDOUT. If no sort columns are specified, then the input table is sorted on entire rows. If nonexistent columns are specified, they are silently ignored.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql sort -n -r NAME AMT < inputtable
    

Output on STDOUT:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Holmes  65      D       1111
                Hart    77      D       1111
                Perry   77      B       244
                Bush    44      A       133
                Jones   77      X       77
                Hansen  44      A       23
    

4.47 soundex

Usage: nosql soundex [-n|--name name] column [table]

Compute the soundex codes for a table column. Takes a NoSQL table and prints the soundex code associated with each data value in a specified column.

This operator reads a table from STDIN and prints a new table to STDOUT. The output table is the input table with an added leading column (the 'soundex' column). The new column contains the soundex code of the data in 'column'. The new column name is the same as 'column', with an "s" prepended to it, or the name specified with option '-n' if present. For instance, if the requested column is 'Name', then the computed soundex column default name will be 'sName'.

Example :

input file table

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Holmes  65      D       1111
                Hart    77      D       1111
                Perry   77      B       244
                Bush    44      A       133
                Jones   77      X       77
                Hansen  44      A       23
    

      nosql soundex NAME < table
    

Output table :

            sNAME   NAME    COUNT   TYP AMT
            -----   ----    -----   --- ---
            N500    Holmes  65      D   1111
            Z000    Hart    77      D   1111
            H452    Perry   77      B   244
            H630    Bush    44      A   133
            P600    Jones   77      X   77
            B200    Hansen  44      A   23
    

4.48 split

Usage: nosql split [-o|--output] [headerfile] < table

Separates the header from the body of a table. Splits a table apart, separating the header section from the body. The header data is written to whatever file was specified on the command line, or to /dev/null if not specified.

This NoSQL operator reads a table from STDIN and writes the table header to a file and the table body to STDOUT.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql split headfile < inputtable
    

Output on STDOUT:

                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111


      cat headfile

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
    

4.49 subtotal

Usage: nosql subtotal [options] B_column ... [-s|--subtotal] column ...

Options:

-t|--total

Include totals.

-T|--total-only

Total Only Option. Just do totals on specified columns. In this case the '-s' flag is not necessary.

This operator lists subtotals of specified column(s) whenever the value of specified break columns(s) (B_column(s)) changes. A single break column will produce a sub-total of all specified columns on each line. If there is more than one break column given then in addition whenever the value of the first break column changes an additional line will be printed showing the sub-total for that group.

If no break column is given the first column is used; if no subtotal column is given then all columns of type numeric are sub-totaled.

This operator reads a table via STDIN and writes a table via STDOUT. Options may be abbreviated.

Example table (named small):

      name      amt    typ   count  n
      ----      ---    ---   -----  -
      Hansen    39     A     23     3
      Hansen    9      A     3      3
      Hansen    9      B     3      4
      Jones     42     B     144    5
      Jones     4      B     14     5
      Hart      9      C     3      5
      Hart      2      C     55     6
      Hart      2      D     55     6
      Hobbs     57     X     7      4
      Hobbs     5      X     57     4
    

The output from the command:

      nosql subtotal name -s amt < small | nosql print
    

would be:


      name      amt
      ------    ---
      Hansen     57
      Jones      46
      Hart       13
      Hobbs      62

    

The output from the command:

      nosql subtotal name typ -s amt count < small | nosql print
    

is shown in Table 18.


                             Table 18
  
               Output from the nosql 'subtotal' operator

                      name    typ     amt  count
                      ------  ---     ---  -----
                      Hansen  A        48     26
                      Hansen  B         9      3
                                       57     29
      
                      Jones   B        46    158
                                       46    158
      
                      Hart    C        11     58
                      Hart    D         2     55
                                       13    113
                      
                      Hobbs   X        62     64
                                       62     64
      
    

4.50 summ

Usage: nosql summ [options] [column ...]

Options:

-cu|--count-unique

A Count of the unique values for each column given.

-cun|--count-unique-null

Like option '-cu' but also shows counts of null (empty) and blank values (have only space chars), if either exist.

-cuu|--count-unique-each

A Count of each unique value for each column given.

-cu2|--count-unique-multiple

Like option '-cuu' but shows only counts greater than one.

-m|--compute

The min, average, max, total for each column given.

-v|--revert

Inverse option. Selects all columns except those named.

Produces "summary" information about a table. If no columns are given then information about all columns is produced. A Count of the data rows is always shown.

This operator reads a table via STDIN and writes a summary report via STDOUT. Options may be abbreviated.

4.51 tabletolist

Usage: nosql tabletolist < table

Converts a table to "list" format. This NoSQL operator reads a table from STDIN and converts it to "list" format on STDOUT.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql tabletolist < inputtable
    

Output on STDOUT:


                NAME  Bush
                COUNT 44
                TYP   A
                AMT   133

                NAME  Hansen
                COUNT 44
                TYP   A
                AMT   23

                NAME  Jones
                COUNT 77
                TYP   X
                AMT   77

                NAME  Perry
                COUNT 77
                TYP   B
                AMT   244

                NAME  Hart
                COUNT 77
                TYP   D
                AMT   1111

                NAME  Holmes
                COUNT 65
                TYP   D
                AMT   1111

    

4.52 tabletoshell

Usage: nosql tabletoshell [options] < table

Options:

-t|--truncate

Simply truncate the 'value' part of each assignment at the first occurrence of the single-quote char ('), instead of translating the latter to '&#39;'. See explanations below.

-p|--prefix P

Prefix each output variable (column) name with string 'P'.

Converts a single-record table to shell variable format (VARIABLE='value'), handy for grabbing a table row into a shell program. If the table contains more than one row of data, then the assignments will be those from the last row. Any single-quotes in the 'value' part of the assignment are escaped with the ASCII sequence '&#39;', not to cause troubles to the invoking shell, or they are used simply as upper boundary of the 'value' part of the assignment if '-t' is specified.

This operator reads a table from STDIN and prints the corresponding shell variable assignments to STDOUT.

Example of usage from within a shell script:

      eval `nosql tabletoshell < table`
    

4.53 template

Usage: nosql template < table

Builds a table template file for the input NoSQL table.

This NoSQL operator reads a table from STDIN and prints the corresponding table template file to STDOUT, in a format suitable for 'maketable'.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql template < inputtable
    

Output on STDOUT:

                    1      NAME
                    2      COUNT
                    3      TYP
                    4      AMT
    

4.54 trim

Usage: nosql trim [-m|--strip-all] [-e|--empty-rows] < table

Removes surrounding blanks from column values. If option '-m' is specified then multiple blanks between the words are stripped to one single blank. If option '-e' is specified then empty rows are removed from the input table.

4.55 ucfirst

Usage: nosql ucfirst < table

Turns to lower-case the column names and capitalizes the first letter of each one, to improve column name readability.

This NoSQL operator reads a table from STDIN and writes a table to STDOUT.

Example:

Input table inputtable:

                NAME    COUNT   TYP     AMT
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23 
                Jones   77      X       77 
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

Command:

      nosql ucfirst < inputtable
    

Output on STDOUT:

                Name    Count   Typ     Amt
                ----    -----   ---     ---
                Bush    44      A       133
                Hansen  44      A       23
                Jones   77      X       77
                Perry   77      B       244
                Hart    77      D       1111
                Holmes  65      D       1111
    

4.56 uniq

Usage: nosql uniq [options] column ... < table

Options:

-D|--count

Diagnostic output. Prints number of rows removed on STDERR.

-g|--groups

Group option. Instead of producing unique rows, produce only groups of repeated rows, with a blank row between. In this case the '-D' option shows the number of groups.

-v|--revert

Inverse option. Selects all columns except those named.

Reads the input table and compares adjacent rows. The second and succeeding copies of repeated rows, considering only the selected columns, are removed. That is, adjacent rows are considered equal if the data values in all of the selected columns are equal. The remaining rows are written to STDOUT.

Note that repeated rows must be adjacent in order to be found. Normally this means that the input table should be sorted on the selected columns for this capability to work properly.

This NoSQL operator reads a table from STDIN and writes a table to STDOUT. Options may be abbreviated.

4.57 unlock

Usage: nosql unlock table [table ...]

Removes the lock file(s) created by 'nosql lock' against the same table(s).

4.58 version

Usage: nosql version

Prints the NoSQL version and a short copyright notice to STDOUT.

4.59 weed

Usage: nosql weed [-o|--output] [weedfile] < table

Removes invalid records from a table. Discards any records whose number of TAB-separated fields does not match the number of fields in the table header. The failing rows are optionally written to the file specified on the command line, or to /dev/null if no output file is specified.

This NoSQL operator reads a table from STDIN and writes the purified table to STDOUT.

4.60 whatis

Usage: nosql whatis [string] Displays a short description of all the NoSQL operators which name contains string.

4.61 write

Usage: nosql write [options] [-o|--output] outputtable < inputtable

Options:

-f|--raw-file

Do not regard the input file as a NoSQL table, but rather as a generic, unstructured file. This must be used when the input file lacks a valid NoSQL header, or the write operation will fail.

-s|--no-stdout

Suppress writing the table to STDOUT.

-K|--key field

Keep the table sorted on field when writing the input table back to disk. If unspecified, the table is kept sorted on the first (leftmost) field by default.

-u|--unique

Besides sorting on the key field, the output table is also kept unique on the same field. Rows that are duplicated on that field will be silently dropped. This option should be used with care.

-o|--output

This option does nothing. It has been added for better compliancy with the GNU coding standards.

Reads a table via STDIN and [over]writes the table given as a command line argument, printing the output table also to STDOUT. The function of this operator is somewhat similar to that of the UNIX utility tee(1).

This operator can be safely used in constructs like :

      nosql compute ... < table | nosql write -o table | ...
    

without worrying that the original input table be destroyed before the first pipeline has completed.

If no output stream is desired on STDOUT then the latter can be directed to /dev/null, as usual, or the '-s' command line switch may be used.

The table may be an existing file, or it may be automatically checked out from RCS. In the latter case it will be checked back into RCS after the operation is complete. The default action is that if the table does not exist an attempt will be made to find the table under RCS.

To start RCS versioning on a table for the first time, the following command can be used:

      ci [-u] table
    

From now on, write will handle all interactions with RCS automatically.

WARNING: 'write' does not do any locking on the output table. Any locking, if necessary, must be handled by the invoking program.

Example of usage from within a Bourne shell script, with locking :


      if nosql lock outputtable
      then
        nosql compute ... < inputtable | nosql write -s -o outputtable
        nosql unlock outputtable
      fi
    

where inputtable and outputtable can be the same.

This fragment of code ensures that either the lock can be set, or no modifications to 'outputtable' will be done.

This operator is especially meant to be used in programs that need to modify tables 'in place'. It can optionally take a backup copy of the target table, which will have the same name as the destination table, prefixed by a comma (,).


Next Previous Contents