pilot-qof: Extensions manual.

What is QSF?  QSF - QOF Serialization Format - lays out a QOF object in a series of XML tags. The format consists of two component formats:

QSF object files will contain user data and are to be exported from QOF applications under user control or they can be hand-edited. QSF maps contain application data and can be created by application developers from application source code. Tools may be created later to generate maps interactively but maps require application support as well as an understanding of QOF objects in the import and output applications and are intended to remain within the scope of application developers rather than users.

A QSF file written by one QOF application will need an appropriate QSF map before the data can be accessed by a different application using QOF. Any QSF objects that are not defined in the map will be ignored. QSF files written and accessed by the same application can use maps if necessary or can simply import the QSF data as a whole.

If you need to convert pilot-qof data into other formats, take a look at the example stylesheets installed with pilot-qof. You are welcome to contribute new or amended stylesheets - just post them on the QOF-devel mailing list. A few simple bash scripts are also included to show how to reference the installed XSL stylesheets.

These stylesheets convert all data in the QSF XML file. Use the query support within pilot-qof to filter out unwanted records before conversion.

Referencing pilot-qof XSL stylesheets. The examples in this manual reference pilot-qof XSL stylesheets by a unique resource identifier (URI) as opposed to a location (URL) on the local filesystem or the internet. This identifier serves to ensure that pilot-qof XSL is distinguishable from other XSL files with the same name on the system and, more importantly, to provide a more memorable and system-independent identifier. i.e. the URI will work on any system, no matter how the package maintainer configures the installation directories. To use these URI's (which in the case of pilot-qof refer to XSL at http://www.data-freedom.org/xsl/) you need to install not just xsltproc but also xml-core or "recursion errors" will occur. Both xsltproc and xml-core are recommended by pilot-qof and should be installed by anyone wanting to use the examples in this manual.

It could be considered rude to require network access for XSL conversions so although the examples use the data-freedom.org site as an identifier, the examples also specify the xsltproc option --nonet which prevents xsltproc from attempting to get the XSL from the internet. Currently, the stylesheets are not actually available online because the data-freedom.org string is used as an identifier, not a location, although it can be used as both.

Whilst some stylesheet content can be converted back into XML (and thence into QSF XML), this is not always possible. If two way conversions are important to you, please investigate the export functionality of the target programme and ask on the QOF-devel mailing list.

http://lists.sourceforge.net/lists/listinfo/qof-devel

Depending on your package manager, catalog.xml, the XSL stylesheets and scripts should be installed in /usr/share/xml/pilot-qof/xsl/ , or on Fink /sw/share/xml/pilot-qof/xsl/.

XSL stylesheets included with pilot-qof. When XML_CATALOG_FILES is exported correctly:

export XML_CATALOG_FILES="/usr/share/xml/pilot-qof/xsl/catalog.xml"
      

stylesheets can be referenced by the www.data-freedom.org/xsl/ URI in combination with the --nonet option to xsltproc. Current stylesheets are works in progress, but include:

Scripts. XSL conversions can require a lot of commands and options, most of which do not change between runs. pilot-qof includes sample scripts that demonstrate how to ease the process.

XML_CATALOG_FILES="/usr/share/xml/pilot-qof/xsl/catalog.xml" \
 xsltproc --nonet http://www.data-freedom.org/xsl/pilot-qof-calcurse-todo.xsl $1
    

$1 indicates the first argument passed to the script (in this case an XML file). XML_CATALOG_FILES is an environment variable, set just for this script. Where stylesheets in this manual can use other options, these are documented in a shortened form that omits the environment variable and the URL of the stylesheet.

Sample bash scripts are neither installed into system-wide locations nor available within the default system path. You are advised to copy the scripts into user-specific locations to make it easier to adjust the script behaviour.

You are free to copy and modify these stylesheets to your own requirements, including translations and customised formats. Depending on your package manager, this is often best done by copying the installed file to a local directory before modifying it. pilot-qof does not reference these stylesheets directly. If your modifications could be useful to others, please contribute them to pilot-qof via the QOF-devel mailing list or pilot-qof SourceForge project site.

http://sourceforge.net/projects/pilot-qof/

Localisation (l10n) of stylesheet output. Where possible, each stylesheet supports providing translatable strings and user-specific encodings via external parameters. This can make the command line very long so is best performed using a script. Each descriptive word in the output is configured as a string parameter for the stylesheet and can be replaced with a translated version. HTML output supports setting the HTML language (as would be specified in the <html> lang attribute) and the encoding (as would be specified in the <meta> charset value in the content attribute).

If you use these stylesheets via a scripting language - like bash, PHP or Perl - you could automate the translation by passing values obtained from a normal gettext PO file. Copy the translatable strings into your script file as normal variables then mark up those variables for translation by gettext. When the script is called, gettext will assign the translated values to the variables and your script can simply echo those values to the calls to the XSL parser routines.

It is important that the HTML language and the encoding match each other AND the expected content of the HTML output generated from the pilot-qof data.

When providing translated strings, the same constraints apply as if you were using gettext and a normal PO file: the context and format of the translation should match the intention expressed in the default value of the parameter. If the default is plural, the translation should be plural. If the default is capitalised, the translation should be capitalised - subject to grammatical rules for that language - even if the parameter name itself is not capitalised.

Each stylesheet specifies the translatable strings in a block near the top of the file, marked as for the attention of translators. Common settings, like the HTML language parameter and the encoding support, are in the pilot-qof-std.xsl stylesheet and are available in all HTML stylesheets by using the string parameters html_lang and encoding.

Examples. Convert a QSF XML file containing pilot_address records, into individual VCard .vcf files in the vcards/ directory - which must already exist. Files are named according to either the Company or FirstName_LastName of the pilot_address record.

$ xsltproc --nonet -o vcards/ http://www.data-freedom.org/xsl/pilot-qof-address-vcard1.xsl addresses.xml
      

Convert the output from pilot-qof --invoice-city or --invoice-vendor options into an HTML invoice. The invoice uses CSS in the invoice.css file. This stylesheet needs records from pilot_address, pilot_expenses and pilot_datebook databases but ignores pilot_todo records that may be present. Calendar data is used to create the main invoice, expenses from the same day are added and address data is added to describe the invoice.

$ xsltproc --nonet -o invoices/ http://www.data-freedom.org/xsl/pilot-qof-invoice-xhtml.xsl invoice.xml
      

This example overrides the default mileage-rate (0.30 currency units per unit distance) and the default hourly-rate (20 currency units per hour) provided by the stylesheet with user specific values.

$ xsltproc --nonet --stringparam mileage-rate 0.45 --stringparam hourly-rate 21 \
-o invoices/ http://www.data-freedom.org/xsl/pilot-qof-invoice-xhtml.xsl invoice.xml
      

The stylesheet currently defaults to Pounds Sterling and - depending on support in your shell - the currency-symbol can also be overridden:

$ xsltproc --nonet --stringparam currency-symbol $ \
-o invoices/ http://www.data-freedom.org/xsl/pilot-qof-invoice-xhtml.xsl invoice.xml
      

If your shell complains about using a currency-symbol on the command line, e.g. when trying to use Euro (€), copy the stylesheet to somewhere below your /home/ directory and edit the currency-symbol in the file. Wikipedia has a reasonably complete list of entity references: http://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references

Redirection, scripts and backups. Although it is fine to pipe pilot-qof output to other processes like xsltproc, (note the terminal dash - to denote that xsltproc should expect the XML content on stdin) :

$ pilot-qof -x whole_year.xml --invoice-city -t 2006-03-01 | \
xsltproc --nonet http://www.data-freedom.org/xsl/pilot-qof-invoice-xhtml.xsl -
        

Just remember that it might not be so wise to do this:

$ pilot-qof -a --invoice-city -t 2006-03-01 | \
xsltproc --nonet http://www.data-freedom.org/xsl/pilot-qof-invoice-xhtml.xsl -
       

Why? Because using -a means that your Palm data is not being backed up in this process (because the XML sent to xsltproc by the pipe is not saved and cannot be recreated from the HTML). Now, of course, you can sync your Palm twice but reading from the backup file is a lot faster.

Reporting Bugs. Please do NOT report bugs in pilot-QOF to pilot-link or GnuCash. Report bugs via the QOF-devel mailing list. http://lists.sourceforge.net/lists/listinfo/qof-devel

What is QOF?  QOF provides a set of C language utilities for performing generic structured complex queries on a set of data held by a set of C/C++ objects. This framework allows programmers to add query support to their applications without having to hook into a SQL Database. QOF provides the framework, the new QOF XML backend (QSF) provides the off-line storage and a format for a data stream between different applications, including converting QOF objects.

What does QOF have to do with pilot-link?  QOF frees the data from the application, allowing queries, data mining, abstraction and data interchange without having to recompile the original application, let alone program the filters. Central to QOF are the ideas of databases and objects. A Palm database is an object in QOF. A Palm database record is an instance of the object, called an Entity. QOF then collates all supported objects to form one data source that can be queried. QOF allows the programmer to easily find all other instances of any one type, just by having one instance of that type. Objects are collected in books and a book can contain any number of different objects - depending on the application running QOF. The book is the central data source within QOF and a book can be queried or written out to XML using the QOF Serialisation Format: QSF. See Chapter 1.

Timezone issues.  Pilot-link itself does not directly support users who cross timezones with their Palm - it may be wise that users are reminded that if timezone issues are important to their DateBook or Expenses data, the user should take the sensible precaution of changing the Palm Preferences to use the same timezone as the computer running pilot-qof. Once in QOF, all dates and times are UTC so the QSF XML files can be freely exchanged across timezones without complication.

The Palm default currency table includes old European currencies that have been absorbed into the Euro. pilot-qof uses € for the Euro symbol.

The ID is a Palm constant used to identify this currency internally and has no particular sequence or geographic / regional significance.

In the following table, the currency_code on the Palm is shown first. The country name is as specified in the Palm default currency table. The fraction is that used by the Palm itself (i.e. if you change an expense amount from a currency with a fraction of 100 to one with a fraction of 1, the expense amount is rounded down to the nearest whole unit. Any decimal places are lost). The symbol is the Palm symbol for that currency and may differ from symbols used elsewhere for the same currency. The mnemonic is the relevant GnuCash mnemonic for the matching gnc_commodity.

American users should note that the Palm uses $ for the Singapore Dollar. The American dollar is $US. Using the Singapore Dollar to force the Palm to show only $ instead of $US will cause conversion problems if you need to use pilot-qof data in GnuCash or CashUtil. This is a decision made by Palm, not pilot-qof.