This program takes output files from MediaSystems Adora system and builds a
queue structure containing all the files suitable for a pulse database to

In order to process the file correctly, there is a ROUTING code either in the
FipHdr OR as a Tag in the data PLUS an (optional) FORMAT code for the name of a
parameter file to use.
    For ROUTING FipHdr QR or Tag <ROUTING> in the <START> section
    For FORMAT  FipHdr QF or Tag <FORMAT> in the <START> section

Valid ROUTINGs are
    CLASSLINKER - create a PCS ClassLinker structure
    PULSE       - create individual Pulse files and send to spool/2ftp
                with FipHdr
    SINGLE      - create single file per ad and add FipHdr
                plus DU:(fip-dest) and send to spool/2go
    ALL     - create one file for all ads and add FipHdr
                plus DU:(fip-dest) and send to spool/2go

The FORMAT code is use as the name of the parameter file in tables/setup and,
if not there, defaults to the ROUTING code.

The program first goes and gets its parameter file :
    tables/setup/XADORA.(Format Code)
    eg tables/setup/XADORA.PULSE for Pulse files.

The format of this file is :
    ; comment line
    toptag: XML tag which is the wrapper around a single ad
        Do not specify the '<>/' chrs.
        default: FIP-ADORA-DATA

    datatag: Tag signifying that Screen data follows
        Do not specify the '<>/' chrs.
        default: DATA

    convert-style:  AdTypes/Styles which need to be converted.
        (these two are exactly the same thing)
        There are normally several adtypes each specified on its own line
        convert-style:"Complete Text T"   break:<$f"HelveticaNeue-Condensed">
        Sub parameters are ( all optional):
            start:  (Markup to insert at the top of the data)
            end:    (Markup to insert after the data)
            break:  (Markup to insert when a BREAK code is found)
            pub:    (Pub code)
        The 'pub:(pub code)' is used to specify the markup for a
        particular publication code. Where there are multiple lines
        for the same Style, always specify the 'pub' lines
        before the default. Pub code refers to the <PUBCODE> tag.
    ignore-adtype: Adtype/Style to ignore for this Route.
    ignore-style:  Adtype/Style to ignore for this Route.

    xml-format: XML character format/style/name to be converted.
    justif-format: Format name to be converted.
        Sub parameters are the same a 'convert-style'
        Any formats NOT specified are just ignored.

    xml-font-change: XML font change from default for style
        default: change is ignored
        xml-font-change:"Euro Sans"   "<$f\042Euro Sans 33\042>"

    xml-leading-change:(FipSeq) Quark and other page makeup can change leading
using in-line markup
        Use this for mapping changes - the new value is in FipHdr field E0
        xml-leading-change:<*p(0,0,0,\E0,0,0,g,"International English")>

    extra-fiphdr: Extra FipHdr information for output files
        Used only for Types - PULSE and FIP
        default: none

    pulse-epsf-fiphdr:  FipHdr fields for pulse epsf and text
        These default to the adding a 'DF' FipHDr field which
        is used to select the FTP parameter file for that type.
        defaults are    pulse-epsf-fiphdr:#DF:PULSE.EPSF

    use-old-urn: If there is an Old URN, use that rather than the current.

    fiphdr-routing-field: FipHdr field containing the routing code if not QR
        eg fiphdr-routing-field:DU

    classlinker-ignore-subclass: Ignore the Subclass for ClassLinker

    hires-path: Main data path for the High Resolution data files
        default is /home/ds/lithos/highres

    map-drive: Map a PC drive letter to a Unix Path
        default for all is the 'data-path'
        syntax - map-drive:(drive letter) (spaces/tab) (UnixPath)
        eg  map-drive:K /home/ds/bigdisk/low-availability

        Force the case of all Header fields - URN, PUB etc.
        default is no-change.

    image:(name)    magic:(magic)   script:(script)
        Define an external script to be used for converting non-epsf
        hires files. See example below for the two types of TIFF.

    allow-top-spaces: Normally for all preceding spaces and blank lines
        are stripped from the top of the converted file.
        This preserves spaces etc.

    justif-file-before: Markup and other data to insert BEFORE any data
            in the output file.
    justif-before:  Markup and other data to insert BEFORE any data
            in the output file.
    justif-after:   Markup and other data to insert AFTER any data
            in the output file.
    justif-file-after: Markup and other data to insert AFTER any data
            in the output file.

        The 2 chrs which start and stop a format
        default is  justif-format-start:\xD1
    justif-meta:    Meta chr for formats chr    default:{
    justif-break:   Break Code chr      default:\320 (hex D0)
        The 2 chrs which start and stop a comment
        default is  justif-comment-start:\xEE
        The 2 chrs which start and stop hidden text
        default is  justif-hide-start:\xFD
    justif-chr: Replacement chr or string for a specific chr in the data.
        You will need to check 'Justif Manager' for the codes
        used on your system.
        This is used only for special chrs. Functions are covered
        by the other parameters. Generally only accented chrs and
        Pi's will need to be mapped.
        Both chr and string are in FipSeq and will be parsed.
        eg  justif-chr:\xDB <$f$BI>
            justif-chr:\xE0 \r
            justif-chr:@    <\\@>
        For xml entities, use \e9999 or \uAAAA to specify big numbers
            \e is a 4 digit DECIMAL number
            \u is a 4 digit HEX number
        These two give exactly the same result :
            xml-chr:\e8364  (EURO)
            xml-chr:\u20ac  (EURO)
        Remeber that both these take 4 digit numbers - so zero pad pls.
            xml-chr:\e0732  ^
        If a high end (ie not latin1 and > decimal 255) entity is not specified in
the 'xml-chr' parameters, use this to force the chr to a single default chr.
        default is '?'

    justif-end-of-line: Used to replace the CR in the data. default is <\\n>
        An optional 'pub:' parameter may be added for those Pubs which
        need a different eoln. These should be specified BEFORE the
            ; End of line - for Adsmart we just want the Space
            justif-end-of-line:\s   pub:AD

        use the <*J> etc commands at the beginning of the line (ie not at the end
where Justif has them)
        no  - do NOT output the command at start of each line (default)
        yes - do and start with setting as quad Left
        left    - do and start with setting as quad Left
        right   - do and start with setting as quad Right
        center  - do and start with setting as quad Centre
        justified - do and start with setting as quad Justified
        force - do and start with setting as quad ForceJustified
        (only the first letter is crucial)
    justif-qc:C Quark code for a justif Quad Centre - \xE0
    justif-qr:R Quark code for a justif Quad Right - \xE1
    justif-ql:L Quark code for a justif Quad Left - \xE2
    justif-qj:J Quark code for a justif Quad Justif/Middle  - \xE4
        Single Justif chrs for Quark Quads,
        These are flagged if want-quark-quads:yes
        To make the Justif QL be a Quark *J,

    res-fork-software:  EtherShare, Ushare, None
        default is ethershare
    res-fork-path:      Offset to the resource fork
        default is /.rsrc
    res-fork-type-epsf: 4 letter code for the EPSF type
        default: 'EPSF'
    res-fork-type-text: 4 letter code for the TEXT type
        default: 'TEXT'
    res-fork-creator:   4 letter code for the creator
        default: 'FIPO'

    ignore-first-eoln: (pub)    Make the first end of line a space
        for this pub..
    strip-multiple-spaces:  Used to do just that !
        Send all ClassLinker files to spool/2ftp and do NOT build a
        normal Classlinker file structure
        default is NO
        if FTPing Classlinker files, use this to NOT bother with the log
        default is YES if 'ftp-classlinker' has been set.

        Change the size of the URN - defaults to 6
    new-urn:    FipSeq to alter the URN
        eg to add a leading zero

    xml-hyphen: (FipSeq)
        Replacement text for a hyphen for the XML feed (ie NOT Justif)
            default is '-'
        For quark, pls remember to double any backies : xml-hyphen:<\\->


    xml-default-leading:8 for 8pts

ClassLinker normally writes it data files to a structure on its local drive. If
this happens to be on another system - NFS drives for example - it is a good
idea to check that drive is actually mounted beforehand.

A parameter file keyword, 'chkexists' can be used to check whether a specified
file or queue exists before the input queue is scavenged (or output written).
As filenames starting with a dot are ignored, one way to implement this is to
touch a file in this input queue. eg :
    cd /mountedOnRemote/spool/input
    touch .DontZapMePls
While in the parameter file we could have something like :
Or if the classlinker-path is /tweedle/dum
will look for a file or folder called '/tweedle/dum/Tester'

Input parameters are (all optional) :
    -1 : (path/filename) run this single file and stop  default: spooled
    -C : do NOT check EPSFs for embedded low res    default: do
    -d : done queue for incoming        default: none
    -e : extra suffix for Parameter files   default: none
        ie -e DTI and the ROUTING is FREEAD
        do not specify the '.'
        This suffix is also added to any DF: for ftp files.
    -i : input queue to scan        default: spool/xadora
    -l : do NOT log incoming files      default: log files
    -L : log ALL incidents plus files   default: log files
    -O : Check File/Queue name for output       default: none
        see above
        If the path starts with a '/', it is considered a hard path
        otherwise it is considered relative to the output queue.
    -w : file wait interval         default: 5 secs
    -x : debugging - show XML tags      default: no
        use only with the -1 single file option
    -X : debugging - show XML tags and stop.default: no
        use only with the -1 single file option
    -z : name of parameter file     default: XADORA
        in tables/setup if not default
    -v : print version no and exit

XML version

xml.a - formats/styles

Note that the CHARACTER styles/formats are decoded using 'xml-format'

So if the

; CharacterFormats
xml-format:"Ad Text"  start:"<$f\042Helvetica 45 Light\042>"
xml-format:"Ad Bold"  start:"<$f\042Helvetica 55 Roman\042>"
; Fractions are shrunk!
xml-format:"Fraction"   start:"<$f\042Helvetica Fraction\042h90"
xml-format:"Ad Reference Number" start:"<$f\042Helvetica 45 Light\042>"
xml-format:"Euro Symbol"  start:<$f\042Euro Sans\042>"

xml.b fonts - changing within a style

use xml-font-change where the charstyle stays the same but the font changes
xml-font-change:"Euro Sans"   start:<$f\042Euro Sans\042>"
on changing back, the full format is used as in 'xml-format'

xml.c - Watch out for high number entities/chrs

; <!ENTITY ensp  "&#8194;"> <!-- en space, U+2002 ISOpub -->
; <!ENTITY emsp  "&#8195;"> <!-- em space, U+2003 ISOpub -->
; <!ENTITY thinsp  "&#8201;"> <!-- thin space, U+2009 ISOpub -->
; <!ENTITY ndash   "&#8211;"> <!-- en dash, U+2013 ISOpub -->
; <!ENTITY mdash   "&#8212;"> <!-- em dash, U+2014 ISOpub -->
; <!ENTITY euro   "&#8364;"> <!--  euro sign, U+20AC NEW -->

xml-chr:\e8194  <\\s>
xml-chr:\e8195  <\\s><\\s>
xml-chr:\e8201  <\\p>
xml-chr:\e8211  -
xml-chr:\e8212  <\\m>
;;xml-chr:\u20ac    EURO

xml.d mapping bold, italic, underline and hyphens

    default is '-'
    default is '<b>'
    default is '<u>'
    default is '<i>'
    all these default to '<$>'

-- Example of converting TIFFs to EPSF using image alchemy
; What convertions do we want to do (double quotes to keep the spaces)
image:tiffII    magic:"II*\000" script:"/usr/forum/bin/alchemy \QN \QN.eps -Q -o
-e --."
image:tiffMM    magic:"MM\000*" script:"/usr/forum/bin/alchemy \QN \QN.eps -Q -o
-e --."

The script MUST generate a file in spool/epsf with the same filename as the
input with an Extra extension of '.eps'.

Note that FipHdr fields available for 'image':
    QF - (optional) input parameter giving Format file - WEB for XADORA.WEB
    QR - (optional) input parameter giving Routing - PULSE, CLASSLINKER, etc
    QN - Full Path and Filename of the input file to be converted
    SN - name of the input file to the this program

Example of input Data.
The Stored procedure produces an XML feed with three types of main tags
    START       - giving the parameters chosen.
    FIP-ADORA-DATA  - structure for each Ad
    EXIT        - stating number of ads processed and exit code.

<?xml version="1.0" standalone="yes" ?>
<PUB/>Canterbury Extra
<ADTYPE/>Space Booking P
Qse0,30m bd,%0RQse9,1RQu45Rp~nADTo jumble sale at the old school house in
nADTo}yQse9,0RQif %s ne 0Rb ohich; is great value, hon>
.... etc etc

FipHdr fields for PULSE and FIP/SINGLE

The following FipHdr fields are added to each indvidual Ad file created for
both Pulse and/or Fip routings :
    QT  AdType
    QP  PubCode
    QD  InsertDate
    QE  InsertDate as CCYYMMDD
    QC  Class
    QK  ClassKey
    QS  SubClass
    QO  oldURN
    QN  URN
    QH  Hires
    QX  full adinsert URN with insertion sequence no
    QM  Material
    QV  Matver
    QB  1000000 - Material number
    QY  1000000 - Matver number
    QW  OutputType - TEXT or EPSF
            any ad which has a ad-type or style specified in by 'convert-adtype:' or
'convert-style:' is TEXT; all others are EPSF

    (QL is used for FTPing the ClassLinker Log)
    (QF is used for FTPing the ClassLinker data resource fork folder name)
    (QZ is used for FTPing the ClassLinker data type folder name
        EpsfFiles or QuarkText)

Also remember QF and QR are FORMAT and ROUTING

-- For the XML variant,
    QJ is used as the Vertical in points when floating the ad vertically (only for
the top and tail)
        (if Centred - QJ is half the difference)
    Q1 is the total depth (<Height100mm> in points)
    Q2 is the total width (<Width100mm> in points)
    Q3 is same as QJ BUT is NOT present/blank if < 2pts (2pts is the min ptsize in
    Q4 is same as QJ BUT is minimum 3pts
    QL is used as the Horizontal in points when floating the ad vertically (only
for the top and tail)
        (if Centred - QL is half the difference)

    ; Q0 is an option on Q3 which is ONLY there if > 2pts
    option:Q0   Q3

Version Control
;003r4  31mar06 Adora XML version ;a-h XML
    ;i 20sep06 bugette trailing spaces removed in xml-chr too early
    ;j-l 21sep06 new XMLinternals (;l for bugette in justif-chr/xml-chr)
    ;m 22nov06 bug in XML where style has no font and bugette2 for xml-chr
    ;n-p1 31may07 added VertAlignment and conv-adtype ... end: (;3p1 TagAtt)
    ;p2 25jun08 add fiphdr for all EPSadd fiphdr for all EPSF
    ;q 31apr09 new, flexi tags...
    ;r1 18sep09 STYLE/STYLES are the same now ;2 20may10 slash in OldUrn ;3 5apr12
        ;4 18dec13 interpret leading-style
;002n   08sep00 'chkexits/wait' for NFS
    and started outputData and ftp for CLASSLINKER
    ;a-b 28feb01 cleanup plus ignore-adtype
    ;c-d added new-urn and allowed CR
    ;e 18aug03 added QM/QV
    ;f-h 05sep03 added quark justif <*C>R/L/J etc
    ;i 08nov03 added xED support for Justif warings (in Green)
    ;j-n 18aug04 added QB and QY for million - material/matver
        plus -e for extra Parameter name
        plus justif-file-before/after
;001a   29jul99 added FIP-SINGLE and QF/<FORMAT>
    ;a 28sep99 save Pub PP, PubDate PD, Section PS - better ALLname

(copyright) 2017 and previous years FingerPost Ltd.