xmlwire - newsmlwire - nitfwire

This program will bring in and interpret an XML file especially NITF and NewsML
or yours-or-anybodyelse's-xml

It uses a parameter file to extract header fields from the incoming stream.
Usually this is to generate routing information.

The actual data - HEAD and BODY - is left as is.

** Note The data is sliced into files by specifying :
    Either  an soh/eot  which is the DEFAULT - start with a SOH - binary 1 - and end
with EOT - binary 4

    Or  an envelope-tag which is the top and tail main tag
                *** IF using envelope-tag, you MUST manually set soh:\000 and eot:\000 ***

The parameter file in tables/wire defaults to the name of the service (-n) and
has the keywords:
    fiphdr:(2-letter code)  (optional subkeywords)
        Either  tagdata:(name of tag)
                specify the tag name which contains the data required.
        Or  tagattrib:(name of tag),(name of attribute)
                specify the tag name and the attribute name which
                contains the data required.
        Or  data: (FipSeq)
                general data to add to a FipHdr field.
        For any of the tag options, use 'dup' to flag duplicated fields.
            dup:(optional separator)
                This field may be duplicated. Duplicate fields are separated
                with a space unless a separator chr is also specified.
                increment the Fiphdr field by this number each time
        For 'tagattrib', there can also be a 'key' parameter for selecting
        the data of an attribute ONLY if there is Key attribute with
        its data equal to a certain string:
            eg: if the tag is <meta name="category" content="f"/>
            fiphdr:NC   tagattrib:meta,content  key:name=category
            Double quotes around the Key Data are optional unless there
            are embedded spaces. The Key Data can be in FipSeq.
        As some FipHdr fields have distinct meanings - SN, DU, DP etc - please use 2
        codes starting N or Q.
    eg  fiphdr:NA   tagdata:itemid  dup:+
            get the data from each <ITEMID> field. If there is more than one,
            they are separated by a '+'.
    dest: (one or more Fip Destinations separated by space or '+')
                This can be overridden by the DX: FipHdr field. Note that all
                destinations MUST be in the tables/sys/USERS file. As per normal
                case is important, so ZAPME and zapme are 2 different destinations.
                eg. dest:logcopy+outsgml.
    extrafip: (FipSeq)  String to parse and add to the FipHdr as extra fields
                eg: extra:DB:rude#QH:YY
    before: (FipSeq)    String to parse and add at the top of the file.
    after: (FipSeq)     String to parse and add at the end of the file.
    number:octal|dec|hex    In FipSeq, make all escaped numbers Octal, Dec or Hex.
                default is octal
    soh: (FipSeq chr)   Start of Header character   default: SOH (binary 1)
                This MUST be a chr between NUL (0) and space (binary 32 decimal).
                eg: soh:\023
                ** If there is NO SOH chr, you MUST specify soh:\000
    eot: (FipSeq chr)   End of Transmission character   default: EOT (binary 4)
                This MUST be a chr between NUL (0) and space (binary 32 decimal).
                and different to 'soh:'
                ** If there is NO EOT chr, you MUST specify eot:\000
    hdrchr: (FipSeq string) : (FipSeq Chr or String)
                Translate Sgml escaped chr back into a single chr or a string
                Note that case IS important
                This will take &XXXX; and translate it.
                eg. hdrchr:lt:<
    tagNL: (FipSeq string)  Add a EndOfLine string BEFORE a start of tag
                eg tagNL:\r\r\n     default:\n for NL only
    endtagNL: (FipSeq string) Add a EndOfLine string BEFORE an End tag
                eg endtagNL:\r\n    default: nothing
    envelope-tag: (tag name)
    nitftag: (tag name)
        The main enveloping tag which wraps HEAD and BODY, if it
            is not <NITF>.....</NITF>.

        eg  nitftag:ttnitf
        The end tag of this will be used to finish the file if there
        is no 'eot:' specified.
            Do Not specify the '/', '<', '>' chrs
            Do Not specify <?xml... or !doctype.
    comment-tag: (tag name)
        Normally this is set to '!--' 
        Turn Off with a blank
    dump-data: This makes a copy in /fip/dump of all incoming raw data
    delay-queue: (path for delay)
    delay-tag: (tagname)
        If the date in this tag is in the future, the file is left in the
delayy-queue specified.
        The date should be in any CCYYMMDDHHNNSS format (only numbers are
        eg  2007-02-14 10:30:21
    ack-msg: (fipSeq)
        Send this string at the end of a successfully received message.
        eg To send the date and time
        default: none

Current Limitations are :
    No more than 2000 tags may be specified.

Note that start/end Paragraph tags <P> and </P> are stripped from FipHdr fields

Input Parameters :
    Mandatory :
    -n : name of this wire (usually the agency)     default: none
either  -s : 'px25', 'sx25', Spider name or TTY device name default: none
    -p : Terminal Server port number (Not for TTY)      default: none
or  -P : Port on UnixBox to use for outbound        default: none
        This is only used where Terminal Servers canNOT run reverse telnet
        correctly and need to set a Permanent Virtual Connection across
        the ethernet to a fixed port number on the UnixBox
    Optional :
    -A : name of the archive file if not the -n name field  default: 'name'
    -C : connection timeout                 default: 60 secs
        Timeout between attempts/messages if unable to connect.
    -d : the name of a DUPLICATE wire where 2 copies of the same
        file is required (SD header field).     default: none
    -L : For Passive (-P) type connections, reestablish the 
        link after disconnection            default: no 
    -o : Name of output format (DF field)           default: NITFWIRE
    -O : output folder in spool             default: spool/2brouted
    -q : quiet mode - not log Connects/Disconnects      default: do
    -r : the name of a DIFFERENT routing table to 'name'
        (SR field : used by iproute)            default: name
    -S : do NOT strip leading/trailing spaces from tags default: do
    -T : Timeout in seconds              default: none
        If this time is exceeded WITHOUT an End-of-file, the
        file is terminated.
    -x : Wakeup string for Pad/modem etc        default: none
    -y : Banner string to strip from terminal server.default: SpiderBanner
    -Y : There is no banner to strip        default: as above
    -Z : do NOT archive any incoming files      default: archive
    -v : Print the version number and exit

Version Control
;003s2  09may00 mods for AP including keys for tagattrib data
    ;a 05apr01 cleanups
    ;b-d 10jul01 morphed into XMLwire and added ftimeout
    ;e-g 19sep03 added -O output folder
    ;h speedy
    ;i-l 17feb05 added -q and dump-data
    ;m 06mar05 added delay-que/tag
    ;n-r 28aug06 dup FipHdr fields better (r big buffer overflow)
    ;s2 14may07 added ack-msg
;002e   28nov97 ignore/strip <P> from Fip headers etc ;a minors
    ;b/c  9dec97 inbuf up to WIRBUF plus added ohdrbuf;
    ;d 23apr98 woops - altzone for HG
    ;e 17jun98 endtagNL added

(copyright) 2017 and previous years FingerPost Ltd.