nxwire

nxwire

This program connects to the BizWire NX v4 client FireHose and grabs files.

It allows ALL NX traffic for a server to be sucked in and treated like an
incoming data stream like a wire service.

A small FipHdr is added with date and time fields, sender and receiver logons
before the file is passed on - normally to spool/2brouted for 'iproute' to sort
out.

RoundRobin may be set for the output folder and also adds the RR FipHdr field

There are two modes
    - normal running where the program sits on the port processing data until
stopped manually
    - catchup mode which requests for any OLD files which have been queued
waiting.
        This instance will drag all these files off and then stop
        This is flagged by a '-C' input switch on startup.
When the program is started in 'normal running' mode, it will automatically
start a second instance in CatchUp mode.(use -X to inhibit the second instance)

The program reads a parameter file in tables/wire which has the same name as
the '-n' service.
Parameters may be
    ; comment
    log-level:0,10,20                   default: 0
        0-errors only
        10-plus connects/disconnects
        20- files too
    log: Custom log line for files (in FipSeq)      default: filename, queue, size
    extra-fiphdr : Extra FIP header information     default: none
        For fixed header info in FIP. eg #QA:AA#QB:BASIC
        As this flag is normally the last specified, its contents can be used to
overwrite any unique fields such as DU, DP, SN etc.
    dump-data:yes/no    Dump raw incoming data      default: no
    archive-name: Name of the archive file          default: same as Service
    max-round-robin: (no or 1-49)               default: no
        add a round-robin number to the output queue (same as -R)
    connection-retries: (goes)              default: 5
    connection-timeout: (secs)              default: 30
    reset-timeout: (secs)                   default: 30
    log-timeout: (secs)                 default: 90
    tcp-keep-alive: (y/n)                   default: yes
    tcp-ka-count: (probes)                  default: 2
    tcp-ka-interval: (secs)                 default: 60 secs
    tcp-ka-idle: (secs)                 default: 65 secs
        The default is keep-alives are ON; if nothing arrives in (idle) 65 secs, then
a KA is sent every (interval) 60 secs - but after 2 goes (count) without
response the connection is broken.

Plus the usual FipSeq - partial, filter, valid etc

Input Parameters :
    -n : Name of service (SU field)             default: NXWIRE
All Optional :
    -d : done folder                    default: none
        This holds a copy of all incoming data files from every source
        The structure is
            (done folder) / (date)_(logon) eg 20110921_fip / (filename as written to the
output folder)
        It can be purged with an entry in maintenance (zapfiplog)
        eg if '-d raw.data' and we want the last 30 days data
            /fip/bin/ipdelque -q/fip/spool/raw.data -i1 -a30
    -D : display connection and data (use only for debugging)   default: no
    -k : keep alive idle time               default: 65 secs
    -K : keep alive interval time               default: 60 secs
    -I : wire Id use to denote which nxwire if more than one. default: none
    -o : Output folder in /fip/spool            default: spool/2brouted
    -O : Name of output format (DF field)           default: NXWIRE
    -p : port number of the FireHose            default: none
        Normally NX v4 uses port 9001
    -r : the name of a DIFFERENT routing table to 'name'
        (SR field : used by iproute)            default: name
    -R : Running Round Robin on the output folder       default: no
    -s : hostname/internet address to select a test host    default: localhost
    -w : log timeout                default: 90
    -Z : do NOT archive any incoming files      default: archive
    -v : Print the version number and exit

Plus for CatchUps
    -C : Single shot catchup. The program stops on completion   default: normal
running
    -X : do NOT run a second CatchUp instance       default: if normal, do
        This is for normally running which will automatically run a CatchUp on
startup.

Version Control
;000t   23sep12 original version ;m funny catchup fudge - and redid catchup
    ;n added -d doneque
    ;o tuning ;p-r log max ;s-t 1may13 added tcp-keepalives

(copyright) 2017 and previous years FingerPost Ltd.