nxwire

This program connects to the BizWire NX v4 client FireHose and grabs files.

It allows ALL NX traffic for a server to be sucked in and treated like an
incoming data stream like a wire service.

A small FipHdr is added with date and time fields, sender and receiver logons
before the file is passed on - normally to spool/2brouted for 'iproute' to sort
out.

RoundRobin may be set for the output folder and also adds the RR FipHdr field

There are two modes
	- normal running where the program sits on the port processing data until
stopped manually
	- catchup mode which requests for any OLD files which have been queued
waiting.
		This instance will drag all these files off and then stop
		This is flagged by a '-C' input switch on startup.
When the program is started in 'normal running' mode, it will automatically
start a second instance in CatchUp mode.(use -X to inhibit the second instance)

The program reads a parameter file in tables/wire which has the same name as
the '-n' service.
Parameters may be
	; comment
	log-level:0,10,20					default: 0
		0-errors only
		10-plus connects/disconnects
		20- files too
	log: Custom log line for files (in FipSeq)		default: filename, queue, size
	extra-fiphdr : Extra FIP header information		default: none
		For fixed header info in FIP. eg #QA:AA#QB:BASIC
		As this flag is normally the last specified, its contents can be used to
overwrite any unique fields such as DU, DP, SN etc.
	dump-data:yes/no	Dump raw incoming data		default: no
	archive-name: Name of the archive file			default: same as Service
	max-round-robin: (no or 1-49)				default: no
		add a round-robin number to the output queue (same as -R)
	connection-retries: (goes)				default: 5
	connection-timeout: (secs)				default: 30
	reset-timeout: (secs)					default: 30
	log-timeout: (secs)					default: 90
	tcp-keep-alive: (y/n)					default: yes
	tcp-ka-count: (probes)					default: 2
	tcp-ka-interval: (secs)					default: 60 secs
	tcp-ka-idle: (secs)					default: 65 secs
		The default is keep-alives are ON; if nothing arrives in (idle) 65 secs, then
a KA is sent every (interval) 60 secs - but after 2 goes (count) without
response the connection is broken.

Plus the usual FipSeq - partial, filter, valid etc

Input Parameters :
	-n : Name of service (SU field)				default: NXWIRE
All Optional :
	-d : done folder					default: none
		This holds a copy of all incoming data files from every source
		The structure is
			(done folder) / (date)_(logon) eg 20110921_fip / (filename as written to the
output folder)
		It can be purged with an entry in maintenance (zapfiplog)
		eg if '-d raw.data' and we want the last 30 days data
			/fip/bin/ipdelque -q/fip/spool/raw.data -i1 -a30
	-D : display connection and data (use only for debugging)	default: no
	-k : keep alive idle time				default: 65 secs
	-K : keep alive interval time				default: 60 secs
	-I : wire Id use to denote which nxwire if more than one. default: none
	-o : Output folder in /fip/spool			default: spool/2brouted
	-O : Name of output format (DF field)			default: NXWIRE
	-p : port number of the FireHose			default: none
		Normally NX v4 uses port 9001
	-r : the name of a DIFFERENT routing table to 'name'
		(SR field : used by iproute)			default: name
	-R : Running Round Robin on the output folder		default: no
	-s : hostname/internet address to select a test host	default: localhost
	-w : log timeout				default: 90
	-Z : do NOT archive any incoming files		default: archive
	-v : Print the version number and exit

Plus for CatchUps
	-C : Single shot catchup. The program stops on completion	default: normal
running
	-X : do NOT run a second CatchUp instance		default: if normal, do
		This is for normally running which will automatically run a CatchUp on
startup.

Version Control
;000t	23sep12 original version ;m funny catchup fudge - and redid catchup
	;n added -d doneque
	;o tuning ;p-r log max ;s-t 1may13 added tcp-keepalives

(copyright) 2014 and previous years FingerPost Ltd.