ipwheel (Sat Oct 25 2014 01:31:01)

ipwheel

IPWHEEL finds where our illustrious file is to go. It scans queue 2go for
incoming files.

For each new file, it reads in the FIP header (if there is one) and checks for
the destination (DU)- which may be multiple DUs. For each DU, the USERS file is
scanned for other information - DP, DQ, DC etc. If there is no valid entry for
the DU in USERS, the file is woops'ed to the Intercept queue.
Syntax of the USERS file is :

	; comment

	(destination)= (tab) (fip-hdr1):(data1) (tab) (hdr2):(data2) ... (EOL)
where	destination is the DU field
	fip-hdr is the header field or flag
	data is the optional data for that field/flag
For example
	; USERS file
	racing=		DP:edsys3	DQ:racing-spo	DC:j11
	sunrac=		DP:rs6000	DQ:save_racing	DC:ibm
	indrac=		DP:adobesvr	DQ:Racing.Desk	DC:indesign

If the file is a binary file (FBIN) or the source and destination chrsets are
the same (hdr field SC = DC), then the file is linked directly to the output
queuefor that destination. Otherwise it is sent via IPXCHG with the CQ field
filled with the queue of the next stage.

IPWHEEL also checks for redundancy. If it is running on the primary Unixbox for
that service, it processes the first file and junks the rest. If it running on
the secondary, the file is not processed unless the primary system is down.

The fun bit (?) is where you want to override Source Redundancy with
Destination Redundancy! Confused ? So am I and I wrote it. If a file has an SZ
field (usually added by IPROUTE from the incoming Wire routing file), normally
the 1st system specified is the Primary and the Second is the secondary.
BUT you may want to serve one network from one UnixBox and another group of
destinations from another UnixBox ad infinitum. To do so, use the DP_REDUN file
in tables/sys to override the primary/secondary pair for that destination.
Works a treat! Syntax is :
	; for comment
	EdsysName (tab/spc) PrimaryUnixHostname (spcs) SecHost (eoln)
	where Edsysname is the SAME as the DP in USERS
		and should be specified WITHOUT any system number
Syntax of the tables/sys/DEST_REDUN file is (remember this file is OPTIONAL) :
	; for comment
	EdsysName (tab/spc) PrimaryUnixHostname (spcs) SecHost (eoln)
	where Edsysname is the SAME as the DP in USERS
		and should be specified WITHOUT any system number
 For example
	; DEST_REDUN file
	atexedit	fipA	fipB
	ccisender	fipC	fipD
	opi1		fip99.bigpaper.co.zz	fip66.bigpaper.co.zz

Localhost is not a valid hostname in DEST_REDUN. Please take care to enter ALL
your editorial system hosts in DEST_REDUN if it is being used.

An optional parameter file may be specified using the '-E' input switch. Valid
parameters are
	- the usual parse parameters - partial, option, repeat, sum etc
	- folder-size: (name of folder under spool)	max: (maxsize)
		If a file whose size if bigger than 'maxsize' arrives for the output folder
(fiphdr field DQ) specified, then it is dropped into (foldername)_big and NOT
(foldername)
		There can be 50 such folders specified.
	- file-trace: (valid actions - yes/no, + hdr + in + out + (delete is ignored
in ipwheel))
	  file-trace-name: (UniqueId in FipSeq if not default -  SU-SN-HR)
	  file-trace-log: (extra File Tracing logging in FipSeq)
		Trace a file thru fip.
		Actions are 'Yes' - log, 'No' - dont; if NOT NO, then in addition to logging,
we can save a copy of the FipHdr, Input file and/or Output
		The Hdr and files are left in /fip/log/file_trace (which should be purged in
the nightly maintenance zapfiplog)
	These can be overridden by FipHdr fields in each file if a particular file -
or service needs tracing.
		(notice the '_' if using in the FipHdr but '-' if parameters)
	eg	ZFILE_TRACE: hdr
		ZFILE_TRACE_NAME: louislouis
		ZFILE_TRACE_LOG: seqno.ZQ

If the ZI hdr flag is set, a copy of the file is saved in the archive log
unless either it is a redundant file or the XZ hdr flag is on signifying a
resend. The name of the archive file is in two parts :
	1 - A day-of-the-year number which is found either from the HS Fiphdr field or
defaults to the current DOY.
	2 - the contents of the SU FipHdr field (ie the Source)
		this will be overridden if there is an SL FipHdr.

If the destination is flagged as 'delete' the file is zapped and no processing
- not even archive - is done.  If the destination is flagged as 'logdel' the
file is zapped and no processing except archive is done.

Input options (all optional) :
	-q or -i : queue to scan			default: spool/2go
	-x : queue for xchg				default: spool/xchg
	-n : queue for 2net				default: spool/2net
	-t : scan time					default: 1 sec
	-m : multi redundant files. For redundant incoming services, the normal
		arbitration is between two files. This switch is used where
		more than 2 are expected.
			default: allow the first file of every two.
	-h : hostname for multiple ethernets		default: (hostname)
		Name of this Unixbox if different from the reboot name.
	-B : do NOT read the FipHdr-in-text for FBIN files	default: do
	-c : make copy (link) in spool/2taste if valid file. default: no
		but ignore all traffic sent to 'logdel' or 'delete'
		Only Archived traffic that is NOT a resend is sent.
		the parameters can be one of the following
			S-saved data only (no logdel or delete)
			Z-deleted copy too
			X-all data (except logdel or delete)
	-C : filter traffic to spool/2taste default: no
		Only traffic with this filter (or nothing) in FipHdr ZWHEEL_FILTER is passed
	-d : check for changes to DEST_REDUN		default: no
		the default is to read once on startup
	-D : pass everything - including 'logdel' and 'delete'. def:no
	-e : Stub of the name of the editorial system	default: atex
		Wheel checks if the DP starts with this.
		eg -e mac	for editorial systems called macone, mactwo etc
		Files are slotted into 2macone, 2mactwo etc
		This will override the env variable FIP_def_EDSYS.
	-E : name of an optional parameter file in sys			default: none
	-F : FipHdr field for RoundRobin to be added to output file	default: none
		(see -X input switch below)
	-G : WINNT/2k - grab the first file only	default: no
	-H : log all the hosts/folders for each file	default: no
		Use this to trace where traffic is going !
	-p : BreakIn Priority For HiPriority News	default: 1
		Agency Bdcast traffic.
	-P : FipHdr Field for High-Priority News 	default: PR
		Agency Bdcast traffic
	-Q : only send one instance of the file to any one	default: send all
		local queue In this case only, the full 'DU' field
		is copied to 'ZU'. This works for files for this
		Host ie DP:localhost or DP is this host name
	-r : For DEST_REDUN - do not resolve any hostnames in DP: SZ: or DEST_REDUN
							 default resolve all names
	-R : For DEST_REDUN - do NOT use DNS to resolve names before checking
		DEST_REDUN. For this all the DEST_REDUN entries MUST be the
		true hostnames.				 default resolve all names
	-S : (size) if a file is bigger than this size and needs to be
		'xchg'ed, it is sent to spool/xchgbig NOT spool/xchg	default: spool/xchg
	-T : display timing stats for this stage
	-u : track DUs for uniqueness - only pass one copy of	default: do NOT track
		a file for each destination. Use this where a file MAY be sent
		to the same destination (DU) but only one copy is required.
	-U : add UNO field as the HR FipHdr field to ALL files.
		For Unisys : -U 0 = 0 -> 255; -U 1 = 0 -> 127; -U 2 = 128 -> 255
		For AFX Unos use -U 3	There is NO default.
		-U 4 is time-seqno
		-U 5 is time-ipaddress-seqno
		-U 6 is time-ipaddress-seqno with 6 digit seqno
		-U 7 is for timing stats
			(add -T to display timing stats for this stage)
		The Uno is ONLY added if the HR field does NOT exist.
	-W : 10 ths of a second - MUST wait if nothing processed	default: 0
	-X ; send to multiple xchg queues in round-robin order		default: no
		-X 3 will send to xchg1, xchg2 and xchg3
		maximum value is 29 (9 up to version 263o7)
		-X4 -F RR will send to 4 folders and add RR:n to the FipHdr
	-y : 2nd or supplementary USERS file		default: none
		this is used in addition to the main USERS file
	-z : name of USERS file in tables/sys		default: USERS
	-Z : never archive anything - ignore ZI,ZO,ZX	default: archive
	-9 : do NOT run in Speedy mode			 default: no
	-v : version and exit

How does redundancy work here ... good question.- see the web pages.

Note that the editorial system name is significant. Please do not mix Fip
Logical hostnames - like 'edone', 'edtwo' with proper TCP hosts ie do NOT call
a space Sparc 'edthree'.


(copyright) 2014 and previous years FingerPost Ltd.