iproute (Sat Oct 25 2014 01:31:01)


IPROUTE may be used in three areas :
	- normal use as the routing stage for all wire files arriving on a system and
passing the files to 'ipwheel' to distribute.
Two lessers uses are :
	- applying extra Metadata according selections of FipHdr fields
		This is triggered by the '-METAONLY' input switch
	- moving incoming files to folders in /fip/spool depending on FipHdr fields
		This is triggered by the 'DEST-IS-FOLDER:' keyword in a single parameter file
OR the -Q input switch.

Normal routing is :

IPROUTE reads a file generated by one of the WIRE programs and attempts to
decide where to route it. This it does by looking at the FIP Header fields DU
(destination or end user), SU (logical source or user i.e. wire name) or SR for
the alternate routing filename, SC (source character set) and SH (source

A lookup table with the name of SU header field (in uppercase) (or SR if it is
optionally there) is inspected to find where the file should be sent. This
parameter file can be forced to be always the same if the -S input switch is
used to specify it.

If the file has a SD (duplicate wire) then the file is copied (well, linked)
with the new name.

Optionally if the file has a DU then no further processing needs to be done.
This is flagged by the -r or %REUSEDU flags.

This program does NOT send it - it merely creates a link in the output queue
2GO in the form :


On input you can change the default settings of DU, SU, SN and SC.

All parameters are optional - although there MUST be routing file for each
incoming feed even if it contains a single comment line.

Normally the minimum file will contain :
	- comments (naturally!)
	- a default destination (using the default-dest: or %DU keyword)
	- a source character set (using the default-chrset or %SC keyword)

Other normal keywords are :
	- a number of routing lines
	- redun-hosts or %SZ field if there is a redundant feed
	- %LOG to display the headline or other meaningful fields.

Note that the '%' in front of each keyword in the parameter file is no longer
required. So each of these keywords may be specified as either:
or	hdrkey:mykeywordfile

The syntax of the routing file is :
	; comment line (comments are also valid on the rest of line).
	DU: (default destination setting if not matched)
		eg: %DU:logdel
		which will send all unrouted copy to the archive and no further
		eg: %DU:apspo+spoprt+chrisq
		which will send unrouted copy to the three destinations
	SC: (set source character set)
	SZ: (primary,secondary host for redundant wires)
		SZ:fip1, gate2
		Note BOTH systems must be specified, neither need be the
		system that the file first came in on
	KEY: (keyword file containing keywords to search for)
		A file in tables/keyword/DAILY_WORDS exists with a list of
		keywords to search for on separate lines. If such a Keyword is
		found it is added to the FipHdr as the HK field. A match is made
		is a string is found in the stream of data. To look only for words,
		use the PIPE, '|' to specify the begining/end.
		ie	mac	will find 'macintosh', 'machine', 'applemac'
			|mac	will find the first two only
			mac|	will find 'applemac' only
			|mac|	will find 'mac' only
		A word is one that starts/ends with a space or ends with ',.;?!:)([]{}'.
		Apostrophes are ignored.
	KEY-TEXT-MAX: (number or FipSeq)
		Maximum number of chrs to check in the Keyword file
		Use this to trim the search to the first 400 chrs for example
		default: up to 64k
	HDRKEY: (Also look in these fields for the Keywords)
		Look in the K field of the SH (source header) plus the AB field
		for the same keywords as the text.
		The syntax is denote each field with a preceeding backslash
		plus a space to separate : "AB (space) QQ (space) FQ"
	CASE: (keywords and text searches are case SENSITIVE)
		Normally words specified in the Keyword file are case insensitive.
		So an entry like 'Humbug' will find :
			HUMBUG, humbug, hUmBuG
		Set this flag to force the program to only search for an EXACT match
			- 'Humbug' in this example.
	CONDCASE: (Condition lines are case SENSITIVE)
		The test in a condition line is normally caseINsensitive. So the following
		line will trigger on any case like 'TV', 'tv', 'tV' or 'Tv' :
			1	c=tv		>qpstv		; send tv pages to qps
		Almost all wire services send codes which are case-insensitive.
		AFP's EAA service however has category codes like 'FsI' or 'Sss'
		where 's' and 'S' have different meanings. Use '%CONDCASE' to
		flag this.
	LOG: (Custom logging : text, unix and octal chrs, FIP hdr fields)
		Each file though is logged as a one line entry in the Item Log
		(log/ALL) The default format is :
	   date time iproute !i Source(SU) Dest(DU) FileName(SN) Keyword(HK)
	   eg:	Sat Jan 22 15:15:55 iproute !i : AP	wsport APN0174 eggs
		which is represented by %LOG:SUtDUsSNsHK
	NOLOGCTL: Strip all control chrs - such as BELL - from the log file
		default is to display all chrs
	NOLOG:	Do NOT log each item through at all
	NOLOGDELETE:	Do NOT log any item which is sent to 'delete'
	WILD: (Change the wild Chr from '*' to this chr)
	SINGLEWILD: (Change the single wild Chr from '?' to this chr)
		wild and single wild chrs may be used in any condition below
		to signify any chr or any string of chrs
		eg: %WILD:%
			1	c=Z%L
		will prove true for any category code (XC field of the Source
		Header (SH)) that begins Z and ends L.`
	WORD-BOUNDARY: (Chr in FipSeq)
		This is a single punctuation chr which can be used to mean the START or END
of a Word or field
				1	WT=|ABC*
			will look for the contents of the WT Fiphdr field STARTING ABC
				1	WT=*XYZ|
			will look for the contents of the WT Fiphdr field ENDING XYZ
		The following chrs may NOT be used + > " # _ - =
		There is no default and so word/field boundaries are NOT tracked normally.
	TEXT: (no of chrs to look to routing information)
		This defaults to 8k and is 64k maximum.
	MAX-DOLLAR-T: (no of chrs for $t - first bit of text)
		This can be up to a maximum of 2000 chrs - default is 80.
	HASH-IN-HDR: (FipSeq)
or	HASH-IN-DOLLAR-1: (FipSeq)
		Hash character ('#') to be used in $t and $1 to $9. (Remember '#' is an
		end of FipHdr character, so it needs to be mapped out of the way)

		Number of lines at the top of the file to ignore BEFORE starting the hash
		Use this if a date/time is on the first line or so.
		default: none
		Start generating the hash-key (%x) AFTER this character/string.
		default:none - ie no start string meaning ALL data from the start.
		Stop generating the hash-key (%x) on this character/string.
		default:none - ie no end string meaning data continues to the end
		; stop hashing at the ^C/end-of-text as the datetime is ALWAYS different
		Use this string to generate the hash
		default use the data in the file
		eg	hash-key-data:XS-XN-XC-XP-XK

	TYPE-MARKERS: (2 punctuation chrs to be used as markers)
		For Tests, types-of-characters may also be used.
		There MUST be two, non-space characters used for the start
		and end of the types which must be punctuation and NOT the same
		you have used for wither wilds or word boundarys.
		eg	look for (space) (3 uppercase letters) (4 numbers) (spc)
			1	XK=*<SUUUNNNNS>*	+someDestination
		Note the start and end markers may be the same chr.
		You can also mix exact strings with types :
		eg	look for 'ABC' (4 numbers) (spc)
			1	XK=*ABC<NNNNS>*		>weHaveABCfolks
		Destination (DU) in sys/USERS to send 2nd (and other) duplicates
		Strip re-route duplicate files which do NOT have the same
		default is - no deduplicate.
		A common de-dup destination might be 'logdel' to log a copy only.
		The incoming files list is in log/route/DEDUP_(SU)
	DE-DUPLICATE-HEAD: (FipSeq field for the Headline)
		(this depends on DE-DUPLICATE-DEST)
		THis can be a combie or some other sequence of FipSeq to generate a unique
		eg 	; dedup on name - catcode - priority - take number - taketime
			fixed:Q1	SN-NC-NP-NQ-NT
		default is 'XK'
		To use the hash key '%x' in a De-Dup head, unfortunately you cannot add it
in a 'fixed:'
			fixed:Q1   XK-XC-Q0
	DE-DUPLICATE-DIFF: (number of bytes)
		This allows for slight differences in the text of the files. This fudge
factir is
		the allowable difference. So DE-DUPLICATE-DIFF:100 will catch files that have
the same
		Headline (and Source) and the file length is within 100 plus/minus.
		ie if a file is 987 bytes long and the original was 1086, it is de-duped.
		BUT if it is 1187, it will be considered a new file and allowed thru.
		If you do NOT want to dedup on size, set the value to -1 eg:
			default: 0 for zero bytes difference.
	DE-DUPLICATE-MAX: (number of Items)
		(this depends on DE-DUPLICATE-DEST)
		Total number of entries to be tracked in the time period
			default: 100
		As duplicate files normally follow each other, generally this total dows not
have to be very big.
	DE-DUPLICATE-TIME: (number of seconds)
		Time period for duplicates in seconds.		default: 3600 for 1 hour
		This is the name of the 'collection' of files to check for duplicates and/or
		It is normally the Source (SU) but can be FipSeq.
		This is need to define the uniqueness of the data.
		It is used to create the name of the log file in log/route.
	DE-DUPLICATE-BALANCE: (balance group)			default: none
		Name of a Group in the sys/BALANCE file to syncronise the DEDUP file across
		Note there will normally be up to a second delay in balancing, so duplicate
files arriving on both system within the baalnce time may be un-duped.
		The new items are APPENDED to the DEDUP file on the remotes. As cleanup (iw
trimming according to time) is only done on a new file on that system, remote
systems which never receive files (ie pure DR sites) will need an occasional
full copy - or nightly maintenance should be used to zap the files in log/route
on those systems.
		Ignore the Destination/DU for this traffic	default: do NOT ignore
		Use this where the two files may have different DUs because of round-robin

	THROTTLE-MAX: (number of Items in FipSeq)
		Only allow this number of files through for the time period.
		Use this where you not only want to deduplicate but also restrict data.
		Note that throttling also uses de-duplications. In particular, DD-DEST, HEAD,
MAX and/or TIME must be specified.
		The parameter can either be a number OR a FipSeq Fiphdr field :
		In this example the max is normally 10 files during the period unless the
FipHdr SU is 'AFX' which allows 99 files.
			option:Q1	SU,,,,AFX
			fixed:Q2	Q199$o
			combie:Q3	Q2,10
		default: no max to allow ALL files through
		Extra FipHdr information to add to any throttled copy	default: none
		This could be a banner or some extra Meta data for the next process.
	LOCALE: (locale for this source)
		Use a different 'locale' ( look at the man pages for locale)
		Most computers are set to US English and never changed.
		Use this parameter to customise any date/time or Chr translations
		The parameter MUST be a valid locale on your system!
		To find out what valid locale exist, type 'locale -a' on unix.
		Eg	; Set for for Brazil, portugese
	ADDHDR: Extra, optional, fixed FipHdr information to added
		AFTER any routing and before the file is passed on.
		Often this is add extra fixed fields or to change the unique
		name for redundancy. eg:
	ADDHDR-BEFORE-ROUTING: Extra, optional, fixed FipHdr information
		added BEFORE any routing is checked.
			; map the extra fields to LCC
			; eg of incoming : KI:LCC MRG LCCX EMRG
			; we want KI:LCC MRG EMRG
			unique:Q2	Q1
		If using any FipSeq - like in the example -, they MUST be
		specified ABOVE/BEFORE the addhdr-before-routing line in the
		parameter file. Any specified below will be ignored.
	ADDHDR-BEFORE-SCRIPT: Extra, optional FipHdr information generated by an
external program or script
	ADDHDR-SCRIPT: Extra, optional FipHdr information generated by an external
program or script
		ADDHDR-SCRIPT:/fip/local/find_iim.pl E4 > E3
		E3 hold the name of a TMP file to create that will be read for the list.
		E4 hold the name of a TMP file that holds the contents of the incoming file
			this is a copy of the incoming file but with a safe filename.
		** if you are going to use the FipHdr fields from the script for routing or
logging, pls use the '-BEFORE-' versions
		This is a 'must use' if using the Fip version of rdjpgcom.
		eg ADDHDR-BEFORE-SCRIPT:/fip/3rdparty/rdjpgcom -fiphdr E4 > E3
		NOTE - that the script uses Forward Slashes for the file names. So Windows
uses may need to convert in the script.
	REUSEDU: If the DU exists for this service, use it and ignore the
		routing table.  Normally if a DU exists, it is ignored and the
		tables searched.
	US-TICKERS: Look for tickers in text - (NASDAQ:MSFT) (NYSE:FPOST)
		and stick in the R* FipHdr fields.
			up to 9 codes can be saved.
			format is (Exchange) (colon) (Ticker)
		This may be of value for Business Wire and PRnewswire services.
		(Note for versions 158z - currently please do NOT specify BOTH BIZWIRE-HEADER
	TICKER-TAG:<.> start, separater and end chrs for US-Tickers.	default is (:)
for (NYSE:F)
	REPLACE-TICKER: Replace a US-Ticker in text (as bracketed by (:) or <.>) with
this FipSeq
		Several agencies - Reuters, DPA - send a Ticker code after the
		company name : BP Plc <BP.LSE> Use this to replace the ticker with other
		The ticker data is available as FipHdr Z1
		eg	REPLACE-TICKER:<a href="http://www.nasty.metabank.org/tickers=Z1>Z1</a>
		These are lists of FipHdr fields with tickers and names to search in text.
			fixed:T9	V3 (<a href="T0T4:T5">T5</a>) $o
	BIZWIRE-HEADER: Interpret the Business Wire Header at the top of text.
		( BW)(TEST-TEST-2)(ABC) This is a Test; Please Disregard This Test
		These are put in B* FipHdr fields.
			BU for Source	'BW'
			BK for Keywords	'TEST-TEST-2'
			B1 to B9 for Ticker codes	'ABC'
			BH for Headline	'This is a Test; Please Disregard This Test'
				The headline MAY be multiline, any end-of-lines are forced to a space.
			BD: Keywords			'EUROPE'
			BI: Industry Keywords	'MULTIMEDIA'
			BF: for Supplier Source	'Ardvark Associates'
			For RNS work,
				BS for ShortName	'Abbey National PLC'
				BC for FSA Category	'SUS'
				BZ for Sequence Number	00000007
				BR for Time of Receipt
	PRNEWSWIRE-HEADER: Interpret the PR Newswire Header at the top of text.
		These are put in B* FipHdr fields.
			BR for Ticker codes	'ABC'
			BH for Headline	'This is a Test; Please Disregard This Test'
			The headline MAY be multiline, any end-of-lines are forced to a space.
			BD: Subject Keywords	'ASI PDT'
			BI: Industry Keywords	'ERN TLS'

	AFPXML: Look for AFP tagged keywords - <org>DASSAULT AVIATION<orgid
idsr="ISIN" etc
		and stick in the R* FipHdr fields.
			R1 - First TickerCode	- ISIN*FR0000121725*DASSAULT AVIATION
			R2 - Second TickerCode	- SICOVAM*12172*DASSAULT AVIATION
			R9 - Ninth Ticker Code
				format is (ORG) * (Code) * (Company Name)
		There is a variant - AFPXML-link which also looks for the first occurrence
		of the name of the company in the text and adds
	BLOOMKWD: Look for Bloomberg-style keywords - NI EUROPE NI M&A NI FIP etc
		and stick in the HX FipHdr field.
		Look for the first ANPA-style Bold line and put in this FipHdr field.
		The Anpa bold is started with a '^'. In theory the headline is
		only one line, but many suppliers make it over two or more. So
		this flag will take data until a '^ @ < > ='
		Look for Reuters-style Dateline, Advisory and Bylines and put in
		the 5 FipHdr fields noted ie L1 implies :
			KM	DateLine
			MH	ByLine
			MN	Advisory
			B8	Correction Advisory
			B7	Embargo Advisory
	REUTERS-SIGNOFF-SCRIPT:/fip/local/r_naomi_contacts.pl lang=QL fh=bu,bx
output=E3 data='E5'
		Script for pulling apart the Reuters Signoff
		Look in the KK FipHdr field for UPDATE or WRAPUP and create a Revision FipHdr
			M9	This Revision|Last|Next|Y/N
		Put ShiftOut and ShiftIn around the Dateline for BUI
		Cleanup any extraneous Bui chrs in the text
		Number of lines to look for a Reuters Dateline - default 15
		Minimum is 1, max is 25
	ALLOWSPACES: Leave all spaces in the data field for comparison.
		Normally a field will have leading, trailing and multiple spaces stripped.
		eg WITHOUT this option a field containing (pipes show start and end of data)
			|	This is a	line with tabs   and spaces in	|
		will be tested as :
			|This is a line with tabs and spaces in|
	SEND-SINGLE-COPY: If the same destination is specified more than once,
		this option will only send a single copy of the file rather
		than as many times as it is specified.
	SEQNO-BALANCE-GROUP: (balance group)
	SEQNO-BALANCE-QUEUE: (balance queue)
	SEQNO-NAME: (Name for a Seqno)
		This will save and balance a seqno to this balance group
		If no QUEUE is specified, the default is spool/2balance
		If no NAME is specified, the default is the name of the SU (or SR) FipHdr
	CHECK-SEQNO: (fiphdr field)
		Check this FipHdr field contains a value which increments
		This is used to log any missing seqnos
	CHECK-SEQNO-MIN: (minimum value)
	CHECK-SEQNO-MAX: (maximum value)
		Set bounds for the sequence number checking - defaults are 1 and 9999
	SEQNO-RESETS-AT: (Hour:minute)
		Sequence number is reset at this time - default is 0000 for midnight
	FIPHDR-FOR-EXPAND: (FipHdr field)
		The fiphdr field which needs to be expanded - no default
	EXPAND-CODE: (code)	(codes to add)
		Add extra codes to a FipHdr field if the 'code' exists.
		The 'code's are space separated.
		There can be one or many (up to 300) 'expand-code' lines. eg:
			expand-code:A	 C D E M O T U MTL GRO SOF OIL
			expand-code:C	 C MTL GRO SOF
			expand-code:D	 D T
		Use this to create a fake output sequence number in the FipHdr
		field ZQ. If there is a number to this parameter it will be used
		as the amount to increment - default is 2.
		eg default create ZQ:3-4, the next ZQ:5-6
		if create-ZQ:12, file will have ZQ:22-33 then ZQ:34-45 etc.
		if create-ZQ:1,  file will have ZQ:2-2 then ZQ:3-3 etc.

	DEST-IS-FOLDER: (yes/no)
		Destination names are actually folders under /fip/spool
		default is DU are destinations in the sys/USERS file.
		This overrides the -Q switch
		Note all folder names are forced lowercase.
		These folders are now speedy-fip if that option is running.
		The DU is NOT appended in this case - so if you are putting into spool/2go,
add fiphdr fields
			1	SU=SPTZ	 #DU:dev2sc+sptz2saves#SC:no
			1	SU=SPTZ	 >2go

	DONE-QUEUE: (Folder name in FipSeq)
		Put the input file into this folder after processing
		This overrides the -f Input switch.
	BEFORE: (FipSeq)
	AFTER:	(FipSeq)
		Text to conditionally add at the top or bottom of text
	SCRIPT: Optional Script to run AFTER the file has been sent successfully.
		eg	script:/fip/local/FTP_NEXT_STAGE YI-YS
	TRACKER-SCRIPT: Optional Tracking script - run AFTER script (if any)
		default: none

	MAX-ROUND-ROBIN: (no or 1-49)			   default: no
		add a round-robin number to the output queue (same as -R)
	ROUND-ROBIN-FIPHDR: (2 letter FipHdr code)		default: none
		both these parameters need to be specified
	BIG-FILE-LIMIT: (size in bytes)				default: none
	BIG-FILE-FIPHDR: (2 letter FipHdr code)			default: none
		Flag YES or NO if the file is bigger than the limit.
		both these parameters need to be specified
		This is added as a FipHdr before routing and can be used to Route on.

	FILE-TRACE: yes/no, + hdr + in + out
		Trace all files using this parameter file	default: no
	FILE-TRACE-NAME: (FipSeq) unique name of this file	default: SU-SN-HR
	FILE-TRACE-LOG: (FipSeq) additional logging		default: none
		These can be individually turned on by FipHdr fields in the incoming data :
		(notice the '_' if using in the FipHdr but '-' if parameters)

The rest of the file is a series of conditions or rules are applied to each
incoming file.
The format of these is :
	tab (level_no) tab (condition)
	tab (level_no) tab (condition) tab > (destination)
	tab (level_no) tab (condition) tab (+opt additional DU)
	tab (level_no) tab (condition) tab (>) DU1+DU2+DU3..+DUn)
	tab (level_no) tab (condition) tab (+DU1+DU2+DU3..+DUn)

	- Each rule is checked starting from the top of the file.
	- where tab can be any combination of tabs and spaces.
	- where the level_no is a number. If it is greater then 1, then this line is
		only checked IF the preceeding level was matched.
	- each rule is one and only one line.
	- destinations - which MUST be entries in the tables/sys/USERS file - are
specified as :
		or	>dest1+dest2
		or	+dest3
		- Multiple destinations are separated by a '+'.
		- If a line has NO destination, processing continues with the next line.
		- If a line has one or more destinations starting with a '>', Searching
			STOPS and the file sent to those destinations (without the '>' or '+').
			NO more tests are processed.
		- If a line has one or more destinations starting with a '+', Searching
			CONTINUES while those destinations are saved for sending later (without the

	- Syntax of a condition is :

			(FipHdr Field) = (data)
		eg:		1	p=3	>prio3

		If the FipHdr field is a single letter it is from the Source Header (SH).
			eg for standard wires
				s is service designator
				n is item number
				p is priority
				c is category
				w is number of words
				k is keyword or headline
		which has a format in the data file like :
			SH::SEAA:N0436:P3:CI:W0260:RUSA /AFP-EC08:KUS-Asia-radio

		If the FipHdr field is two letters, it is that actual field such as EK in
this example.
		eg:		1	EK=88	>2FatLadies

		Routing may also be on data in the text using the '$$'as the base. eg:
			2	$$="New York Times"	>nyt
		For data, the condition is not EQUALS but CONTAINS.
		ie The following will be true :
				AbcdefThe New York Timeszzzzzzz
			and	New York Timesn

		Actual type-of-file or the 'magic number' can be tested with '$f'
		eg	; Is it a Word 2000 file ?
			1	$f=xCFxD0xE0x11	>translateWord2000

		If a Keyword file is being searched, Routing can be done on a match.
		In this case '%%' is used and the name of the Keyword file.
		(The %KEY parameter is ignored in this case).
		In this example a file is held in 'tables/keyword/FOREIGN_LIST'.
			1	c=INT				; for Category = International ..
			2	%%=FOREIGN_LIST	>mainforeign	; Main For queue for matches
			2	c=*		>junkforeign	; Rest to junk

	- Condition is either '=' to signify equals or '#' meaning NOT equal. eg:
		1	c=INT	>allints
		1	c#INT	>noints

	- in the case where several tests need to be made to determine a destination
		then the preceeding lines have no destination
			; If category starts 's' ..
				1	c=s*
			; send a COPY of all high priority files to the editor, and keep searching
					2	p=1		+hied
			; if Keyword is SOCCER - send to the soccer destination
					2	k=SOCCER	>soccer		; send race cards
			... etc
		There can be up to 100 levels if you really need.

	- Use wild strings and chrs using (by default) '*' for strings and '?' for a
single chr.
		(These can be changed using %WILD or %SINGLEWILD as above)
	eg	7	k=*SOCCER*
		will be true for XK fields such as :
			These are the Soccer Headlines
			And so for soccer
	- Case - upper or lowercase - are NORMALLY ignored so that SOCCER' and
'soccer' are equal
		unless specifically noted
			either	with the '%CONDCASE' flag for FipHdr fields
			or	with the '%CASE' flag for Text and Keyword matches.

	- Note that if any spaces are embedded, double quotes should surround the
		3	k="Liverpool Beatles"

	- Testing for something or nothing !
		The following test will always be true, no matter whether there
		is a FipHdr L1, a blank L1 or an L1 with data.
			1	L1=*	>IamTrue
		So to test if there IS something there :
			1	L1#""	>thereIsSomut
		Or nothing (ie no L1 or L1 with no data) :
			1	L1=""	>thereisNothing

	- Special Case - you can also add extra FipHdr information on the end of a
		(or destinations)
		1	p=1	>w4rns+rnsEd#JH:RNS-PRIORITY
	or and example where you want to continue
		1	p=1
			2	c=HHH	+w4rns+rnsEd
		Note that the '#' sign marks the end of any destinations and the begining of
		The FipHdr bit may be in FipSeq :
			1	p=1	>w4rns+rnsEd#JH:RNS-TIME-$d

For example :

; routing table for AP
; default destination is junk
; chrset is 'anpa'
; If category starts 's' ..
	1	c=s*
; level 2 - so for all category 's' etc, if the priority is '1' ..
; send to sptpri1 and do not look any further down the table.
		2	p=1	>spthipri	; Priority 1 to sports hi pri
		2	SU=AP	>rac		; send race cards
	1	c=t		>tv		; tv pages

Note that if you use one of the $1 to $9 FipSeq functions to get the 1st to
9th line of text, any HASH is replaced by x9D.

Note also that a couple of extra builtin FipHdrs exist
	%x	- generate a 32bit hash of the first 64k of the data

Input parameters are (all optional) :
	-1 : (minus one) filename for single-shot	default: folder is spooled
	-b : Balance ALL data files - balance queue	default: (if -B specified)
	-B : Balance ALL data files - balance group	default: not balanced
	-c : default SC	- source chrset			default: ascii
	-d : default DU - destination			default: woops
	-D : display contents of header fields as they are checked	def.no
		This is ONLY valid when using '-1' single shot and NO other
		output is generated - ie no files, just the check.
		eg iproute -1/fip/spool/test/abc.wire.9001.1234 -D
	-f : done queue for the input file		default: file is deleted
		This can be overridden by the DONE-QUEUE parameter
	-h : hostname to use				def: name of this host
		This is used for the XP flag for redundant feeds and should
		only be necessary on systems connected to more than one
		ethernet where the base hostname is NOT that to be used.
	-G : WINNT/2k - grab the first file only	default: no
	-H : display contents of header fields in HTML	def.no
			This also requires the -D input switch
	-i : input queue in /fip/spool		default: 2brouted
	-o : output queue in /fip/spool		default: 2go
	-l : (minus el) do NOT log			default: log
	-METAONLY : special case of iproute - pls see below.	default: no
	-m : name of the top MetaData file 		default: METADATA or (WinNT -
	-n : default SN - filename			default: UNK000
	-Q : all destinations are really folders NOT DU	default: DU
		This can be selectively turned off for a single parameter file
		with DEST-IS-FOLDER:no
	-r : Do NOT search for a routing if a DU exits.	def: always use table
		With this switch, if a DU field exists, it is used and the
		routing table ignored. Normally the contents of the DU are
		ignored and are overwritten with whatever destination
		is selected via the table. This switch is for all incoming
		services. If only one or two services need this, use the
		%REUSEDU flag in the routing table for that service (see above).
	-R : Running Round Robin on the output folder	   default: no
	-s : default SU - source (usually the agency)	default: noname
	-S : force the parameter file to ALWAYS be this. default: FipHdr SR or SU is
	-t : interval between scans of the queue	default: 3 secs
	-X : ignore SZ redundancy			default: use SZ
		This stops a copy of an incoming file being sent automatically
		to the redundant system as in the SZ line of the routing file.
	-W : 10 ths of a second - MUST wait if nothing processed	default: 0
	-8 : input is NOT speedy (valid on a speedy system only)
		ie the input folder is NOT piped but the rest of the system is.
	-9 : run in Speedy mode			 default: no
	-v : print version number and exit

-- The above defaults are ONLY used if any defaults are NOT set in the
parameter file (%DU, %SC etc).

-- Note that if 'iproute' is used for files which are not directly from the
wires - perhaps files which have to pass through twice, A selection of existing
FipHdr fields are zapped before processing if present.
 These are :
	- XZ (resent file flag - do NOT archive)
	- XS (do not send to other systems flag)
	- CX and DC (for xchg)
	- CQ change queue for xchg
	- SZ (to denote redundant systems)
	- XP and CP : source and destination FIP Hosts

This has no effect on FipHdr fields being ADDED at this stage (such as SZ being
added by the %SZ line) - only to delete any old FipHdr fields inserted by
another program.

The XZ flag stops 'ipwheel' from archiving the data.
The XS flag stops 'ipwheel' from sending to other systems.
CX and DC are usually added by 'ipwheel' to denote which XCHG to use for
SZ is used to send a copy of the file over to the redundant system.
XP and CP are the source and destination FIP Hosts (ie system file arrived on
and where it was sent to)

-- If you do not want to log every time, then either use the parameter to STOP
ALL logging :
or, just a single, incoming file, add the FipHdr field ROUTE_LOG_EVERY:(999)
or, in the parameter file, for a filter :
	1	NN=grabeachsecond	#ROUTE_LOG_EVERY:600
	1	NN=grabeachsecond	>delete
The value for ROUTE_LOG_EVERY is the number of seconds to skip before loggin
the next occurance OR a negetive numbre for No logging at all. In the example
above, the first file AFTER 600 secs/5mins is logged.

-- Why would you send a file through 'iproute' twice ??

Perhaps you have several publications. The main wire feeds come into one (pair
of) systems and most of the data is sorted and processed on that system for the
main publication.
But a smaller publication - or a specialist publication such as a Sports or
Financial magazine wants a subset of data with totoally different sorting
It is easy to split a copy of all the wire files using a line like :
route file :
	1	c=*	+otherpub	; copy all to destination otherpub
USERS file
	otherpub	DP:smlpubhost
and send to the 2nd pubications servers when 'iproute' would resort properly.
USERS file on 'smlpubhost' :
	otherpub	DP:localhost	DQ:2brouted	SC:no DC:no

-- File tracing thru the Fip

There are a series of parameters common to serveral Fip Programs which will log
what is happening to all files, a service or a single file.

	file-trace: (valid actions - yes/no, + hdr + in + out)
		Actions are 'Yes' - log, 'No' - dont; if NOT NO, then in addition to logging,
we can save a copy of the FipHdr, Input file and/or Output
		(delete is ignored in iproute))

	file-trace-name: (UniqueId in FipSeq if not default -  SU-SN-HR)

	file-trace-log: (extra File Tracing logging in FipSeq)
		The Hdr and files are left in /fip/log/file_trace (which should be purged in
the nightly maintenance zapfiplog)

	These can be overridden by FipHdr fields in each file if a particular file -
or service needs tracing.
		(notice the '_' if using in the FipHdr but '-' if parameters)
	eg	ZFILE_TRACE: hdr
		ZFILE_TRACE_NAME: louislouis

---------------- META ONLY ------------------------
Normally 'iproute' is used as the routing stage for all wire files arriving on
a system.

There is a special case where a second copy of the program may be used to add
MetaData (in the form of FipHdr fields) only to any incoming files. This is
flagged by the input switch -METAONLY. In addition the input switches '-i' and
'-o' for input and output folders MUST be specified. This is to make sure that
the 2nd copy does NOT interfere with the main copy flow.

You MUST also define an input and output folders (-i and -o).
 eg for the SYSTEM file :
	meta	iproute	-METAONLY -m testmetadata.fip -i 2meta -o 2edsys

The parameter to the -m switch is a file in tables/metadata whose syntax is
different from the normal route file :
	; comment
	check-file:(name of file)

Example of a MetaData file
; Sports - look in /fip/tables/metadata/PSP.FIP

There can be several check-file(s) which all reside in tables/metadata.

  The syntax of the check-file is the normal 'ROUTE' as described above except

  1. 3 new keywords
			These give the strings of FipHdr to add to the file if the tests are true or
		DEFAULT-RESULT: (valid/invalid)
			The default result if all conditions are met is either 'valid' or 'notvalid'
or 'invalid' (the last 2 are the same !)

  2. The only valid destinations are
			Stop looking in this MetaData file as the result is valid
			Continue with any further Metadata files specified in the top file
			Stop looking in this MetaData file as the result is invalid
			Continue with any further Metadata files specified in the top file
			Stop looking in this MetaData file as the result is valid
			Do NOT continue looking any further
			Stop looking in this MetaData file as the result is valid
			Do NOT continue looking any further
			Stop checking at this level in this MetaData file
			Continue checking at the next (level - 1)
				1	SU=PA
					2	XC=HHH
						; if Prioriy is 1 or 2, drop down to the next level 2
						3	XP=1	>continue
						3	XP=2	>continue
						3	XP=*	>stop-invalid
					2	XC=PPP

						etc etc etc
  Example of a check-file:
; ------------------------------------------------------
; if still valid at the end, add VT:PSC

; we have wilds

; we assume that it is valid...


; ------------------------------------------------------
; If Status is NOT New or Priority 1 - always ignore
	1	NV#New	>stop-all-invalid
	1	IP=1	>stop-all-invalid

; ------------------------------------------------------
; want Product Codes G or SPO or E (continue to next level 1 or end)
	1	NO=*
		2	NO=|G|		>continue
		2	NO=|SPO|	>continue
		2	NO=|E|		>continue
		2	NO=*		>stop-invalid

; ------------------------------------------------------
; *** End ***

Version Control
;162f52	23oct07 added extra-fiphdr-script
	;b 14jan08 allowed DE_DUPLICATE-DIFF:-1 for NO checking at all
	;c 26apr08 keyword stripped ';' as comment - needed for '&aring;'
	;d 31jul08 -b, -B added for dataBalGroup, balQue and SX
	;e 27aug08 bugette with Reuters which are ONLY Xref/Signoff
	;f1-4 18feb09 allow speedy with DEST-IS-FOLDER ;2 allow de-duplicate-time of
0=dont test time;3 NOLOGDELETE:
		;4-5 3jun09 allow SD to tbe multiples SD:AP1,AP2,AP3
		;6 22oct09 CHG - redid addhdr-script so it is the same as ipspool (with
		;7 16sep10 added DE-DUPLICATE-BALANCE
		;8-9 24sep10 added ADDHDR-BEFORE-SCRIPT:..
		;10 10oct10 ignore dedup if DU=logdel or delete !
		;11 23oct10 bugette with DEDUP and no file
		;12-14 3mar11 wrinkle with speedy and do NOT allow ';' to mean comments on
non-route lines and added %x for CRC
		;15 10may11 bugette in chk_route (WIN only)
		;16-18 11jul11 added -8 for NOT speedy in but speedy out and $1 back; added
HASH-IN-HDR as -$1 or-$t
		;19-20 7dec11 added time for done queue duplicates
		;21-26 30jan12 allow xml in Reuters-dateline and added HASH-KEY-STOP
		;27  4jul12 added KEY-TEXT-MAX
		;28-32 19jul12 do NOT add DU for DEST-IS-FOLDER and added ROUTE_LOG_EVERY as
FipHdr field
		;33-35 20nov12 bugette of addhdr-before-route with no routing lines
		;36 8mar13 added big-file-limit and max-round-robin and allow parsed routing
line : 1 c=* >2xmlRR
		;37-38 7may13 wrinkle with ignoring the FipHdr in '1 c=* >abc+def#SC:ghi'
		;39 6jun13 allow SH to change in ADDHDR, ADDHDR-BEFORE-ROUTING
		;40 12sep13 added DE-DUPLICATION-IGNORE-DU
		;41-43  1oct13 bugette in DE-DUP balance
		;44 3jan14 added SHOW-DEDUP:yes/no
		;45-48 16jan14 added SCRIPT and TRACKER-SCRIPT and refresh SN if it has
changed with ADDhdr etc
;161z2	03dec04 bugette with more than one kwd search (1 %%=KWDFILE >abc)
	;a 15dec04 added -f done queue and DONE-QUEUE:
	;b-h 17dec04 bugette with >a#SC:a and %SC:def (f-Roy xref)
	;i-j 28apr05 added REPLACE-TICKER
	;k-l 16may05 mod for mtime/nfs waits
	;m-p 05jul05 use AFP <org> markup in text and buglette in size of 1st dedup
	;q-r 03sep05 added euro-tickers and euro-names
		added BEFORE and AFTER
	;w 1aug06 rename copied file if already exists
	;x-y2 17nov06 bugette in test if singlewild '?' is first chr
	;z2 08may07 added CREATE-ZQ
;160z	14nov03 tuning dedups (size is now JUST the data part)
	;a 12dec03 added reuters-lines:
	;b 12jan04 added anpa-headline:
	;c-g 16feb04 cleanup HTML display and added expand-code
	;h 01apr04 added RTR code for Advisory->Emb or Corr
	;i 25apr04 added stop-all-valid/invalid for Metadata
	;j-o 06jun04 bugette in AFPXML plus added REUTERS-BUI-DATELINE
	;p-q 27jul04 no BUI in Dateline pls and single spaces in Dateline
	;r-s 01sep04 -9 speedy (s for R-Adv)
	;t-v 22sep04 added REUTERS-CHK4BUIS and tweak to Advisories
	;w 04oct04 extra FipHdr lines are now added to FipHdr immediately
		.. plus Pilcrow again
	;x-y 03nov04 RTRS - case of Xref after a Signoff
	;z  17nov04 added dest-is-folder: and default-dest and -w filewait
;159z	06jan03 minor mod to Reuters/Byline
	;a 13jan03 RTRS - added UPDATE/WRAPUP revision
	;b-g 30jan03 RTRS - added Genre
	;h 04mar03 RTRS - Signoff
	;i 25may03 RTRS - get/zap HASHes in MetaFields
	;j 29may03 RTRS - add xB6-Pilcrow for extra lines in Advisories
	;k-l 04jun03 RTRS - strip trailing spaces/lines from Advisory
			and bugette for signoff
	;m 29jun03 VON as a By plus poss correction advisory
	;n 04jul03 added REUTERS-PILCROW
	;o 23jul03 added PRNEWSWIRE-HEADER
	;p 24jul03 redid $f for filetype - which was NOT working well
	;q-v 04aug03 added de-duplicates
	;w-y 06oct03 stripp and <Fip...> markers for Datelines
	;z 31oct03 added timings
;158zi	28dec01	added TYPE-MARKERS and LOCALE and US-TICKERS
		;a 22jan02 restrict size of market for US ticker to 20
		;b 05feb02 added -1 for single shot
		;c 12feb02 bugette with wildstrings
			ie k=*sport* sometimes true erroneously
		;d/e 27feb02 added $t and $1-9 for ADDHDR
		;f/g 03apr02 added BIZWIRE-HEADER
		;h 14apr02 ADDHDR can now contain FipSeq for other FipHdrs
		;i 18apr02 widgets put in - fixed, partial, combie etc
		;j/k/l 21apr02 added check-seqno, seqno-resets-at, check-seqno-max,
		;m 22may02 allow CC!="abc" as well as CC#"abc" and defaultSU put back in
		;n 28may02 bugette in word boundary code
		;o  5jun02 bugette in ADDHDR - not allowing $1 etc
		;p 13jun02 allow more than 2 source redundancy hosts (SZ).
		;r 18jun02 added -D and -H for display only
		;s/t/u 24jun02 added addhdr-before-routing
				and -G to grab-first-file
		;v 16sep02 added $f for checking the file magic number
		;w 17oct02 use calculated DU even if there is an incoming DU
		;x/y 26nov02 added %REUTERS-DATELINE
		;zi 11dec02 added more BIZWIRE kwds at end
;157b	12nov01 added -METAONLY/-m and word-boundary
;156p	16jul99	added %SEND-SINGLE-COPY:
		;b zap FipHdr DP if in the incoming file.
		;c 02jul00 added %AFPXML
		;d 07aug00 bugette with multiple DUs which blow the filename.
		;e 06nov00 added ExtraFipHdr info on the end of a dest.
		;f 22nov00 bugette - when using duplicates (SD), sometimes overwrote original
file !
		;g 02apr01 bugette - SendSingleCopy sometimes duplicated the 1st DU
		;h/j 21may01 redid ExtrafipHdr
		;k/l/m 25aug01 strip NULs from data part of file for comparison
		;o 05sep01 redid ability to run external program
		;p 28sep01 AFP changed their XML .... for %AFPXML

(copyright) 2014 and previous years FingerPost Ltd.