ippost (Sat Oct 25 2014 01:31:01)


IPPOST is used to send a single file to a group of destinations or a
combination of groups and clients. Its main use is for syndication, library,
telex, fax and mailbox. It compares FIP header fields to the GROUPS parameter
file in tables/sys.

IPPOST scans the 'post' queue and for each file it checks the source,
destination (DU) and sub-destination (DA hdr field) and decides which
destinations it is to go.

Slightly different rules are used for differing types of traffic.
- All syndication and library clients MUST be specified in the GROUPS file.
- All mailbox clients MUST be specified in the mail/LOGON file.
- All ftp, internet-mail, dialup, telex and fax destinations are allowed to
pass through.
	Fax clients get sent to POSTFAX, telex to POSTTLX, ftp to POSTFTP and dialups
to POSTDIAL - all of which must be in USERS

Note that client names are case INsensitive so it matters not at all whether
the name is Upper or lower or both.

The groups file has the following syntax :
	; comment
	group: (name)	(list of clients in this group)
		eg :	group:syn	profile mead	dtimes ql
	client: (name)	type: (type of client)	number: (telex of fax number)
or	client: (name)	type:other	dest: (destination)
		eg:	client:gorby	type:telex	number:064-77878
		eg:	client:gorby	type:other	dest:russia

WHERE Client types are :
	mail	- mailbox, files loaded into spool/mail/logon
		where logon is in tables/mail/LOGON.
	syn	- syndication, file moved to spool/syn/clientname
	lib	- library download, file moved to spool/lib/in
		for further processing by iplibfilter or iplib
	pc	- pc, file is translated and moved to
		spool/pc/clientname with a pc style filename
	bdcast	- news agency broadcast, file moved to spool/clientname
	dialup	- send file using dialup	(probably 'ipdial')
	ftp	- send file using ftp		(probably 'ipftp')
	fax	- send file using fax
	telex	- send file using telex
	alertmsg or atexmsg - msg for ipbox logon (or the Atex j11)
	unixmail - mail for unix sendmail, file send to spool/2go
		for wheel to distribute using DU TOUNIXMAIL, DA client
	other	- send to another destination for this client.
		The 'dest' subparameter MUST exist for this client.
		If not the default dest is OTHER.
	ignore - ignore all files for this client !!

	Note that it is just the first letter of the client type that is checked and
is unique (or first two for FTP).

Various subparameters are possible for each client line (except for ALERTMSG) :
	Note that spaces are used to separate the subparams, so embed with double
quotes (or use 's'
		if you need to enter them in a string.
	type:	as above. This is the only mandatory subparameter.
	number:	for EMAIL	this is the email address of the subscriber.
		for TELEX	this is the telex number of the subscriber.
		for FAX		this is the telephone number of the subscriber.
		for DIAL	this is the telephone number of the subscriber.
		eg	client:fippo	type:unixmail	number:dot@fingerpost.co.uk
	dest:	for other	this is the DU or destination (in tables/sys/USERS)
	fiphdr:	by default the FipHdr is stripped off the output file.
		Use this to retain it.  Generally this is used to retain
		valuable FipHdr information for 'bdcast', 'other' types.
		There is no data with this subparam.
	extra:	Add more FipHdr fields to the outbound FipHdr.
		Any FipSeq can be added. Remember to add 'fiphdr:' too to carry
		over any old fipHdr fields, otherwise they will be ignored.
		eg	client:drongo type:bdcast extra:nQD:PostTime=$h:$nn
	before:	Add data at the top of the output file.
		eg	client:helene type:telex before:"Telex fpr The Mekonsn"
	after:	Add data at the end of the output file.
	xchg:	Run 'ipxchg' against the output file before sending/storing it.
		eg	client:daswong	type:bdcast	xchg:cr2nl
		where there is an xchg parameter file called tables/xchg/CR2NL
		For FTP, Fax and Unixmail this xchg is NOT run by 'ippost' but
		is flagged so that 'ipxchg' is run in the usual Fip Copy Flow.
	convnl: Convert all CR or NL to CR NL to create a Dos/Windows
		readable file.  This is automatically triggered for 'type:pc'
	outfmt:	The Output format file - FipHdr field DF or (for bdcast) DB.
	queue:	Place all files in this queue under spool/syn. This overrides
		the default which is :
			for dial	spool/2go
			for ftp		spool/2go
			for fax		spool/2go
			for unixmail	spool/2go
			for syn		spool/syn/(name of client)
			for bdcast	spool/(name of client)
		This allows several clients or sub-clients to share a transmission.

Any errors such as No Such Routing, are sent back to the sender if there is
both a source header field (SU) and a sub-source or author field (SA). In any
case they are logged.

Text is NOT translated except for 'pc' clients where NL-> CR NL unless FBIN.

For Alert messages, (ie if DU:ALERTMSG or ATEXMSG) then DA holds the logon for
the message. IPPOST checks the tables/mail/ALERTMSG (if it exists) or the
ATXMSG file (if it exists) and sends to a DU of MSGFORALERT or MSGFOR ATEX
	If not then the tables/mail/LOGON file is checked - if there, it is added to
the spool/mail/LOGON; otherwise it is sent back if there is an SA else it is

The destination for Msgs to Atex is MSGFORATX_NETWORK (uppercase) which should
be in the destination file (tables/sys/USERS) and uses output format (DF) of
ATXMSG which should be in tables/out, tables/news or tables/gateway.

For Fax, telex, ftp, unixmail (ie internet-mail) and dialup - If a specific
'queue' subkeyword is present for that client, the files are stuffed in that
folder. Otherwise all files are moved to spool/2go for 'ipwheel' to distribute
using a destination of :
	unixmail- TOUNIXMAIL
	w4	- POSTW4
	ftp	- POSTFTP
	dialup	- POSTDIAL
	fax	- POSTFAX
	telex	- POSTTLX

All incoming syndication and library files are saved in saves/DU - ie saves/lib
for library. (NOTE version change - this is now valid ONLY if the '-S' input
switch is specified).

Input Parameters are (all optional) :
	-q : input queue			default: spool/post
	-i : input queue			default: spool/post
	-d : done queue for incoming files	default: none
		Normally files are deleted after processing.
	-g : name of parameter file		default: tables/sys/GROUPS
	-t : scan time for the input queue	default: 5 secs
	-l : do NOT return logs to sender	default: log all errors
	-L : log all client entries for each file	default: no
	-m ; report errors as log type "ox"	default: report as a failure "x"
	-M ; do NOT report if the client does NOT exist	default: do
	-z : name of parameter file		default: tables/sys/GROUPS
	-Z : name of the optional POST file	default: sys/POST
	-v : version number and exit

An optional Parameter file - tables/sys/POST can be used for:
	; comment
	okmsg:rnLooks good to mern
	dudmsg:rnAbsolute rubbish, Thanksrn
	fiphdr-not-found: (2 letter FipHdr field)
	log-copy:(FipHdr DU(s))
	log-raw-copy:(FipHdr DU(s))
	balance-group:(Balance Group)
	balance-all-group:(Balance Group)
	balance-this-type: (POST type of file)
	round-robin-dial: (number)
	round-robin-ftp: (number)
	round-robin-unixmail: (number)
	round-robin-fax: (number)
	round-robin-telex: (number)
	add-unique-id: (optional FipHdr field)
	global-extra-fiphdr:	(FipHdr to be added to all files with 'fiphdr')
	newSN: (FipSeq for the SN)
	newDA: (FipSeq for the DA)
	chkexists: (Full Path/file or queuename)
	chkexists-timeout: (time between trying/logging if path is not there)
	xchg-program: Replacement path/file for ipxchg		default:/fip/bin/ipxchg
	xchg-queue: Replacement output folder for xchg		default: xchg
	wheel-queue: Replacement output folder for wheel/2go	default: 2go
	syn-queue: Replacement output folder for syn		default: syn
	extra-email-fiphdr: (FipHdr)
		Send a copy of the incoming file to the email addresses in this FipHdr (if it
is non-blank)
	check-primary-server: pseudo-host name that is specified in
	log-line: (FipSeq) log line

In more detail...
 - change the 2 messages  and to define any client types :
	; comment
	okmsg:rnLooks good to mern
	dudmsg:rnAbsolute rubbish, Thanksrn

 - any entry in the FipHdr DA field which is NOT specified in the GROUPS file
(or LOGON etc for mail types) is normally ignored/deleted. To send them to
another destination for further processing, use
where destination is the usual entry in the sys/USERS file.
Plus	fiphdr-not-found: (2 letter FipHdr field)
		This adds a FipHdr with either a '+' if the client is valid and '-' if not.

 -  make a copy of all files going to valid clients and send to this
	log-copy:(FipHdr DU(s))
eg	log-copy:w4xpost
Use this for W4 or another logging system.
	log-raw-copy: (ditto) is similar but is the IMPUT file (ie raw copy) rather
than the OUTPUT file which may have been through an xchg (UNIX only)

 -  make a mirror copy of all files going to valid clients using balance
	balance-group:(Balance Group)
eg	balance-group:balanceData
Normally only files NOT being passed to 'ipwheel' or 'ipxchg are balanced as
they too will balance their files.

To balance EVERYTHING, use balance-all-group:

Alternatively use 'balance-this-type' to balance ONLY files from that type.
ie	balance-this-type:syn

Alternatively use 'balance-on-queue-only' to balance ONLY files which are for a
client with a 'queue' subparameter.
ie	balance-on-queue-only:

 - Use several outputs for this type of delivery.
	round-robin-dial: (number)
	round-robin-ftp: (number)
	round-robin-unixmail: (number)
	round-robin-fax: (number)
	round-robin-telex: (number)
		Eg	round-robin-ftp:3
		So the first file for ftp is sent to POSTFTP1
			the second	to POSTFTP2
			the third	to POSTFTP3
			the fourth	to POSTFTP1 etc

	add-unique-id: (optional FipHdr field)
		This adds a new FipHdr field containing a unique ID
		which is made from the IPaddress, time and sequence number.
		It is quite long !

	global-extra-fiphdr:	(FipHdr to be added to all files with 'fiphdr')
		Use this to add more FipHdr fields to an existing hdr.
		It works only for those clients WITH the 'fiphdr:' specified.
		It is is addition to the 'extra:' keyword specified on each line.

	newSN: (FipSeq)
		The FipHdr field SN (source name) is used to track the file
		thru the system. If there is traffic which does NOT have an SN,
		use this to add one. eg
			;  test if there IS an SN
			combie:Q1	SN
			; add the SN if there is NOT one - use the original if there is.
		In this example, if there IS an SN wth something in it, that is used
		Otherwise, use the contents of WO, WR, WD and WE with dashes between.

	newDA: (FipSeq)
		The FipHdr field DA (sub-destinations) contains a list of all
		the groups and/or clients to send to.
		If there are files which do NOT have a DA, use this to add one.
			;  test if there IS an DA
			combie:QA	DA|AA,default
		Here we say use the QA FipHdr field which will be either the
		DA if there, or the AA FipHdr field if not, or the string
		'default' if neither have data.

	check-primary-server: pseudo-host name that is specified in
		Use this when an ipspool on 2 systems is accessing the same remote folder -
in a redundant way.
		that is used whether the current host should be getting the files or not.
	ie in the parameter file is
		and in the DEST_REDUN is
			; psuedohost	primary secondary
			postsvr	fip1	fip2
		and in the SYSTEM file for both fip1 AND fip2 there is  the same 'ippost'
line with the same param
eter file
		If fip1 is up,  'ippost' on fip1 will always get while on fip2 it will just
		If fip1 is down 'ippost' on fip2 will start processing the files.

- CHKEXISTS is a complete pathname of either a standing file or a queue. This
is used to check that NFS drives are actually mounted BEFORE writing new files.
eg : chkexists:/data1/nfs/helios/.Desktop
WinNT/2k remote drives (also called UNC) may be specified as :
(note the double \ for each 
 and the 'ignore-escapes to ignore the dots in the IP address or hostname being
The use of fwd slashes '/' rather than '' make the string much easier to read
Use 'chkexists' to make sure the drive is available first

- log-line: (FipSeq) log line
	A couple of temporary fipHdr fields are added
		E1 - index number of the client
		E2 - name of the client
		E3 - type of method - FTP, EMAIL etc
	default is "client.E2 : SN DU DA (E1.E3)"

The incoming file may optionally have a FipHdr POST_LOGCOPY: (FipSeq) to
override the 'logcopy' parameter. The fipHdr can have NO data to disable just
for that one file.

The incoming file may optionally have a FipHdr POST_CLIENT_ONLY: which
signifies that the sub-destination (DA:) must be a client only - all groups are

Where sections of FipHdr fields are required or changes to the output style,
use keywords : fixed, partial, combie, optional, repeat, newdate and/or style.
(see The SysAdmin manual for more information).

	They are normally specified :
		fixed:QZ	1234543
		partial:QT	ST,3,2,U,<,>
		combie:QY	ep|na,(0000000)a
		option:QE	ep,11,7,s
		repeat:QK	XK,-,3
	or	repeat:QP	PK,,4,#X
		style:QS	XN,%.03d
		replace:QZ	XK	a=b c=d ee=f

The current maximum number of clients in a group or a single transmission is

Version Control
;055x17	30mar00 added Sort of Groups and the Groups ptr table
	;a 28nov00 added POST_DIAL as a type
	;b 01dec00 added add-unique-id:
	;c/d 03feb01 added global-extra-fiphdr and newSN
	;e 08feb01 added round-robin-telex
	;f 15may01 bugette with case of first chr for Sort of clients
	;g 17aug01 added 'newDA' as FipSeq for DA
	;h 18mar02 added balance-this-type/DGtype
	;i 22mar02 added POST_RESEND_QUEUE as FipHdr
	;j 23mar02 cleanup of alerts
	;k-m 17dec03 WINNT cleanup
	;n 09feb04 bugette - allow 40 chrs for SN if none in FipHdr
	;o 03nov04 make names up to 100 chrs from 20.
	;p 11oct05 added FipRemote and Email
	;q 17jan06 added chkexists...
	;r-u 17aug08 added xchg/wheel/syn-queue and xchg-program
	;v 14sep06 added validClientFipHdr + or -
	;w 06nov06 added extra-email-fiphdr
	;x15 28feb07 added FipHdr POST_LOGCOPY: ;x1 strparse logcopy ;x2 log-raw-copy
		;x3 pramfile as -Z ;x4-7 added -L ;8 POSTW4 and reorder fiphdr
		;10 5jan09 added POST_CLIENT_ONLY:
		;11 28jun10 allow queue:/...
		;12 Win2k missing clients better
		;13-15 22jun11 added check-primary-server ;15 added log-line:
		;16 added speedy
		;17 30apr14 added file-trace

(copyright) 2014 and previous years FingerPost Ltd.