vwire (Sat Oct 25 2014 01:31:01)


Parameter-defined message format input program.

The header is parsed and put in the FIP header field SH. The file is slotted
into the spool/2brouted queue for IPROUTE to process and route.

Input Parameters :
	Mandatory :
	-n : name of this wire (usually the agency)		default: none
	-s : Spider name or TTY device name			default: none
	-p : Spider port number					default: none
or	-P : Port on UnixBox to use for passive connections	default: none
		This is only used where Terminal Servers canNOT run reverse telnet
		correctly and need to set a Permanent Virtual Connection across
		the ethernet to a fixed port number on the UnixBox
	Optional :
	-L : For Passive (-P) type connections, reestablish the
		link after disconnection			default: no
	-o : Output Folder if not spool/2brouted		default: spool/2brouted
	-x : Wakeup string for pad or modem			default: none
		This wakes up the pad on startup.
	-y : Banner string to strip from terminal server.default: SpiderBanner
	-Y : There is no banner to strip		  default: as above
	-z : parameter file in tables/wire if different to -n name
	-v : Print the version number and exit

The parameter table can have :

	; comment line with preceding semicolon
	soh:	start of header string				default: none
		Normally there is only one 'soh' string but there
		can be up to 10. For example AFP slow speed lines can
		start either 'zczc' or 'glgl'.
	stx:	start of text string				default: none
	eot:	end of text string				default: none
		Normally there is only one 'eot' string but there
		can be up to 10. For example where an incoming line is used
		for both IPTC (end with a single EOT chr) and IATA (ends
		with 'nnnn').
	timeout: timeout in secs for end of file (used if no EOT). default:none
		ie timeout:30	VWIRE will wait for a gap of 30 seconds
		with no data received before closing the file.

If no EOT is specified, timeout will default to 10 seconds; otherwise it
defaults to zero.

	connect: connect string for dialup			default: none
	discon:  disconnect string for dialup			default: none
			discon:NO CARRIERr
	modemok: String from the modem signalling OK		default: none
		The idea is to NOT echo this string if we get it.
	banner: String to output on connection			default: none
		Used only for dialup and
or	fixhdr:	 fixed header field(s) - used to add more FipHdr
		fields or to force them to a fixed text. If more than
		one field is added; use '#' or n' to separate
		them. Note this is a FipSeq field so any
		other FipHdr, partials etc can be used.		default: none
	dup:	name of a DUPLICATE wire where 2 copies of the same
		file is required (SD).				default: none
	route:	use this routing table (SR : used in iproute)
		default is name as per -n
	case:	header is case INsensitive	- yes/no	default: no
			case:yes 	means ZCZC = ZcZc = zczc = ZCzc
			case:no		means ZCZC is ONLY ZCZC
	bits:	no of bits - 5, 6, 7, 8				default: 7
	ansbck:	ansbck string to be send back on eot		default: none
	ansfile: send file back as answer back			default: no
		  The parameter is the number of lines to send back; if 0
		  the whole file is returned. Only the data part is sent.
	anseof: For ansfile, optional end of file string	 default: none
	answait: For ansfile, Wait in seconds after file has been received
		  before the 'ansfile' is sent back. Only used if 'ansfile'
		  or 'ansback' has been specified.		default: 5 secs
	binary:	binary file - do NOT put a header on the files	default: no
		 - yes/no
	bau2asc: Translate ALL from 5level Baudot to 7level Ascii. def: no
		This only applies when used with 'bits:5'.	default: strip
	keepbauesc:	Do NOT strip the Baudot Escape chrs (033 and 037)
		Normally these are stripped.
		To Letters is a '<', while To Numbers is a '>'.
		This only applies when used with 'bits:5' and 'bau2asc:'.
	chrset:	Source character Set (SC)			default: ascii
		This is generally overwritten at the routing stage.
	outfmt:	Output format for IPOUT (DF)			default: vwire
	enq:	Incoming enq chr or string to prompt enqmsg msg	default: none
	enqmsg:	String to be sent back on enq			default: none
	archive: archive log name if NOT that of name (-n)	default: name
	noarchive: Do NOT archive any incoming data		def: do archive
	hdrlines: header is no of lines 			default: no
	linesep: lines separator in header
		if not either CR or CR NL 			default: no
		If there are a variable number of lines in some fields of the
		header, disable ALL lines (except forthe first with 'linesep:'
	striphdr: Strip hdr	- number of lines to strip	default: yes
				- NO = do not strip at all
	keepeol: do NOT ignore blank lines in the hdr		default: ignore
	keepall: pass thru all including SOH STX and EOT	default: no
	hdrprint: allow only ascii printables in the source hdr (SH)
		Specify 'hdrprint:no' if you want non-printables. default: yes
		Note that undesirables are converted to printable with a leading '^'
		Eg a 23 is two chrs "^S"
	toptxt: Information to add BEFORE the text in the file	default: none
		made of fixed data and FIP header fields.
		Specify as tottxt:nXK - from SU)n
	endtxt: Information to add AFTER the text in the file	default: none
		made of fixed data and FIP header fields.
		Specify as endtxt: nXK - from SU)n
	txtsoh:	ignore duplicate soh in text			default: no
	wild:	wild chr if not '*'	ie wild:$		default: *
		This must be the first parameter in the file or you could get
		strange results.
		Vwire uses a '*' to indicate that default string should be
		used for that parameter.
		If you have any strings starting with a '*' then change this
		wild chr to a character NOT used to start a message.
	seqno: add a sequence no to the filename
		and increment automatically			default: no
	ignoreblank: Ignore/Strip files with no text		default: no
		- possibly check messages for example
	filename: text and source header fields to make up the filename
		fixed text is within punctuantion (which is ignored)
		eg : (ABC)SN or -ABC-SN to give ABC plus the SerDes and Number
		default is S for service designator and N for item number: SN
	newname: A different method of specifying the filename. The parameter
		for newname is a parsebale string :
		eg	newname:ABCXSXN	or newname:ABC.$S
		If newname is specified, it takes precedent over filename.
	keyword: Name of a Keyword file in tables/keyword. The result(s) if any
		is placed in the HK header field and can be sent back with
		the 'ansbck' parameter.
	debug:	Create a debug file in spool/wiretest showing how the header
		has been put together				default: no
	trailer: String for the start of the trailer		default: none
		Trailer lines are added to the Fip Hdr and are called 'SE' for
		Extra lines are forced to be a space.
	striptlr: Strip the trailer from the text part.		default: no
	timmsg:	Message appended to the Text of a file which	default: none
		was timed out
	timattn: String to Escape to the modem on timeout	default: none
	timclr:	String to clear down modem
			If there is a timattn it is done beforehand.
	checkhdr: End of header string for interactive wires.	default not used
		Where VWIRE is used for human input, this is the string which
		terminates the header (started by the 'soh' string'). If we
		have NOT received a valid 'soh', the 'chkbad' string is
		sent back. If valid, the 'okhdr' string is sent.
		This can be used to confirm to the remote user that their
		copy is being received properly.
	okhdr:	Ok string for check header.	ie file open, send data
	badhdr:	NOT Ok string for check header. ie no file open, start again
	echo:	echo header, data and any rubbish back to sender. def: no echo
	allownohdr: allow data to pour in even though there MAYBE no hdr. def:no
		Normally we want a valid header for the file.
	zapondel:  Delete previous chr if a DEL or BS (backspace) arrives in data. def
: no
	logon: logon sequence of strings to send and to receive (see below)
							default: not used.
	waitchr:176	wait one sec chr		default: tilde (0176)
	chunk-file-size: (size in bytes)		default: single file
		Use this to specify the maximum take size chunk
		Any incominmg file bigger than this is sent in multiple
		files with FipHdr field D1: holding the take number.
		The last take also sets D2 to 'END'.
		The SN field also has a 3 digit extention containing the take
		number so redundancy does NOT kick in to stip alternate files.
		Otherwise the whole FipHdr is sent each time.
		Use this option with care ! - The Chunk Size MUST be big
		enough to contain the incoming hdr plus the first bit of data.
		There are some functions which work badly with this option
		such as Trailer FipHdr fields and Ack/Enqs.
		The minimum size is 400 bytes.
		Normally chunked takes are in take-filename order. This parameter
		changes this to filename-take order.
	dump-data: This makes a copy in /fip/dump of all incoming raw data
		Show timings		default: no

		see below

Note that all strings can be FipSeq - header fields (XN), System variables,
printable chrs, unix escape chrs (r, n) or octal numbers (03).
In addition, at the end of the file, the following extra variables can also be
used :
	$c - no of characters in the file
	$w - no of words in the file
		which is no of chrs divided by 8 or the environment
		variable FIP_VWIRE_WORD
	$l - no of lines in the file
		which is no of chrs divided by 80 or the environment
		variable FIP_VWIRE_LINE
	$q - time taken for the transmission in seconds

Where sections of FipHdr fields are required or changes to the output style,
use keywords : fixed, partial, combie, optional, repeat, newdate and/or style.
(see The SysAdmin manual for more information).

	 They are normally specified :
		  fixed:QZ	 1234543
		  partial:QT	ST,3,2,U,<,>
		  combie:QY	ep|na,(0000000)a
		  option:QE	ep,11,7,s
		  repeat:QK	XK,-,3
	 or	repeat:QP	PK,,4,#X
		  style:QS	 XN,%.03d

Where VWIRE is being used in an interactive environment - using ENQ, Timeouts
etc -  please remember to turn ECHO OFF on the modem - for Hayes modems, using

How to find out what the data looks like - reverse engineering at its worst!

Unless you have a nice clean specification in front of you, there is every
chance you are going to have to dig for the information about what header
information - priority, category, filename etc - are being sent and where in
the data stream can you find them.

Firstly you need to get some data!

If you have on diskette or someone can FTP some examples - fine, use them.

If not, put up a very simple VWIRE parameter file with 'dump-data' OR collect
the RAW data using 'ipdump' against the same input port as you want to run this
copy of 'vwire' (but remember to take the 'vwire' or any other program down
before starting 'ipdump' - only ONE program can sit on a port at any one time)

For example, if the line in the SYSTEM file is
	pcdial	in	vwire -s termserv2 -p 31203 -n PCDIAL
then in a telnet window (at the shell prompt, not in 'ip'):
	ipdump -s termserv2 -p 31203 -n pcdial -z
or	ipdump -s termserv2 -p 31203 -n pcdial -z -L
(the second displays incoming data n the screen as it arrives - useful for slow
moving lines) and wait for data. Control C stops the programs and gets you back
to the prompt.
In this case, 'ipdump' dumps data to /fip/dump/PCDIAL_(JulianDate)

To test your parameter file, this data can be run using a second copy of
'vwire' and 'sffport' to send the data :
	In a telnet window (at the shell prompt, not in 'ip'):
		vwire -P 9111 -n pcdial
	In another telnet window (at the shell prompt, not in 'ip'):
		cd /fip/dump
		sffport -s (hostname) -p 9111 (DumpfileName)
	eg if the host is fip1 and the dump file is PCDIAL_102
		sffport -s fip1 -p 9111 PCDIAL_102
Hint - always add the 'debug:' line to the parameter file as it will log a copy
a what it thinks as the header to a file in /fip/spool/wiretest.
Hint 2 - always remember to comment out the 'debug:' line once it is working !
(And probably zap the contents of spool/wiretest in the nightly maintenance too

Now assuming you have some data, we need to try and work out the basics :
(everythingelse can wait till these are done!)
	1. Can there be multiple Start-of-file and End-Of-files ?
		you can have up to 10 'soh', 'stx' and 'eot' lines
	2. Is there a chance there could be no end of data - like a journalist
forgetting the end-of-text marker ? - If so use 'timeout' to close the file
whithin a few seconds of the last chr arriving.
	3. Where does the Header end and the Data/text begin ?
		Is there a single chr or string we can put in the 'stx' line
		Or are we just counting lines using 'hdrlines'
	4. Do you need an answerback or an ENQ - use the relevant line.
	5. For each Header line, how can we break apart the individual fields ?

There are two ways of breaking apart the individual fields in the header.
	- if the fields look like they are in FIXED positions and/or always follow the
same sequential order :	use the 'line' parameter (default)
		eg	line 1 is :	abc123 c 4 998
			line 2 is :	Headline about a famous man
	- if the fields are like HTML/XML attributes and have names which may be in
any sequence, use the 'attribute-style' line plus 'att-fiphdr' for each field.
		eg	cat=LIFE pri=33 keyword="FOOD GREED SIDS LUXURY" Headline="Food poisoning
at London's only 6-star greasy spoon"

Style 1 Fixed Positions using 'line' - default

To specify fields for each line of the header :
	line:(num):(sep)  (hdrfld):(len):(type):(startString):(endString) (eol)
		'num' is line number starting from 1
		'sep' is (optional) end-of-line seq if not CR and/or LF
		'hdrfld' is the single letter describing this field in the
			FIP hdr field 'SH'. (see later) These are accessible
			by specifying XZ in later programs where 'Z' is hdrfld.
		'len' is the length of the string or 0 for any length.
		'type' is :
			a-alphabetic	eg : letters only , aBnnBBBnn
			u-uppercase	eg : ABCDEF
			l-lowercase	eg : abcdef
			n-number	eg : 1234567890
			x-alphanumeric	eg : all numbers and letters (any case)
			p-printable	eg : Printable chrs plus Space
			s-space		eg : space, tab, CR, NL or FF
			b-binary 	eg : anything !
			c-control 	eg : chr is < octal 040 or >= 0177
			z-anpa hdr field eg: alnum plus punctuation (NOT space)
			t-punctuation	eg : .,<>!"#$%^&*()_+-=[]{}:;@;
			Note the actual definitions of what is chrs are defined
			for each of these types is defined by SunOS/AIX using
			the 'locale'. Use the 'man' pages for more information.
		'startString' is the (optional) start string for the field.
		'endString' is the (optional) end string for the field.

An example using an IPTC header :
Input file data will look something like :
	(soh) PAL1234 (spc) 3 (spc) HHH (spc) 104 (spc) PA ADVISORY (cr) (nl)
	PRESS Advisory re First editions (cr) (nl)
Parameter file:
	; Vwire parameter file for PA
	soh:	01
	stx:	02
	eot:	03
	line:1	S:3:a N:4:n *:0:s P:1:n *:0:s C:3:x *:0:s w:0:n *:0:s R:0:p
	line:2	k:0:p
All other fields/lines etc left as defaults

	This will save the following fields for later use :
		XS - service designator		= PAL
		XN - service seq number		= 1234
		XP - priority			= 3
		XC - category code		= HHH
		XW - no of words		= 104
		XR - reference field		= PA ADVISORY
		XK - keyword			= PRESS Advisory re etc

Note that the default line separator is CR or NL in whatever chrset is used.

Style 2 Attibute style using 'attribute-style' and 'att-fiphdr'

Specify 'attibute-style' once in the parameter file and a 'att-fiphdr' for each
line you wish to save.

	; style is one line per parameter ending with some combination of CR and/or NL
	; the default style is to have spaces only between attributes
	att-fiphdr:(Incoming Header name):(FipHdr)

So Headline will be in the SH (SourceHeader) field K and accessible using XK


The DEBUG: keyword enables you to finetune the parameter file. If enabled, it
creates a file in 'spool/wiretest' with the following type of information :

--- New File at Tue Apr 18 13:27:28 ---

--- Parse the Header ...

--- No STX and hdrlines=none : NoHeader ...

--- FileName:WETH002|

--- Output the Header ...

--- Output the Text ...
Some test data

Turn DEBUG off and on by adding/commenting out 'debug:' in the parameter file
and stopping/starting the relevant VWIRE using 'ip' the user interface.

For Logons, The strings are in uucp-like format where the syntax is :

	(send1string) (space) (receive1string) (spc) (s2) (spc) (r2) etc
ie you send1, wait 5secs for r1, send s2 wait 5secs for r2 etc.

eg:	logon:~"" Userid: ~rootr Password? ~bongosr LanRoverE_FIP# ~"connect
port5r" options

This mimics the following sequence :
@ Userid: root

Shiva LanRover/E PLUS, Version 5.7 98/11/06
LanRoverE_JV1# connect port5
Connecting to Serial5 at 9600 BPS.
Escape character is CTRL-^ (30).

Type the escape character followed by C to get back,
or followed by ? to see other options.

To send a space, use s or dbl quote that string.
To wait a second - send a tilde (or redefine to your Wait chr)
All keywords - setup, logon etc -  are optional - if one does not exist, it is
Embedded spaces may be specified by enclosing within double quotes.

Null fields can be specified by two consequetive double quotes - "". If a wait
s required howver ALWAYS SPECIFY BEFORE the dbl quotes eg: ~~~"" for a 3 second
wait with nothing returned.

(copyright) 2014 and previous years FingerPost Ltd.