vwire
Parameter-defined message format input program.
The header is parsed and put in the FIP header field SH. The file is slotted
into the spool/2brouted queue for IPROUTE to process and route.
Input Parameters :
Mandatory :
-n : name of this wire (usually the agency) default: none
either
-s : Spider name or TTY device name default: none
-p : Spider port number default: none
or -P : Port on UnixBox to use for passive connections default: none
This is only used where Terminal Servers canNOT run reverse telnet
correctly and need to set a Permanent Virtual Connection across
the ethernet to a fixed port number on the UnixBox
Optional :
-L : For Passive (-P) type connections, reestablish the
link after disconnection default: no
-o : Output Folder if not spool/2brouted default: spool/2brouted
-x : Wakeup string for pad or modem default: none
This wakes up the pad on startup.
-y : Banner string to strip from terminal server.default: SpiderBanner
-Y : There is no banner to strip default: as above
-z : parameter file in tables/wire if different to -n name
-v : Print the version number and exit
The parameter table can have :
; comment line with preceding semicolon
soh: start of header string default: none
Normally there is only one 'soh' string but there
can be up to 10. For example AFP slow speed lines can
start either 'zczc' or 'glgl'.
soh:zczc
soh:glgl
stx: start of text string default: none
stx:02
eot: end of text string default: none
Normally there is only one 'eot' string but there
can be up to 10. For example where an incoming line is used
for both IPTC (end with a single EOT chr) and IATA (ends
with 'nnnn').
eot:04
eot:nnnn
timeout: timeout in secs for end of file (used if no EOT). default:none
ie timeout:30 VWIRE will wait for a gap of 30 seconds
with no data received before closing the file.
If no EOT is specified, timeout will default to 10 seconds; otherwise it
defaults to zero.
connect: connect string for dialup default: none
connect:CONNECT
discon: disconnect string for dialup default: none
discon:NO CARRIERr
modemok: String from the modem signalling OK default: none
modemok:OKr
The idea is to NOT echo this string if we get it.
banner: String to output on connection default: none
Used only for dialup and
extra-fiphdr:
or fixhdr: fixed header field(s) - used to add more FipHdr
fields or to force them to a fixed text. If more than
one field is added; use '#' or n' to separate
them. Note this is a FipSeq field so any
other FipHdr, partials etc can be used. default: none
dup: name of a DUPLICATE wire where 2 copies of the same
file is required (SD). default: none
route: use this routing table (SR : used in iproute)
default is name as per -n
case: header is case INsensitive - yes/no default: no
case:yes means ZCZC = ZcZc = zczc = ZCzc
case:no means ZCZC is ONLY ZCZC
bits: no of bits - 5, 6, 7, 8 default: 7
ansbck: ansbck string to be send back on eot default: none
ansfile: send file back as answer back default: no
The parameter is the number of lines to send back; if 0
the whole file is returned. Only the data part is sent.
anseof: For ansfile, optional end of file string default: none
answait: For ansfile, Wait in seconds after file has been received
before the 'ansfile' is sent back. Only used if 'ansfile'
or 'ansback' has been specified. default: 5 secs
binary: binary file - do NOT put a header on the files default: no
- yes/no
bau2asc: Translate ALL from 5level Baudot to 7level Ascii. def: no
This only applies when used with 'bits:5'. default: strip
keepbauesc: Do NOT strip the Baudot Escape chrs (033 and 037)
Normally these are stripped.
To Letters is a '<', while To Numbers is a '>'.
This only applies when used with 'bits:5' and 'bau2asc:'.
chrset: Source character Set (SC) default: ascii
This is generally overwritten at the routing stage.
outfmt: Output format for IPOUT (DF) default: vwire
enq: Incoming enq chr or string to prompt enqmsg msg default: none
enqmsg: String to be sent back on enq default: none
archive: archive log name if NOT that of name (-n) default: name
noarchive: Do NOT archive any incoming data def: do archive
hdrlines: header is no of lines default: no
linesep: lines separator in header
if not either CR or CR NL default: no
If there are a variable number of lines in some fields of the
header, disable ALL lines (except forthe first with 'linesep:'
striphdr: Strip hdr - number of lines to strip default: yes
- NO = do not strip at all
keepeol: do NOT ignore blank lines in the hdr default: ignore
keepall: pass thru all including SOH STX and EOT default: no
hdrprint: allow only ascii printables in the source hdr (SH)
Specify 'hdrprint:no' if you want non-printables. default: yes
Note that undesirables are converted to printable with a leading '^'
Eg a 23 is two chrs "^S"
toptxt: Information to add BEFORE the text in the file default: none
made of fixed data and FIP header fields.
Specify as tottxt:nXK - from SU)n
endtxt: Information to add AFTER the text in the file default: none
made of fixed data and FIP header fields.
Specify as endtxt: nXK - from SU)n
txtsoh: ignore duplicate soh in text default: no
wild: wild chr if not '*' ie wild:$ default: *
This must be the first parameter in the file or you could get
strange results.
Vwire uses a '*' to indicate that default string should be
used for that parameter.
If you have any strings starting with a '*' then change this
wild chr to a character NOT used to start a message.
seqno: add a sequence no to the filename
and increment automatically default: no
ignoreblank: Ignore/Strip files with no text default: no
- possibly check messages for example
filename: text and source header fields to make up the filename
fixed text is within punctuantion (which is ignored)
eg : (ABC)SN or -ABC-SN to give ABC plus the SerDes and Number
default is S for service designator and N for item number: SN
newname: A different method of specifying the filename. The parameter
for newname is a parsebale string :
eg newname:ABCXSXN or newname:ABC.$S
If newname is specified, it takes precedent over filename.
keyword: Name of a Keyword file in tables/keyword. The result(s) if any
is placed in the HK header field and can be sent back with
the 'ansbck' parameter.
debug: Create a debug file in spool/wiretest showing how the header
has been put together default: no
trailer: String for the start of the trailer default: none
Trailer lines are added to the Fip Hdr and are called 'SE' for
Extra lines are forced to be a space.
striptlr: Strip the trailer from the text part. default: no
timmsg: Message appended to the Text of a file which default: none
was timed out
timattn: String to Escape to the modem on timeout default: none
timattn:+++
timclr: String to clear down modem
If there is a timattn it is done beforehand.
timclr:ath0rn
checkhdr: End of header string for interactive wires. default not used
Where VWIRE is used for human input, this is the string which
terminates the header (started by the 'soh' string'). If we
have NOT received a valid 'soh', the 'chkbad' string is
sent back. If valid, the 'okhdr' string is sent.
This can be used to confirm to the remote user that their
copy is being received properly.
okhdr: Ok string for check header. ie file open, send data
badhdr: NOT Ok string for check header. ie no file open, start again
echo: echo header, data and any rubbish back to sender. def: no echo
allownohdr: allow data to pour in even though there MAYBE no hdr. def:no
Normally we want a valid header for the file.
zapondel: Delete previous chr if a DEL or BS (backspace) arrives in data. def
: no
logon: logon sequence of strings to send and to receive (see below)
default: not used.
waitchr:176 wait one sec chr default: tilde (0176)
chunk-file-size: (size in bytes) default: single file
Use this to specify the maximum take size chunk
Any incominmg file bigger than this is sent in multiple
files with FipHdr field D1: holding the take number.
The last take also sets D2 to 'END'.
The SN field also has a 3 digit extention containing the take
number so redundancy does NOT kick in to stip alternate files.
Otherwise the whole FipHdr is sent each time.
Use this option with care ! - The Chunk Size MUST be big
enough to contain the incoming hdr plus the first bit of data.
There are some functions which work badly with this option
such as Trailer FipHdr fields and Ack/Enqs.
The minimum size is 400 bytes.
chunk-in-file-order:
Normally chunked takes are in take-filename order. This parameter
changes this to filename-take order.
dump-data: This makes a copy in /fip/dump of all incoming raw data
timing-stats:yes/no
Show timings default: no
attrbute-style:
att-fiphdr:
see below
Note that all strings can be FipSeq - header fields (XN), System variables,
printable chrs, unix escape chrs (r, n) or octal numbers (03).
In addition, at the end of the file, the following extra variables can also be
used :
$c - no of characters in the file
$w - no of words in the file
which is no of chrs divided by 8 or the environment
variable FIP_VWIRE_WORD
$l - no of lines in the file
which is no of chrs divided by 80 or the environment
variable FIP_VWIRE_LINE
$q - time taken for the transmission in seconds
Where sections of FipHdr fields are required or changes to the output style,
use keywords : fixed, partial, combie, optional, repeat, newdate and/or style.
(see The SysAdmin manual for more information).
They are normally specified :
fixed:QZ 1234543
partial:QT ST,3,2,U,<,>
combie:QY ep|na,(0000000)a
option:QE ep,11,7,s
repeat:QK XK,-,3
or repeat:QP PK,,4,#X
style:QS XN,%.03d
Where VWIRE is being used in an interactive environment - using ENQ, Timeouts
etc - please remember to turn ECHO OFF on the modem - for Hayes modems, using
ate0.
---------------------------------------------------------------------
How to find out what the data looks like - reverse engineering at its worst!
Unless you have a nice clean specification in front of you, there is every
chance you are going to have to dig for the information about what header
information - priority, category, filename etc - are being sent and where in
the data stream can you find them.
Firstly you need to get some data!
If you have on diskette or someone can FTP some examples - fine, use them.
If not, put up a very simple VWIRE parameter file with 'dump-data' OR collect
the RAW data using 'ipdump' against the same input port as you want to run this
copy of 'vwire' (but remember to take the 'vwire' or any other program down
before starting 'ipdump' - only ONE program can sit on a port at any one time)
For example, if the line in the SYSTEM file is
pcdial in vwire -s termserv2 -p 31203 -n PCDIAL
then in a telnet window (at the shell prompt, not in 'ip'):
ipdump -s termserv2 -p 31203 -n pcdial -z
or ipdump -s termserv2 -p 31203 -n pcdial -z -L
(the second displays incoming data n the screen as it arrives - useful for slow
moving lines) and wait for data. Control C stops the programs and gets you back
to the prompt.
In this case, 'ipdump' dumps data to /fip/dump/PCDIAL_(JulianDate)
To test your parameter file, this data can be run using a second copy of
'vwire' and 'sffport' to send the data :
In a telnet window (at the shell prompt, not in 'ip'):
vwire -P 9111 -n pcdial
In another telnet window (at the shell prompt, not in 'ip'):
cd /fip/dump
sffport -s (hostname) -p 9111 (DumpfileName)
eg if the host is fip1 and the dump file is PCDIAL_102
sffport -s fip1 -p 9111 PCDIAL_102
Hint - always add the 'debug:' line to the parameter file as it will log a copy
a what it thinks as the header to a file in /fip/spool/wiretest.
Hint 2 - always remember to comment out the 'debug:' line once it is working !
(And probably zap the contents of spool/wiretest in the nightly maintenance too
!)
Now assuming you have some data, we need to try and work out the basics :
(everythingelse can wait till these are done!)
1. Can there be multiple Start-of-file and End-Of-files ?
you can have up to 10 'soh', 'stx' and 'eot' lines
2. Is there a chance there could be no end of data - like a journalist
forgetting the end-of-text marker ? - If so use 'timeout' to close the file
whithin a few seconds of the last chr arriving.
3. Where does the Header end and the Data/text begin ?
Is there a single chr or string we can put in the 'stx' line
Or are we just counting lines using 'hdrlines'
4. Do you need an answerback or an ENQ - use the relevant line.
5. For each Header line, how can we break apart the individual fields ?
There are two ways of breaking apart the individual fields in the header.
- if the fields look like they are in FIXED positions and/or always follow the
same sequential order : use the 'line' parameter (default)
eg line 1 is : abc123 c 4 998
line 2 is : Headline about a famous man
- if the fields are like HTML/XML attributes and have names which may be in
any sequence, use the 'attribute-style' line plus 'att-fiphdr' for each field.
eg cat=LIFE pri=33 keyword="FOOD GREED SIDS LUXURY" Headline="Food poisoning
at London's only 6-star greasy spoon"
Style 1 Fixed Positions using 'line' - default
----------------------------------------------
To specify fields for each line of the header :
line:(num):(sep) (hdrfld):(len):(type):(startString):(endString) (eol)
where
'num' is line number starting from 1
'sep' is (optional) end-of-line seq if not CR and/or LF
'hdrfld' is the single letter describing this field in the
FIP hdr field 'SH'. (see later) These are accessible
by specifying XZ in later programs where 'Z' is hdrfld.
'len' is the length of the string or 0 for any length.
'type' is :
a-alphabetic eg : letters only , aBnnBBBnn
u-uppercase eg : ABCDEF
l-lowercase eg : abcdef
n-number eg : 1234567890
x-alphanumeric eg : all numbers and letters (any case)
p-printable eg : Printable chrs plus Space
s-space eg : space, tab, CR, NL or FF
b-binary eg : anything !
c-control eg : chr is < octal 040 or >= 0177
z-anpa hdr field eg: alnum plus punctuation (NOT space)
t-punctuation eg : .,<>!"#$%^&*()_+-=[]{}:;@;
Note the actual definitions of what is chrs are defined
for each of these types is defined by SunOS/AIX using
the 'locale'. Use the 'man' pages for more information.
'startString' is the (optional) start string for the field.
'endString' is the (optional) end string for the field.
An example using an IPTC header :
Input file data will look something like :
(soh) PAL1234 (spc) 3 (spc) HHH (spc) 104 (spc) PA ADVISORY (cr) (nl)
PRESS Advisory re First editions (cr) (nl)
(stx)
text
(eot)
Parameter file:
; Vwire parameter file for PA
soh: 01
stx: 02
eot: 03
line:1 S:3:a N:4:n *:0:s P:1:n *:0:s C:3:x *:0:s w:0:n *:0:s R:0:p
line:2 k:0:p
All other fields/lines etc left as defaults
This will save the following fields for later use :
XS - service designator = PAL
XN - service seq number = 1234
XP - priority = 3
XC - category code = HHH
XW - no of words = 104
XR - reference field = PA ADVISORY
XK - keyword = PRESS Advisory re etc
Note that the default line separator is CR or NL in whatever chrset is used.
Style 2 Attibute style using 'attribute-style' and 'att-fiphdr'
-------------------------------------------------------------
Specify 'attibute-style' once in the parameter file and a 'att-fiphdr' for each
line you wish to save.
; style is one line per parameter ending with some combination of CR and/or NL
; the default style is to have spaces only between attributes
attribute-Style:n
att-fiphdr:(Incoming Header name):(FipHdr)
att-fiphdr:cat:C
att-fiphdr:headline:K
So Headline will be in the SH (SourceHeader) field K and accessible using XK
Using DEBUG
-----------
The DEBUG: keyword enables you to finetune the parameter file. If enabled, it
creates a file in 'spool/wiretest' with the following type of information :
--- New File at Tue Apr 18 13:27:28 ---
--- Parse the Header ...
--- No STX and hdrlines=none : NoHeader ...
--- FileName:WETH002|
--- Output the Header ...
SU:prweth
ST:vwire
SP:rack0-ts1,1105
SN:WETH002
SC:ascii
DF:vwire
ZI:
--- Output the Text ...
Some test data
Turn DEBUG off and on by adding/commenting out 'debug:' in the parameter file
and stopping/starting the relevant VWIRE using 'ip' the user interface.
For Logons, The strings are in uucp-like format where the syntax is :
(send1string) (space) (receive1string) (spc) (s2) (spc) (r2) etc
ie you send1, wait 5secs for r1, send s2 wait 5secs for r2 etc.
eg: logon:~"" Userid: ~rootr Password? ~bongosr LanRoverE_FIP# ~"connect
port5r" options
This mimics the following sequence :
@ Userid: root
Password?
Shiva LanRover/E PLUS, Version 5.7 98/11/06
LanRoverE_JV1# connect port5
Connecting to Serial5 at 9600 BPS.
Escape character is CTRL-^ (30).
Type the escape character followed by C to get back,
or followed by ? to see other options.
To send a space, use s or dbl quote that string.
To wait a second - send a tilde (or redefine to your Wait chr)
All keywords - setup, logon etc - are optional - if one does not exist, it is
i
gnored.
Embedded spaces may be specified by enclosing within double quotes.
Null fields can be specified by two consequetive double quotes - "". If a wait
i
s required howver ALWAYS SPECIFY BEFORE the dbl quotes eg: ~~~"" for a 3 second
wait with nothing returned.
(copyright) 2014 and previous years FingerPost Ltd.