vwire

vwire

Parameter-defined message format input program.

The header is parsed and put in the FIP header field SH. The file is slotted
into the spool/2brouted queue for IPROUTE to process and route.

Input Parameters :
    Mandatory :
    -n : name of this wire (usually the agency)     default: none
either
    -s : Spider name or TTY device name         default: none
    -p : Spider port number                 default: none
or  -P : Port on UnixBox to use for passive connections default: none
        This is only used where Terminal Servers canNOT run reverse telnet
        correctly and need to set a Permanent Virtual Connection across
        the ethernet to a fixed port number on the UnixBox
    Optional :
    -L : For Passive (-P) type connections, reestablish the
        link after disconnection            default: no
    -f : Extra FIP header information           default: none
    -o : Output Folder if not spool/2brouted        default: spool/2brouted
    -x : Proxy or Wakeup string for network device          default: none
        This wakes up the pad on startup or Connects via a proxy
    -y : Banner string to strip FROM terminal server.default: SpiderBanner
    -Y : There is no banner to strip          default: as above
    -z : parameter file in tables/wire if different to -n name
    -v : Print the version number and exit

The parameter table can have :

    ; comment line with preceding semicolon
    soh:    start of header string              default: none
        Normally there is only one 'soh' string but there
        can be up to 10. For example AFP slow speed lines can
        start either 'zczc' or 'glgl'.
            soh:zczc
            soh:glgl
    stx:    start of text string                default: none
            stx:\002
    eot:    end of text string              default: none
        Normally there is only one 'eot' string but there
        can be up to 10. For example where an incoming line is used
        for both IPTC (end with a single EOT chr) and IATA (ends
        with 'nnnn').
            eot:\004
            eot:nnnn
    timeout: timeout in secs for end of file (used if no EOT). default:none
        ie timeout:30   VWIRE will wait for a gap of 30 seconds
        with no data received before closing the file.

If no EOT is specified, timeout will default to 10 seconds; otherwise it
defaults to zero.

    connect: connect string for dialup          default: none
            connect:CONNECT
    discon:  disconnect string for dialup           default: none
            discon:NO CARRIER\r
    modemok: String from the modem signalling OK        default: none
            modemok:OK\r
        The idea is to NOT echo this string if we get it.
    banner: String TO OUTPUT On connection          default: none
        Used only for dialup and proxy
    extra-fiphdr:
or  fixhdr:  fixed header field(s) - used to add more FipHdr
        fields or to force them to a fixed text. If more than
        one field is added; use '#' or \n' to separate
        them. Note this is a FipSeq field so any
        other FipHdr, partials etc can be used.     default: none
    dup:    name of a DUPLICATE wire where 2 copies of the same
        file is required (SD).              default: none
    route:  use this routing table (SR : used in iproute)
        default is name as per -n
    case:   header is case INsensitive  - yes/no    default: no
            case:yes    means ZCZC = ZcZc = zczc = ZCzc
            case:no     means ZCZC is ONLY ZCZC
    bits:   no of bits - 5, 6, 7, 8             default: 7
    ansbck: ansbck string to be send back on eot        default: none
    ansfile: send file back as answer back          default: no
          The parameter is the number of lines to send back; if 0
          the whole file is returned. Only the data part is sent.
    anseof: For ansfile, optional end of file string     default: none
    answait: For ansfile, Wait in seconds after file has been received
          before the 'ansfile' is sent back. Only used if 'ansfile'
          or 'ansback' has been specified.      default: 5 secs
    binary: binary file - do NOT put a header on the files  default: no
         - yes/no
    bau2asc: Translate ALL from 5level Baudot to 7level Ascii. def: no
        This only applies when used with 'bits:5'.  default: strip
    keepbauesc: Do NOT strip the Baudot Escape chrs (033 and 037)
        Normally these are stripped.
        To Letters is a '<', while To Numbers is a '>'.
        This only applies when used with 'bits:5' and 'bau2asc:'.
    chrset: Source character Set (SC)           default: ascii
        This is generally overwritten at the routing stage.
    outfmt: Output format for IPOUT (DF)            default: vwire
    enq:    Incoming enq chr or string to prompt enqmsg msg default: none
    enqmsg: String to be sent back on enq           default: none
    archive: archive log name if NOT that of name (-n)  default: name
    noarchive: Do NOT archive any incoming data     def: do archive
    hdrlines: header is no of lines             default: no
    linesep: lines separator in header
        if not either CR or CR NL           default: no
        If there are a variable number of lines in some fields of the
        header, disable ALL lines (except forthe first with 'linesep:'
    striphdr: Strip hdr - number of lines to strip  default: yes
                - NO = do not strip at all
    keepeol: do NOT ignore blank lines in the hdr       default: ignore
    keepall: pass thru all including SOH STX and EOT    default: no
    hdrprint: allow only ascii printables in the source hdr (SH)
        Specify 'hdrprint:no' if you want non-printables. default: yes
        Note that undesirables are converted to printable with a leading '^'
        Eg a \023 is two chrs "^S"
    toptxt: Information to add BEFORE the text in the file  default: none
        made of fixed data and FIP header fields.
        Specify as tottxt:\n\XK - from SU)\n
    endtxt: Information to add AFTER the text in the file   default: none
        made of fixed data and FIP header fields.
        Specify as endtxt: \n\XK - from SU)\n
    txtsoh: ignore duplicate soh in text            default: no
    wakeup: (FipSeq)
    proxy-connect: (FipSeq)
        Proxy connect or Wakeup string for network device   default: none
        This wakes up the pad on startup or Connects via a proxy
        (same as -x)
    keep-alive: (FipSeq)                    default: no keep alives sent
    keep-alive-interval: 60                 default: 60
        Send keep-alive string if there has been NO traffic for the interval time (in
seconds)
        eg keep-alive:\r\n
    wild:   wild chr if not '*' ie wild:$       default: *
        This must be the first parameter in the file or you could get
        strange results.
        Vwire uses a '*' to indicate that default string should be
        used for that parameter.
        If you have any strings starting with a '*' then change this
        wild chr to a character NOT used to start a message.
    seqno: add a sequence no to the filename
        and increment automatically         default: no
    ignoreblank: Ignore/Strip files with no text        default: no
        - possibly check messages for example
    filename: text and source header fields to make up the filename
        fixed text is within punctuantion (which is ignored)
        eg : (ABC)SN or -ABC-SN to give ABC plus the SerDes and Number
        default is S for service designator and N for item number: SN
    newname: A different method of specifying the filename. The parameter
        for newname is a parsebale string :
        eg  newname:ABC\XS\XN   or newname:ABC.\$S
        If newname is specified, it takes precedent over filename.
    keyword: Name of a Keyword file in tables/keyword. The result(s) if any
        is placed in the HK header field and can be sent back with
        the 'ansbck' parameter.
    debug:  Create a debug file in spool/wiretest showing how the header
        has been put together               default: no
    trailer: String for the start of the trailer        default: none
        Trailer lines are added to the Fip Hdr and are called 'SE' for
        Extra lines are forced to be a space.
    striptlr: Strip the trailer from the text part.     default: no
    timmsg: Message appended to the Text of a file which    default: none
        was timed out
    timattn: String to Escape to the modem on timeout   default: none
            timattn:+++
    timclr: String to clear down modem
            If there is a timattn it is done beforehand.
            timclr:ath0\r\n
    checkhdr: End of header string for interactive wires.   default not used
        Where VWIRE is used for human input, this is the string which
        terminates the header (started by the 'soh' string'). If we
        have NOT received a valid 'soh', the 'chkbad' string is
        sent back. If valid, the 'okhdr' string is sent.
        This can be used to confirm to the remote user that their
        copy is being received properly.
    okhdr:  Ok string for check header. ie file open, send data
    badhdr: NOT Ok string for check header. ie no file open, start again
    echo:   echo header, data and any rubbish back to sender. def: no echo
    allownohdr: allow data to pour in even though there MAYBE no hdr. def:no
        Normally we want a valid header for the file.
    zapondel:  Delete previous chr if a DEL or BS (backspace) arrives in data. def
: no
    logon: logon sequence of strings to send and to receive (see below)
                            default: not used.
    waitchr:\176    wait one sec chr        default: tilde (0176)
    chunk-file-size: (size in bytes)        default: single file
        Use this to specify the maximum take size chunk
        Any incominmg file bigger than this is sent in multiple
        files with FipHdr field D1: holding the take number.
        The last take also sets D2 to 'END'.
        The SN field also has a 3 digit extention containing the take
        number so redundancy does NOT kick in to stip alternate files.
        Otherwise the whole FipHdr is sent each time.
        Use this option with care ! - The Chunk Size MUST be big
        enough to contain the incoming hdr plus the first bit of data.
        There are some functions which work badly with this option
        such as Trailer FipHdr fields and Ack/Enqs.
        The minimum size is 400 bytes.
    chunk-in-file-order:
        Normally chunked takes are in take-filename order. This parameter
        changes this to filename-take order.
    dump-data: This makes a copy in /fip/dump of all incoming raw data
    timing-stats:yes/no
        Show timings        default: no

    attrbute-style:
    att-fiphdr:
        see below

Note that all strings can be FipSeq - header fields (\XN), System variables,
printable chrs, unix escape chrs (\r, \n) or octal numbers (\003).
In addition, at the end of the file, the following extra variables can also be
used :
    \$c - no of characters in the file
    \$w - no of words in the file
        which is no of chrs divided by 8 or the environment
        variable FIP_VWIRE_WORD
    \$l - no of lines in the file
        which is no of chrs divided by 80 or the environment
        variable FIP_VWIRE_LINE
    \$q - time taken for the transmission in seconds

Where sections of FipHdr fields are required or changes to the output style,
use keywords : fixed, partial, combie, optional, repeat, newdate and/or style.
(see The SysAdmin manual for more information).

     They are normally specified :
          fixed:QZ   1234543
          partial:QT    ST,3,2,U,<,>
          combie:QY ep|na,(0000000)a
          option:QE ep,11,7,s
          repeat:QK XK,-,3
     or repeat:QP   PK,,4,#X
          style:QS   XN,%.03d

Where VWIRE is being used in an interactive environment - using ENQ, Timeouts
etc -  please remember to turn ECHO OFF on the modem - for Hayes modems, using
ate0.

---------------------------------------------------------------------
How to find out what the data looks like - reverse engineering at its worst!

Unless you have a nice clean specification in front of you, there is every
chance you are going to have to dig for the information about what header
information - priority, category, filename etc - are being sent and where in
the data stream can you find them.

Firstly you need to get some data!

If you have on diskette or someone can FTP some examples - fine, use them.

If not, put up a very simple VWIRE parameter file with 'dump-data' OR collect
the RAW data using 'ipdump' against the same input port as you want to run this
copy of 'vwire' (but remember to take the 'vwire' or any other program down
before starting 'ipdump' - only ONE program can sit on a port at any one time)

For example, if the line in the SYSTEM file is
    pcdial  in  vwire -s termserv2 -p 31203 -n PCDIAL
then in a telnet window (at the shell prompt, not in 'ip'):
    ipdump -s termserv2 -p 31203 -n pcdial -z
or  ipdump -s termserv2 -p 31203 -n pcdial -z -L
(the second displays incoming data n the screen as it arrives - useful for slow
moving lines) and wait for data. Control C stops the programs and gets you back
to the prompt.
In this case, 'ipdump' dumps data to /fip/dump/PCDIAL_(JulianDate)

To test your parameter file, this data can be run using a second copy of
'vwire' and 'sffport' to send the data :
    In a telnet window (at the shell prompt, not in 'ip'):
        vwire -P 9111 -n pcdial
    In another telnet window (at the shell prompt, not in 'ip'):
        cd /fip/dump
        sffport -s (hostname) -p 9111 (DumpfileName)
    eg if the host is fip1 and the dump file is PCDIAL_102
        sffport -s fip1 -p 9111 PCDIAL_102
Hint - always add the 'debug:' line to the parameter file as it will log a copy
a what it thinks as the header to a file in /fip/spool/wiretest.
Hint 2 - always remember to comment out the 'debug:' line once it is working !
(And probably zap the contents of spool/wiretest in the nightly maintenance too
!)

Now assuming you have some data, we need to try and work out the basics :
(everythingelse can wait till these are done!)
    1. Can there be multiple Start-of-file and End-Of-files ?
        you can have up to 10 'soh', 'stx' and 'eot' lines
    2. Is there a chance there could be no end of data - like a journalist
forgetting the end-of-text marker ? - If so use 'timeout' to close the file
whithin a few seconds of the last chr arriving.
    3. Where does the Header end and the Data/text begin ?
        Is there a single chr or string we can put in the 'stx' line
        Or are we just counting lines using 'hdrlines'
    4. Do you need an answerback or an ENQ - use the relevant line.
    5. For each Header line, how can we break apart the individual fields ?

There are two ways of breaking apart the individual fields in the header.
    - if the fields look like they are in FIXED positions and/or always follow the
same sequential order : use the 'line' parameter (default)
        eg  line 1 is : abc123 c 4 998
            line 2 is : Headline about a famous man
    - if the fields are like HTML/XML attributes and have names which may be in
any sequence, use the 'attribute-style' line plus 'att-fiphdr' for each field.
        eg  cat=LIFE pri=33 keyword="FOOD GREED SIDS LUXURY" Headline="Food poisoning
at London's only 6-star greasy spoon"

Style 1 Fixed Positions using 'line' - default
----------------------------------------------

To specify fields for each line of the header :
    line:(num):(sep)  (hdrfld):(len):(type):(startString):(endString) (eol)
    where
        'num' is line number starting from 1
        'sep' is (optional) end-of-line seq if not CR and/or LF
        'hdrfld' is the single letter describing this field in the
            FIP hdr field 'SH'. (see later) These are accessible
            by specifying \XZ in later programs where 'Z' is hdrfld.
        'len' is the length of the string or 0 for any length.
        'type' is :
            a-alphabetic    eg : letters only , aBnnBBBnn
            u-uppercase eg : ABCDEF
            l-lowercase eg : abcdef
            n-number    eg : 1234567890
            x-alphanumeric  eg : all numbers and letters (any case)
            p-printable eg : Printable chrs plus Space
            s-space     eg : space, tab, CR, NL or FF
            b-binary    eg : anything !
            c-control   eg : chr is < octal 040 or >= 0177
            z-anpa hdr field eg: alnum plus punctuation (NOT space)
            t-punctuation   eg : .,<>!"#$%^&*()_+-=[]{}:;@;
            Note the actual definitions of what is chrs are defined
            for each of these types is defined by SunOS/AIX using
            the 'locale'. Use the 'man' pages for more information.
        'startString' is the (optional) start string for the field.
        'endString' is the (optional) end string for the field.

An example using an IPTC header :
Input file data will look something like :
    (soh) PAL1234 (spc) 3 (spc) HHH (spc) 104 (spc) PA ADVISORY (cr) (nl)
    PRESS Advisory re First editions (cr) (nl)
    (stx)
        text
    (eot)
Parameter file:
    ; Vwire parameter file for PA
    soh:    \001
    stx:    \002
    eot:    \003
    line:1  S:3:a N:4:n *:0:s P:1:n *:0:s C:3:x *:0:s w:0:n *:0:s R:0:p
    line:2  k:0:p
All other fields/lines etc left as defaults

    This will save the following fields for later use :
        XS - service designator     = PAL
        XN - service seq number     = 1234
        XP - priority           = 3
        XC - category code      = HHH
        XW - no of words        = 104
        XR - reference field        = PA ADVISORY
        XK - keyword            = PRESS Advisory re etc

Note that the default line separator is CR or NL in whatever chrset is used.

Style 2 Attibute style using 'attribute-style' and 'att-fiphdr'
-------------------------------------------------------------

Specify 'attibute-style' once in the parameter file and a 'att-fiphdr' for each
line you wish to save.

    ; style is one line per parameter ending with some combination of CR and/or NL
    ; the default style is to have spaces only between attributes
    attribute-Style:\n
    att-fiphdr:(Incoming Header name):(FipHdr)
    att-fiphdr:cat:C
    att-fiphdr:headline:K

So Headline will be in the SH (SourceHeader) field K and accessible using \XK

Using DEBUG
-----------

The DEBUG: keyword enables you to finetune the parameter file. If enabled, it
creates a file in 'spool/wiretest' with the following type of information :

--- New File at Tue Apr 18 13:27:28 ---

--- Parse the Header ...

--- No STX and hdrlines=none : NoHeader ...

--- FileName:WETH002|

--- Output the Header ...
SU:prweth
ST:vwire
SP:rack0-ts1,1105
SN:WETH002
SC:ascii
DF:vwire
ZI:

--- Output the Text ...
Some test data

Turn DEBUG off and on by adding/commenting out 'debug:' in the parameter file
and stopping/starting the relevant VWIRE using 'ip' the user interface.

For Logons, The strings are in uucp-like format where the syntax is :

    (send1string) (space) (receive1string) (spc) (s2) (spc) (r2) etc
ie you send1, wait 5secs for r1, send s2 wait 5secs for r2 etc.

eg: logon:~"" Userid: ~root\r Password? ~bongos\r LanRoverE_FIP# ~"connect
port5\r" options

This mimics the following sequence :
@ Userid: root
Password?

Shiva LanRover/E PLUS, Version 5.7 98/11/06
LanRoverE_JV1# connect port5
Connecting to Serial5 at 9600 BPS.
Escape character is CTRL-^ (30).

Type the escape character followed by C to get back,
or followed by ? to see other options.

To send a space, use \s or dbl quote that string.
To wait a second - send a tilde (or redefine to your Wait chr)
All keywords - setup, logon etc -  are optional - if one does not exist, it is
i
gnored.
Embedded spaces may be specified by enclosing within double quotes.

Null fields can be specified by two consequetive double quotes - "". If a wait
i
s required howver ALWAYS SPECIFY BEFORE the dbl quotes eg: ~~~"" for a 3 second
wait with nothing returned.

(copyright) 2017 and previous years FingerPost Ltd.