iphdrchk

iphdrchk

This program checks the headers of files for syntax and content.

It is normally used as part of the News Agency Broadcast module to filter badly
formatted files and act accordingly - for example send message back to the
sender, add or alter data or to ignore/spike the file.

These checks can be made on any combination of FIP header fields and/or text
strings in the data.

The parameter files holding the filters/tests are in tables/hdrchk. There is a
file per Source (FIP header field SU) unless overridden by the contents of the
FIP header field SK.

If there is no SU or SK field the default parameter file is
tables/hdrchk/HDRCHK.

The format of each parameter file is :

    ; comment
    dest:(DU) source:(SU) if:(test) and:(test) pass:(flag) fail:(flag) (eoln)
    default:        (action...) (eoln)
    flag: (number)      (action...) (eoln)
    notflag: (number)   (action...) (eoln)
    tofield: (new FIPhdr field for the DU)  (eoln)
    name: (replacement name for file : replaces SN) (eoln)
    wild: (single chr)          (eoln)
    singlewild: (single chr)        (eoln)
    table: (tablename)  (list of table entries ..)  (eoln)
    reptable: (tablename)   (list of table entries ..)  (eoln)

where :

dest: is the contents of the destination or DU field. Normally this will be a
single destination but multiples are valid if the whole DU is specified as in
the field. Ie if the incoming DU contains 'aline+bline+CLINE' then specify :
    dest:aline+bline+CLINE  (etc ...)
with NO embedded spaces.  This keyword MUST start the line.

Subparameters for 'dest:' include :
-----   source: the the source or SU field. It is optional.
-----   if: test condition syntax is:
        if: (FIP Header field) = (condition)
    or  if: (FIP Header field) != (condition)
       The operation equal, '=', can also be NOT equal '!='.
       The FIP header field can be any valid field. Source Header fields
        (in SH) are preceeded by X, ie XC for category. It can also
        be a new sub field using partial:, option: or combie:
       The condition is any FIP sequence string (ie text, octal chrs, unix
        escape chars) plus wild cards using '*' for a string and '?'
        is single wild card chr. To search for a string/chr embedded
        somewhere in a field, uses a '*' before and after.
        Note that the condition string is case_insensitive.
       To test for a null field or one containing spaces only, use double
        quotes. eg: if:TY=""
       However, use the 'no:' keyword to test whether a field exists or not.
-----   and: A second (optional) test condition using the same syntax as 'if'.
-----   no: Valid if there is NO field of that name in the FIP Header. Syntax: 
            no: (FIP Header field)
-----   search: search a predefined table for this Header field. syntax is :
        search: (FIP Header field) = (table name)
    or  search: (FIP Header field) != (table name)
        where table name is that defined by a 'table:' line.
-----   pass:  If the test(s) succeed 'pass' sets a flag number which can be in
        the range of 1 to 200.
-----   fail: If the test(s) fail, 'fail' sets a flag. The ok and error flags
        are the same and it is for the User to decide where to use
        which flag.
There should be either or both 'pass' and/or 'fail'. If both are specified,
take care to make sure the flags set are different !
    Valid flags are between 1 and 200.

flag: uses as parameter a flag number that corresponds to that set by the
'pass'
    subkeyword in one or more 'dest' lines. Its subparameters describe what
    kind of action is required for this case.
notflag: is the same as 'flag' but for the 'fail'ures.
    These keywords MUST start the line.

Subparameters for 'flag' and 'notflag' are the action parameters which are a
series of optional keyword:parameters separated by spaces or tabs:
-----   newdest: (destinations) replace contents of DU with these.
        This allows hdrchk to act as a Group sender.
-----   addhdr: (new FIP header fields) add/replace FIP header fields.
        The parameter can be a FIP Seq string including octal numbers,
        System variables etc which is checked at runtime for each file.
-----   msg: (destination):(message)    send the Message in a file to that
                destination (DU). This creates a file called
                ERddhhmm for day, hour, minute with a SC of 
                'ascii'. If the message is not specified, the
                default message is sent.
-----   script: (scriptname)    run this script.
-----   log: (string)       log this string in the Item Log file.
-----   action: (action)    action can be 'delete' - delete the file
                'ok' - pass the file thru ignoring errors
                'woops' - send the file to the woops Que

You may also define a default action if a file meets NONE of the criteria:
    default: (default action if no flags are set at all)
        where the action is any or all of the actions above.
A default is advised.

The table keyword is specified by :
    table: (tablename)  (list of table entries ..)  (eoln)
The list of table entries are a number of entries delimited by spaces and/or
tabs. If embedded spaces/tabs are needed, double quote the entry. The condition
is considered passed (or failed if the "!=" is used) if at least one of the
entries is matched. 
The reptable keyword is specified by :
    reptable: (tablename)   (list of table entries ..)  (eoln)
This will create a new header field "H?" where ? is 1,2,3 etc for the 1st
reptable specified etc.
Each entry has the format
    (key),(replacement string)
    eg  reptable:boat   oar,"yes pls"     rowlocks,"no thanks"
For readability of the parameter file, for 'table' and 'reptable', you can
either specify all the entries on one line or have multiple lines with the same
table name:
    reptable:category   a,1 b,2 c,3 d,4 e,5 f,6
or  reptable:category   a,1
    reptable:category   b,2
    reptable:category   c,3
    reptable:category   d,4
    reptable:category   e,5
    reptable:category   f,6

If the FIP header field containing the destination is NOT the 'DU' field, the
name of the field can be specified using :
    tofield:XX where XX is the 2 letter header field to be used.
        This can NOT be a Source Header field.
eg  tofield:QD

If the contents of the 'SN' FIP header field containing the filename is NOT
that required, it can be replaced using the 'name' keyword. The parameter to
'name' can be any FIP sequence of fixed, text, FIP header fields, unix escape
chrs and octal chrs.
    name: (FIPseq)  (eoln)
eg  name:\$d\SNoink
In this example, a file whose previous SN was 'PIGGY123' which is run on 3rd
April would have a new name of '03PIGGY123oink'

For the tests, wild card strings and single characters are normally '*' and '?'
respectively. These can be changed by the 'wild:' and 'singlewild:' keywords.
The parameter to these is a single printable character. eg :
    wild:%
The following characters should NOT be used as they have other syntactical
uses:
 colon, hash, equal, semi-colon, double quotes, exclamation.

Where sections of FipHdr fields are required or changes to the output style,
use keywords : fixed, partial, combie, optional, repeat, newdate and/or style.
(see The SysAdmin manual for more information).

    They are normally specified :
        fixed:QZ    1234543
        partial:QT  ST,3,2,U,<,>
        combie:QY   ep|na,(0000000)a 
        option:QE   ep,11,7,s
        repeat:QK   XK,-,3  
    or  repeat:QP   PK,,4,#X
        style:QS    XN,%.03d 

Example of a parameter file :
;
;   Hdr Check file
;
; allowable parameters are
default: action:woops  log:"Rubbish file received - woops : \SN from \SU"
;
dest:xxx    if:XC!=AAA  and:XC!=AAB pass:33 fail:47
dest:reuspo source:reuter   if:XC=S* and:XK=bc-sport-oly*   pass:5  fail:10
dest:reu    source:reuter   if:XC!=S*   pass:29 fail:255
;
flag:5  action:ok   log:Ok\s5
flag:10 action:woops    log:NO\s10
flag:5  newdest:yyy+aaa addhdr:#QD:0101#ZI: msg:hdrerr

Note there can be multiple flags all adding Fip Hdr fields.
Example :
dest:bout   if:PR=1  pass:1  
dest:bout   if:PR=10 pass:10  

dest:bout   if:TX=x pass:98
dest:bout   if:TX=y pass:99

flag:98 action:ok   addhdr:#XX:\$Z  log:"This one X"
flag:99 action:ok   addhdr:#YY:\$Z  log:"This one Y"
;
flag:1  addhdr:#PP:1
flag:10 addhdr:#PP:10

A file such as :
    #sn:no1#pr:1#du:bout#tx:x
will become
    #sn:no1#pr:1#du:bout#tx:x##XX:0006#PP:1#SN:no1

Input parameters are :
    Mandatory :
    -i : input queue to scan    default: spool/hdrchk
    -o : output queue       default: spool/2go
    Optional
    -D : ignore any Delay times default: look for the DE field
    -f : file creep time for files arriving over the network
        This is usually around 5 seconds for RCP files. default: none
    -t : scan time for directory    default: 5 secs
    -z : name of selection file default: tables/hdrchk/HDRCHK
    -l : log every incoming file    default: do NOT log
    -Q : NOT quiet - complain (log) if the parameter file does not
        exist. Normally if the file does not exist, the default file is.
                    default: no log and use default file
    -v : display version number and exit.

(copyright) 2017 and previous years FingerPost Ltd.