ipwheel

ipwheel

IPWHEEL finds where our illustrious file is to go. It scans queue 2go for
incoming files.

For each new file, it reads in the FIP header (if there is one) and checks for
the destination (DU)- which may be multiple DUs. For each DU, the USERS file is
scanned for other information - DP, DQ, DC etc. If there is no valid entry for
the DU in USERS, the file is woops'ed to the Intercept queue.
Syntax of the USERS file is :

    ; comment

    (destination)= (tab) (fip-hdr1):(data1) (tab) (hdr2):(data2) ... (EOL)
where   destination is the DU field
    fip-hdr is the header field or flag
    data is the optional data for that field/flag
For example
    ; USERS file
    racing=     DP:edsys3   DQ:racing-spo   DC:j11
    sunrac=     DP:rs6000   DQ:save_racing  DC:ibm
    indrac=     DP:adobesvr DQ:Racing.Desk  DC:indesign

If the file is a binary file (FBIN) or the source and destination chrsets are
the same (hdr field SC = DC), then the file is linked directly to the output
queuefor that destination. Otherwise it is sent via IPXCHG with the CQ field
filled with the queue of the next stage.

IPWHEEL also checks for redundancy. If it is running on the primary Unixbox for
that service, it processes the first file and junks the rest. If it running on
the secondary, the file is not processed unless the primary system is down.

The fun bit (?) is where you want to override Source Redundancy with
Destination Redundancy! Confused ? So am I and I wrote it. If a file has an SZ
field (usually added by IPROUTE from the incoming Wire routing file), normally
the 1st system specified is the Primary and the Second is the secondary.
BUT you may want to serve one network from one UnixBox and another group of
destinations from another UnixBox ad infinitum. To do so, use the DP_REDUN file
in tables/sys to override the primary/secondary pair for that destination.
Works a treat! Syntax is :
    ; for comment
    EdsysName (tab/spc) PrimaryUnixHostname (spcs) SecHost (eoln)
    where Edsysname is the SAME as the DP in USERS
        and should be specified WITHOUT any system number
Syntax of the tables/sys/DEST_REDUN file is (remember this file is OPTIONAL) :
    ; for comment
    EdsysName (tab/spc) PrimaryUnixHostname (spcs) SecHost (eoln)
    where Edsysname is the SAME as the DP in USERS
        and should be specified WITHOUT any system number
 For example
    ; DEST_REDUN file
    atexedit    fipA    fipB
    ccisender   fipC    fipD
    opi1        fip99.bigpaper.co.zz    fip66.bigpaper.co.zz

Localhost is not a valid hostname in DEST_REDUN. Please take care to enter ALL
your editorial system hosts in DEST_REDUN if it is being used.

An optional parameter file may be specified using the '-E' input switch. Valid
parameters are
    - the usual parse parameters - partial, option, repeat, sum etc
    - folder-size: (name of folder under spool) max: (maxsize)
        If a file whose size if bigger than 'maxsize' arrives for the output folder
(fiphdr field DQ) specified, then it is dropped into (foldername)_big and NOT
(foldername)
        There can be 50 such folders specified.
    - file-trace: (valid actions - yes/no, + hdr + in + out + (delete is ignored
in ipwheel))
      file-trace-name: (UniqueId in FipSeq if not default -  \SU-\SN-\HR)
      file-trace-log: (extra File Tracing logging in FipSeq)
        Trace a file thru fip.
        Actions are 'Yes' - log, 'No' - dont; if NOT NO, then in addition to logging,
we can save a copy of the FipHdr, Input file and/or Output
        The Hdr and files are left in /fip/log/file_trace (which should be purged in
the nightly maintenance zapfiplog)
    These can be overridden by FipHdr fields in each file if a particular file -
or service needs tracing.
        (notice the '_' if using in the FipHdr but '-' if parameters)
    eg  ZFILE_TRACE: hdr
        ZFILE_TRACE_NAME: louislouis
        ZFILE_TRACE_LOG: seqno.\ZQ

If the ZI hdr flag is set, a copy of the file is saved in the archive log
unless either it is a redundant file or the XZ hdr flag is on signifying a
resend. The name of the archive file is in two parts :
    1 - A day-of-the-year number which is found either from the HS Fiphdr field or
defaults to the current DOY.
    2 - the contents of the SU FipHdr field (ie the Source)
        this will be overridden if there is an SL FipHdr.

If the destination is flagged as 'delete' the file is zapped and no processing
- not even archive - is done.  If the destination is flagged as 'logdel' the
file is zapped and no processing except archive is done.

Input options (all optional) :
    -q or -i : queue to scan            default: spool/2go
    -x : queue for xchg             default: spool/xchg
    -n : queue for 2net             default: spool/2net
    -t : scan time                  default: 1 sec
    -m : multi redundant files. For redundant incoming services, the normal
        arbitration is between two files. This switch is used where
        more than 2 are expected.
            default: allow the first file of every two.
    -h : hostname for multiple ethernets        default: (hostname)
        Name of this Unixbox if different from the reboot name.
    -B : do NOT read the FipHdr-in-text for FBIN files  default: do
    -c : make copy (link) in spool/2taste if valid file. default: no
        but ignore all traffic sent to 'logdel' or 'delete'
        Only Archived traffic that is NOT a resend is sent.
        the parameters can be one of the following
            S-saved data only (no logdel or delete)
            Z-deleted copy too
            X-all data (except logdel or delete)
    -C : filter traffic to spool/2taste default: no
        Only traffic with this filter (or nothing) in FipHdr ZWHEEL_FILTER is passed
    -d : check for changes to DEST_REDUN        default: no
        the default is to read once on startup
    -D : pass everything - including 'logdel' and 'delete'. def:no
    -e : Stub of the name of the editorial system   default: atex
        Wheel checks if the DP starts with this.
        eg -e mac   for editorial systems called macone, mactwo etc
        Files are slotted into 2macone, 2mactwo etc
        This will override the env variable FIP_def_EDSYS.
    -E : name of an optional parameter file in sys          default: none
    -F : FipHdr field for RoundRobin to be added to output file default: none
        (see -X input switch below)
    -G : WINNT/2k - grab the first file only    default: no
    -H : log all the hosts/folders for each file    default: no
        Use this to trace where traffic is going !
    -p : BreakIn Priority For HiPriority News   default: 1
        Agency Bdcast traffic.
    -P : FipHdr Field for High-Priority News    default: PR
        Agency Bdcast traffic
    -Q : only send one instance of the file to any one  default: send all
        local queue In this case only, the full 'DU' field
        is copied to 'ZU'. This works for files for this
        Host ie DP:localhost or DP is this host name
    -r : For DEST_REDUN - do not resolve any hostnames in DP: SZ: or DEST_REDUN
                             default resolve all names
    -R : For DEST_REDUN - do NOT use DNS to resolve names before checking
        DEST_REDUN. For this all the DEST_REDUN entries MUST be the
        true hostnames.              default resolve all names
    -S : (size) if a file is bigger than this size and needs to be
        'xchg'ed, it is sent to spool/xchg_big NOT spool/xchg   default: spool/xchg
    -T : display timing stats for this stage
    -u : track DUs for uniqueness - only pass one copy of   default: do NOT track
        a file for each destination. Use this where a file MAY be sent
        to the same destination (DU) but only one copy is required.
    -U : add UNO field as the HR FipHdr field to ALL files.
        For Unisys : -U 0 = 0 -> 255; -U 1 = 0 -> 127; -U 2 = 128 -> 255
        For AFX Unos use -U 3   There is NO default.
        -U 4 is time-seqno
        -U 5 is time-ipaddress-seqno
        -U 6 is time-ipaddress-seqno with 6 digit seqno
        -U 7 is for timing stats
            (add -T to display timing stats for this stage)
        The Uno is ONLY added if the HR field does NOT exist.
    -W : 10 ths of a second - MUST wait if nothing processed    default: 0
    -X ; send to multiple xchg queues in round-robin order      default: no
        -X 3 will send to xchg1, xchg2 and xchg3
            (and xchg_big1 xchg_big2 etc if using -S big filter)
        maximum value is 29 (9 up to version 263o7)
        -X4 -F RR will send to 4 folders and add RR:n to the FipHdr
    -y : 2nd or supplementary USERS file        default: none
        this is used in addition to the main USERS file
    -z : name of USERS file in tables/sys       default: USERS
    -Z : never archive anything - ignore ZI,ZO,ZX   default: archive
    -9 : do NOT run in Speedy mode           default: no
    -v : version and exit

How does redundancy work here ... good question.- see the web pages.

Note that the editorial system name is significant. Please do not mix Fip
Logical hostnames - like 'edone', 'edtwo' with proper TCP hosts ie do NOT call
a space Sparc 'edthree'.

(copyright) 2017 and previous years FingerPost Ltd.