Keywords for MOS parameter file are :
    ; comment line
    fiphdr:(2letterCode)    tag:(tagname)   multi:no/yes/KEY-storyID
        see 'FIPHDR' section below

    process-mosromgr: (yes/no/only)
        yes - process both old fip style AND mosromgr style data
        only    - ONLY process mosromgr sty;e
        no  - ignore mosromgr (default)

    mosromgr-script: (script)
        script to generate FINAL or INCOMPLETE mosromgr feeds
        (NOTE - use 'rundown-extra-fiphdr' to add extra FipHdr info to add to the
output file - normally it would include a DU:(dest) for sys/USERS eg
        ; RUNDOWN/TRIGGER only ...

    outque:  Output Queue for the output file.
            This default to the '-o' input switch which defaults to spool/2go.
            If the first chr is NOT a '/', it is assumed under spool.
            If there is an 'outque' and an input switch '-o', the -o value is used.
            UNLESS the -V switch is on were -o is used over outque.
    doneque:    Done Queue for the raw input file.
            This default to the '-d' input switch which has no default.
            If the first chr is NOT a '/', it is assumed under spool.

    newname-trigger: (FipSeq)
        name of a new trigger output file

    before: (FipSeq)    String to parse and add at the top of the file.
    after: (FipSeq)  String to parse and add at the end of the file.
    beffile: (Path/filename) Contents of a file in FipSeq to parse and add at the
                top of the file (after 'before')
    aftfile: (Path/filename) Contents of a file in FipSeq to parse and add at the
                bottom of the file (before 'after')
    script: (path and name) Script to run AFTER processing.         default: none
            The output filename and path is added to the script before running.
    log:    Custom log line for the Fip Item log in FipSeq
        default is name of the parameter file (DF) and filename (SN)

    balance-data:  Send data this Balance Group (see 'ipbalan')
        Use this to make sure any internal data and sequence numbers are always
updated on any companion systems.

    hash-in-fiphdr: (FipSeq chr)            default:\035
        A hash/pound (#) in a FipHdr field is mapped to this chr
        Normally hashes are end-of-field in a FipHdr. So it needs to be mapped to
something else and we changed it here on outbound.

    normalise-time: (no, 2digit mins)
        if set, any time that is NOT within this number of mins is rounded to the
        the increment is by 10 for 10 mins, so normalise-time:10
            roEdStart=...20:55:01 will round UP to 2100
            roEdStart=...20:54:59 will round DOWN to 2050
        do not use for any time which may trigger over midnight!
        default: do not normalise/change the release time
    output-blank-rundowns: (yes/no)
        if a new rundown (ie roCreate or roReplace) does NOT have anything in, create
a blank output.
        default: no

    min-output-gap: (no of secs)
        minimum gap between MosRoMgr rundown outputs        default: 29 secs

    check-primary-server: pseudo-host name that is specified in
        Valid for -Z TEXT and -Z TRIGGER options only
        Use this when ipmos is running duplicated on more than 1 server but we ONLY
want a single output feed.
        ie in the parameter file MOS_TEXT is
        and in the DEST_REDUN is
            ; psuedohost    primary secondary
            mossvr      fip1    fip2    fip3    fip4
        and in the SYSTEM file for all 4 fips, there is a line
            mos1    local   ipmos -Z TEXT
        Then if fip1 is up, the ipmos on fip1 will always send while on fip2 etc it
will just check/loop.
            if fip1 is down, the ipmos on fip2 will automatically start sending in its

    mos-environment: (2 letter FipHdr)
        This is the FipHdr field which holds the environment - eg enps_live,
        If the -E input switch is on, then whatever is in that will overwrite any
metadata in the file with the same FH.
        It is used to separate the different feeds from different environments

    ncs-id: (ncs)
        all the NCSids of incoming file to match if shortening the ID.  default
        one ncsID per line :

    zap-words: (word)
        chaff words in the ncsID that can be ignored
        There can be multiple 'zap-words' entries

    ; ignoring some feeds - if the fiphdr contents matches any one of the list of
    channel-fiphdr:(FipHdr field)
    ignore-channel:(channel name)
        There can be multiple 'ignore-channel' entries

For -Z TRIGGER, up to 10 outputs can be specified
    To tune these, there are a series of parameters - where ? is the output key
from 0-9
    output-?-valid-rundown: (FipSeq)
        Test for a valid rundown eg: \TN
        If there is NO data returned, that output is ignored
    output-?-DU: (FipSeq)
        A valid distinations to send this particulat output to.
        If there is NO data returned, that output is ignored
    output-?-extrafh: (FipSeq)
        Extra FipHdr info to add
    output-?-top: (filename of a file in tables/setup)
    output-?-tail: (filename of a file in tables/setup)
        Name of the template (in FipSeq) holding the Rundown Top and Tail
    output-?-item-top: (filename of a file in tables/setup)
    output-?-item-tail: (filename of a file in tables/setup)
        Name of the template (in FipSeq) holding the Item (ie each element) Top and
    output-?-item-format: (optional format script to run)
        Name of a script/runline to process the item data
        Note that temp FipHdr fields are created for E3-output file and E4-input file
        eg output-1-item-format:/fip/bin/ipxchg -1\E4 -N\E3
    output-?-item-abstract: (Size)
        Size of the abstract in bytes - default is 0 for all data
    output-?-overflow (String in FipSeq)
    output-?-overflow-end (String in FipSeq)
        Optional flag Strings for start and end of of flagging data AFTER abstract
length and AFTER all data
            output-1-overflow:<span id="roMoreTxt\N0" class="dispNone">\s
    output-?-item-link  (filename of a file in tables/setup)
    output-?-item-link-end (filename of a file in tables/setup)
        Xpath/Link items (ie placements in output file refering to the actual data)

Where sections of FipHdr fields are required or changes to the output style,
use keywords : fixed, partial, combie, optional, repeat, newdate and/or style.
(see The SysAdmin manual for more information).

    They are normally specified :
        fixed:QZ    1234543
        partial:QT  ST,3,2,U,<,>
        combie:QY   ep|na,(0000000)a
        option:QE   ep,11,7,s
        repeat:QK   XK,-,3
    or  repeat:QP   PK,,4,#X
        style:QS    XN,%.03d

FipHdrs :
    These are hard coded and do not need to be added
        MA  mosID
        MB  ncsID
        MC  messageID
        MD  roID
        ME  roSlug
        MF  roChannel
        MG  roEdStart
        MH  roEdDur
        MI  mosExternalMetadata
        MJ  mosScope
        MK  Kwmos
        ML  StartTime
        MM  ProgramName
        MN+MO   storyID - if more than 1, MO always has current 2nd
        MP+MQ   storySlug - if more than 1, MQ always has current 2nd
        MR  Approved
        MS  Actual (ENPS) or StoryDuration (OM) (Duration in a story)
        MT  ReadTime (Duration in a story) if no Actual
        MU  ENPSitemType

    from route/MOSWIRE
        MX  UID of mosAction roID/storyID hash
        MY  hash on contents

    These are generated in ipmos for -Z TRIGGER only :
        M0  output id 1-9 - as in 3 in rundown-3-tail:
        M1  3 letter day of week (from StartTime if it exists) - SUN
        M2  full day of week (from StartTime if it exists) - Sunday
        M3  4 digit time (from StartTime if it exists) - 1900
        M4  RunDown name (if Kwmos and Starttime exists) - WSAFRF
        M5  Full name (if Kwmos and Starttime exists) - WSAFRFSUN1900
        M6  UTC Date (if Kwmos and Starttime exists) - 20160431
        M7  Full name of Month
        M8  Year
        M9  Day of month - leading space (no longer has th or st after)
    Plus these :
        M0  storySend status : INSERT or UPDATE (for -Z TEXT only)
        N*  a-z/0-9 fields in list
            eg NF - item sequence/Id (8 hex digit id) for News <file>\NF</file>
        N0  Line seqno (Trigger only)
        N1  Start time (Trigger only)
        N2  Duration (Trigger only)
        N3  Transmission or Output Seqno (Text only)
        N4  overflow data '+' or blank (Trigger only)
        RX  roID - might be/possibly shortened

        RQ  Rundown Date
        RN  for the output filename (Text: dec seqno, Trigger: Rundown name)
        RO  short roID used(not used) in TRG for MosRoMgr only
        RI  for the output filename - for text it is 2 hex numbers
        RU  mos environment
    This is generated in moswire :
        S2  mosAction
Input Parameters are (all optional) :
    -1 : path/filename for single shot      default: spooled
        The input file is NOT deleted
        If this does NOT start with a '/', it is assumed relative to the current
    -d : done queue for original raw data       default: none-input deleted
        this can be in fipseq
    -D : display tags               default: no
        use this ONLY when running '-1' single shot to
        display all tags, attributes and levels and their data.
        ie use to debug/tune.
    -i : input queue
        If this does NOT start with a '/', it is assumed under spool.
            if flag -Z RUNDOWNS spool/xmos_rundowns
            if flag -Z TEXT     spool/xmos_text
            if flag -Z TRIGGER  /fip/data/mos/trigger or /fip/data/mosromgr/trigger if -m
    -l : do NOT log files in            default: log
    -mosromgr : run in mosromgr style only      default: old fip style
    -o : output queue               default: spool/2go
        If this does NOT start with a '/', it is assumed under spool.
    -T : (scan time) watch for Trigger to output (-Z trigger only)  default: 30
    -w : file wait for files arriving across a network. def: none
    -X : re process all data for each element   default: process on change
    -z : default parameter file in tables/setup default: tables/setup/MOS_(-Z
    -Z : type of processing - RUNDOWNS, TEXT, TRIGGER   default: rundowns
    -v : print version number and exit


-- Issue of duplicates

Often data may be 'ipmossed' twice - once for Text and once for Rundowns. While
there are usually different ROs, some data may be required for both.

Or perhaps the same mos elements are received on multiple systems and need to
be dedupped.

So in the route/MOSWIRE, 2 separate desinations are set but with a means of
putting a unique key in the MX fiphdr field.
Eg add these to route/MOSWIRE :

; MY: hash on contents
; MX: UID of mosAction roID/storyID hash

; uniqueness-headline
; MY (hash)
; S2 mosAction
; StoryId or RoId
combie:Q6   S9|S3
repeat:Q7   Q6,+,1
fixed:MX    \S2-\Q7-\MY

; add a hash of the data in MY before so we can use it ..
; and put the unique key in MX

; and also get rid of any duplicates in iproute before ipmos
; DEDUP time should be same as redun AND ipmos

--- ---
Version Control
;04e-j  18mar22 -mosromgr ;c 4apr22 added min-output-gap ;d 22apr22 woops
buglette in 2min gap ;efghi 4may22 newname-trigger
    ;j missing FINAL if KWMOS exists as new bits of FipHdr were never being added
;03t    25nov16 zap on roDelete ;d insert. move and replace - allow thru with only
a new storyID
    ;g 4jan17 added ncs-id ;h 1feb17 better FipHdr ;ij 7feb17 files with spec chrs
;klm redid Move and replace
    ;n-o 4apr17 fiphdr xchg and FIP_maxFipHdrSize and roMetaReplace now handles
'name' and 'maps'
    ;p normalise-time added and output blank rundowns
    ;r 11apr19 use roEdStartTime and if not there use the time of the rundown
    ;s 13may21 incr max ROWS to 3000
    ;t 17nov21 in trigger, ignore blank stories starting 'Story 99'

;02z    14apr16 3 flavours -Z rundowns trigger text ;b date issue for XCHG in
trigger ;cd 23may16 added Text-outputSeqno
    ;e-f 5jun16 check for duplicates and check ModTime
    ;h-j added check-primary-server ;k-p cleanups and bug fix if FipHdr too large
    ;q-s cleanups and added test-slug
    ;t added M0 for Text new0/1=INSERT/UPDATE
    ;u added environment-fiphdr
    ;v 27oct16 bugette in abs/chunks
    ;w  2nov16 added order of -111 for storyDelete so it can be scanned for ....
    ;x  8nov16 track mosEnv as incoming FipHdr too (use -E as default) and added
-h extraFHin
    ;y 12nov16 mods for OM (actually de-ENPSed)
    ;z 15nov16 del is now -666, MOVE/INS/REPL but unable to place is -222, story
now gets all FH, added MS-StoryDuration

;01m    20nov12 original version/30apr14 ;j cleanup ;k News/xml/txt added ;l-m
added funny bbc text

(copyright) 2024 and previous years FingerPost Ltd.