ipmos
ipmos
Keywords for MOS parameter file are :
; comment line
fiphdr:(2letterCode) tag:(tagname) multi:no/yes/KEY-storyID
see 'FIPHDR' section below
process-mosromgr: (yes/no/only)
yes - process both old fip style AND mosromgr style data
only - ONLY process mosromgr sty;e
no - ignore mosromgr (default)
mosromgr-script: (script)
script to generate FINAL or INCOMPLETE mosromgr feeds
(NOTE - use 'rundown-extra-fiphdr' to add extra FipHdr info to add to the
output file - normally it would include a DU:(dest) for sys/USERS eg
; RUNDOWN/TRIGGER only ...
rundown-extra-fiphdr:\nDU:mosromgr2uced+mosromgr2wsp\n
outque: Output Queue for the output file.
This default to the '-o' input switch which defaults to spool/2go.
If the first chr is NOT a '/', it is assumed under spool.
If there is an 'outque' and an input switch '-o', the -o value is used.
UNLESS the -V switch is on were -o is used over outque.
doneque: Done Queue for the raw input file.
This default to the '-d' input switch which has no default.
If the first chr is NOT a '/', it is assumed under spool.
newname-trigger: (FipSeq)
name of a new trigger output file
before: (FipSeq) String to parse and add at the top of the file.
after: (FipSeq) String to parse and add at the end of the file.
beffile: (Path/filename) Contents of a file in FipSeq to parse and add at the
top of the file (after 'before')
aftfile: (Path/filename) Contents of a file in FipSeq to parse and add at the
bottom of the file (before 'after')
script: (path and name) Script to run AFTER processing. default: none
The output filename and path is added to the script before running.
log: Custom log line for the Fip Item log in FipSeq
default is name of the parameter file (DF) and filename (SN)
balance-data: Send data this Balance Group (see 'ipbalan')
Use this to make sure any internal data and sequence numbers are always
updated on any companion systems.
hash-in-fiphdr: (FipSeq chr) default:\035
A hash/pound (#) in a FipHdr field is mapped to this chr
Normally hashes are end-of-field in a FipHdr. So it needs to be mapped to
something else and we changed it here on outbound.
normalise-time: (no, 2digit mins)
if set, any time that is NOT within this number of mins is rounded to the
nearest
the increment is by 10 for 10 mins, so normalise-time:10
roEdStart=...20:55:01 will round UP to 2100
roEdStart=...20:54:59 will round DOWN to 2050
do not use for any time which may trigger over midnight!
default: do not normalise/change the release time
output-blank-rundowns: (yes/no)
if a new rundown (ie roCreate or roReplace) does NOT have anything in, create
a blank output.
default: no
min-output-gap: (no of secs)
minimum gap between MosRoMgr rundown outputs default: 29 secs
check-primary-server: pseudo-host name that is specified in
tables/sys/DEST_REDUN
Valid for -Z TEXT and -Z TRIGGER options only
Use this when ipmos is running duplicated on more than 1 server but we ONLY
want a single output feed.
ie in the parameter file MOS_TEXT is
check-primary-server:mossvr
and in the DEST_REDUN is
; psuedohost primary secondary
mossvr fip1 fip2 fip3 fip4
and in the SYSTEM file for all 4 fips, there is a line
mos1 local ipmos -Z TEXT
Then if fip1 is up, the ipmos on fip1 will always send while on fip2 etc it
will just check/loop.
if fip1 is down, the ipmos on fip2 will automatically start sending in its
place
mos-environment: (2 letter FipHdr)
This is the FipHdr field which holds the environment - eg enps_live,
om_staging
If the -E input switch is on, then whatever is in that will overwrite any
metadata in the file with the same FH.
It is used to separate the different feeds from different environments
ncs-id: (ncs)
all the NCSids of incoming file to match if shortening the ID. default
(ignored)
one ncsID per line :
ncs-id:NCS15
ncs-id:NCS17
ncs-id:DRNCS17
zap-words: (word)
chaff words in the ncsID that can be ignored
There can be multiple 'zap-words' entries
zap-words:NCS
zap-words:MOS
zap-words:MOS
; ignoring some feeds - if the fiphdr contents matches any one of the list of
ignore-channels
channel-fiphdr:(FipHdr field)
channel-fiphdr:AG
ignore-channel:(channel name)
There can be multiple 'ignore-channel' entries
ignore-channel:POLITICS
For -Z TRIGGER, up to 10 outputs can be specified
To tune these, there are a series of parameters - where ? is the output key
from 0-9
output-?-valid-rundown: (FipSeq)
Test for a valid rundown eg: \TN
If there is NO data returned, that output is ignored
output-?-DU: (FipSeq)
A valid distinations to send this particulat output to.
If there is NO data returned, that output is ignored
output-?-extrafh: (FipSeq)
Extra FipHdr info to add
output-?-top: (filename of a file in tables/setup)
output-?-tail: (filename of a file in tables/setup)
Name of the template (in FipSeq) holding the Rundown Top and Tail
output-?-item-top: (filename of a file in tables/setup)
output-?-item-tail: (filename of a file in tables/setup)
Name of the template (in FipSeq) holding the Item (ie each element) Top and
Tail
output-?-item-format: (optional format script to run)
Name of a script/runline to process the item data
Note that temp FipHdr fields are created for E3-output file and E4-input file
eg output-1-item-format:/fip/bin/ipxchg -1\E4 -N\E3
-DMOS_RUNDOWN_HTML,MOS_RUNDOWN_HTML2
output-?-item-abstract: (Size)
Size of the abstract in bytes - default is 0 for all data
output-?-overflow (String in FipSeq)
output-?-overflow-end (String in FipSeq)
Optional flag Strings for start and end of of flagging data AFTER abstract
length and AFTER all data
eg
output-1-item-abstract:500
output-1-overflow:<span id="roMoreTxt\N0" class="dispNone">\s
output-1-overflow-end:</span>
output-?-item-link (filename of a file in tables/setup)
output-?-item-link-end (filename of a file in tables/setup)
Xpath/Link items (ie placements in output file refering to the actual data)
Where sections of FipHdr fields are required or changes to the output style,
use keywords : fixed, partial, combie, optional, repeat, newdate and/or style.
(see The SysAdmin manual for more information).
They are normally specified :
fixed:QZ 1234543
partial:QT ST,3,2,U,<,>
combie:QY ep|na,(0000000)a
option:QE ep,11,7,s
repeat:QK XK,-,3
or repeat:QP PK,,4,#X
style:QS XN,%.03d
FipHdrs :
These are hard coded and do not need to be added
MA mosID
MB ncsID
MC messageID
MD roID
ME roSlug
MF roChannel
MG roEdStart
MH roEdDur
MI mosExternalMetadata
MJ mosScope
MK Kwmos
ML StartTime
MM ProgramName
MN+MO storyID - if more than 1, MO always has current 2nd
MP+MQ storySlug - if more than 1, MQ always has current 2nd
MR Approved
MS Actual (ENPS) or StoryDuration (OM) (Duration in a story)
MT ReadTime (Duration in a story) if no Actual
MU ENPSitemType
from route/MOSWIRE
MX UID of mosAction roID/storyID hash
MY hash on contents
MZ TEXT type TVCNEWSWIRE WSWEBNEWS VCSNEWS
These are generated in ipmos for -Z TRIGGER only :
M0 output id 1-9 - as in 3 in rundown-3-tail:
M1 3 letter day of week (from StartTime if it exists) - SUN
M2 full day of week (from StartTime if it exists) - Sunday
M3 4 digit time (from StartTime if it exists) - 1900
M4 RunDown name (if Kwmos and Starttime exists) - WSAFRF
M5 Full name (if Kwmos and Starttime exists) - WSAFRFSUN1900
M6 UTC Date (if Kwmos and Starttime exists) - 20160431
M7 Full name of Month
M8 Year
M9 Day of month - leading space (no longer has th or st after)
Plus these :
M0 storySend status : INSERT or UPDATE (for -Z TEXT only)
N* a-z/0-9 fields in list
eg NF - item sequence/Id (8 hex digit id) for News <file>\NF</file>
N0 Line seqno (Trigger only)
N1 Start time (Trigger only)
N2 Duration (Trigger only)
N3 Transmission or Output Seqno (Text only)
N4 overflow data '+' or blank (Trigger only)
RX roID - might be/possibly shortened
RQ Rundown Date
RN for the output filename (Text: dec seqno, Trigger: Rundown name)
RO short roID used(not used) in TRG for MosRoMgr only
RI for the output filename - for text it is 2 hex numbers
RU mos environment
This is generated in moswire :
S2 mosAction
Input Parameters are (all optional) :
-1 : path/filename for single shot default: spooled
The input file is NOT deleted
If this does NOT start with a '/', it is assumed relative to the current
path.
-d : done queue for original raw data default: none-input deleted
this can be in fipseq
-D : display tags default: no
use this ONLY when running '-1' single shot to
display all tags, attributes and levels and their data.
ie use to debug/tune.
-i : input queue
If this does NOT start with a '/', it is assumed under spool.
default:
if flag -Z RUNDOWNS spool/xmos_rundowns
if flag -Z TEXT spool/xmos_text
if flag -Z TRIGGER /fip/data/mos/trigger or /fip/data/mosromgr/trigger if -m
set
-l : do NOT log files in default: log
-mosromgr : run in mosromgr style only default: old fip style
-o : output queue default: spool/2go
If this does NOT start with a '/', it is assumed under spool.
-T : (scan time) watch for Trigger to output (-Z trigger only) default: 30
secs
-w : file wait for files arriving across a network. def: none
-X : re process all data for each element default: process on change
-z : default parameter file in tables/setup default: tables/setup/MOS_(-Z
type)
-Z : type of processing - RUNDOWNS, TEXT, TRIGGER default: rundowns
-v : print version number and exit
---NOTES---
-- Issue of duplicates
Often data may be 'ipmossed' twice - once for Text and once for Rundowns. While
there are usually different ROs, some data may be required for both.
Or perhaps the same mos elements are received on multiple systems and need to
be dedupped.
So in the route/MOSWIRE, 2 separate desinations are set but with a means of
putting a unique key in the MX fiphdr field.
Eg add these to route/MOSWIRE :
; MY: hash on contents
; MX: UID of mosAction roID/storyID hash
; uniqueness-headline
; MY (hash)
; S2 mosAction
; StoryId or RoId
combie:Q6 S9|S3
repeat:Q7 Q6,+,1
fixed:MX \S2-\Q7-\MY
; add a hash of the data in MY before so we can use it ..
ADDHDR-BEFORE-ROUTING:MY:\%x\n
; and put the unique key in MX
ADDHDR:MX:\MX\n
; and also get rid of any duplicates in iproute before ipmos
NOLOGDELETE:
DE-DUPLICATE-HEAD:MX
DE-DUPLICATE-DEST:delete
DE-DUPLICATE-DIFF:0
DE-DUPLICATE-MAX:10000
; DEDUP time should be same as redun AND ipmos
DE-DUPLICATE-TIME:10800
--- ---
Version Control
;04e-j 18mar22 -mosromgr ;c 4apr22 added min-output-gap ;d 22apr22 woops
buglette in 2min gap ;efghi 4may22 newname-trigger
;j missing FINAL if KWMOS exists as new bits of FipHdr were never being added
;03t 25nov16 zap on roDelete ;d insert. move and replace - allow thru with only
a new storyID
;g 4jan17 added ncs-id ;h 1feb17 better FipHdr ;ij 7feb17 files with spec chrs
;klm redid Move and replace
;n-o 4apr17 fiphdr xchg and FIP_maxFipHdrSize and roMetaReplace now handles
'name' and 'maps'
;p normalise-time added and output blank rundowns
;r 11apr19 use roEdStartTime and if not there use the time of the rundown
;s 13may21 incr max ROWS to 3000
;t 17nov21 in trigger, ignore blank stories starting 'Story 99'
;02z 14apr16 3 flavours -Z rundowns trigger text ;b date issue for XCHG in
trigger ;cd 23may16 added Text-outputSeqno
;e-f 5jun16 check for duplicates and check ModTime
;h-j added check-primary-server ;k-p cleanups and bug fix if FipHdr too large
;q-s cleanups and added test-slug
;t added M0 for Text new0/1=INSERT/UPDATE
;u added environment-fiphdr
;v 27oct16 bugette in abs/chunks
;w 2nov16 added order of -111 for storyDelete so it can be scanned for ....
;x 8nov16 track mosEnv as incoming FipHdr too (use -E as default) and added
-h extraFHin
;y 12nov16 mods for OM (actually de-ENPSed)
;z 15nov16 del is now -666, MOVE/INS/REPL but unable to place is -222, story
now gets all FH, added MS-StoryDuration
;01m 20nov12 original version/30apr14 ;j cleanup ;k News/xml/txt added ;l-m
added funny bbc text
(copyright) 2024 and previous years FingerPost Ltd.