CHK 734 - filtim v soon ipdelay Files which are embargoed and messages for clients who wish to take advantage of cheaper telecom tariffs are saved in a delay queue until their time of departure is exceeded. This program scans the delay directory once a minute and checks the FIP header field DE which contains the release time. If this time has passed, then the file is moved to spool/2go for program IPWHEEL to distribute normally. The format of the FipHdr DE field is sone of two formats : 1. a long number which matches the unix time function (default) 2. a UTC string with/without timeoffset (if the input switch -UTC is specified) If no DE FipHdr exists, 'CE' is used instead. FipHdr CE can have one of four formats : 1. If the '-R' input switch is specified : CE:ssss where ssss is the number of seconds to delay by from ths moment the file arrived. 2. If the '-r' input switch is specified : CE:hh:mm+d where hh is the hour, mm is the minute and d is no of days relative from now 3. If the '-r' input switch is NOT specified : CE:hh:mm_dd_mm where dd/mm is the day/month 4. If the '-r' input switch is NOT specified and the USADATE environment variable is set : CE:hh:mm_mm_dd where mm/dd is the month/day if parameter usadate is set In CE or DE, random spaces may be inserted (and are ignored) for readability. Any punctuation can be used as separators except HASH and you are advised NOT to used Unix metachrs like '*' etc (except for DE, UTC version where the punctuation BEFORE any timeoffset must be '+' or '-') If there is a parameter file, it is (optionally) called by the '-z' switch and holds just FipSeq - for the -N switch for example plus 'early-warning' messages While the default is to process each and every file, the -N input switch can be used to create and use a unique key in the hold/delay folder. So any new item with the same key will overwrite any older version. Whether the timer continues or is restarted (default) depends on a second switch : -x. If specified, any new item with the same unique filename (-N) will NOT reset the start time With -x also set, the delay time is NOT updated if LONGER than the original. EG -N and -x : if a file with a delay of 5 mins (CE:300) arrives and then a second with the same key arrives at 92 seconds with a CE of 300 or more, then the new version is updated 108 seconds after. But if the second file has a CE:95 it will be output just 3 seconds later. Note that -N uses a 2 letter FipHdr field as the Key and the contents of this FipHdr will be used; it can be an existing FipHdr in all the files or a FipSeq one - ie the result of a combie/fixed etc The optional Early Warning messages are to pre-announce or early warn that a file is about to be released. This is triggered ONLY for those files that have non-empty FipHdr field as specified by the early-warning-fiphdr field early-warning-fiphdr: (FipHdr) early-warning-extra: (FipSeq) Extra FipHdr info to add to any EarlyWarning file - but NOT the main trigger early-warning: (no of secs before release time) (FipHdr and message in FipSeq) There can be a number (up to 20) early-warnings eg early-warning-fiphdr: QP combie:QP PR|XP early-warning:3600 DI:File \SN will be released in one hour early-warning:600 DI:File \SN will be released in 10 mins early-warning-extra: DA:\CA\n eary-warning-script: (script to run) Same as the -S input switch Run the script when outputting the early-warning message check-primary-server: pseudo-host name that is specified in tables/sys/DEST_REDUN Use this when an ipdelay on 2 systems is accessing the same remote folder - in a redundant way. that is used whether the current host should be getting the files or not. ie in the parameter file is check-primary-server:webserver and in the DEST_REDUN is ; psuedohost primary secondary webserver fip1 fip2 and in the SYSTEM file for both fip1 AND fip2 there is a line rem1 local ipdelay Then if fip1 is up, the ipdelay on fip1 will always send while on fip2 it will wait, check at the last moment and NOT send if fip1 is down, the 'ipspool' on fip2 will start getting the files. hostname: (hostname) Hostname to use for this server for using with redundancy balance-group: (group) Name of the Group in sys/BALANCE for sending to ipdelay on the secondary system extra-fiphdr: (FipHdr fields in FipSeq) Add these FipHdr fields/setting to all output files replace-DU:(FipSeq) replace the DU of the output file to this default: same as incoming script: (script to run) Same as the -s switch below Run this script when releasing the file log-line: (FipSeq) replacement log line if input switch '-L' default: output-queue: (FipSeq) outque: (FipSeq) Output folder for files - this can be in FipSeq to vary the output path; if under /fip/spool, you do NOT need to specify the /fip/spool default is spool/2go This overrides the -o input switch combie:QQ AQ,2edsys outque:/fip/spool/\QQ done-queue: (FipSeq) doneque: (FipSeq) Done folder for incoming files - this can be in FipSeq to vary the output path This overrides the -d input switch doneque:/fip/data/raw.data/\$e\$y\$i\$d_delay_done hold-queue: (FipSeq) holdque: (FipSeq) Holding folder for files in delay default uses the name of the input folder and replaces 'delay' with 'hold' (or adds '_hold' if there is none) This overrides the -h input switch error-queue: (FipSeq) errque: (FipSeq) When a script is run at the end of the delay period, files which ERROR are placed in this folder Input parameters are (all optional) : -9 : do NOT run speedy on a speedy system -d : done queue default: input is deleted after sending -D : replacement DU default: same as incoming -f : extra fiphdr info to add to each file default: none this may be overwritten by the 'extra-fiphdr' parameter, if one exists in the parameter file -F : when released from hold, create new FipHdr H* fields default: use any existing fields -G : resolve all times against GMT with the released time/date default: local system time (overwriting any existing fields) default: no -h : holding queue for items to be sent. default: spool/hold_delay this may be overwritten by the 'holdque' parameter, if one exists in the parameter file -i : queue to scan default: spool/delay -l : do NOT log when the file is released. default: log each file when released -L : log incomings too default: log each file when released -M : Maximum no of days you can delay a file default: 90 days -N : 2 letter FipSeq field to create a unique filename in the hold/delay folder. see notes above default: all items are different -o : output queue for sending default: spool/2go if this is 'delete' the file is deleted (usually after running a script). this may be overwritten by the 'outque' parameter, if one exists in the parameter file -q : queue to scan default: spool/delay -i and -q are interchangeable -Q : do not complain if hold file is missing default: complain -r : date in the CE FipHdr is RELATIVE to today default: date is specified in dd/mm format -R : time in the CE FipHdr is number of secs to delay default: date is specified in dd/mm format -s : scriptname default: no script name This will be triggered Before the file is released. Take care not to run for a long time as the file is NOT sent until the script has finished. -T : release file this many seconds BEFORE time. default: 3 -UTC : time in the DE FipHdr field is in UTC format default: DE holds a unix no-of-secs-since-epoch ie 20050928123400 or 20071129T115959+0430 -w : wait in seconds after a file has been released default: 1 second -W : file stable time for NFS or scripted files default: 0 secs -x : any new item with the same unique filename (-N) will NOT reset the start time default: restart delay time -Y : default delay in seconds if there is NO CE or DE default: 0 secs -z : name of optional parameter file default: none -v : print version no and exit The environment variable for USA style dates is setenv FIP_USA_DATE yes The maximum number of items is 5000. Version Control ;11x32 19apr99 added -R and and max items 200 -> 4000 ;a 18jun99 added delete as an option and -Q for quiet ;b 14jan00 WINNT mods for new version ;c 01feb00 added -M for maximum no of days ;d 27may02 on restart, -R now checks file-time. (max is now 5000) ;e 8dec02 added -F for new H*: FipHdr fields with date/time of release ;f 28apr03 added -T delay offset (default is 3) ;g 06jun04 bugette in WINNT stuff ;h-j 23sep04 speedy for outque ;k 09mar05 added -UTC for DE is in UTC time ;l 17sep05 added -N (fiphdr) ;m-n 11aug06 added -z pram file and early-warning: ;o-p 03nov06 added DR and balance ;q 15nov06 added offset to UTC ;r-t 17nov06 early-warning-extra: added and -d doneque ;u-w 15mar07 slice time reduced.. ;x 23jul07 added -w - wait plus bugette in copy - not using the right fiphdr! ;9 10mar09 bugette for win2k and DST ;10 31mar11 added -x replace ;11-16 poss occasional bugette in hdr and added log-line ;17 added file-trace ;18-22 23jan15 added -Y defaultCE and doneque strparse ;23 4dec16 added replace-DU ;24 24aug17 added chk_size to output filename in case of v long names ;25-26 22dec17 bugette - did NOT save the incoming filename (in hold) - so might miss some FipHdrs ;27-28 buglettes ;29-32 1apr20 outque in FipSeq ; 31 chksize/zap X for output and doneq ;32 to close spOut (copyright) 2021 and previous years FingerPost Ltd.