ipfprep This program prepares incoming data to tweak it before IPFORMAT is run against it. Each line of data - or each record if it is not a line - is checked against a series of tests to try and determine the type of line it is. In the parlance of this program is tries to find a KEY. According to this key the record is either thrown away or, more usually, labelled with a record type. This record type is normally at the beginning of the line as that is the easiest for IPFORMAT to catch and work on. A series of parameter lines in the normal FIP syntax are used to describe the patterns and tests. (see below for a fuller description of all the variables). Firstly the file is described with the parameters : filtyp - type of file recsep - each record ends with ... reclen - (optional fixed record length) wild - if you are using WILD cards, what chr signifies The next thing is a series of FMTs and KEYs which try to test the line eg : fmt:BLANK s:0 b:0 key:BLANK tst:2="" delete ie split the line into two fields - the first contains all the leading spaces and the second is the rest of the line. If field 2 has nothing in it, it is a blank line. The action is strip the line. Normally if the result of the tests is good, you just want to label the line. In this case, if fields 3, 6, 9 and 12 each have a single space and field 1 is NOT a space, then we prefix "ZOG," to the begining of the line fmt:ZOG b:1 b:9 s:1 b:1 b:9 s:1 b:1 b:9 s:1 b:1 b:9 s:1 b:0 key:ZOG chk:3 chk:6 chk:9 chk:12 tst:1#" " addkey:1:ZOG, You can also create new records from the fields you have created : fmt:STX b:40 s:0 n:0 s:0 b:0 key:STX chk:1 chk:3 tst:5="" delete newrec:STX,{3},{1} This deletes the original record and adds a new one starting "STX," followed by field 3 comma field 1. Lastly it is good idea to have a catch all for the lines you missed or for the majority of lines : ; setup the default keyname for text files defkey: addkey:1:DAT, This just prefixes four chrs "DAT," at position 1 of the line. Note there should be at least one space/tab after 'defkey:' and any (or all) default action - addkey/delete/newrec Notes - the NAME of a 'fmt' must be unique but there can be several 'key's for that fmt which may be testing different things - and creating newrecords. ; Headers Par Value - strip fmt:HDR p:10 b:0 key:HDR tst:1="Par Value " delete newrec:PAR,next key:HDR tst:1="Div NvG " delete newrec:DIV,next key:HDR tst:1="Yld Notes " delete newrec:YLD,next key:HDR tst:1="Chg on wk " delete newrec:CHG,next key:HDR tst:1="Earn/Shr " delete newrec:ERN,next Here the original record is replaced by one of five records with no extra data. Each FMT is checked sequentially - ie top down in the parameter file. If a single key matches, then the parameter file is checked for more key lines with the same name. Processing stops on the next 'fmt' record. This means all the keys for a fmt should follow immediately after as in th examples above. Note that if you have a 'newrec' and do not want to delete the original, the new one is inserted BEFORE the original. FMT - Syntax : and a single line , field marked [] are optional fmt:(name) (typ1):(len1)[:(startpos1):(startstring1):(endstr1)] ... (typn):(lenn) where name is the name of the record format (case insensitive) tabs or spaces are separators between fields colons are separators between parms for the same field length or size of field is 0 for variable size type is a-alphabetic, u-uppercase, l-lowercase n-number, p-printable, s-space(or tab, CR, NL or FF) b-binary (ie anything), x-alphanumeric, c-control (ie < 040 or >= 0177), z-anpa hdr field (ie alnum plus non-quad/format punct. t-punctuation startposn is the starting posn in the record from posn1 startstring is the starting string for the field endstring is the end string for the field KEY - Syntax : key:(name) chk:(fldx) tst:(fldx)=(string) tst:(fldn)#(string) addkey:(posn):(len) delete: newrec:(syntax for newrec) key:(name) same name as FMT previously specified chk:3 chk field 3 is of the format is the right length or if field 3 is varibale length, has 1 or more. tst:7=ABC Does field 7 equal ABC tst:9#" " Does field 9 NOT equal 4 spaces Note that tst is case sensitive. addkey:1:3 Add a key of the name to the record at 1 delete: Delete this record newrec:PAG0003 Add a record before with data PAG0003 (recsep) DO NOT ADD THE END OF LINE it is done automatically RECKEY Check the type and size of the key Syntax : reckey: : (length) : (type) where length or size of key is 0 for no size check type is a-alphabetic, u-uppercase, l-lowercase n-number, p-printable, s-space(or tab, CR, NL or FF) b-binary (ie anything), x-alphanumeric, c-control (ie < 040 or >= 0177), z-anpa hdr field (ie alnum plus non-quad/format punct. t-punctuation Input parameters are : Optional : Either -1 : filename (and optional path) of a single file def: spool queue The input file is NOT deleted in this case. or -i : input queue default: spool/fprep -o : output queue default: spool/form -f : file creep time for files default: 0 arriving across the network - normally make it 5 secs -t : scan time of directory default: 5 secs -l : do NOT log every incoming file default: log -z : use the basename of the file for the parameter file if there is .. .. no FP FIP header field default: use FP only -Z : always use this parameter file default: use the FP field. -v : display version number and exit. Example : ; ; type of file is text filtyp:t ; each record is separated by a CR CR NL recsep:\r\r\n ; strip all cr nl combinations including blank lines ; this boils the 'recsep' into a single CR stripeol:y ; There are no reclen and reckey commands - comment them out ;;reclen:121 ;;reckey:3:u ; change the wild card chr to a dollar (normally it is a star '*') wild:$ ; blank lines - to be stripped - put first to catch all of them fmt:BLANK s:0 b:0 key:BLANK tst:2="" delete ; text 8 col lines fmt:TX8 s:1 p:38 p:1 key:TX8 tst:3="-" addkey:1:TX8 ; format of the trailer records ; (spc) AJ (4 num) (2 spc) (10 prinatable repeated 4 times) fmt:TRAILER s:1 u:2 n:4 s:2 p:28 p:28 p:28 p:28 ; a trailer record needs to be deleted - and subrecords created from it key:TRAILER chk:1 tst:2=AJ chk:3 chk:4 delete key:TRAILER chk:1 tst:2=AJ chk:3 chk:4 tst:5#" " newrec:E{5} key:TRAILER chk:1 tst:2=AJ chk:3 chk:4 tst:6#" " newrec:E{6} key:TRAILER chk:1 tst:2=AJ chk:3 chk:4 tst:7#" " newrec:E{7} key:TRAILER chk:1 tst:2=AJ chk:3 chk:4 tst:8#" " newrec:E{8} ; Main Heading - check the syntax of fields 1 and 3 and make sure f2 = '1' fmt:HEAD S:8 N:1 S:20 P:14 P:12:44 p:4:56 key:HEAD chk:1 tst:2=1 chk:3 addkey:1:HD1 key:HEAD chk:1 tst:2=1 chk:3 tst:4="CITROEN" newrec:PAG0003 key:HEAD chk:1 tst:2=1 chk:3 tst:4="PEUGEOT" newrec:PAG0024 key:HEAD chk:1 tst:2=1 chk:3 tst:4="RENAULT" newrec:PAG0104 key:HEAD chk:1 tst:2=1 chk:3 tst:4="TALBOT" newrec:PAG0105 key:HEAD chk:1 tst:2=1 chk:3 tst:4="FORD" newrec:PAG0106 key:HEAD chk:1 tst:2=1 chk:3 tst:4="MAZDA" newrec:PAG0107 key:HEAD chk:1 tst:2=1 chk:3 tst:4="LADA" newrec:PAG0108 key:HEAD chk:1 tst:2=1 chk:3 tst:4="ROVER" newrec:PAG0109 key:HEAD chk:1 tst:2=1 chk:3 tst:4="NISSAN" newrec:PAG0141 key:HEAD chk:1 tst:2=1 chk:3 tst:4="BMW" newrec:PAG0144 ; 1st Col headers fmt:COL n:0 s:39 p:10 p:10 p:10 p:10 p:10 p:10 p:10 p:10 key:COL tst:1#0 tst:3#" " addkey:1:COL ; 2nd Col headers fmt:CL2 s:40 p:10 p:10 p:10 p:10 p:10 p:10 p:10 p:10 key:CL2 chk:1 tst:2#" " addkey:1:CL2 ; Section Head fmt:SECTION s:1 u:1 t:1 p:0 key:SECTION chk:1 chk:2 tst:3="-" addkey:1:SCT ; SubSection Head fmt:SUBSECT s:1 u:1 n:0 t:1 p:0 key:SUBSECT chk:1 chk:2 chk:3 tst:4="-" addkey:1:SUB ; setup the default keyname for text files defkey: addkey:1:TX8 (copyright) 2024 and previous years FingerPost Ltd.