ipstatd This is the daemon for the statistics database. It uses a parameter file is used to route and process incoming files. This parameter is normally the name of the database to update which is normally the contents of the FipHdr field DB and defaults to tables/stat/DATABASE. It first uses a selection table to decide what the type of data this really is. As the list is top down, only the first valid selection is processed. IPSTATD then uses the 'type' to run a sequence of programs predefined in the parameter file. The syntax for the DATABASE file is : ; comment ; the following is a selection line (hdr field) = string [opt (tab) & (hdr) = string ...] (tab) >typename (nl) type: (typename) (program to run) trace: (typename) test: (typename) To describe the Selection syntax in detail : (hdr field) = string [opt (tab) & (hdr) = string ...] (tab) >typename (nl) Each selection is on a single line. If necessary, multiple conditions can be specified with the '&' to 'and' them. The operation equal, '=', can also be NOT equal '!='. Source Header fields (in SH) are preceeded by X, ie XC for category. A '*' is used a wild card string; a '?' is single wild card chr. To search for a string/chr embedded somewhere in a field, uses a '*' before and after. If embedded spaces are needed in the string-to-be-searched, use an '*'. Note that the search string is case_insensitive. Both the selection file and the main file are scanned completely, so that one file may be sent to none, one or several destinations according to the same or different criteriae. For the 'type' parameter : - '$i' refers to the input file name (Note \$i is still the FIP System Variable 'month') - all queues and files are assumed to be under /fip/spool - Never assume however that the path environment has been setup, so we advise you specify full pathnames for the programs. - all 'type' lines MUST precede the selection - ie be above. - FIP System variables and Header fields can be accessed. - there can be one or many or very many type lines. - any program can be run - if a script/program returns an error, it is logged in the Item log and further processing stops. If a 'type' exists for a typename, ipstatd will NOT run ipformat but will run what is specified - which may be ipformat of course. The 'trace' parameter is used for setup, tuning and testing a new type. All it does is tell IPSTATD to log each line in the Item log. EG: trace:shares Trace MUST always be specified in the DATABASE file BEFORE the types (ie on a line nearer the top of the file) and all types must be before the selection lines. The 'test' parameter does the same as 'trace' BUT NONE of the programs are actually run. This allows you just to check syntax etc. EG: test:racecards Test MUST always be specified in the DATABASE file BEFORE the types (ie on a line nearer the top of the file) and all types must be before the selection lines. Example of a sequence of programs for a 'type'. Here, if the file starts 'borsen', IPFORMAT is strted twice (sequencially, serially - ie NOT at the same time) using two parameter files in tables/form/text : WKENDSHARES and DAILYSHARES . ; shares : for both weekday and weekend type:shares ipformat -p dailyshares -i $i type:shares ipformat -p wkendshares -i $i ; Selection for type called 'shares' SN=borsen* > shares test:geges Going into the MUI, ip, and doing a 'l' to list the log (or 'm' to more) gives: Sat Mar 4 11:52:30 ipstatd !i : Incoming File : geges : : geges Sat Mar 4 11:52:30 ipstatd !f : Test/NotRun : /fip/bin/ipformat -p geges -i /fip/spool/form/geges -D -S NAGS Sat Mar 4 11:52:30 ipstatd !f : Test/NotRun : /fip/bin/ipxchg -1 formsave/NAGS -D geges -F -o formsave Sat Mar 4 11:52:30 ipstatd !f : Test/NotRun : /bin/sort +0 -3 -o formsave/NAGS.done formsave/NAGS Sat Mar 4 11:52:30 ipstatd !f : Test/NotRun : /bin/mv formsave/NAGS.done 2go/#SN:geges#DU:nagsdone Other Points worth noting (ish) .. Break out - If either the input parameter -x or a header field FZBO is present, the input file is 'broken apart' into blocks, records and fields. The resultant file is called (dest)_(SN) in spool/formtest where dest is as above and SN is the filename. If an incoming file matches none of the tests, it is deleted and an error logged. In the selection file, remember to specify long names first. In the following example, type 'sunrac2' never gets processed as all files will be typped as 'sunrac' XK:RAC* >sunrac XK:RACING* >sunrac2 Input parameters are (all optional) : -c : name of a queue into which copies of all incoming files are made. default: no copies -f : file creep time default: 0 -i : queue to scan default: spool/2stat -l : do NOT log every incoming file/destination default: log -n : run the program at reduced priority default: nice 5 -p : processing file to use default: tables/stat/DATABASE -s : run files serially (ie one after the other) default: parallel -t : scan time of directory default: 3 secs -T : Always trace types. This is the same as the 'trace' parameter used for setup, tuning and testing a new type. All it does is tell IPSTATD to log each line in the Item log. def: no -x : debugging ON - ALL incoming files will be 'broken out' in formtest parameter is 'o'ctal, 'd'ecimal or 'h'ex. default: off -z : calm down time default: 5 secs To attempt to let ipformat finish one type before the next -v : display version number and exit. (copyright) 2022 and previous years FingerPost Ltd.