ipxchg IPXCHG translates the Character Set and any Style changes that are needed to process any incoming file. It uses a parameter file, in tables/xchg, to define what the processing should be and how it should be done. The incoming file is normally passed on by 'ipwheel' as part of the pipline to get the data to the destination. IPXCHG can also be run as a once-hit against a particular file. In order to determine what parameter file to use, 'ipxchg' uses FipHdr fields, if they exist. This can be overridden by the '-D' input switch which will force the xchg to be that name. Otherwise the 'CX' field is first looked for. If there is no 'CX', the 'SC' field is compared to the 'DC' field. If they are different, the xchg file used will be SC2DC. ie if 'SC:AP' and 'DC:HTML', then the xchg parameter file will be 'AP2HTML'. If there is NO SC, DC or CX and NO -D input switch, a string at the beginning of the filename (up to the first HASH/POUND SIGN) can be used. The actual parameter file is in tables/xchg and the filename will be forced Uppercase. Syntax for the xchg parameter file held in tables/xchg : one parameter per line ; Comments lines may be inserted at any time and are preceeded by a ';'. : Comment lines can also be preceeded by a colon in this case (type) [ (flag) ] (delimiter) (1st parameter) [ (delimiter) (2nd parameter) ...] where items in [] are optional - type is a CASE SENSITIVE type of parameter - c, D, x etc There should be NO leading spaces/tabs. To aid legibility, other letters and spaces may be put AFTER the type but take care NOT to use letters or punctuation, eg these are equivalent: chrset :ascii and c:ascii - A flag is any number between 1 and 100. It allows conditional xchgs to be done. Ie the flag must have be set ON using the 'o' command beforehand or the line is ignored. - Delimiter can be any punctuation EXCEPT backslash or Hash. it must also be consistent : z:SPC:after z_SPC_after - Parameter string is any FIP sequence of fixed text, Octal numbers (\000 format) and Unix escape chrs (\n etc). In some cases - eg 'r' for Add FipHdr - the data can be any FipSeq string including System Variables and existing FipHdr fields. The first parameter string is called the TO or SEARCH field. The second parameter (if there) is FROM or REPLACE field. There are no mandatory parameters - the defaults are : c: ASCII Chrset is ASCII b: 8 : 8 Input stream is 8 bit chr and output is 8 bit Default is : No escapes, xchgs or other mutilations. List of Types _____________ Syntax Example ------ ------- x, X - An xchg; from is the source string, to is the replacement string. The From or search string can be 1-255 chrs long; the replacement string can be 0-255 chrs long. x:fromString:toString x:+++:bong The difference between 'x' and 'X' is : 'x' the search string is case SENSITIVE 'X" the search string is case INsensitive ie X:head:hhh will replace 'HeAd' or 'HEAD' or 'head' etc with 'hhh' c -This refers to parameter file tables/chrset/PA (see below for format of this external chrset file) c:(main chrset) c:PA There are a number of hard-coded chrsets which do NOT need to be in chrset: c:ascii 7 bit ascii c:isoascii 8 bit ascii - latin1 c:ascii2baudot automatic conversion from ascii to baudot c:c2let,c2num the two baudot sets c:utf8 8 bit unicode - see doc on utf c:utf16 16 bit unicode - see doc on utf c:ucs2 16 bit unicode (Microsoft's notepad/Sql Server version) Multiple chrsets can be defined as comma separated names : c:PA,PA2,PA3 There can be up to 10 chrsets. or if only one alternate set needs to be defined : C - This is the alternate chrset - for 5 or 6 bit sets for example C:alt chrset C:c2num So these two are equivalent : c:c2let,c2num c:c2let C:c2num h - This is the header chrset. Parameter z:chghdr must be specified for the fields to translate (see below). If there is no 'h:' the main chrset is used. h:hdr chrset h:swhdr Please do NOT use characters NL, Tilde (octal 176) and Colon in Header fields They will be moved to octal 371, 372 and 373 respectively if you do ! e, E - Escape chrs to the alternate and back to the main chrset for multi-chrset For example where multi-layer chrsets such as BAUDOT and TTS are used, e:escape chr from main e:\033 E:escape chr from alt E:\037 Note the default is to pass these chrs through - use z:ZAPESC to strip. OR these can be specified as e:(Esc chr):(Number of chrset) where the number is the chrset in the 'c:' line above : c:PA,PA2,PA3 PA is 1, PA2 is 2 and PA3 is 3 there can be up to 20 escape chrs. The actual chr MUST be unique so octal 005 cannot escape to both PA and PA2 at the same time. But more than one escape chr can point to the same chrset. Escapes MUST always be specified lower down the parameter file than Chrsets. s, S - Escape chrs for a single Chr from the alternate and then revert back to the main chrset for multi-chrset For example where multi-layer chrsets such as ISO-7901 in its many guises. s:single chr escape from main s:& S:single chr escape from alt S:& Where we want to escape to a secondary chrset for a single chr. Note the default is to pass these chrs through - use z:ZAPESC to strip. d, D - Delete all between from and to strings. Lowercase d strips BOTH FROM and TO strings, Upper D strips only the FROM. d:ignore from:to d:<:> D:ignore from:to D:<MDNO:<MD i, I - Ignore all from the beginning until this string Strip all from this string to the end of file From version 47l conditional flags are NOT ignored. i:Ignore Upto string i:\002 I:Strip all from this string I:\003 Note that this string is the RAW data - ie NO chrset translation has been done beforehand In either case, if the string is NOT found, then data is NOT stripped. See also z:RAWDATA-BEFORE and z:RAWDATA-AFTER which work in the same way except the data is preserved in RAWDATA* and stripped in i/I m - MultiByte chrs for UTF/Unicode m+\u2012+- m+\u2013+{endash} m+\u2014+{emdash} One common one is the Euro : m+\u20ac+EURO\s a, A - Add text at beginning or end This text is ALWAYS added and any conditional flags ignored. a:String for begining of file a:{m1Top Table{m0\n A:String to end file A:\r\003NAME \$d-\$m-\$y\004 From version 44m onwards, up to 100 adds may be specified. a:Top line\n a:Line 2\n a:Line 3\n From 44w, both After and Before Adds may be dependent on a flag A +\n(always) A #22+\n(not 22) A #33+\n(not 33) A 22+\n(yes 22) A 33+\n(yes 33) A +\nbibi\n f, o - Set flag OFF or ON - see below for more information on flags To set flag on or off for conditional Xchgs. f:Search string:Flag number f:\r\nREU:23 o:Search string:Flag number o:NNNN:23 The flag is turned ON/OFF immediately AFTER the string The string itself is left untouched - use an 'x:' line to strip that. Uppercase 'F' and 'O' may be used for case-INsensitive search strings. p, P - flip-flop a flag ON and OFF To set flag alternatively on then off for conditional Xchgs. p:Search string:Flag number p:*T:44 The flag is turned ON/OFF immediately AFTER the string The string itself is left untouched - use an 'x:' line to strip that. Uppercase 'P' may be used for case-INsensitive search strings. k, K - Kill or delete characters either before or after the search string k: search string:No to delete before k:\r\n:5 K: search string:No to delete after K:RHUBARB:93 The string itself is left untouched - use an 'x:' line to strip that. r - Add a new Fip Header Field after processing has finished. The SEARCH file holds the Hdr field name and the REPLACE field holds data. Note that any FipSeq data can be used but all FF, CR, DEL and NL are replaced by '_'. r-TI-Time is \$h:\$n:\$b Several 'r's may be specified. The new fields are added at the end of the Fip Hdr so they may be used to override existing Hdr fields but will in turn be overridden by any Hdr fields in the filename. Note the '-' used as separators as ':' is used as part of the string y, Y - Yank or save text between from and to strings Lowercase y saves from the end of the FROM field to the BEGINNING of the TO field, while Uppercase Y saves to the END of the TO. y:[ptsize is:\n Y:<save>:<unsave> u, U - UNYank or unsave text previously saved using 'y' or 'Y' Lowercase u adds the text at the BEGINNING of the FROM field, while Uppercase U adds it AFTER the FROM. The TO field is ignored. u:<add before this> U:=AddAfter= w, W - Allow wild character and strings The first character of the Lowercase w is the wild CHARACTER ie a single character, while the first character of the Uppercase W is the wild string. w:? wild chr is '?' W:* wild string is '*' Note that only one wild string is allowed in the TO or SEARCH string of any test and it can NOT be the first chr of the SEARCH string. There is no restriction on the number of wild Chrs in either search replace strings. When used in REPLACE the saved chrs are output in the same order as found. The maximum number of chrs and string is 1024. ; wild CHR is '?' w:? ; wild STRING is '*' W:* x:ABCD???HIJ:ab?fg* An incoming string of 'ABCDEFGHIJ' will be stripped to 'abEfgFG' Note that you should make sure that the two wilds are not in any other SEARCH or REPLACE string anywhere else in the parameter file or you will get funny results : W .* x .[p*].<ptsize=*> x .@*&%$.abcde will probably NOT give the required values for the second, fixed string. Note that by default the wild card/string is positional - use it and lose it. To use the Wild string as a full string each time - as in the example : x .[cp*].[cp*][cl*] use the option 'z:ALLWILD' To use the contents of the wild buffer in a later xchg line, use the option 'z:KEEPWILD' z - Setup Option z:option z:FCC: Options include: FCC idiot upper and lower-case. Note you may force an Uppercase letter by using a '~' as a precedent chr. NUL do NOT strip NULs ie binary zero chrs NULS are normally deleted from the output automatically TOPSPC strip all spaces, tabs, carriage return and line feeds at the top of the file BOTTOMSPC strip all spaces, tabs, carriage return and line feeds at the bottom of the file EOLSPC strip all spaces and tabs BEFORE the end of a line (all other spaces on the line are preserved) An end-of-line is either CR and/or NL chr - but optionally a single different character may be specified eg z:eolspc:\222 SPC or SPC:before strip DUPLICATE spaces BEFORE xchgs SPC:after strip DUPLICATE spaces AFTER xchgs LINE or LINE:before strip DUPLICATE newlines/line feeds BEFORE xchgs LINE:after strip DUPLICATE newlines/line feeds AFTER xchgs IGNORE-DATA Ignore the data part of the file. Any fipHdr and before/after are processed but all data from the file is stripped. ABSTRACT-DATA:(number) Only preserve the first 'n' characters Any fipHdr and before/after are processed but all data AFTER the first 'n' are stripped. RAWDATA-BEFORE:(FipSeq string) RAWDATA-AFTER:(FipSeq string) Save/Store the data before/after this string with NO changes This is similar to i/I except the data is preserved in RAWDATA* and stripped in i/I FIPHDR-IS-DATA-TOO Allow xchgs on the whole of the file including the FipHdr part! NOFIPHDR or NOHDR Do NOT add the FIP header to the data. FIPHDR-ONLY or HDRONLY Only process the FipHdr fields - leave the data alone - see below. CHGHDR Also process/replace header fields as well as the data part FipHdrs to process can be specified by listing after this flag. eg : z:chghdr:QA,SH,DI z:chghdr with NO FipHdrs defaults to the Source Header only (SH). The header chrset can be specified by the 'h:' parameter or will default to the same as the primary chrset as in the 'c:' parameter. Note that if the fields also exist in the filename, they are also changed. Note you cannot replace individual subfields of the SH - like XP, XF etc. In this case, use NEWHDR to make a copy of that subfield first and then chghdr it. z:newhdr:AP \XP h:anpapriority z:chghdr:ap Note using z:hdronly AND z:chghdr:A1,A2 will result in changes to A1 and A2 only, All data and other FipHdrs will be untouched CONVERT-FIPHDR: (options - comma separated if there are more than one) eg: z:convert-fiphdr:utf8,map Convert the FipHdr field(s) from the existing chrset and recode any high-end chr in a particular style of entity. UTF8 Convert the COMPLETE FipHdr to UTF8 MAP Convert the field(s) in z:chghdr:xx using the map-unicode file specified Eg if the incoming data is Russian (CP1251) variety, put the unicode map in tables/unicode and : z:chghdr:IH z:unicode-map:CP1251.TXT z:convert-fiphdr:utf8,map Note you will always want to specified UTF8 too for this option ENTITIES:Quark ENTITIES:InDesign ENTITIES:Dec ENTITIES:Hex ENTITIES:Rtf ENTITIES:Json TRIM-HDR-SPACES: yes/no remove all multiple spaces and any spaces at the beginning and end of all FipHdr fields WORDMAC If the file is in the Mac version of the Microsoft Word format, the header and trailer are stripped, leaving just the text section. This works for Versions 5 and 6 only. An optional parameter may be added to STRIP the data if NOT Word (eg XLS) WORDPC If the file is in the Mac version of the Microsoft Word format, the header and trailer are stripped, leaving just the text section. This works for Versions 6 thru 9 (ie all WordBinary formats from 48f35). Optional parameters may be added to STRIP the data if NOT Word (eg XLS) and Save Meta as MA-MS: fiphdr fields z:wordpc:strip z:wordpc:strip,meta Unless z:nohdr' is specified, a FipHDr of M9 will be the numeric part of the code page for the data eg MZ:65001 for UTF8 For the Meta-Fiphdrs : MA - just the numeric part of the code page of metadata eg MA:1252 for Latin1 MB - Title MC - Subject MD - Author ME - Keywords MF - Comments MG - Template MH - Last Saved By MI - Revision Number MJ - Total Editing Time MK - Last Printed ML - Create Time/Date MM - Last saved Time/Date MN - Number of Pages MO - Number of Words MP - Number of Characters MQ - Thumbnail MR - Name of Creating Application MS - Security - From DocumentSummaryInfo .. NB - Category NN - Manager NO - Company ZAPESC For escaped chrsets - such as 5 level baudot or 6 level TTS, delete/ignore the escape chrs. ALLWILD Use the whole of the wild string in a REPLACE, not just the remainder. KEEPWILD: Normally the wild card buffer is only available for the one Xchg line and then zapped after use. This allow it to be used later on. PARTIAL new FIP header field created from existing field z:partial:qc XK,3,8,u ie To make QC, take K header field from the Source Header (SH) posn 3 for 8 chrs, UPPERCASE. This field is only used internally and NOT stuffed in the output file unless saved with NEWHDR. (refer to SysAdmin FipSeq doc for more information) FIXHDR additional FIP header information z:fixhdr:QZ 1234543 ie If QZ is specified, replace with 1234543 This field is only used internally and NOT stuffed in the output file unless saved with NEWHDR. (refer to SysAdmin FipSeq doc for more information) OPTION test a FipHdr field before outputting z:option:QR ZZ,3 This field is only used internally and NOT stuffed in the output file unless saved with NEWHDR. (refer to SysAdmin FipSeq doc for more information) REPEAT create a new FipHdr field from an existing field z:repeat:QR ZZ,-,3 This field is only used internally and NOT stuffed in the output file unless saved with NEWHDR. (refer to SysAdmin FipSeq doc for more information) COMBIE create a new FipHdr field from one of two existing fields or use a default z:combie:QR ZZ|ZR,(defaultpls) This field is only used internally and NOT stuffed in the output file unless saved with NEWHDR. (refer to SysAdmin FipSeq doc for more information) STYLE: create a new FipHdr field from an existing field and pad it with spaces, zeros or justify it using a 'printf' type style z:style:QZ XZ,%.04f This field is only used internally and NOT stuffed in the output file unless saved with NEWHDR. (refer to SysAdmin FipSeq doc for more information) REPLACE: create a new FipHdr field from an existing field and replace one or more strings/chrs. z:replace:qK XK "BC-"="" (refer to SysAdmin FipSeq doc for more information) NEWDATE: create a new FipHdr field containing the date offset from the current datee and time z:newdate:QD days+3 "\ZH - \ZD" This field is only used internally and NOT stuffed in the output file unless saved with NEWHDR. (refer to SysAdmin FipSeq doc for more information) SUM:, VALID:, LOOKUP:, UNIQUE:, FILTER:, MERGE:, PERL:, RECODE: (refer to SysAdmin FipSeq doc for more information) NEWHDR: create and SAVE a new FipHdr field. Any FipSeq can be added. z:newhdr:JF FormatIs.\XF z:newhdr:JN \XN+\XS=\SN This field is added to the FipHdr at the start of processing and can therefore be 'hdrXchged' etc. (refer to SysAdmin FipSeq doc for more information) Please note that any FipSeq (z:REPLACE: z:COMBIE: etc) MUST be specified first BEFORE any NEWHDR lines in the parameter file. TEXT2FIPHDR: Extract a line at the top of the file and make it a FipHdr field (and normally/optionally remove from text). A line is ended by any combination of CR and NL This is done on the RAW, incoming data,; not the output. At the top of text : This is done on the RAW, incoming data,; not the output. At the bottom of text : Specify with a negative line number : -1 for the last, -2 2nd last etc Lines are counted AFTER z:bottomspc and and Strip from (I:) If there is a trailing NL, CR or CRNL, they are ignored syntax z:text2fiphdr: (line number):(fiphdr):(optional flags) eg z:text2fiphdr:1:DI:no The optional flags can be : yes - (stripfromdata) - also delete that data or no - do NOT strip or xml - ignore any xml tags and attributes and only get data or html - ignore up to <body> or </body> The default strip is YES. There can be 100 z:text2fiphdr lines (note you cannot strip XML) REDUCEBEFORE: (chr) REDUCEAFTER: (chr) Reduce all multiple occurances of a single chr to a one The character can be in FipSeq - \031 reducebefore acts After the 'chrset' and Before any xchgs reduceafter acts after all processing. There can be up to 100 reducesbefores and afters. WRAPTEXT: (line length) : (End-of-Line string) : (optional alt eoln chr) Wrap text after all processing to a specified number of chrs per line. Add the End-of-Line string for long lines. The default line length is 64. The default end of line is NL or \n There is no optional alternative eoln chr (in addition to NL and CR) PRESERVE-SPACE-IN-WRAP: yes/no Normally the wordwrap string implies a replacement of a space. Sometimes - for quoted printable for example - the wordwrap is in addition to any other data So for these cases z:preserve-space-in-wrap:yes default: no IGNORE-XML-IN-WRAP: yes/no Ignore any tags within <...> in calculating wordwrap default: no for leave as part of the calculation IGNORE-ENTITIES-IN-WRAP: yes/no Ignore any entities within &...; in calculating wordwrap default: no for leave as part of the calculation WRAP-LONG-WORDS:(length) add a space after this number of characters to wrap long words eg z:wrap-long-words:77 default: no NUMBER: (octal/decimal/hex) Change the number system for FipSeq numbers to this The default is octal. Note this takes effect for ALL parameters in the file. FILEBEFORE: (complete pathname to a FipSeq file) BINARY-FILEBEFORE: (complete pathname to a FipSeq file) Add the contents of this file AFTER the FipHdr (if there is no 'z:nohdr' and before the 'a:' text string before and before the data. There can be up to 100 file befores. The file can be contain any text and FipSeq except the binary version will copy the data byte for byte with no checking or parsing for FipSeq. z:filebefore:/fip/tables/customer/Header.fipseq z:binary-filebefore:/fip/spool/savefiles/\SN z:filebefore:/fip/spool/savefiles/ThirdFile FILEAFTER: (complete pathname to a FipSeq file) BINARY-FILEAFTER: (complete pathname to a FipSeq file) ditto for after. The data from this file is add AFTER data and AFTER the 'A' text string after. There can be up to 100 file afters. INPUT-FILEBEFORE: INPUT-FILEAFTER: Special case of before and after files where the data is the binary contents of the input data file with NO changes. This is rarely used - as an extra attachment with a preceeding mimetype, for example. FILEFIPHDR: (complete pathname to a FipSeq file) a file containing FipHdr lines to be included at the end of the FipHdr before any NEWHDR fields (if z:nohdr has been specified, it is processed - so any new FipHdrs will be useable - but NOT written) USE-SX: replace/increment/merge/no USE-EXTERNAL-FILE: replace/increment/merge/no If and only if the file is just a FipHdr file and uses 'SX:' to point at an external data file, 'I'ncrement- make a new SX: file with the changes; the FipHdr file is output. The existing SX file is untouched This is the default if USE-SX is specified with no parameter. 'R'eplace existing SX: file; the FipHdr file is output 'M'erge the FipHdr and the Data to make a single output file NO-ignore any data in the file whose path is in SX LOCALE: (valid locale) Change the locale from the System Locale to this The locale MUST be valid ! Always use the Locale WITHOUT a chrset if possible and the Locale WITH single byte (iso8859) and not UTF8 Even if the the input and/or output data are UTF8, we transform the single byte. (if Linux, use 'locale -a' to find the name ; check what is available with 'sudo dpkg-reconfigure locales') FLAG-IF-FIPHDR: (flag number) : (FipHdr field) Turn on a flag if a FipHdr exists and has a least one character in. z:flag-if-fiphdr:42:XC There is an optional string after the FipHdr which is a further test that the FipHDr equals this string. z:flag-if-fiphdr:44:SU:AFP or use either != or # to signify NOT z:flag-if-fiphdr:44:SU#AFP USA-DATE-FORMAT \$A date format should be MM/DD (default is DD/MM) usa-data-format:no will also force it DD/MM CONVERT-TO-ENTITIES: (dec|hex|quark|indesign|rtf|json|jsonslash) Convert all non-basic ascii chars to the ϧ notation. eg a pound sign (xA6 or octal 236) becomes ž Use this for XML feeds that do not want UTF8. The option can be : dec - decimal notation ì hex - hex notation ¦ quark - quark express notation <\#236> indesign - adobe indesign notation <0x00A6> rtf - rtf format \u998? (assumes \uc1 has been set in the header) json - json format \u00A6 jsonslash - json format \u00A6 in jsonslash the '/' slash is also escaped ('\/') BTW The JSON spec says you CAN escape forward slash, but you don't have to. A reverse solidus must be escaped, but you do not need to escape a solidus. in rfc8259#page-9, slash is both UNescaped and escaped ! Default is decimal notation (eg ì) MAX-ENTITY: (number) : (replacement chr) default: no max, all entities allowed Largest number allowed for entities. Use this to strip emojis and weirdo entities. if specified, the anything above the number is replaced by the RepChr (which defaults to a '*') eg ; make the Euro symbol the biggest ... x20AC = 8364 z:max-entity:8364:+ ; for hex number preceed with 'x' z:max-entity:x20AC:+ ENTITY-HASH:(fipseq chr) Replacement chr for the HASH '#' in an entity created by CONVERT-TO-ENTITIES for dec, hex and quark options Use this if any FipHdr fields need converting too. OUTPUT-ENTITY-HASH:(fipseq chr) Replacement chr for the HASH '#' in an entity Use this only if you are using the -O switch to convert back to HASH ! CONVERT-ENTITIES: CONVERT-FROM-ENTITIES: (json, rtf, quark, indesign, xml) default is xml Convert any text with entities in ϧ or &xAA; notation to a single chr JSON - all json encoded chrs such as \" \n \u00a3 RTF - \u123? Quark - <\@> <\#236> InDesign- <0x00A3> XML/Html- £ ì (plus named entities if used with a file - see below) Normally use this with z:convert-to-utf8 as most of the numbers will be > 255 dec. For XML, if the ENTITY-HASH is specified, that character will also be tested for For XML you can also use a 'read-entity-file' to convert symbols such as ' READ-ENTITY-FILE: (name of file in tables/xchg) Many websites/xml feeds encode certain chrs as 'entities' - either named or numbered: ie ' for a ' € for the euro sign numbers like £ for a PoundSign This uses a W3C entity file to automatically resolve Html/Xml entities before the rest of the xchg is run If the entity is NOT in the file, it is left in the data and not converted. so &russiandoll; would be left alone These are exactly the same format/files as used by ipsgml and are 'borrowed' from w3c http://www.w3.org/TR/xhtml1/DTD/ appendix 2 They look like <!ENTITY copy "©"> <!-- copyright sign, U+00A9 ISOnum --> Multiple files can be specified : z:read-entity-file:xhtml-lat1.ent z:read-entity-file:xhtml-special.ent z:read-entity-file:xhtml-symbol.ent CONVERT-TO-UCS2: Convert the incoming data to the UCS2 variant of Unicode. This process is done on ALL xchged data as an absolutely last stage. UCS2 is the format that MicroSoft Word uses by default for example. CONVERT-TO-UTF8: Convert the incoming data to the UTF8 variant of Unicode. This process is NOT done on any xchged data. If you need the reverse process, you will have to find all the funny chrs and map them to whatever eg x +\u2034+(euro) CONVERT-TO-BASE64: CONVERT-FROM-BASE64: Convert the data to or from Base64 when converting TO base64, - Adjust line lengths with the optional number (default 72) - Only the data part is converted - which allows befores and/or filebefores to add lines like: Content-Transfer-Encoding: base64 - Note that if you are encoding PDF files, you nearly always need to preserve NULs - using z:nul See the example below. CONVERT-TO-UUENCODE: (TO uuencoded - not implemented yes) CONVERT-FROM-UUENCODE: Convert the data to or from uuencoded UNMATCHED-UNICODE: (FipSeq String) Where the chrset of the incoming data is UTF8 or UTF16 or UCS2 (as specified by 'c:utf8' or 'c:ucs2' for example), use this to map any hi-byte unicode chr to a replacement. z:unmatched-unicode:* UNICODE-MAP (file of character maps in tables/unicode) Convert the data from the source set to unicode using a Unicode Mapping table Use where the source encoding such as BIG5 or KOI-8 and iyu need to map to Unicode Note that you need also to specify how to represent the unicode chr - which will often be > 255. So select an output encoding such as 'z:convert-to-utf8' 'z:convert-to-ucs2' Also the input chrset should be 'c:isoascii' For standard maps - look at http://www.unicode.org/Public/MAPPINGS/ or http://tcl.apache.org/sources/tcl/tools/encoding In these lookup tables, '#' is a comment. Each chr is on a separate line: (raw hex chr) (spc) (unicode chr) spc [ opt comment] eg from KOI-8.TXT 0x9F 0x00F7 # DIVISION SIGN 0xA0 0x2550 # BOX DRAWINGS DOUBLE HORIZONTAL 0xA1 0x2551 # BOX DRAWINGS DOUBLE VERTICAL 0xA2 0x2552 # BOX DRAWINGS DOWN SINGLE AND RIGHT DBL 0xA3 0x0451 # CYRILLIC SMALL LETTER IO REVERSE-UNICODE-MAP (file of character maps in tables/unicode) Convert the data from unicode to the source. The syntax and explanation is the same as for UNICODE-MAP (above) Note that normally the input chrset will be 'c:utf8' or 'c:ucs2' If you have both a UNICODE-MAP and a REVERSE-UNICODE-MAP, the REVERSE is always done first UNICODE-MAP-IS-7BIT:yes Some code maps - like BIG5 or SHIFTJIS - are 7 bit codes not 8 as the mapping start at xA0 not xFF ie anything from xA0 needs to be mapped default is NO (or unspecified) for 8 bit eg ; map from SHIFTJIS z:unicode-map:SHIFTJIS.TXT ; note it is a 7bit jobbie z:unicode-map-is-7bit:yes ; .. and remember to output in UTF8 z:convert-TO-utf8 UNICODE-MAP-BASE: (ascii | isoascii | none) fill in any missing elements with the basic ASCII set ascii - basic 7 bit from 0 to x7F isoascii- basic 8 bit from 0 to xFF none - ignore it ! default - none MAP-DATA: (no | yes) default is yes If you want to ONLY map one of more FipHdr fields, use this to NOT map data. ie - convert ONLY DI and XK c:isoascii z:map-data:no z:convert-to-utf8 z:chghdr:DI,XK z:unicode-map:CP1277.txt FLIPLINES Flip a line of data from left2right to right2left - or viceversa Use this for mixed data - Hebrew-English or Arabic-English z:fliplines:ltr text default is left-to-right, so flip Hebrew z:fliplines:rtl text is right-to-left, so flip Latin9 z:fliplines:rtl,xml text is right-to-left, so flip Latin9 and ignore any xml z:fliplines:xml,x40,x80 text is ltr, and flip any chrs between x40 and x80 A line end is CR, NL or CRNL CONVERT-FIPHDR-TO-QP: (orig FipHdr field):(new):(language):(length):(WrapChr) Convert a single FipHdr to Quoted-Printable for use in a Mime header (as used by the 'Subject' line in smtp) If no language is specified, iso-8859-1 is the default. If no length is specified, the whole field is converted otherwise this is the maximum allowed on the line (normally 76). If length is specified and the language is 'utf-8', the chrs are converted to and from beforehand so that complete chrs are included. The Optional WrapChr is a FipSeq chr which signals to ipsmtp to wrap here rather than trimming the Fiphdr to that size. eg z:convert-fiphdr-to-qp:DI:DJ:koi-8 DI:DJIA at 3pm is 9999.3 generates DJ:=?koi-8?Q?=DJIA=20at=203pm=20is=209999.3?= CONVERT-ENCODED-WORDS: (all|hdr|data) This converts the FipHdr or the Data or Both from Mime Encoded Words format Use this to convert Email subject lines like Subject: =?windows-1255?B?5uQg8uXjIPDx6eXvIOH54ensIODw4/jl?= An Encoded Word line (usually an email field) has the format '=? (chrset) '?' (type) '?' (data) '?=' This only works within text, so if the data is in a FipHdr field, use this and Z:FIPHDR-IS-DATA-TOO in the xchg parameter file (and make sure you have no other xchg lines which could damage the FipHdr !). ANCHOR: Search for common web urls and wrap an <A> tag around them Urls starting with 'http://', https://, ftp://', 'www.', 'ftp.' and 'mailto:' are converted. eg www.fippo.org becomes <a href="www.fippo.org">www.fippo.org</a> This works for Text only and can have a negetive effect on wordwrap. OUTPUT-FILENAME: Output filename in FipSeq Normally the output filename has a few fipHdr fields added. This is the same as the -N input switch except output-filename is used for the xchg it is specified in while -N acts on the LAST xchg (not just -1 single) PRESERVE-FILENAME: The input filename will be used as the output filename. Normally the output filename has a few fipHdr fields added. This is the same as the -P input switch, except only files using this xchg will be preserved COPY-FILE: (full path and filename in FipSeq) Up to 30 copies of the original file can be made eg Z:COPY-FILE:/fip/data/nefarious/ends/\$d/\SU_\EN ZIP-PROGRAM: (optional - path to a gzip or zip program /fip/3rdparty/zip (default /bin/gzip) UNZIP-PROGRAM: (optional - path to a GUNzip or unzip program /fip/3rdparty/zip (default /bin/gunzip) ZIP-FILE-EXT: (optional file extension) eg gzip -d requires files with an ext of 'gz' ZIP:(before/after) UNZIP:(before/after) Zip/Gzip or UNgzip/zip the contents of the file before or after processing Normally you would want to zip AFTER xchgs and UNzip BEFORE If you UNzip after, do not touch the data part at all and set Z:NUL so that NULs are preserved ZLIB:(before/after) UNZLIB:(before/after) ZLIB compress or uncompress the contents of the file before or after processing SCRIPT: (script and parameters in FipSeq) Run a script AFTER processing the file Care must be taken NOT to run a script on a file that normally is written to a spooled queue as that file probably will not exist when ipxchg runs the script! So you do not run out disk space, the script could delete the input file after use OR delete them all in the nightly maintenance - 'zapfiplog' Tmp FipHdr fields are created for script : E1 - original input file (note that if this has spaces in, pls quote) E2 - input file - for this xchg (ie will be a file in /fip/x for Multiples) E3 - output file - rarely want this as it has probably disappeared if in /fip/spool E4 - name of the xchg eg Z:SCRIPT:/fip/local/fip_sortoutname.pl datafile='\E2' outputname='\E3' meta=\$d/\SU_\EN SCRIPT-HASH:(fipseq chr) Replacement chr for the HASH '#' in E2 and E3 Default is \035 - so your script/program must cater for any Hashes - like in FipHdr fields - by mapping this chr back to a '#' SEQNO-NAME: (fipSeq) Use a dedicated seqno for this traffic default: use common one eg z:seqno-name:GNS\DU REPLACE-TAG: (original xml tag) (spaces) (replacement tag) used to replace a tag with its attributes Also use 'z:replace-tag-attX' (where X is 1-9) to put the attributes into a temporary, private FipHdr area, so you can use FipSeq to reouput them in the new tag. EG - Incoming data has format : <ticker companyLongName="Reed Elsevier NV Cert." companyShortName="REED ELSEVIER NV" sedol="4148810" zzcode="ELS" isin="NL0000349488" countryCodeId="NL">Reed Elsevier</ticker> But we need the style of : Manpower <symbol tickercode="MAN" country="US" cusip="56418H100" isin="US56418H1005">MAN</symbol> shares led the gainers, rising 2.5 percent to $44.52. z:replace-tag:ticker \T0 <symbol tickercode="\T6" country="\T5" cusip="\T3" isin="\T4">\T6</symbol> z:replace-tag-att1:companyLongName z:replace-tag-att2:companyShortName ; Sedol is T3 z:replace-tag-att3:sedol z:replace-tag-att4:isin z:replace-tag-att5:countryCodeId z:replace-tag-att6:zzcode LOG-LINE: (extra information for logging in FipSeq) z:log-line:\SN company=\A1 indcode=\A2 isin=\A3 The default is \SN \CX for filename and xchg(s) ipxchg needs the '-l' input switch for this to print ALWAYS-LOG: Always log ignoring the -l switch NEVER-LOG: Never log ignoring the -l switch DONE-QUEUE: Name of a folder which the incoming file is copied to default - incoming file is deleted if spooled MAX-SINGLE-FIPHDR-SIZE: max length that a single fiphdr field can be. default is 8196-sep2012 (was 2024) The minimum is 1024 and maximum is 31000 INTERPRET-BOM: (yes/no) Look for the Byte Order Mark and cater for this automatically This is usually the first two or three bytes of a Windows or Adobe text file. (first two bytes as BigEndian 0xFE 0xFF, LittleEndian 0xFF 0xFE or 0xEF 0xBB 0xBF for UTF8) Some common programs from Microsoft, such as Notepad and Visual C++ add BOMs to UTF-8 files by default. Google Docs adds a BOM when a Microsoft Word document is downloaded as a .txt file. default is yes (from version 48f46) MASK-EMAIL: (yes/no) MASK-NUMBERS: (yes/no/ or a number) Blank out any of these with 'x' if found For numbers, only mask out if the run of numbers (ignoring spaces +-.,) is equal or more than this (to mask telephone numbers) MASK-SOCIAL-MEDIA: (yes/no) This blanks out the word (hopefully the address) after twitter:, snapchat:, facebook:, skype:, instagram: An Example of an xchg : ----------------------- ; use the MAC chrset ie /fip/tables/chrset/MAC c:mac ; strip all text from notes mode to the next mode change D:<MDN>:<MD ; strip formats d:\016:\017 ;.. replace CR CAN (Octal 15 then 30) by CR x:\r\030:\r ; .. otherwise make CAN a space (chg Octal 30 to 40) x:\030:\s x:<MDB>:<B> x:<MDR>:<P> x:<MDI>:<I> x:<MDF>:<F> ; use a different delimiter for changes sake x-\300e-e ; strip duplicate spaces both before and after the xchgs ; default is BEFORE the xchgs z:SPC:before z:SPC:after Multiple Passes --------------- From version 47b you can run multiple xchgs in the same run by specifying in the CX Fiphdr file (usually in the USERS file) stocks DP:edsys1 DQ:junk-edp CX:Stocks1,Stocks2,Stocks3 SC:NO DC:YES DF:stocks This will run the same file through 3 xchgs successively - tables/xchg/STOCKS1, STOCKS2 and STOCKS3 - before sending to the Editorial System. Previously the OLD method was ... Note that only one pass of the file can be made in one go. So to run multiple passes, add dummy addresses in the tables/sys/USERS file eg : stocks DP:localhost DQ:2go CX:Stocks1 SC:NO DC:YES DU:stocks1 stocks1 DP:localhost DQ:2go CX:Stocks2 SC:NO DC:YES DU:stocks2 stocks2 DP:edsys1 DQ:junk-edp CX:Stocks3 SC:NO DC:YES DF:stocks How to use flags ---------------- - ON 'o' and OFF 'f' and FlipFlop 'p' lines do not affect the data themselves. - The flag is turned on/off AFTER the string. - add spaces around the flag for better legibility, These are the same : x77:\n\n\n:\r\n x 77 :\n\n\n:\r\n - NOT all lines are affected : x, X, I, o, f, r, k, K, d, D - you can use flags within flags : o 22 +\n+32 Only if flag 22 is ON, a newline will turn 32 ON. - Test for a flag being OFF with a '#' before the number : f # 19 :abc:33 if flag 19 is OFF, string 'abc' will turn flag 33 OFF (if 33 was ON) o # 19 |abc|33 if flag 19 is OFF, string 'abc' will turn flag 33 ON (ignoring if 33 was ON or OFF) - Flags 1 to 100 are acceptable. - You want to set flags on and off before any xchgs which use them as the xchg may zap the data you need to key off. ie Turn the first 'A' after a NL to a 'B': o:\n:32 x 32 :A:B f 32 :A:32 will NEVER turn flag 32 OFF as the preceeding line will have modified it ! Better do : o:\n:32 f 32 :A:32 x 32 :A:B From version 44j, you may specifiy multiple flags on/off by putting a non-number between : o +\n+1,2,3,4 Current limitations are : ------------------------- Max size of an XCHG parameter file is 64k chrs. Max size of a search OR a replacement string is 255 chrs each. Max number of flags is 100 (1-100). Running IPXCHG from a Script ---------------------------- The program can be used in a script to run against a single file using the '-1' input switch. In this case, if the output file exists, it is OVERWRITTEN. If the output queue is the same as the input, the input FILE is overwritten. The '-D/-F/-o' switches are also advised in this case to be sure where the output file will end up. Eg /fip/bin/ipxchg -1 formwork/SHARES.RUBY -F -o formsave -D crunchy will process (and delete after) the file from spool/formwork using the xchg file tables/xchg/CRUNCHY and leave the xchged file in formsave. or ipxchg -1 Cruddo -D mangle will process Cruddo in the current queue using xchg file MANGLE. Note that the output queue is ALWAYS under /fip/spool. Special Case of Running an Xchg against the FipHdr -------------------------------------------------- Using 'z:hdronly' allows you to ONLY modify data in the FipHdr and leave the data part completely untouched. Only a subset of commands are useable - some are obviously ignore like Word format. Others not taken into account are : - 'h' for hdr field translate (use normal 'c' for chrset) - 'a' and 'A' for adding before and after the data part You may find it advisable to use flags to limit the damage an xchg can do to a particular FipHdr field. For example - we want to xchg LAG, SOC and ORO in the 'NC' FipHdr field and and word ManU in the 'NZ' : ; no chrset chg here c:isoascii ; turn ON HdrOnly z:hdronly ; Flag 1 is ON during the NC field only - turned off at the end of the line ; note we use pipes as delimiters as we have a colon ':' as part of the data o |\nNC:| 1 f |\n| 1 ; Only search for these if flag 1 is on x 1 |LAG|PAR:12000000:LAG x 1 |SOC|PAR:21000000:SOC x 1 |ORO|PAR:21000000:ORO ; Flag 2 is ON during the NZ field only - turned off at the end of the line o |\nNH:| 2 f |\n| 2 ; Only search for these if flag 2 is on x 2 |ManU|Liverpool Example of using ipxchg to encode mail attachments -------------------------------------------------- Use this for sending PDFs for example therough 'ipsmtp' ; main parameter file ; default chrset c:isoascii ; preserve NULs z:nul ; convert to base64 with a line length of 75 chrs z:convert-to-base64:75 ; read and parse the mime-header template beforehand z:filebefore:/fip/tables/xchg/MIMEHDR.BEFORE ; At the end - finish the last part A+\r\n--180191Fip\SN180191--\r\n ... and MIMEHDR.BEFORE could look like: Content-Type: multipart/mixed;\r boundary="180191Fip\SN180191"\r\n\r --180191Fip\SN180191\r Content-Type: text/plain\r\n\r The following is a PDF from the Fippo --180191Fip\SN180191\r Content-Disposition: attachment; filename="\SN"\r Content-Type: application/pdf\r Content-Transfer-Encoding: base64\r\n\r File tracing thru the Fip ------------------------- There are a series of parameters common to several Fip Programs which will log what is happening to all files, a service or a single file. file-trace: (valid actions - yes/no, + hdr + in + out) Actions are 'Yes' - log, 'No' - dont; if NOT NO, then in addition to logging, we can save a copy of the FipHdr, Input file and/or Output (delete is ignored in ipxchg)) file-trace-name: (UniqueId in FipSeq if not default - \SU-\SN-\HR) file-trace-log: (extra File Tracing logging in FipSeq) The Hdr and files are left in /fip/log/file_trace (which should be purged in the nightly maintenance zapfiplog) These can be overridden by FipHdr fields in each file if a particular file - or service needs tracing. (notice the '_' if using in the FipHdr but '-' if parameters) eg ZFILE_TRACE: hdr ZFILE_TRACE_NAME: louislouis ZFILE_TRACE_LOG: seqno.\ZQ Picking up data in the file and reusing as a FipHdr --------------------------------------------------- Say you want to pick up the funny date/time stamp at the end of a wire file.. ; ; create TI field from wire file ; needs ipxchg.44d plus (13nov00) stuff the lot into the FipHdr 'TI' r-TI-* ; a '*' is a wild string W:* ; do not zapt the wild buffer after the 'x' line z:keepwild ; strip from the ETX to the EST x:\003*EST: ; ** ENDS ** --------------------------------------------------- Input flags ----------- Input options are (all optional) : Either -i : queue in spool to scan default: xchg Or -1 : name of a single file to xchg default: use as spooler In this case the input file is NOT deleted and the output is forced to the folder '-o' field (if specified) or the filename '-N' (if specified) Or -s : buffer string default: use as file spooler -o : default destination queue if no CQ field is present. default: 2atex Normally a FIP hdr field, CQ, determines the output queue. If that is missing, this field overwrites the default. If this does NOT starts with a '/', it is assumed to be under /fip/spool. -O : display the resultant, xchged file and do NOT save to disk. Use this inside a CGI for example default: disk -A : allow W4_ABSTRACT_DC default: ignore this will run the xchg and copy the file to a temp area for ipw4 the original file is untouched, except the W4_ABTRACT_FILE points to the xchged abstract -d : name of the default xchg default: no default If there is more than one,separate by a comma eg -d DEFAULT1ST,DEFAULT2ND (48d3 onwards) Use '-L' to NOT log when this is used. -D : force the xchg file to this def: use SC2DC or CX -e : Supplementary xchg filename default: none This name is appended to the CX or SC2DC file name. This enables a 2nd group of xchges to be used. -E : name of a single entity file that should be run on ALL data. def: none see Z:READ-ENTITY-FILE: above -f : file creep time default: 0 For files arriving from a network, this is the wait time Generally this should be 5 secs for RCPed files etc. -F : force all output to the output destination queue and ignore the CQ field. default: use CQ -G : WINNT/2k - grab the first file only default: no -h : Extra FipHdrs to be added to the input fiphdr default: none -I : ignore any include-files in the parameter file default: allow ie ignore z:FILEBEFORE:abc.pl -l : log items through default: no -L : do NOT complain/log files requesting non-existent xchgs - just use the default default: complain -N : output filename (or path and filename) for single files (-1) only. default: overwrite input file -P : Preserve the filename for the output file default: if spooled, chged. -r : move the original file to this queue default: delete it -R : move the original file to this queue default: delete it and Change the name to CCYYMMDD.HHMMSS.(name) -S : display any message that is being logged. default: log only. -t : scan time of directory default: 2 secs -u : default is to USE-SX/USE-EXTERNAL-FILE, default use-sx must be specified in the param file (this only has effect if SX exists in the incoming file of course) -x : do NOT delete the original default: delete it This only applies to spooled files; running single mode will NOT delete the input. -X : do NOT allow multiple xchgs in the CX FipHdr field . default: allow -W : 10 ths of a second - MUST wait if nothing processed default: 0 -Y : one or more flags to set ON at the start of the run default: none -Y 33,44 -9 : do NOT run speedyFip with named pipes default: do if available -v : print version number and exit default: no Version Control 48g42 ; 17apr07 added convert-to-entities:indesign ;a-b 14may07 added Z:WRAP-LONG-WORDS:(length) ;c 05nov07 added xml for TEXT2FIPHDR ;d WINNT bugette ;d1-2 poss bug if single chr X and already-zapped data) ;3 allow multixchg for default ;4 text2fiphdr html added ;5 bugette with R:A3:abc ;67 allow quoted filenames for -1 and winnt bugette ;89 hdronly in multis bugette ;e4 01feb09 added convert-to/from-uudecode/uuencode ;2 TOPSPC after processing too ;f1 25mar09 added -A allowW4Abs and W4_ABSTRACT_DC: ;2 bugette with yanl/unyank and mulitple CX ;3-4 added z:convert-hdr:utf8,unicodemap and copy-file ;5-6 IGNORE-XML-IN-WRAP for entities too ! ;7 added ABSTRACT-DATA ;8 added W4_ABSTRACT_CX ;9 entities: rtf ;10 7dec09 bugette in wrap-long-words ;11 15dec09 bugette with anchor tags being xchged (they should NOT be) ;12 bugette with missing multiple xchg ;13-14 bugette - dont use BOTTOMSPC with HDRONLY ;15 3jun10 added z:convert-from-unicode ; 16 added ALWAYS-LOG and NEVER-LOG and EOLSPC ;17-18 5aug10 added MAP-DATA:no/yes ;18 allow blank Z:COPY-FILE using an option .. Z:COPY-FILE:\V1abc\$o ;19 16sep10 CHG - now looks for SC2DC before start of filename ;20-21 15nov10 better filename sizing / Win2k isspace bug ;22-24 wrinkleette in fipSpeedy and CONVERT-ENTITIES added ;25-27 19may11 added TRIM-HDR-SPACES ;28 29nov11 make sure output folder exists for -1/-N ;29 bugette in 'X' if flag specified but flag is never turned on ;30-2 9mar12 tuned TRIM-HDR-SPACES ;33 1may12 added z:fliplines:RTL and XML ;34 14jun12 added z:done-queue:(folder) ;35-44 14jul12 redid Word binary for all versions including Meta ;45 4sep12 added MAX-SINGLE-FIPHDR-SIZE ;46 8nov12 added INTERPRET-BOM to turn it off ! ;47 5dec12 woops bugette with convert-fiphdr-to-qp with multiple lines ;48 3jan13 read-unicode-map can be FipSeq ; 49-50 bugette with WORD ;51-53 bugette in QP ;54 28oct13 added convert-to-entities:JSON ;55-60 chasing bugette in Word-DocumentSummaryInfo ;61-65 27jan14 fudge/fix latin1->utf8 ! ;66 29apr14 file-trace added and ignore after if added (wrongly) to EOLSPC !! ;67 20jun14 maxCopies 10->30 ;68 30jun14 bugette in convert-to-entities:JSON ;69 30jul14 unicodes 100 -> 1000 ; 70 if z:hdronly do NOT convert NL convert-to-entities:JSON ;71-75 23aug14 allow \uFFFF as replace string .. (72-74 was BAD for utf8) ;76-77 bugette in replacement text ;78-80 added z:script ;81-85 cleanups ;85 bugette in flags not zapping buffer ;86-89 added z:output-filename and added speedy on -1 ;90-92 11apr16 added mask-email and mask-numbers ;92 allow output-name in spooled version ;93 WIN EACCES ;94 added z:SEQNO-NAME:ss\AB ;95-96 added MAX-ENTITY ;97 22sep16 bugette if readonly and -1 ;98 if FipSeq process on entry not with the other lines (11413) ;g1 31oct16 added z:mask-social-media:yes ;2 bugette in script - need space at end ;4-5 socmedia can bse caseINsens ;6-7 allow COPY-FILE in multixchg ;8 29apr17 use-sx merge/increment/replace buglette ;9 8jun17 bug in uudecode for short lines (chg for SEC) ;10 1aug17 added convert-from-entities:xml/json/rtf/quark/indesign ;11 30oct17 added z:preserve-space-in-wrap ;12-13 21dec17 bugette in SX when useSX:increment ;14 22aug18 added -s input buffer ;15-16 4oct18 bugette in \$W and multiple Xchgs ;17 11dec18 missing a case where script needed to be triggered ;18-19 24dec18 bugette in use-sx:i ;20-21 12dec19 redid linkage for WORD - uses the FAT better ;22 29jul20 tuned W4_ABSTRACT_CX ;23 28nov20 woops - bugette in DOC processing ;24 4jan21 allow s/r > 255 chrs ;25 21jan21 BUG in v big entity fields plus tuned s/r > 255 ;26-27 10mar21 UCS2 now handles emoijs ;28-29 12apr21 added FILEFIPHDR ;30-31 29jul21 added gzip/zlib and abstract int->long ;32 8oct21 tuning JSON conv-entities ;33 19apr22 Abstract better ;34-36 16aug22 uuencode better which unencoding files that are mainly compressed ;37 12nov23 wrapALT added ;38 15feb24 allow unicode chrs in CHRSET and added z:convert-to-entities:jsonslash to NOT escape slash ;39-42 19feb24 bug in g38 unicode chrs and allow flag in m 22:\u00A0:xxx ;40 added flag to z 22:unmatched-unicode ;047z bugette with multiple xchgs losing CQ ;a-b 19apr06 added z:replace-tag ;c 15may06 bugette if flag set on input or FLAG-IF-FIPHDR without other 'o' and 'f' ;de 25may06 added z:convert-to-entities at last and -N newname for single ;fg 28may06 added z:ignore-xml-in-wrap ;h-k 22jun06 added z:convert-to-entities:quark and ENTITY-HASH ;l 25jul06 allow flags in i/I strips and fixed bugette in -Y startflags ;m-n 14aug06 wrinkle in convert264 one too many '=' at end ;o 21sep06 added WrapChr to CONVERT-FIPHDR-TO-QP ;p 23oct06 bugette with c:utf8/z:convert-to-utf8 ;q-s 13nov06 added convert-encode-words: ;t 08dec06 added <@\> to quark-entities and removed TAB ;u-v 08jan07 allow ^ and % in parameter file names (v bugette in wild) ;w-y 08feb07 added z:FLIPLINES ;z 17mar07 added -3 extraFH3 plus Winnt -O is now binary ;046z3 mod for unicode-map (single bytes bugette) ;b 26jan04 added INPUT-FILEBEFORE/AFTER for original, raw file added CONVERT-FIPHDR-TO-QP ;c 30mar04 chrset can now be hex too \x4E ;d-e 19apr04 bugettes with unicode in hdr and include_file ;f-i 28apr04 added z:anchor ;j 29jun04 added max length to Convert-to-QP ;k-l 16aug04 added Multiple xchgs.. ;m-q 07sep04 added -9 speedy and x/m speedup and -I ;r-s 01apr05 add unicode-map-is-7bit for CP850 and KOI7 ;t 06may05 bugette with wildchrs (emmettlanebug) and the hex in CONVERT-FIPHDR-TO-QP should be UCASE ;u 30jun05 bugette in map-unicode - sometimes making a mapped chr NUL ;v 02aug05 bugette in i/I if string > 120 chrs ;w-z3 22sep05 added CONVERT-TO-UCS2 and read-entity-file - z3 thsxhg->STDBUF ;045z 16apr02 added UNMATCHED-UNICODE ;a/b 18apr02 added 'm' for multibyte chr as in m+\u20ac+EURO\s ;c 25jun02 added -G for WINNT/2k grab first file ;d 31jul02 z:NEWHDR - new fipHdr field may also have '$', '-' and/or '_' ;e 09aug02 z:FIPHDR-IS-DATA-TOO added ;f/g 15aug02 ':' is only matched/moved to 372 in FipHdr SH ;h 30oct02 bugette in flags where files has thousands of flags ;i 21dec02 woops - allow h:utf8 too where UTF8 chrs are in the fiphdr ;j 22dec02 allow z:text2fiphdr for lines at the END of text ;k 14jan03 bugette with h:utf8 ;m 16jan03 CHANGE - d:<:> will now be ignored if the end is not found ;n 03mar03 bottomspc now zaps replaced copy too ;o 10mar03 added IGNORE-DATA ;p-q 27mar03 better filenames for Win2k plus bugette in NEWHDR with NEWDATE ;r-t 10apr03 added FLAG-IF-FIPHDR ;u-w 14apr03 added USA-DATE-FORMAT/minors ;x-y 25sep03 bottomspc now zaps beforehand too ;z 31oct03 timings ;044z 18aug00 allow multiple filebefores/afters and -h for fiphdr and -O for stdout ;a/b 19aug00 added text2fiphdr ;c 10oct00 finally got the repeated chr bug ;d 03nov00 ..but that blew Yank/Unyank ;e 12nov00 allow SN length to be variable and allow wilds in 'r' ;f 20nov00 bug - stripped FipHdr for zero length data part ;g 06mar01 possible bug with flags and big files. ;h 27mar01 bugette - strips to/from (i/I) were parsed twice ;i 16apr01 bugette with flags again ;j/k 05jul01 allowed multiple flags on/off o +\n+1,2,3,4 ;m 03sep01 bugette with a/A and no data ;o/p/q 13sep01 added binary-filebefore/after and move_it added ;r 03oct01 bugette with FileBefore and no data in file. ;s 25oct01 bugette with text2fiphdr and stripping. ;t 14nov01 added FlipFlipFlags. ;u 19nov01 added z:bottomspc. ;v 26nov01 added c:utf8, c:utf16 and z:convert-to-utf8 ;w 29nov01 added flags to A-add after ;x 29jan02 added convert-to/from-base64 ;y 25feb02 bugette in d and D where 2nd string may be matched early ;z 15apr02 c:utf8 not working brilliantly. (copyright) 2024 and previous years FingerPost Ltd.