ipxbin

ipxbin

This program is used to put Atex J11 Binary files into some sort of order.
Theses fields are mirror images of the J11 data and are flagged with the FJ11
header flag.

These files are normally sent to the Unixbox via MIDO/BINFIP/IPXNET.

IPXBIN spools a directory for Atex J11 files and extracts the header
information according to its parameter file.

The input file is either zapped or can be left in a done queue.

Both Classified (IAS) and Editorial files may be sent over with the header
being processed accordingly.

Incoming files may be sent directly to the output queue (specify input switch
-R) OR a j11 header field, usually the 'TO' field, can be compared against a
routing table. The latter defaults to setup/BINROUTE. The 'TO' field can be
changed in the BINTRANS parameter file as described above using 'hdrto'.

It then strips the header, converting individual fields to FIP header fields
according to a translation table which is normally tables/setup/BINTRANS.

Keywords for parameter file are :
    ; comment
    hdr: Save the j11 Editorial hdr field in a Fip field. Syntax :
        hdr: (j11 hdr field) (tab) (FipHdr Field)
        eg : to Save MS from the J11 in DI for FIP :
            hdr:MS  DI
    arecord: 2 letter code and length of an IAS Classified Arecord field.
        These are in sequential order or use; any fillers - use a junk
        code (often the same junk code will do). While it is good
        practise to use the same codes as for IAS, Be careful NOT to
        reuse Fields that FIP uses for programs downstream.
        eg: arecord:AB  10
            arecord:AD  4
            arecord:ZZ  7   ; filler
            arecord:CC  12  .. etc
    adflag: 2 bytes which signals the header is Classified and NOT Editorial.
        Normally this is \301\301
        eg: aflag:\240\333
    type: Type of processing for the Data
        The default is ascii text, unhnjed, without markup.
        b = binary file - do not process
        t = typesetter file - strip the 1st two blocks
            and then treat as a binary
        m - allow modes - see mode table
        n - preserve nuls
        h - leave text hnjed
        k - preserve markup.
        q - translate all atex quads (2-5) to NewLine/LineFeed.
            and 012 to 220 octal (see also atex012)
        eg : type:hk  
            will leave markup and modes
    modes: Modes to strip. All text in these modes are ignored.def: 2,6,7,8,9
    pchr:  Precedent chr for mode switches.     default: {
        eg mode change to bold is {M1
    nohdr: Do NOT put a Fip Hdr on this file.   default: do
    addfiphdr: Some extra, fixed information to default: none
        add to the outbound Fip Header.
    allj11hdr: Translate all j11 hdr fields using their atex
        codes. Extra data which is not in a hdr field
        in added as junk fields starting $$ default: no
        See note below on allj11hdr.
    befhdr: FipSeq to add before Header is output.  default: none
    afthdr: FipSeq to add after  Header is output.  default: none
    befqdata: FipSeq to add before Qdata is output. default: none
    aftqdata: FipSeq to add after  Qdata is output. default: none
    beftxt: FipSeq to add before Text is output.    default: none
    afttxt: FipSeq to add after  Text is output.    default: none
        Note that the outputfile is HDR, then QDATA, then TEXT
    atex012: Chr to replace Atex 012 chr and translate all atex quads (2-5)
        to NewLine/LineFeed. (see also type:q above). def: do NOT chg
    softquad: Where non-quad end-of-lines need to be preserved, this is the
            string, which can be one or more fipseq chrs, to flag
            such a being. A non-quad eoln is, for example, an hnj
            loose line. A quad is QL to QM (2, 3, 4, 5).
    nomodes: Ignore mode changes            default: insert
        This does NOT insert {M1 etc on changes of modes.
    data:  Replace the data of the file with this FipSeq.
        The default is to use the file data.
    force:u/l Force all data Upper or lower case.   default: leave as is.
    filename: Make this the output filename. default:all Fip Hdr fields
        To make just the original J11 name :
            filename:\SN
        default: New filename containing the following Fip fields
            SC2DC BIN2SUN for an xchg, CQ of '2go', DU, SN, SC, HS
        To preserve the original filename and all incoming Fip Hdr
        fields, use the 'preservename:' keyword.
    preservename: To preserve the original filename and all
        incoming Fip Hdr fields         default: see above
        In this case, only the DU field will be added IF a new one exists.
    addqdata: Add the qdata UNTOUCHED at the top of the file.
                            default: strip qdata
    hdrto:  If routing, the J11 field which contains the routing code
        Eg: hdrto:NO            default: TO field
    hdrslug: The J11 field which contains the name of the file
        Eg: hdrslug:CA          default: SL field
    nodefroute: Flag to NOT use the default Destination/Queue if no match
        is found in the routine file. This is only used if routing is
        ON.                 default: use the default
    routewild: Wild Card chr for BINROUTE entries.  default: '*'
        A wild card can start and/or end a match field for a BINROUTE
        entry: Eg : *-spo   /home/sport
    doneq:  Pathname for input files once processed. default: deleted
    script: Name of an option program or script to run once the output file
        has been written.           default: none
        The full path and filename of the file is passed as the
        parameter.  If extra parameters are required, place these after
        the script name.            default: no script
        Eg:     script:printme laserbeam scotty
        will give the following parameters to script/program 'printme'
            $1  'laserbeam'
            $2  'scotty'
            $3  fullpath and filename
Pls never allow your script to loop as ipxbin waits for completion before
continung !

Where sections of FipHdr fields are required or changes to the output style,
use keywords : fixed, partial, combie, optional, repeat. newdate and/or style.
(see The SysAdmin manual for more information).

    They are normally specified :
        fixed:QZ    1234543
        partial:QT  ST,3,2,U,<,>
        combie:QY   ep|na,(0000000)a 
        option:QE   ep,11,7,s
        repeat:QK   XK,-,3  
    or  repeat:QP   PK,,4,#X
        style:QS    XN,%.03d 

Normally the BINROUTE file is disabled using the 'noroutefile:' keyword as you
will always want to force the file to a specific destination or queue.
The BINROUTE file has the style of :
    ; comment line
    (contents of j11 TO field)  (tabs/spaces)   (FIP DU or destinations)
OR  (contents of j11 TO field)  (tabs/spaces)   (Full Unix Output Path)
where :
    the j11 field 'TO' can be any field in the incoming file.
    the contents of the j11 field are case insensitive.
    The program decides on whether to use the destination or the path dependning
on the first character of the string : if it is a '/' or a '.' , it assumes it
is a path.
EG :
    ; BINROUTE file for typesetter
    proof   /data1/opi/dash_proofer
    set /data1/opi/lasercomp
    remote  /data1/opi/ricoh

The default queue/destination is the first valid entry in the file.
A false destination/queue of 'delete' means just that !
Multiple destinations may be specified separated by '+' with no embedded
spaces.
For queues, only a single queue is acceptable. If a queue name has an embeded
space, use double quotes to embed. The double quotes are stripped.

Second example :
    ;   this is a comment line
    ; first valid line is the default
    abc zzz
    ; delete as a destination means ignore this file
    def delete
    ; if the destination starts with a '/' or '.', it is a queue
    chj /rasta/oinky
    lt  ./lovely/hairdo
    ; Use the '+' to separate multiple destinations
    fnf fnf+fns+sss
    fns fns

The Input parameters are :
    Optional :
Either  -1 : single shot - run once and stop        default: spool
        The parameter is the name if the input file.
        If it starts with a '/', the input queue is ignored.
        This does NOT delete the input file.
Or  -i : input queue to scan            default: spool/xbin
        This will delete the input file unless the '-x' flag is set.
    -o : output queue               default: spool/2go
        "-o ./" to place the output path relative to the
        input queue (spooled) or current directory (single shot)
    -z : parameter file in tables/setup     default: BINTRANS
    -r : editorial routing file in tables/setup default: BINROUTE
    -R : no routing - all files are sent to the output queue. default: route
    -l : do log incoming files          default: no log
    -t : sleep time betwix scans of the queue   default: 5 sec
    -w : file creep time for files arriving across a network. default: 0
    -u : logon for the owner of the output files
                    default: logon that started ipxbin
    -x : do NOT delete files after sending      default: yes
    -s : do NOT strip trailing spaces from arecord fields   default: yes
    -v : print version number and exit

-- Normal syntax for a single shot is :
    ipxbin -1 xsndsnd/Bigfilename -x -R -z xbin.nyt -o xbindon
    This reads a file n a sunque from your 'pwd' called xsndsnd/Bigfilename.
    Once processed the raw file will be stuffed into another subque xbindon.
    Parameter file in tables/setup is XBIN.NYT (note uppercase)
    -x = do NOT delete raw file.
    -R = do NOT use a routing file.

-- Note that using both keywords 'allj11hdr' and 'fiphdr' together needs to be
thought about. Remember that any fields starting 'X' - Xk for example, is
assumed to be in the Source Header (SH), so any fields starting 'X' are not
accesible through the normal FipSeq routines. Ditto for J11 hdr fields starting
'$'. And of course Fip overrides any fields hat are system dependent like 'SN',
'DU' etc.

So 'allj11hdr' is better used with the no-fip-hdr flag 'nohdr', for files that
will be processed with program 'sffhdr' which can stuff anything. eg:
        allj11hdr:
        nohdr:
        afthdr:~~~~\n
(using 'afthdr' to write an End-Of-Hdr string of 4 tildes which the j11 cannot
produce in a header field -  sffhdr would use input option '-e "~~~~\n"' to
catch the marker.)

-- Note that the outputfile is HDR, then QDATA, then TEXT which is different
from the Atex. This is to get a proper header on the file before you start
playing with Qdata. Use befqdata/afqdata strings to help your script/program
determine when Qdata starts (if it exists) and ends.

-- For background information, the format of the incoming file is :
    (Qdata flag byte) (length of qdata) (qdata)
    (flag byte) (508 bytes of data) (flag byte) (508 bytes of date) ...
where the flag byte is Q for Qdata, H for Hdr block or T for start of text.

The sequence is ALWAYS Qdata (if it exists) then Header (if it exists) then
Text(if it exists) .

There will be 0 or 1 qdata blocks, 0 to 6 header blocks and 0 or 1 Text block
flag. The Text block flag is ONLY for the first block of data.

The format of the data is :
    Qdata : as per Atex File system manual for editorial,
        else program dependent.
    Hdr : as per Atex file system for editorial or ARECORD for IAS.
    Text : normally 16bit words UNLESS you tell it otherwise.

Other env varis can be used to define where the system is :
    SFF_HOME    where the home or top queue is.     default: /fip
            eg  setenv  SFF_HOME    /ripexpress/underware
    SFF_LOG     where the log files queue is        default: (SFF_HOME)/log
    SFF_SPOOL   where the data queues are       default: (SFF_HOME)/spool
    SFF_TMP     where the tmp data queues is        default: (SFF_HOME)/x
            THIS MUST BE ON THE SAME UNIX VOLUME as SFF_SPOOL queues.
            ie if spools are on /data99 which is hard disk /dev/sd0, you MUST also
            have the TMP queue on the same disk/partition

NOTE that for all BUT SFF_HOME, if the parameter starts with a '/' then it is a
hard, absolute path; if not then the spool area is under SFF_HOME.
    eg  setenv  SFF_SPOOL   /data7      will look under /data7 for queues
    while   setenv  SFF_SPOOL   data7       will look under /fip/data7

(copyright) 2017 and previous years FingerPost Ltd.