sffmail

sffmail             - to read from stdin
    sffmail -o queuename
    sffmail -h FipHdr information

Simple program to read in stream of data and stuff it on disk.

It is used in two main areas - capturing email inbound and capturing file sent
via the W4 module.

When used for receiving mail sent from an email server - usually kicked off by
'sendmail' - then an entry in '/etc/aliases' is required for EACH logon tracked
:

    fip: "| /fip/bin/sffmail -o mail_store -h RZ:fip"

It is a good idea to add a FipHdr field - like RZ in the above example -
containing the actual logon as mail from ListServers etc do not usually have a
single address.

For most recent versions of sendmail, you will need to reload aliases once you
have modified it using something like :
    /usr/lib/sendmail -bi
or  /usr/sbin/newaliases

and on rs6000   refresh -s sendmail

Please check your particular system what is required.

We can also 'sffmail' for receiving files from the browser using W4 using the
'-w' switch.

Input switches :
    -e : do NOT add all environment variables as FipHdr fields  default: yes
    -E : add all environment variables as FipHdr fields default: yes
        *note this option was changed in version 01s to ALWAYS put out the envs
    -o : output file path and queue         default: spool/xsmtp
    -h : Fip Header information         default: none
        -h "#DU:mailsort#SC:ascii#SN:\$e\$y\$i\$d\$z"
    -w : use with w4 incoming files         default: no
    -l : log incoming to fip LOG            default: no
    -m : For w4, message to send back if no message file
        default:<HTML><Head></HEAD><BODY>Thank you for \\SV from \\SA - reference
\\SN</BODY></HTML>
    -M : For w4 name of a file of FipSeq to send back
        default: /fip/web/template/recvfile.answer
        Note old versions (pre 01f) used a default of
            /fip/web/setup/w4.incoming.answer
        recvfile.answer uses a stylesheet /fip/web/pages/css/recvfilex.css
    -r : For w4 multiple files  - filename of the TOP
    -R : For w4 multiple files  - filename of the TAIL
    -d : folder into which we can put the raw files. default: none
        use this for debugging
    -u : default DU if none specified (mainly for W4 traffic). def:
w4remoteUnverified
    -z : name of parameter file in tables/mail  default: CHECK.MIME
    -v : version and exit

For those switches with parameters, the parameter MUST be separated by a space.

FipHdr fields filled in are :
For W4 -
    SP - REMOTE_ADDR of the sendert
    SN - Name given by sender
    ST - Content-type
    SW - Full Path of file sent on sender's box
    SV - filename on sender's box
    SA - Logon
    S1 - Publication of that Logon (if there is one)
    S2 - Section of that Logon (if there is one)
    S3 - UserType of that Logon (if there is one)
    S4 - HTTP Host - proxy server
    S5 - IPaddress of the sender
    S6 - Receiving system hostname
    S7 - Receiving system Protocol
    S8 - realname/logon
    S9 - localhost name
    E* - (optional) any mime headers in 'x-(yyy)-SU: (spc) (data)' format where
yyy is the string in the xmime-key: parameter
    PLUS a multitude of date and time fields (the 'HX's)
    PLUS any field with a name="FIP-XX" in the template is added to the FipHdr
    PLUS any mime headers in 'x-fip-SU: (spc) (data)' format

    If there is no DU for ipwheel to match, a default is used -
"w4remoteUnverified" (from version 01p)
eg
    <input type="hidden" name="FIP-DU" value="remote2cci">
    <input type="hidden" name="FIP-SU" value="w4remote">
    <input type="radio" name="FIP-WB" value="holding" checked>
    <input type="input" name="FIP-WK" value="" size=15>
    <input type="input" name="FIP-WN" value="" size=30>
    <input type="input" name="FIP-WI" value="" size=5>

Note there is also an (optional) parameter file with a list of mime-types to
allow or disallow. The parameter file is in tables/mail/CHECK.MIME unless
specified otherwise using the '-z' input switch.

The syntax for the parameters are in the normal Fip standard and the keywords
are :
    ; comment
    round-robin: (number)              default: none
    round-robin-fiphdr: (2 letter FipHdr field) default: none
        Round-Robin the output files and add the RR number to the fipHdr.
        Both parameters are required - the Number is the MAXimum.
        eg to leave the output in folder1 to folder9
            round-robin:9
            round-robin-fiphdr:RR
        and a suitable output folder might be   /fip/spool/xchg\RR
        Note that the round-robin number is NOT added automatically to any output
folder - ie you MUST specify a FipHdr as in /fip/spool/2xml\RR

    save-data-path: (pathname for data)
        This puts the data of the incoming data in a file in this folder and creates
a FipHdr file that contains 2 FipHdrs containing the full path/filename
            SX: and FTP_EXTERNAL_FILE:
        (ipbalan uses SX and ipftp uses FTP_EXTERNAL_FILE)
            eq  save-data-path:/fip/data/jpegs/\$e\$y\$i\$d/
        Use this for big files that you do not want to copy around the Fip Spool
area.
        ** if specified, ALL non-standalone files will be split like this **

    default-allow-mime-types: yes/no/allow/disallow
        Make the default to either allow all (except those specifically disallowed)
or vice versa
        default is yes - ie allow everything in.
        yes or allow are the same
        no or disallow are the same
    allow-empty-mime-type: yes/no/allow/disallow
        default is yes to allow all files with NO mimetype specified to be received.
        yes or allow are the same
        no or disallow are the same

    allow-mime-type
    disallow-mime-type
        Specify the mimetype you want to accept or reject
        There can be up to 100 allow/disallow-mime-types
        eg  allow-mime-type:text/plain
        There are no defaults
        Note only the string entered is checked, so :
            allow-mime-type:text
        will allow ANY text variant - text/plain, text/html, texturally/wonderful.

Normally, if you want to control the types of files received, then :
either  1. make the default ALLOW and list the types to DISALLOW
        default-allow-mime-type:yes
        disallow-mime-type:application/msword
        disallow-mime-type:binary
or  2. make the default DISALLOW and list the types to ALLOW
        default-allow-mime-type:no
        allow-mime-type:text/plain
        allow-mime-type:text/rtf
        allow-mime-type:x-fip-quark

Example :
; DISallow all by default and explictly allow a couple
default-allow-mime-types:no

; nothing there - reject it !
allow-empty-mime-types:no

; list of those we want
allow-mime-type:text
allow-mime-type:image/jpeg
allow-mime-type:application/rtf
allow-mime-type:application/x-macbinary
allow-mime-type:application/mac-binhex40
allow-mime-type:application/octet-stream
allow-mime-type:application/zip
allow-mime-type:application/x-msexcel

The following parameters may also be specified and are the equivalent for the
same inputs in the data stream.
        extra-fiphdr:   Add these extra FipHdr fields to each inbound file
        w4-default-dest:    (Fip DU or destination in sys/USERS)
        Send all files from senders who have NOT logged on to this destination.
        default is 'w4remoteUnverified'

        filebefore
        fileafter   contents of these files will be added to the output data - before
or after the main data.

        The following 'w4-message*' parameters allow replacement of the standard
messages returned to the client.
        (if marked 'file', it is the name of a file in web/templates)

either      w4-message-file     file - complete HTML page template for a single
incoming file
or      w4-message      FipSeq - complete HTML page in FipSeq
or      w4-message-top      file - top response template
        w4-message-meta     file - first response - poss with FipHdrs and textarea text
        w4-message-item     file - template for a each FILE sent
        w4-message-missing  file - if there were NO files send/received (using ..-top
and ..-tail)
        w4-message-tail     file - tail response template

        w4-message-nothing  FipSeq - no files message (if no .-top)
        w4-message-no-files file template for 'No Files' (if no .-top)
        w4-message-bad-authentication   file template for 'No Files' (if using
chk-fipid or check-apikey)
        w4-message-invalid-mimetype
            default is ST:<font color=red>** File ignored - MimeType disallowed - \\ST,
please resend in acceptable format</font>
        w4-message-hr       FipSeq replacement for <hr> between files

        w4-log-accepted: (FipSeq)
            Fip Item log line for files accepted and forwarded on to other parts of Fip
            default is no string
        w4-log-rejected: (FipSeq)
            Fip Item log line for rejected files
            default is "** File ignored from \\SU - MimeType disallowed - \\ST"

        rawque: (FipSeq)
            Leave a copy of the incoming file in this folder
            eg  rawque:/fip/data/raw.data/\$e\$y\$i\$d_sffmail
            default: none

        package-uid-fiphdr: (2 letter FipHdr)
        package-que: (FipSeq folder name)
        package-queue: (FipSeq folder name)
        package-name: (FipSeq file name)
        balance-package: (balance group for package)

        hash-in-fiphdr: (FipSeq chr)            default:none
            A hash/pound (#) in a FipHdr field is mapped to this chr
            Normally hashes are end-of-field in a FipHdr. So it needs to be mapped to
something else and we changed it here on outbound.

        response-mime-type:( mimetype of the response back)
            default: text/html

        xmime-key: (string)
        xmime-key0: (string)
        ..
        xmime-key9: (string)
            Save any mime headers including those starting 'x-(key)-(subkey):(data)
            These are saved in E0-9
            eg  xmime-key3:x-boat-anchor
                xmime-key5:x-boat-engine
            data =  x-boat-anchor:long, blue
                x-boat-engine:steam
            gives   E3:long, blue
                E5:steam
            Note you can also use the FipSeq filter for the same effect
            filter:E3   ZE  X-BOAT-ANCHOR:*
            filter:E5   ZE  X-BOAT-ENGINE:*
                in this case, add them to the FipHdr with 'extra-fiphdr:E3:\E3\nE5:\E5\n'

        fiphdr-fipid:(2 letter FipHdr field to hold the FipId)
            use this to save the FipId
            (it requires the Fipid to be in the HTTP_REFERER environment variable)

        fiphdr-setup: (2 letter code)   FipHdr field for Setup field
        fiphdr-subset: (2 letter code)  FipHdr field for SubSet field
        fiphdr-service: (2 letter code) FipHdr field for Service field
        fiphdr-uid: (2 letter code) FipHdr field for Uid field
        fiphdr-action: (2 letter code)  FipHdr field for Action or Variant field
        fiphdr-api-key: (2 letter code) FipHdr field for API-KEY
        fiphdr-api-code: (2 letter code)    FipHdr field for API-CODE
        fiphdr-api-response: (2 letter code)    FipHdr field for API-RESPONSE
            api-response can be raw, json, xml, html and may be used to vary the format
of the response (if setup so !)

        check-fipid:no/yes
        check-apikey;no/yes
            Check either the Fipid or the ApiKey for IPaddress range, commands, expiry
and logon
            If a FipId and/or an APIkey exists, it is checked and metadata added; but
only if the check is set to YES, will it be stopped.
            So if set to YES, these params are used to filter UNauthenticated files from
the incoming stream
            If set to NO, they are NOT checked and NO metadata stored

        trigger-extra: (ExtraFipHDr for an extra trigger file)
            Use this FipHdr info to create an extra file AFTER all the others have been
processed.
            eg trigger-fiphdr:\nRF:trigger\n
            This requires an optional input variable on the form (usually hidden) :
            eg  <!-- AFTER all the data, metadata and files.. trigger -->
                <input type="hidden" name="SFFMAIL_TRIGGER" value="XX:##UID##">
            The supplementary file is triggered - maybe for tracking purposes - to a
different DU (using meta specified in the FH parameter)

        debug-file:(pathnme in FipSeq)
            put all incoming data into a debug file
            debug-file:/fip/data/\$e\$y\$i\$d_debug/\$h\$n\$b_xremote_\$z.fip

Note that if defining Unix escape chrs in scripts, you will probably need to
escape the escapes ! eg : \n becomes \\n

Other env varis can be used to define where the system is :
    SFFMAIL_Z   name of a parameter file
            This is overridden by the '-z' input switch.
    SFF_HOME    where the home or top queue is.     default: /fip
            eg  setenv  SFF_HOME    /ripexpress/underware
    SFF_LOG     where the log files queue is        default: (SFF_HOME)/log
    SFF_SPOOL   where the data queues are       default: (SFF_HOME)/spool
    SFF_TMP     where the tmp data queues is        default: (SFF_HOME)/x
            THIS MUST BE ON THE SAME UNIX VOLUME as SFF_SPOOL queues.
            ie if spools are on /data99 which is hard disk /dev/sd0, you MUST also
            have the TMP queue on the same disk/partition

NOTE that for all BUT SFF_HOME, if the parameter starts with a '/' then it is a
hard, absolute path; if not then the spool area is under SFF_HOME.
    eg  setenv  SFF_SPOOL   /data7      will look under /data7 for queues
    while   setenv  SFF_SPOOL   data7       will look under /fip/data7

Version Control
;2h34   05sep04 added zippy and sum: and bugette in copying data..
    ;c 17oct06 allow alphanumeric Fipids
    ;d-g 11mar07 added FIP-MULTI-FILE
    ;h1-7 21dec10 64bit issue ;h2 6dec13 bugette - extra data before start of blob
;3 HR
    ;8-10 2jun15 added outname, outque, SX and RR
    ;11 WINNT again ;12 hash-in-fiphdr/isHash added; 13-15 added trigger and
extra-fiphdr better
    ;16-17 4oct15 added pkg for updating and new HR ;18-21 balance-package
    ;22-33 action/variant ;34 8may18 added fipseq_extras
;001z   27mar99 chj added w4 bits
    ;a 29may99 mods to Fiplogon
    ;b 21oct99 added H* fields
    ;c 01jan00 envs added
    ;d 01may00 log added
    ;e 17jan01 added -m/-M/-d
    ;f 20feb01 response template is now in /fip/web/templates
    ;g/h 21feb01 Allow big textarea called FIP-DATA as data
    ;j 25jul01 bugette - w4 new file overwrote SW (fullpathname)
            not XX (short-just filename)
    ;k 20sep01 RADIUS_* -> use RealName.
    ;l/m 24jan02 cleaned up multiple files for w4
    ;n/o 07feb02 added -r/-R
    ;p 09aug02 added default DU for w4 if none specified and -u
    ;r 20aug02 seqno increments for every file now - not JUST for multiples
    ;s 04sep02 allow mime headers -> FipHdrs if in xxx-fip-SU: format
    ;t 13dec02 added tracking of mime-types with accept/rejects (param file)
    ;u 04jun03 added extra logon info for w4
    ;v 21jul03 zapped trailing CR on some binary files
    ;w 24jul03 chg of param file
    ;y 17nov03 small adjustments incase the Fip-Data field is > 128k
    ;z 22jan04 bug tracking...

(copyright) 2018 and previous years FingerPost Ltd.