imapwire

imapwire

This periodically attachs to, checks for and grabs new articles in a mailbox on
a remote IMAP server.

It is normally started by 'iptimer' with FipSeq for the mailbox name, password
etc

The whole document is then left, normally, in spool/xsmtp for 'ipchkmail' to
pull apart the Header etc.

The parameter file, normally tables/wire/IMAP, is read for the the names of the
mailbox to scan.
    ; comment

    mailbox: (mailbox name on the remote IMAP server)

        password:(FipSeq/in plain)
            no default
        delete:(yes/no)
            Delete files that have been grabbed - and have the Ok-to-delete flag set (ie
old)
            This is the IMAP Expunge command for the mailbox.
            default is NO
        sendto:(newaddress)
            sendto allows you to specify another name for the DA field
            IPPOST will use this to route. By default
            the Fip Hdr field DA will hold the logon name.
        fiphdr:(FipSeq)
            Add to the FIP hdr - perhaps the DU field to change the destination.
            default: none
        inbox: (inbox name)
            default INBOX

    eg
    mailbox:chris   password:zongle fiphdr:#XX:here delete:yes

Optional
    grab-every:(seconds)
        Connect, logon and check for news every X seconds.
        The default is 600 seconds (5 mins) while the minimum
        is 5 seconds.
        The '-t' input switch can also be used.

    defdest: (default Fip Destination (DU FipHdr field) default: "imap"
    chrset: (Source character set ie SC header field)   default: ascii
    imap-host: (hostname or IP address of the host to attach to) nodefault
        (see also -s input switch)
    imap-port: (Port number of the host)            default: 143
        Unless use-tls is set where the default is port 993
        (see also -p input switch)
    response-timeout: (timeout in seconds wanting for the rmote to replay)
                                default: 60 secs
    extra-fiphdr: (more FipHdr information to add)      default: none
    skip-balance-group: name of a balance group (in tables/sys/BALANCE) to
distribute
        the skip file when changed (see doc on 'ipbalan') - for ipftp and webwire.
        This is often used where a second system could be used as a redundant server
        if the main system fails. (see also -B input switch)
    skip-balance-queue: name of queue under /fip/spool  default 2balance
    proxy-server: If using a proxy, these are the name and port to aim at.
    proxy-port:
    proxy-logon: This is the logon and password to get thru the firewall
        if required. The format is (logon) (colon) (password) and is
        converted to base 64.
        proxy-logon:Y2hyaXMuaHVnaGpvbmVzOnBhbnRoZXIK=

        To generate :
            echo -n "logon:password" | sffb64 -i
        eg  echo -n "chris:sleekpanther" | sffb64 -i
        gives   Y2hyaXM6c2xlZWtwYW50aGVy
            proxy-logon:Y2hyaXM6c2xlZWtwYW50aGVy=
    proxy-is-squid:yes/no   Is the proxy a Squid ?  default: no
        For Proxies - Please see note below

    use-tls:yes/implicit/explicit/no
        The commends are for a ftp running over SSL/TLS on the remote server
        default is NO
        no      - normal, standard FTP on (normally) port 21 for the control
        yes or explicit - connect (normally) on port 110 in clear then use SSL for
USER, PASS and data
        implicit    - connect (normally) on port 993: use SSL for all conversations
    tls-auth: (XXX)
        AUTH type for TLS/SSL               default: TLS

    ssl-method: (1,2,3,23,999)
        Version number to use for TLS/SSL       default: 999 for current default (2 or 3)
    ssl-password: (password)
    ssl-passwd: (password)                default: none
        Optional password if the handshake requires a shared secret
    ssl-cert: (name of a PEM certificate file)      default: none
    ssl-root-cert: (name of a root PEM certificate file)    defaunt: none
        Optional certificates - held in tables/ssl

    output-folder: (folder name)
    output-folder1: (folder name)
    ..
    output-folder9: (folder name)
        if the folder does NOT start with a '/', it is assumed to be
        Note these override the default and '-o' input switch..

Where sections of FipHdr fields are required or changes to the output style,
use
 keywords : fixed, partial, combie, optional, repeat, newdate and/or style.
(see The SysA
dmin manual for more information).

    They are normally specified :
        fixed:QZ    1234543
        partial:QT  ST,3,2,U,<,>
        combie:QY   ep|na,(0000000)a
        option:QE   ep,11,7,s
        repeat:QK   XK,-,3
    or  repeat:QP   PK,,4,#X
        style:QS    XN,%.03d
        replace:QN  NN  abc=DEF def=GHI
        newdate:QT  hours+3 "\ZD"

--- Gmail using IMAP

To access a Gmail account :
** beforehand, you must logon to the Gmail account
    - select settings
    - click on Forwarding and POP3/IMAP
    - select enable IMAP
    -( select Auto expunge OFF if you have more than one person/program accessing
!)

- you must use imapwiressl

- add the following to the parameter file

; Use TLS
tls-auth:ssl
use-tls:implicit

; Imap Host
imap-host:imap.gmail.com
imap-port:993

--- Using TIMER to kick off
Easy !

1. wire/IMAP.FIP

; Use FipSeq for the attributes
; If using W4 or Prestige, copy WN to another FipHdr field - RU in this case
mailbox:\WN password:\W7 fiphdr:\W3#RU:\WN#

; hostname of the exchange server
imap-host:(hostname here)

; If you have more than one fip - make sure the other system is up-to-date
; ** Add balskips group in sys/BALANCE - for each host: group:balskips
host:(hostname) ignore-localhost: nofiphdr:
skip-balance-group:balskips

; ---------------
2. setup/TIMER_IMAP.FIP

; If you have more than one fip and are running Primary/Secondary :
; ** Add wiresvr to sys/DEST_REDUN (or use an existing entry)
check-primary-server:wiresvr

group:imap
track-status:no
bandwidth-stats:no

; If you have more than one fip - make sure the other system is up-to-date
; ** Add balskips group in sys/BALANCE - for each host: group:balskips
host:(hostname) ignore-localhost: nofiphdr:
skip-balance-group:balskips

; Then for EACH mail address - add this line - emailaddress does not NORMALLY
need a domain
client:(emailaddress)   type:imap fiphdr:XX:extraStuff days:X every:1
passwd:(password)

; ---------------
3. sys/SYSTEM

; add the line ...
imap    mail    iptimer -n timer_imap.fip

; ---------------
4. check sys/BALANCE and sys/DEST_REDUN as above

--- Testing

If things do NOT look like they are working, you can run imapwire manually with
the -1 and -D to run once and display the handshake.

So if the line in the SYSTEM file is
    imap    wires   imapwire -s mail.bignastycorp.com -n imap.fip
.. You can test from a terminal/CMD with
    imapwire -s mail.bignastycorp.com -n imap.fip -1 -D
or if using ssl
    imapwiressl -s mail.bignastycorp.com -n imap.fip -1 -D

--- Note Imapwire saves the last item, date and time and UID in a file for each
mailbox in /fip/fix/imapwire

The three items are editable on 3 lines, so you can mess around at your peril
if you need !

::::::::::::::
imap_mail.zingle.com_fip$2011%hoho_inbox
::::::::::::::
408
14-Oct-2011
174223

--- Input switches are :
Mandatory :
    -s : Hostname where the IMAP is running.    default: none
Optional :
    -1 : one single pass and then stop      default: continuous
    -B : default balance group for skip files   default: none
        (see skip-balance-group parameter)
    -D : display the conversation with the remote server    default: no
        valid ONLY with the -1 for single shot; used for debugging troublesome
connections
    -h : extra FipHdr information           default: none
        This is in FipSeq and should normally be quoted
        Note this is the means that 'iptimer' sends variable information to imapwire
        eg : -h"SN:hello#TC:200401031"
    -l : do NOT log anything except errors      default: log files only
    -L : log every file and every connection    default: log files only
    -n : name of the service            def: name of the parameter file
    -o : Next fip queue for incoming files      default: spool/2go
    -p : port number on the remote host     default: 143
    -t : sleep in seconds between connections/accesses  default: 600 secs
    -U : restart on this UID            default: use last saved in the fix file
    -x : Proxy server host or IP address        default: none
    -X : Proxy server port              default: 80
    -y : Proxy logon                default: none
    -Y : Proxy server is Squid          default: no
    -z : parameter file             default: wire/IMAP
    -v : display version number and exit.

Version Control
;001v   31dec08 cleanups ;f note_balance_action ;g-h 16feb11 added TLS
        ;i-j 31oct11 make sure date is valid and better error msgs
        ;k-m 02apr12 bugette with tls ;m unlink tmp on singleshot
        ;n 21jul12 added output-folderX
        ;o-s 15oct12 bugette - missed first file if folder is reset (or zapped and
remade)
        ;t-u 26may13 added expunge (finally) ;v added 993 as default for use-tls

(copyright) 2017 and previous years FingerPost Ltd.