imapwire
imapwire
This periodically attachs to, checks for and grabs new articles in a mailbox on
a remote IMAP server.
It is normally started by 'iptimer' with FipSeq for the mailbox name, password
etc
The whole document is then left, normally, in spool/xsmtp for 'ipchkmail' to
pull apart the Header etc.
The parameter file, normally tables/wire/IMAP, is read for the the names of the
mailbox to scan.
; comment
mailbox: (mailbox name on the remote IMAP server)
password:(FipSeq/in plain)
no default
delete:(yes/no)
Delete files that have been grabbed - and have the Ok-to-delete flag set (ie
old)
This is the IMAP Expunge command for the mailbox.
default is NO
sendto:(newaddress)
sendto allows you to specify another name for the DA field
IPPOST will use this to route. By default
the Fip Hdr field DA will hold the logon name.
fiphdr:(FipSeq)
Add to the FIP hdr - perhaps the DU field to change the destination.
default: none
inbox: (inbox name)
default INBOX
eg
mailbox:chris password:zongle fiphdr:#XX:here delete:yes
Optional
grab-every:(seconds)
Connect, logon and check for news every X seconds.
The default is 600 seconds (5 mins) while the minimum
is 5 seconds.
The '-t' input switch can also be used.
defdest: (default Fip Destination (DU FipHdr field) default: "imap"
chrset: (Source character set ie SC header field) default: ascii
imap-host: (hostname or IP address of the host to attach to) nodefault
(see also -s input switch)
imap-port: (Port number of the host) default: 143
Unless use-tls is set where the default is port 993
(see also -p input switch)
response-timeout: (timeout in seconds wanting for the rmote to replay)
default: 60 secs
extra-fiphdr: (more FipHdr information to add) default: none
skip-balance-group: name of a balance group (in tables/sys/BALANCE) to
distribute
the skip file when changed (see doc on 'ipbalan') - for ipftp and webwire.
This is often used where a second system could be used as a redundant server
if the main system fails. (see also -B input switch)
skip-balance-queue: name of queue under /fip/spool default 2balance
proxy-server: If using a proxy, these are the name and port to aim at.
proxy-port:
proxy-logon: This is the logon and password to get thru the firewall
if required. The format is (logon) (colon) (password) and is
converted to base 64.
proxy-logon:Y2hyaXMuaHVnaGpvbmVzOnBhbnRoZXIK=
To generate :
echo -n "logon:password" | sffb64 -i
eg echo -n "chris:sleekpanther" | sffb64 -i
gives Y2hyaXM6c2xlZWtwYW50aGVy
proxy-logon:Y2hyaXM6c2xlZWtwYW50aGVy=
proxy-is-squid:yes/no Is the proxy a Squid ? default: no
For Proxies - Please see note below
use-oauth:yes/no
Use OAUTH to grab/use an access-token or Bearer token eg for Gmail access
default is NO
use-ssl:yes/implicit/explicit/no
use-tls:yes/implicit/explicit/no
The commends are for a ftp running over SSL/TLS on the remote server
default is NO
no - normal, standard FTP on (normally) port 21 for the control
yes or explicit - connect (normally) on port 110 in clear then use SSL for
USER, PASS and data
implicit - connect (normally) on port 993: use SSL for all conversations
tls-auth: (XXX)
AUTH type for TLS/SSL default: TLS
ssl-method: (1,2,3,23,999)
Version number to use for TLS/SSL default: 999 for current default (2 or 3)
ssl-password: (password)
ssl-passwd: (password) default: none
Optional password if the handshake requires a shared secret
ssl-key: (name of a certiticate key file) default: none
ssl-cert: (name of a certificate file) default: none
ssl-root-cert: (name of a root PEM certificate file) defaunt: none
Optional certificates are in tables/ssl unless name starts with '/'
ssl-verify: yes/no verify server certificates default: yes
ssl-ciphers: (list) acceptable ciphers
(use 'openssl ciphers' to list)
default: "HIGH:!aNULL:!kRSA:!SRP:!PSK:!CAMELLIA:!RC4:!MD5:!DSS"
ssl-display: yes/no display SSL connection details default: no
output-folder: (folder name)
output-folder1: (folder name)
..
output-folder9: (folder name)
if the folder does NOT start with a '/', it is assumed to be
Note these override the default and '-o' input switch..
-- For accessing Oauth protected assets
oauth-refresh-script: (Script in FipSeq) script to generate the access_token
using a refresh_token
These 5 FipHdrs are use to generate, check, add/renew permissions to access
the remote data - normally Gmail.
oauth-client-fiphdr: (FipHdr) default: IC
oauth-secret-fiphdr: (FipHdr) default: IS
oauth-access-fiphdr: (FipHdr) default: IA
oauth-refresh-fiphdr: (FipHdr) default: IR
oauth-expiry-fiphdr: (FipHdr) default: IX
-- Where sections of FipHdr fields are required or changes to the output style,
use keywords : fixed, partial, combie, optional, repeat, newdate and/or style.
(see The SysAdmin manual for more information).
They are normally specified :
fixed:QZ 1234543
partial:QT ST,3,2,U,<,>
combie:QY ep|na,(0000000)a
option:QE ep,11,7,s
repeat:QK XK,-,3
or repeat:QP PK,,4,#X
style:QS XN,%.03d
replace:QN NN abc=DEF def=GHI
newdate:QT hours+3 "\ZD"
--- Gmail using Oauth and IMAP
--- Gmail using SSL and IMAP
Generally this is being phased out for Oauth - see above
To access a Gmail account :
** beforehand, you must logon to the Gmail account
- select settings
- click on Forwarding and POP3/IMAP
- select enable IMAP
-( select Auto expunge OFF if you have more than one person/program accessing
!)
- you must use imapwiressl
- add the following to the parameter file
; Use TLS
tls-auth:ssl
use-tls:implicit
; Imap Host
imap-host:imap.gmail.com
imap-port:993
--- Using TIMER to kick off
Easy !
1. wire/IMAP.FIP
; Use FipSeq for the attributes
; If using W4 or Prestige, copy WN to another FipHdr field - RU in this case
mailbox:\WN password:\W7 fiphdr:\W3#RU:\WN#
; hostname of the exchange server
imap-host:(hostname here)
; If you have more than one fip - make sure the other system is up-to-date
; ** Add balskips group in sys/BALANCE - for each host: group:balskips
host:(hostname) ignore-localhost: nofiphdr:
skip-balance-group:balskips
; ---------------
2. setup/TIMER_IMAP.FIP
; If you have more than one fip and are running Primary/Secondary :
; ** Add wiresvr to sys/DEST_REDUN (or use an existing entry)
check-primary-server:wiresvr
group:imap
track-status:no
bandwidth-stats:no
; If you have more than one fip - make sure the other system is up-to-date
; ** Add balskips group in sys/BALANCE - for each host: group:balskips
host:(hostname) ignore-localhost: nofiphdr:
skip-balance-group:balskips
; Then for EACH mail address - add this line - emailaddress does not NORMALLY
need a domain
client:(emailaddress) type:imap fiphdr:XX:extraStuff days:X every:1
passwd:(password)
; ---------------
3. sys/SYSTEM
; add the line ...
imap mail iptimer -n timer_imap.fip
; ---------------
4. check sys/BALANCE and sys/DEST_REDUN as above
--- Testing
If things do NOT look like they are working, you can run imapwire manually with
the -1 and -D to run once and display the handshake.
So if the line in the SYSTEM file is
imap wires imapwire -s mail.bignastycorp.com -n imap.fip
.. You can test from a terminal/CMD with
imapwire -s mail.bignastycorp.com -n imap.fip -1 -D
or if using ssl
imapwiressl -s mail.bignastycorp.com -n imap.fip -1 -D
To test AND GRAB NOTHING, add the -V switch too
imapwiressl -s mail.bignastycorp.com -n imap.fip -1 -D -V
--- Note Imapwire saves the last item, date and time and UID in a file for each
mailbox in /fip/fix/imapwire
The three items are editable on 3 lines, so you can mess around at your peril
if you need !
::::::::::::::
imap_mail.zingle.com_fip$2011%hoho_inbox
::::::::::::::
408
14-Oct-2011
174223
--- Input switches are :
Mandatory :
-s : Hostname where the IMAP is running. default: none
Optional :
-1 : one single pass and then stop default: continuous
-B : default balance group for skip files default: none
(see skip-balance-group parameter)
-d : display the conversation with the remote server default: no
and pause between files for you to hit return to continue
valid ONLY with the -1 for single shot; used for debugging troublesome
connections
-D : display the conversation with the remote server default: no
-h : extra FipHdr information default: none
This is in FipSeq and should normally be quoted
Note this is the means that 'iptimer' sends variable information to imapwire
eg : -h"SN:hello#TC:200401031"
-k : on Display, send a NOOP instead of a CAPABILITY before LOGON
default: send CAPABILITY
-K : do NOT send anything before LOGON default: send CAPABILITY
-l : do NOT log anything except errors default: log files only
-L : log every file and every connection default: log files only
-n : name of the service def: name of the parameter file
-o : Next fip queue for incoming files default: spool/2go
-p : port number on the remote host default: 143
-t : sleep in seconds between connections/accesses default: 600 secs
-V : do NOT grab any files - used with -D -1 to test only default: run and
grab
-U : restart on this UID default: use last saved in the fix file
-x : Proxy server host or IP address default: none
-X : Proxy server port default: 80
-y : Proxy logon default: none
-Y : Proxy server is Squid default: no
-z : parameter file default: wire/IMAP
-v : display version number and exit.
Version Control
;02a-h 16jun18 fipseq_extras ;b ;c extra SSL details added ;d added defUseSSL
to reset on connect ;e socks4/5 added
;f 1mar22 added oauth2 for gmail plus fixed bug when > 64k of ids on SEARCH
plus TLS remhost added
;g-h 8jul22 DU and SC are notw FipSeq
;01z 31dec08 cleanups ;f note_balance_action ;g-h 16feb11 added TLS
;i-j 31oct11 make sure date is valid and better error msgs
;k-m 02apr12 bugette with tls ;m unlink tmp on singleshot
;n 21jul12 added output-folderX
;o-s 15oct12 bugette - missed first file if folder is reset (or zapped and
remade)
;t-u 26may13 added expunge (finally) ;v added 993 as default for use-tls
;wx 7mar17 send CAPABILITY before LOGIN so we can check if LOGINDISABLED if
-D !
;y 10apr17 made logon,password etc parseable
;z 16jun17 more logging
(copyright) 2022 and previous years FingerPost Ltd.