ipsmtp This feeds and receives from Internet Mail. It sends to an SMTP server and can receive either on a local server (Unix only) or from a POP3 server. We define OUTGOING as FIP -> SMTP and the reverse (SMTP or POP3 -> FIP) INCOMING. Routing Files Outbound ---------------------- Messages for sending are put in the input queue, usually spool/2smtp, with the FipHdr field DA containing one or more Mail Address (separated by commas if multiple) - such as wizzo@fingerpost.co.uk. Messages can be sent directly to the spool/2smtp queue, routed normally by Fip or as a group client for Syndication by IPPOST. In the latter case, the client is specified as type:unixmail and there is a special DU called TOUNIXMAIL in USERS which has the syntax : TOUNIXMAIL= DP:(hostname) DQ:2smtp DC:(whatever) where hostname is the name of the UnixBox doing the transfer and whatever is the character set required. The From field or sender is the header field SU or, if it exists, the SA (Source Author) which is formatted as SA@SU. SU is normally the Source system or type while SA is the normally the logon or sender. If the SA is an Atex style 'LOGON ;dd/mm,hh/nn' then only the logon is used. So if the SA:fippo and SU:liverpool-fc.co.uk, the sender becomes fippo@liverpool-fc.co.uk If the SA is ALWAYS the full address - like SA:hedgehog@flea-pit.com, then use the '-F' input switch to NOT add the SU header field. If neither SU nor SA exist, the default sender is 'fip' unless overridden by the 'defsender:' in the parameter file. Note that this name must be in the passwd file as a valid logon. The data of the outgoing messages will be converted if 8 bit or control chrs to the normal '=XX' hex format used. Routing Files Inbound --------------------- Messages from the Internet are sent to the special DU, FIPMSG which is either hardcoded to a specific destination, or more usually, directed to IPPOST to detetmine if the recepient is on the editorial system OR is a mailbox (IPBOX) customer. Testing ------- If things do NOT look like they are working, you can run ipsmtp/ipsmtpssl manually with the -1 and -D to run once and display t he handshake. Parameter File -------------- The parameter file, normally tables/mail/SMTP, is read for the logon names to scan on a regular basis plus the destination to send these to. The syntax of the parameter file is : ; comment defsender: (default sender) default: fip Used ONLY if there is NO fipHdr field SU nor SA. This is used for the 'From:' field for outgoing messages. forcesender: (Force sender to this logon) default: no This is the default reply-to address. It must be a valid logon and domain. It is also used for the 'From:' field for ALL outgoing messages if there is NO 'forcefrom' (see below) or FipHdr field 'DJ' in the file to send irrespective of whether there is a FipHdr SA or SU. It should be JUST the address : forcesender: lucy@sky-with-diamonds.com forcefrom: (Force the From field) This is the From field in the outgoing file which does not have to be the same as the sender's address. If using the '-h' host input switch it can contain a 'pretty' version : eg forcefrom: "MP3 king Steinhoff" <dohrayme@singalong.com> defdest: (default Fip Destination (DU FipHdr field) default: FIPMSG chrset: (Source character set ie SC header field) default: ascii newDA: FipHdr field to replace the DA field which default: DA holds the email address of the person to sent to. eg newDA:DZ This can also be an combination field : combie:QA DA|DZ|NA newDA:QA hash-in-address: (FipSeq Chr) Some old-style email addresses have a leading hash/pound/# sign - but a hash is a field sepearator in FipHdr. So another chr is used in the DA FipHdr and mapped at the last moment. default is 243 hash-in-fiphdr: (FipSeq Chr) Sometimes a Subject line might have a hash/pound/# sign - but a hash is a field sepearator in FipHdr. So another chr is used in the DA FipHdr and mapped at the last moment. default is '#' (ie a hash is a hash and that is where the FipHdr field ends !) eg in tables/route file, map a HASH out of the way : HASH-IN-HDR:35 in the ipsmtp parameter file, to map back : hash-in-fiphdr:35 newDI: FipHdr field to replace the DI field which default: DI is used for the 'Subject' field. newDJ: FipHdr field to replace the DJ field which default: DJ is used for the 'From' field. balance-seqno: Send the Sequence number to this Balance Group (see 'ipbalan') balance-delete: Delete the Data on remote systems once sent using this Balance Group (see 'ipbalan') forcedomain: (domain name) default: do not Strip any domain in the FipHdr field SA and add this one in its place. forcedomain:commies.com So if the SA:albert@redsunderthebed.com, then the from field becomes From: albert@commies.com strip-domain-on-send: Send only the logon/mail name default: domain too and strip any domain name. defsubject: (default Subject line if nothing specified) default: "Message for you" att-message: Message to replace an attachment with. default:** Attachment has been Stripped, pls contact System Managers for (name of attachment file) log-line: (FipSeq) Additional information for the log file entry in ALL extra-log-fiphdr: Fipseq FipHdr fields to add to any transmission log This is triggered on a file by file basis by the FipHdr field ZSMTP_LOG:(dest) in the file to send - (dest) being a valid DU in tables/sys/USERS. Use this to pass FipHdr fields from the incoming file to the outgoing default: none eg. extra-log-fiphdr:nSI:SIn send-new-fiphdr: Fipseq FipHdr fields to add to any outgoing file to act as defaults etc send-extra-fiphdr: Fipseq FipHdr fields to add to any outgoing file to act as defaults etc The NEW variant is added BEFORE the real FipHdr, so can be used for Defaults (and the real FipHdr fields can override these). The EXTRA variant will add at the end of the file's FipHdr and so will override any setting, This is triggered on a file by file basis by the FipHdr field default: none eg. send-extra-fiphdr:nZSMTP_TYPE:text/plainn stream-connections:yes default: no Do NOT drop the connection/reconnect for every file. Once a connection is up to a mailserver, any subsequent file for the same server is sent in the same transmission. (using SMTP only) fiphdr: (Map incoming mail field to a Fip Hdr field) default: none fiphdr:DI Subject fiphdr:AZ Message-Id ignore: (Mail header field to always ignore) default: none ignore: Status ignore: Apparently-To ignore: X-Msmail-Priority decode: unencode: Decode any base64 sections. default: no decode: This will NOT decode attachments decode:text This will NOT decode attachments decode:all This will decode body AND attachments maxattempts: (no of attempts before the file is sent to woops) default: 10 This checks the FipHdr 'DO' which is managed internally. (UNIX only) logeachfile:(dest) Send a Success/failed msg to this destination for each file. There is no default. This log file is just a FipHdr with the following extra fields : DR-File Sent OK DR:ok or DR:error DG-Will Retry later DG:retrying or DG:stopped DT-Some message text DT:No connection DO-Number of last attempt DO:5 HT-Date and Time HT:25 Nov 98 15:35:25 default: no log created. The DR and DG messages can be changed by using : log-dr-ok:(FipSeq) log-dr-error:(FipSeq) log-dg-stop:(FipSeq) log-dg-retry:(FipSeq) eg: log-dr-ok:File VN sent to Remote Number DA ($h:$n) msgeachfile:(FipSeq) Additional information to add to the FipHdr of the 'logeachfile' msg. This should be in FipHdr format and be in FipSeq. It can be used to pass FipHdr fields in the outgoing file into the log file. eg msgeachfile: DF:logdialnSS:SSn default: nothing added default-smtp-host: (hostname or IPaddress) For sending emails out - Same as the '-h' Input Switch alternate-smtp-host: (hostname or IPaddress) a failover hostname for if the default fails connection-timeout: (secs) Max time in seconds for the connection to the SMTP server The call is aborted if it takes longer than this. If the remote is across the computer room, this time can be reduced to 15 or 20 secs; if over the internet, 120 secs is the norm. default: 120 for 120 seconds, minimum is 10 send-single-smtp: send smtp in one file per session which is then cut and reopened for the next file/session. hostname: (hostname to use) Cosmetic ! timing-stats: yes/no Show timing stats if the incoming file has the HE/HR fiphdr default: no use-sx: or use-external-file: if there is an SX FipHdr field with a path to the data file, use that rather than the data in the input file. wrap-subject-chr: (FipSeq chr) If the Subject field has this chr, wrap the line default: none This might have been added by ipxchg : z:convert-fiphdr-to-qp:DI:DJ:utf-8:02 wrap-subject-chr:02 force-subject-wrap: (No of chrs) Normally subject lines at this total default: 76 eg force-subject-wrap:76 check-primary-server: pseudo-host name that is specified in tables/sys/DEST_REDUN Use this when for POP3 accesses ONLY where ipsmtp on 2 systems is accessing the same POP3 account(s) in a redundant way. It is used to decide whether the current host should be getting the files or not. ie in the parameter file is check-primary-server:timersvr and in the DEST_REDUN is ; psuedohost primary secondary timersvr fip1 fip2 and in the SYSTEM file for both fip1 AND fip2 there is the same 'ipsmtp' line with the same parameter file If fip1 is up, 'ipsmtp' on fip1 will always get while on fip2 it will just check/loop. If fip1 is down 'ipsmtp' on fip2 will start processing the files. use-tls:yes/implicit/explicit/no The commends are for a ftp running over SSL/TLS on the remote server default is NO no - normal, standard FTP on (normally) port 21 for the control yes or explicit - connect (normally) on port 110 in clear then use SSL for USER, PASS and data implicit - connect (normally) on port 995: use SSL for all conversations tls-auth: (XXX) AUTH type for TLS/SSL default: TLS ssl-method: (1,2,3,23,999) Version number to use for TLS/SSL default: 999 for current default (2 or 3) ssl-password: (password) ssl-passwd: (password) default: none Optional password if the handshake requires a shared secret ssl-cert: (name of a PEM certificate file) default: none ssl-root-cert: (name of a root PEM certificate file) defaunt: none Optional certificates - held in tables/ssl Then for EACH logon you wish to track INCOMING mail from the Internet specify a line begining : logon: (logon name) default: none or unix: (unixlogonname) default: none These two parameters are identical. There is also a means, for UnixBoxes only, to poll for ALL mail for ALL the members of a group (as in the /etc/group file) : group: (unixgroupname) default: none Whether 'logon', 'unix' or 'group' there are other (optional) parameters: sendto:(newaddress) fiphdr:(FIP hdr sequence) striphdr: keephdr: pop3pwd: (Password for POP3) where sendto allows you to specify another name for the DA field IPPOST will use this to route. By default the Fip Hdr field DA will hold the logon name. fiphdr allows you to add to the FIP hdr. This may be any - perhaps the DU field to change the destination. striphdr will strip the SendMail header except for the From and Subject lines pop3pwd specifies the password to use IF you are accessing a POP3 server to receive incoming mail. It is NOT required for UnixBoxes scanning local mail queues. To add a new logon : 1. Read the 'passwd' file for the exact Unixlogon name - eg newboy 2. Enter this name in SMTP file as : logon:newboy 3. If this is the same name as in file ATXMSG, enter line newboy edone where edone is the name of your editorial system 4. If 'newboy' is really called 'dirtyoldman' on the 'atex' editorial system, the entries should be: SMTP file - logon:newboy sendto:dirtyoldman ATXMSG file - dirtyoldman atex To bringup IPSMTP : 1. Create logons as above 2. Check you have destinations in the USERS file : FIPMSG= DP:fipmain DQ:post DC:atexeol MSGFORATX_EDSYS= DP:edsys3 DQ:atexq-grp DC:atexeol where 'fipmain' is the name of this UnixBox. Destination FIPMSG is generated by IPSMTP. Destination MSGFORATX_* is generated by IPPOST IF the name is in the FIPMSG file. eg if the Editorial System is called ATEX (environment variable FIP_EDSYS) and teh queue required is email-wir on system3 : MSGFORATX_ATEX= DP:atex3 DQ:email-wir DC:atexeol 3. Input Queue spool/2smtp needs to be added 4. IPPOST needs to be running too (unless you change the FIPMSG DP/DQ to somewhere else) 5. Outbound Format is ATXMSG for IPGTWY/IPOUT/IP2WTR etc. Please check if this exists and has the correct setup The current version can scan up to 1000 Unixmail boxes. Where sections of FipHdr fields are required or changes to the output style, use keywords : fixed, partial, combie, optional, repeat, newdate and/or style. (see The SysA dmin manual for more information). They are normally specified : fixed:QZ 1234543 partial:QT ST,3,2,U,<,> combie:QY ep|na,(0000000)a option:QE ep,11,7,s repeat:QK XK,-,3 or repeat:QP PK,,4,#X style:QS XN,%.03d replace:QN NN abc=DEF def=GHI For outbound traffic, FipHdr fields used are : Mandatory DA holds one or more remote email addresses multiple addresses are separated by a comma. Can be overridden by 'newDA' Can be changed by 'forcedomain' Optional SA holds the sender's email address can be overridden by 'forcesender' (actual address) can be overridden by 'forcesender' (from field) can be defaulted by 'defsender' DI holds the Subject: field DJ holds the From: field ZSMTP_BINARY for sending Binary files ZSMTP_ENCODE holds the Content-Transfer-Encoding default to Content-Transfer-Encoding:quotedprintable ZSMTP_TYPE holds the Content-Type. no default ZSMTP_LOG will send a log of the SMTP transmission to this destination (for SMTP outbound traffic using the '-h' input switch only) ZSMTP_HOST IPaddress or hostname of the mail server if NOT the default (for SMTP only). Input Switches -------------- Input parameters are : all optional : -1 : Get once only then stop for POP3 or Receive only. default: spool Either -p (path) or -3 (pop3 server) must be used with this option This also incorporates -Q so no queu is scanned and no mail sent. -3 : Hostname where the pop3 daemon is running. default: none Note -p and -3 are mutually exclusive. -a : queue for non-text attachments default: none default is that attachments are passed through in text -A : destination for non-text attachments default: none default is that attachments are passed through in text -C : do NOT convert unprintables to "=XX" format default: do -d : do NOT delete each unix msg default: delete -D : display the conversation with the remote server default: no valid ONLY with the -1 for single shot; used for debugging troublesome connections -f : do NOT output the original filename as the first line of text default: output name -F : do NOT output the original sending system (SU) as the From field default: From: SU -h : Hostname where the smtp daemon is running. default: that system -H : do NOT add an extra Mime Headers as the file is in the correct format default: add To/From/Subject -i : Receive spooled queue if you are using 'sffmail' default: using Path -k : alternate host name default: none use this for checking primary/secondary for clustered systems where the actual hostname differs from the cluster name ie if the host name may be fipcluster-a or fipcluster-b but there is a cluster hostname of fipcluster -K : quiet mode - message if NOT the primary system default: do not message -l : do NOT log each msg in or out default: log -m : (ipchkmail folder name) default: process inside ipsmtp Do not convert incoming SMTP and POP3 files immediately but pass them to ipchkmail runing on this folder which does a better job! especially for attachments-within-attachments -o : Next fip queue for incoming files default: spool/2go -O : Done queue for spooled files out default: none -p : Path on this system holding mailboxes default: depends on the OS Unix only. The full pathname MUST be stated. Note -p and -3 are mutually exclusive. -q : Send Queue to spool default: spool/2smtp -Q : Do NOT send any mail and therefore do NOT scan any input queue default: scan -r : eolns for the data part should be CR NL default: just NL -R : do NOT resolve hostnames default: do This will resolve any hostnames in the DEST_REDUN file (if there is one) -S : do NOT split files on the "From ..." line default: do -t : scan time of queue default: 3 secs -T : log the trace of each transaction default: do not Normally a one line log of each file sent is stored in log/remote_trace with a name of 'date_Address'. -w : for file->Sendmail, set line length. default: 76 + translate and do NOT translate unprintables to "=XX" format To NOT wrap text and no translations -w 0 This can be ignored for a single file if it has the FipHdr field ZSMTP_BINARY: -W : for files outbound, set JUST the line length def: 76 + translate but still translate unprintables. -z : parameter file default: mail/SMTP -v : display version number and exit. Now a statement of sanity (?) about incoming mail from the Internet : If like most of the civilised world you prefer to keep well away from anything 'sendmail' can throw at you, then let ipsmtp just grab its files from the sendmail internal queue. Or even better use POP3 and forget about it all - no setup, no nothing extra! If you are unlucky enough to have a Sendmail specialist on your staff, before sending for the men in white coats, point out there is a '-p path for unix mailboxes' switch. Your guru should dictate (they always dictate - though they claim they are working with your best interests in mind) where he is willing to drop files for those logons you are interested in and add the -p switch to the tables/sys/SYSTEM file. Easy huh ! Note there was a subtle change at version 15 which changed the default from ATXMSG to FIPMSG. Version Control ;016v15 09nov01 cleanup after failed connections better ;a 16mar02 added remote_trace and -O done queue ;b-c 03jun03 cleanup line-enders in mimehdr and streaming for smtp and added ZSMTP_HOST ;d-i 22sep03 added send-extra-fiphdr: and stream-connections:yes ;j-l 29jun04 added -1 and speedy ;m-p 17sep05 added alternate-smtp-host and hostname: and log-line ;q 10aug06 added remote_trace style timings to send file done msg as H4 ;r-t 21sep06 allow quotes in email address and wrap long subject lines ;u 13apr07 added extra-mime-headers: ;v4 06feb08 added hash-in-address ;2 modette for size of email list ;3-4 added check-primary-host for POP3 and chkmail-folder: ;5-6 bugette with send_smtp and logging long DA ;7 3sep10 added use-sx ;8-9 10nov10 bug in pop3 when Uidl has non-alnum ;10-11 19may11 bugette when TLS does not work and b64 '=' padding ;12 22aug13 added POP3 error message on logon/pwd and added -1 for display ;13-14 30sep13 added hash-in-fiphdr ;15 30apr14 file-trace (copyright) 2014 and previous years FingerPost Ltd.