nntpwire (Sat Oct 25 2014 01:31:01)

nntpwire

This periodically attachs to, checks for and grabs new articles in groups on a
remote NNTP server.

The whole docuement is then left, normally, in spool/xsmtp for 'ipchkmail' to
pull apart the Header etc.

The parameter file, normally tables/wire/NNTP, is read for the the names of all
the groups to scan for.
	; comment
	logon: (logon on the remote server)		default: none
	password: (password on the remote server)	default: none

	group: (group name on the remote NNTP server)
		sendto:(newaddress)
		fiphdr:(FIP hdr sequence)
	where	sendto allows you to specify another name for the DA field
			IPPOST will use this to route. By default
			the Fip Hdr field DA will hold the logon name.
		fiphdr allows you to add to the FIP hdr. This may be any
			- perhaps the DU field to change the destination.

		eg	group:ap.online.newsbriefs
			group:ap.agate.sports.all

		There can be up to 1000 groups specified.

Optional
	grab-every:(seconds)
		Connect, logon and check for news every X seconds.
		The default is 600 seconds (5 mins) while the minimum 
		is 5 seconds.
		The '-t' input switch can also be used.

	list-groups: (Fip Destination)
		This will list the groups available when the program logs on the
		first time and once a day at midnight.
		The resulting file is send to the destination specified.
		This file is always sent to spool/2go as it does NOT have
		a mime-header.

	proxy:(name of Proxy server)				default: none
	proxy-port: (only if a proxy server is specified)	default: 23
	proxy-is-squid: (yes/no)				default: none
		Please see note below

	defdest: (default Fip Destination (DU FipHdr field)	default: "nntp"
	chrset:	(Source character set ie SC header field)	default: ascii
	response-timeout: (timeout in seconds wanting for the remote to reply)
								default: 60 secs
	connection-timeout: (timeout in seconds wanting to connect to the remote)
								default: 120 secs
	extra-fiphdr: (more FipHdr information to add)		default: none

--------- Running redundant nntpwire on two Fip systems
	check-primary-server: pseudo-host name that is specified in
tables/sys/DEST_REDUN
		that is used whether the current host should be getting the files or not.
		ie	check-primary-server:remotewire
		and in the DEST_REDUN is
			; psuedohost	primary secondary
			remotewire	fip1	fip2
		Then if fip1 is up, the 'nntpwire' on fip1 will always get while on fip2 it
will just check/loop.
			if fip1 is down, the 'nntpwire' on fip2 will start getting.
	current-balance-group: name of a balance group (in tables/sys/BALANCE) to
distribute the current totals file when changed (see doc on 'ipbalan')
		This is often used where a second system could be used as a redundant server
if the main system fails.


Where sections of FipHdr fields are required or changes to the output style,
use
 keywords : fixed, partial, combie, optional, repeat, newdate and/or style.
(see The SysA
dmin manual for more information).

	They are normally specified :
		fixed:QZ	1234543
		partial:QT	ST,3,2,U,<,>
		combie:QY	ep|na,(0000000)a
		option:QE	ep,11,7,s
		repeat:QK	XK,-,3
	or	repeat:QP	PK,,4,#X
		style:QS	XN,%.03d
		replace:QN	NN	abc=DEF def=GHI
		newdate:QT	hours+3	"ZD"

Input switches are :
Mandatory :
	-s : Hostname where the NNTP is running.	default: none
Optional :
	-1 : one single pass and then stop		default: continuous
	-b : start at this id				default: get all files
		use with care !  Only valid with the -1 switch and a single group
	-B : ignore missing articles			default: abort on a missing article
		see below for a small note on the AP NNTP server.
	-D : display the conversation with the remote server	default: no
		valid ONLY with the -1 for single shot
		used for debugging troublesome connections
	-e : end at this id				default: get all files
		use with care !  Only valid with the -1 switch and a single group
	-l : log every file throught			default: no logging
	-n : name of the service		def: name of the parameter file
	-o : Next fip queue for incoming files		default: spool/xsmtp
	-p : port number on the remote host		default: 119
	-t : sleep in seconds between connections/accesses	default: 600 secs
	-z : parameter file				default: wire/NNTP
	-v : display version number and exit.

-- Why would you want to use the the -B switch ?

Some NNTP servers - AP's for example - do not give a proper first-id of a
group.

ie they might say there are 877 files in a group and the last one is 416998 -
but the first is 512.

So when 'nntpwire' starts at 512, it errors with '423 Bad Article Number'

So use the -B switch (with the -1 single) to ignore these spurious errors and
drain the group manually before letting into production.

-- Squid as Proxy

If using Squid, 'nntpwire' uses CONNECT, so there will need to be a mod to
squid.conf as NNTP is not on the default list of allowed ports for Squid.
Eg
- 1. quick fudge
If you are 'almost' using the default squid.conf, the only ACL that allows
CONNECT is 'SSL_ports'.
So just add port 119 on the end of the line :
	acl SSL_ports port 443 563
becomes	acl SSL_ports port 443 563 119

- 2. slightly more proper is to add a new ACL ...

add these
	# create a new acl called nntp
	acl nntp port 119		# nntp
	# add to Safe_ports too
	acl Safe_ports port 119		# nntp
	# Strip all HTTP headers from remote - not sure you really need this but ..
	header_access All deny nntp

change this
	# Deny CONNECT to other than SSL ports
	http_access deny CONNECT !SSL_ports
to this
	http_access deny CONNECT !SSL_ports !nntp

plus check you have the defaults
	acl CONNECT method CONNECT
	# Deny requests to unknown ports
	http_access deny !Safe_ports


(copyright) 2014 and previous years FingerPost Ltd.