nntpwire

nntpwire

This periodically attachs to, checks for and grabs new articles in groups on a
remote NNTP server.

The whole docuement is then left, normally, in spool/xsmtp for 'ipchkmail' to
pull apart the Header etc.

The parameter file, normally tables/wire/NNTP, is read for the the names of all
the groups to scan for.
    ; comment
    logon: (logon on the remote server)     default: none
    password: (password on the remote server)   default: none

    group: (group name on the remote NNTP server)
        sendto:(newaddress)
        fiphdr:(FIP hdr sequence)
    where   sendto allows you to specify another name for the DA field
            IPPOST will use this to route. By default
            the Fip Hdr field DA will hold the logon name.
        fiphdr allows you to add to the FIP hdr. This may be any
            - perhaps the DU field to change the destination.

        eg  group:ap.online.newsbriefs
            group:ap.agate.sports.all

        There can be up to 1000 groups specified.

Optional
    grab-every:(seconds)
        Connect, logon and check for news every X seconds.
        The default is 600 seconds (5 mins) while the minimum 
        is 5 seconds.
        The '-t' input switch can also be used.

    list-groups: (Fip Destination)
        This will list the groups available when the program logs on the
        first time and once a day at midnight.
        The resulting file is send to the destination specified.
        This file is always sent to spool/2go as it does NOT have
        a mime-header.

    proxy:(name of Proxy server)                default: none
    proxy-port: (only if a proxy server is specified)   default: 23
    proxy-is-squid: (yes/no)                default: none
        Please see note below

    defdest: (default Fip Destination (DU FipHdr field) default: "nntp"
    chrset: (Source character set ie SC header field)   default: ascii
    response-timeout: (timeout in seconds wanting for the remote to reply)
                                default: 60 secs
    connection-timeout: (timeout in seconds wanting to connect to the remote)
                                default: 120 secs
    extra-fiphdr: (more FipHdr information to add)      default: none

--------- Running redundant nntpwire on two Fip systems
    check-primary-server: pseudo-host name that is specified in
tables/sys/DEST_REDUN
        that is used whether the current host should be getting the files or not.
        ie  check-primary-server:remotewire
        and in the DEST_REDUN is
            ; psuedohost    primary secondary
            remotewire  fip1    fip2
        Then if fip1 is up, the 'nntpwire' on fip1 will always get while on fip2 it
will just check/loop.
            if fip1 is down, the 'nntpwire' on fip2 will start getting.
    current-balance-group: name of a balance group (in tables/sys/BALANCE) to
distribute the current totals file when changed (see doc on 'ipbalan')
        This is often used where a second system could be used as a redundant server
if the main system fails.

Where sections of FipHdr fields are required or changes to the output style,
use
 keywords : fixed, partial, combie, optional, repeat, newdate and/or style.
(see The SysA
dmin manual for more information).

    They are normally specified :
        fixed:QZ    1234543
        partial:QT  ST,3,2,U,<,>
        combie:QY   ep|na,(0000000)a
        option:QE   ep,11,7,s
        repeat:QK   XK,-,3
    or  repeat:QP   PK,,4,#X
        style:QS    XN,%.03d
        replace:QN  NN  abc=DEF def=GHI
        newdate:QT  hours+3 "\ZD"

Input switches are :
Mandatory :
    -s : Hostname where the NNTP is running.    default: none
Optional :
    -1 : one single pass and then stop      default: continuous
    -b : start at this id               default: get all files
        use with care !  Only valid with the -1 switch and a single group
    -B : ignore missing articles            default: abort on a missing article
        see below for a small note on the AP NNTP server.
    -D : display the conversation with the remote server    default: no
        valid ONLY with the -1 for single shot
        used for debugging troublesome connections
    -e : end at this id             default: get all files
        use with care !  Only valid with the -1 switch and a single group
    -l : log every file throught            default: no logging
    -n : name of the service        def: name of the parameter file
    -o : Next fip queue for incoming files      default: spool/xsmtp
    -p : port number on the remote host     default: 119
    -t : sleep in seconds between connections/accesses  default: 600 secs
    -z : parameter file             default: wire/NNTP
    -v : display version number and exit.

-- Why would you want to use the the -B switch ?

Some NNTP servers - AP's for example - do not give a proper first-id of a
group.

ie they might say there are 877 files in a group and the last one is 416998 -
but the first is 512.

So when 'nntpwire' starts at 512, it errors with '423 Bad Article Number'

So use the -B switch (with the -1 single) to ignore these spurious errors and
drain the group manually before letting into production.

-- Squid as Proxy

If using Squid, 'nntpwire' uses CONNECT, so there will need to be a mod to
squid.conf as NNTP is not on the default list of allowed ports for Squid.
Eg
- 1. quick fudge
If you are 'almost' using the default squid.conf, the only ACL that allows
CONNECT is 'SSL_ports'.
So just add port 119 on the end of the line :
    acl SSL_ports port 443 563
becomes acl SSL_ports port 443 563 119

- 2. slightly more proper is to add a new ACL ...

add these
    # create a new acl called nntp
    acl nntp port 119       # nntp
    # add to Safe_ports too
    acl Safe_ports port 119     # nntp
    # Strip all HTTP headers from remote - not sure you really need this but ..
    header_access All deny nntp

change this
    # Deny CONNECT to other than SSL ports
    http_access deny CONNECT !SSL_ports
to this
    http_access deny CONNECT !SSL_ports !nntp

plus check you have the defaults
    acl CONNECT method CONNECT
    # Deny requests to unknown ports
    http_access deny !Safe_ports

(copyright) 2017 and previous years FingerPost Ltd.