ampswire
ampswire
This periodically attachs to, checks for and grabs new articles in groups on a
remote AMPS server using WNDP
The whole docuement is then left in (outputfolderstub)(fileextension). The
default folder stub is 'xamps_', so :
txt files will be left in /fip/spool/xamps_txt
xml files will be left in /fip/spool/xamps_xml
jpg files will be left in /fip/spool/xamps_jpg
htm files will be left in /fip/spool/xamps_htm
The parameter file, normally tables/wire/AMPS, is read for the the names of all
the groups to scan for.
; comment
logon: (logon on the remote server) default: none
password: (password on the remote server) default: none
class: Class of data default: ALL
see the amps doc for a full description
includes XMLNews-story
ANPA
IPTC
HTML
MarketData
ALL
KNRIDDER-NITF
PRNewswire-Raw
wndp-version: Version of WNDP default: 3
xmlgen-version: Version of XML default: 1
include-filter: (service name) default: ALL
exclude-filter: (service name) default: none
There can be up to 1000 filters specified.
force-output: yes/no default: no
force all output to the output folder
default is use the output folder name as a stub and append the file 'ext'
eg if the file is 'txt' and output:xamps_
output folder is xamps_txt
in this case NO extension is added !
process-additions:ignore/process/file default: process
AMPS does not usually send duplicate data files but sends a '.add' file which
denotes the original one sent plus new metadata.
ignore - just ignore all references
process - find the original and send that with the new metadata
file - treat '.add' messages as just other file
Note that the 'process' option uses /fip/data/amps and this should be purged
nightly for anything over 4 or 5 days (check with the AMPS setup how far back
it might go to)
extra-fiphdr-ext:(ext) (FipSeq to add) default: none
add this extra fiphdr information to each file with this file extention.
extra-fiphdr-ext:TXT #SR:ALLPA
extra-fiphdr-ext:XML #DY:RIPOUTTXT.XML
default-ext:(ext) default: none
if the incoming file has an extention which is NOT mapped by an existing
extra-fiphdr-ext, use this one.
It also will leave it in the xamps_(ext) folder rather than making a new
folder.
extra-fiphdr-class:(ext) (FipSeq to add) default: none
add this extra fiphdr information to each file with this file extention.
extra-fiphdr-class:JPG #AB:JPEG
round-robin: (number) default: none
round-robin-fiphdr: (2 letter FipHdr field) default: none
Round-Robin the output files and add the RR number to the fipHdr.
Both parameters are required - the Number is the MAXimum.
eg round-robin:9
round-robin-fiphdr:RR
and a suitable output folder might be /fip/spool/xchg\RR
Commons extensions are
TXT text - anpa or iptc
XML xml variants; prnews
JPG pix jpegs
HTM html files
Optional
chrset: (Source character set ie SC header field) default: ascii
response-timeout: (timeout in seconds wanting for the remote to reply)
default: 60 secs
connection-timeout: (timeout in seconds wanting to connect to the remote)
default: 120 secs
extra-fiphdr: (more FipHdr information to add) default: none
current-balance-group: name of a balance group (in tables/sys/BALANCE) to
distribute the current totals file when changed (see doc on 'ipbalan')
This is often used where a second system could be used as a redundant server
if the main system fails.
log-line: (FipSeq)
Custom log line for files in (ie depends on -l 20)
eg log-line:\S2 \SF (\S1) \SA -> \SN
Fiphdr fields for each incoming file are :
SN - filename
SU - from the '-n' input switch or AMPS
SC - 'ascii' or chrset parameter
SA - route from the request
ST - class from the request
SE - file extention
S1 - size from the request
S2 - data/time from the request
timing-stats:yes/no
Show timings statistics default: no
Where sections of FipHdr fields are required or changes to the output style,
use keywords : fixed, partial, combie, optional, repeat, newdate and/or style.
(see The SysAdmin manual for more information).
They are normally specified :
fixed:QZ 1234543
partial:QT ST,3,2,U,<,>
combie:QY ep|na,(0000000)a
option:QE ep,11,7,s
repeat:QK XK,-,3
or repeat:QP PK,,4,#X
style:QS XN,%.03d
replace:QN NN abc=DEF def=GHI
newdate:QT hours+3 "\ZD"
Input switches are :
Mandatory :
-s : Hostname where the AMPS is running. default: none
Optional :
-A : Processing ADD - additions default: process
ignore - just delete them
process - copy the original data and add the new routing, send as a new file
file - just send like a new file
-d : done folder default: none
This holds a copy of all incoming data files from every source
The structure is
(done folder) / (date)_(logon) eg 20110921_fip / (filename as written to the
output folder)
It can be purged with an entry in maintenance (zapfiplog)
eg if '-d raw.amx' and we want the last 30 days data
/fip/bin/ipdelque -q/fip/spool/raw.amx -i1 -a30
-D : display the conversation with the remote server default: no
used for debugging troublesome connections
do NOT add in the SYSTEM file
-F : Force the output folder to the '-o' and do NOT add the file type
extension
-I : id of this instance default: ignored
Where there are several copies of 'ampswire' running with the same -n
-l : log level default: 10 for connect/disc messages too
-l 0 errors only
-l 10 plus connect/disconnect
-l 20 plus each file
-n : name of the service def: AMPS or name of -z param file
-o : Next fip queue for incoming files default: spool/xamps_ZZZ
the file type is added as the last 3 letters of the output folder
-p : port number on the remote host default: 39030
-V : major version default: xmlgen
-V wndp for wndp
-V xmlgen for xmlgen
-z : parameter file default: wire/AMPS
this file MUST exist - ut can be a single comented line.
-v : display version number and exit.
-- Copy Flow
For text files which are ANPA or IPTC, spool the output folder with ipdskwir :
eg a snippet from the SYSTEM file
; ampswire using default parameter file wire/AMPS and default folders -
xamps_txt
amps local ampswire -s localhost
; txt files - process and route as normal
atxt local ipdskwir -i xamps_txt -n AZMEDIA
; jpgs - move to pix system directly
apix local ipftp -i xamps_jpg -n AZPIX2EDIT
-- Maintenance
The zapfiplog entry if using the process-additions:process option (which is the
default) :
If the -n name is different to AMPS, then change the 2nd 'amps' in the 2
strings
if [ -d /fip/data/amps/amps ]; then
/fip/bin/ipdelque -q /fip/data/amps/amps -A 2 -i 0
fi
Version Control
;1b19a 29mar05 original version
;a3-5 added process-addition
;b1-8 better XML ;9-10 added round-robin ;11 added -d ;12 bugette in -d
;b13-14 added extra-fiphdr-class and better ext ;15-16 -I wireId
;b17-19 6jan16 added default-ext
(copyright) 2024 and previous years FingerPost Ltd.