iptimer to get by ftp

Using iptimer to get files by ftp.

This is a slightly more complex copy flow than ftp-get to ftp-send,for getting files from a number of sites via ftp and delivering them to an external destination – in this case by ftp again.

Iptimer is used to run a task normally ipftp, httpwire at timed intervals.

; Timer file for 'get' files from Group output servers.
group:evouchers
track-status:yes

client:get1 url:111.22.333.11 display: type:f template:genericgetandzap match:*.pdf 
ignore:/DIRECTORY/TO?GET/FILES/FROM fiphdr:QD:cwd_q days:M.Tu.W.Th.F.S.Su time: start:16:00 end
:23.59 every:10 logon:xxx passwd:yyy dest:delivery

client:get2 url:111.22.333.12 display: type:f template:genericgetandzap match:*.pdf 
ignore:/DIRECTORY/TO?GET/FILES/FROM fiphdr:QD:cwd_q days:M.Tu.W.Th.F time: start:23:00 end
:23.59 every:15 logon:xxx passwd:yyy dest:delivery

The template used for the ftp is a file called /fip/tables/ftp/genericgetandzap. This is a generic ftp template, that takes the fields given above and plugs them into the ftp parameter file on the fly.

The fields map as follows:

  • url=W0
  • match=W1
  • ignore=W2
  • logon=W6
  • dests=W9
  • password=W7
; 'GENERICGET' -
; This is a generic ftp parameter file that runs ftp gets based on 
; information given by iptimer.

; See /fip/tables/setup/TIMER.*

; ----------------------------------
; which system should do the getting ?
; see sys/DEST_REDUN
check-primary-server-for-getfiles:pdfgetter
; ----------------------------------

; Remote Host - URL/IP Address
remhost:\W0

; Remote Host - Logon
logon:\W6

; Remote Host - Password
password:\W7

;;cd beforehand ??
option:V2		 W2
ftpbefore:\V2fipon-error:abort\$o
ftpbefore:\V2cwd \W2\$o

; Get all files
get-and-zapall:\W1
;;get-and-zapall:*.pdf
;;get-allfiles:\W1

; add some extra fipHdr fields
get-extra-fiphdr:SU:\WN\nQU:\WN_\WG#\W3

; Output file destination
dest:\W9

; Different Output Queue ?
option:V3		 WQ
fixed:V4		  WQ,2test
outque:\V4


; Create a skip list
;skip-files:\WN.\WG

; Log file
logfile:/fip/log/ftp/\WN.\WG.\$D

Links

* iptimer
* ipftp

Example files

IpTimerFtpExample parameter files – getandzap and getandmove with iptimer