ftp-in to ftp-out

Simple ftp get to ftp delivery

This is a very simple copy flow. Files are simply got from a number of internal servers, and dropped into a folder for further ftp’ing on to the external site of a company called ‘DELIVERY’.

It is suitable for a small and relatively stable number of local sites where we need to fetch the data regularly and deliver it as soon as we have got it.

If you have a larger number of similar sites, and would like more complicated scheduling a copyflow using iptimer to trigger the ftp gets might be more appropriate. See IptimerFtp for a means of running this type of copy flow.

In the /fip/tables/sys/SYSTEM file

get1	  local	ipftp -G 600 -z MYFIRSTGET -Z -o 2delivery -q
get2	  local	ipftp -G 600 -z MYSECONDGET -Z -o 2delivery -q
get3	  local	ipftp -G 600 -z MYTHIRDGET -Z -o 2delivery -q
delivery local	ipftp -i 2delivery -d 2delivery_done -z 2delivery -Z 

For the getting:
The syntax

get1	 local	ipftp -G 600 -z MYFIRSTGET -Z -o 2delivery -q

means run an ftp get every 10 mins (600 secs), using the parameter file /fip/tables/ftp/MYFIRSTGET and deliver the files to /fip/spool/2delivery. Be quiet about (-q) occasions when you ftp to the site and do not find any files there.

In the parameter file /fip/tables/ftp/MYFIRSTGET you then specify the server, logon and password, andy special instructions (like a cd before the file) and any particular logging requirements. You may also wish to add some header information at this point.

; ----------------------------------
; which system should do the getting ?
; see sys/DEST_REDUN
check-primary-server-for-getfiles:pdfgetter
; ----------------------------------
; where are we getting it from ?
; Remote server - Name/IP Address
remhost:172.100.100.6

; Remote server - Logon
logon:logon

; Remote server -Password
password:password

; Filename on remote server
newname:\EN

; Commands to execute at the beginning of the session
; Note fipdelays to slow things up for NT
ftpbefore:fipdelay
; exit if the directory doesn't exist
ftpbefore: fipon-error:abort
ftpbefore:cwd /Place/To/PickFiles/Upfrom

; Files to get ...
get-and-zapall:*.pdf

; Commands to execute before sending each file - especially important 
; if fetching from a windows server
ftpbeffile:fipdelay

; destination - this isn't used as the files are put straight
; into the outbound q,  but I have forced it anyway
; as otherwise the default is woops which looks distracting in the logs
dest:pdfpurchaser

; Add a little fip header information so that we can improve the logging ...
get-extra-fiphdr:\nQ1:MYFIRSTGET\nQ2:\W0\nQ3:\WN\n

; Log file name
; NB - may need to change this to reflect the source of the files
logfile:/fip/log/ftp/MYFIRSTGET.\$D\$M\$E\$Y

; *** End ***

This will therefore get any files on the remote sever in /Place/To/PickFiles/Upfrom tand delivery them into /fip/spool/2delivery where they would site except for the fact that:

2delivery	  local	ipftp -i 2delivery -d 2delivery_done -z 2delivery -Z 

is monitoring the 2 delivery queue. This process will ftp the files according to a parameter file /fip/tables/ftp/2delivery, dump the original copies in /fip/spool/2delivery_done. Files are delivered into a directory on the remote site called ‘special’.

The parameter file /fip/tables/ftp/2delivery contains:

; '2delivery' -
;
; Delivery of pdfs send to the delivery ftp server
; These pdfs are fetched from various internal sites and 
; then sent on to 'delivery'

; NB there is a primary and a secondary server
; so a pseudohostname is used, which is referenced to the actual servers 
; in /fip/tables/sys/DEST_REDUN

; Lucy 7th February 2005

; Remote server - Name/IP Address
remhost:133.122.211.97

; Remote server - Logon
logon:delivery_Logon

; Remote server -Password
password:secret

; Filename on remote server
newname:\SN

; Commands to execute at the beginning of the session
; Note fipdelays to slow things up for NT
ftpbefore:fipdelay
ftpbefore:mkd special
ftpbefore:fipdelay
ftpbefore:cwd special
ftpbefore:fipdelay

; in case there is a file with the same name
; Commands to execute before sending each file
ftpbeffile:dele \SN
ftpbeffile:fipdelay
ftpbeffile:fipdelay

; Log file name
; NB - may need to change this to reflect the source of the files
logfile:/fip/log/ftp/DELIVERY.\$D\$M\$E\$Y

; Minimal logging - logs success and failure only - 
; comment out if you want wordy session debugging
;;minimum-log:

; *** End ***

Other things needed:

1) Purging of 2delivery_done.

This would normally be done with the nightly maintenance and can be done by adding a line to /fip/local/zapfiplog.

# delete ‘delivery’ files older than 4 days ..

if [ -d /fip/spool/2delivery_done ]; then
		  /fip/bin/ipdelque -q /fip/spool/2delivery_done -m 4 -i 1
fi

2) Entry in /fip/tables/sys/DEST_REDUN to determinw which of a pair of machines is the primary ‘getter’.

This enables you to have these system file entries set up and running on two machines, wna only the primary machine actually do any getting. The secondary machine will remain passive until the primary machine is down.

; 'DEST_REDUN' - Destination redundancy
;
;					Primary			Secondary
; Name			 system			 system
; ----			 -------			---------
remotesvr		 jj-dist1		  jj-dist2
pdfgetter		 jj-dist2		  jj-dist1

–Links
* [[Ipftp][ipftp]]