ftpwire FTPWIRE is a very simple FTP daemon. Use 'ipftp' when you need to send data using FTP - or Grab data from a remote FTPd site. Use 'ftpwire' when a remote host needs to send to the Fip. The file is slotted into the spool/2brouted queue for IPROUTE to process and route. To send to ftpwire, use the normal 'ftp' program : ftp -P 9130 (hostname of the Fip) or ftp hostname 9130 FTP uses 2 ports - one for control and the other for data. In normal FTPese, these are 21 and 20. 'ftpwire' use the -P input switch to define the Control port. If it is started with port 21, then it will use port 20 as a fixed port to send data from/to. If the parameter 'fixed-data-port' is set, that number is used. Otherwise the dataport is just the first available port. Remember ports 20 and 21 are reserved ports on Unix/Linux and 'root' must start any program using a port < 1024 (and you must make sure ftpd in inet.d or xinet.d is disabled or on a different port) If using WIN2K, you MUST specify a unique wireid using the -I input switch and specify several instances of ftpwire in the SYSTEM file for the number of instances you want. If using Unix/Linux, only one instance of ftpwire should be in the SYSTEM file and the -E 99 switch is used to determine the number of simultaneous inputs. One useful builtin is merging FipHdrs - if the incoming file has a FipHdr, then any 'extra-fiphdr' info, datetime FipHdrs, system FipHdrs are all merged. -- Optional parameter file is under tables/wire and the '-n' input switch and defaults to FTPWIRE. A '.FIP' extension may be added. Syntax : ; comment line banner: Replacement banner outque: (folder name in FipSeq) Output folder - this overrides the -O input switch logon-file:(filename) See below - default LOGON.(-n) or LOGON.FTPWIRE.FIP allow-blank-pwd: (yes/no) If there is a blank password in the logon file, accept ANY password. default-no password-is-encoded: (yes/no) Password in normal logon file or w4 lists is encoded (default: no) use-w4-logon-list:(name eg FIPO.EXTERNAL) Use this Fip w4 LogonList file for authentication and ignore all other auth methods such as LOGON.FTPWIRE.FIP w4-logon-restrict-templates:(name(s) eg RADIUS_FTPUSER,RADIUS_FTPADMIN) If using LogonList, only allow logons with this template(s) w4-logon-restrict-pubs:(pub name(s) eg PIF,BIG,DAILY) If using LogonList, only allow logons with this pub code w4-logon-topq: w4-logon-outq: w4-logon-copyq: w4-logon-fiphdr: w4-logon-options: if using LogonList, use these default values (see Logon below for fuller explanation of each) use-etc-passwd: yes/no (Linux/Unix only) - if YES, use the /etc/passwd file for authentication and ignore all other auth methods such as LOGON.FTPWIRE.FIP default: no external-path: (path) for etc/passwd, Only allow logons with a home folder starting with this path eg external-path:/home/ftp expects all the home folders below /home/ftp default: nothing specified for all folders external-shell: (shell) For /etc/passwd, Only allow logons that run this shell external-shell:/bin/nologin default: nothing specified for all shells external-min-uid: (Uid number) For /etc/passwd, Only allow logons with Uids higher than this number to logon Stop anyone logging on a root ! default: 400 external-min-gid: (Gid number) For /etc/passwd, Only allow logons with Group ids higher than this number to logon default: none nat-pasv-address:(Ip address) if using a proxy and/or NAT addresses, this is the address of THIS host to the outside world. It is used in the PORT command extra-fiphdr: (fipseq) Additional FipHdr info to be added to each incoming file. default none extra-fiphdr-ext: (fipseq) Additional FipHdr info to be added to each incoming file if the file has this extension. default none eg if a file is abc1.jpg extra-fiphdr-ext:jpg SR:FTP_JPEGS replace-space: (fipseq) Replace a space in the filename or folder with this chr. default: SPC (see also allow-spaces:) replace-hash: (fipseq) Replace hash chr in the filename or folder with this chr. default: '#' replace-unsafe: (fipseq) Replace control and meta chrs in the filename or folder with this. default: '-' Ftpwire will make a filename 'safe' for the system by cleaning meta chrs '/' FipHdr ZO contains the safe filename and SN is the 'given' no-archive: do NOT archive the data in the daily archive files in log/data (ZI: fiphdr field) display-log (yes/no/file/logon) (same as -D) Display all commands for each connection default: no Use this to trace problem connections Option 'file' will log the transactions in a dated file in log/ftp Option 'logon' will log the transactions in a dated file for that logon in log/ftp hourly-logs: (time) Remote Trace log files are normally daily (/fip/log/remote_trace/(date) Use this to add a hour extension (only 60 for 60 mins is currently valid) (can be overriden by 'J' or '-J' in the LOGON file for each logon) cwd-code:200 CWD response code MUST be a 3 digit number between 200 and 299. There is some confusion whether the response should be 257 or 200. default: 257 msg-215: String to replace the normal 215 response of 'UNIX Type: L8 Euripides 99' pwd-quote: (' " or a space) Quote for response to a PWD command. Valid options are doublequote ", singlequote ' or a space default: " allow-relogon: (yes/no) Set this to NO to inhibit relogon in the same session default:yes add-list-dots:yes/no Add the two folders for '.' and '..' for a long list default: no force-folder-names:lower/upper/nochg Force the case of any request for folder names default: nochange force-file-names:lower/upper/nochg Force the case of any request for file names default: nochange Use these to handle Win2k <->Linux case issues - where a case-INsensitive client is talking to a case-SENS servers One suggestion is to force all folders and files lowercase and set both of these ..:lower force-passive-mode: (yes/no) By default FTP connections are 'active'. But most Firewalls will only allow 'passive' connections nowadays. So you can request the remote client to go passive using 'force-passive-mode:yes'. In this case the 2 FTP commands which attempt to pass data PORT and EPRT are turned into errors with a message telling the remote site to turn PASV/PASSive on. balance-group; (Balance Group name) Balance group for balancing doneque items default: none / no balancing This group MUST be in sys/BALANCE balance-folder: (folder under spool) Balance queue for balancing doneque items default: 2balance move-on-read-group: (Balance Group name) Balance group for redundant move-on-read default: none / no balancing This needs a move-on-read-folder to be specified move-on-read-folder: (FipSeq name of a folder) Name of folder for files once read default: none default-move-on-read: (yes/no) Default for Move on Read default: no min-data-port: (number) max-data-port: (number) minimum (default is 32000) and maximum (def 64000) range of part number for the DATA port fixed-data-port: (number) A single, fixed data port number. Default is port 20 if the Control port is set to 21 (-P 21), otherwise it is a high port within the range specifiied by the min and max (see above) linger-on-close: Timeout in secs for the data to be sent for Passive connections For VERY slow connections, increase this already-enormous number, default: 20 (secs) to allow up to 20 secs max before cutting the call connection-retries: (number) default 5 send-timeout: (number) default 60 secs recv-timeout: (number) default 60 secs connection-timeout: (number) default 20 secs session-timeout: (number) default 1200 secs (20 mins) of no activity between-files-timeout: (number) default 1200 secs (20 mins) of no activity round-robin: (number) default: none round-robin-fiphdr: (2 letter FipHdr field) default: none round-robin-offset-fiphdr: (2 letter FipHdr field) default: none Round-Robin the output files and add the RR number to the fipHdr. Both parameters are required - the Number is the MAXimum. eg to leave the output in folder1 to folder9 round-robin:9 round-robin-fiphdr:RR round-robin-offset-fiphdr:RO and a suitable output folder might be /fip/spool/xchg\RR (This can be in the LOGON file or the default input switch -O xchg\\R (remember double backers) Note that the round-robin number is NOT added automatically to any output folder - ie you MUST specify a FipHdr as in /fip/spool/2xml\RR The round-robin-offset-fiphdr allows the RR number to be offset by the -R input switch which is the base offset - default 1 So if '-R 8' and 'round-robin:10', the output will be in (folder)8 to (folder)17 So WITHOUT the -R switch both round-robin-fiphdr and round-robin-offset-fiphdr will give the same number. allow: (IPaddress to allow) disallow: (IPaddress to block) use this for blacklist/whitelist certain addresses A '*' (star) can be used to indicate ALL eg 10.3.3.* disconnect-limit: (number) number of logon/password errors before connection is broken default: 30 unsuccessful attempts disable-limit: (number) number of logon/password errors before logon is blacklisted default: 30 unsuccessful attempts allow-site-fiphdr: (yes/no) see SITE FIPHDR below default: no allow-anon: (yes/no) Allow anonymous logons default-no anon-desc: For Anon logons, fill in the default logon fields.... anon-topq: anon-outq: anon-fiphdr: anon-standalone: anon-display: Override the default display setting (same as -D) timing-stats: (yes/no) generate Timing stats (default is now YES) save-data-path: (pathname for data) This puts the data of the incoming data in a file in this folder and creates a FipHdr file that contains 2 FipHdrs containing the full path/filename SX: and FTP_EXTERNAL_FILE: (ipbalan uses SX and ipftp uses FTP_EXTERNAL_FILE) eq save-data-path:/fip/data/jpegs/\$e\$y\$i\$d/ Use this for big files that you do not want to copy around the Fip Spool area. ** if specified, ALL non-standalone files will be split like this ** -- Logons and Passwords There are 3 (main) types of Authentication. - default - using a Fip pipe delimited file called LOGON_FTPWIRE.FIP (or the parameter of the '-n' input switch. - for Unix/Linux, using the normal /etc/passwd file - using the Fip w4 LogonList file The following describes the parameter file syntax for the default. Please see the relevant (external) doc for the others. The types of logon/password are - anonymous logon use input switch -A to allow (disallowd by default) use parameters to optionally add more information anon-desc - just a note for logging anon-fiphdr - extra fiphdr to add to each file anon-topq - top folder for LIST and GETs default is LISTs and GETS are not allowed anon-copyq - folder holding a copy of any incoming file anon-curq - under - full logon and password (normal running) - logon and allow any password - just leave the password blank and add -B The Logon file is in tables/wir and is called LOGON.(name) where name is the '-n' switch or FTPWIRE.FIP by default fields in the Password file are pipe delimited and are : field 0 Enabled or Disabled flag E/D 1 LogonName 2 Password 3 last mod time (used by the user interface only) 4 Description/Comment 5 optional home folder which is revealed as '/' to the remote if this is blank, then LS and GETs are returned as 'no such folder/file' 6 Output folder for any incoming files. If it does NOT start with a '/', the folder is under /fip/spool This can be in FipSeq If blank, the default output folder is used. 7 optional Copy folder where an exact copy of the incoming file this can be the same as 5-home folder if the remote needs to see the file 8 any optional ExtraFipHdr info 9 Option single letter Flags (Negate by adding a dash/hyphen prefix - ie to make sure files are NEVER deleted '-Z' S-standalone output file (original filename and no fiphdr) C-standalone copy file (original filename and no fiphdr) H-For Standalone copy, add a FipHdr D-display all commands for this client only (ie -D for this one client) F-allow extra FipHdr data to be added BEFORE the Store as a site command SITE FIPHDR #SU:ZIBBLE#CX:ZIBBLE2EDITO Z-allow delete of any Standalone Copy ('C') files T-allow sessionTimeout of 24 hours for this client (default is 20 mins or the session-timeout parameter) U-allow Last File Timeout of 24 hours for this client (default is 20 mins or the session-file-timeout parameter) M-allow client to MKDIR a sub folder R-allow client to RMDIR a sub folder G-allow client to GET a file (which is the default if 'S'tandalone or 'C'opy is on - so to Disallow, use '-G') Q-on a CD/CWD, check the folder really does exist J-set hourly logs for the remote_trace log file W-if in standalone mode, overwrite files X-Send on Rename - Files are held in the 5-HomeFolder until a rename when they are moved to the 6-Outputfolder 2 further suboptions in () are optional for Pre strings and Post strings (pre=[string in FipSeq]) and (post=[FipSeq string]) where [FipSeq string] is any parsable text ! - if a source sends files preceeded by _^_(filename) then X (pre=_^_) will detect these to be SendOnRename - if a source sends files with a '.tmp' extension and then renames them to '.xml' or someother file type, use X (post=.tmp) NOTE that any files WITHOUT either the pre or post strings will be sent immediately NOTE you cannot rename files if using V-virtual list V-Virtual list - use this to hold a list of files sent by the remote. LIST, SIZE and MDTM commands will show the files Only files from the sender IN THIS SESSION are shown - none from previous sessions or from any concurrent session. NOTE you cannot sendOnRename or rename files if using V-virtual list L-Logging options (sub options in following brackets) C - do NOT log connections/disc N - NewLogon eg E|Pittlewire|zong|0|Mr Pittles Image Agency||2edsys||#DF:PITTLEWIRE.FIP#EQ:pittle| E|Brittle||0|Mr Brittle HardHat Agency||xchg|#CX:B2FIP#DF:BRITTLE.FIP#EQ:brittle| -- FipHdr fields added to each file UNLESS the Standalone option has been flagged SP IPaddress of the remote host SN Filename given ZO Safe filename SU -n input switch or FTPWIRE SA logon name SC chrset - defaults to ASCII S1 client description for the logon file S2 wire id S3 session id S4 current PWD S5 any extra subfolder in a put command S6 current Client system details (if offered by remote) Input Parameters : Optional : -9 : do not use Speedy on a Speedy system -A : allow anonymous logons default: always need a logon/pwd -B : allow blank passwords default: always need a pwd if the password field is blank in the LOGON file, no checking takes place. -d : done folder default: none This holds a copy of all incoming data files from every source The structure is (done folder) / (date)_(logon) eg 20110921_fip / (filename as written to the output folder) It can be purged with an entry in maintenance (zapfiplog) eg if '-d raw.ftpwire' and we want the last 30 days data /fip/bin/ipdelque -q/fip/spool/raw.ftpwire -i1 -a30 -D : display all commands for each connection default: no Use this to trace problem connections -E : maximum number of threads default: 1 up to a max of 200 (not Win2k) Note this is also a hardware limit in that small systems may not be able to run more than 50 or so -F : allow SITE FIPHDR commands default: no -I : wire id default: 0 used to track which instance of a multi-ftpwire system a file arrived/logged -j : balance queue for balancing doneque items default: 2balance -J : balance group for balancing doneque items default: -none- no balancing This group MUST be in sys/BALANCE -l : log level -n : name of this wire default: FTPWIRE -O : Name of the output folder if not default default: spool/2brouted This folder will be under /fip/spool -P : Port for control default: 9130 -P 21 is normal -R : round-robin base - see above default: 1 -s : local hostname or ipaddress default: all local addresses where a server has multiple ip address/hostnames, use '-s' to restrict connections to a single address -S : default is standlone and not Fip default: next folder is fip in this case do NOT add a Fiphdr and preserve exactly the incoming filename this can be overridden in the Logon file -T : log timing stats default: no -v : Print the version number and exit -- Log levels for -l input switch are : default (-1) errors only are logged 0 connections/disconnections 10 logons 20 each file in or out -- The -D input switch will display all cmds etc as they come in eg Thu Jun 11 12:27:44 id.35 ++ New Connection FTPWIRE 22.214.171.124 w.0.id.35.tot.1 Thu Jun 11 12:27:44 id.35 Send.23 220 Fip FTP service~~| Thu Jun 11 12:27:44 id.35 Recv.10 USER zibbly| Thu Jun 11 12:27:44 id.35 Send.20 331 Enter password~~| Thu Jun 11 12:27:44 id.35 Recv.12 PASS *******| Thu Jun 11 12:27:45 id.35 Send.14 230 Logon ok~~| Thu Jun 11 12:27:45 id.35 Recv.3 PWD| Thu Jun 11 12:27:45 id.35 Send.7 257 /~~| Thu Jun 11 12:27:45 id.35 Recv.4 PASV| Thu Jun 11 12:27:45 id.35 Send.45 227 Entering Passive Mode (10,1,1,35,125,1)~~| Thu Jun 11 12:27:45 id.35 Recv.6 TYPE I| Thu Jun 11 12:27:45 id.35 Send.8 200 ok~~| Thu Jun 11 12:27:45 id.35 Recv.34 STOR 0223-42-2009-IT0001347175.zip| Thu Jun 11 12:27:45 id.35 Send.10 150 go..~~| Thu Jun 11 12:27:45 id.35 Recv 1460 on Data Port Thu Jun 11 12:27:45 id.35 Recv 1460 on Data Port Thu Jun 11 12:27:45 id.35 Recv 1460 on Data Port Thu Jun 11 12:27:45 id.35 Recv 1460 on Data Port Thu Jun 11 12:27:45 id.35 Recv 1460 on Data Port Thu Jun 11 12:27:45 id.35 Recv 1460 on Data Port Thu Jun 11 12:27:45 id.35 Recv 1460 on Data Port Thu Jun 11 12:27:45 id.35 Recv 538 on Data Port Thu Jun 11 12:27:45 id.35 .. Incoming File logon.zibbly file.0223-42-2009-IT0001347175.zip -> /home/hohoho [ferdy.#XX:extrastuff] Thu Jun 11 12:27:45 id.35 Send.17 226 end of file~~| Thu Jun 11 12:27:45 id.35 Recv.4 QUIT| Thu Jun 11 12:27:45 id.35 Send.10 221 bibi~~| Version Control ;1j01 1sep17 added hourly-logs and J/-J logon feature ;0j98 5jun09 original version ;h3 spoof TYPE ;8 CWD -> 250;9 added F and H ;10-11 added -Z ;12-14 RR added ;h15-18 mixup with multicard servers and RR; 18 display as option ;1920 ;j1 12sep09 speedier version ;2 15oct09 added disable/disconnect-limits ;3 minor cleanup for GET ;4 allow * in black/whitelist ;5 tuning ;6-8 added SIZE and FEAT ;9 track disableds better ;10 2dec10 added SITE FIPHDR ;12 20may11 added FipHdr inbound if nec ;14-16 12dec11 added 'z' for standalone delete/14jan11 woops - disable .. ;17-23 22may12 RRbase added -R and bugette with speedy ;20 added min/max-data-port ;21-23 added display-log and ms in log ;24 18oct12 added external /etc/passwd for linux ;25-7 12nov12 added Mkdir and Rmdir as logon options and FIP_SHADOW added ;28-31 5jan14 added w4 logon list too ;31 woops permissions on ls for DIR ;32 added -j -J for balQue and balGrp ;33-36 17jun14 added option Q to check that the CD /queue really did exist ! ;35 getuid for shadow ;36 balance dels ;37 17dec14 added remote_trace, buglette in mkdir, added L8 as type to force binary(MOD) ;38 added X-sendOnRename ;39 added force-passive-mode: ;40-42 cleanups ;43 bugette in RR - missing the last number ;44 bugette in widget ;45 added uid and gid and SX/use-sx ;46-9 added pwdcrypt ;50-56 log cleanup and added 'H' to preserve the FipHdr of 'C'-copyfile ;57 better handling of attempted logons in Linux ;58 added ZO/safe SN ;59 balance SX too ! ;60 21sep15 DELE can be Standalone AND CopyStandalone ;61-62 24sep15 mod to portData close on slow, ungraceful connections and added linger-on-close ;63-68 reset_data close socket YES/NO (and disable linger for WINNT! ;69-82 31oct15 SX includes ipaddress for uniqueness, 'X' can have a precedence chr, mods to ACCEPT to allow wrong ordered commands ;83-86 22nov15 added 'V' for virtual list to confuse hackers ;87-89 for V in MKDIR ;90-92 14apr16 better Restart (REST) and added vsftpd-compatible and replace-unsafe/hash/space ;93 bugette - rename with multiple spaces ;94 26jan17 allow both logonFile and w4logonlist ;95-96 30jan17 added quiet logging options ;97-8 added '#' for merge fiphdr and recode added (copyright) 2018 and previous years FingerPost Ltd.