ftpwire FTPWIRE is a very simple FTP daemon. Use 'ipftp' when you need to send data using FTP - or Grab data from a remote FTPd site. Use 'ftpwire' when a remote host needs to send to the Fip. The file is slotted into the spool/2brouted queue for IPROUTE to process and route. To send to ftpwire, use the normal 'ftp' program : ftp -P 9130 (hostname of the Fip) or ftp hostname 9130 FTP uses 2 ports - one for control and the other for data. In normal FTPese, these are 21 and 20. 'ftpwire' use the -P input switch to define the Control port. If it is started with port 21, then it will use port 20 as a fixed port to send data from/to. If the parameter 'fixed-data-port' is set, that number is used. Otherwise the dataport is just the first available port. Remember ports 20 and 21 are reserved ports on Unix/Linux and 'root' must start any program using a port < 1024 (and you must make sure ftpd in inet.d or xinet.d is disabled or on a different port) If using WIN2K, you MUST specify a unique wireid using the -I input switch and specify several instances of ftpwire in the SYSTEM file for the number of instances you want. If using Unix/Linux, only one instance of ftpwire should be in the SYSTEM file and the -E 99 switch is used to determine the number of simultaneous inputs. One useful builtin is merging FipHdrs - if the incoming file has a FipHdr, then any 'extra-fiphdr' info, datetime FipHdrs, system FipHdrs are all merged. -- Optional parameter file is under tables/wire and the '-n' input switch and defaults to FTPWIRE. A '.FIP' extension may be added. Syntax : ; comment line banner: Replacement banner outque: (folder name in FipSeq) Output folder - this overrides the -O input switch logon-file:(filename) See below - default LOGON.(-n) or LOGON.FTPWIRE.FIP allow-blank-pwd: (yes/no) If there is a blank password in the logon file, accept ANY password. default-no password-is-encoded: (yes/no) Password in normal logon file or w4 lists is encoded (default: no) use-w4-logon-list:(name eg FIPO.EXTERNAL) Use this Fip w4 LogonList file for authentication and ignore all other auth methods such as LOGON.FTPWIRE.FIP w4-logon-restrict-templates:(name(s) eg RADIUS_FTPUSER,RADIUS_FTPADMIN) If using LogonList, only allow logons with this template(s) w4-logon-restrict-pubs:(pub name(s) eg PIF,BIG,DAILY) If using LogonList, only allow logons with this pub code w4-logon-topq: w4-logon-outq: w4-logon-copyq: w4-logon-fiphdr: w4-logon-options: if using LogonList, use these default values (see Logon below for fuller explanation of each) use-etc-passwd: yes/no (Linux/Unix only) - if YES, use the /etc/passwd file for authentication and ignore all other auth methods such as LOGON.FTPWIRE.FIP default: no external-path: (path) for etc/passwd, Only allow logons with a home folder starting with this path eg external-path:/home/ftp expects all the home folders below /home/ftp default: nothing specified for all folders external-shell: (shell) For /etc/passwd, Only allow logons that run this shell external-shell:/bin/nologin default: nothing specified for all shells external-min-uid: (Uid number) For /etc/passwd, Only allow logons with Uids higher than this number to logon Stop anyone logging on a root ! default: 400 external-min-gid: (Gid number) For /etc/passwd, Only allow logons with Group ids higher than this number to logon default: none nat-pasv-address:(IP address) if using a proxy or firewall, which does NOT automatically NAT internal to external addresses, this is the external address of THIS host to the Extranet world. It is used in the PORT command sent to the remote server. extra-fiphdr: (fipseq) Additional FipHdr info to be added to each incoming file. default none extra-fiphdr-ext: (fipseq) Additional FipHdr info to be added to each incoming file if the file has this extension. default none eg if a file is abc1.jpg extra-fiphdr-ext:jpg SR:FTP_JPEGS replace-space: (fipseq) Replace a space in the filename or folder with this chr. default: SPC (see also allow-spaces:) replace-hash: (fipseq) Replace hash chr in the filename or folder with this chr. default: '#' replace-unsafe: (fipseq) Replace control and meta chrs in the filename or folder with this. default: '-' Ftpwire will make a filename 'safe' for the system by cleaning meta chrs '/' FipHdr ZO contains the safe filename and SN is the 'given' no-archive: do NOT archive the data in the daily archive files in log/data (ZI: fiphdr field) display-log (yes/no/file/logon) (same as -D) Display all commands for each connection default: no Use this to trace problem connections Option 'file' will log the transactions in a dated file in log/ftp Option 'logon' will log the transactions in a dated file for that logon in log/ftp hourly-logs: (time) Remote Trace log files are normally daily (/fip/log/remote_trace/(date) Use this to add a hour extension (only 60 for 60 mins is currently valid) (can be overriden by 'J' or '-J' in the LOGON file for each logon) cwd-code:200 CWD response code MUST be a 3 digit number between 200 and 299. There is some confusion whether the response should be 257 or 200. default: 257 msg-215: String to replace the normal 215 response of 'UNIX Type: L8 Euripides 99' pwd-quote: (' " or a space) Quote for response to a PWD command. Valid options are doublequote ", singlequote ' or a space default: " allow-relogon: (yes/no) Set this to NO to inhibit relogon in the same session default:yes add-list-dots:yes/no Add the two folders for '.' and '..' for a long list default: no force-folder-names:lower/upper/nochg Force the case of any request for folder names default: nochange force-file-names:lower/upper/nochg Force the case of any request for file names default: nochange Use these to handle Win2k <->Linux case issues - where a case-INsensitive client is talking to a case-SENS servers One suggestion is to force all folders and files lowercase and set both of these ..:lower force-passive-mode: (yes/no) By default FTP connections are 'active'. But most Firewalls will only allow 'passive' connections nowadays. So you can request the remote client to go passive using 'force-passive-mode:yes'. In this case the 2 FTP commands which attempt to pass data PORT and EPRT are turned into errors with a message telling the remote site to turn PASV/PASSive on. balance-group; (Balance Group name) Balance group for balancing doneque items default: none / no balancing This group MUST be in sys/BALANCE balance-folder: (folder under spool) Balance queue for balancing doneque items default: 2balance move-on-read-group: (Balance Group name) Balance group for redundant move-on-read default: none / no balancing This needs a move-on-read-folder to be specified move-on-read-folder: (FipSeq name of a folder) Name of folder for files once read default: none default-move-on-read: (yes/no) Default for Move on Read default: no min-data-port: (number) max-data-port: (number) minimum (default is 32000) and maximum (def 64000) range of part number for the DATA port fixed-data-port: (number) A single, fixed data port number. Default is port 20 if the Control port is set to 21 (-P 21), otherwise it is a high port within the range specifiied by the min and max (see above) linger-on-close: Timeout in secs for the data to be sent for Passive connections For VERY slow connections, increase this already-enormous number, default: 20 (secs) to allow up to 20 secs max before cutting the call connection-retries: (number) default 5 send-timeout: (number) default 60 secs recv-timeout: (number) default 60 secs connection-timeout: (number) default 20 secs session-timeout: (number) default 1200 secs (20 mins) of no activity between-files-timeout: (number) default 1200 secs (20 mins) of no activity use-tls:yes/implicit/explicit/auth/no The commands are for a ftp running over SSL/TLS on the remote server NOTE - ipftpssl and NOT ipftp must be used for SSL/TLS default is NO no - normal, standard FTP on (normally) port 21 for the control yes or explicit - connect (normally) on port 21 in clear then use SSL for USER, PASS and data auth - connect (normally) on port 21 in clear, use SSL for USER, PASS then return to clear for non-data commands - but use SSL for all data implicit - connect (normally) on port 990: use SSL for all control and data tls-auth: (XXX) AUTH type for TLS/SSL default: TLS Valid entries are TLS, SSL, TLS-C (whatever that is !) and something starting 'X-' which will be something homegrown ! NOTE that for all versions of SSL the method string is "SSL" (this string is case sensitive according to the RFC) eg tls-auth:SSL ssl-method: tls tls1 tls1.1 tls1.2 sslv2 sslv3 sslv2and3 Version number to use for TLS/SSL default: 999 for current default (2 or 3) (only the digits are significant, so add other text to make it readable) For 'modern' connection, pls do NOT use sslv2 ! as it is deemed insecure If default it will check the available list and pick the highest. The default is currently 23 which on a modern server is sslv3 and tls1_2 !) ssl-password: (password) ssl-passwd: (password) default: none Optional password if the handshake requires a shared secret ssl-cert: (name of a PEM certificate file) default: none ssl-root-cert: (name of a root PEM certificate file) defaunt: none Optional certificates - held in tables/ssl ssl-verify: yes/no verify certificates default: yes ssl-ciphers: (list) acceptable ciphers (use 'openssl ciphers' to list) default: "HIGH:!aNULL:!kRSA:!SRP:!PSK:!CAMELLIA:!RC4:!MD5:!DSS" round-robin: (number) default: none round-robin-fiphdr: (2 letter FipHdr field) default: none round-robin-offset-fiphdr: (2 letter FipHdr field) default: none Round-Robin the output files and add the RR number to the fipHdr. Both parameters are required - the Number is the MAXimum. eg to leave the output in folder1 to folder9 round-robin:9 round-robin-fiphdr:RR round-robin-offset-fiphdr:RO and a suitable output folder might be /fip/spool/xchg\RR (This can be in the LOGON file or the default input switch -O xchg\\R (remember double backers) Note that the round-robin number is NOT added automatically to any output folder - ie you MUST specify a FipHdr as in /fip/spool/2xml\RR The round-robin-offset-fiphdr allows the RR number to be offset by the -R input switch which is the base offset - default 1 So if '-R 8' and 'round-robin:10', the output will be in (folder)8 to (folder)17 So WITHOUT the -R switch both round-robin-fiphdr and round-robin-offset-fiphdr will give the same number. allow: (IPaddress to allow) disallow: (IPaddress to block) use this for blacklist/whitelist certain addresses A '*' or '0' (star or zero) can be used to indicate a range eg 10.3.3.* An extra number with a preceeding space can be used to set a loglevel (see below for values) disconnect-limit: (number) number of logon/password errors before connection is broken default: 30 unsuccessful attempts disable-limit: (number) number of logon/password errors before logon is blacklisted default: 30 unsuccessful attempts allow-site-fiphdr: (yes/no) see SITE FIPHDR below default: no allow-anon: (yes/no) Allow anonymous logons default-no anon-desc: For Anon logons, fill in the default logon fields.... anon-topq: anon-outq: anon-fiphdr: anon-standalone: anon-display: Override the default display setting (same as -D) timing-stats: (yes/no) generate Timing stats (default is now YES) save-data-path: (pathname for data) This puts the data of the incoming data in a file in this folder and creates a FipHdr file that contains 2 FipHdrs containing the full path/filename SX: and FTP_EXTERNAL_FILE: (ipbalan uses SX and ipftp uses FTP_EXTERNAL_FILE) eq save-data-path:/fip/data/jpegs/\$e\$y\$i\$d/ Use this for big files that you do not want to copy around the Fip Spool area. ** if specified, ALL non-standalone files will be split like this ** -- Logons and Passwords There are 3 (main) types of Authentication. - default - using a Fip pipe delimited file called LOGON_FTPWIRE.FIP (or the parameter of the '-n' input switch. - for Unix/Linux, using the normal /etc/passwd file - using the Fip w4 LogonList file The following describes the parameter file syntax for the default. Please see the relevant (external) doc for the others. The types of logon/password are - anonymous logon use input switch -A to allow (disallowd by default) use parameters to optionally add more information anon-desc - just a note for logging anon-fiphdr - extra fiphdr to add to each file anon-topq - top folder for LIST and GETs default is LISTs and GETS are not allowed anon-copyq - folder holding a copy of any incoming file anon-curq - under - full logon and password (normal running) - logon and allow any password - just leave the password blank and add -B The Logon file is in tables/wir and is called LOGON.(name) where name is the '-n' switch or FTPWIRE.FIP by default fields in the Password file are pipe delimited and are : field 0 Enabled or Disabled flag E/D 1 LogonName 2 Password 3 last mod time (used by the user interface only) 4 Description/Comment 5 optional home folder which is revealed as '/' to the remote if this is blank, then LS and GETs are returned as 'no such folder/file' 6 Output folder for any incoming files. If it does NOT start with a '/', the folder is under /fip/spool This can be in FipSeq If blank, the default output folder is used. 7 optional Copy folder where an exact copy of the incoming file this can be the same as 5-home folder if the remote needs to see the file 8 any optional ExtraFipHdr info 9 Option single letter Flags (Negate by adding a dash/hyphen prefix - ie to make sure files are NEVER deleted '-Z' S-standalone output file (original filename and no fiphdr) C-standalone copy file (original filename and no fiphdr) H-For Standalone copy, add a FipHdr D-display all commands for this client only (ie -D for this one client) F-allow extra FipHdr data to be added BEFORE the Store as a site command SITE FIPHDR #SU:ZIBBLE#CX:ZIBBLE2EDITO Z-allow delete of any Standalone Copy ('C') files T-allow sessionTimeout of 24 hours for this client (default is 20 mins or the session-timeout parameter) U-allow Last File Timeout of 24 hours for this client (default is 20 mins or the session-file-timeout parameter) M-allow client to MKDIR a sub folder R-allow client to RMDIR a sub folder G-allow client to GET a file (which is the default if 'S'tandalone or 'C'opy is on - so to Disallow, use '-G') Q-on a CD/CWD, check the folder really does exist J-set hourly logs for the remote_trace log file W-if in standalone mode, overwrite files X-Send on Rename - Files are held in the 5-HomeFolder until a rename when they are moved to the 6-Outputfolder 2 further suboptions in () are optional for Pre strings and Post strings (pre=[string in FipSeq]) and (post=[FipSeq string]) where [FipSeq string] is any parsable text ! - if a source sends files preceeded by _^_(filename) then X (pre=_^_) will detect these to be SendOnRename - if a source sends files with a '.tmp' extension and then renames them to '.xml' or someother file type, use X (post=.tmp) NOTE that any files WITHOUT either the pre or post strings will be sent immediately NOTE you cannot rename files if using V-virtual list V-Virtual list - use this to hold a list of files sent by the remote. LIST, SIZE and MDTM commands will show the files Only files from the sender IN THIS SESSION are shown - none from previous sessions or from any concurrent session. NOTE you cannot sendOnRename or rename files if using V-virtual list L-Logging options (sub options in following brackets) C - do NOT log connections/disc N - NewLogon eg E|Pittlewire|zong|0|Mr Pittles Image Agency||2edsys||#DF:PITTLEWIRE.FIP#EQ:pittle| E|Brittle||0|Mr Brittle HardHat Agency||xchg|#CX:B2FIP#DF:BRITTLE.FIP#EQ:brittle| -- FipHdr fields added to each file UNLESS the Standalone option has been flagged SP IPaddress of the remote host SN Filename given ZO Safe filename SU -n input switch or FTPWIRE SA logon name SC chrset - defaults to ASCII S1 client description for the logon file S2 wire id S3 session id S4 current PWD S5 any extra subfolder in a put command S6 current Client system details (if offered by remote) Input Parameters : Optional : -9 : do not use Speedy on a Speedy system -A : allow anonymous logons default: always need a logon/pwd -B : allow blank passwords default: always need a pwd if the password field is blank in the LOGON file, no checking takes place. -d : done folder default: none This holds a copy of all incoming data files from every source The structure is (done folder) / (date)_(logon) eg 20110921_fip / (filename as written to the output folder) It can be purged with an entry in maintenance (zapfiplog) eg if '-d raw.ftpwire' and we want the last 30 days data /fip/bin/ipdelque -q/fip/spool/raw.ftpwire -i1 -a30 -D : display all commands for each connection default: no Use this to trace problem connections -E : maximum number of threads default: 1 up to a max of 200 (not Win2k) Note this is also a hardware limit in that small systems may not be able to run more than 50 or so -F : allow SITE FIPHDR commands default: no -I : wire id default: 0 used to track which instance of a multi-ftpwire system a file arrived/logged -j : balance queue for balancing doneque items default: 2balance -J : balance group for balancing doneque items default: -none- no balancing This group MUST be in sys/BALANCE -l : log level -n : name of this wire default: FTPWIRE -O : Name of the output folder if not default default: spool/2brouted This folder will be under /fip/spool -P : Port for control default: 9130 -P 21 is normal -R : round-robin base - see above default: 1 -s : local hostname or ipaddress default: all local addresses where a server has multiple ip address/hostnames, use '-s' to restrict connections to a single address -S : default is standlone and not Fip default: next folder is fip in this case do NOT add a Fiphdr and preserve exactly the incoming filename this can be overridden in the Logon file -T : log timing stats default: no -v : Print the version number and exit -- Log levels for -l input switch are : default (-1) errors only are logged 0 connections/disconnections 10 logons 20 each file in or out -- The -D input switch will display all cmds etc as they come in eg Thu Jun 11 12:27:44 id.35 ++ New Connection FTPWIRE 220.127.116.11 w.0.id.35.tot.1 Thu Jun 11 12:27:44 id.35 Send.23 220 Fip FTP service~~| Thu Jun 11 12:27:44 id.35 Recv.10 USER zibbly| Thu Jun 11 12:27:44 id.35 Send.20 331 Enter password~~| Thu Jun 11 12:27:44 id.35 Recv.12 PASS *******| Thu Jun 11 12:27:45 id.35 Send.14 230 Logon ok~~| Thu Jun 11 12:27:45 id.35 Recv.3 PWD| Thu Jun 11 12:27:45 id.35 Send.7 257 /~~| Thu Jun 11 12:27:45 id.35 Recv.4 PASV| Thu Jun 11 12:27:45 id.35 Send.45 227 Entering Passive Mode (10,1,1,35,125,1)~~| Thu Jun 11 12:27:45 id.35 Recv.6 TYPE I| Thu Jun 11 12:27:45 id.35 Send.8 200 ok~~| Thu Jun 11 12:27:45 id.35 Recv.34 STOR 0223-42-2009-IT0001347175.zip| Thu Jun 11 12:27:45 id.35 Send.10 150 go..~~| Thu Jun 11 12:27:45 id.35 Recv 1460 on Data Port Thu Jun 11 12:27:45 id.35 Recv 1460 on Data Port Thu Jun 11 12:27:45 id.35 Recv 1460 on Data Port Thu Jun 11 12:27:45 id.35 Recv 1460 on Data Port Thu Jun 11 12:27:45 id.35 Recv 1460 on Data Port Thu Jun 11 12:27:45 id.35 Recv 1460 on Data Port Thu Jun 11 12:27:45 id.35 Recv 1460 on Data Port Thu Jun 11 12:27:45 id.35 Recv 538 on Data Port Thu Jun 11 12:27:45 id.35 .. Incoming File logon.zibbly file.0223-42-2009-IT0001347175.zip -> /home/hohoho [ferdy.#XX:extrastuff] Thu Jun 11 12:27:45 id.35 Send.17 226 end of file~~| Thu Jun 11 12:27:45 id.35 Recv.4 QUIT| Thu Jun 11 12:27:45 id.35 Send.10 221 bibi~~| -- Secure FTP -------------------------------------- Confusingly there are two - completely different - 'Secure' FTPs plus a defunct company called SecoueFTP etc etc 1. a more secure version of ordinary FTP which uses SSL/TLS in the same way http and httpS work for secure web sites. 2. a file copy layer which sits ontop of SSH - which has nothing at all to do with ordinary FTP The extra confusion is that SSH uses SSL - so dont mix the two up ! ((This is from the FileZilla website as they are the kings of ftp : .. TLS (FTPS) vs SSH (SFTP) FTPS (FTP encrypted with TLS) should not be confused with SFTP (SSH). The latter is a completely different protocol. .. Explicit vs Implicit FTPS FTPS (FTP over TLS) is served up in two incompatible modes. If using explicit FTPS, the client connects to the normal FTP port and explicitly switches into secure (TLS) mode with "AUTH TLS", whereas implicit FTPS is an older style service that assumes TLS mode right from the start of the connection (and normally listens on TCP port 990, rather than 21). In a FileZilla client this means prefixing the host with "FTPES://" to connect an "explicit" FTPS server, or "FTPS://" for the legacy "implicit" server (for which you will likely also need to set the port to 990). Thank you FileZilla)) How do you know which is the one you want ? - What is the port number on the remote server ? port 21 - it is BOTH normal FTP and the SSL/TLS version (port 21 is the same as normal FTP) test with ordinary 'ftp' client port 990 - it is ONLY SSL/TLS version test with 'telnet' to (remhost) 990 and cut the connection once you are satisfied it connects port 22 - it is ONLY the sftp on top of SSH test with 'ssh' or 'sftp' --- 1. SSL/TLS This is the easy one.. YOU MUST USE ftpwiressl for any/all ssl/tls traffic as ftpwire blocks these commands. Prerequsite is the SSL layer which nowadays comes as standard on most platforms. Otherwise it can be downloaded from the installation kit or the website of the OS - Sun for Solaris, RedHat, Suse etc http://www.openssl.org For *nix, if you have gcc installed, it is usually easier (!) to compile from the latest sources at http://www.openssl.org/source/ For Win2k, there are precompiled versions at http://www.openssl.org/related/binaries.html which points at http://www.slproweb.com/products/Win32OpenSSL.html NOTE you generally have to also add the 'MicroSoft Visual C++ 2008 Redistributables' (vcredit) Pick the 32bit Light version unless you are running very very high volume stuff. To get ipftp to use SSL/TLS, all you need to do is add the keyword use-tls:(param) where param is no - normal, standard FTP on (normally) port 21 for the control (ie DO NOT use SSL/TLS) yes or explicit - connect (normally) on port 21 in clear then use SSL/TLS for USER, PASS and data auth - connect (normally) on port 21 in clear, use SSL/TLS for USER, PASS then return to clear (but use SSL/TLS for all data) This is normally the only version which works if using a Proxy server or a really nasty Firewall implicit - connect (normally) on port 990: use SSL/TLS for all control and data If passwords and/or certicates are required, use the ssl-cert etc to add --- 2.a SSH/SFTP ftpwire does NOT current do sftp over ssh. Version Control ;1a08 1sep17 added hourly-logs and J/-J logon feature ;a1-6 better TLS ;7 log-level and log in allow/disallow ;8 7feb19 nat-pasv-address was reversed for Linux ;0j98 5jun09 original version ;h3 spoof TYPE ;8 CWD -> 250;9 added F and H ;10-11 added -Z ;12-14 RR added ;h15-18 mixup with multicard servers and RR; 18 display as option ;1920 ;j1 12sep09 speedier version ;2 15oct09 added disable/disconnect-limits ;3 minor cleanup for GET ;4 allow * in black/whitelist ;5 tuning ;6-8 added SIZE and FEAT ;9 track disableds better ;10 2dec10 added SITE FIPHDR ;12 20may11 added FipHdr inbound if nec ;14-16 12dec11 added 'z' for standalone delete/14jan11 woops - disable .. ;17-23 22may12 RRbase added -R and bugette with speedy ;20 added min/max-data-port ;21-23 added display-log and ms in log ;24 18oct12 added external /etc/passwd for linux ;25-7 12nov12 added Mkdir and Rmdir as logon options and FIP_SHADOW added ;28-31 5jan14 added w4 logon list too ;31 woops permissions on ls for DIR ;32 added -j -J for balQue and balGrp ;33-36 17jun14 added option Q to check that the CD /queue really did exist ! ;35 getuid for shadow ;36 balance dels ;37 17dec14 added remote_trace, buglette in mkdir, added L8 as type to force binary(MOD) ;38 added X-sendOnRename ;39 added force-passive-mode: ;40-42 cleanups ;43 bugette in RR - missing the last number ;44 bugette in widget ;45 added uid and gid and SX/use-sx ;46-9 added pwdcrypt ;50-56 log cleanup and added 'H' to preserve the FipHdr of 'C'-copyfile ;57 better handling of attempted logons in Linux ;58 added ZO/safe SN ;59 balance SX too ! ;60 21sep15 DELE can be Standalone AND CopyStandalone ;61-62 24sep15 mod to portData close on slow, ungraceful connections and added linger-on-close ;63-68 reset_data close socket YES/NO (and disable linger for WINNT! ;69-82 31oct15 SX includes ipaddress for uniqueness, 'X' can have a precedence chr, mods to ACCEPT to allow wrong ordered commands ;83-86 22nov15 added 'V' for virtual list to confuse hackers ;87-89 for V in MKDIR ;90-92 14apr16 better Restart (REST) and added vsftpd-compatible and replace-unsafe/hash/space ;93 bugette - rename with multiple spaces ;94 26jan17 allow both logonFile and w4logonlist ;95-96 30jan17 added quiet logging options ;97-8 added '#' for merge fiphdr and recode added (copyright) 2019 and previous years FingerPost Ltd.