ftpwire

ftpwire

FTPWIRE is a very simple FTP daemon.

Use 'ipftp' when you need to send data using FTP - or Grab data from a remote
FTPd site.
Use 'ftpwire' when a remote host needs to send to the Fip.

The file is slotted into the spool/2brouted queue for IPROUTE to process and
route.

To send to ftpwire, use the normal 'ftp' program :
    ftp -P 9130 (hostname of the Fip)
or  ftp hostname 9130

FTP uses 2 ports - one for control and the other for data. In normal FTPese,
these are 21 and 20. 'ftpwire' use the -P input switch to define the Control
port. If it is started with port 21, then it will use port 20 as a fixed port
to send data from/to.
If the parameter 'fixed-data-port' is set, that number is used.
Otherwise the dataport is just the first available port.

Remember ports 20 and 21 are reserved ports on Unix/Linux and 'root' must start
any program using a port < 1024 (and you must make sure ftpd in inet.d or
xinet.d is disabled or on a different port)

If using WIN2K, you MUST specify a unique wireid using the -I input switch and
specify several instances of ftpwire in the SYSTEM file for the number of
instances you want.

If using Unix/Linux, only one instance of ftpwire should be in the SYSTEM file
and the -E 99 switch is used to determine the number of simultaneous inputs.

-- Optional parameter file is under tables/wire and the '-n' input switch and
defaults to FTPWIRE. A '.FIP' extension may be added.
    Syntax :
    ; comment line
    banner:             Replacement banner
    outque: (folder name in FipSeq) Output folder - this overrides the -O input
switch
    logon-file:(filename)       See below - default LOGON.(-n) or LOGON.FTPWIRE.FIP
    allow-blank-pwd: (yes/no)   If there is a blank password in the logon file,
accept ANY password. default-no
    password-is-encoded: (yes/no)   Password in normal logon file or w4 lists is
encoded (default: no)

    use-w4-logon-list:(name eg FIPO.EXTERNAL)
                    Use this Fip w4 LogonList file for authentication and ignore all other
auth methods such as LOGON.FTPWIRE.FIP
    w4-logon-restrict-templates:(name(s) eg RADIUS_FTPUSER,RADIUS_FTPADMIN)
                    If using LogonList, only allow logons with this template(s)
    w4-logon-restrict-pubs:(pub name(s) eg PIF,BIG,DAILY)
                    If using LogonList, only allow logons with this pub code
    w4-logon-topq:
    w4-logon-outq:
    w4-logon-copyq:
    w4-logon-fiphdr:
    w4-logon-options:       if using LogonList, use these default values (see Logon
below for fuller explanation of each)

    use-etc-passwd: yes/no      (Linux/Unix only) - if YES, use the /etc/passwd file
for authentication and ignore all other auth methods such as LOGON.FTPWIRE.FIP
                    default: no
    external-path: (path)       for etc/passwd, Only allow logons with a home folder
starting with this path
                    eg  external-path:/home/ftp     expects all the home folders below /home/ftp
                    default: nothing specified for all folders
    external-shell: (shell)     For /etc/passwd, Only allow logons that run this
shell
                    external-shell:/bin/nologin
                    default: nothing specified for all shells
    external-min-uid: (Uid number)  For /etc/passwd, Only allow logons with Uids
higher than this number to logon
                    Stop anyone logging on a root !
                    default: 400
    external-min-gid: (Gid number)  For /etc/passwd, Only allow logons with Group
ids higher than this number to logon
                    default: none

    nat-pasv-address:(Ip address)   if using a proxy and/or NAT addresses, this is
the address of THIS host to the outside world. It is used in the PORT command
    extra-fiphdr: (fipseq)      Additional FipHdr info to be added to each incoming
file. default none
    extra-fiphdr-ext: (fipseq)  Additional FipHdr info to be added to each incoming
file if the file has this extension. default none
                    eg if a file is abc1.jpg extra-fiphdr-ext:jpg   SR:FTP_JPEGS
    replace-space: (fipseq)     Replace a space in the filename or folder with this
chr.        default: SPC
                    (see also allow-spaces:)
    replace-hash: (fipseq)      Replace hash chr in the filename or folder with this
chr.        default: '#'
    replace-unsafe: (fipseq)    Replace control and meta chrs in the filename or
folder with this.   default: '-'
                    Ftpwire will make a filename 'safe' for the system by cleaning meta chrs
'/'
                    FipHdr ZO contains the safe filename and SN is the 'given'
    no-archive:         do NOT archive the data in the daily archive files in log/data
(ZI: fiphdr field)
    display-log (yes/no/file/logon) (same as -D) Display all commands for each
connection      default: no
                    Use this to trace problem connections
                    Option 'file' will log the transactions in a dated file in log/ftp
                    Option 'logon' will log the transactions in a dated file for that logon in
log/ftp
    cwd-code:200    CWD response code MUST be a 3 digit number between 200 and 299.
            There is some confusion whether the response should be 257 or 200. default:
257
    msg-215:    String to replace the normal 215 response of 'UNIX Type: L8 Euripides
99'
    pwd-quote: (' " or a space) Quote for response to a PWD command.
            Valid options are doublequote ", singlequote ' or a space  default: "
    allow-relogon: (yes/no)
            Set this to NO to inhibit relogon in the same session   default:yes
    add-list-dots:yes/no    Add the two folders for '.' and '..' for a long list
default: no
    force-folder-names:lower/upper/nochg    Force the case of any request for folder
names   default: nochange
    force-file-names:lower/upper/nochg  Force the case of any request for file
names   default: nochange
            Use these to handle Win2k <->Linux case issues - where a case-INsensitive
client is talking to a case-SENS servers
            One suggestion is to force all folders and files lowercase and set both of
these ..:lower
    force-passive-mode: (yes/no)    By default FTP connections are 'active'. But most
Firewalls will only allow 'passive' connections nowadays. So you can request
the remote client to go passive using 'force-passive-mode:yes'. In this case
the 2 FTP commands which attempt to pass data PORT and EPRT are turned into
errors with a message telling the remote site to turn PASV/PASSive on.

    balance-group; (Balance Group name) Balance group for balancing doneque items   
default: none / no balancing
        This group MUST be in sys/BALANCE
    balance-folder: (folder under spool) Balance queue for balancing doneque items  
default: 2balance
    move-on-read-group: (Balance Group name) Balance group for redundant
move-on-read    default: none / no balancing
            This needs a move-on-read-folder to be specified
    move-on-read-folder: (FipSeq name of a folder)  Name of folder for files once
read    default: none
    default-move-on-read: (yes/no)  Default for Move on Read                default: no

    min-data-port: (number)
    max-data-port: (number)
        minimum (default is 32000) and maximum (def 64000) range of part number for
the DATA port
    fixed-data-port: (number)   A single, fixed data port number.
        Default is port 20 if the Control port is set to 21 (-P 21),
        otherwise it is a high port within the range specifiied by the min and max
(see above)

    linger-on-close: Timeout in secs for the data to be sent for Passive
connections
        For VERY slow connections, increase this already-enormous number,
        default: 20 (secs) to allow up to 20 secs max before cutting the call

    connection-retries: (number)    default 5
    send-timeout: (number)      default 60 secs
    recv-timeout: (number)      default 60 secs
    connection-timeout: (number)    default 20 secs
    session-timeout: (number)   default 1200 secs (20 mins) of no activity
    between-files-timeout: (number) default 1200 secs (20 mins) of no activity

    round-robin: (number)               default: none
    round-robin-fiphdr: (2 letter FipHdr field) default: none
    round-robin-offset-fiphdr: (2 letter FipHdr field)  default: none
        Round-Robin the output files and add the RR number to the fipHdr.
        Both parameters are required - the Number is the MAXimum.
        eg to leave the output in folder1 to folder9
            round-robin:9
            round-robin-fiphdr:RR
            round-robin-offset-fiphdr:RO
        and a suitable output folder might be   /fip/spool/xchg\RR
            (This can be in the LOGON file or the default input switch -O xchg\\R
(remember double backers)
        Note that the round-robin number is NOT added automatically to any output
folder - ie you MUST specify a FipHdr as in /fip/spool/2xml\RR
        The round-robin-offset-fiphdr allows the RR number to be offset by the -R
input switch which is the base offset - default 1
        So if '-R 8' and 'round-robin:10', the output will be in (folder)8 to
(folder)17
        So WITHOUT the -R switch both round-robin-fiphdr and
round-robin-offset-fiphdr will give the same number.

    allow: (IPaddress to allow)
    disallow: (IPaddress to block)
        use this for blacklist/whitelist certain addresses
        A '*' (star) can be used to indicate ALL    eg 10.3.3.*
    disconnect-limit: (number)
        number of logon/password errors before connection is broken
        default: 30 unsuccessful attempts
    disable-limit: (number)
        number of logon/password errors before logon is blacklisted
        default: 30 unsuccessful attempts
    allow-site-fiphdr: (yes/no) see SITE FIPHDR below   default: no

    allow-anon: (yes/no)        Allow anonymous logons  default-no
    anon-desc:          For Anon logons, fill in the default logon fields....
    anon-topq:
    anon-outq:
    anon-fiphdr:
    anon-standalone:
    anon-display:           Override the default display setting (same as -D)

    timing-stats: (yes/no)      generate Timing stats (default is now YES)

    save-data-path: (pathname for data)
        This puts the data of the incoming data in a file in this folder and creates
a FipHdr file that contains 2 FipHdrs containing the full path/filename
            SX: and FTP_EXTERNAL_FILE:
        (ipbalan uses SX and ipftp uses FTP_EXTERNAL_FILE)
            eq  save-data-path:/fip/data/jpegs/\$e\$y\$i\$d/
        Use this for big files that you do not want to copy around the Fip Spool
area.
        ** if specified, ALL non-standalone files will be split like this **

-- Logons and Passwords

There are 3 (main) types of Authentication.
    - default - using a Fip pipe delimited file called LOGON_FTPWIRE.FIP (or the
parameter of the '-n' input switch.
    - for Unix/Linux, using the normal /etc/passwd file
    - using the Fip w4 LogonList file

The following describes the parameter file syntax for the default. Please see
the relevant (external) doc for the others.

The types of logon/password are
    - anonymous logon
        use input switch -A to allow (disallowd by default)
        use parameters to optionally add more information
            anon-desc   - just a note for logging
            anon-fiphdr - extra fiphdr to add to each file
            anon-topq   - top folder for LIST and GETs
                        default is LISTs and GETS are not allowed
            anon-copyq  - folder holding a copy of any incoming file
            anon-curq   - under
    - full logon and password (normal running)
    - logon and allow any password - just leave the password blank and add -B

The Logon file is in tables/wir and is called LOGON.(name) where name is the
'-n' switch or FTPWIRE.FIP by default

fields in the Password file are pipe delimited and are :
field       0   Enabled or Disabled flag    E/D
        1   LogonName
        2   Password
        3   last mod time (used by the user interface only)
        4   Description/Comment
        5   optional home folder which is revealed as '/' to the remote
            if this is blank, then LS and GETs are returned as 'no such folder/file'
        6   Output folder for any incoming files. If it does NOT start with a '/', the
folder is under /fip/spool
            This can be in FipSeq
            If blank, the default output folder is used.
        7   optional Copy folder where an exact copy of the incoming file
                this can be the same as 5-home folder if the remote needs to see the file
        8   any optional ExtraFipHdr info
        9   Option single letter Flags
            (Negate by adding a dash/hyphen prefix - ie to make sure files are NEVER
deleted '-Z'
            S-standalone output file (original filename and no fiphdr)
            C-standalone copy file (original filename and no fiphdr)
            H-For Standalone copy, add a FipHdr
            D-display all commands for this client only (ie -D for this one client)
            F-allow extra FipHdr data to be added BEFORE the Store as a site command
                SITE FIPHDR #SU:ZIBBLE#CX:ZIBBLE2EDITO
            Z-allow delete of any Standalone Copy ('C') files
            T-allow sessionTimeout of 24 hours for this client (default is 20 mins or
the session-timeout parameter)
            U-allow Last File Timeout of 24 hours for this client (default is 20 mins or
the session-file-timeout parameter)
            M-allow client to MKDIR a sub folder
            R-allow client to RMDIR a sub folder
            G-allow client to GET a file (which is the default if 'S'tandalone or 'C'opy
is on - so to Disallow, use '-G')
            Q-on a CD/CWD, check the folder really does exist
            W-if in standalone mode, overwrite files
            X-Send on Rename - Files are held in the 5-HomeFolder until a rename when
they are moved to the 6-Outputfolder
                2 further suboptions in () are optional for Pre strings and Post strings
                (pre=[string in FipSeq]) and (post=[FipSeq string]) where [FipSeq string]
is any parsable text !
                - if a source sends files preceeded by _^_(filename) then X (pre=_^_) will
detect these to be SendOnRename
                - if a source sends files with a '.tmp' extension and then renames them to
'.xml' or someother file type, use X (post=.tmp)
                NOTE that any files WITHOUT either the pre or post strings will be sent
immediately
                NOTE you cannot rename files if using V-virtual list
            V-Virtual list - use this to hold a list of files sent by the remote. LIST,
SIZE and MDTM commands will show the files
                Only files from the sender IN THIS SESSION are shown - none from previous
sessions or from any concurrent session.
                NOTE you cannot sendOnRename or rename files if using V-virtual list

eg

E|Pittlewire|zong|0|Mr Pittles Image
Agency||2edsys||#DF:PITTLEWIRE.FIP#EQ:pittle|
E|Brittle||0|Mr Brittle HardHat
Agency||xchg|#CX:B2FIP#DF:BRITTLE.FIP#EQ:brittle|

-- FipHdr fields added to each file UNLESS the Standalone option has been
flagged
    SP  IPaddress of the remote host
    SN  Filename given
    ZO  Safe filename
    SU  -n input switch or FTPWIRE
    SA  logon name
    SC  chrset - defaults to ASCII
    S1  client description for the logon file
    S2  wire id
    S3  session id
    S4  current PWD
    S5  any extra subfolder in a put command
    S6  current Client system details (if offered by remote)

Input Parameters :
Optional :
    -9 : do not use Speedy on a Speedy system
    -A : allow anonymous logons             default: always need a logon/pwd
    -B : allow blank passwords              default: always need a pwd
        if the password field is blank in the LOGON file, no checking takes place.
    -d : done folder                    default: none
        This holds a copy of all incoming data files from every source
        The structure is
            (done folder) / (date)_(logon) eg 20110921_fip / (filename as written to the
output folder)
        It can be purged with an entry in maintenance (zapfiplog)
        eg if '-d raw.ftpwire' and we want the last 30 days data
            /fip/bin/ipdelque -q/fip/spool/raw.ftpwire -i1 -a30
    -D : display all commands for each connection       default: no
        Use this to trace problem connections
    -E : maximum number of threads              default: 1
        up to a max of 200 (not Win2k)
        Note this is also a hardware limit in that small systems may not be able to
run more than 50 or so
    -F : allow SITE FIPHDR commands             default: no
    -I : wire id                        default: 0
        used to track which instance of a multi-ftpwire system a file arrived/logged
    -j : balance queue for balancing doneque items      default: 2balance
    -J : balance group for balancing doneque items      default: -none- no balancing
        This group MUST be in sys/BALANCE
    -l : log level
    -n : name of this wire                  default: FTPWIRE
    -O : Name of the output folder if not default       default: spool/2brouted
        This folder will be under /fip/spool
    -P : Port for control                   default: 9130
        -P 21 is normal
    -R : round-robin base - see above           default: 1
    -s : local hostname or ipaddress            default: all local addresses
        where a server has multiple ip address/hostnames, use '-s' to restrict
connections to a single address
    -S : default is standlone and not Fip           default: next folder is fip
        in this case do NOT add a Fiphdr and preserve exactly the incoming filename
        this can be overridden in the Logon file
    -T : log timing stats                   default: no
    -v : Print the version number and exit

-- Log levels for -l input switch are :
    default (-1)    errors only are logged
    0       connections/disconnections
    10      logons
    20      each file in or out

-- The -D input switch will display all cmds etc as they come in

eg
Thu Jun 11 12:27:44 id.35 ++ New Connection FTPWIRE 195.185.192.221
w.0.id.35.tot.1
Thu Jun 11 12:27:44 id.35   Send.23 220 Fip FTP service~~|
Thu Jun 11 12:27:44 id.35   Recv.10 USER zibbly|
Thu Jun 11 12:27:44 id.35   Send.20 331 Enter password~~|
Thu Jun 11 12:27:44 id.35   Recv.12 PASS *******|
Thu Jun 11 12:27:45 id.35   Send.14 230 Logon ok~~|
Thu Jun 11 12:27:45 id.35   Recv.3  PWD|
Thu Jun 11 12:27:45 id.35   Send.7  257 /~~|
Thu Jun 11 12:27:45 id.35   Recv.4  PASV|
Thu Jun 11 12:27:45 id.35   Send.45 227 Entering Passive Mode
(10,1,1,35,125,1)~~|
Thu Jun 11 12:27:45 id.35   Recv.6  TYPE I|
Thu Jun 11 12:27:45 id.35   Send.8  200 ok~~|
Thu Jun 11 12:27:45 id.35   Recv.34 STOR 0223-42-2009-IT0001347175.zip|
Thu Jun 11 12:27:45 id.35   Send.10 150 go..~~|
Thu Jun 11 12:27:45 id.35   Recv 1460 on Data Port
Thu Jun 11 12:27:45 id.35   Recv 1460 on Data Port
Thu Jun 11 12:27:45 id.35   Recv 1460 on Data Port
Thu Jun 11 12:27:45 id.35   Recv 1460 on Data Port
Thu Jun 11 12:27:45 id.35   Recv 1460 on Data Port
Thu Jun 11 12:27:45 id.35   Recv 1460 on Data Port
Thu Jun 11 12:27:45 id.35   Recv 1460 on Data Port
Thu Jun 11 12:27:45 id.35   Recv 538 on Data Port
Thu Jun 11 12:27:45 id.35 .. Incoming File logon.zibbly
file.0223-42-2009-IT0001347175.zip -> /home/hohoho [ferdy.#XX:extrastuff]
Thu Jun 11 12:27:45 id.35   Send.17 226 end of file~~|
Thu Jun 11 12:27:45 id.35   Recv.4  QUIT|
Thu Jun 11 12:27:45 id.35   Send.10 221 bibi~~|

Version Control
;0j93 5jun09 original version
    ;h3 spoof TYPE ;8 CWD -> 250;9 added F and H ;10-11 added -Z ;12-14 RR added
    ;h15-18 mixup with multicard servers and RR; 18 display as option ;1920
    ;j1 12sep09 speedier version ;2 15oct09 added disable/disconnect-limits ;3
minor cleanup for GET
    ;4 allow * in black/whitelist
    ;5 tuning ;6-8 added SIZE and FEAT ;9 track disableds better ;10 2dec10 added
SITE FIPHDR
    ;12 20may11 added FipHdr inbound if nec
    ;14-16 12dec11 added 'z' for standalone delete/14jan11 woops - disable ..
    ;17-23 22may12 RRbase added -R and bugette with speedy ;20 added
min/max-data-port ;21-23 added display-log and ms in log
    ;24 18oct12 added external /etc/passwd for linux
    ;25-7 12nov12 added Mkdir and Rmdir as logon options and FIP_SHADOW added
    ;28-31 5jan14 added w4 logon list too ;31 woops permissions on ls for DIR
    ;32 added -j -J for balQue and balGrp
    ;33-36 17jun14 added option Q to check that the CD /queue really did exist !
;35 getuid for shadow ;36 balance dels
    ;37 17dec14 added remote_trace, buglette in mkdir, added L8 as type to force
binary(MOD)
    ;38 added X-sendOnRename ;39 added force-passive-mode: ;40-42 cleanups
    ;43 bugette in RR - missing the last number ;44 bugette in widget ;45 added
uid and gid and SX/use-sx
    ;46-9 added pwdcrypt ;50-56 log cleanup and added 'H' to preserve the FipHdr
of 'C'-copyfile
    ;57 better handling of attempted logons in Linux ;58 added ZO/safe SN ;59
balance SX too !
    ;60 21sep15 DELE can be Standalone AND CopyStandalone
    ;61-62 24sep15 mod to portData close on slow, ungraceful connections and added
linger-on-close
    ;63-68 reset_data close socket YES/NO (and disable linger for WINNT!
    ;69-82 31oct15 SX includes ipaddress for uniqueness, 'X' can have a precedence
chr, mods to ACCEPT to allow wrong ordered commands
    ;83-86 22nov15 added 'V' for virtual list to confuse hackers ;87-89 for V in
MKDIR
    ;90-92 14apr16 better Restart (REST) and added vsftpd-compatible and
replace-unsafe/hash/space
    ;93 bugette - rename with multiple spaces

(copyright) 2017 and previous years FingerPost Ltd.