httpwire

httpwire/httpwiressl

This program sits on a funny port 9066 rather than port 80 - and pretends to be
a http or https server.

It allows a browser/program to POST files which are treated like an incoming
data stream like a wire service.

A better way of doing this may well be using Apache and scripts like
'fip_recvfile.pl' and 'sffmail'. Use this program where that is a political
nono - for security reasones perhaps - or just too much work to set up.

A small FipHdr is added with date and time fields, remote IPaddresses etc
before the file is passed on - normally to spool/xsmtp for 'ipchkmail' to sort
out.

To install on the normal http port 80, you will need to take Apache/Notes
Domino/IIS/NetScape iPlanet down first before replacing it (so please do make
sure no-one else needs anything on the http server on that system !).
You also need to be 'root' on a Unix box to be able to start httpwire with a
port lower than 1024.

It can optionally also/or be a simple HTTP file server where files are
requested with a GET. If there is a 'get-top-folder' parameter specified in the
paramater, all files under this folder may be GOT. There is also an optional
subparameter - mime to specify what it is being called eg

http://ftp.doodlefoodle.com:9699/20091007_bm/1216_90177_bia070b1_7557634.pdf?mime=application/pdf

    if the 'get-top-folder:/fip/data/bmstaff', then file
/fip/data/bmstaff/20091007_bm/1216_90177_bia070b1_7557634.pdf will be returned
with a mime type of application/pdf

For POSTs or GETs, actual reply messages - XML, Plan text, JSON or whatever -
can be customised by creating files in tables/wire/httpwire. These files can be
called just the HTTP return code : 201, 404, 501 (and -1 for timeouts) or can
have the SU prefixed :
eg if the run line is
        'httpwire -s+ -P9876 -n json'

HTTPWIRE needs a parameter file to start with - in tables/wire - which defaults
to HTTPWIRE.  Often it contains just a couple of comment lines !
    ; comment
    script: Script to run BEFORE returning the answerback (in FipSeq)
        The return code is put into a temporary FipHdr field V5 if you need to use it
in the reply message
        script:/fip/local/unzip_docx.pl debug=yes /fip/data/ohno/\EN
    code-script-error: (number)
        HTTP return code if the script errors.      default: 200
    reply-template-path: (file path for return message)
        data is assumed to be in FipSeq
        If in the parameter file there is
            ; path and filestub of reply messages
            reply-template-path:/fip/tables/wire/httpwire/JSON
            httpwire will look in tables/wire/httpwire for a file of plain text or
FipSeq called 'JSON.200' and if not found, one called '200'.
        Default is /fip/tables/wire/httpwire with no file prefix for files called
200, 404, -1 (for timeouts)
    parse-reply:yes/no
        if display-file/diplay-file-binary is NOT set, use this to Parse-the-reply OR
leave it as binary
        ie the reply can be in FipSeq and codes/words/strings replaced outbound from
:
            - the contents of FipHdr of the file (if there is one)
            - tmp FipHdr fields V1 to V5 (see below)
            - field SM has any reply message and S2 has the reply result code.
                S2 will normally be 200 or 404

    allow-post: (yes/no)
        default is yes to allow incoming files to be POSTed
        if yes, you MUST specify an output-folder (or more than one)
    output-folder: (folder name)
    output-folder1: (folder name)
    ..
    output-folder9: (folder name)
        if the folder does NOT start with a '/', it is assumed to be
        Note these override the default and '-o' input switch..

    get-top-folder: (folder)
        Pls make sure permissions are set so that the correct logon has read access
to all files.
        default: none ie no GETs are serviced
    default-mime-type: (mime type)
        If allowing GETs, set the default mimetype
        default: text/plain (or look for mime=xxx in the incoming url)

    use-ssl: (yes/no)
        use https (use-ssl: yes) or http (no, default) - BUT you cannot have both !
        this overrides input switch -SSL
    ssl-method: tls tls1 tls1.1 tls1.2 sslv2 sslv3 sslv2and3
        Version number to use for TLS/SSL       default: 999 for current default (2 or 3)
        (only the digits are significant, so add other text to make it readable)
        For 'modern' connection, pls do NOT use sslv2 ! as it is deemed insecure
        If default it will check the available list and pick the highest.
        The default is currently 23 which on a modern server is sslv3 and tls1_2 !)
    ssl-password: (password)
    ssl-passwd: (password)            default: none
        Optional password if the handshake requires a shared secret
    ssl-cert: (name of a PEM certificate file)      default: none
    ssl-root-cert: (name of a root PEM certificate file)    defaunt: none
        Optional certificates - held in tables/ssl
    ssl-key: (name of a PEM key file)           default: none
    ssl-verify: yes/no  verify certificates     default: yes
    ssl-ciphers: (list) acceptable ciphers
        (use 'openssl ciphers' to list)
        default:  "HIGH:!aNULL:!kRSA:!SRP:!PSK:!CAMELLIA:!RC4:!MD5:!DSS"

    for GETting files ...
    code-default: (3 digit code in range 200-599)
        reply code for ok-done if no file       default: 404
    code-getmissing: (3 digit code in range 200-599)
        reply code for a missing file           default: 404
    code-getnofile: (3 digit code in range 200-599)
        reply code for no file specified        default: 404

    for POSTing/receiving data ..
    reply-mime-type: (mime type)    default is text/html
        This is just the Content-type mime header in all cases except JSON where if
            reply-mime-type:application/json
        then the return codes are checked JSON.200 etc
    reply-default: (Fipseq)
        reply message for ok-done if no action      default:"Thank you"
    code-post-nodata: (3 digit code in range 200-599)
        reply code for if no data was sent      default: 404
    reply-post-nodata: (FipSeq)
        no default
    code-post-systemerror: (3 digit code in range 200-599)
        reply code for if unable to write data      default: 501
    reply-post-systemerror: (FipSeq)
        no default

    extra-get-httphdr: (FipSeq)
        Add extra HTTP mime header lines for GET and POST
        REMEMBER to add \r\n to the end of each line
        eg extra-get-httphdr:Content-Disposition:
name="\AN.\QE"\r\nX-FipHdr-DI:\DI\r\n
        no default
    extra-reply-httphdr: (FipSeq)
        Add extra HTTP mime header lines for a reply message
        REMEMBER to add \r\n to the end of each line
        eg extra-reply-httphdr:Pragma: no-cache\r\n
        no default

    allow: (IPaddress to allow)
    disallow: (IPaddress to block)
        use this for blacklist/whitelist certain addresses

    log-folder: (FipSeq)
        folder to put log files             default: no logging
    log-name: (FipSeq)
        Name of a log file to write/append to       default: no logging
    log-script: (FipSeq)
        Name of a script to run against the log file    default: no logging

    dump-data: (yes/no/FipSeq)
        Save /Dump a copy of the each request and response and data in a dump file in
/fip/dump default:no

Httpwire automatically adds a number of temporary FipHdr fields for each
access.
For GETting files, the following are useable
    EN - the Get or Post command
    V1 - Path/Filename as from EN
    V2 - Metadata as added by the ..&meta=xxx
    V3 - MimeType as added by the ..&mime=xxx
    V4 - Filename as added by the ..&name=xxx or just the filename from the EN
For incoming files (using a POST for example) the
    EN - the Get or Post command
    V1 - Path/Filename as from EN
    V5 - script result (if running a script)

Input Parameters :
Mandatory
    -n : name of this service               default: none
        This is also the name of a parameter file in tables/wire
Optional :

    -A : name of the archive file if not the -n name field  default: 'name'
    -c : the chrset of the source (SC header field)     default: ascii
    -C : Do NOT reestablish the link after disconnection    default: do
    -d : the name of a DUPLICATE wire where 2 copies of the same
        file is required (SD header field).     default: none
    -D : Display incoming data              default: no
        Use this for debugging incoming connections and data.
    -f : Extra FIP header information           default: none
        For fixed header info in FIP. eg -f #QA:AA#QB:BASIC
        As this flag is normally the last specified, its contents
        can be used to overwrite any unique fields such as DU, DP,
        SN etc.
    -I : id of this instance                default: ignored
        Where there are several copies of 'wire' running with the same -n
    -K : Close the connection after each POST       default: keep alive
    -s : hostname/internet address to select        default: systemname on boot
    -h : hostname/internet address to select        default: systemname on boot
        for servers with more than one card/address
        -s and -h are identical
        use '-s +' to listen on all ip addresses on that box.
    -l : Log items thru
    -L : detailed log thru                  default: no
    -o : Output folder in /fip/spool            default: spool/xsmtp
        Note this will be overridden if there are any 'output-folderX:' parameters in
the parameter file.
    -O : Name of output format (DF field)           default: HTTPWIRE
    -P : port number to use                 default: 9066
        Note that the normal http port is 80. On some platforms - Linux
        for example - Fip has to be started by 'root' to use ports under 1024.
    -r : the name of a DIFFERENT routing table to 'name'
        (SR field : used by iproute)            default: name
    -SSL : Force HTTPS (ie TLS/SSL)             default: no - http:
    -t : timeout with no data               default: 10 secs
            After this the connection is closed.
    -T : top folder for gets                default: none
    -u : logon for files created if NOT that
        which was used to start 'httpwire'      default: same
    -V : HTTPS TLS/SSL method to use            default: loop around
    -Z : do NOT archive any incoming files          default: archive
    -v : Print the version number and exit

-- TLS/SSL notes:
 Use -SSL input switch to use - and optionally change the version number with
-V.

Prerequsite is the SSL layer which nowadays comes as standard on most
platforms. Otherwise it can be downloaded from the installation kit or the
website of the OS - Sun for Solaris, RedHat, Suse etc
    http://www.openssl.org
For *nix, if you have gcc installed, it is usually easier (!) to compile from
the latest sources at http://www.openssl.org/source/
For Win2k, there are precompiled versions at
    http://www.openssl.org/related/binaries.html
which points at
    http://www.slproweb.com/products/Win32OpenSSL.html
NOTE you generally have to also add the 'MicroSoft Visual C++ 2008
Redistributables' (vcredit)
Pick the 32bit Light version unless you are running very very high volume
stuff.

Version Control
;0s16   23may00 original version from smtpwire
    ;f-m 17apr02 added -D display data and -C
    ;n 31oct03 added timings
    ;o 24sep04 speedy
    ;p 08apr06 added script and display-file-binary
    ;q 27feb09 added output-folder0-9:
    ;r3 14sep09 better GET ;4-7 04jan10 added TLS/SSL, added -I ;8 better 404
messages
    ;s3 13apr10 uploadform better, added blacklists, (ignores favicon and gets
css)
     ;3-7 16apr12 added reply* ;8 issue with HTTP/1.1
     ;9-14 15feb17 openssl 1.1.0 rework and stop /../../
     ;15-16 20mar22 added better logging and dump

(copyright) 2024 and previous years FingerPost Ltd.