httpwire
httpwire/httpwiressl
This program sits on a funny port 9066 rather than port 80 - and pretends to be
a http or https server.
It allows a browser/program to POST files which are treated like an incoming
data stream like a wire service.
A better way of doing this may well be using Apache and scripts like
'fip_recvfile.pl' and 'sffmail'. Use this program where that is a political
nono - for security reasones perhaps - or just too much work to set up.
A small FipHdr is added with date and time fields, remote IPaddresses etc
before the file is passed on - normally to spool/xsmtp for 'ipchkmail' to sort
out.
To install on the normal http port 80, you will need to take Apache/Notes
Domino/IIS/NetScape iPlanet down first before replacing it (so please do make
sure no-one else needs anything on the http server on that system !).
You also need to be 'root' on a Unix box to be able to start httpwire with a
port lower than 1024.
It can optionally also/or be a simple HTTP file server where files are
requested with a GET. If there is a 'get-top-folder' parameter specified in the
paramater, all files under this folder may be GOT. There is also an optional
subparameter - mime to specify what it is being called eg
http://ftp.doodlefoodle.com:9699/20091007_bm/1216_90177_bia070b1_7557634.pdf?mime=application/pdf
if the 'get-top-folder:/fip/data/bmstaff', then file
/fip/data/bmstaff/20091007_bm/1216_90177_bia070b1_7557634.pdf will be returned
with a mime type of application/pdf
For POSTs or GETs, actual reply messages - XML, Plan text, JSON or whatever -
can be customised by creating files in tables/wire/httpwire. These files can be
called just the HTTP return code : 201, 404, 501 (and -1 for timeouts) or can
have the SU prefixed :
eg if the run line is
'httpwire -s+ -P9876 -n json'
HTTPWIRE needs a parameter file to start with - in tables/wire - which defaults
to HTTPWIRE. Often it contains just a couple of comment lines !
; comment
script: Script to run BEFORE returning the answerback (in FipSeq)
The return code is put into a temporary FipHdr field V5 if you need to use it
in the reply message
script:/fip/local/unzip_docx.pl debug=yes /fip/data/ohno/\EN
code-script-error: (number)
HTTP return code if the script errors. default: 200
reply-template-path: (file path for return message)
data is assumed to be in FipSeq
If in the parameter file there is
; path and filestub of reply messages
reply-template-path:/fip/tables/wire/httpwire/JSON
httpwire will look in tables/wire/httpwire for a file of plain text or
FipSeq called 'JSON.200' and if not found, one called '200'.
Default is /fip/tables/wire/httpwire with no file prefix for files called
200, 404, -1 (for timeouts)
parse-reply:yes/no
if display-file/diplay-file-binary is NOT set, use this to Parse-the-reply OR
leave it as binary
ie the reply can be in FipSeq and codes/words/strings replaced outbound from
:
- the contents of FipHdr of the file (if there is one)
- tmp FipHdr fields V1 to V5 (see below)
- field SM has any reply message and S2 has the reply result code.
S2 will normally be 200 or 404
allow-post: (yes/no)
default is yes to allow incoming files to be POSTed
if yes, you MUST specify an output-folder (or more than one)
output-folder: (folder name)
output-folder1: (folder name)
..
output-folder9: (folder name)
if the folder does NOT start with a '/', it is assumed to be
Note these override the default and '-o' input switch..
get-top-folder: (folder)
Pls make sure permissions are set so that the correct logon has read access
to all files.
default: none ie no GETs are serviced
default-mime-type: (mime type)
If allowing GETs, set the default mimetype
default: text/plain (or look for mime=xxx in the incoming url)
use-ssl: (yes/no)
use https (use-ssl: yes) or http (no, default) - BUT you cannot have both !
this overrides input switch -SSL
ssl-method: tls tls1 tls1.1 tls1.2 sslv2 sslv3 sslv2and3
Version number to use for TLS/SSL default: 999 for current default (2 or 3)
(only the digits are significant, so add other text to make it readable)
For 'modern' connection, pls do NOT use sslv2 ! as it is deemed insecure
If default it will check the available list and pick the highest.
The default is currently 23 which on a modern server is sslv3 and tls1_2 !)
ssl-password: (password)
ssl-passwd: (password) default: none
Optional password if the handshake requires a shared secret
ssl-cert: (name of a PEM certificate file) default: none
ssl-root-cert: (name of a root PEM certificate file) defaunt: none
Optional certificates - held in tables/ssl
ssl-key: (name of a PEM key file) default: none
ssl-verify: yes/no verify certificates default: yes
ssl-ciphers: (list) acceptable ciphers
(use 'openssl ciphers' to list)
default: "HIGH:!aNULL:!kRSA:!SRP:!PSK:!CAMELLIA:!RC4:!MD5:!DSS"
for GETting files ...
code-default: (3 digit code in range 200-599)
reply code for ok-done if no file default: 404
code-getmissing: (3 digit code in range 200-599)
reply code for a missing file default: 404
code-getnofile: (3 digit code in range 200-599)
reply code for no file specified default: 404
for POSTing/receiving data ..
reply-mime-type: (mime type) default is text/html
This is just the Content-type mime header in all cases except JSON where if
reply-mime-type:application/json
then the return codes are checked JSON.200 etc
reply-default: (Fipseq)
reply message for ok-done if no action default:"Thank you"
code-post-nodata: (3 digit code in range 200-599)
reply code for if no data was sent default: 404
reply-post-nodata: (FipSeq)
no default
code-post-systemerror: (3 digit code in range 200-599)
reply code for if unable to write data default: 501
reply-post-systemerror: (FipSeq)
no default
extra-get-httphdr: (FipSeq)
Add extra HTTP mime header lines for GET and POST
REMEMBER to add \r\n to the end of each line
eg extra-get-httphdr:Content-Disposition:
name="\AN.\QE"\r\nX-FipHdr-DI:\DI\r\n
no default
extra-reply-httphdr: (FipSeq)
Add extra HTTP mime header lines for a reply message
REMEMBER to add \r\n to the end of each line
eg extra-reply-httphdr:Pragma: no-cache\r\n
no default
allow: (IPaddress to allow)
disallow: (IPaddress to block)
use this for blacklist/whitelist certain addresses
log-folder: (FipSeq)
folder to put log files default: no logging
log-name: (FipSeq)
Name of a log file to write/append to default: no logging
log-script: (FipSeq)
Name of a script to run against the log file default: no logging
dump-data: (yes/no/FipSeq)
Save /Dump a copy of the each request and response and data in a dump file in
/fip/dump default:no
Httpwire automatically adds a number of temporary FipHdr fields for each
access.
For GETting files, the following are useable
EN - the Get or Post command
V1 - Path/Filename as from EN
V2 - Metadata as added by the ..&meta=xxx
V3 - MimeType as added by the ..&mime=xxx
V4 - Filename as added by the ..&name=xxx or just the filename from the EN
For incoming files (using a POST for example) the
EN - the Get or Post command
V1 - Path/Filename as from EN
V5 - script result (if running a script)
Input Parameters :
Mandatory
-n : name of this service default: none
This is also the name of a parameter file in tables/wire
Optional :
-A : name of the archive file if not the -n name field default: 'name'
-c : the chrset of the source (SC header field) default: ascii
-C : Do NOT reestablish the link after disconnection default: do
-d : the name of a DUPLICATE wire where 2 copies of the same
file is required (SD header field). default: none
-D : Display incoming data default: no
Use this for debugging incoming connections and data.
-f : Extra FIP header information default: none
For fixed header info in FIP. eg -f #QA:AA#QB:BASIC
As this flag is normally the last specified, its contents
can be used to overwrite any unique fields such as DU, DP,
SN etc.
-I : id of this instance default: ignored
Where there are several copies of 'wire' running with the same -n
-K : Close the connection after each POST default: keep alive
-s : hostname/internet address to select default: systemname on boot
-h : hostname/internet address to select default: systemname on boot
for servers with more than one card/address
-s and -h are identical
use '-s +' to listen on all ip addresses on that box.
-l : Log items thru
-L : detailed log thru default: no
-o : Output folder in /fip/spool default: spool/xsmtp
Note this will be overridden if there are any 'output-folderX:' parameters in
the parameter file.
-O : Name of output format (DF field) default: HTTPWIRE
-P : port number to use default: 9066
Note that the normal http port is 80. On some platforms - Linux
for example - Fip has to be started by 'root' to use ports under 1024.
-r : the name of a DIFFERENT routing table to 'name'
(SR field : used by iproute) default: name
-SSL : Force HTTPS (ie TLS/SSL) default: no - http:
-t : timeout with no data default: 10 secs
After this the connection is closed.
-T : top folder for gets default: none
-u : logon for files created if NOT that
which was used to start 'httpwire' default: same
-V : HTTPS TLS/SSL method to use default: loop around
-Z : do NOT archive any incoming files default: archive
-v : Print the version number and exit
-- TLS/SSL notes:
Use -SSL input switch to use - and optionally change the version number with
-V.
Prerequsite is the SSL layer which nowadays comes as standard on most
platforms. Otherwise it can be downloaded from the installation kit or the
website of the OS - Sun for Solaris, RedHat, Suse etc
http://www.openssl.org
For *nix, if you have gcc installed, it is usually easier (!) to compile from
the latest sources at http://www.openssl.org/source/
For Win2k, there are precompiled versions at
http://www.openssl.org/related/binaries.html
which points at
http://www.slproweb.com/products/Win32OpenSSL.html
NOTE you generally have to also add the 'MicroSoft Visual C++ 2008
Redistributables' (vcredit)
Pick the 32bit Light version unless you are running very very high volume
stuff.
Version Control
;0s16 23may00 original version from smtpwire
;f-m 17apr02 added -D display data and -C
;n 31oct03 added timings
;o 24sep04 speedy
;p 08apr06 added script and display-file-binary
;q 27feb09 added output-folder0-9:
;r3 14sep09 better GET ;4-7 04jan10 added TLS/SSL, added -I ;8 better 404
messages
;s3 13apr10 uploadform better, added blacklists, (ignores favicon and gets
css)
;3-7 16apr12 added reply* ;8 issue with HTTP/1.1
;9-14 15feb17 openssl 1.1.0 rework and stop /../../
;15-16 20mar22 added better logging and dump
(copyright) 2024 and previous years FingerPost Ltd.