httpwire (Sat Oct 25 2014 01:31:01)

httpwire

This program sits on a funny port 9066 rather than port 80 - and pretends to be
a http or https server.

It allows a browser/program to POST files which are treated like an incoming
data stream like a wire service.

A better way of doing this may well be using Apache and scripts like
'fip_recvfile.pl' and 'sffmail'. Use this program where that is a political
nono - for security reasones perhaps - or just too much work to set up.

A small FipHdr is added with date and time fields, remote IPaddresses etc
before the file is passed on - normally to spool/xsmtp for 'ipchkmail' to sort
out.

To install on the normal http port 80, you will need to take Apache/Notes
Domino/IIS/NetScape iPlanet down first before replacing it (so please do make
sure no-one else needs anything on the http server on that system !).
You also need to be 'root' on a Unix box to be able to start httpwire with a
port lower than 1024.

It can optionally also/or be a simple HTTP file server where files are
requested with a GET. If there is a 'get-top-folder' parameter specified in the
paramater, all files under this folder may be GOT. There is also an optional
subparameter - mime to specify what it is being called eg


http://ftp.doodlefoodle.com:9699/20091007_bm/1216_90177_bia070b1_7557634.pdf?mime=application/pdf

	if the 'get-top-folder:/fip/data/bmstaff', then file
/fip/data/bmstaff/20091007_bm/1216_90177_bia070b1_7557634.pdf will be returned
with a mime type of application/pdf

For POSTs or GETs, actual reply messages - XML, Plan text, JSON or whatever -
can be customised by creating files in tables/wire/httpwire. These files can be
called just the HTTP return code : 201, 404, 501 (and -1 for timeouts) or can
have the SU prefixed :
eg if the run line is
		'httpwire -s+ -P9876 -n json'

HTTPWIRE needs a parameter file to start with - in tables/wire - which defaults
to HTTPWIRE.  Often it contains just a couple of comment lines !
	; comment
	script: Script to run BEFORE returning the answerback (in FipSeq)
		The return code is put into a temporary FipHdr field V5 if you need to use it
in the reply message
		script:/fip/local/unzip_docx.pl debug=yes /fip/data/ohno/EN
	code-script-error: (number)
		HTTP return code if the script errors.		default: 200
	reply-template-path: (file path for return message)
		data is assumed to be in FipSeq
		If in the parameter file there is 
			; path and filestub of reply messages
			reply-template-path:/fip/tables/wire/httpwire/JSON
	 		httpwire will look in tables/wire/httpwire for a file of plain text or
FipSeq called 'JSON.200' and if not found, one called '200'.
		Default is /fip/tables/wire/httpwire with no file prefix for files called
200, 404, -1 (for timeouts)
	parse-reply:yes/no
		if display-file/diplay-file-binary is NOT set, use this to Parse-the-reply OR
leave it as binary
		ie the reply can be in FipSeq and codes/words/strings replaced outbound from
:
			- the contents of FipHdr of the file (if there is one)
			- tmp FipHdr fields V1 to V5 (see below)
			- field SM has any reply message and S2 has the reply result code.
				S2 will normally be 200 or 404 

	output-folder: (folder name)
	output-folder1: (folder name)
	..
	output-folder9: (folder name)
		if the folder does NOT start with a '/', it is assumed to be 
		Note these override the default and '-o' input switch..

	get-top-folder: (folder)
		Pls make sure permissions are set so that the correct logon has read access
to all files.
		default: none ie no GETs are serviced
	default-mime-type: (mime type)
		If allowing GETs, set the default mimetype
		default: text/plain (or look for mime=xxx in the incoming url)

	ssl-method: (1,2,3,23,999)
		Version number to use for TLS/SSL		default: 999 for current default (2 or 3)
	ssl-password: (password)
	ssl-passwd: (password)					default: none
		Optional password if the handshake requires a shared secret
	ssl-cert: (name of a PEM certificate file)		default: none
	ssl-root-cert: (name of a root PEM certificate file)	defaunt: none
		Optional certificates - held in tables/ssl

	for GETting files ...
	code-default: (3 digit code in range 200-599)
		reply code for ok-done if no file		default: 404
	code-getmissing: (3 digit code in range 200-599)
		reply code for a missing file			default: 404
	code-getnofile: (3 digit code in range 200-599)
		reply code for no file specified		default: 404

	for POSTing/receiving data ..
	reply-mime-type: (mime type)	default is text/html
		This is just the Content-type mime header in all cases except JSON where if
			reply-mime-type:application/json
		then the return codes are checked JSON.200 etc
	reply-default: (Fipseq)
		reply message for ok-done if no action		default:"Thank you"
	code-post-nodata: (3 digit code in range 200-599)
		reply code for if no data was sent		default: 404
	reply-post-nodata: (FipSeq)
		no default
	code-post-systemerror: (3 digit code in range 200-599)
		reply code for if unable to write data		default: 501
	reply-post-systemerror: (FipSeq)
		no default

	extra-get-httphdr: (FipSeq)
		Add extra HTTP mime header lines for GET and POST
		REMEMBER to add rn to the end of each line
		eg extra-get-httphdr:Content-Disposition:
name="AN.QE"rnX-FipHdr-DI:DIrn
		no default
	extra-reply-httphdr: (FipSeq)
		Add extra HTTP mime header lines for a reply message
		REMEMBER to add rn to the end of each line
		eg extra-reply-httphdr:Pragma: no-cachern
		no default

	allow: (IPaddress to allow)
	disallow: (IPaddress to block)
		use this for blacklist/whitelist certain addresses

Httpwire automatically adds a number of temporary FipHdr fields for each
access.
For GETting files, the following are useable
	EN - the Get or Post command
	V1 - Path/Filename as from EN
	V2 - Metadata as added by the ..&meta=xxx
	V3 - MimeType as added by the ..&mime=xxx
	V4 - Filename as added by the ..&name=xxx or just the filename from the EN
For incoming files (using a POST for example) the
	EN - the Get or Post command
	V1 - Path/Filename as from EN
	V5 - script result (if running a script)

Input Parameters :
Mandatory
	-n : name of this service				default: none
		This is also the name of a parameter file in tables/wire
Optional :

	-A : name of the archive file if not the -n name field	default: 'name'
	-c : the chrset of the source (SC header field)		default: ascii
	-C : Do NOT reestablish the link after disconnection	default: do
	-d : the name of a DUPLICATE wire where 2 copies of the same
		file is required (SD header field).		default: none
	-D : Display incoming data				default: no
		Use this for debugging incoming connections and data.
	-f : Extra FIP header information 			default: none
		For fixed header info in FIP. eg -f #QA:AA#QB:BASIC
		As this flag is normally the last specified, its contents
		can be used to overwrite any unique fields such as DU, DP,
		SN etc.
	-I : id of this instance				default: ignored
		Where there are several copies of 'wire' running with the same -n
	-K : Close the connection after each POST		default: keep alive
	-s : hostname/internet address to select		default: systemname on boot
	-h : hostname/internet address to select		default: systemname on boot
		for servers with more than one card/address
		-a and -h are identical
		use '-s +' to listen on all ip addresses on that box.
	-l : Log items thru
	-L : detailed log thru					default: no
	-o : Output folder in /fip/spool			default: spool/xsmtp
		Note this will be overridden if there are any 'output-folderX:' parameters in
the parameter file.
	-O : Name of output format (DF field)			default: HTTPWIRE
	-P : port number to use					default: 9066
		Note that the normal http port is 80. On some platforms - Linux
		for example - Fip has to be started by 'root' to use ports under 1024.
	-r : the name of a DIFFERENT routing table to 'name'
		(SR field : used by iproute)			default: name
	-SSL : Force HTTPS (ie TLS/SSL)				default: no - http:
	-t : timeout with no data				default: 10 secs
			After this the connection is closed.
	-T : top folder for gets				default: none
	-u : logon for files created if NOT that
		which was used to start 'httpwire'		default: same
	-V : HTTPS TLS/SSL method to use			default: 23 for 2 or 3
	-Z : do NOT archive any incoming files			default: archive
	-v : Print the version number and exit

-- TLS/SSL notes:
 Use -SSL input switch to use - and optionally change the version number with
-V.

Prerequsite is the SSL layer which nowadays comes as standard on most
platforms. Otherwise it can be downloaded from the installation kit or the
website of the OS - Sun for Solaris, RedHat, Suse etc
	http://www.openssl.org
For *nix, if you have gcc installed, it is usually easier (!) to compile from
the latest sources at http://www.openssl.org/source/
For Win2k, there are precompiled versions at
	http://www.openssl.org/related/binaries.html
which points at
	http://www.slproweb.com/products/Win32OpenSSL.html
NOTE you generally have to also add the 'MicroSoft Visual C++ 2008
Redistributables' (vcredit)
Pick the 32bit Light version unless you are running very very high volume
stuff.

Version Control
;000s7	23may00 original version from smtpwire
	;f-m 17apr02 added -D display data and -C
	;n 31oct03 added timings
	;o 24sep04 speedy
	;p 08apr06 added script and display-file-binary
	;q 27feb09 added output-folder0-9:
	;r3 14sep09 better GET ;4-7 04jan10 added TLS/SSL, added -I ;8 better 404
messages
	;s3 13apr10 uploadform better, added blacklists, (ignores favicon and gets
css)
		;3-7 16apr12 added reply*

(copyright) 2014 and previous years FingerPost Ltd.