icewire
useHttp
POST
icewire
Icewire is an ICE client which connects to a server and collects packages for
the subscriber.
These packages can be updates of weather, financial data, sports results,
backup for Wire services if the satellite is down, graphics, software. In fact
most things.
The program can be used either :
- on a timed basis to get regular known pages.
- on demand by sending a file into spool/icepoll with the FipHdr field DF set
to the parameter file required.
There is a TUNING mode to be used for setting up a new link and trying to clean
up the relevant parameter file WITHOUT sending (possibly) live data to the
required destination.
- This shows the handshaking that taking place
- Use this in particular when accessing the server via a proxy
- To run, choose your parameter file in tables/icewire and run 'icewire'
manually in a window:
icewire -T AUS.STOX | more for prompt before calls
or icewire -A -T AUS.STOX | tee aussies for no prompting
There are Two (sometimes three) types of parameter file :
1. Main Parameter file which sets up the polling of certain pages at set times
(if any).
2. A Page Description file for each site/page accessed.
----- Main Parameter file -----
The syntax of the Main Parameter File - by default tables/icewire/ICE_POLL :
; comment line
poll:(pram file) day:(MTuWThFSaSu) time:20:30 mustget:
In detail, the 'poll' keyword :
Pram file is the name of the Page Description file - see below for its syntax
day: Day of week to run the job :
M Monday
Tu Tuesday
W Wednesday
Th Thursday
F Friday
Sa Saturday
Su Sunday
X Every day.
Case is NOT important.
Commas (but NOT spaces) may be used to separate.
Default is every day.
either
time: Time of the day on 24 hour clock. Default is 18:00.
or
every: interval between grabs Default: none
every: (mins) [(optional) start:(starttime) end:(endtime)
every:30 start:07:30 end:19:00
The minimum interval is 10 mins.
eg:
poll:AP day:ALL time:20:10
Get the Page file tables/icewire/AP every day at 20:10
poll:Forex day:MTuWThF time:16:30
poll:Forex day:MTuWThF time:16:40
Get the Page file tables/icewire/FOREX every week day at 16:30 and 16:40
There can be none or up to 200 polls in the main parameter file.
Note that the page is grabbed ONLY if the program is running.
----- Page Description Parameter files -----
The individual Page description parameter files are also in tables/icewire. The
syntax of these are :
; comment start with a semi colon like this
MANDATORY
url: Full url of the page. default: none
There MUST be one and only one 'url:' specified.
You can also specify the page, cgi and any subparameters.
eg url:www.fingerpost.co.uk
url:www.big-press-org/sports/baseball/index.htm
url:www.marketlook.co.uk/scripts/Summary.dll?HandleSummary
dest: Fip Destination for the files default: ICEDATA
This is the 'DU' FipHdr field as per the USERS file.
eg dest:w3saves
OPTIONAL:
port: Port number of the Remore Server. default: 80
nofiphdr: Do NOT add a Fip Hdr to the file. default: yes pls
source: Fip Source of the files. (FipHdr 'SU'). default: ICE_POLL
Unless 'noarchive' is specified, all data files will be archived
under this name in log/data.
noarchive: Do NOT archive these files in log/data. default: archive
httphdr: Extra lines of HTTP header you may need. default: none
Remember to add a NL at the end of each line.
eg httphdr:Authorization: Basic AbGtGgbhpdOkOTE=\n
post: Post a Form default: get url
see below for processing a form using method=POST.
filename: Filename for the output file in FipSeq. default: ICE\$Z
If this does NOT start with a '/' it is left under the
Output Queue as specified on startup (default spool/2go)
eg filename:AFP\$d.\$z
wild: (FipSeq) Character used as a Wild String for default: '*'
'matchlinks/ignorelinks'.
eg wild:\377
singlewild: (FipSeq) Character used as a single default: '?'
Wild chr for 'matchlinks/ignorelinks'.
eg singlewild:!
number: (o|d|h) Number system for FipSeq default: octal
octal, decimal or hexidecimal
The following are all equivalent :
number:octal
before:\040
number:decimal
before:\032
number:hex
before:\020
before: FipSeq String to add before any data. default: none
after: FipSeq String to add after any data. default: none
script: Script to run on ths data of the incoming file. default: none
log: FipSeq custom logging for the item log.
extra: Extra FipHdr fields to be added default: none
to the output file. To separate FipHdr fields, pls use a '#'.
eg extra:ZH:NYNZ#DI:Headline News#QZ:333
keep-alive: yes/no default: no
Just that ! default:no
http-version: 1.0 or 1.1 default:1.0
only-get-if-modified: (FipSeq message if not found) default: get
This will check the remote server for the time the page was
last modified. This does not work with old servers and some
set to HTTP/1.0.
If modified since, the page is read
If not, the optional message is sent
If there is no message, no data is sent - just a note in the item log
one-output-file: Put ALL data in a single output file.
The default is one file per page/access
Use this with 'values' to create a single output file.
This ONLY uses the FipHdr of the first file if 'values' have been specified.
end-of-document: Where a site is sending really really crap HTML - or XML
use this to state what the last tag.
For no checking at all : end-of-document:
Default: end-of-document:</HTML>
proxy-server: If using a proxy, these are the name and port to aim at.
proxy-port:
proxy-logon: This is the logon and password to get thru the firewall
if required. The format is (logon) (colon) (password) and is
converted to base 64.
proxy-logon:Y2hyaXMuaHVnaGpvbmVzOnBhbnRoZXIK=
To generate :
echo -n "logon:password" | sffb64 -i
eg echo -n "chris:sleekpanther" | sffb64 -i
gives Y2hyaXM6c2xlZWtwYW50aGVy
proxy-logon:Y2hyaXM6c2xlZWtwYW50aGVy=
Note that in the FipHdr - unless the 'nofiphdr' keyword has been requested, the
following fields will be filled in :
Day and time in the normal HH,HD,HY etc fields
ST host
SP port
SF path/filename
Input Parameters (all optional) :
either -i : scrolled queue default: spool/icepoll
This allow a variety of parameter files to be run
or -1 : Run a single time and exit default: spool
The parameter is the name of the individual parameter file
in tables/icewire (ie NOT The top or main parameter file)
or -T : Tuning mode default: spool
Display links and data for the page requested. Runs only that page
and then exits.
The parameter is the name of the individual parameter file
in tables/icewire (ie NOT The top or main parameter file)
-A : In Tuning mode, do NOT prompt default: prompt
before searching a link
-o : output queue in 'spool' default: spool/2go
This is ignoring in Tuning mode.
-w : Wait in seconds between accessing links. default: 5
-z : parameter file in 'tables/icewire'. default: ICE_POLL
-v : Print the version number and exit
---- Other Notes ----
-- Proxies Proxies Proxies Proxies Proxies Proxies Proxies Proxies Proxies
Proxies
When running through a proxy server, you will need :
1. hostname of the proxy server
2. port number on the proxy server if it is NOT port 80
3. (optionally) a logon and password
If this information is NOT available, normally you can find it easily from any
PC or Mac on the internal network using a browser like Netscape or IExplorer.
Start a NEW copy of either of these. - It must be a new copy to check on
logons etc.
Under 'Preferences' or 'Internet Options' there should be a 'Connections'
section and under that, the host name or ip address plus host name of any proxy
used.
Note that often the main Fip server is NOT running DNS and will not be able to
resolve external hostnames, so the IP address must be used in this case.
Enter these values in the Fip parameter file as :
proxy-server:195.13.83.7 (no default)
proxy-port:412 (this defaults to port 80)
Use the Browser to attempt to access a ice site outside the firewall - like
'www.fingerpost.co.uk'.
If you are asked for a password to get through, you will probably need to add a
'proxy-logon' parameter too unless the keeper of the Firewall has made a hole
through just for you.
The data for 'proxy-logon' is in base64 in the format (logon) (colon)
(password).
Use 'sffb64' to generate this string :
On a Sparc echo -n "chris:magicman" | sffb64 -i
On Linux echo "chris:magicman" | sffb64 -i
On Winnt type "chris:magicman" | sffb64 -i
proxy-logon:Y2hyaXM6bWFnaWNtYW4===
The actual 'You need to Logon, Pal' message is a '407 Authentication Required'
message.
-- Others Others Others Others Others Others Others Others Others Others Others
--Where a logon and password is requested as part of the Browser - ie a pop-up
from Netscape or IExplorer, NOT an HTML form - you will need to add a
'Authorization' line. This will be true if you get a message like :
HTTP/1.0 999 Authorization failure
... etc etc etc ...
Assuming you know your logon and password :
1. Use uuencode or sffb64 to generate a Base64 string
echo -n "logon:passwd" | sffb64 -i
2. Add an extra line to the parameter file with the result of the sffb64 line
using 'httphdr'.
Syntax: Authorization (colon) (spc) Basic (spc) (Base64 of logon:password)
(\n FipSeq for NL)
Eg httphdr:Authorization: Basic AbGtGgbhpdOkOTE=\n
(copyright) 2025 and previous years FingerPost Ltd.