ipremsvr
ipremsvr
This program handles the connection with the outside world.
There are three basic modes which it can work :
1. Permanent connection for a single client
2. Start-stop connections for a group of clients
This uses the -g input switch to hold a list of clients.
3. Local copy for a group of clients using other transmissions
such as email/SMTP and FTP which are serviced by 'ipsmtp' and 'ipftp'
This uses the -LOCAL input switch for each client.
It copes with connections which are
either client or server initiated
ie Active or Passive - also called Push or Pull
either broadcast or FipServer protocol
ie fipagent or not - handshaking or serial-over-ip
either permanent or start-stop connections
It adds the format and modifies the chrset (or converts the file type for
non-text files) according to the client's requirements.
'ipremsvr' first reads a parameter file in either
tables/remote/groups/GROUPNAME if the '-g' switch is on
or tables/remote/clients/CLIENTNAME.
This will point to a format file too which is in tables/remote/formats
if a text-format has been specified , that is also read...
The GROUP file (if used) has the following syntax:
; comment
client: (clientname)
There can be up to 10 clients specified. Each one will have
a client file.
connection-started-by:client or server
Who makes the initial call - client connecting to the server
or vice versa. default is server
(if not LOCAL client)
port: local port if connection started by client.
(if not LOCAL client)
Up to 10 clients may be specified - ALL using the same 'connection-started-by'.
The 'connection-type' is 'start-stop' unless forced to be LOCAL by the '-L'
input switch
The CLIENT file has the following syntax:
; comment
delivery-method:bdcast or server or local
The two options are 'bdcast' for one-way traffic (default) and
'server' for fip-agent type traffic.
If using the -L for Local, this is ignored.
If using the 'fipremote' program at the remote site and the
delivery-method is 'broadcast', make sure that the 'end-of-message' in
remote's 'REMOTE.FIP' parameter file is matched in the format for
'ipremsvr'.
connection-started-by:client or server
Who makes the initial call - client connecting to the server
or vice versa. Default is server
If using the -L for Local, this is ignored.
connection-type: permanent or start-stop
The connection is dropped for Start/Stop clients immediately
after all valid data has been sent and acknowledged.
This is used for low-data clients and allows other clients
to be serviced almost at the same time.
If using the -L for Local, this is ignored.
local-folder: for LOCAL clients, this determines the local folder to stuff
data files.
The default is /fip/spool/remotes/(clientname forced lowercase)
output-filename : Define the filename (for both local and remote
clients. The default is (SERVICENAME) (SEQUENCE NO)
eg output-filename:\$d\R9\R7
local-extra-fiphdr: (fipSeq) Extra metadata to add to the FipHdr for LOCAL
client files.
default: none
add-local-fiphdr: (yes/no) Add/do not add the FipHdr to the LOCAL client
files.
default: add
host: hostname of the remote
port: port of the remote
password: Password the client uses to logon
no-password: There is no Password handshake
no-logon-required: There is no Password handshake
(These two keywords have the same effect)
service: name of a service that this client will receive
There can be as many 'service' lines as required.
Each line specifies one and only one service.
The name of the service MUST be exactly the same as specified
in the main SERVICES file as used by 'ipremdata'.
Eg :
service:europe
service:biz
service:commodities
service:funnies
ignoreforservice: Ignore certain files in a particular service
eg ignoreforservice:epd QA=pp
if the FipHdr field QA is 'pp' then ignore that file on service EPD
http-proxy-host: Hostname of Proxy HTTP server. default: direct connection
http-proxy-port: Port Number of Proxy server. default: direct connection
http-proxy-logon: This is the logon and password to get thru the firewall
if required. The format is (logon) (colon) (password) and is
converted to base 64.
http-proxy-logon:Y2hyaXMuaHVnaGpvbmVzOnBhbnRoZXIK=
To generate (use 'type' on Winnt and just 'echo' for other Unix) :
echo -n "logon:password" | sffb64 -i
eg echo -n "chris:sleekpanther" | sffb64 -i
gives Y2hyaXM6c2xlZWtwYW50aGVy
http-proxy-logon:Y2hyaXM6c2xlZWtwYW50aGVy=
protocol-header-file:
protocol-trailer-file:
Used where the protocol, like HTTP, requires extra information
before and/or after the data.
text-format: (name of a format file in remote/formats)
encrypt-size: (number)
encrypt-string: (FipSeq string of encrypt-size)
These state a length and string to map to the data.
They MUST be exactly the same as the Remote system.
allow-ip-address: 192.3.22.1
reject-ip-address: (IP addresses/mask)
Allow or reject connections from named systems.
up to 100 addresses may be specified.
if one or more 'allow's are specified, then the
client MUST connect from one of these addresses.
if start-stop,
reconnect-every: Number of seconds to wait
do-not-connect-on-days:sat,sun
do-not-connect-before-time:09:21
do-not-connect-after-time:19:30
check-message-timeout:600
Number of seconds of quiet before a Check Message is sent
default is 0 for No check messages.
check-message-file:
Name of a file in tables/remote which contains a Data file
with a suitable FipHdr to use for a Check Message
The default is CHECK_MESSAGE
response-timeout:60
Timeout for 'server' type connections waiting for an
acknowledgement for the data.
speed: (number)
Pace the data out at this speed (in bps).
Use this where a serial line with no flow control is on
the circuit. eg speed:9600
default: not specified.
allow-remote-repeats: yes/no
Allow the remote user to specify a sequence number to restart transmissions
on.
default is no
fiphdr-client-id: (2 chr FipHdr field)
FipHdr field in the incoming data which holds the client id which will be
used for resends.
default the absolute sequence id.
dump-data:
archive the data sent in a dump file in /fip/dump/DOY_CLIENT
do-not-send-duplicates:
If a single file is in more than one service, then it is normally
a copy is normally sent for EACH service.
Use this flag to send only one copy.
There is an item in the log which says that rg second and subsequents are NOT
sent.
send-all-old-data:
Normally all files are sent to a client.
However if the client disconnects at the end of a day (ie before
midnight) and does not reconnect for hours afterwards, they
will normally ONLY get files from midnight.
Use this option to force all of the previous day too.
send-no-old-data:
Send no back data previous to the logon time (see above)
create-ZQ:
Use this to create a fake output sequence number in the FipHdr
field ZQ. If there is a number to this parameter it will be used
as the amount to increment - default is 2.
eg default create ZQ:3-4, the next ZQ:5-6
if create-ZQ:12, file will have ZQ:22-33 then ZQ:34-45 etc.
if create-ZQ:1, file will have ZQ:2-2 then ZQ:3-3 etc.
track-high-priority:yes/no
If 'ipremdata' is also set to track these, this will send the
HiPris before any other traffic.
This option is NOT for Groups - only single Clients.
This automatically flags 'do-not-send-duplicates:'
log-duplicates:yes/no
create-md5-signature:
check-md5-signature:
Create the MD5 signature of the incoming file and send to the FipRemote
running in Server mode
The file is logged if in error at the remote end.
default: none
output-md5: (FipSeq)
Output this string with the MD5 signature as FipHdr RM AFTER the data has
been sent
Use this for straight braodcast feeds
eg - top and tail with an ETX and EOT
output-md5:\003\RM\004
default: none
encrypt-script:
External script to encrypt or compress the data.
There is one and only one parameter to this script which is
the name of the file to encrypt and in which the new contents
will be stuffed.
encrypt-script:/fip/local/remote_pgp
log-message: (FipSeq)
logging information for fipremote default: nonw
The text-format file has the Syntax of :
; comment
before: Text to add at the top of the data file.
after: Text to add at the bottom of the data file.
filebefore: File to add at the top of the data file.
fileafter: File to add at the bottom of the data file.
zaptags: If the source data is in XML (or HTML or any other tagged
format), zap all the tags.
eg tag:P \r\n\t
tag: Replace a start tag with some string (eg <p>)
endtag: Replace an END tag with some string (eg </p>)
alonetag: Replace an Alone tag with some string (eg <br/>)
Syntax: endtag:(tag) (replacement data in FipSeq)
There can be a 'tag' for each that needs replacing.
Do not use these with wrap - pretty meaningless if you do anyway !
It is case insensitive.
tag:P <text>
endtag:P </text>
alonetag:br <padline/>\n
force-binary: If the text starts with this string, assume it is binary
and make NO changes to end-of-line of characher maps
force-binary:<p>
chr:(chr) (replacement data in FipSeq)
Syntax: chr:(chr) (replacement data in FipSeq)
There can be a 'chr' for each character that needs replacing.
chr:\006 +Bell+
chr:\037 <TD><!-- \037 -->
hdrchr:(chr) (replacement data in FipSeq)
As for 'chr' but for FipHdr fields only.
Use this to map XML entities for example
hdrchr:& &
hdrchr:< <
hdrchr:> >
hdrchr:" '
txchr:(chr) (replacement data in FipSeq)
As for 'chr' but for 'before', 'after'. 'filebefore','fileafter'
Use this for any '#' in the FipHdr fields for example
paragraph-marker: End of line sequence in FipSeq
or eoln: End of line sequence in FipSeq
default is to leave the end-of-line as in the data
ignore-eoln: Test for eoln - if therem, do NOT add the eoln
eg eoln:</p>\n<p>
ignore-eoln:</p>
In this case if there is a single preceeding <p>, no changes to the eoln will
be made
If not, all end of lines (usually NL or CR NL) are replaced with '</p> NL
<p>'
This works on a file by file basis so files with/without <p> can be
interlaced in the output
default - none
wordwrap: Wrap the text at a number of characters per line.
This will wrap on the nearest space to the end of the line.
default is to not change the data at all.
eg Wrap the text at 72 characters.
wordwrap:72
ww-eoln: For word-wrapped lines, End-of-line sequence in FipSeq.
default is CR NL ie '\r\n'
eg ww-eoln:\f
add-wordwrap-for-service: ONLY wordwrap for this service. There can
be several lines each specifying a single service.
remove-wordwrap-for-service: Remove any existing wordwrap for files for
a service. There can be several 'remove-wordwrap-for-service'
lines, one per service.
Eg : if files for service 'biz' are wrapped at 70 chrs and
you want to have NO wordrap :
remove-wordwrap-for-service:biz
This is done BEFORE applying any wordwrap , so if you want to
put a wordwrap on 40 chrs:
remove-wordwrap-for-service:biz
wordrap:40
convert-to-utf8:
Convert the incoming data to the UTF8 variant of Unicode.
max-data-size-for-service:(servce name) (no of chrs)
To create an abstract wire of the first n chrs for a particular service
Headlines only, set this to 0 (zero)
eg max-data-size-for-service:biz 0
default is -1 for ALL data/text
repeat-service-name: Service code for Repeated items. default is 'RPT'
To use the original, use a blank 'repeat-service-name:'
service-name-alias: Original Service code and replacement
eg service-name-alias:XX2 SPT
There can be up to 1000 aliases
The aliased name does not have to be unique
Input Parameters :
Mandatory
Either -g : name of a Groups file in tables/remote/groups default: none
or -n : name of a single Client file in remote/clients default: none
Optional
-N : replacename name for the client default: -n
This allows a 2 clients to use the same parameter file
but have different idents.
Resending files from Fip will go to both clients
Care MUST be taken to have UNIQUE client names on a single server
-s : hostname as override to parameter file default: name
for client-started feeds
eg -s + to listen on all ports
-P : port as override of 'port:' in parameter file default: port
for client-started feeds
-Q : do NOT send Seqnos (ie remote is fipremote < 2e12) default: yes
-l : do NOT log each item default: do
-LOCAL : only process this client (or Group) locally default: network
-T : do NOT log the trace of each transaction default: do
Normally a one line log of each file sent is stored in
log/remote_trace with a name of 'date_client'.
Use this flag to stop this trace.
-V : version level default: 3
(use for old remdata)
-v : Print the version number and exit
FipHdr fields POSSIBLY used in the incoming data and used by ipremdata/svr :
R0- clientname for traffic for a particular client only
This is used for resends and system messages
R0:MegaCorp
R6- Mime Type of file
R6:TEXT - used for flagging to use the TEXT format
Any other value/string is user-defined.
RX- Full path and filename on this device
FipHdr fields added by ipremdata are :
R1- date CCYYMMDD
R1:20000630
R2- hour
R2:09
R3- Overall System item number
R3:06132
R4- Uno
R4:200006300906132
R5- SystemId - not used by RemSvr
FipHdr fields added by ipremsvr are :
R7- Service item number
R8- Client item number
R9- Service name, usually a service code
R9:COM
--------------------------------------------------------------
Version Control
;2z17 15dec03 cleanups
;d-f 27feb04 cleanup Groups
;g-h 29jul04 added 'track-high-priority:'
;i-k 15aug04 added check-md5-signature and encrypt-script
;l 07jan05 bugette over chg of day
;m-n 26jan06 added allow-remote-repeats:yes/no (n dump md5)
;o 24sep06 added tag, endtag, zaptags and alone tags,
;p-q 04nov06 added -P port as override to parameter file
;r 06nov06 added send-no-old-data: and -N
;s-u 22nov06 added ignore-eoln
;v-w5 11dec06 added -k remkey
;x-y6 8may07 added service-name-alias and StuffFile bugette (y4 no-pwd for
fipremote)
;y7-10 for txchr: (10 for space in service-..alias)
;z1-9 17aug08 added take-size/fiphdr/top/tail (2 bugfix for output buffer) (5
new services buglette)
;10-11 27apr10 added local-fiphdr and local-extra-fiphdr
;12 added SEQNO and tracking for super bad links
;13-17 added log-message and -Q for no SEQNO
;001z 22feb01 no-password/no-logon-required added for fipagent too
;a 20mar01 added proxies
;b 27mar01 added wordwrap properly
;c/d/e 20apr01 bugette in non-encrypted feeds
;f 27jun01 WINNT
;g 27nov01 closeup if bad logon.
;h 10dec01 added do-not-send-duplicates.
;i/j 08jan02 added hdrchr
;k/l 09mar02 added remove-wordwrap-for-service
;m 11apr02 added convert-to-utf8/base64
;n 26apr02 added add-wordwrap-for-service
;o 02may02 ignore the Missing msg if Pos=0
;p/q 09may02 added send-all-old-data:
;r/s 12jun02 finally got the svce bug !!
;t 28sep02 added max-data-size-for-service:
;u 27mar03 added ignoreforservice:
;v-y 23may03 added createZQ
;z 05dec03 added repeat-service-name
(copyright) 2025 and previous years FingerPost Ltd.