ipremsvr This program handles the connection with the outside world. There are three basic modes which it can work : 1. Permanent connection for a single client 2. Start-stop connections for a group of clients This uses the -g input switch to hold a list of clients. 3. Local copy for a group of clients using other transmissions such as email/SMTP and FTP which are serviced by 'ipsmtp' and 'ipftp' This uses the -LOCAL input switch for each client. It copes with connections which are either client or server initiated ie Active or Passive - also called Push or Pull either broadcast or FipServer protocol ie fipagent or not - handshaking or serial-over-ip either permanent or start-stop connections It adds the format and modifies the chrset (or converts the file type for non-text files) according to the client's requirements. 'ipremsvr' first reads a parameter file in either tables/remote/groups/GROUPNAME if the '-g' switch is on or tables/remote/clients/CLIENTNAME. This will point to a format file too which is in tables/remote/formats if a text-format has been specified , that is also read... The GROUP file (if used) has the following syntax: ; comment client: (clientname) There can be up to 10 clients specified. Each one will have a client file. connection-started-by:client or server Who makes the initial call - client connecting to the server or vice versa. default is server (if not LOCAL client) port: local port if connection started by client. (if not LOCAL client) Up to 10 clients may be specified - ALL using the same 'connection-started-by'. The 'connection-type' is 'start-stop' unless forced to be LOCAL by the '-L' input switch The CLIENT file has the following syntax: ; comment delivery-method:bdcast or server or local The two options are 'bdcast' for one-way traffic (default) and 'server' for fip-agent type traffic. If using the -L for Local, this is ignored. If using the 'fipremote' program at the remote site and the delivery-method is 'broadcast', make sure that the 'end-of-message' in remote's 'REMOTE.FIP' parameter file is matched in the format for 'ipremsvr'. connection-started-by:client or server Who makes the initial call - client connecting to the server or vice versa. Default is server If using the -L for Local, this is ignored. connection-type: permanent or start-stop The connection is dropped for Start/Stop clients immediately after all valid data has been sent and acknowledged. This is used for low-data clients and allows other clients to be serviced almost at the same time. If using the -L for Local, this is ignored. local-folder: for LOCAL clients, this determines the local folder to stuff data files. The default is /fip/spool/remotes/(clientname forced lowercase) output-filename : Define the filename (for both local and remote clients. The default is (SERVICENAME) (SEQUENCE NO) eg output-filename:\$d\R9\R7 local-extra-fiphdr: (fipSeq) Extra metadata to add to the FipHdr for LOCAL client files. default: none add-local-fiphdr: (yes/no) Add/do not add the FipHdr to the LOCAL client files. default: add host: hostname of the remote port: port of the remote password: Password the client uses to logon no-password: There is no Password handshake no-logon-required: There is no Password handshake (These two keywords have the same effect) service: name of a service that this client will receive There can be as many 'service' lines as required. Each line specifies one and only one service. The name of the service MUST be exactly the same as specified in the main SERVICES file as used by 'ipremdata'. Eg : service:europe service:biz service:commodities service:funnies ignoreforservice: Ignore certain files in a particular service eg ignoreforservice:epd QA=pp if the FipHdr field QA is 'pp' then ignore that file on service EPD http-proxy-host: Hostname of Proxy HTTP server. default: direct connection http-proxy-port: Port Number of Proxy server. default: direct connection http-proxy-logon: This is the logon and password to get thru the firewall if required. The format is (logon) (colon) (password) and is converted to base 64. http-proxy-logon:Y2hyaXMuaHVnaGpvbmVzOnBhbnRoZXIK= To generate (use 'type' on Winnt and just 'echo' for other Unix) : echo -n "logon:password" | sffb64 -i eg echo -n "chris:sleekpanther" | sffb64 -i gives Y2hyaXM6c2xlZWtwYW50aGVy http-proxy-logon:Y2hyaXM6c2xlZWtwYW50aGVy= protocol-header-file: protocol-trailer-file: Used where the protocol, like HTTP, requires extra information before and/or after the data. text-format: (name of a format file in remote/formats) encrypt-size: (number) encrypt-string: (FipSeq string of encrypt-size) These state a length and string to map to the data. They MUST be exactly the same as the Remote system. allow-ip-address: 18.104.22.168 reject-ip-address: (IP addresses/mask) Allow or reject connections from named systems. up to 100 addresses may be specified. if one or more 'allow's are specified, then the client MUST connect from one of these addresses. if start-stop, reconnect-every: Number of seconds to wait do-not-connect-on-days:sat,sun do-not-connect-before-time:09:21 do-not-connect-after-time:19:30 check-message-timeout:600 Number of seconds of quiet before a Check Message is sent default is 0 for No check messages. check-message-file: Name of a file in tables/remote which contains a Data file with a suitable FipHdr to use for a Check Message The default is CHECK_MESSAGE response-timeout:60 Timeout for 'server' type connections waiting for an acknowledgement for the data. speed: (number) Pace the data out at this speed (in bps). Use this where a serial line with no flow control is on the circuit. eg speed:9600 default: not specified. allow-remote-repeats: yes/no Allow the remote user to specify a sequence number to restart transmissions on. default is no fiphdr-client-id: (2 chr FipHdr field) FipHdr field in the incoming data which holds the client id which will be used for resends. default the absolute sequence id. dump-data: archive the data sent in a dump file in /fip/dump/DOY_CLIENT do-not-send-duplicates: If a single file is in more than one service, then it is normally a copy is normally sent for EACH service. Use this flag to send only one copy. There is an item in the log which says that rg second and subsequents are NOT sent. send-all-old-data: Normally all files are sent to a client. However if the client disconnects at the end of a day (ie before midnight) and does not reconnect for hours afterwards, they will normally ONLY get files from midnight. Use this option to force all of the previous day too. send-no-old-data: Send no back data previous to the logon time (see above) create-ZQ: Use this to create a fake output sequence number in the FipHdr field ZQ. If there is a number to this parameter it will be used as the amount to increment - default is 2. eg default create ZQ:3-4, the next ZQ:5-6 if create-ZQ:12, file will have ZQ:22-33 then ZQ:34-45 etc. if create-ZQ:1, file will have ZQ:2-2 then ZQ:3-3 etc. track-high-priority:yes/no If 'ipremdata' is also set to track these, this will send the HiPris before any other traffic. This option is NOT for Groups - only single Clients. This automatically flags 'do-not-send-duplicates:' log-duplicates:yes/no create-md5-signature: check-md5-signature: Create the MD5 signature of the incoming file and send to the FipRemote running in Server mode The file is logged if in error at the remote end. default: none output-md5: (FipSeq) Output this string with the MD5 signature as FipHdr RM AFTER the data has been sent Use this for straight braodcast feeds eg - top and tail with an ETX and EOT output-md5:\003\RM\004 default: none encrypt-script: External script to encrypt or compress the data. There is one and only one parameter to this script which is the name of the file to encrypt and in which the new contents will be stuffed. encrypt-script:/fip/local/remote_pgp log-message: (FipSeq) logging information for fipremote default: nonw The text-format file has the Syntax of : ; comment before: Text to add at the top of the data file. after: Text to add at the bottom of the data file. filebefore: File to add at the top of the data file. fileafter: File to add at the bottom of the data file. zaptags: If the source data is in XML (or HTML or any other tagged format), zap all the tags. eg tag:P \r\n\t tag: Replace a start tag with some string (eg <p>) endtag: Replace an END tag with some string (eg </p>) alonetag: Replace an Alone tag with some string (eg <br/>) Syntax: endtag:(tag) (replacement data in FipSeq) There can be a 'tag' for each that needs replacing. Do not use these with wrap - pretty meaningless if you do anyway ! It is case insensitive. tag:P <text> endtag:P </text> alonetag:br <padline/>\n force-binary: If the text starts with this string, assume it is binary and make NO changes to end-of-line of characher maps force-binary:<p> chr:(chr) (replacement data in FipSeq) Syntax: chr:(chr) (replacement data in FipSeq) There can be a 'chr' for each character that needs replacing. chr:\006 +Bell+ chr:\037 <TD><!-- \037 --> hdrchr:(chr) (replacement data in FipSeq) As for 'chr' but for FipHdr fields only. Use this to map XML entities for example hdrchr:& & hdrchr:< < hdrchr:> > hdrchr:" ' txchr:(chr) (replacement data in FipSeq) As for 'chr' but for 'before', 'after'. 'filebefore','fileafter' Use this for any '#' in the FipHdr fields for example paragraph-marker: End of line sequence in FipSeq or eoln: End of line sequence in FipSeq default is to leave the end-of-line as in the data ignore-eoln: Test for eoln - if therem, do NOT add the eoln eg eoln:</p>\n<p> ignore-eoln:</p> In this case if there is a single preceeding <p>, no changes to the eoln will be made If not, all end of lines (usually NL or CR NL) are replaced with '</p> NL <p>' This works on a file by file basis so files with/without <p> can be interlaced in the output default - none wordwrap: Wrap the text at a number of characters per line. This will wrap on the nearest space to the end of the line. default is to not change the data at all. eg Wrap the text at 72 characters. wordwrap:72 ww-eoln: For word-wrapped lines, End-of-line sequence in FipSeq. default is CR NL ie '\r\n' eg ww-eoln:\f add-wordwrap-for-service: ONLY wordwrap for this service. There can be several lines each specifying a single service. remove-wordwrap-for-service: Remove any existing wordwrap for files for a service. There can be several 'remove-wordwrap-for-service' lines, one per service. Eg : if files for service 'biz' are wrapped at 70 chrs and you want to have NO wordrap : remove-wordwrap-for-service:biz This is done BEFORE applying any wordwrap , so if you want to put a wordwrap on 40 chrs: remove-wordwrap-for-service:biz wordrap:40 convert-to-utf8: Convert the incoming data to the UTF8 variant of Unicode. max-data-size-for-service:(servce name) (no of chrs) To create an abstract wire of the first n chrs for a particular service Headlines only, set this to 0 (zero) eg max-data-size-for-service:biz 0 default is -1 for ALL data/text repeat-service-name: Service code for Repeated items. default is 'RPT' To use the original, use a blank 'repeat-service-name:' service-name-alias: Original Service code and replacement eg service-name-alias:XX2 SPT There can be up to 1000 aliases The aliased name does not have to be unique Input Parameters : Mandatory Either -g : name of a Groups file in tables/remote/groups default: none or -n : name of a single Client file in remote/clients default: none Optional -N : replacename name for the client default: -n This allows a 2 clients to use the same parameter file but have different idents. Resending files from Fip will go to both clients Care MUST be taken to have UNIQUE client names on a single server -s : hostname as override to parameter file default: name for client-started feeds eg -s + to listen on all ports -P : port as override of 'port:' in parameter file default: port for client-started feeds -Q : do NOT send Seqnos (ie remote is fipremote < 2e12) default: yes -l : do NOT log each item default: do -LOCAL : only process this client (or Group) locally default: network -T : do NOT log the trace of each transaction default: do Normally a one line log of each file sent is stored in log/remote_trace with a name of 'date_client'. Use this flag to stop this trace. -V : version level default: 3 (use for old remdata) -v : Print the version number and exit FipHdr fields POSSIBLY used in the incoming data and used by ipremdata/svr : R0- clientname for traffic for a particular client only This is used for resends and system messages R0:MegaCorp R6- Mime Type of file R6:TEXT - used for flagging to use the TEXT format Any other value/string is user-defined. RX- Full path and filename on this device FipHdr fields added by ipremdata are : R1- date CCYYMMDD R1:20000630 R2- hour R2:09 R3- Overall System item number R3:06132 R4- Uno R4:200006300906132 R5- SystemId - not used by RemSvr FipHdr fields added by ipremsvr are : R7- Service item number R8- Client item number R9- Service name, usually a service code R9:COM -------------------------------------------------------------- Version Control ;2z16 15dec03 cleanups ;d-f 27feb04 cleanup Groups ;g-h 29jul04 added 'track-high-priority:' ;i-k 15aug04 added check-md5-signature and encrypt-script ;l 07jan05 bugette over chg of day ;m-n 26jan06 added allow-remote-repeats:yes/no (n dump md5) ;o 24sep06 added tag, endtag, zaptags and alone tags, ;p-q 04nov06 added -P port as override to parameter file ;r 06nov06 added send-no-old-data: and -N ;s-u 22nov06 added ignore-eoln ;v-w5 11dec06 added -k remkey ;x-y6 8may07 added service-name-alias and StuffFile bugette (y4 no-pwd for fipremote) ;y7-10 for txchr: (10 for space in service-..alias) ;z1-9 17aug08 added take-size/fiphdr/top/tail (2 bugfix for output buffer) (5 new services buglette) ;10-11 27apr10 added local-fiphdr and local-extra-fiphdr ;12 added SEQNO and tracking for super bad links ;13-15 added log-message and -Q for no SEQNO ;001z 22feb01 no-password/no-logon-required added for fipagent too ;a 20mar01 added proxies ;b 27mar01 added wordwrap properly ;c/d/e 20apr01 bugette in non-encrypted feeds ;f 27jun01 WINNT ;g 27nov01 closeup if bad logon. ;h 10dec01 added do-not-send-duplicates. ;i/j 08jan02 added hdrchr ;k/l 09mar02 added remove-wordwrap-for-service ;m 11apr02 added convert-to-utf8/base64 ;n 26apr02 added add-wordwrap-for-service ;o 02may02 ignore the Missing msg if Pos=0 ;p/q 09may02 added send-all-old-data: ;r/s 12jun02 finally got the svce bug !! ;t 28sep02 added max-data-size-for-service: ;u 27mar03 added ignoreforservice: ;v-y 23may03 added createZQ ;z 05dec03 added repeat-service-name (copyright) 2018 and previous years FingerPost Ltd.