AP Atom install

Contents

%TOC% (needed)

Overview

    • Install and configure AP Atom on FingerPost

Prerequisites

    • /fip/bin/webwire, version 05y7
    • /fip/bin/iptimer, version 02n20
    • /fip/bin/ipsgml, version 19f33
    • /fip/bin/ipxchg, version 48f18
    • /fip/bin/sffb64, version 00h
    • NB: all versions can also be more recent
    • zip file atom_lala.<date>.zip
    • apatom specific cgi’s and templates only or (optionally) w4_lala..zip
    • AP Atom logon and password
    • Are http connections on port 80 allowed from FIP server outside through the firewall?
    • Zip Files

        • Unzip tables and w4 zip files provided by FingerPost

      Logons

        • Add to either to template in /fip/web/logon/templates or logon name in /fip/web/logon/names for the admin user:
        • options:Manage AP Atom:/fip-cgi/w4_apatom.pl:directory_window
        • Feed Setup

          Add publication and request entitlements

          • Logon to w4 and click on
          • Manage AP Atom
          • Click on
          • Add an new Publication
          • APatom_2
            • Name the publication (ie: your newspaper name)
            • fill in logon and password provided by AP. Other fields are optional.
            • If you plan on running a 2nd FIP system for redundancy then you will need to name Fip Balance Group and Fip Primary Host
            • Fip Balance Group is defined in tables/sys/BALANCE. Any hosts which need webwire skip-lists to keep track of files already accessed from apatom should be in this group. For example:
            • =group:fipadmin host:(hostname1) ignore-localhost: nofiphdr:=
            • =group:fipadmin host:(hostname2) ignore-localhost: nofiphdr:=
            • Fip Primary Host is defined in tables/sys/DEST_REDUN. This is an alias which will try to use the primary server first for bringing in apatom files or, failing that, the secondary server. For example:
            • =atomsvr (hostname1) (hostname2)=
            • Click on
            • Add
            • atom-nav1-logon

              * You should see in green:
              “Adding a Publication…”

              * Next click on “3.Admin” in navigation links on upper right-hand part of screen.

              atom-nav2-logon

              * You should see new Pub listed and a remark in red:
              “Pls download Product and Package entitlements from AP before continuing”

              * Click on link underneath *Feed/Entitlements*

              atom-nav3-ent

              * And then click on the *Download current entitlements from AP* link.
              atom-nav4-ent
              * After a few momennts you should see a message in green: “Request for entitlements OK – there are xxxx products + packages”

              atom-nav5-ent

              * Click on “3.Admin” in navigation links. Under “Rebuild Parameters” click on *-rebuild-*

              atom-nav6-reb

              * You should see a message in green: “…Rebuilding TIMER and WEBWIRE parameter files”

              atom-nav7-reb

              NB:”Rebuild” reads through options for this Pub in files added to /fip/tables/apatom from the browser front-end and then builds new parameter files in /fip/tables/webwire and /fip/tables/setup.

              * Click on “3.Admin” in navigation links to move up a level and next add a wire group…

              Add wire groups

              * A wire group should correspond to a feed on the AP satellite
              * For example, if the satellite provides three feeds on three different ports:

              apdata wires wire -s ws3 -p 2102 -U -Y -a -t -n apdata
              apfea wires wire -s ws3 -p 2103 -U -Y -a -t -n apfea
              apspt wires wire -s ws3 -p 2104 -U -Y -a -t -n apspt

              * …then you will almost certainly want to create three wire groups for AP Atom

              * Click on *-view-*, under “Wire Groups” then *Add a new Wire Group*

              atom-nav8-grp

              atom-nav9-grp

              * *Wire Group:* to avoid (?) confusion name the Wire Group the same as the current AP satellite feed
              * *Grab Interval:* is in minutes. Anything between 1 and 10 minutes OK. 2 should be fine.
              * *Fip Source (SU):* If testing along with satellite feed on same server, then *Fip Source (SU)* should have a different name than the satellite feed. (Hint: Satellite feed name with wire or vwire program is devined by the -n switch).
              * *Data type:*
              * for a _text_ feed choose __text-nitf__
              * or if an imitation of old satellite feed is needed, choose __text-anpa__
              * for _photos_ choose __pix-jpg__
              * for _graphics_ there is a choice between __graphics-pdf__ and __graphics-illustrator__ or, if you want both in the same feed, __graphics-pdf&illustrator__
              * *MaxTimeout:* default is 300. You may need to exand this to 600 or 1200 for big photo packages.
              * *MaxItems:* default is 25. You should bump this up for text feeds but it may be best to leave it at the default for photos.
              * *ProductsOrSavedSearches:* if it’s already an AP product or package then leave it as “Products”. “Saved Searches” are for special searches set up with your logon/password on the AP site.
              * click *Add* and then *Return to View <Pub>*

              atom-nav10-grp

              * Click on “3.Admin” in navigation links to move up a level and next assign code to a wire group…

              Assign codes to wire groups

              * click on link under “Feed/Entitlements”

              atom-nav8-grp

              * Here you want to add all the codes to match the old satellite feed.
              NB: there is no exact match or even documentation for trying to
              duplicate a satellite feed on AP Atom. You need to view your entitlements and
              select the most likely candidates based on the description.

              * your only wire group under the “wire group” section should be “unassigned codes.” Click on *unassigned codes*

              * there should now be a drop down list with any wire groups you have added. Go through the list and click the check boxes to assign codes to wire groups.

              atom-nav23-ass
              * when finished hit the *Change” button at the bottom of the page and a message in green should appear, confirming your changes.
              atom-nav25-ass.PNG

              * Click on “3.Admin” in navigation links to move up a level
              * And then click on *-Rebuild-*

              Server setup

              * assuming that there is still an AP satellite feed, add a new routing table by copying from the old feed
              * the new routing table must be named to match the Fip Source (SU:) given to the wire group
              * for example, in tables/route, cp APSPT to ATOMSPT
              * when first testing it may be best to send all data in the new routing table to “logdel” to check before sending to the editorial system.

              * in tables/sys/USERS add:
              * ap_atom_2br= DP:localhost DQ:2brouted DC:\SC
              * ap_atom_xml= DP:localhost DQ:2atomxml DC:\SC
              * w4atom= DP:localhost DQ:2w4 DC:\SC WL: WN: WT: WV: WD: WJ: WH: WA: WY:
              * w4atom1= DP:fipXXXX1 DQ:2go DC:\SC DU:w4atom
              * w4atom2= DP:fipXXXX2 DQ:2go DC:\SC DU:w4atom
              * ap_atom_ignore DP:fip4 DQ:pixignore DC:\SC
              * ap_atom_pix DP:fip4 DQ:2edsys EQ:pix4pix DC:\SC DF:APATOM2PIX
              * ap_atom_pix DP:finger1 DQ:2edsys EQ:2ftp DC:\SC DF:APATOM2PIX
              * ; oddies
              * ap_atom_sptagt DP:finger1 DQ:spttest DC:\SC

              * in tables/sys/WAIT add:
              * : 2atomxml : */2atomxml

              * in tables/sys/SCAN or ../SCANSVCE add:
              * queue:2atomxml

              * in tables/sys/MAINT add:
              * /data/APATOM 0

              * in tables/sys/DEST_REDUN add:
              * atomsvr (hostname1) (hostname2)

              * in tables/sys/BALANCE add:
              * group:fipadmin host:(hostname1) ignore-localhost: nofiphdr:
              * group:fipadmin host:(hostname2) ignore-localhost: nofiphdr:

              * in local/zapfiplog add:
              * /fip/bin/ipdelque -q /fip/spool/atom.done -A 12 -i 0
              * /fip/bin/ipdelque -q /fip/data/apatom/blob -A 12 -i 0

              * in tables/setup/TIMER.AP.ATOM.<Pub> add alias defined in DEST_REDUN:
              * check-primary-server:atomsvr

              * and in the same file add balance group defined in tables/sys/BALANCE:
              * skip-balance-group:fipadmin

              Start feed

              * To finally start the feed add a runline to tables/sys/SYSTEM and/or x/SYSTEM:
              * apatom local iptimer -n timer.ap.atom.<Pub> -l -E 60
              * (-n switch is the name of the file created in tables/setup)
              * if any of the wire groups have a Data Type of “text-anpa”, then add an ipdskwir runline:
              * dskwir local ipdskwir -i atomdskwir -a -9 -n ATOMANPA

              Check feed/entitlements

              * In case no data is coming in on a certain feed – or any feeds – you can check both the connection with AP and if any data is available for a product.

              * Navigate to *2.Manage…* and click *Daily operations*
              atom-nav16-daily

              * Click on *-check-* under “Check Feed/Entitlements”

              atom-nav17-daily

              * Click on the prodcuct code number under the “code” column
              atom-nav18-daily
              * If access to AP is OK and if data is available for the product code, you should get a list of files, sorted by AP sequence number.

              atom-nav19-daily

              * If access is OK, but data is not available for the product code there will be a “no data…” message.

              atom-nav19a-daily

              Bandwidth

              * The amount of data moving in and out of the FIP server via the AP Atom Module is logged in comma-delimeted text files in YYYYMMDD sub-directories under /fip/log/bandwidth_stats. Each text file covers 1 hour of activity. These may be downloaded as csv files or as a graphic.

              * Navigate to *2.Manage…* and click *Daily operations*

              atom-nav16-daily
              * Then click on *-view-* under “Bandwidth Stats”

              atom-nav17-daily

              * The date defaults to today. To view other days click on the dates underneath “Other dates…” Click on the XX.ATOM_webwire.fip file to download as csv
              * The comma delimeted file displays:
              * field 1 – start time
              * field 2 – wire group
              * field 3 – bytes tx
              * field 4 – bytes rx
              * field 5 – diff in millisecods

              atom-nav20a-daily

              * Or scroll down and view html graph of average total bytes for each hour throughout the day
              atom-nav20-daily

              Resends

              * Every once in a while it may be necessary to request a resend of AP Atom data from a certain time and date. You will need to know the *Sequence Number* to send from as well as the *Sequence Number Time*. In case you don’t know the sequence number, just put in roughly the time you want and Atom should start from the nearest sequence number to that time.

              * Navigate to *2.Manage…* and click *Daily operations*
              atom-nav16-daily

              * Under *Access Details for publication…* click on the wire group to modify.

              atom-nav21-seq

              * Note that the time and date needs to be in AP’s own brand of Zulu time format. Click on *change* to write the changes or *ignore* to back out.

              atom-nav22-seq

              Copy Flow

              * Copy flow for a wire group through FIP will depend on the data type. Here are 3 tables which will guide the files as far as the routing stage. The iproute parameter file will be the value of the SU: hdr field defined when setting up the wire group.

              Text in NITF

              *program* *spool/* *table/* *example runline in SYSTEM*
              1.iptimer setup/TIMER.AP.ATOM.<pub> =apatom local iptimer -n timer.ap.atom.daily_bump -l -E 60=
              2.webwire webwire/AP.ATOM.<pub>
              3.ipsgml 2xml sgml/AP.ATOM.TOP =xml local ipsgml -i 2xml -z ap.atom.top=
              4.iproute 2brouted route/<SU:>

              Text in raw ANPA (new – webwire version 5y7 or later)

              %TABLE{ sort=”off” headerbg=”navy” tablerules=”rows”}%

              *program* *spool/* *table/* *example runline in SYSTEM*
              1.iptimer setup/TIMER.AP.ATOM.<pub> =apatom local iptimer -n timer.ap.atom.daily_bump -l -E 60=
              2.webwire webwire/AP.ATOM.ANPA.<pub>
              3.ipxchg xchg xchg/APATOMANPA, APATOMANPA2
              4.iproute 2brouted route/<SU:>

              Text in raw ANPA (old)

              *program* *spool/* *table/* *example runline in SYSTEM*
              1.iptimer setup/TIMER.AP.ATOM.<pub> =apatom local iptimer -n timer.ap.atom.daily_bump -l -E 60=
              2.webwire webwire/AP.ATOM.ANPA.<pub>
              3.ipdskwir atomdskwir =dskwir local ipdskwir -i atomdskwir -a -9 -n ATOMANPA=
              4.iproute 2brouted route/<SU:>

              Pictures and Graphics

              *program* *spool/* *table/* *example runline in SYSTEM*
              1.iptimer setup/TIMER.AP.ATOM.<pub> =apatom local iptimer -n timer.ap.atom.daily_bump -l -E 60=
              2.webwire webwire/AP.ATOM.BLOB.<pub>
              3.iproute 2brouted route/<SU:>

              Troubleshooting

              Connecting to AP

              * No http access out of the FIP server
              * When *downloading current entitlements from AP* and it takes a long time to get a response message in the browser, until this finally appears:

              APatom_ts3

              …and this message appears in the FIP log:

              APatom_ts5

              * Incorrect logon or password to AP server:
              * When *downloading current entitlements from AP* and this error message appears:

              APatom_ts2

              Data

              * Recurring duplicate files
              * If the same files get processed every 5 minutes over and over again, it could be that too many files (default 100) are being requested in too little time (default 300 seconds). This message will appear in the log:

              APatom_ts6