ipdelque

ipdelque

This program deletes files from a queue. This can be a single shot (default) or
you can specify a delay which the program will wait before restarting ad
infinitum.

Normally a 30 second wait is applied before the program starts in case the
wrong queue etc has been selected.

Optionally only files that are older (ie last time modified) than a certain
number of days or minutes can be deleted. This can be changed to checking
creation time by input switch -c.

For queues that accept networked files, a file creep time can be set.

IPDELQUE is often used in the nightly maintenance script ZAPFIPLOG to trim
holding queues and sundry save areas.

Strings in data can be searched and files deleted or NOT accordingly using the
-g/-G switch (plus -s for case sensitivity if necessary).
    eg: ipdelque -q yuko -g "txtclass=edito*"
will delete files containing txtclass=edito* in /fip/spool/yuko.
    (Note the * is NOT a wild card here so we have quoted the string)

Filenames can be checked too with -k or -K.
    eg: ipdelque -q zibbo -K ttt
will delete all files in zibbo EXCEPT those with '.ttt' somewhere in the name

Input parameters are :
    Mandatory :
    -q : input queue name               default: none
        This can be a full pathname starting '/'
        or a relative to current path if starting '.' as in './xyz'
        otherwise it is assumed to be a queue under /fip/spool.
        it can also be in FipSeq :
        eg -q \$e\$y\$i\$d_xmp_error (if in the SYSTEM file)
        or -q \\$e\\$y\\$i\\$d_xmp_error (if run from a shell)
    Optional :
    -a : age subqueues in this queue        default: no
        syntax is -a 8 for eight days worth
        - move any files in the queue to a new subqueue called
(queue)/(queue).CCYYMMDD
        - if the subparameter > 0, zap any subqueues older than that number of days
    -A : age subqueues and files in this queue  default: no
        Syntax is -A 3 for deleting all queues over 3 days old
        This variation is for queues AND files that begin   YYMMDD or YYYYMMDD
            eg  970925-reuters
                20001231-31223-SPORTS
                20180925_soccer_fixtures.xml
        All files or queues older than the date the program is run (minus the
subparameter) are deleted.
        Any files or queues whose name does not start with a date are ignored.
    -F : ignore date in Folder name
        when used with -A, the date in the folder name is ignored and the last MOD
datetime is used
    -l : log all deleted files
        default: normally only one QUEUE DELETED msg appears
    -L : do NOT log if no files were deleted    default: (as above)
    -m : delete files if more than x days only  default: all files
        This deletes all files up to midnight before the day.
    -M : delete files if more than x mins only  default: all files
    -c : check time of creation         default: last modify
    -C : check file name                default: no checking beforehand
        only zap files if this checkfile exists
        This is to check that UNC, NAS and other remote drives are actually
mounted/accessible beore deleting
    -d : WINNT only drive letter for queue  default: drive Fip started on
    -E : add ExtraFipHdr information        default: none
    -f : file wait                  default: 0
        (ie wait x secs for file to finish before processing)
    -g : search string in data - if present,    default: all files
        delete THIS file but NOT those without it.
    -G : search string in data - if present,    default: all files
        do NOT delete THIS file but delete those without it.
    -h : for search string in data, ignore the FipHdr. default: all data
    -H : run on this host only          default: all
        use this were the same zipfiplog is on several servers
    -i : wait in secs before starting to delete default: wait 30 secs
        -i 10   wait 10 seconds beforehand
        -i 0    do NOT wait before deleting
        If the '-t' switch (spooling) is also on, this wait is applied to the FIRST
scan only, after which it is ignored.
    -k : filename mask - if present,        default: all files
        delete THIS file but NOT those without it.
    -K : filename mask - if present,        default: all files
        do NOT delete THIS file but delete those without it.
    -OLD : for -a make the aged queue YYMMDD not CCYYMMDD
    -Q : DO NOT create the folder if it does not exist  default: create
    -r : delete any Mac Resource forks in ./.rsrc   default: no
    -R : Run ONLY if the server is Primary      default: always process
        Only the primary host actually does the delete  default: delete
        Eg if there is an entry in the tables/sys/DEST_REDUN file :
            nassvr  fip-l01v-p  fip-l02v-p
        and you run ipdelque -R nassvr -q /uncdrive/somefolder/subq/
        then fip-l02v-p will only run the delete if fip-l01v-p is down
        This can be used run to zap on shared drives in conjunction with -C (chkfile)
from only one of the Fips
        (Use -S if you want to run on the Secondary(ies) only)
    -s : search string is case SENSitive        default: case insens.
    -S : Run ONLY if the server is Secondary (ie opposite of -R)    default: always
process
    -t : sleep between scans of the directory.  default: run once only
        The minimum scan time is 10 seconds -t 10
    -T : do NOT delete any files starting with a dot    default: delete
    -X : TEST only; just display filename that should be zapped (ver 08k+ only)
default: delete files/folders
    -V : log the name of each item deleted      default: log totals only
    -Z : zap the top folder (-q) too        default do NOT
        This option is disabled if you also specify any of : mask (-k/-K), aged
queues (-a/-A), search (-g/-G)
    -v : version no and exit

From version 8k onwards there is also a -X input switch which displays the
files that would be deleted - BUT does not delete ! All previous version just
zap !

Version Control
;8p20a  25sep97 added -A and -d
    ;a 8oct97 added rmdir for NT
    ;b 1may98 -a/-A now zaps queues on too
    ;c/d 20dec99 BUG -a/-A and 00->100
    ;e 12oct00 -A now zaps files (and folders) called '20001006.etc'
    ;f 20nov00 allow spaces in WINNT folder names for -A
    ;g 10jul02 zap res fork files too
    ;h 03mar04 added -L
    ;i 29nov04 bugette with spooled version not changing -m:BAD MOD PLS DONT USE
    ;j-m 09feb05 reworked those times/dates again and added -X and better Winnt
log drive
    ;n 02may06 -t 0 is now invalid - minimum time is 10 secs
    ;o 17jun06 added -Q
    ;p5 18jul06 bugette minutes -M 3 not working correctly ;2 cosmetic ;3-4
win-use del /S /F /Q
        ;5 *nix- top and tail with apos
        ;6 allow Fipseq for -q queue and -E (extraFipHdr)
            eg ipdelque -q /fip/data/raw.data/\$e\$y\$i\$d_ap -i 0 -M 2
        ;7-8 27jan13 added -R DestRedun Pseudo host when checking from 2 or more
systems and -C chkfile
        ;9-11 Linux NFS fudge - false ELOOP on readdir -
https://bugs.centos.org/view.php?id=6161
        ;12 11apr16 added -S run on secondary
        ;13-14 16mar17 added -X to zap TOP folder too !
        ;15-16 added -H run on this host only
        ;17 allow -A 0
        ;18 added -F where the data-in-file/folder name is ignored
        ;19 allow YYYYMM dates in -A and better logging
        ;20a added -V log items as well as totals

(copyright) 2024 and previous years FingerPost Ltd.