Review of Trashman
by
Shawn M. Gordon
President SMGA
Introduction
How many times has this happened to you, you decide to write some code based on another program, you text the program in, make a bunch of changes, and then instead of saving it to a new file you overwrite the original. Have you ever done that without a backup available of the original file? I know I have, and it is a miserable experience. How about doing a wildcard purge of a bunch of files and you suddenly realize you are in the production account instead of the development account? Without an up to the minute backup you can end up with a real problem. This, and other situations are where Trashman can help you.
You know all those great little PC utilities like disk defragging and Undelete that you wish you had on your HP? Well, some of your wishes are finally coming true. Software Research Northwest (also referred to as SRN, but you won’t find them in the phone book that way) has come out with a true undelete utility for the HP 3000 MPE/iX 4.0 and later, systems (you may need an OS patch on 4.0). This product takes advantage of the HP supplied AIF’s to trap system call’s to FCLOSE so it can keep a copy of the file instead of letting the FCLOSE purge it.
Trashman was originally part of the Diamond Nuggets Collection of utilities from Software Research Northwest, but it has since been beefed up and moved out on it’s own. For those of you unfamiliar with the Nuggets utilities I suggest calling and finding out, there are some really great programs in there, and they are all very reasonably priced. I tested the software on an HP 3000 918LX running version ??? of MPE/iX
Features
The real difference here between Trashman and a PC based undelete utility is that the PC utility tries to recover the file from disk since a deleted file is just marked as free, and will stay there until something overwrites it’s location on disk. This technique isn’t that reliable if you really want full recovery capability so Trashman takes the approach of essentially renaming the file to it’s own account instead of purging it. This also gives you the ability to keep multiple versions of a file since you can control the number of generations of a file that you will keep around.
Let me first go over the programs and syntax that you will be using and what they do. First off, Trashman builds an account called LANDFILL, no other account name can be used for the Trashman software, and no files purged in this account will be ‘safe-purged’. Safe-Purged files are files that are available for recovery. For a file to be safe-purged it must be defined in the Trashman configuration file (covered in the next section). Trashman does not safe-purge privilaged files (e.g., database files), temporary files, KSAM files, message files, circular files or spool files. It will also not safe-purge files that are purged via purgegroup, purgeacct, restore or store operations.
Rubbish is the process that intercepts purge requests and determines whether or not a file is to be safe-purged. This process can only be activated or deactivated by the Landfill Manager. Once activated, all purge and fclose commands will be processed through Trashman to see if they are eligible to be safe-purged.
The Landfill Manager is the person responsible for configuring Trashman. To become the Landfill Manager one must simply logon as MGR.LANDFILL and run the Trashman program. Only one person can run this program at a time. The duties include setting up configurations, start the Rubbish process, running the Compost job to clean up the Landfill account. The Landfill Manager can also Kill any safe-purged file on a selective basis.
Compost is a job that runs while Trashman is active. Periodically it checks the Landfill for safe-purged files that have exceeded their savetime parameter. Expired safe-purged files are permanently removed from the MPE/iX file system.
Recycle is the process that you and the users will use to recover a safe-purged file. It has both a command line and window interface (This is based on another SRN product called Wingspan and it looks very good). Only the original owner of the file or the Landfill Manager can Recycle a safe-purged file.
To Kill a file is to permanently remove it from the MPE/iX file system (really purge it). The Landfill Manager can Kill any safe-purged file, other users can only Kill files that they originally safe-purged.
So let’s get into what it is you can do with this program, and how you can go about doing it. After installing the software you will need to set up and compile a configuration files. First I will show a sample configuration file, and then I will explain the different portions and what your options are for each portion.
[account rules]
@@ : maxfile=-1;maxsize=-1;
[include rules]
/@/ :versions=3;filecode=(0..-1);filesize=(0..-1);warning=yes
[exclude rules]
/HPSPOOL/ : filecode=(0..-1);filesize=(0..-1);
/HPOFFICE/ : filecode=(0..-1);filesize=(0..-1);
[end]
Okay, the first section in our config files was ‘[account rules]’, this is where we can define the maximum number of sectors and the maximum number of files that a given account can use within the Landfill. You can assign these on an Account by Account basis, or specify @ as I did for ‘All Accounts’. By specifying a value of -1 in ‘maxfile’ and ‘maxsize’ we are setting these values to ‘unlimited’ or until the disk spaces runs out, whichever comes first. You can specify an exact number as well if you wish.
Before I get into the next two sections I want to point out that all of the Include and Exclude rules use POSIX naming conventions for the file name. this means that the names are case sensitive and you will use the / key instead of the familiar period ‘.’. Here are the the Posix naming conventions lay out for our familiar MPE FILE.GROUP.ACCOUNT
Account-wide Specifications
/
Group-wide Specifications
/
File Specifications
/
So using a real world example;
All the files in the SYS account /SYS/
All the files in the PUB group, SYS account /SYS/PUB/
A Single file called SL, PUB group /SYS/PUB/SL
Now let’s cover the data in the Include Rules section. Include rules are used to qualify safe-purge operations on specified accounts, groups, or files. A file must be represented by an Include rule in order to be safe-purged. If a file is represented by more than one Include rule, the rule that is more global is the one that applies. For example, an /@/@ takes precedence over /SYS/PUB.
The format of an Include rule is /ACCOUNT/GROUP followed by a rule. You can have multiple rules on a single line as long as they are seperated by a semi-colon, but for readability I would recommend putting one per line.
There can be only one Include rule section in the configuration file. It must be preceded by <[>Include Rule], case is not important. Now here is a list of the options available in the Include rule.
COMMAND = YES | NO
This option provides for safe-purge operations from programmatic calls. This would include calling the COMMAND intrinsic with a PURGE command in it, or calling FCLOSE with a purge disposition.
CREATOR – This is the MPE USER.ACCOUNT, either parameter may be wildcarded:
CREATOR=MANAGER.SYS
CREATOR=@.@
FILECODE – This is the file code of the file and it may be specified in one of two ways, either as a range or a list.
FILECODE = (m..n), where ‘m’ is less than or equal to ‘n’, and ‘n’ could equal -1 to indicate any.
For example:
For any filecode…..FILECODE=(0..-1) (default)
For filecodes between 1,000 and 2,000…………FILECODE=(1000..2000)
For a list of filecodes……FILECODE=1029,1030,1032,1033
This option is great as a sort of a really minimal version control on source code, especially COBOL where you have the file code EDTCT to look for. There is an appendix in the back of the manual that gives you a list of all file codes, the mnemonic associated with it and a description of what it is.
FILESIZE – The range of file sizes to safe-purge. The format of this is identical to FILECODE, but instead of -1 meaning any code it denotes any size.
INTERACTIVE = YES | NO
This option indicates if the safe-purge operation will take place for interactive users (not jobs).
SAVETIME = <[>days,] hours
Specifies the amount of time in days and hours that a safe-purged file will stay in the Landfill. When the Compost program runs it will check to see what files have exceeded this time limit and will then permenantly remove the expire
d safe-purged file. The default is one day.
VERSIONS – The number of copies of the same file that can exist in the Landfill. The default is one version, if a file has reached the max number of versions and another copy of the file is safe-purged the oldest version will be lost. If WARNING is set to YES then you should receive a message that this is happening.
WARNING = YES | NO | ERROR
This is a warning message that is generated when a file is safe-purge.
When set to YES an interactive process will be notified anytime a file cannot be safe-purged because it was excluded in the configuration file. For batch and programmatic access the message is sent to the Trashman log file.
When set to NO there is no notification of any kind, this is the default.
When set to ERROR, only errors will be reported.
So using this information and comparing it to our sample configuration file we can determine the following;
All files are included, up to three versions of the same file will be kept, all filecodes are applicable, a file of any size will be safe-purged, and warnings will be given.
The last section in the configuration file is the ‘Exclude Rules’. It works exactly like the Include rules section, but you have fewer options, they are;
COMMAND
CREATOR
FILECODE
FILESIZE
INTERACTIVE
The parameters are specified exactly as they are for the Include rules. So for our example configuration file we have specified the following. We will exclude all files in the HPSPOOL and HPOFFICE account regardless of the filecode or filesize.
Finally the configuration file must terminate with the statement [end].
Ok, the hard part is over and all that’s left to do is to recycle safe-purged files.
The fast, quick way to recycle a file is to issue the RECYCLE command from the command line specifying a file or fileset. Any files matching the criteria will then be ‘recycled’ back to disk. I wanted to see an option on the command line that would also just list what files were available for recycling based on the specified fileset.
The more interesting, and flexible option for recycling files is to run the RECYCLE program, this can be done by just typing RECYCLE and not specifying a fileset, see figure 1 for a sample of the start up screen for RECYCLE.
This first screen let’s you specify the fileset that you want to deal with, it defaults to everything in your logon account. Once you have the screen filled in, you will then press function key 6 to ‘Accept’ it.
Figure 2 shows the screen that comes up after your selection process is complete. This has a summary of what has qualified, and some of the include rules that apply to them. At this point you can ‘tag’ or ‘untag’ files for inclusion for either recycling or killing. You use the cursor control keys to maneuver through this screen, and function key 8 to exit. Figure 3 shows the dialog once you start recycling tagged files (killing them works the same way). You can selectivly do them one at a time, or use function key 3 to have all selected files be recycled. Figure 3 is based on different sample data than is figure 2.
Figure 4 gives an example of what happens when you try to recycle a file that has a name conflict. This is based on a fileset that has VERSIONS specified, and brings up an interesting point about using the VERSIONS option in the configuration file. This can be used as a basic version control system for source code, however it is not going to work with all editors. It does work with EDIT/3000, but it doesn’t work with the CSL editor QUAD and it doesn’t work with Robelle’s Qedit. This is because of the way that they open the files, it makes it impossible to get a correct picture when the file is FCLOSE’d.
Having VERSIONS on can also cause some confusion when recycling files, this is why I would like to see some options when recycling to either purge or rename the existing file instead of just preventing the recycle to take place.
As you can see it is very easy to work with the software once it is installed and set up. You really can’t get much easier than this, and remember, these files are 100% recoverable as long as they have been safe-purged. There is no chance of them magically disappearing. They will only go away when the parameters you have set up have been satisfied.
Usability (also installation)
Although the configuration file isn’t hard to set up, and the rules are pretty straight forward, it is by far the most difficult part of Trashman. What I’m saying is that it isn’t difficult at all. There are some things that you will need to change in your computer operations procedures to make sure you take full advantage of Trashman by making sure it is always active, but that should be pretty simple.
Installation consists of enabling procedure exits by running PEUTIL.PUB.SYS and issuing a single command, restore a job, stream the job and then you are done. You will probably want to set up their UDC system wide, or put their command file group in your HPPATH variable, but that is pretty much it.
Reliability
Well I tried to fake out Trashman by setting a small size amount for a rule on a group and account and then trying to purge just one file that exceeded that amount. I got a warning that the files was not being safe-purged, and then the file was purged. I got the warning because I had WARNING enabled on the Include Rule. I would have liked to have seen one more option for WARNING which would be ASK so you could make the purge optional at purge time.
Another test I tried was with setting the number of files to a value and then tried purging more files than I had allowed. Again I got the warning when I exceeded the limit, and the file was then purged. I ran the same tests for the FILECODE and CREATOR options of the Include Rules and it appeared that I always got correct results.
Performance
Where you would actually expect to see a performance hit is in an increased time in purging files, since the file isn’t being purged, but being moved to another account. Strangely enough, not only don’t you see a performance hit, it actually seems to be more forgiving than the PURGE command. On large files it can talk a LONG time under MPE/iX to actually purge it, Trashman in most instances does a RENAME of the file behind your back into the Landfill account. This means that the purge is almost always going to be instantaneous and incur almost no disk overhead.
The Compost background job, even with a timeout of 5 seconds almost never impacts the CPU enough to even tell that it is running. So aside from the possibility of running low on disk space because you have your retention set too long or you have a LOT of safe-purged files, you should never see a performance hit, and most of the time you will see a slight boost in performance.
Supportability (including Doc)
The documentation is well laid and out clearly written. They explain each section of the product in order, from installation to use, and explain all the files and groups that are installed and what they contain. There is even an appendix at the end that describes all the different file codes and what they are. I haven’t seen a list of those in one place in a long long time. The manual is only 79 pages so that is pretty nice and concise, it also includes an Index at the end.
The beginning of the manual talks about some of the other Nugget utilities, what they do and who they would apply to. It made me want to look at those too. They also spend some time explaining the Graphical User Interface (GUI) that the Recycle process uses via the Wingspan product, how to use it, and potential terminal emulator conflicts and how to correct them.
The support is also quit good, the folks at Software Research Northwest really know their products, which is kind of amazing given the quantity of products that they have.
Summary
I am going to make this point again since this is the fourth low cost product in a row that I have reviewed “A low price does not mean low quality”. There are many good and useful things out there for reasonable prices, so support the low prices and buy the products if they seem useful.
What can you say about an undelete utility? As long as it works, then it has done as promised. Trashman goes a bit farther than that in giving you the ability to control how many generations of a file will be maintained, and it is 100% in it’s ability to undelete a file. The graphical interface, via the Wingspan product, to the Recycle utility is really great, they have done a top notch job of simulating a GUI on a regular HP terminal.
As great as the interface looked, the Tagging and Untagging functions were a little strangly implemented. For example, you would type a T or U to Tag or Untag, but you would actually have to hit the key twice to get the terminal read to satisfy the two character read. That is one of the draw backs to a terminal based GUI since the arrow keys actually generate two characters you have to read both characters, so there is no way to do a one character read and satisfy the demand to read the arrow keys.
I would have liked a way to just list files available for recycling at the command level. I also felt that the Recycle program should give you the option of renaming or purging a file if there was a naming conflict during the recycle process as well. I mentioned these to tech support and the developer thought they were good ideas, so we should see them soon. This is the first release of the product and I am sure as more people use it and Software Research Northwest get’s more input we will see the product grow and become even easier to use than it already is.
So bottom line, I like it, it works, it’s cheap, and it could save your job in an emergency. Too many of us put of disaster recovery until after the disaster, now’s your chance to plan ahead.
At-a-Glance box
Trashman, version 1.0 from Software Research Northwest Inc.
17710 100th Ave. SW
Vashon Island WA 98070
Phone (206) 463-3030
FAX (206) 463-9393
$960 Per copy, $480 each for additional copy. Support is $90 for first CPU and then $45 for each additional CPU. One 79 page manual is included.