Test Drive of Shadow D/R
by
Shawn M. Gordon
President SMGA
What it can do?
In a less than crowded market place for data replication products, Lund has revived Shadow from it’s acquisition of the Carolian products from a few years ago. They’ve blown off the dust, fixed the bugs, added som enhancements, and released it as Shadow D/R. The concept behind Shadow is that it will perfectly copy data in an HP 3000 Image/SQL database between 2 or more HP 3000’s.
Shadow only works with Image/SQL in this release, no Allbase, KSAM or MPE files can be shadowed. For some people that’s a problem, for many other’s it’s not. By shadowing your data you can buy a number of things very easily. Offsite, realtime backup of your critical data. Distributed processing so that you can load balance between machines, imagine a batch only machine where reports could duke it out without affecting online users.
How does it work?
Shadow D/R is implemented by taking advantage primarily of Image logging. This feature comes with Image, and will maintain a transaction log separate to the database of each transaction. You can control this yourself to a certain degree by using DBBEGIN..DBEND pairs in your code, but it’s not necessary. Shadow makes use of NS services to transport the data to the shadow machine and roll’s the transactions out of the log file, into the database. From this perspective Shadow doesn’t have much to do, but it’s more complex than that.
There are monitor processes on both systems to keep track of the ‘heartbeat’ between the two systems making sure that communication is always there. If you loose your connection for some reason, like a system going down, then Shadow will alert the still active machine. There is a high water mark that you can set for how many transactions can queue on the master machine before Shadow will start archiving to tape. This is based on the assumption that both machines could go down at any time once the first machine goes down. This is an interesting feature, and a good idea for disaster recovery purposes.
There is typically one background job for each account/group that you have databases that you want to shadow, which also means there is a config file for each. Take a look at figure 1 for an example config file. The format is rather similar to a schema file, but here is where you specify how and where the remote system is located, the database to shadow, the name of the log file that will contain the information, how the job should log on.
Features
Shadow has a number of handy features for use in a production shop. There is the BACKUP command which will suspend updates on the shadow system to allow you to back it up. All changes will queue on the master system until a RESUME command is issued. This can give you a theoretical 24 by 7 access to your production database if you aren’t concerned about the records that queue up during the backup period.
As mentioned earlier, Shadow will start archiving to tape if the log’s hit their high water mark. The RECOVER command will roll data into the database from the tape. Their are a couple of options to tell Shadow whether to drop incomplete transactions or not.
There are a number of other commands to start and stop Shadow configurations, see what is running in Shadow, verify a config file, etc. Nothing very glamorous, but it get’s the job done. One thing that Shadow would benefit from in this regard is a unified config and maintainance program as opposed to the loose config files laying around.
Installation and Documentation
Installation is very easy, your standard RESTORE and job stream to configure the accounting structure. The manual is very well written and easy to follow. It’s slightly over 120 pages, but this includes a table of contents, an index, and a number of appendixes with things like the error messages and frequently asked questions.
The manual is arranged to first give you an overview of what shadowing is, why and how you would use it, and how it works. It then moves logically forward to how to configure the system, how to use it, and a description of all commands. Overall a very easy read, and quit friendly, however it could really use a demo guide that walked you through an example of the whole process. The only real problem is that it still refers to the SHADOW account, and not the LPS account that it’s distributed in. The default config file contains the same errors.
Test Drive
I built a new data base from scratch so I would have a fresh environment. There are two main tricks to getting going, one is getting the Image logging enabled, which isn’t straight forward for most people. The other is making sure you have the right node information in the config file. Here again, a very clear short, canned tutorial would do wonders to speed this process up.
As part of the price of the product, Lund will send a tech out and help you get set up and going. While this is very generous, it isn’t strictly necessary because the product isn’t that hard. I spent about 20 minutes on the phone with them clearing up my NS connection because I was using the name from HOSTS.NET.SYS and not the name from NMMGR of the remote machine. Once that was set up, away we went.
A nice thing with Shadow is that you don’t have to set up anything on the shadow machine, other than making sure you have the database over there that you want to shadow to. Once the job fires off, just start adding, changing and deleting records from your master database. You can easily inquire off the shadow machine without any problems. The tests I ran had the data showing up just as quickly as I could check, I didn’t do any cpu second counts, but it was instant as far as I could manually verify.
I tried a number of things to test the fault tolerance like severing the NS connection, aborting the shadow job on the master, on the slave. Shadow handled everything correctly. When severing the connection, it would start writing out to tape when required, transactions would queue up and resynchronize very easily. I was very happy with how fast and easy it was to use.
Comparisons
The obvious comparison here is to Netbase from Quest, and possibly Silhouette if it’s still around. I have years of experience using Netbase, and it is a very complete product. I have no experience with Silhouette, so I can’t comment.
From the perspective of straight Image shadowing, Shadow D/R is much easier than Netbase, and seems to have less process overhead. From the perspective of a complete solution for every type of file, and porting different processes access around the network to different machines depending on the access type, then Netbase is definitely in the lead. I don’t know that a full comparison between the two products is entirely fair, since once you are out of Image, Shadow can’t do anything, and Netbase still has tons of options left, you pay for these options of course, but if you need them, it’s worth it.
Conclusions
It’s nice to start to have some choice in this arena. Lund has done a nice job with Shadow, it’s fast, it’s easy to set up and maintain, it works, and it’s fairly priced. The overhead on your system is pretty light as well. I am anxious to see support of non-Image files, as I think this would really round the product out. The manual needs to be better in it’s explanation of setting up the image log files. This can be confusing for many people, and they don’t have a step by step tutorial
Using Image log files as the basic mechanism is both elegant and limiting. It makes Image easy, but other file types very difficult. I’m told by Lund that they will support other file types and the CI by the end of the year. Despite this current limitation, if you are Image-centric, and haven’t addressed disaster recovery, you should definitly take a look at Shadow D/R as one of your options.
Road Report
Shadow D/R version E.01.05
Lund Performance Solutions
240 Second Ave. SW
Albany, OR 97321
Phone 541-926-3800
FAX 541-926-7723
email: info@lund.com
http://www.lund.com
Shadow D/R includes the software required for both systems to enable you to shadow to a second machine. The NS services that it makes use of are included with the base operating system so no further software is required.
Shadow D/R for the HP 3000 runs on all HP 3000 Series 900s. The software is tier based ranging from $6,000 to $21,500. Discounting on multiple CPUs. Support ranges from $1,200 to $4,300 per year and includes phone in, electronic support and new releases of the software. All prices are in US dollars.
Figure 1 BEGIN << Sample SHADOW configuration file >> LOCAL=!JOB shadow,mgr/nimbus.smga,pub;hipri;outclass=lp,1,1;pri=bs: COMMANDS = shadowp1.pub.smga: << Pipe to CP >> RESPONSES = shadowp2.pub.smga: << Pipe from CP >> SAVEFILE = shadowsv.pub.smga: << Saves environment >> USE logproc WITH dumb.pub.smga/FUGAZI: MODE = 1: << mode=1/locking >> WAIT = 6000: << ten seconds >> MAXRECS = 1000: << before requesting tape >> DSLINE = OMEGA: << 1 LAN Lines >> REMOTE=hello shadowsp,mgr.smga;pri=bs;hipri: LOG = logfl001.pub.smga: TAPEDEV = tape: << Tape device class >> END. << of configuration file >>