Pages

Friday, March 23, 2007

Flying 300 Miles to Correct an Error!

OK ... thought it might be time to repeat this story. I frequently use it in my PM classes as an object lesson in detail and quality. It has a place in my IT classes as well, since it points out an age old issue. I hope you at least get a little chuckle from this one!

This goes back to the very late 70s and early 80s. I was working on a project with an advanced concept for the time. The product we were developing needed to run equally well on any major IBM platform. At the time we selected the 360/370 series running OS-VS, OS-MVS, DOS-VS, DOS-VSE, VM, and the older IBM System 3. Most of the code was in COBOL for ease of portability. Some of the code had to be in assembly language since we were dealing with communications channels specific to the systems. We obviously tried to limit the amount of this code to keep things simpler.

Things went very well during development and initial testing, but the full roll out was rocky to say the least. Finding available equipment to test all these combinations was a major problem. The OS versions were easy -- the company owned the right platforms. For DOS and DOS-VSE, we finally contacted a number of service bureaus and found one that fit our budget. Our local IBM office was kind enough to lease us time on an IBM System 3, so we thought we were home free.

For those of you that aren't familiar with it, the IBM System 3 was a strange beast. Its primary inputs consisted of punch cards and floppy disks. The punch cards weren't very standard, however. They were tiny cards with three rows of punches -- 96 columns arranged in rows of 32. This took a special key punch which made tiny round holes. I bought a special card box to hold the "master" copy we produced at the IBM facility. The local office only supported punch cards, so this became our primary medium.

This scheme worked fine, until we encountered a client in Detroit without a card reader. Our local IBM office connected me with an IBM Data Center in Indianapolis where I could convert our master card deck on to a floppy disk. They even sent one of their local support people with me to assist. We tried running the job several times, but always got a warning on a certain card. Looking at a listing of the COBOL code, we saw a blank enclosed in quotes (" "). Thinking this was nothing out of the ordinary and needing to get over to the client, we packed up the newly made floppy disk and headed out. This was to be a simple, overnight trip.

Next day at the client site, everything installed and compiled smoothly. But getting the software to communicate was another issue. We were unable to connect to the communication channel. Traces showed we sent one message which was immediately rejected. We puzzled over this for hours. Then I remembered the card that kept giving us problems when copying to the floppy. I called up the programmer and had him take a look at the statement. Uh oh! Big trouble! The quotes actually enclosed a non-printable ASCII character which was an essential part of the initial handshake between computers. Had I looked at the actual punch card instead of the listing, I would have seen an odd punch combination it took special pains to make.

So now we had a fix -- the programmer gave us an equivalent statement with the decimal value so it would be visible. But the client had no facility to edit the floppy disks. Next day we hopped on a plane and flew back to Indianapolis (about 300 miles). We fixed up the cards, copied them to floppy again, then went back to the customer. Success! Everything went without a hitch.

Now there are several lessons learned here, so be careful not to miss any! These mistakes cost an extra flight for two and turned an overnight trip into three or four days. First, set a programming standard prohibiting non-printable characters in quotes in the code. These are too easy to overlook. Next, whenever testing text files, always check the file for non-printable characters. I was hit by this one again a few months ago due to a programming error. Dumping the file in hex showed the offending character. Also, look at the actual source, not the copy. Had we looked at the punch card the first time, we might have caught this before leaving the Data Center. And finally, always be prepared for the unexpected! This was so long ago, I don't recall what I did for clothing the extra days -- I'm sure I had to buy an extra shirt or two while on the road.

No comments: