Isam problems

I have just "fixed" a bug that appears to be a comppiler error, and I am
a bit worried that it is going to crop up later and bite us. The
platform is Windows(98), and the compiler is MF NetExpress Version 3.0.
I was wondering is anybody else has seen something similar.

The Isam File had a key length of 120 characters, and was built as a
temporary file by stringing a bunch of fields into that key. The total
length of the fields going into the key varies, but never exceeds 104.

The file is then closed, and re-opened as an input only file. The
process then goes into a loop:

read sequential
store key
read sequential in loop
restore key, and re-start at stored position.

About halfway through, I got an error that I had an invalid key on the
restore record position using the stored key.  My first thought,
obviously, was that somewhere the program had stepped on the stored key.
Not so.  I put in a patch storing it in a second place, and doing a
compare of the two versions before the restart. The key from storage was
unchanged.

Second thought ... somewhere in the program, the record had been
deleted.  Not so. I made sure the file was only open in input during the
second pass, and also did a file compare of it as built, and after the
error occured.

Finally, in desparation, I reduced the key size by one character. It
worked!  (Note that the code has been working for YEARS). I put it back
... it failed. Obviously the storage location was too short ... nope,
not so. The storage location was also 120 characters, and I changed it
to 119 when I changed the key.

It seems that with a key length of 120, it will not work, and with a key
length of 119, it will not.  There are alternate keys, but all reading
was being done using the primary. The key I am trying to do the read
with comes out of the file, so there is no way it should be invalid. The
file status is 23, which agrees with the invalid key.

As the saying goes ... WTF? Any ideas, or do we just pray it continues
to work?

Donald

Donald Tees <donald_tees@sympatico.ca> wrote in message news:msDAc.52038$7H1.1795747@news20
.bellglobal.com...
> I have just "fixed" a bug that appears to be a comppiler error,

When the reason for an anomaly isn't obvious after a reasonable amount
of research and debugging, it is normal to "grasp at straws",
only to end up feeling foolish when the real cause is finally found.

Been there, done that.

> and I am a bit worried that it is going to crop up later and bite us.

I'd venture a guess that it will.

> The platform is Windows(98), and the compiler is MF NetExpress Version 3.0
.
> I was wondering is anybody else has seen something similar.

Sorry.  I'm a mainframe guy.
>
> The Isam File had a key length of 120 characters, and was built as a
> temporary file by stringing a bunch of fields into that key. The total
> length of the fields going into the key varies, but never exceeds 104.

Idle though:  Does the "String" operation always use blanks inter-field?
>
> The file is then closed, and re-opened as an input only file. The
> process then goes into a loop:
>
> read sequential
> store key
> read sequential in loop
> restore key, and re-start at stored position.
>
> About halfway through, I got an error that I had an invalid key on the
> restore record position using the stored key.  My first thought,
> obviously, was that somewhere the program had stepped on the stored key.

Sound reaonable.

> Not so.  I put in a patch storing it in a second place, and doing a
> compare of the two versions before the restart. The key from storage was
> unchanged.
>
> Second thought ... somewhere in the program, the record had been
> deleted.

I've been bitten by that one before, albeit on a checkpoint restart.

> Not so. I made sure the file was only open in input during the
> second pass, and also did a file compare of it as built, and after the
> error occured.
>
> Finally, in desparation, I reduced the key size by one character. It
> worked!  (Note that the code has been working for YEARS).

I've heard that one before, and I appreciate where you are coming from,
but it still does NOT mean that the logic is error free, although it does
make it more likely that the issue is data-related.

> I put it back
> ... it failed. Obviously the storage location was too short ... nope,
> not so. The storage location was also 120 characters, and I changed it
> to 119 when I changed the key.
>
> It seems that with a key length of 120, it will not work, and with a key
> length of 119, it will not.

Huh?  I'll assume that the last word "not" shouldn't be there.
Otherwise, you just lost me.

> There are alternate keys, but all reading
> was being done using the primary. The key I am trying to do the read
> with comes out of the file, so there is no way it should be invalid. The
> file status is 23, which agrees with the invalid key.
>
> As the saying goes ... WTF? Any ideas, or do we just pray it continues
> to work?
>
> Donald

Could you be hitting EOF within the loop, causing a read-after-EOF condition
?
Do you have a checkpoint-restart routine?  Could it be interfering?
What is the key before the problem key?  Could you have a duplicate?

That's all I have off the top of my head.  Hope you find the error.
If it was up to me, I'd persevere until I found the cause.

Hugh Candlin wrote:
> Donald Tees <donald_tees@sympatico.ca> wrote in message news:msDAc.52038$7
H1.1795747@news20.bellglobal.com...
> 
>
>
> When the reason for an anomaly isn't obvious after a reasonable amount
> of research and debugging, it is normal to "grasp at straws",
> only to end up feeling foolish when the real cause is finally found.
>
> Been there, done that.

Me too.  Thats why I posted it.

> Idle though:  Does the "String" operation always use blanks inter-field?

No, there is nothing interfield, it is string by length.

>
> Huh?  I'll assume that the last word "not" shouldn't be there.
> Otherwise, you just lost me.

Correct (what was that last w? avoid "NOT"?)  ;<(

> Could you be hitting EOF within the loop, causing a read-after-EOF conditi
on?
> Do you have a checkpoint-restart routine?  Could it be interfering?
> What is the key before the problem key?  Could you have a duplicate?

We are definitely hitting the end of file at places, but that is not
illegal on an Isam file. It is quite legal to read sequential, trigger
an end of file, then read using a key from the middle again. Also, in
this particular case, that is not true. In fact, it does not happen in
this program unless the program is working ... it happens on the last
set of records, after we get by this.

I also checked the records before and after the culprit record, looking
for exactly the sort of thing you mention, but all keys appear to be
unique ... oviously so for several records for and aft the one that will
not read.
>
> That's all I have off the top of my head.  Hope you find the error.
> If it was up to me, I'd persevere until I found the cause.
>

If it was up to me, I might as well. Unfortunately, it is an end of year
program that has been patched to the point it got the job done, and has
become extremely low priority (until next year, of course).

Donald

I would change the file to DYNAMIC and the restart to START NOT LESS followe
d by
READ NEXT followed by compare for equal. I would also look for a character l
ess
than space in byte 120.


Donald Tees <donald_tees@sympatico.ca> wrote:

>I have just "fixed" a bug that appears to be a comppiler error, and I am
>a bit worried that it is going to crop up later and bite us. The
>platform is Windows(98), and the compiler is MF NetExpress Version 3.0.
>I was wondering is anybody else has seen something similar.
>
>The Isam File had a key length of 120 characters, and was built as a
>temporary file by stringing a bunch of fields into that key. The total
>length of the fields going into the key varies, but never exceeds 104.
>
>The file is then closed, and re-opened as an input only file. The
>process then goes into a loop:
>
>	read sequential
>	store key
>	read sequential in loop
>	restore key, and re-start at stored position.
>
>About halfway through, I got an error that I had an invalid key on the
>restore record position using the stored key.  My first thought,
>obviously, was that somewhere the program had stepped on the stored key.
>Not so.  I put in a patch storing it in a second place, and doing a
>compare of the two versions before the restart. The key from storage was
>unchanged.
>
>Second thought ... somewhere in the program, the record had been
>deleted.  Not so. I made sure the file was only open in input during the
>second pass, and also did a file compare of it as built, and after the
>error occured.
>
>Finally, in desparation, I reduced the key size by one character. It
>worked!  (Note that the code has been working for YEARS). I put it back
>... it failed. Obviously the storage location was too short ... nope,
>not so. The storage location was also 120 characters, and I changed it
>to 119 when I changed the key.
>
>It seems that with a key length of 120, it will not work, and with a key
>length of 119, it will not.  There are alternate keys, but all reading
>was being done using the primary. The key I am trying to do the read
>with comes out of the file, so there is no way it should be invalid. The
>file status is 23, which agrees with the invalid key.
>
>As the saying goes ... WTF? Any ideas, or do we just pray it continues
>to work?
>
>Donald
>
>
>
>
>
>

Donald Tees wrote:
>
> I have just "fixed" a bug that appears to be a comppiler error, and I am
> a bit worried that it is going to crop up later and bite us. The
> platform is Windows(98), and the compiler is MF NetExpress Version 3.0.
> I was wondering is anybody else has seen something similar.
>
>

This reply is chronologically subsequent to the replies posted up to
today at 4:33 p.m.

It appears that you've spent a lot of time on this problem and might be
getting oversaturated with it.  I have a purely hypothetical suggestion
- based on years of experience  with similar impossible things - which
is: The Problem is Always Somewhere Else.  I have no practical
suggestions, since you've tried everything obvious, but try looking
somewhere else in the program.  It sometimes works because the problem
really is somewhere else; and sometimes by looking somewhere else your
subconscious is freed up to find the answer on its own.

Best of luck
PL

Actually I can believe this. I've run into more than one
"anomaly" with Micro Focus ISAM files, particularly if
there are alternate indexes. I can easily see a key conflict
between primary and alternate keys causing something like
this even if it "shouldn't" in a VSAM file on the mainframe.
You could have stepped on an alternate key. MF wants all
the alternate key stuff there and correct, even if you don't
ever use it.

Suggestions, first rebuild the file. Second carefully
check that ALL the key definitions match EXACTLY between
the program and the files. Check the lengths, record lengths,
key lengths and positions, things like "WITH DUPLICATES" in
the select if the AIX's allow them. Are you overflowing
an alternate key? Is the key really 120 or are there positions
or lengths off by 1 because of relative to 0? Just a few
things I thought of.

Donald Tees wrote:

Some other things:
1. How big is the actual record - including key? Is it possible the record
exceeds your max permitted size?
2. Is the problem exactly reproducible? That is, same error, same place, on
the same file?
3. After building the file, read it back sequentially and build another
file. Use the other file in your process. Same error?
4. What's magic about 120? Does you compiler allow a primary key that large?

JerryMouse wrote:
> Donald Tees wrote:
>
> Some other things:
> 1. How big is the actual record - including key? Is it possible the record
> exceeds your max permitted size?

Not that large. Well under 1k. No, it is not even near.

> 2. Is the problem exactly reproducible? That is, same error, same place, o
n
> the same file?

Yes.

> 3. After building the file, read it back sequentially and build another
> file. Use the other file in your process. Same error?

Yes, multiple times.

> 4. What's magic about 120? Does you compiler allow a primary key that larg
e?
>
>
That is indeed the problem.  Yes, it is no where near the limit.

Donald

JerryMouse wrote:
> Donald Tees wrote:
>
> Some other things:
> 1. How big is the actual record - including key? Is it possible the record
> exceeds your max permitted size?
> 2. Is the problem exactly reproducible? That is, same error, same place, o
n
> the same file?
> 3. After building the file, read it back sequentially and build another
> file. Use the other file in your process. Same error?
> 4. What's magic about 120? Does you compiler allow a primary key that larg
e?
>
>

PS to that last.

I can also put a break point in, and stop the program immediately after
the first read, close the file, and make a backup of it. Then let it
abort, and compare the file *after* the abort.  They are the same. There
is absolutely no doubt it read the key from the file on a read next,
then failed to read it using the same key when read randomly(giving and
invalid key and a file status 23).  Also no doubt that there is no
duplicate record at, before, or behind the culprit record. mmm maybe
Robert is right ... say they last character was a null.  Then when the
key was moved to working storage, it was changed to blank. That should
not happen, should it?  PIC X(n) to PIC X(n) should *never* change data
in Cobol. Can an alpha-numeric key have an illegal byte in it?

Donald

Donald Tees <donald_tees@sympatico.ca> wrote:

> say they last character was a null.  Then when the
>key was moved to working storage, it was changed to blank. That should
>not happen, should it?  PIC X(n) to PIC X(n) should *never* change data
>in Cobol. Can an alpha-numeric key have an illegal byte in it?

Moving pic x to pic x will not change data, but writing that pic x to a file
might. In the case of line sequential files, Micro Focus inserts nulls befor
e
characters less than space. I realize your file is indexed, not line sequent
ial,
but there might be something similar going on.