BPAM NCP=2 PURGE

Hello:

I am trying to fix an old bug (over 10 years) in one of my programs which
reads PDS members with NCP=2.  I use the technique suggested in the DFP
Using Datasets manual, namely:

FIND DCB2,MEMBER,D
READ DECB1,SF,DCB2
READ DECB2,SF,DCB2
LOOP EQU *
CHECK DECB1
. . . process 1st buffer
READ DECB1
CHECK DECB2
. . . process 2nd buffer
READ DECB2
B LOOP
. . .
DCB2  DCB DSORG=PO,NCP=2,MACRF=R

This works fine as long as the program exits the loop via EODAD or SYNAD.
The problem arises when the program reads multiple members and decides to
exit the loop before EODAD or SYNAD to, in effect, "flush" the rest of the
member, leaving an outstanding READ which has not been CHECKed.  The
program could, of course, issue a CHECK against the next DECB to wait for
completion, but I'd like to avoid that, for obvious performance reasons
(waiting may take a while and the data is of no interest to the program).

If I don't CHECK the last DECB, the program eventually hangs on a CHECK
macro when it tries to process other members. How long the program takes
to hang seems timing-dependent and doesn't occur all the time (or in the
same CHECK) in the real world, although I can recreate the problem
reliably on my P/390 when nothing else runs on it.

Issuing PURGE, even when it returns R15=0, seems to have some effect (I'd
be hard-pressed to say exactly what), but doesn't really solve the
problem:

XC    NPPL,NPPL               Clear NPPL
MVI   NPPLOPT1,NPPLNPPL       NPPL format
OI    NPPLOPT1,NPPLDS         purge one data set
OI    NPPLOPT1,NPPLPOST       post ECBs
MVC   NPPLDSID+1(3),DCBDEBA   COPY 24-BIT DEB ADDRESS TO NPPL
MVI   NPPLCC,X'7F'            COMP CODE
PURGE NPPL                    PURGE OUTSTANDING REQUESTS

Issuing TCLOSE doesn't seem to make any difference either.

I tried to look at the ICQE, the SAMB, the IOBs to see if I could trick my
way through a work-around, to no avail.  My gut feeling is that FIND and
PURGE aren't enough to tell BPAM that it should reset the SAMB/ICQE/IOBs,
etc, the way it does when it hits CE/DE.

The problem has occured for over 10 years when NCP>1.  When it happens,
people cancel the program and restart it after adding NCP=1 on the DD
statement.  I tried to fix the problem ten years ago, but gave up.
I spent the last 4 days trying to fix it to no avail.

The program needs NCP>1 for performance, and be able to stop reading
certain members at any time.  Can anyone help ?  I haven't tried
MULTACC/MULTDSN yet.  Could they help?

Thanks.

Gilbert Saint-Flour
http://gsf-soft.com/

Gilbert,

I'll take a guess and say that you observe a hang
when you reuse a DECB that has not been CHECKed and
the triggered I/O has not completed.  I'll say that
with the same program logic if the I/O completed
before you could reuse the DECB then no hang occurs.

This would fit in with your observations of timing
dependence.

I don't know what you should do.

What I would do is issue a CHECK for any DECB for
which I've issued an I/O macro and just wait it out.
In fact, I think I've seen code (not written by me)
which issues a WAIT macro for all outstanding DECBs
for which the data being transferred (if any) is no
longer of interest.

I suppose the advantage of the WAIT macro is that it
verifies that the event has completed but will not
trigger entry into any DCB-related exit routine.

IIRC the ECB is at the start of the DECB, but verify
that before you code up the WAIT logic (if you do).

Considering code complexity vs. potential time saved
I would not be interested in exploring I/O purge logic.
However, if you think that would be lots of fun, don't
let me discourage you.    :)

Cheers,
Greg

On Wed, 14 Jan 2004 16:27:06 +1100 Greg Price <grogon@groginon.com> wrote:

:>I'll take a guess and say that you observe a hang
:>when you reuse a DECB that has not been CHECKed and
:>the triggered I/O has not completed.  I'll say that
:>with the same program logic if the I/O completed
:>before you could reuse the DECB then no hang occurs.

:>This would fit in with your observations of timing
:>dependence.

:>I don't know what you should do.

:>What I would do is issue a CHECK for any DECB for
:>which I've issued an I/O macro and just wait it out.
:>In fact, I think I've seen code (not written by me)
:>which issues a WAIT macro for all outstanding DECBs
:>for which the data being transferred (if any) is no
:>longer of interest.

:>I suppose the advantage of the WAIT macro is that it
:>verifies that the event has completed but will not
:>trigger entry into any DCB-related exit routine.

The bigger advantage of the WAIT macro is if you have several pending events
you can get control whenever any of them complete.

CHECK is limited to a single DECB.

:>IIRC the ECB is at the start of the DECB, but verify
:>that before you code up the WAIT logic (if you do).

:>Considering code complexity vs. potential time saved
:>I would not be interested in exploring I/O purge logic.
:>However, if you think that would be lots of fun, don't
:>let me discourage you.    :)

--
Binyamin Dissen <bdissen@dissensoftware.com>
http://www.dissensoftware.com

Director, Dissen Software, Bar & Grill - Israel

I have a problem understanding why your code has been designed the
way it is. (And I seriously believe that your "problem" is simply a result
of the design! You do not seem to cater for channel end or device end
interrupts that may occur after you close the "connection" to the dataset).

As far as I can see you are reading one or more members in partitioned
datasets and each member is read sequentially but not necessarily to its
end. So why not implement the standard LIOCS logic for such operations?

Open:
locate the member (sequential file) to be read and initiate control blocks
Start reading the first record into one of the two I/O buffers (READ)
return to caller

Get:
if the read operation is still in progress wait for this to complete
(CHECK)
(and handle any exception that may be signalled)
start reading the next record into the other I/O buffer
return to caller with the address of the just completed buffer

Close:
if a read operation is in progress wait for this to complete
finish up control blocks
return to caller.

I cannot see any advantage in initiating a read for the second record
before the caller begins processing the first and so on. On the contrary
there is for instance a significant possibility that the channel in certain
cases may reorder the channel programs queue and in fact retrieve
the second record before the first because of hardware conditions
invisible to you.

But I may be wrong - after all it is some fifteen or twenty years since
I was writing channel programs myself.

regards Sven

"Gilbert Saint-Flour" <usenet5678@yahoo.com> wrote in message
news:pan.2004.01.13.17.06.09.150878@yahoo.com...
> Hello:
>
> I am trying to fix an old bug (over 10 years) in one of my programs which
> reads PDS members with NCP=2.  I use the technique suggested in the DFP
> Using Datasets manual, namely:
>
>   FIND DCB2,MEMBER,D
>   READ DECB1,SF,DCB2
>   READ DECB2,SF,DCB2
> LOOP EQU *
>   CHECK DECB1
>  . . . process 1st buffer
>   READ DECB1
>   CHECK DECB2
>  . . . process 2nd buffer
>   READ DECB2
>   B LOOP
>  . . .
> DCB2  DCB DSORG=PO,NCP=2,MACRF=R
>
> This works fine as long as the program exits the loop via EODAD or SYNAD.
> The problem arises when the program reads multiple members and decides to
> exit the loop before EODAD or SYNAD to, in effect, "flush" the rest of the
> member, leaving an outstanding READ which has not been CHECKed.  The
> program could, of course, issue a CHECK against the next DECB to wait for
> completion, but I'd like to avoid that, for obvious performance reasons
> (waiting may take a while and the data is of no interest to the program).
>
> If I don't CHECK the last DECB, the program eventually hangs on a CHECK
> macro when it tries to process other members. How long the program takes
> to hang seems timing-dependent and doesn't occur all the time (or in the
> same CHECK) in the real world, although I can recreate the problem
> reliably on my P/390 when nothing else runs on it.
>
> Issuing PURGE, even when it returns R15=0, seems to have some effect (I'd
> be hard-pressed to say exactly what), but doesn't really solve the
> problem:
>
> XC    NPPL,NPPL               Clear NPPL
> MVI   NPPLOPT1,NPPLNPPL       NPPL format
> OI    NPPLOPT1,NPPLDS         purge one data set
> OI    NPPLOPT1,NPPLPOST       post ECBs
> MVC   NPPLDSID+1(3),DCBDEBA   COPY 24-BIT DEB ADDRESS TO NPPL
> MVI   NPPLCC,X'7F'            COMP CODE
> PURGE NPPL                    PURGE OUTSTANDING REQUESTS
>
> Issuing TCLOSE doesn't seem to make any difference either.
>
> I tried to look at the ICQE, the SAMB, the IOBs to see if I could trick my
> way through a work-around, to no avail.  My gut feeling is that FIND and
> PURGE aren't enough to tell BPAM that it should reset the SAMB/ICQE/IOBs,
> etc, the way it does when it hits CE/DE.
>
> The problem has occured for over 10 years when NCP>1.  When it happens,
> people cancel the program and restart it after adding NCP=1 on the DD
> statement.  I tried to fix the problem ten years ago, but gave up.
> I spent the last 4 days trying to fix it to no avail.
>
> The program needs NCP>1 for performance, and be able to stop reading
> certain members at any time.  Can anyone help ?  I haven't tried
> MULTACC/MULTDSN yet.  Could they help?
>
> Thanks.
>
>  Gilbert Saint-Flour
>  http://gsf-soft.com/
>
>

In article <pan.2004.01.13.17.06.09.150878@yahoo.com>,
Gilbert Saint-Flour <usenet5678@yahoo.com> wrote:
>Hello:
>
>I am trying to fix an old bug (over 10 years) in one of my programs which
>reads PDS members with NCP=2.  I use the technique suggested in the DFP
>Using Datasets manual, namely:
>
>  FIND DCB2,MEMBER,D
>  READ DECB1,SF,DCB2
>  READ DECB2,SF,DCB2
>LOOP EQU *
>  CHECK DECB1
> . . . process 1st buffer
>  READ DECB1
>  CHECK DECB2
> . . . process 2nd buffer
>  READ DECB2
>  B LOOP
> . . .
>DCB2  DCB DSORG=PO,NCP=2,MACRF=R
>
>This works fine as long as the program exits the loop via EODAD or SYNAD.
>The problem arises when the program reads multiple members and decides to
>exit the loop before EODAD or SYNAD to, in effect, "flush" the rest of the
>member, leaving an outstanding READ which has not been CHECKed.  The
>program could, of course, issue a CHECK against the next DECB to wait for
>completion, but I'd like to avoid that, for obvious performance reasons
>(waiting may take a while and the data is of no interest to the program).
>
>If I don't CHECK the last DECB, the program eventually hangs on a CHECK
>macro when it tries to process other members. How long the program takes
>to hang seems timing-dependent and doesn't occur all the time (or in the
>same CHECK) in the real world, although I can recreate the problem
>reliably on my P/390 when nothing else runs on it.


Criminently!  The causation  (UnCHECKed DECB) is identified, the remedy (CHE
CK
the unCHECKed DECB)is identified, and the 'obvious' fix hasn't been implemen
ted
in TEN YEARS.

FIND DCB2,MEMBER,D
READ DECB1,SF,DCB2 
READ DECB2,SF,DCB2 
LOOP EQU * 
CHECK DECB1
. . . process 1st buffer
READ DECB1 
CHECK DECB2
. . . process 2nd buffer
READ DECB2 
B LOOP

{{ loop exit }} 
|    READ DECB1
|   [clear 'flag1']
|   [if 'flag2' set]
|    READ DECB2
|   [clear 'flag2']


This is *ELEMENTARY* 'housekeeping' logic for asynchronous operations

If I remember anything at all from my IOCS programming you probably
mean CHECK - not READ in your sample code at loop exit?

regards Sven

"Robert Bonomi" wrote
............
> Criminently!  The causation  (UnCHECKed DECB) is identified, the remedy
(CHECK
> the unCHECKed DECB)is identified, and the 'obvious' fix hasn't been
implemented
> in TEN YEARS.
>
>        FIND DCB2,MEMBER,D
>        READ DECB1,SF,DCB2 
>        READ DECB2,SF,DCB2 
>      LOOP EQU * 
>        CHECK DECB1
>       . . . process 1st buffer
>        READ DECB1 
>        CHECK DECB2
>       . . . process 2nd buffer
>        READ DECB2 
>        B LOOP
>
>  {{ loop exit }} 
>   |    READ DECB1  <------------- CHECK (I suppose)???
>   |   [clear 'flag1']
>   |   [if 'flag2' set]
>   |    READ DECB2  <------------- CHECK (I suppose)???
>   |   [clear 'flag2']
>
>
> This is *ELEMENTARY* 'housekeeping' logic for asynchronous operations
>

In article <TQeNb.1302$hd.27458@news2.e.nsc.no>,
Sven Pran <no.direct@mail.please> wrote:
>If I remember anything at all from my IOCS programming you probably
>mean CHECK - not READ in your sample code at loop exit?

Spoilsport!  By deliberately putting an 'obvious' bug in the illustration, i
t
forces somebody trying to use it to _understand_ what they're actually doing
.

*grin*



That said, the 'feature' got into this illustration because I cut-and-pasted
the _wrong_ line.  And you are absolutely correct, that is what I .meant_.

One of the nice things about writing code for _people_ to read, instead of
computers -- "Do what I meant, not what I said" *does* work most of the time
!


Thanks for the catch.


>
>regards Sven
>
>"Robert Bonomi" wrote
>............ 
>(CHECK 
>implemented 
>
>

"Gilbert Saint-Flour" <usenet5678@yahoo.com> wrote in message
news:pan.2004.01.14.20.37.34.619235@yahoo.com...
> On Wed, 14 Jan 2004 16:55:19 +0100, Sven Pran wrote:
> 
>
> For performance reasons.
> 
>
> I do not take care of CE/DE directly, you're right, and that's probably my
> problem and I started this thread to find some help to solve it. When I
> issue PURGE then FIND, SAM could take the hint and take care of CE/DE for
> me - apparently, it doesn't.
> 
>
> Again, for performance.
>
> My example only showed NCP=2 for simplicity.  In the real world of
> my programs, NCP is calculated from the BLKSIZE to not exceed the old
> SAM-E constants (NCP=30,BUFF=240K) that are hard-coded in the SAMB.  As a
> result, the NCP value varies between 8 and 30.
>
> On my ADCD system, SYS1.MACLIB is defined with BLKSIZE=6160, which results
> on the program using NCP=30.  If I'm only interested in processing one or
> two blocks for each member on the average, not issuing CHECKs for dropped
> READs results in significant savings in elapsed and connect times.
>
> Another reason for trying to find a solution, of course, is the challenge.

Have you verified that you get an improvement in performance by coding this
way
instead of using standard sequential LIOCS logic?

You may be surprised, and you may doubt me, but when processing a dataset
(like a member in a PDS) sequentially there is only one way to improve
performance
beyond the LIOCS coding I described and that is by having all the records
preloaded
in memory so that you avoid disk accesses entirely when processing data.

The point is that because a disk device is unable to process more than one
channel
program at the same time there is nothing to save by issuing another READ
for the
same device while another READ is still pending. This holds true even when
you have
several different access paths to the same device because the device only
has one set
of head assemblies.

So sorry pal, I don't think I buy your reasoning!

regards Sven

Xref: kermit comp.lang.asm370:8104


"glen herrmannsfeldt" <gah@ugcs.caltech.edu> wrote in message
news:VaiNb.52555$sv6.127618@attbi_s52...
> Gilbert Saint-Flour wrote:
> 
>
> (snip)
> 
> 
a 
>
> (snip)
> 
challenge.
>
> So performance, and not necessarily size.
>
> Can you do a multiple WAIT for all the ECB's, and then CHECK each
> one after it completes?   That is, have multiple sets of NCP
> buffers, so you can start the next before the previous ones are
> finished.  I don't know how far ahead you could get,
> but that might require significantly more buffers.
>
> It does seem like there should be a way to tell it that you
> are done with one.

He can, but that will not help him.

A typical application where you wait on multiple ECB's is TSO where you
do not know which ECB will fire next (it depends upon which user presses
an attention or interrupt key on his terminal), but when an ECB fires you
process the interrupt for that ECB and then return to your multiple wait.

Sven

"Gilbert Saint-Flour" <usenet5678@yahoo.com> wrote in message
news:pan.2004.01.14.20.44.53.29119@yahoo.com...
> On Wed, 14 Jan 2004 18:17:37 +0100, Sven Pran wrote:
> 
>
> Thanks, but this is not helping, perhaps because I didn't make myself
> perfectly clear: I'm looking for a way to AVOID issuing the extra CHECKs.
>
>  Gilbert Saint-Flour

Any asyncronously Started IO requires completion with for instance CHECK.

A fundamental rule when programming is to avoid testing a program until it
works,
any program should be tested until it works in compliance with the rules for
the
operating system!

Sven