Clipper - ASORT Very Slow sometimes !

Author

ASORT Very Slow sometimes !

Mel Smith

2006-04-25, 9:55 pm

Hi,

I use ASORT in a Tbrowse routine to re-sort an array of approx 2K string
elements. This is activated by a Key Press (F4).

Each of the array elements is approx 60 chars long.

My sort is like the following:

AMyArray := ASORT(AMyArray,{|X,Y| UPPER(SUBSTR(X,5,4) <
UPPER(SUBSTR(Y,5,4 ) } )

Sorting this array the First Time, takes about 12 seconds on my IBM Win
98SE machine (600 MHz) -- which annoys my user.

The user may wish to re-sort the array using another slightly different
part of the string (but it is similar to the above ASORT(), and so presses
the <F4> key again. This second re-sort takes less than a second.

So, the first re-sort using ASORT takes about 12 seconds, but *all
succeeding* 'sorts' take less than one second, *including the one above *
that took so long in the first instance ?? !!

Question: Why do succeeding 'sorts' take less time to complete than the
first one ??

TIA for any further info.

-Mel Smith

Ross McKenzie

2006-04-26, 3:55 am

On Tue, 25 Apr 2006 21:53:37 -0600, "Mel Smith" <medsyntel@aol.com>
wrote:

>Hi,
>
> I use ASORT in a Tbrowse routine to re-sort an array of approx 2K string
>elements. This is activated by a Key Press (F4).
>
> Each of the array elements is approx 60 chars long.
>
> My sort is like the following:
>
> AMyArray := ASORT(AMyArray,{|X,Y| UPPER(SUBSTR(X,5,4) <
>UPPER(SUBSTR(Y,5,4 ) } )
>
> Sorting this array the First Time, takes about 12 seconds on my IBM Win
>98SE machine (600 MHz) -- which annoys my user.
>
> The user may wish to re-sort the array using another slightly different
>part of the string (but it is similar to the above ASORT(), and so presses
>the <F4> key again. This second re-sort takes less than a second.
>
> So, the first re-sort using ASORT takes about 12 seconds, but *all
>succeeding* 'sorts' take less than one second, *including the one above *
>that took so long in the first instance ?? !!
>
>Question: Why do succeeding 'sorts' take less time to complete than the
>first one ??
>
>
>TIA for any further info.
>
>-Mel Smith
>
>

Hi Mel,

Subsequent sorts are faster because the earlier data is buffered in
memory. Perhaps you could do an initial "sort" while your flashscreen
is displaying on program startup.

Regards,

Ross McKenzie
ValuSoft
Melbourne Australia

valusoft AT optusnet DOT com DOT au

Mel Smith

2006-04-26, 6:55 pm

Ross said:

> Subsequent sorts are faster because the earlier data is buffered in
> memory. Perhaps you could do an initial "sort" while your flashscreen
> is displaying on program startup.

Ross,

Thanks for the info/guidance above.

The array is already sorted before my Tbrowse is entered (i.e., I create
a sorted array of these 2K strings just prior to entering the TBrowse app).
My user wants to look at the array (of Job Numbers and Customer Codes) in
order by either of these 'keys' (at the flick of the <F4> key)).

So, I start out by providing the array sorted in the most commonly
needed order (i.e., by Cust Code), Then pressing the <F4> Key 'flicks' the
sorting order. But the 1st 'flick' takes approx 12 seconds. The user (of
course) thinks that he hasn't pressed the <F4> key hard enough, so he hits
it again ! (and again). Oh me ... (time for the CLEAR TYPEAHEAD command
here :))) )

Anyway, since I have approx three major arrays hanging around in memory
at the time of starting up the TBrowse, is it possible that 'garbage
collection' also might be taking up the time ???

TIA,

-Mel Smith

Stephen Quinn

2006-04-26, 6:55 pm

Mel

Try it with 2 copies of the array and sort that one in the other order, use 2
TBrowses and switch between them when the user clicks the F4
aArray1 := ASort( aArray1, {|| /*CustCode Order*/} )
aArray2 := AClone( aArray1 )
aArray2 := ASort( aArray2, {|| /*JobNo Order*/} )

oTb1 := TBrowse( aArray1 )
oTB1:Cargo := 'CUSTCODE'
oTb2 := TBrowse( aArray2 )
oTB2:Cargo := 'JOBNO'
// Key handler code
nKey = K_F4
dispbegin()
// Probably save index numbers to keep the hilite on the same row,
etc...
IF oTB:Cargo = 'JOBNO'
// switch to Custcode
oTB := oTB1
ELSE
// switch to JobNo
oTB := oTB2
ENDIF
dispend()

The next question to ask yourself is does he really need to see 2000 records in
an array<g>, why not in a indexed (temp) DBF

HTH
Steve

Mel Smith

2006-04-26, 6:55 pm

Stephen,

Thanks for the good hints !!

Will mull on them and let you know what transpires.

btw, I was thinking of two arrays also, but this latest 'flicking' back
and forth is a late development -- before this there was only one sorting
sequence and comprised only a few hundred jobs. Now my customer's business
is growing fast, and I'm trying to handle all his new requests (including
this new 'double-sorting'), and I'm going mad !!!!

(btw, its a structural steel company that's providing major construction
to Alberta's booming oil sands projects.)

Thanks again.

-Mel Smith

"Stephen Quinn" <steveq@NOSPAMintegritynet.com.au> wrote in message
news:IfM3g.16644$vy1.16582@news-server.bigpond.net.au...
> Mel
>
> Try it with 2 copies of the array and sort that one in the other order,
use 2
> TBrowses and switch between them when the user clicks the F4
> aArray1 := ASort( aArray1, {|| /*CustCode Order*/} )
> aArray2 := AClone( aArray1 )
> aArray2 := ASort( aArray2, {|| /*JobNo Order*/} )
>
> oTb1 := TBrowse( aArray1 )
> oTB1:Cargo := 'CUSTCODE'
> oTb2 := TBrowse( aArray2 )
> oTB2:Cargo := 'JOBNO'
> // Key handler code
> nKey = K_F4
> dispbegin()
> // Probably save index numbers to keep the hilite on the same row,
> etc...
> IF oTB:Cargo = 'JOBNO'
> // switch to Custcode
> oTB := oTB1
> ELSE
> // switch to JobNo
> oTB := oTB2
> ENDIF
> dispend()
>
> The next question to ask yourself is does he really need to see 2000
records in
> an array<g>, why not in a indexed (temp) DBF
>
> HTH
> Steve
>
>

Mike

2006-04-27, 6:55 pm

I can't imagine why 12 seconds. Maybe Blinker is swapping the code for
the asort into memory and it is a big chunk. Once in memory it is
quick. I think it was in the extend.lib. Can you force that into
memory?

Mel Smith

2006-04-27, 6:55 pm

Mike said:

> I can't imagine why 12 seconds. Maybe Blinker is swapping the code for
> the asort into memory and it is a big chunk. Once in memory it is
> quick. I think it was in the extend.lib. Can you force that into
> memory?
>

Mike here is the actual code fragment:

This code is in the middle of my oTB browser, and when the user hits
<F4>, I go (in the beginning) to the 'ELSE' block. This is where I hit the
12-second delay (and where you see my ' wait a few moments' message

btw, as tthe uiser inputs the job number or customer code, I
automatically move to the appropriate line of the 200-line block

CASE NLASTKEY = K_F4
IF LSELBYJOB // THEN 'FLICK' TO FALSE
LSELBYJOB := .F.
// NOW HAVE TO RE-SORT THE JOBARY
CLEAR TYPEAHEAD
aOPTIONS := ASORT(aOPTIONS,,,{|X,Y| UPPER(SUBSTR(X,12,6)) <
UPPER(SUBSTR(Y,12,6))})
@ 21,39 SAY "Job No" COLOR "GR+/G"
LJOBORDCH := 2
ELSE
LSELBYJOB := .T.
CLEAR TYPEAHEAD
@ 24,00 SAY "Please Wait a Few Moments !!" COLOR CWARN
// start of the delay here (but *only* on the 1st pressing of the
<F4> key)
aOPTIONS := ASORT(aOPTIONS,,,{|X,Y| UPPER(SUBSTR(X,5,4)) <
UPPER(SUBSTR(Y,5,4))})
@ 24,00
// end of 12-second delay here

@ 21,39 SAY "CustCd" COLOR "GR+/G"
LJOBORDCH := 1
ENDIF

oTB:CONFIGURE()
SETJOBSEL(LJOBORDCH)

LJOBSCAN := LCODSCAN := LDESSCAN := ""

loop // to start of browser input loop and for re-display of my
array

Very Puzzling !

-Mel Smith

Mike

2006-04-28, 6:55 pm

Nothing wrong with that code.

Is the array full (Declare 2048 and load 1800?)? If not, maybe setting
the empties to blanks instead of null.

Mike

Mel Smith

2006-04-28, 6:55 pm

Mike said:

> Is the array full (Declare 2048 and load 1800?)? If not, maybe setting
> the empties to blanks instead of null.

Mike,

Yes, the array is always full when I enter the browse routine (which is
copied from the SuperLib sachoice() function (within s_achoi.prg) , and
highly modified by me for incremental searching and toggling thru two sort
orders, plus other needed stuff.

My user can select among Active Jobs, Dormant Jobs, or Both. Picking
'Both' results in a 15-second primary delay, picking Dormant results in
approx 8-second, and Active produces about 12 seconds.

After he selects the type of jobs to be presented, I build a new
Aoptions array, and pass it as an argument to the browser (pickjob()). I
have tried declaring the array in the 'caller' routine as both Private and
Local. Absolutely no difference with either way.

In my previous post, I typo'ed 200 lines instead of 2000 lines in the
possible array size.

In more tests yesterday at my office, I was surprised on one occasion to
have *no* delay at all, but in re-trying it the delay returned :(((

I guess I'll just forget about my wasting time (and also yours) with
this problem, and tell my user to "Deal with it !"

(but its still puzzling. Maybe we're back to garbage collection again ???)

Thanks again.

-Mel Smith

E. Fridman

2006-04-28, 6:55 pm

Mel,

Is your application linked in protected mode?

Mel Smith

2006-04-29, 6:55 pm

E.Fridman said:

> Is your application linked in protected mode?

Yes, it is.

btw, I sent him my blinker .lnk file to peruse. I assume he'll make a
comment about it early next w

.

-Mel Smith

sali

2006-04-29, 6:55 pm

"Mel Smith" <medsyntel@aol.com> wrote in message
news:4bf55vF10tg59U1@individual.net...
> Mike said:
>
>
>
> Mike,

>
> (but its still puzzling. Maybe we're back to garbage collection again
???)
>
> Thanks again.
>
> -Mel Smith
>

have you tried to modify sort option, instead of recalculating sort key each
time, to use allready prepared keys in additional rows, like:
key1:
ASORT(aOPTIONS,,,{|X,Y|UPPER(SUBSTR(X,12
,6))<UPPER(SUBSTR(Y,12,6))})
key2:
ASORT(aOPTIONS,,,{|X,Y|UPPER(SUBSTR(X,5,
4))<UPPER(SUBSTR(Y,5,4))})

so, instead 1-dimensional array aoptions[n], to use 2-dimensioanl
aoptions[n][3], where aoptions[i][1] is original row, aoptions[i][2] is key1
(f.e. upper(substr(aoptions[i][1],12,6))), and aoptions[i][3] is key2 (f.e.
upper(substr(aoptions[i][1],5,4))), so to have:
key1:
asort(aoptions,,,{|x,y|x[2]<y[2]})
key2:
asort(aoptions,,,{|x,y|x[3]<y[3]})

also, what clipper version 5.2 or 5.3?
think that clipper 5.3 is faster on string manipulation, since able to move
just *pointer* to string body, while 5.2 performs full *memcopy* od string
body. when sorting arrays instead of string [since array is allways pointer
operated], this memory copy in 5.2 may be avoided.

Mel Smith

2006-04-30, 6:55 pm

Sali said:

> have you tried to modify sort option, instead of recalculating sort key
each
> time, to use allready prepared keys in additional rows, like:
> key1:
> ASORT(aOPTIONS,,,{|X,Y|UPPER(SUBSTR(X,12
,6))<UPPER(SUBSTR(Y,12,6))})
> key2:
> ASORT(aOPTIONS,,,{|X,Y|UPPER(SUBSTR(X,5,
4))<UPPER(SUBSTR(Y,5,4))})
>
> so, instead 1-dimensional array aoptions[n], to use 2-dimensioanl
> aoptions[n][3], where aoptions[i][1] is original row, aoptions[i][2] is
key1
> (f.e. upper(substr(aoptions[i][1],12,6))), and aoptions[i][3] is key2
(f.e.
> upper(substr(aoptions[i][1],5,4))), so to have:
> key1:
> asort(aoptions,,,{|x,y|x[2]<y[2]})
> key2:
> asort(aoptions,,,{|x,y|x[3]<y[3]})

No, I haven't thought of doing this extra multi-dimensional array work
for this problem. It brings in a bunch of whole new problems and re-coding
for my incremental search code in the same app. But, I may have to do it.

I guess my real need is to understand *why* it happens, and why it
sometimes *does not* happen.

>
> also, what clipper version 5.2 or 5.3?
> think that clipper 5.3 is faster on string manipulation, since able to
move
> just *pointer* to string body, while 5.2 performs full *memcopy* od string
> body. when sorting arrays instead of string [since array is allways
pointer
> operated], this memory copy in 5.2 may be avoided.
>

I use Clipper 5.2e for this application.

Thanks for your comments and suggestions.

-Mel Smith

Mike

2006-04-30, 6:55 pm

Mel,

I agree that it looks like garbage collection. For testing, can you
try adding an inkey() before the asort()?

Mike

Sponsored Links