Home > Archive > Cobol > December 2007 > Speed with Fujitsu Cobol
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Speed with Fujitsu Cobol
|
|
|
| I have a customer that has + or - 40 clients and a server with windows 2000,
HP PROLIANT, dual processor 2,8 MHZ, 2GB RAM to run an application made in
fujitsu powercobol, using indexed files.
The system is very slow.
That configurations recommend to me to improve the situation.
Many thanks
Joe
| |
| Richard 2007-12-17, 6:56 pm |
| On Dec 18, 6:39 am, "Euro" <e...@euromercante.pt> wrote:
> I have a customer that has + or - 40 clients and a server with windows 2000,
> HP PROLIANT, dual processor 2,8 MHZ, 2GB RAM to run an application made in
> fujitsu powercobol, using indexed files.
> The system is very slow.
> That configurations recommend to me to improve the situation.
One of the problems of using indexed files on a system like this is
that the client is not just 'reading a record', it is searching the
index and transferring multiple blocks of data across the network so
that it can follow all the index levels and _then_ it reads the
record.
You need to start by analysing the problem. Measure the network
traffic, monitor the disk activity. Check the level of record locks
and collisions.
It may be, for example, that the clients are spending significant
amounts of time waiting for records locked by another client, or it
may be that the network is overloaded, or that the disk is
continuously active.
Some of these may be fixable (eg upgrade to gigabit network, or change
to SATAII or spread the load across more drives), while other problems
will need rewriting the application.
For example if there are a significant number of locked record clashes
do you simply redo the read immediately (and thus overload the network
and disk) or do you 'sleep' for a short time and then retry.
If, eg, you keep filling window lists with all records on each
interaction you may be better to load it once and hold it in an array
in the client to refill the list from the next time.
Performance is a function of application design and not just of
hardware configuration.
| |
| Pete Dashwood 2007-12-17, 6:56 pm |
|
"Richard" <riplin@azonic.co.nz> wrote in message
news:4f6f0113-9318-47ef-b723-1d56b06db0a9@d21g2000prf.googlegroups.com...
> On Dec 18, 6:39 am, "Euro" <e...@euromercante.pt> wrote:
>
> One of the problems of using indexed files on a system like this is
> that the client is not just 'reading a record', it is searching the
> index and transferring multiple blocks of data across the network so
> that it can follow all the index levels and _then_ it reads the
> record.
>
> You need to start by analysing the problem. Measure the network
> traffic, monitor the disk activity. Check the level of record locks
> and collisions.
>
> It may be, for example, that the clients are spending significant
> amounts of time waiting for records locked by another client, or it
> may be that the network is overloaded, or that the disk is
> continuously active.
>
> Some of these may be fixable (eg upgrade to gigabit network, or change
> to SATAII or spread the load across more drives), while other problems
> will need rewriting the application.
>
> For example if there are a significant number of locked record clashes
> do you simply redo the read immediately (and thus overload the network
> and disk) or do you 'sleep' for a short time and then retry.
>
> If, eg, you keep filling window lists with all records on each
> interaction you may be better to load it once and hold it in an array
> in the client to refill the list from the next time.
>
> Performance is a function of application design and not just of
> hardware configuration.
>
Excellent response, Richard. Mine would have been along the same lines. The
problem is not necessarily PowerCOBOL, it is in all the factors you mention.
Pete.
--
"I used to write COBOL...now I can do anything."
| |
| HeyBub 2007-12-17, 6:56 pm |
| Richard wrote:
> On Dec 18, 6:39 am, "Euro" <e...@euromercante.pt> wrote:
>
> One of the problems of using indexed files on a system like this is
> that the client is not just 'reading a record', it is searching the
> index and transferring multiple blocks of data across the network so
> that it can follow all the index levels and _then_ it reads the
> record.
>
> You need to start by analysing the problem. Measure the network
> traffic, monitor the disk activity. Check the level of record locks
> and collisions.
>
> It may be, for example, that the clients are spending significant
> amounts of time waiting for records locked by another client, or it
> may be that the network is overloaded, or that the disk is
> continuously active.
>
> Some of these may be fixable (eg upgrade to gigabit network, or change
> to SATAII or spread the load across more drives), while other problems
> will need rewriting the application.
>
> For example if there are a significant number of locked record clashes
> do you simply redo the read immediately (and thus overload the network
> and disk) or do you 'sleep' for a short time and then retry.
>
> If, eg, you keep filling window lists with all records on each
> interaction you may be better to load it once and hold it in an array
> in the client to refill the list from the next time.
>
> Performance is a function of application design and not just of
> hardware configuration.
Agreed. the PowerCOBOL indexed file system is very efficient (i.e., fast),
but it can't over-ride the programmer's decision on file and record locking.
At the very least, the programs should be locking RECORDS, not FILES.
If the file I-O is sequestered in a subroutine, it's a simple matter to add
a few lines to log how often a lock occurs and the time it takes to resolve
collisions and locks.
The Realia ISAM system kept its own logs which could be accessed via a
stand-alone utility.
Another thought: It is possible, I guess, that the file is so hoplessly
convoluted that a reorganization is appropriate. Fujitsu has a utility
program for that purpose, or the programmer can write a reorg back-pocket
program for infrequent use.
| |
| Robert 2007-12-17, 9:56 pm |
| On Mon, 17 Dec 2007 17:39:30 -0000, "Euro" <euo@euromercante.pt> wrote:
>I have a customer that has + or - 40 clients and a server with windows 2000,
>HP PROLIANT, dual processor 2,8 MHZ, 2GB RAM to run an application made in
>fujitsu powercobol, using indexed files.
>The system is very slow.
>That configurations recommend to me to improve the situation.
Are programs holding locks for a long time i.e. while the user ponders what to do?
The initial read should be without lock. When the program really needs to do an update,
it should do a second read with lock, check a timestamp to detect collision, quickly do a
rewrite to release the lock.
Some collisions can be handled by the application. For example, when starting to enter an
order there are 1,000 widgets available. When saving the order, there are 998 available.
That's not a collision. Just subtract 1 from the 998.
Make sure programs are not opening files in exclusive mode (OPEN WITH LOCK or LOCK MODE
EXCLUSIVE) for each access.
| |
|
| Thanks a lot for the suggestions, however the same application run in a
system with the Windows 2003 Server has a completely different reply.
This is that I do not obtain a explain.
I do not use OPEN WITH LOCK or LOCK MODE
EXCLUSIVE) will be each access, but yes i use READ WITH LOCK and UNLOCK xxxx
Thanks one more time
Joe
"Richard" <riplin@azonic.co.nz> escreveu na mensagem
news:4f6f0113-9318-47ef-b723-1d56b06db0a9@d21g2000prf.googlegroups.com...
> On Dec 18, 6:39 am, "Euro" <e...@euromercante.pt> wrote:
>
> One of the problems of using indexed files on a system like this is
> that the client is not just 'reading a record', it is searching the
> index and transferring multiple blocks of data across the network so
> that it can follow all the index levels and _then_ it reads the
> record.
>
> You need to start by analysing the problem. Measure the network
> traffic, monitor the disk activity. Check the level of record locks
> and collisions.
>
> It may be, for example, that the clients are spending significant
> amounts of time waiting for records locked by another client, or it
> may be that the network is overloaded, or that the disk is
> continuously active.
>
> Some of these may be fixable (eg upgrade to gigabit network, or change
> to SATAII or spread the load across more drives), while other problems
> will need rewriting the application.
>
> For example if there are a significant number of locked record clashes
> do you simply redo the read immediately (and thus overload the network
> and disk) or do you 'sleep' for a short time and then retry.
>
> If, eg, you keep filling window lists with all records on each
> interaction you may be better to load it once and hold it in an array
> in the client to refill the list from the next time.
>
> Performance is a function of application design and not just of
> hardware configuration.
>
| |
|
| or...
If I were an 'IT Consultant' who had never known how to design and
code a system I could say...
"You need a bigger server". Pocket the consultancy fee and walk.
Unfortunately, I'm sure we've all seen it done.
When you say 40 clients, are these clients actually running the
application or are they using Terminal Services/Remote Desktop ?
| |
|
| Normally they are all use the application that is in a "server" (folder
sharing ).
Very rare remote desktop is use or terminal services,
only for tele-maintenance.
Regards
"razor" <iruddock@blueyonder.co.uk> escreveu na mensagem
news:ebceb43a-a6cc-4051-9bf7-a566286532c8@r29g2000hsg.googlegroups.com...
> or...
>
> If I were an 'IT Consultant' who had never known how to design and
> code a system I could say...
>
> "You need a bigger server". Pocket the consultancy fee and walk.
>
> Unfortunately, I'm sure we've all seen it done.
>
> When you say 40 clients, are these clients actually running the
> application or are they using Terminal Services/Remote Desktop ?
>
| |
| HeyBub 2007-12-18, 7:55 am |
| Euro wrote:
> Thanks a lot for the suggestions, however the same application run in
> a system with the Windows 2003 Server has a completely different
> reply. This is that I do not obtain a explain.
> I do not use OPEN WITH LOCK or LOCK MODE
> EXCLUSIVE) will be each access, but yes i use READ WITH LOCK and
> UNLOCK xxxx
> Thanks one more time
> Joe
Windows 2000 is limited to TEN concurrent connections.
"By default, Windows 2000 Professional is limited to no more than 10
concurrent connections. Anyone trying to connect after the maximum number of
connections is reached receives an error message, and the connection is
refused."
There you are.
| |
|
| Excuse me, if I did not inform, but I am speak of Windows 2000 server !!!
"HeyBub" <heybub@gmail.com> escreveu na mensagem
news:13mfd74sqne34a4@corp.supernews.com...
> Euro wrote:
>
> Windows 2000 is limited to TEN concurrent connections.
>
> "By default, Windows 2000 Professional is limited to no more than 10
> concurrent connections. Anyone trying to connect after the maximum number
> of connections is reached receives an error message, and the connection is
> refused."
>
> There you are.
>
| |
| Michael Mattias 2007-12-18, 6:55 pm |
| I'd bet at least ninety percent of run-time performance is determined before
you write the first line of source code.. by your application design.
That extra hour or so you spend *thinking* about file access, network
traffic, indexes, record/table layouts, overall program flow, etc pays for
itself.
Sure, it pays for itself but a few seconds or milliseconds at a time, but
over hundreds of thousands or millions of iterations it *does* pay for
itself.
MCM
| |
|
| In article <tJQ9j.121$El5.23@newssvr22.news.prodigy.net>,
Michael Mattias <mmattias@talsystems.com> wrote:
>I'd bet at least ninety percent of run-time performance is determined before
>you write the first line of source code.. by your application design.
Hmmmmm... given how I have seen systems age, Mr Mattias, I'm not quite
sure where 'application design' properly fits in. For example:
Away back when... Mr Able got budget approved to put together... something
that gives annual totals by location.
Then, later... Ms Baker wants what Mr Able has... but she wants it broken
down by department within location.
Later... Mr Charlie wants what Ms Baker and Mr Able have... but he wants
it broken down by manager within department within location and totalled
by quarter.
And again... Ms Dexter wants what Mr Charlie and Ms Baker and Mr Able
have... but she wants it updatable by manager within department within
location on a nightly basis.
Then... Mr Echo wants (see above)... but updatable and queryable across
each way and displayed in realtime.
The application design for what Mr Echo wants would be, from what I have
seen of such things, rather different from what Mr Able started with...
but in my experience what frequently happens is that the system already in
place is turned over to someone with a dictum that starts with 'All ya
gotta do is...'
>That extra hour or so you spend *thinking* about file access, network
>traffic, indexes, record/table layouts, overall program flow, etc pays for
>itself.
If one searches this newsgroup for the phrase "stepped out for a smoke"
one might find an anecdote about how 'thinking time' has been valued by a
Project Leader.
>Sure, it pays for itself but a few seconds or milliseconds at a time, but
>over hundreds of thousands or millions of iterations it *does* pay for
>itself.
Now that just Doesn't Make Sense, Mr Mattias... for what reason would a
Manager, evaluated on a quartely basis, spend extra money in order to
benefit those who Come After? Let *Them* pay for optimising... get that
code into Prod, fast!
DD
| |
| Richard 2007-12-18, 6:55 pm |
| On Dec 18, 11:37 pm, razor <irudd...@blueyonder.co.uk> wrote:
> or...
>
> If I were an 'IT Consultant' who had never known how to design and
> code a system I could say...
>
> "You need a bigger server". Pocket the consultancy fee and walk.
>
> Unfortunately, I'm sure we've all seen it done.
If you were an 'IT Consultant' you would say "You need an XYZ server",
pocket the consultancy, pocket the kickback from XYZ, and charge for
installation, training and support at consultancy rates while sending
in a part-time student to do it.
> When you say 40 clients, are these clients actually running the
> application or are they using Terminal Services/Remote Desktop ?
| |
| Richard 2007-12-19, 3:55 am |
| On Dec 18, 11:02 pm, "Euro" <e...@euromercante.pt> wrote:
> Thanks a lot for the suggestions, however the same application run in a
> system with the Windows 2003 Server has a completely different reply.
> This is that I do not obtain a explain.
> I do not use OPEN WITH LOCK or LOCK MODE
> EXCLUSIVE) will be each access, but yes i use READ WITH LOCK and UNLOCK xxxx
>
Perhaps the Windows 2003 server had gigabit LAN and the 2000 has
10/100 with 10Mbit hubs.
Perhaps the W2003 has SATA2 drives with RAID 0 (striped) for speed and
the 2000 has a single IDE mode 4.
Perhaps the W2003 has several Gbytes of RAM while the W2000 has only
512Mbyte and is thrashing its swap file.
Perhaps the W2000 is part of a botnet and is spending 75% of its time
sending out spam emails.
Only when you start collecting numbers will you be able to work out
what to do.
| |
| HeyBub 2007-12-19, 7:55 am |
| Euro wrote:[color=darkred]
> Excuse me, if I did not inform, but I am speak of Windows 2000 server
> !!!
>
> "HeyBub" <heybub@gmail.com> escreveu na mensagem
> news:13mfd74sqne34a4@corp.supernews.com...
Okay, so how many CALs (Client Access Licenses) is the 2000 Server
authorized?
| |
| donald tees 2007-12-20, 9:56 pm |
| HeyBub wrote:
> Richard wrote:
>
> Agreed. the PowerCOBOL indexed file system is very efficient (i.e., fast),
> but it can't over-ride the programmer's decision on file and record locking.
> At the very least, the programs should be locking RECORDS, not FILES.
>
> If the file I-O is sequestered in a subroutine, it's a simple matter to add
> a few lines to log how often a lock occurs and the time it takes to resolve
> collisions and locks.
>
> The Realia ISAM system kept its own logs which could be accessed via a
> stand-alone utility.
>
> Another thought: It is possible, I guess, that the file is so hoplessly
> convoluted that a reorganization is appropriate. Fujitsu has a utility
> program for that purpose, or the programmer can write a reorg back-pocket
> program for infrequent use.
>
>
The point on file/record locking is a good one. While the Fujitsu Isam
is very fast, I have found that the MS open/close routines are very
slow. Using file level lockout on a record by record basis will slow a
system to a crawl, and is highly dependent on the number of files in the
directory. That is one of the few things that can be checked without
looking at actual code.
Donald
| |
| Richard 2007-12-20, 9:56 pm |
| On Dec 21, 3:36 am, donald tees <donaldt...@execulink.com> wrote:
> HeyBub wrote:
>
>
>
>
>
>
>
>
>
>
>
> The point on file/record locking is a good one. While the Fujitsu Isam
> is very fast, I have found that the MS open/close routines are very
> slow. Using file level lockout on a record by record basis will slow a
> system to a crawl, and is highly dependent on the number of files in the
> directory. That is one of the few things that can be checked without
> looking at actual code.
How would you know that it used "file level lockout on a record by
record basis" without looking at the code.
In fact Euro said he doesn't do that. He uses READ WITH LOCK (ie
single record lock).
| |
| donald tees 2007-12-20, 9:56 pm |
| Richard wrote:
> On Dec 21, 3:36 am, donald tees <donaldt...@execulink.com> wrote:
>
> How would you know that it used "file level lockout on a record by
> record basis" without looking at the code.
>
True enough ... though I was saying that the number of files in the
directory can be checked without referring to the code. That does affect
open/close timing significantly. Vagueness on my part.
> In fact Euro said he doesn't do that. He uses READ WITH LOCK (ie
> single record lock).
Yes, I read that a couple posts further on.
Donald
| |
|
| On Dec 17, 11:39 am, "Euro" <e...@euromercante.pt> wrote:
> I have a customer that has + or - 40 clients and a server with windows 2000,
> HP PROLIANT, dual processor 2,8 MHZ, 2GB RAM to run an application made in
> fujitsu powercobol, using indexed files.
> The system is very slow.
> That configurations recommend to me to improve the situation.
>
> Many thanks
> Joe
Joe - Which version of the Fujitsu COBOL runtime is in use?
There was a pretty serious performance issue when opening files using
versions prior to 6.1a - I think 6.1a had the patch available.
Versions 7-9 are fine though.
| |
|
| I use version 7, with the last updates.
Regards,
Joe
"Thane" <thaneh@softwaresimple.com> escreveu na mensagem
news:6644b901-39c3-474c-852c-b90abb2b05b0@18g2000hsf.googlegroups.com...
> On Dec 17, 11:39 am, "Euro" <e...@euromercante.pt> wrote:
>
> Joe - Which version of the Fujitsu COBOL runtime is in use?
>
> There was a pretty serious performance issue when opening files using
> versions prior to 6.1a - I think 6.1a had the patch available.
> Versions 7-9 are fine though.
|
|
|
|
|