VAST application freeze / hang

We have a VAST application, running in production for about 8-9 years,
originally written in VisualSmalltalk, later converted to VisualAge
Smalltalk and currently is in 6.0.2. It has interfaces to Oracle 9i
(using IBM API for Oracle 8) for authentication, retrieving and storing
data. MQ interface allows the application to store and retrieve
business related data with a third party vendor. Some of the functions
in the app do call Web Services written in Java. It either uses SOAP
protocol or simple xml. Few of the functions even use ActiveX controls
which displays a Java screen within the application. It also has a
connectivity to another internal application application (which
connects to Genesys T server) thru TCP socket on a given port. The
application runs on Win2000 platform and the runtime image size is
about 9MB.

The problem reports that we receive from our production sites is that
the application freezes after it has run for few hours, say 4 or 5
hours. It is not consistant either. They also said that it has been
happening for a year and half, but they would just reboot the system
and work fine for rest of the day. When we analyzed, we realized that
image size is growing upto 70 or 80MB over the time, but would never
come back. After a little bit of battle, we found out that there are
lot of instances of modal window hanging around the system. We cleaned
it up, for most part of it, if not completely. With this change, image
size dont grow more than 40MB now, but the freeze still occurs.

We have been in touch with IBM, and they are helping, but we couldn't
identify the cause of the freeze yet. The freeze could be either one or
all of the following:
1) The application hangs - Can't click on a menu, a button, etc.
2) The application hangs, but also the OS - nothing can be done on the
PC
3) The application hangs, OS hangs, and it sometimes kill other apps,
such as MS Outlook, IE explorer.

At this point, I am unsure if it is our application that is causing the
issue, or is it any other within the desktop, but the issue is focused
on us, because our application is the one they are using as front end
to serve the customers.

I have surfed through lot of discussion threads in this newsgroup, and
there were some interesting stuffs that were written around 1998. They
all talked about memory increase, dangling references & garbage
collection. While I am going back to the application to see if we can
find out more, if anyone has any ideas that can be shared with me, it
will be highly appreciated. Thanks.

-- Mani

Mani,

Have you run this application under the debug/checked build of the version
of Windows yet?

For example if the application fails on Windows XP, install the
debug/checked build of Windows XP and run your application under this. The
debug/checked build of Windows is not for retail use (since it is slower),
but it reports problems with unreleased handles, memory allocation errors,
bad memory usage, stack problems and more.

We regularly run Smalltalk MT development and runtime applications under the
debug/checked build to track any issues.

If you need more information, contact me.

Dave Anderson
DAnderson@Genify.com

quote:
Originally posted by Mani Daks 
We have a VAST application, running in production for about 8-9 years,
originally written in VisualSmalltalk, later converted to VisualAge
Smalltalk and currently is in 6.0.2. It has interfaces to Oracle 9i
(using IBM API for Oracle 8) for authentication, retrieving and storing
data. MQ interface allows the application to store and retrieve
business related data with a third party vendor. Some of the functions
in the app do call Web Services written in Java. It either uses SOAP
protocol or simple xml. Few of the functions even use ActiveX controls
which displays a Java screen within the application. It also has a
connectivity to another internal application application (which
connects to Genesys T server) thru TCP socket on a given port. The
application runs on Win2000 platform and the runtime image size is
about 9MB.

The problem reports that we receive from our production sites is that
the application freezes after it has run for few hours, say 4 or 5
hours. It is not consistant either. They also said that it has been
happening for a year and half, but they would just reboot the system
and work fine for rest of the day. When we analyzed, we realized that
image size is growing upto 70 or 80MB over the time, but would never
come back. After a little bit of battle, we found out that there are
lot of instances of modal window hanging around the system. We cleaned
it up, for most part of it, if not completely. With this change, image
size dont grow more than 40MB now, but the freeze still occurs.

We have been in touch with IBM, and they are helping, but we couldn't
identify the cause of the freeze yet. The freeze could be either one or
all of the following:
1) The application hangs - Can't click on a menu, a button, etc.
2) The application hangs, but also the OS - nothing can be done on the
PC
3) The application hangs, OS hangs, and it sometimes kill other apps,
such as MS Outlook, IE explorer.

At this point, I am unsure if it is our application that is causing the
issue, or is it any other within the desktop, but the issue is focused
on us, because our application is the one they are using as front end
to serve the customers.

I have surfed through lot of discussion threads in this newsgroup, and
there were some interesting stuffs that were written around 1998. They
all talked about memory increase, dangling references & garbage
collection. While I am going back to the application to see if we can
find out more, if anyone has any ideas that can be shared with me, it
will be highly appreciated. Thanks.

-- Mani 
 

Mani,

it's seems to be a network related problem (which is very common with JVM). 
You can run the following command to collect data to verify:

netstta -a 300 > netstat-a.txt

(This will collect data every 5 minutes). It will indicate a problem if you 
see connection build-up. Or you send me the data if you need furthe help (ji
eming_wang@hotmail.com).

JW

Well, we found the root cause of the issue and it was two folded.

1. Everytime we opened a modal window, the instance of modal window was
left behind, meaning, references to the modal window was there, so it
could never be GCed. We fixed it and it helped us reduce the memory
usage over the period of time.

2. Everytime we open and close a status window (custom class), there is
a window handle (OSHWnd in VisualAge) left out in the image. As the
time passes, the number of window handle instances kept increasing and
at a point it runs out of window handles. In VisualAge, there is a
method OSWidget>>errorNoMoreHandles, which is supposed to notify the
user about no more handle error, but since it already ran out handles,
it couldnt show the error either. When we added a Trascript show on the
method, we found the culprit being running out of handles. Anyway, we
fixed our handle leak and the problem is no more occuring. We informed
IBM about the bug in errorNoMoreHandles method, and they may fix it in
the future release of VAST.

Thanks for your help, Dave.