Home > Archive > Smalltalk > November 2006 > Modifying a running image
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Modifying a running image
|
|
| Thomas Gagne 2006-11-21, 8:04 am |
| I remember during an address at the 2002 Smalltalk Conference one
presenter lamented too few (any?) Smalltalk shops actually patched
running images. The vision of fixing a running system (brain surgery on
a live patient) hadn't materialized the way it was envisioned by
Smalltalk's creators or is possible.
I was reminded of that while working on the text for an article I'm
writing on a simple way to avoid the object-relational impedance
mismatch using something we call Transaction Oriented Processing (if you
want a draft let me know--I need comments). We commonly load corrected
or enhanced stored procedures into our production database but are
reluctant to do the same for Smalltalk applications. Why? We know we
can do it, but the problem is saving the running image. When a
Smalltalk image is saved /everything/ is saved. It's entire state,
including open files, database connections, sockets, etc. are all saved,
and when a saved image is reanimated all those objects are reanimated as
well--but we don't want them.
Practically speaking, we could probably open a running image, save code,
then parcel-out the fixed classes. That would require our production
images to be configured to read-in parcels when it starts up. While
technically feasibly, it is impractical. We have a distributed
application with multiple Smalltalk images running concurrently and each
one would have to be re-initialized.
It's just not a clean solution.
I was wondering what would happen if a running image knew two things
about itself--which objects belonged to the image and which were
transient. If we could fix or enhance an image in-flight and save the
image without all the transient objects that might make in-flight
modifications more practical, and the saved images more
deterministic--we'd know what's inside them because they wouldn't be
polluted with objects an image doesn't need to recreate on start-up.
Just some random thoughts. I'm curious if anyone else has had similar
thoughts or has come up with solutions.
--
Visit <http://blogs.instreamfinancial.com/anything.php>
to read my rants on technology and the finance industry.
| |
| jarober 2006-11-21, 7:04 pm |
| I responded here:
http://www.cincomsmalltalk.com/blog...ntry=3341554176
On Nov 21, 9:03 am, Thomas Gagne <tga...@wide-open-west.com> wrote:
> I remember during an address at the 2002 Smalltalk Conference one
> presenter lamented too few (any?) Smalltalk shops actually patched
> running images. The vision of fixing a running system (brain surgery on
> a live patient) hadn't materialized the way it was envisioned by
> Smalltalk's creators or is possible.
>
> I was reminded of that while working on the text for an article I'm
> writing on a simple way to avoid the object-relational impedance
> mismatch using something we call Transaction Oriented Processing (if you
> want a draft let me know--I need comments). We commonly load corrected
> or enhanced stored procedures into our production database but are
> reluctant to do the same for Smalltalk applications. Why? We know we
> can do it, but the problem is saving the running image. When a
> Smalltalk image is saved /everything/ is saved. It's entire state,
> including open files, database connections, sockets, etc. are all saved,
> and when a saved image is reanimated all those objects are reanimated as
> well--but we don't want them.
>
> Practically speaking, we could probably open a running image, save code,
> then parcel-out the fixed classes. That would require our production
> images to be configured to read-in parcels when it starts up. While
> technically feasibly, it is impractical. We have a distributed
> application with multiple Smalltalk images running concurrently and each
> one would have to be re-initialized.
>
> It's just not a clean solution.
>
> I was wondering what would happen if a running image knew two things
> about itself--which objects belonged to the image and which were
> transient. If we could fix or enhance an image in-flight and save the
> image without all the transient objects that might make in-flight
> modifications more practical, and the saved images more
> deterministic--we'd know what's inside them because they wouldn't be
> polluted with objects an image doesn't need to recreate on start-up.
>
> Just some random thoughts. I'm curious if anyone else has had similar
> thoughts or has come up with solutions.
>
> --
> Visit <http://blogs.instreamfinancial.com/anything.php>
> to read my rants on technology and the finance industry.
| |
|
|
| Chris Uppal 2006-11-21, 7:04 pm |
| Thomas Gagne wrote:
> I was wondering what would happen if a running image knew two things
> about itself--which objects belonged to the image and which were
> transient. If we could fix or enhance an image in-flight and save the
> image without all the transient objects that might make in-flight
> modifications more practical, and the saved images more
> deterministic--we'd know what's inside them because they wouldn't be
> polluted with objects an image doesn't need to recreate on start-up.
I've occasionally wanted something similar. It could be implemented as a
separate program which massaged the saved image using a GC style algorithm
which knew about transient objects.
Still, I find that in practise just doing the cleanup on image start works
well. Not only does that have the profound advantage that it's actually
possible ;-) it also means that I don't have to do the slow processing at image
save (frequent) but only at image start (infrequent).
Takes a bit of extra code, of course, but nothing too scary (that's with
Dolphin, I assume VW and other Smalltalks have comparable features).
A more functional advantage of cleanup-on-restart is that you could, if you
wished, code it so that the clean restart only happened if the system detected
that it was being restored in a production environment. That would facilitate
off-line debugging and post-mortem investigations.
If the don't-save-transients idea /was/ implemented as off-line
post-processing, then that could share these advantages, but it's not clear
that cleanup-before-restart, including the necessary tagging of transient
objects and processes, would be any easier to use than cleanup-on-restart.
-- chris
| |
| Mike Anderson 2006-11-21, 7:04 pm |
|
Thomas Gagne wrote:
> I was wondering what would happen if a running image knew two things
> about itself--which objects belonged to the image and which were
> transient. If we could fix or enhance an image in-flight and save the
> image without all the transient objects that might make in-flight
> modifications more practical, and the saved images more
> deterministic--we'd know what's inside them because they wouldn't be
> polluted with objects an image doesn't need to recreate on start-up.
Doing /something/ with the transients is relatively straightforward, in
fact, isn't it necessary to avoid errors? However, when you say that
the image doesn't need to recreate them, in fact, in many cases the
image /is/ going to replace them. A database app is going to need to
reconnect to the database, for example.
If you're going to replace the object, and by implication, the whole
assembly of objects around it, then why not attempt to keep the
existing one instead? I have experimented with proxies which
transparently recreate the connection with an external resource when
the image is restarted. If a connection can't be reestablished, then an
exception is thrown, which is equivalent to a connection breaking
normally as far as your coding is concerned. The interesting thing
about this is that reestablishing the object is typically a per-object
behaviour, rather than per-class.
| |
| Thomas Gagne 2006-11-22, 8:04 am |
|
Mike Anderson wrote:
> Thomas Gagne wrote:
>
>
> Doing /something/ with the transients is relatively straightforward, in
> fact, isn't it necessary to avoid errors? However, when you say that
> the image doesn't need to recreate them, in fact, in many cases the
> image /is/ going to replace them. A database app is going to need to
> reconnect to the database, for example.
>
But that's what I want to avoid. While the program is running it needs
the transients. But those transient objects were not in the original
image and they aren't needed, and are in-fact unwanted, in a /virgin/ image.
A long running program accumulates objects, but those objects weren't
there when the image started. After repairing or enhancing a running
image I want the ability to both fix the running program and save the
fix for future incarnations.
--
Visit <http://blogs.instreamfinancial.com/anything.php>
to read my rants on technology and the finance industry.
| |
| Eliot Miranda 2006-11-22, 7:04 pm |
| Chris Uppal wrote:
> Thomas Gagne wrote:
>
>
>
> I've occasionally wanted something similar. It could be implemented as a
> separate program which massaged the saved image using a GC style algorithm
> which knew about transient objects.
>
> Still, I find that in practise just doing the cleanup on image start works
> well. Not only does that have the profound advantage that it's actually
> possible ;-) it also means that I don't have to do the slow processing at image
> save (frequent) but only at image start (infrequent).
>
> Takes a bit of extra code, of course, but nothing too scary (that's with
> Dolphin, I assume VW and other Smalltalks have comparable features).
Right. In fact this is how VW does snapshot. By deferring all cleanups
to startp snapshot doesn't involve shutting anything down, which means
snapshot is just that.
On startup the VW image
- invalidates all general instances of CDatum that have pointers into
the C heap
- invalidates all instances of things that are holding onop OS and
Window Manager handles, such as open files, fonts, windows, pixmaps,
cursors, et al
- rehashes all collections if the pointer size as changed (bringing up a
32-bit image converted to a 64-bit image)
The issue here is making sure the invalidations happen early enough in
system startup that none of the obsolete things get used.
The only things the VM does on snapshot are a scavenge to void eden, a
discard of native code and "stabilizing" all contexts (writing all
active contexts to the heap from the stack cache).
On startup the VM (has to)
- swizzles all pointers to the relevant addresses for the new VM
- byte-swaps all relevant objects (e.g. 2-byte strings)
So some start-time processing is unavoidable.
> A more functional advantage of cleanup-on-restart is that you could, if you
> wished, code it so that the clean restart only happened if the system detected
> that it was being restored in a production environment. That would facilitate
> off-line debugging and post-mortem investigations.
Good idea.
>
> If the don't-save-transients idea /was/ implemented as off-line
> post-processing, then that could share these advantages, but it's not clear
> that cleanup-before-restart, including the necessary tagging of transient
> objects and processes, would be any easier to use than cleanup-on-restart.
The VW experience is that cleanup-on-startup works very well.
--
The surest sign that intelligent life exists elsewhere in Calvin &
the universe is that none of it has tried to contact us. Hobbes.
--
Eliot ,,,^..^,,, Smalltalk - scene not herd
| |
| Mike Anderson 2006-11-23, 4:03 am |
|
Thomas Gagne wrote:
> Mike Anderson wrote:
> But that's what I want to avoid. While the program is running it needs
> the transients. But those transient objects were not in the original
> image and they aren't needed, and are in-fact unwanted, in a /virgin/ image.
>
> A long running program accumulates objects, but those objects weren't
> there when the image started. After repairing or enhancing a running
> image I want the ability to both fix the running program and save the
> fix for future incarnations.
I that case I don't understand your original allusion to *production*
databases. If what you mean is to develop a patch on a development
image, and then apply that patch to a production image, why don't you
just do that (it's common nowadays for applications to check for,
download and install patches for themselves)? If your problem is that
you are not confident to snapshot the production image after applying
the patch, then isn't that your real problem ?
|
|
|
|
|