Home > Archive > Smalltalk > May 2005 > Smalltalk - more usable by others than C++?
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Smalltalk - more usable by others than C++?
|
|
| Thomas Gagne 2005-04-26, 4:03 am |
| Recently I was looking at trying to use a C++ API from Smalltalk and ran
into various problems. Having been down similar roads before and not
having the time, inclination, or compelling reason to overcome them I've
decided to back-burner the whole Smalltalk/C++ project for now.
In a recent ACM Queue article it is reported Bjarne Stroustrup says
there are over 3 million C/C++ programmers, and that number is growing.
I personally hope more of them are C programmers because C++'s
mangling isolates it from other languages, making its libraries and
various other artifacts more difficult to reuse by other languages in
closely-bound environments (like shared libraries). Mangled function
names and the dependency on instances makes any OOP more difficult to
interface with since non-OOP languages don't have direct support for the
API semantics and mangled function names make things even less
straightforward.
Moments ago I was reading a posting in comp.lang.lisp about how Lisp has
few or hard-to-find bindings to OpenGL. I started wondering how similar
Lisp and Smalltalk might be.
Now I'm pondering how Smalltalk is no better than C++ in this regard.
Smalltalk's artifacts, like other OOPs, are not readily accessible
through dynamic libraries. In fact, I don't have any experience myself
accessing Smalltalk code inside a shared library since VW doesn't
provide that mechanism (that I'm aware of). I think I've read Dolphin
or MT might provide that but I'm unsure.
In fact, this may be a failing of all OOP languages that each is
incapable of directly reusing the other's objects and that this obstacle
is doing programmers no favors when trying to deliver software to their
customers. Java has overcome this barrier not by finding a way to cross
it, but by rewriting every conceivable API in Java to make those API's
native. Instead of using an available LDAP and wrapping it there's a
Java LDAP library (that's just one of hundreds of examples).
Since OOPs don't seem able (or willing?) to speak directly to each other
we've had to create all kinds of translators between them. SOAP and
CORBA are two that come to mind, but erecting those bridges have a far
higher expense than any of these languages have talking to C functions.
And I suppose that's the beauty of C. Everyone can talk to it (that I
know of). Java (with a little exercise in JNI), Smalltalk, Lisp,
Python, Ruby, Fortran, COBOL, you name it, it can talk to C.
So what makes a good C API usable? I think GNU's Privacy Guard (GPG) is
a decent design that is easily wrapped and interfaced with using an OOP.
The API returns opaque data structures that must be passed to
functions in a manner similar to how file handles or streams are passed
to open, read, write, and close files.
If someone were to hack a mechanism for making Smalltalk code accessible
to other languages from a shared library I think a similar mechanism
could be constructed. After initializing the library (I won't attempt
to describe how that would work) instance methods could be called by
making their first argument a handle to the instance. The function name
would be resolved inside the scope of the instance handle to avoid
collisions. This would resemble Python's (less than aesthetically
pleasing) "self" reference to functions.
Name mangling would still be a problem. Even if we assume all method
parameters are positional there's still the matter of the method
signature itself. In one of my own methods I'm unsure how pleasing the
signature from a method, "defaultTableHeadersForResult: aTupleCollection
columnList: aColumnList on: aStream" would look as a C-ish function.
void defaultTableHeadersForResult:columnList:
on:
Though there are the same amount of characters, it seems awkward outside
its natural habitat, the Smalltalk IDE. Not unmanageable, just awkward.
Forget for the moment C doesn't allow colons inside names.
By now some of you may be thinking I've wasted a bunch of time with this
posting but I can assure its therapeutic (for me). It just one of my
own 12-step program of realizing how much more difficult some things are
now than they were in the 80s. Sometimes it seems programmers have made
their jobs more difficult instead of less so. My Pascal, Fortran, C,
Cobol and ADA could all call each others' libraries on Data General's
AOS/VS. I could move to a new language without abandoning the assets of
the previous one. I could increase the return on old code by migrating
some functions to newer languages. Heck, we were even able to move into
and out of 16 and 32-bit address spaces. Now such mobility isn't as
easily accessible. Consider how difficult it would be moving to
Smalltalk from Java, Lisp, C#, or anything else.
All these thoughts while at the same time reading about different
languages being implemented on others' VMs. I've read Smalltalk can't
competently be implemented using Java bytecodes but that Java could be
implemented on Smalltalk's (I may have misread that). In the case of
the former it's unknown if such a thing is economically survivable for
vendors.
How can this be made easier? Is someone already making it so? Is there
a VM that will run Lisp, Smalltalk, Java, and C# while at the same time
providing straight-forward access to 3GL-written functions?
| |
| quasimodal 2005-04-26, 4:03 am |
| Have you taken a look at CORBA (Common Object Request Broker
Architecture)? Most of the Smalltalks I have used have had corba
interfaces if not a way to run an ORB, the ones that didn't pre-dated
the spec. Try the site http://linas.org/linux/corba.html; it has a
number of good links for more information and places to find software
for most platforms.
| |
| Hans-Martin Mosner 2005-04-26, 4:03 am |
| Thomas Gagne <tgagne@wide-open-west.com> wrote:
....
> Now I'm pondering how Smalltalk is no better than C++ in this regard.
> Smalltalk's artifacts, like other OOPs, are not readily accessible
> through dynamic libraries. In fact, I don't have any experience myself
> accessing Smalltalk code inside a shared library since VW doesn't
> provide that mechanism (that I'm aware of). I think I've read Dolphin
> or MT might provide that but I'm unsure.
MT specifically supports the building of DLLs which are callable from
other applications, so it is not a deficience of the language but of the
systems which derive from the original image concept.
As soon as you are willing to let the Smalltalk objects live happily and
securely in their own address space and communicate with them through
IPC mechanisms, calling image-based Smalltalk from other languages is
pretty easy. CORBA has already been mentioned, although I consider that
a little heavyweight. There are a number of other mechanisms. For
example, more than 10 years ago our company implemented SUN RPC client
and server code in Smalltalk-80. Doing this kind of stuff is actually
much easier in Smalltalk than in other languages due to its superb
reflective capabilities and other dynamic language features.
But as always, it should be considered whether your question has a
practical background or is just a theoretical consideration. Is there a
real life use case where you need to call Smalltalk from non-Smalltalk
code? If yes, what prohibits the use of such IPC technologies? Inquiring
minds want to know...
Cheers,
Hans-Martin
| |
| Ian Upright 2005-04-26, 9:00 am |
| Thomas Gagne <tgagne@wide-open-west.com> wrote:
[snip]
>Since OOPs don't seem able (or willing?) to speak directly to each other
>we've had to create all kinds of translators between them. SOAP and
>CORBA are two that come to mind, but erecting those bridges have a far
>higher expense than any of these languages have talking to C functions.
However, if you look at Swig, C++ libraries are easily wrappered by a
multitude of langauges. It seems to be the de-facto standard as to how to
integrate "scripting" and higher-level languages and C++ and/or external
C/FORTRAN/ETC languages.
Swig supports the following languages:
Perl
Python
Tcl/Tk
Guile
MzScheme
Ruby
Java
C#
PHP
CHICKEN
Ocaml
Objective-C
Javascript
S#
VW Smalltalk (coming, and with others help, perhaps other dialects)
That is a pretty tall order. Your statement "Don't seem to be able or
willing" seems a little off to me.
Swig modules also support notions of what they call "directors" whereby C++
virtual functions can be overriden by other higher-level language
implementations. So by using C++ directors, one can build C++ classes with
virtual functions, and then implement them in the language you choose. The
language can then compile this result into a library and you can have an
object-oriented interface (C++) implemented in different higher-level
languages, and they can all talk to each other using C++. Directors are
currently only supported in Java, C#, and Python, I believe. Support for
"directors" for the other languages may be in development (and is for
S#/Smalltalk). When complete, S# (a superset of Smalltalk) would be able to
expose a C++ interface to it's behavior, in the form of a shared library or
DLL. S# and Smalltalk MT already support exposing compiled classes/behavior
as a pure C interface. All this is definately a "poor-mans" excuse for
something like a unified .NET VM architecture, however, calling it
nonexistant may be fairly off the mark.
If you look at big libraries like this www.wxwidgets.org, you can see that
it's a C++ library that has been integrated into probably a dozen
higher-level languages. And this library is pretty extensive!
Many C++ libraries today are designed with for Swig integration in-mind.
Some C++ libraries come pre-built for a whole number of languages (using
Swig). Here is one such example: http://librdf.org/
On that note, many are looking at RDF, OWL, and OWL-S as a "data-centric"
focus to cross-systems and cross-language integration, as opposed to
behavior-centric, like SOAP and CORBA. Still, for high-performance and
simplicity, connecting higher-level languages to C and C++ seems to be a
reasonable choice at the moment.
>And I suppose that's the beauty of C. Everyone can talk to it (that I
>know of). Java (with a little exercise in JNI), Smalltalk, Lisp,
>Python, Ruby, Fortran, COBOL, you name it, it can talk to C.
Or using Swig, C++. Even using JNI to connect to a C library is fairly
painful compared to using Swig. You could also use Swig to have FORTRAN
connect to C++.
[snip]
>If someone were to hack a mechanism for making Smalltalk code accessible
>to other languages from a shared library I think a similar mechanism
>could be constructed. After initializing the library (I won't attempt
>to describe how that would work) instance methods could be called by
>making their first argument a handle to the instance. The function name
>would be resolved inside the scope of the instance handle to avoid
>collisions. This would resemble Python's (less than aesthetically
>pleasing) "self" reference to functions.
>
>Name mangling would still be a problem. Even if we assume all method
>parameters are positional there's still the matter of the method
>signature itself.
I'm not quite sure what you're describing here, but it sounds like Swig
directors, and no, name mangling is not a problem, as this has been resolved
in the implementation. If you want more details, read the Swig manual.
>In one of my own methods I'm unsure how pleasing the
>signature from a method, "defaultTableHeadersForResult: aTupleCollection
>columnList: aColumnList on: aStream" would look as a C-ish function.
> void defaultTableHeadersForResult:columnList:
on:
>Though there are the same amount of characters, it seems awkward outside
>its natural habitat, the Smalltalk IDE. Not unmanageable, just awkward.
> Forget for the moment C doesn't allow colons inside names.
[snip]
>How can this be made easier? Is someone already making it so? Is there
>a VM that will run Lisp, Smalltalk, Java, and C# while at the same time
>providing straight-forward access to 3GL-written functions?
A universal VM? The closest things that I'm aware of to this are:
Java VM (cross-platform, yielding poor performance, widespread adoption
despite a poor implementation)
..NET VM (windows specific, better performance, vendor lock-in)
Mono VM (cross-platform, not overly exciting performance, early stages of
development, beginnings of adoption)
AOS/Smallscript VM (excellent dynamic performance, early stages of
development and adoption)
So I may disagree with you in saying that there aren't any efforts to
integrate all these languages, but I wholeheartedly agree, that this
technology sure has miles of headroom to improve upon.
Ian
---
http://www.upright.net/ian/
| |
| Esteban A. Maringolo 2005-04-26, 4:02 pm |
| Thomas Gagne escribi=F3:
> Recently I was looking at trying to use a C++ API from Smalltalk and
ran
> into various problems. Having been down similar roads before and not
> having the time, inclination, or compelling reason to overcome them
I've
> decided to back-burner the whole Smalltalk/C++ project for now.
> [SNIP]
> By now some of you may be thinking I've wasted a bunch of time with
this
> posting but I can assure its therapeutic (for me). It just one of
my
> own 12-step program of realizing how much more difficult some things
are
> now than they were in the 80s. Sometimes it seems programmers have
made
> their jobs more difficult instead of less so. My Pascal, Fortran, C,
> Cobol and ADA could all call each others' libraries on Data General's
> AOS/VS. I could move to a new language without abandoning the assets
of
> the previous one. I could increase the return on old code by
migrating
> some functions to newer languages. Heck, we were even able to move
into
> and out of 16 and 32-bit address spaces. Now such mobility isn't as
> easily accessible.
> Consider how difficult it would be moving to
> Smalltalk from Java, Lisp, C#, or anything else.
That's probably because Smalltalk, Java, Lisp and C# live in another
plane, orthogonal to C, ADA, Pascal and Fortran. And Smalltalk's plane
isn't parallel to Java and C# neither.
> [SNIP]
> How can this be made easier? Is someone already making it so? Is
there
> a VM that will run Lisp, Smalltalk, Java, and C# while at the same
time
> providing straight-forward access to 3GL-written functions?
Perhaps is the plane shift what's missing, you still want to make use
of a "new"* technology from an old technology.
As an analogy, (forgive me if it doesn't fit) think in order of
"technological relevance", in a sense as people name them "1G, 3G, 4G
languages", say, in a timeline you have Pascal, C, ADA in the start;
then Java, Python, Ruby, then Lisp and Smalltalk.
As in real life, time has a direction, C and Pascal doesn't know
anything about Java or Smalltalk, because it's "in the future", the
inverse occurs with Java or Smalltalk in respect to C and Pascal, Java
and Smalltalk knows them, because C is "in the past".
The solutions to interconnect those "new systems" are complicated, is
always easier to make an API call than to make a CORBA request.
And that kind of things is what make many of us "feel" that no real
progress and advance has been made in the last 15 years.
In Dolphin you can implement COM servers, however it isn't
straightforward as an API call, but that's because there is no main()
neither in Smalltalk, and I'm happy with that.
Best regards,
--
Esteban.
| |
| Thomas Gagne 2005-04-26, 4:02 pm |
| Hans-Martin Mosner wrote:
<snip>
>
>
> MT specifically supports the building of DLLs which are callable from
> other applications, so it is not a deficience of the language but of the
> systems which derive from the original image concept.
> As soon as you are willing to let the Smalltalk objects live happily and
> securely in their own address space and communicate with them through
> IPC mechanisms, calling image-based Smalltalk from other languages is
> pretty easy.
<snip>
But IPC mechanisms seem to me a more complicated solution than linking
with a library. Each has advantages. For the purposes of linking and
distribution it's hard to compete against a shared library. For
purposes of distributed processing loosely-coupled alternatives using
IPCs of any variety are hard to beat.
But the latter requires the involvement of 3rd party software. Clients
and servers must be coordinated, there ports available, more software,
GUIs, and moving parts to go awry.
<snip>
> But as always, it should be considered whether your question has a
> practical background or is just a theoretical consideration. Is there a
> real life use case where you need to call Smalltalk from non-Smalltalk
> code? If yes, what prohibits the use of such IPC technologies? Inquiring
> minds want to know...
My company, like any other, has accumulated various software assets I'm
rue to reimplement in other languages. We use C, Smalltalk, and PHP and
each does some things well. The only one that's easily reusable is C.
Both Smalltalk and PHP can *reuse* its code. However some things must
be solved in both Smalltalk and PHP. Principally it violates my (and
hope everyone's) preference to reuse code.
It wouldn't hurt my feelings if more people used open-source shared
libraries written in Smalltalk.
| |
| Esteban A. Maringolo 2005-04-26, 4:02 pm |
| > My company, like any other, has accumulated various
> software assets I'm rue to reimplement in other languages.
> We use C, Smalltalk, and PHP and each does some things well.
If you use Smalltalk as a "language", only a new syntax to do things,
you're missing a lot. _A LOT_.
And that miss is what doesn't help smalltalk "the technology" to
improve, many people see it as a language, not as a system with another
syntax.
Personally I don't see it as a language, its a technology, wich uses a
special syntax named "Smalltalk" (also).
> The only one that's easily reusable is C.
You reuse functions? or simply use them from many places (other
programs)?
However, that's the opposite of what i think about reusability in C,
but is just a personal opinion.
> Both Smalltalk and PHP can *reuse* its code.
What about reusing objects instead of code?
Can you see a difference between use and reuse?
> However some things must be solved in both Smalltalk and PHP.
Like...?
> Principally it violates my (and hope everyone's) preference to reuse
code.
I tend to reuse objects.
Best regards.
| |
|
| Ian:
Swig sounds very interesting and promising, it would have made a great
topic at SS 2005 and sounds like a great CampSmalltalk project if not
one already.
-Charles
| |
| Thomas Gagne 2005-04-26, 4:02 pm |
|
Esteban A. Maringolo wrote:
>
>
> If you use Smalltalk as a "language", only a new syntax to do things,
> you're missing a lot. _A LOT_.
Good advice. I agree. Smalltalk is many things, but it shouldn't be an
island.
<snip>
>
>
>
>
> You reuse functions? or simply use them from many places (other
> programs)?
> However, that's the opposite of what i think about reusability in C,
> but is just a personal opinion.
I don't understand what you mean.
>
>
>
>
> What about reusing objects instead of code?
> Can you see a difference between use and reuse?
Reusing objects is just as problematic. How is PHP to reuse Smalltalk
objects in a tightly-coupled environment? How is C to reuse Smalltalk
objects in a tightly-coupled environment? If Smalltalk is an island
then we shouldn't complain about its lack of use, lack of scripting,
lack of publicity, lack of usage, lack of profits, and lack of customers
clamoring for us to write Smalltalk for them. Customers (ourselves
included) need to be able to migrate to new technologies, Smalltalk
among them. If Smalltalk is an all-or-nothing choice we risk being
negligent advising our clients to bite of more than is necessary to get
the job done.
>
>
>
>
> Like...?
Like business rules that must present themselves in both the application
server and the user interface.
>
>
>
> code.
>
> I tend to reuse objects.
A luxury when your whole world is objects, but the expense for many to
get there is more than they can afford. Even after writing the check
they regret the result trading a bunch of code that did work for brand
new code without a history. More regrettable (IMO) was the choice most
of them made.
| |
| Esteban A. Maringolo 2005-04-26, 4:02 pm |
| Thomas Gagne escribió:
>
>
> Esteban A. Maringolo wrote:
>
> Good advice. I agree. Smalltalk is many things, but it shouldn't be an
> island.
It left the island several years ago, and went orthogonal (even
today), that explains the baloon issue ;-)
> <snip>
> I don't understand what you mean.
Using a function from several places isn't reuse, it is use (many
times). I mean, using sprintf() from an API call in a Smalltalk
String, or Perl, or PHP doesn't mean you're reusing sprintf(), it
only means you're using it many times. Reuse is another different thing.
Having a Person class, which fits in different business domains is
more a kind of reuse. Because you build it for one purpose, and can
use it in other.
Point is a good example, basically it's a class of objects for
windowing support (I guess in the beginning it was there for that
reason), but you can use it for handle other kind of coordinates,
and even subclassify it and give it special behavior.
Reuse is more a kind of "different purpose use".
You can find more on this approach in <www.smalltalking.net>
Copying and pasting C code (or smalltalk code) isn't reuse.
[color=darkred]
> Reusing objects is just as problematic.
> How is PHP to reuse Smalltalk
> objects in a tightly-coupled environment?
> How is C to reuse Smalltalk
> objects in a tightly-coupled environment?
Build an interface, if you like. Smalltalk is floating "above",
(hence the baloon again ;-)
The effort to connect "legacy" technologies to smalltalk is always
hard. Connect a COBOL to Java, then you tell me. Do the inverse and
tell me which is easier.
Make a flat API through sockets, COM, IPC or whatever you have in
your Smalltalk.
> If Smalltalk is an island then we shouldn't complain about its
> lack of use, lack of scripting, lack of publicity,
> lack of usage, lack of profits, and lack of customers
> clamoring for us to write Smalltalk for them.
Perhahps it's an island, a tropical one, if you like.
Wouldn't you live there if you could? :-)
> Customers (ourselves included)
> need to be able to migrate to new technologies, Smalltalk
> among them.
Many people has walked that path, and as stated before, smalltalk
can use "backward components" (C APIs, CORBA, COM, etc).
Once "in the island"... if you need to communicate your smalltalk
you cand build your own tailored solution.
WebServices proliferation, not being as straighforward, nor
perfomant as an C API call is what you have now with spread support
among many "objects" (mostly object oriented) platforms.
> If Smalltalk is an all-or-nothing choice we risk being
> negligent advising our clients to bite of more than is necessary to get
> the job done.
I agree on this, smalltalks are more an all than nothing. But it
doesn't means that you can't coexist with other alternatives.
> Like business rules that must present themselves in both the application
> server and the user interface.
Which is which? Smalltalk the user interface?
> A luxury when your whole world is objects, but the expense for many to
> get there is more than they can afford.
That's a personal decision.
> Even after writing the check they regret the result trading
> a bunch of code that did work for brand new code
> without a history.
> More regrettable (IMO) was the choice most
> of them made.
One additional note: Even considering Smalltalk a good technology
with which do effective reuse, the only _true_ reuse is the
experience, something very valuable, and hardly findable when
developers are used a "human resource" (consumed and discarded, as
any finite resource).
Software reuse is hard to develop, and requires maduration of the
software and developers, which in turn requires time to achieve. And
time is money, so my simple inference is that Software reuse
requires money. :-)
Best regards.
--
Esteban.
| |
| Thomas Gagne 2005-04-26, 8:58 pm |
| Esteban A. Maringolo wrote:
> Thomas Gagne escribió:
<snip>
>
>
>
>
>
> Build an interface, if you like. Smalltalk is floating "above", (hence
> the baloon again ;-)
>
> The effort to connect "legacy" technologies to smalltalk is always hard.
> Connect a COBOL to Java, then you tell me. Do the inverse and tell me
> which is easier.
>
> Make a flat API through sockets, COM, IPC or whatever you have in your
> Smalltalk.
That is not tightly coupled, which is what this thread is about. I've
done that a bunch through middleware and have already published on that
topic.
"Legacy" is used as a pejorative used to describe anything that's
already working and won't employ contractors doing " " stuff. As a
CTO I'm uninterested in keeping programmers employed for the sake of
rewriting applications, functions, methods, anything that already works.
New code should be for doing new things.
I can think of many opportunities to write something new in Smalltalk
that would be valuable to "legacy" code. It would be preferable if
bridging the divide didn't /require/ the more complex interfaces
required to cross that chasm today, CORBA, IPCs, Swig. Smalltalk (what
I'm familiar with) can only run on-top of C code, not beneath it, as
would be the case in a shared library.
>
>
>
>
>
> Perhahps it's an island, a tropical one, if you like.
> Wouldn't you live there if you could? :-)
Even on my tropical island I need transportation to and from the
mainland as well as modern conveniences I'd rather not build myself.
Your description sounds more _Robinson Crusoe_ than _Fantasy Island_.
I'd rather be Mr. Roarke.
| |
| Ian Upright 2005-04-27, 3:59 am |
| Thomas Gagne <tgagne@wide-open-west.com> wrote:
>I can think of many opportunities to write something new in Smalltalk
>that would be valuable to "legacy" code. It would be preferable if
>bridging the divide didn't /require/ the more complex interfaces
>required to cross that chasm today, CORBA, IPCs, Swig. Smalltalk (what
>I'm familiar with) can only run on-top of C code, not beneath it, as
>would be the case in a shared library.
But what is complex about Swig? It's interface is essentially a C++ header
file. I'm sure there could be work done to add some nice user-friendly GUI
tools and such, but I'm not sure how you could make it much easier.
Swig has to be substantially easier than integrating CORBA, IPC's, SOAP,
etc. I wouldn't put it in the same category at all.
And again, Smalltalk MT and S# libraries can be easily used by C code, so
it's only VW and other Smalltalk implementations that may not...
VW ~~ Smalltalk. :) If you want to complain about VW specifically in that
regard, go right ahead, I'm right there with ya!
This is one of the biggest reasons I'm putting effort into using S# in the
first place -- I'm in total agreement. S# also has easy integration with
COM, being a COM client/server as well, integration with .NET, and
supposedly clean Java integration is being worked on as well.
Ian
---
http://www.upright.net/ian/
| |
| Thomas Gagne 2005-04-27, 3:59 am |
| Ian Upright wrote:
<snip>
> This is one of the biggest reasons I'm putting effort into using S# in the
> first place -- I'm in total agreement. S# also has easy integration with
> COM, being a COM client/server as well, integration with .NET, and
> supposedly clean Java integration is being worked on as well.
Has www.smallscript.org been abandoned? It doesn't look like much has
happened since last summer (2004).
| |
| Ian Upright 2005-04-27, 3:59 am |
| Thomas Gagne <tgagne@wide-open-west.com> wrote:
>Ian Upright wrote:
><snip>
>
>Has www.smallscript.org been abandoned? It doesn't look like much has
>happened since last summer (2004).
Not a lot has happened publicly since 2004, but there continues to be
ongoing development internally and with community partners such as myself.
Ian
---
http://www.upright.net/ian/
| |
| Eliot Miranda 2005-04-27, 8:59 pm |
|
Hans-Martin Mosner wrote:
> Thomas Gagne <tgagne@wide-open-west.com> wrote:
> ...
>
>
>
> MT specifically supports the building of DLLs which are callable from
> other applications, so it is not a deficience of the language but of the
> systems which derive from the original image concept.
> As soon as you are willing to let the Smalltalk objects live happily and
> securely in their own address space and communicate with them through
> IPC mechanisms, calling image-based Smalltalk from other languages is
> pretty easy. CORBA has already been mentioned, although I consider that
> a little heavyweight. There are a number of other mechanisms. For
> example, more than 10 years ago our company implemented SUN RPC client
> and server code in Smalltalk-80. Doing this kind of stuff is actually
> much easier in Smalltalk than in other languages due to its superb
> reflective capabilities and other dynamic language features.
>
> But as always, it should be considered whether your question has a
> practical background or is just a theoretical consideration. Is there a
> real life use case where you need to call Smalltalk from non-Smalltalk
> code? If yes, what prohibits the use of such IPC technologies? Inquiring
> minds want to know...
Our first important use-case was to provide a plugin so that VisualWorks
applications could be run within the context of a web browser. In 1998
I did a prototype which itself was based on prototype infrastructure to
allow VW to be embedded in a C program as a DLL. This itself depended
on the THAPI non-blocking interface. When a C program loaded the
prototype a thread was spawned to serve as the VW VM's main thread.
Call-ins to the VW dll went through the threaded call-back mechanism
asynchronously so that the VW system could continue to function even
when no call-ins were executing.
The killer down-side to this architecture for a web plugin is that the
VW heap is in the same address space as the rest of the code in the
browser, including other plugins. This is not a controlled environment
and we were extremely worried about the potential difficulty of
supporting the code when errors in other plugins could easily corrupt
the VW heap. Then John Baker, of RuntimePackager fame, discovered/knew
that Windows window handles were system-wide and could be shared between
processes (just as is the case with X since it has a clean client/server
separation, and MacOS Classic because its a single address space OS).
This meant that a small C plugin could launch VW in its own address
space handing VW the window handle in which it could draw, and use IPC
(John chose shared memory) to communicate between the two. But
conveniently window events are sent directly to the VM process so IPC
did not need to be used for these. This architecture is inherrently
more reliable as other plugins are unable to affect the VW heap[ and
this is the architecture we use today.
It does mean that we have the technology to package VM as a dll in our
back pocket once customer demand is strong enough. Currently no one is
banging down the doors demanding it although there have been a few
enquiries over the years.
--
_______________,,,^..^,,,____________________________
Eliot Miranda Smalltalk - Scene not herd
| |
|
|
| Reinout Heeck 2005-05-01, 9:11 pm |
| Thomas Gagne wrote:
> How would it be to have an Apache
> module for VW smalltalk (or any Smalltalk for that matter?). How
> would it be to call Smalltalk from inside a database using extended
> stored procedures?
I was about to respond along the same lines.
My impression is that making the VM embeddable has a lot of _marketing_
value even if current (more precisely: paying) customers don't express a
need need for it.
To me this sounds as one of those check box items that may generate
significant interest from the hobby/freeware and research communities.
My impression is that embeddability of a language is taken as an
important measure of maturity of an environment.
My 2c,
Reinout
-------
| |
|
| Eliot:
I develop motion simulation programs called freeCAD and CADSM using
VisualWorks. The motion simulation is state of the art but the graphics
is poor. If I can make dll's with my simulation solver, I can embed my
solver into other cad systems like AutoCAD, XXXXXXXXXX, etc more
easily. I can also hide the fact that I am using smalltalk because many
developers do not even want to talk to me because they do not want to
communicate in ways other than through dll's. Having dll's would have
helped me a lot as early as in 1995. I am happy to hear the news -
better late than never!
All the best,
Aik-Siong Koh
| |
| Ian Upright 2005-05-01, 9:11 pm |
| Reinout Heeck <reinout@soops.nl> wrote:
>Thomas Gagne wrote:
>
>I was about to respond along the same lines.
>
>My impression is that making the VM embeddable has a lot of _marketing_
>value even if current (more precisely: paying) customers don't express a
>need need for it.
>
>To me this sounds as one of those check box items that may generate
>significant interest from the hobby/freeware and research communities.
>My impression is that embeddability of a language is taken as an
>important measure of maturity of an environment.
>
>My 2c,
>
>Reinout
>-------
Of course, I'm not in any disagreement. I need these kinds of features so
badly I'm willing to create massive amounts of frameworks and build IDE's
for S#, just because it does this feature (integration with the rest of the
world) better than any other dialect. If VW had such a feature, as well as
native multi-threading, I'd definately consider it as an option -- but since
it doesn't, it's a non-starter for me and doesn't even register on my radar
screen.
I fear of VW/VA growing stagnant in this regard, and I surely hope that they
get things moving in the right direction, becuase I think it would be good
for overall Smalltalk success.
Ian
---
http://www.upright.net/ian/
| |
| Eliot Miranda 2005-05-01, 9:11 pm |
|
Thomas Gagne wrote:
> Eliot Miranda wrote:
> <snip>
>
> That's great to hear, but I'm  . You have a new product that
> allows VW to be used confidentially from more places than it is now
> because customers aren't asking for something you haven't told them you
> have? There must be some marketing wisdom there that eludes me 'cause
> to me it sounds absolutely asinine (I'm referring to your marketing, not
> the product). Think of any product you own and wonder what would have
> happened if the people that thought it up didn't sell it 'cause people
> hadn't asked them for it yet. Hmm.
Its not a marketing issue, its an engineering resource and priority
issue. While we haven't been working on the dll we've been working on
other things. Remember that to productise a prototype one has to write
examples, port and test across all platforms, document it properly,
train support in it, and so on. Writing a prototype is one thing and
producing a product is something of an entirely different order.
> But this case seems even more outrageous because there are competing
> products winning mindshare over your own because they already do what
> you have but aren't releasing. How would it be to have an Apache
> module for VW smalltalk (or any Smalltalk for that matter?). How
> would it be to call Smalltalk from inside a database using extended
> stored procedures?
I'll take that as one customer request :)
--
_______________,,,^..^,,,____________________________
Eliot Miranda Smalltalk - Scene not herd
| |
| Eliot Miranda 2005-05-01, 9:11 pm |
|
Ian Upright wrote:
> Reinout Heeck <reinout@soops.nl> wrote:
>
>
>
>
> Of course, I'm not in any disagreement. I need these kinds of features so
> badly I'm willing to create massive amounts of frameworks and build IDE's
> for S#, just because it does this feature (integration with the rest of the
> world) better than any other dialect. If VW had such a feature, as well as
> native multi-threading, I'd definately consider it as an option -- but since
> it doesn't, it's a non-starter for me and doesn't even register on my radar
> screen.
If it had the feature and lacked native threading but could still
interoperate in a threaded environment (using THAPI) would that work?
The dll we can do relatively quickly (months), native multi-threading we
can't (years).
> I fear of VW/VA growing stagnant in this regard, and I surely hope that they
> get things moving in the right direction, becuase I think it would be good
> for overall Smalltalk success.
>
> Ian
>
> ---
> http://www.upright.net/ian/
--
_______________,,,^..^,,,____________________________
Eliot Miranda Smalltalk - Scene not herd
| |
| Andrei N.Sobchuck 2005-05-03, 4:01 pm |
| Eliot Miranda <eliotm@pacbell.net> wrote:
EM> If it had the feature and lacked native threading but could still
EM> interoperate in a threaded environment (using THAPI) would that work?
EM> The dll we can do relatively quickly (months), native multi-threading we
EM> can't (years).
Why implementing of native multi-threading is so hard?
--
Andrei N.Sobchuck
JabberID: andreis@jabber.ru. ICQ UIN: 46466235.
| |
| Hans-Martin Mosner 2005-05-03, 8:58 pm |
| andrei.sobchuck@gmail.com (Andrei N.Sobchuck) wrote:
> Eliot Miranda <eliotm@pacbell.net> wrote:
> EM> If it had the feature and lacked native threading but could still
> EM> interoperate in a threaded environment (using THAPI) would that work?
> EM> The dll we can do relatively quickly (months), native multi-threading we
> EM> can't (years).
>
> Why implementing of native multi-threading is so hard?
The main problems are in the area of garbage collection. GC in a
multi-threaded environment is still an active research area AFAIK. I
don't know how MT does memory management, but I know the VisualWorks and
Squeak object memory implementations, and for both it would be a
nightmare to support simultaneous access by more than one thread. One
possibility would be to suspend all threads when objects need to be
moved around, but that is actually very often... you would also need to
let the threads get into a safe state (one in which all pointers are in
memory), which means some synchronization overhead. And then there are
things like the various data structures within the VM which would need
to be made thread safe, such as the compiled code cache.
If you have a sponsor who'd be willing to put some money (a lot of
money) into the development of such a VisualWorks VM, I bet Eliot would
be more than happy to attack the problem with a suitably expanded team :-)
Cheers,
Hans-Martin
| |
| Thomas Gagne 2005-05-04, 4:00 pm |
| Hans-Martin Mosner wrote:
> andrei.sobchuck@gmail.com (Andrei N.Sobchuck) wrote:
>
<snip>
>
>
> The main problems are in the area of garbage collection. GC in a
> multi-threaded environment is still an active research area AFAIK. I
> don't know how MT does memory management, but I know the VisualWorks and
> Squeak object memory implementations, and for both it would be a
> nightmare to support simultaneous access by more than one thread. One
> possibility would be to suspend all threads when objects need to be
> moved around, but that is actually very often... you would also need to
> let the threads get into a safe state (one in which all pointers are in
> memory), which means some synchronization overhead. And then there are
> things like the various data structures within the VM which would need
> to be made thread safe, such as the compiled code cache.
Perhaps each thread could be responsible for its own garbage collection?
> If you have a sponsor who'd be willing to put some money (a lot of
> money) into the development of such a VisualWorks VM, I bet Eliot would
> be more than happy to attack the problem with a suitably expanded team :-)
I was thinking about this this morning. With so many other products
already easily integrated with web servers why would someone pay Cincom
to develop one? Without the ability to embed VW in other systems Cincom
cedes that market to other products and vendors. Without that ability
Smalltalk developers are forced to use other languages. Unlike some
Java servers, Cincom's web server isn't robust enough on its own to
replace apache or IIS.
Regrettably, I don't know enough about VA, MT, Dolphin, or other
Smalltalk products to know if they have similar problems.
| |
| Eliot Miranda 2005-05-04, 9:00 pm |
|
Hans-Martin Mosner wrote:
> andrei.sobchuck@gmail.com (Andrei N.Sobchuck) wrote:
>
>
>
> The main problems are in the area of garbage collection. GC in a
> multi-threaded environment is still an active research area AFAIK. I
> don't know how MT does memory management, but I know the VisualWorks and
> Squeak object memory implementations, and for both it would be a
> nightmare to support simultaneous access by more than one thread. One
> possibility would be to suspend all threads when objects need to be
> moved around, but that is actually very often... you would also need to
> let the threads get into a safe state (one in which all pointers are in
> memory), which means some synchronization overhead. And then there are
> things like the various data structures within the VM which would need
> to be made thread safe, such as the compiled code cache.
> If you have a sponsor who'd be willing to put some money (a lot of
> money) into the development of such a VisualWorks VM, I bet Eliot would
> be more than happy to attack the problem with a suitably expanded team :-)
Indeed :) GC is one area. For performance one wants a newSpace per
thread, or at least to dole-out chunks of newSpace to threads to that
allocation within each thread's chunk is unsynchronised. When
scavenging occurs one also wants all threads involved, otherwise pause
times climb linearly with the number of threads. Getting better than
linear speedup with all threads involved is difficult also. The same
goes for the incremental and scan-mark GCs. Note that some Java
implementations have used reference counting for concurrent GC. Good GC
performance in a multi-threaded environment *might* mean a radically
different GC and that means changes to things like
ObjectMemory/MemoryPolicy as well as the VM. I hope it means no more
than careful parallelisation of the existing schemes ;)
The store check must also parallelise nicely, and ours does, requiring
synchronisation only when objects get entered into the remembered
tables. But sequential store buffers or card-marking may exhibit better
performance. All this needs to be tested. I don't think anyone's done
card marking in a system that has objects with an indirection either, so
we'd have to adapt the card marking algorithm somewhat.
There are lots of synchronisation points. For example the allObjects
and allInstances primitives need to stop other threads until they're done.
Analogously, the in-line caches and native code zone need to function
well concurrently. PICs help by reducing send-site rebinding frequency.
But one might want to consider an nzone per thread. Similarly the
stack zone.
Another area of difficulty is in scheduling. The VW (Smalltalk-80)
scheduler is real-time and has predictable scheduling characteristics
that allow for example interactive applications to be written that
exhibit predictable performance. I think it is unacceptable to take the
Java route and merely expose the underlying OS's thread scheduling,
resulting in non-portable behaviour across platforms. Instead the VM
needs to maintain a pool of native threads and schedule Smalltalk
processes within the pool, maintaining real-time scheduling so that (at
least within Smalltalk) there is never a higher-priority process blocked
by a lower-priority one.
This inevitably breaks down when a Smalltalk process calls out to
another language. Should a process that has called-out be considered to
be running or blocked? The system has no way of telling unless there is
information about a given call (e.g. provided in annotations from the
programmer). The safe assumption is that the process is blocked is the
system is not to lock-up when all threads call-out. So the foreign
function interface also has to synchronise, marking the thread as absent
form the pool until return or call-back, and to avoid the pool emptying,
has to ensure there's a spare thread in the pool.
A further difficulty is that debugging the VM is _much_ harder as one
can no longer reliably reproduce the steps leading up to a crash because
with concurrency the exact interleavings of threads determine
behaviour but these cannot be reproduced with conventional debugging
technology. This is a strong argument for avoiding multi-threading
altogether and instead using clusters, where one can reproduce because
one can time-stamp the messages sent between members of a cluster.
Of course, the image also requires significant changes to cope with
concurrency. Most of these are analogs of the above. There is now no
simple way of stopping all other processes as is used by
#valueUnpreemptively. But one might soon want to be able to do
compilations. parcel loads, class redefinitions and the like in
parallel. So there's a long laundry list of issues in the image.
I'm sure I'm forgetting lots of other important issues. But you get the
idea. In short there are person years of work to do a full
multi-threaded implementation and only person months of work to do
embedding as a DLL.
Since "how would you go about writing a multi-threaded VM" was one of my
interview questions at ParcPlace 10 years ago, it should be clear that
one significant meta-issue is precisely how to gather the resources to
do the work. Another is whether architectures like the Cell (each cell
being a core processor with a number of number-crunching "attached"
processors, each cell designed to fit in a cluster) make conventional
SMP VMs sufficiently unattractive that one would not waste time on
multi-threading and get better performance through better exploitation
of newer architectures. I think the opportunity cost of native
multi-threading is the largest for any project we can imagine right now.
That makes it by far the most risky commercially.
--
_______________,,,^..^,,,____________________________
Eliot Miranda Smalltalk - Scene not herd
| |
| Mike Bielser 2005-05-06, 8:58 am |
| Eliot Miranda wrote:
>
<snip...>
> This is a strong argument for avoiding multi-threading altogether and
> instead using clusters, where one can reproduce because one can
> time-stamp the messages sent between members of a cluster.
I know a bit about multithreading in general, but haven't heard about
clusters (in connection with multithreading/multitasking). Do you have
any pointers to articles? (especially related to VM and/or Smalltalk)
Thanks
Mike
| |
| Eliot Miranda 2005-05-06, 4:00 pm |
|
Mike Bielser wrote:
> Eliot Miranda wrote:
>
>
> <snip...>
>
>
>
> I know a bit about multithreading in general, but haven't heard about
> clusters (in connection with multithreading/multitasking). Do you have
> any pointers to articles? (especially related to VM and/or Smalltalk)
Clusters are an alternative to multi-threading. A cluster is a group of
computers connected by high-speed interconnects (typically ethernet).
Check out e.g. Beowolf. If you google cluster, Beowolf is the first
thing that comes up: www.beowolf.org
--
_______________,,,^..^,,,____________________________
Eliot Miranda Smalltalk - Scene not herd
| |
| Alan Knight 2005-05-09, 4:01 pm |
| Thomas Gagne <tgagne@wide-open-west.com> wrote in
news:e4adnbqJvYTHf-XfRVn-3Q@wideopenwest.com:
> I was thinking about this this morning. With so many other products
> already easily integrated with web servers why would someone pay
> Cincom to develop one? Without the ability to embed VW in other
> systems Cincom cedes that market to other products and vendors.
> Without that ability Smalltalk developers are forced to use other
> languages. Unlike some Java servers, Cincom's web server isn't robust
> enough on its own to replace apache or IIS.
It isn't? There are certainly people who do use it that way. There was a
disclaimer in the documentation for years saying that the "Tiny HTTP"
server was only for testing purposes, but we removed it recently because
it didn't seem to actually be true. There are some features missing that
you might want in production, but in terms of robustness I think it does
fairly well.
To my mind, the primary motivation for integrating with a separate web
server is not robustness, it is integration. If I want to mix and match
static pages, PHP, Perl scripts, and whatever else, I don't want
Smalltalk to have to be the one in charge.
However, typically one doesn't do this kind of integration by linking
Smalltalk directly into the web server, because a running Smalltalk is
relatively expensive. So (and the same applies to Java) rather than
linking a separate VM to each of the apache serving processes, you have
them communicate with one (or more) running servers. For that purpose,
all you need is a small gateway process, or even just a reverse proxy.
This is not to say that that works for all circumstances, or that the
ability to link in directly to a running process isn't useful.
Particularly for apache I would say it's less useful because of it's
multiple separate processes. I think it'd be more interesting to be able
to write an ISAPI plugin for IIS directly using Smalltalk. But the
gateway approach works well, in my opinion, for most web serving
applications.
The area I would think would be more useful for callouts is in direct
integration with the other parts of the application. So, if the primary
web pages are written in e.g. PHP, but application functionality is in
Smalltalk, then you might like to be able to have PHP call out to
Smalltalk directly rather than having to use an IPC mechanism. And, of
course, vice versa.
--
Alan Knight [|], Cincom Smalltalk Development
knight@acm.org
aknight@cincom.com
http://www.cincom.com/smalltalk
"The Static Typing Philosophy: Make it fast. Make it right. Make it
run." - Niall Ross
| |
| Thomas Gagne 2005-05-09, 4:01 pm |
|
Alan Knight wrote:
> Thomas Gagne <tgagne@wide-open-west.com> wrote in
> news:e4adnbqJvYTHf-XfRVn-3Q@wideopenwest.com:
>
>
>
> It isn't? There are certainly people who do use it that way. There was a
> disclaimer in the documentation for years saying that the "Tiny HTTP"
> server was only for testing purposes, but we removed it recently because
> it didn't seem to actually be true. There are some features missing that
> you might want in production, but in terms of robustness I think it does
> fairly well.
Unable to change log formats, Server exceptions don't all return 500
messages back to clients, Server configuration (IMO) is finicky --
there's a struggle inside the image re: whether to use data from the
image or data from the configuration file, I'm unaware of an API for
loading other modules for Python, Java, authentication methods, etc.
If Smalltalk is used satisfactorily in-place of apache or IIS then they
didn't need that robust a web server perhaps?
>
<snip>
>
> The area I would think would be more useful for callouts is in direct
> integration with the other parts of the application. So, if the primary
> web pages are written in e.g. PHP, but application functionality is in
> Smalltalk, then you might like to be able to have PHP call out to
> Smalltalk directly rather than having to use an IPC mechanism. And, of
> course, vice versa.
>
Agreed!
| |
| Steven Kelly 2005-05-09, 8:59 pm |
| >>>Unlike some Java servers, Cincom's web server isn't robust
>
Alan Knight wrote:
>
Thomas Gagne <tgagne@wide-open-west.com> wrote
> Unable to change log formats, Server exceptions don't all return 500 messages back to clients, Server configuration
> (IMO) is finicky --
> there's a struggle inside the image re: whether to use data from the image or data from the configuration file, I'm
> unaware of an API for loading other modules for Python, Java, authentication methods, etc.
You can change log formats using WebHitPrintPolicy in the public Store repository. It's configured via webtools.ini and
supports over 20 different fields.
Broadly speaking, the server is robust, supports lots of wonderful things for a dynamic web application server, but
oddly lacks several necessary features for even a basic static file web server. In addition to what Thomas mentions, I
ran into the following:
- no support for HEAD
- HTTP error responses incorrect (unrecognised methods like HEAD send strings instead of codes like 501)
- Last-Modified header is always current time, should read file time for static files
- Cache-Control header doesn't work as it is sent as CacheControl
- Expires header uses wrong time zone (local time, should be UTC)
The good news is these were all easily corrected, and Alan has kindly taken my fixes and incorporated them into the 7.4
dev stream with improvements. I also have a hack that adds a configuration file for specifying new file extension->MIME
type mappings. Our server with these running on it has been as reliable as the IIS 6.0 that sits alongside it - in 6
months we've had one occasion when we had to restart VW and one when we had to restart ISA/IIS.
Steve
--
Steven Kelly, CTO, MetaCase, http://www.metacase.com/blogs/stevek/blogView[color=darkred]
Bill Gates on Domain-Specific Modeling, www.adtmag.com/article.asp?id=9166
| |
| Alan Knight 2005-05-13, 4:15 pm |
| Oh, *those* problems <grin>
I'd have considered most of those missing features rather than robustness
issues, but point taken. Do Java servers have mechanisms for loading in other
language modules?
Do you have any more information about the server exceptions not returning
errors? Is that different from the stuff Steven Kelly was talking about
(which should be fixed in the 7.4 stream). Could you be more specific than
"finicky" about the configuration issues?
The log formats has been a problem for a while, but the person who was
working on login has gotten pulled off to work on a number of other things,
so it keeps getting delayed.
Thomas Gagne <tgagne@wide-open-west.com> wrote in
news:MNednYRZaYkBGuLfRVn-vQ@wideopenwest.com:
>Unable to change log formats, Server exceptions don't all return 500
> messages back to clients, Server configuration (IMO) is finicky --
> there's a struggle inside the image re: whether to use data from the
> image or data from the configuration file, I'm unaware of an API for
> loading other modules for Python, Java, authentication methods, etc.
>
> If Smalltalk is used satisfactorily in-place of apache or IIS then
> they didn't need that robust a web server perhaps?
>
--
Alan Knight [|], Cincom Smalltalk Development
knight@acm.org
aknight@cincom.com
http://www.cincom.com/smalltalk
"The Static Typing Philosophy: Make it fast. Make it right. Make it run." -
Niall Ross
| |
| Thomas Gagne 2005-05-13, 4:15 pm |
| Alan Knight wrote:
> Oh, *those* problems <grin>
>
> I'd have considered most of those missing features rather than robustness
> issues, but point taken. Do Java servers have mechanisms for loading in other
> language modules?
>
> Do you have any more information about the server exceptions not returning
> errors? Is that different from the stuff Steven Kelly was talking about
> (which should be fixed in the 7.4 stream).
I'm not sure. It only seems to happen when you're debugging the server
and kill the exception. Apparently there's a missing #unwind thingy.
Could you be more specific than
> "finicky" about the configuration issues?
Build an image on one box, save it headless, start it on another box.
It won't answer any of the requests because it doesn't recognize the
hostname. Each time I deploy a headless image we must first reset the
configuration to get it to read the local file. Because its headless,
and a production build, we don't save the image again 'cause that would
make it different than what we installed.
>
> The log formats has been a problem for a while, but the person who was
> working on login has gotten pulled off to work on a number of other things,
> so it keeps getting delayed.
It happens. It would be nice to run VW log files through any of the log
analyzers out there in a way compatible with apache log files.
Additionally, there's supposed to be another log file other than
webserve.log where errors are written, but I've never seen anything
written to it. I can't even remember its name.
|
|
|
|
|