Home > Archive > Compilers > May 2005 > The compilation approach in modern languages
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
The compilation approach in modern languages
|
|
|
| What are the new languages that can possibly become mainstream
programming languages in academia/industry, and what is the
compilation technology they follow?
The arrival of Java bought the use of sophisticaed runtimes VM to the
mainstream (compared to the C/C++ native code approach). Does most of
the newer languages follow the similar approach, or most of them
follow the conventional approach of native code generation?
-Ganesh
| |
| Rodney M. Bates 2005-02-12, 8:58 pm |
| It is a common misconception that certain languages are "interpreted"
and others are "compiled" In fact, the distinction between compiled
and interpreted (and hybrids) is not a property of the language, but
only of the implementation.
We now have a traditional native code compiler for Java. There were
hybrid translator/interpreter implementations for C around 15 years
ago.
The one exception is the functional languages, where functions can be,
in effect, manipulated at runtime and then evaluated. Obviously, it
is too late then for a traditional batch compile, and something more
like interpretation is needed, at least some of the time.
Ganny wrote:
> What are the new languages that can possibly become mainstream
> programming languages in academia/industry, and what is the
> compilation technology they follow?
>
> The arrival of Java bought the use of sophisticaed runtimes VM to the
> mainstream (compared to the C/C++ native code approach). Does most of
> the newer languages follow the similar approach, or most of them
> follow the conventional approach of native code generation?
| |
|
| >>It is a common misconception that certain languages are "interpreted"
[color=darkred]
Also, it is true that languages are designed with particular
implementation approaches in mind. It would be a misconception to think
that C was designed for interpreation approach in mind and Java was
designed with native compilation in mind.
-Ganesh
[I think you'll find that Java was designed to be compiled into
interpreted byte codes, native code via JIT from byte codes, or
directly to native code. C was certainly designed with native code in
mind. =John]
| |
| Tony Finch 2005-02-14, 4:03 am |
| "Rodney M. Bates" <rbates@southwind.net> wrote:
>
>The one exception is the functional languages, where functions can be,
>in effect, manipulated at runtime and then evaluated. Obviously, it
>is too late then for a traditional batch compile, and something more
>like interpretation is needed, at least some of the time.
That is not true for most modern functional programming languages
(i.e. ones not descended from lisp) which are designed to be compiled
to native code. It is also true for non-functional languages that have
an "eval" function, which includes many scripting languages.
Tony.
--
f.a.n.finch <dot@dotat.at> http://dotat.at/
[For that matter, people have been compiling Lisp to machine code for
about 45 years. -John]
| |
| Carter Schonwald 2005-02-14, 4:03 am |
| I'm not aware of any general characteristic of functional languages
that precludes batch compilation, especially since such compilers are
not unusual.
The runtime creation of a function merely requires something along the
lines of packing together a pointer to the beginning of the function's
body and a pointer to the current variable environment into a tuple.
If you meant something else, could you please clarify?
| |
| Robert A Duff 2005-02-14, 4:03 am |
| "Rodney M. Bates" <rbates@southwind.net> writes:
> It is a common misconception that certain languages are
> "interpreted" and others are "compiled" In fact, the distinction
> between compiled and interpreted (and hybrids) is not a property of
> the language, but only of the implementation.
True. But it is reasonable to ask what is typical, or what a
particular language designer intended.
Writing a native code generator for a language like one of the the
Unix shells (sh, csh, etc) would be a rather pointless exercise. I'd
say it's reasonable to call 'csh' an interpreted language, even though
in theory one could write a compiler for it. Same for the language of
DOS/Windows .bat files.
> We now have a traditional native code compiler for Java. There were
> hybrid translator/interpreter implementations for C around 15 years
> ago.
> The one exception is the functional languages, where functions can
> be, in effect, manipulated at runtime and then evaluated.
> Obviously, it is too late then for a traditional batch compile, and
> something more like interpretation is needed, at least some of the
> time.
Lots of non-functional languages have this property, too. Some
languages have the ability to calculate a text string at run time, and
then execute (interpret?) that string as a piece of program text. If
this is implemented by generating machine code (at run time) and then
executing the machine code, I'd call that "compiling", not
"interpreting".
Java has the ability to load byte codes at run time. I'd call a JIT
compiler a "compiler", even though it's not a "traditional batch
compiler".
[color=darkred]
> Ganny wrote:
Writing an interpreter is a lot easier than writing a compiler, so I
suspect that most languages at least start out that way. Well, not a
"pure" interpreter, but a translator to something like byte codes,
which are then interpreted.
[color=darkred]
- Bob
[I've seen Unix shell compilers, but I got the impression that the goal
was more to keep people from seeing what they did than to make them run
faster. -John]
| |
| glen herrmannsfeldt 2005-02-14, 4:03 am |
| Rodney M. Bates wrote:
> It is a common misconception that certain languages are "interpreted"
> and others are "compiled" In fact, the distinction between compiled
> and interpreted (and hybrids) is not a property of the language, but
> only of the implementation.
> We now have a traditional native code compiler for Java. There were
> hybrid translator/interpreter implementations for C around 15 years
> ago.
> The one exception is the functional languages, where functions can be,
> in effect, manipulated at runtime and then evaluated. Obviously, it
> is too late then for a traditional batch compile, and something more
> like interpretation is needed, at least some of the time.
There are also macro languages like TeX which can't be compiled for
similar reasons. For TeX, even that category codes for characters
(letter, etc.) can be changed dynamically, so the rules on processing
characters must be followed exactly.
Also, at least according to CS people I talk to, most systems are
somewhere between true compilers and true interpreters.
Most interpreters for programming languages (as opposed to command
processors) compile to an intermediate form, possibly just converting
language keyword tokens to an internal form, sometimes also including
converting numerical constants to internal form.
Most compiled languages generate code including calls to a library,
instead of generating all code inline, as a true compiler should.
Even more obvious, interpretation of format strings for formatted I/O.
-- glen
[You might be surprised what PL/I compilers do with format strings.
I've seen them compiled to thunk-like things that run as coroutines
with the part of the program doing the I/O. -John]
| |
| Nick Maclaren 2005-02-17, 4:03 am |
| Tony Finch <dot@dotat.at> wrote:
>"Rodney M. Bates" <rbates@southwind.net> wrote:
>
>That is not true for most modern functional programming languages
>(i.e. ones not descended from lisp) which are designed to be compiled
>to native code. It is also true for non-functional languages that have
>an "eval" function, which includes many scripting languages.
Yes.
And the equally important (and largely older) class of application
languages. One amusing example here is Genstat - designed entirely
by statisticians, and interestingly unusual :-)
As far as this aspect goes, there are really four levels:
Ones with no general 'eval' operation.
Ones where 'eval' executes entirely within its own scope.
Ones where 'eval' can behave as a declaration.
Ones where 'eval' can change the syntax accepted by the compiler.
It is reasonable to say that languages that allow the last cannot
be compiled and must be interpreted :-)
Regards,
Nick Maclaren.
| |
| Ivan Boldyrev 2005-02-17, 4:03 am |
| On 9020 day of my life Carter Schonwald wrote:
> The runtime creation of a function merely requires something along the
> lines of packing together a pointer to the beginning of the function's
> body and a pointer to the current variable environment into a tuple.
>
> If you meant something else, could you please clarify?
Lisp can coerce list to function. List is generated at runtime ->
function is generated at runtime. The feature rocks, but
implementation must use full compiler or at least interpreter.
But language like [Something]ML and Haskell can be (and are) compiled
statically.
--
Ivan Boldyrev
| |
| Rodney M. Bates 2005-02-17, 4:03 am |
| Tony Finch wrote:
> "Rodney M. Bates" <rbates@southwind.net> wrote:
>
[color=darkred]
> That is not true for most modern functional programming languages
> (i.e. ones not descended from lisp) which are designed to be compiled
> to native code.
I would expect many implementations to do some runtime partial
evaluation when a curried function is partially applied. Does
it never happen? I am sure this is sometimes more than an
efficiency issue. Seems like it could affect termination,
for example.
| |
| Ivan Boldyrev 2005-02-17, 4:03 am |
| On 9020 day of my life Tony Finch wrote:
> [For that matter, people have been compiling Lisp to machine code for
> about 45 years. -John]
Yes, but you have to provide compiler at runtime.
--
Ivan Boldyrev
[Well, sure. Lots of languages can compile at runtime. -John]
| |
| George Neuner 2005-02-17, 4:03 am |
| On 13 Feb 2005 22:46:08 -0500, "Carter Schonwald" <cartazio@gmail.com>
wrote:
>I'm not aware of any general characteristic of functional languages
>that precludes batch compilation, especially since such compilers are
>not unusual.
>
>The runtime creation of a function merely requires something along the
>lines of packing together a pointer to the beginning of the function's
>body and a pointer to the current variable environment into a tuple.
>
>If you meant something else, could you please clarify?
I don't want to start a debate about whether Lisp qualifies as a
"functional" language, however, Lisp programs have access to the
interpreter/compiler at run time and a running program can be changed
or extended on the fly by loading or generating new function
definitions.
Using these features, it is possible to write code that references
functions which don't exist at development time.
ex 1.
(setf code-for-add3 (list 'lambda (list 'x) (list '+ 'x 3)))
(setf code-for-add4 '(lambda (x) (+ x 4)))
:
(compile 'add3 code-for-add3)
(compile 'add4 code-for-add4)
(format t "~A = ~A: ~A~%" (add3 7) (add4 6)
(= (add3 7) (add4 6)))
:
The functions (add3) and (add4) are created from *data*, in this case
lists representing lamba expressions that define them.
The (compile) syntax shown creates named, top level functions that can
be directly referenced. (compile) can also create anonymous functions
which must be funcall'd.
ex2.
(setf code-for-my-function (read))
:
(compile my-function code-for-my-function))
(my-function ...)
:
Sans error checking, this example expects to read the list expression
defining (my-function) from standard input. It could just as well be
read from a file or any other input source.
A batch compile performed at development time could not have created
code for any of (add3), (add4) or (my-function) ... the sources for
the functions were not known until run time. [ I'll grant the
hypothetical "sufficiently smart compiler" might figure out (add4). ]
Many languages have some kind of "eval" for executing arbitrary code.
In Lisp (compile) differs from (eval) in that it has access to the
running program's environment, it does not execute the functions it
creates and it can create functions which are tightly bound to the
running program. In contrast (eval) compiles an expression in a clean
lexical environment creating an anonymous function which it then
executes and discards.
Most (all?) of the recognized functional languages allow functions to
be redefined and anonymous functions to be created, but I don't know
whether they routinely permit the same kind of run time extension
possible in Lisp.
George
| |
| Torben Ęgidius Mogensen 2005-02-19, 3:58 am |
| "Rodney M. Bates" <rbates@southwind.net> writes:
> Tony Finch wrote:
>
>
> I would expect many implementations to do some runtime partial
> evaluation when a curried function is partially applied. Does
> it never happen?
The closest thing to runtime PE for partial applications that I'm
aware of in non-experimental applications is the full laziness
transformation. This (at comile time) extracts subexpressions that
depend only on the first argument of a curried function, so the
evaluation of these can be shared between different applications of
the closure.
No code is actually built at runtime, though.
> I am sure this is sometimes more than an efficiency issue. Seems
> like it could affect termination, for example.
It can if you eagerly evaluate expressions that may be nonterminating.
But the full laziness transformation is applied to lazy languages, so
the evaluation of an expression is postponed until the first use of
its value.
A less mainstream language is MetaML, which does do runtime code
generation on partial applcations, if these are typed so the first
argument has an earlier binding time than the latter. The generated
code is for a virtual machine, which AFAIK is interpreted. Several
functional languages running on virtual machines also have support for
runtime code generation, but this is done explicitly rather than
implicitly on partial applications. And unlike MetaML, you don't get
type safety on generated code.
Torben
| |
| Tony Finch 2005-02-19, 3:58 am |
| "Rodney M. Bates" <rbates@southwind.net> wrote:
>Tony Finch wrote:
>
>I would expect many implementations to do some runtime partial
>evaluation when a curried function is partially applied. Does
>it never happen? I am sure this is sometimes more than an
>efficiency issue. Seems like it could affect termination,
>for example.
The usual implementation is that the partial application is converted
into a thunk which, when the remaining arguments are provided, calls
the function with all the arguments. If the partial application's
argument is statically constant then it may be partially evaluated at
compile time. The implementation cannot not have any effect on
termination, because that would be a bug.
Tony.
--
f.a.n.finch <dot@dotat.at> http://dotat.at/
| |
| Hannah Schroeter 2005-02-19, 3:58 am |
| Hello!
George Neuner <gneuner2@comcast.net> wrote:
>[...]
>Most (all?) of the recognized functional languages allow functions to
>be redefined and anonymous functions to be created, but I don't know
>whether they routinely permit the same kind of run time extension
>possible in Lisp.
Redefinition? When I look at Haskell, SML or ocaml, I don't see a way
to redefine functions (in the sense that you can, at run time, change
the definition).
Okay, in *ML you can define a ref cell which refers to a function of
some specified type and may overwrite it, but this doesn't pose
difficult issues, as you can of course only use lambdas (with their
bodys available to the compiler), already-defined functions (i.e.
body available) or other expressions returning a function to set them,
all of which is representable with a closure (pointer to code which is
known at compile time, and environment, for references to global
functions, the latter may be empty).
So you definitely don't need an interpreter/compiler available at
run-time for neither Haskell nor *ML.
>George
Kind regards,
Hannah.
| |
| Ivan Boldyrev 2005-02-19, 3:58 am |
| On 9023 day of my life Ivan Boldyrev wrote:
> On 9020 day of my life Tony Finch wrote:
>
> Yes, but you have to provide compiler at runtime.
>
> [Well, sure. Lots of languages can compile at runtime. -John]
That's news! I know only about Lisp dialects.
Perhaps, (Common) Lisp/Scheme is different from them because
COMPILE-FILE is *standard* function.
--
Ivan Boldyrev
[In a lot of cases like perl the compilation isn't particularly obvious
to the user. -John]
| |
| Jason Lee Eckhardt 2005-02-20, 8:58 pm |
| >A less mainstream language is MetaML, which does do runtime code
>generation on partial applcations, if these are typed so the first
>argument has an earlier binding time than the latter. The generated
>code is for a virtual machine, which AFAIK is interpreted. Several
>functional languages running on virtual machines also have support for
>runtime code generation, but this is done explicitly rather than
>implicitly on partial applications. And unlike MetaML, you don't get
>type safety on generated code.
Also checkout a slightly more "mainstream" alternative to MetaML,
called MetaOCaml. This language provides support for "multi-stage
programming"-- the ability to construct, combine, and execute code
at runtime (where all such future-stage computations are type safe).
It is built on the popular OCaml system and can be obtained from:
www.metaocaml.org.
| |
| George Neuner 2005-02-28, 3:59 am |
| On 18 Feb 2005 22:49:12 -0500, hannah@schlund.de (Hannah Schroeter)
wrote:
>Hello!
>
>George Neuner <gneuner2@comcast.net> wrote:
>
>
>Redefinition? When I look at Haskell, SML or ocaml, I don't see a way
>to redefine functions (in the sense that you can, at run time, change
>the definition).
Terminology? I didn't mean a function can be changed, I meant that
its name can be re-bound to a different function (possibly eliminating
all references to the first function).
George
| |
| Anton Ertl 2005-02-28, 3:59 am |
| Ivan Boldyrev <boldyrev@cgitftp.uiggm.nsc.ru> writes:
....[color=darkred]
>Perhaps, (Common) Lisp/Scheme is different from them because
>COMPILE-FILE is *standard* function.
Hmm, COMPILE-FILE is not quite what I consider compilation at
run-time. Even in C I can do system("cc ..."); ok, some people have
used that in combination with dlopen/dlsym to achieve a kind of
run-time compilation, but it is relatively cumbersome.
What I am thinking of when I hear Lisp and run-time compilation are
things like macros and back-quote.
Other languages have similar features, and in the standard: In Forth,
"COMPILE,", "POSTPONE" and "LITERAL" are standard words. Prolog has
assert.
etc.
- anton
--
M. Anton Ertl
anton@mips.complang.tuwien.ac.at
http://www.complang.tuwien.ac.at/anton/home.html
| |
| George Neuner 2005-03-01, 8:59 pm |
| On 28 Feb 2005 00:53:47 -0500, anton@mips.complang.tuwien.ac.at (Anton
Ertl) wrote:
>Ivan Boldyrev <boldyrev@cgitftp.uiggm.nsc.ru> writes:
>...
>
>Hmm, COMPILE-FILE is not quite what I consider compilation at
>run-time. Even in C I can do system("cc ..."); ok, some people have
>used that in combination with dlopen/dlsym to achieve a kind of
>run-time compilation, but it is relatively cumbersome.
Lisp has the standard function COMPILE for run-time compilation. It
also has COERCE which (among other uses) can create an interpreted
function at run-time from a suitably structured list.
When called directly on a suitable list, COMPILE calls COERCE to
create a function and then compiles and links the function.
COMPILE-FILE is merely a driver function for COMPILE which takes its
input from a stream.
>What I am thinking of when I hear Lisp and run-time compilation are
>things like macros and back-quote.
Most macro use in Lisp occurs at development time. The typical run
time use is to hot patch a running program and is hidden behind a LOAD
of the new code. A program whose normal operation requires a lot of
run time macro expansion is either doing something incredibly smart or
insanely stupid.
George
| |
| Ivan Boldyrev 2005-03-04, 8:58 pm |
| On 9034 day of my life Anton Ertl wrote:
>
> Hmm, COMPILE-FILE is not quite what I consider compilation at
> run-time. Even in C I can do system("cc ..."); ok, some people have
> used that in combination with dlopen/dlsym to achieve a kind of
> run-time compilation, but it is relatively cumbersome.
.... And unportable. For example, default MS Windows installation has
no C compiler. But every standard Lisp system has compiler.
Lisp has COMPILE function that compiles other function. And you can
convert properly created list to compiled function:
(defun compile-at-runtime (the-list)
(compile nil (coerce the-list 'function)))
> What I am thinking of when I hear Lisp and run-time compilation are
> things like macros and back-quote.
Macros and back-quotes are processed at compile-time.
--
Ivan Boldyrev
| |
| Anton Ertl 2005-03-06, 3:58 am |
| George Neuner <gneuner2@comcast.net> writes:
>On 28 Feb 2005 00:53:47 -0500, anton@mips.complang.tuwien.ac.at (Anton
>Ertl) wrote:
>
>Most macro use in Lisp occurs at development time.
Yes, what I meant here is the ability of Lisp and other languages to
compile at run-time to also run at compile-time, i.e., to use the full
power of the language in macro processing.
The resulting lack of a straight compile-time/run-time boundary is
reflectyed in your use of "development time" (vs. "use time").
Of course, there may be several development times: If a program
generator (e.g., a parser generator is involved), there's the
development time of the generator, and the development time of the
generated program.
> A program whose normal operation requires a lot of run time macro
>expansion is either doing something incredibly smart or insanely
>stupid.
Writing a program generator does not require incredible smarts,
although it may be unidiomatic in Lisp; maybe Lispers prefer to write
meta-interpreters, instead of generators.
- anton
--
M. Anton Ertl
anton@mips.complang.tuwien.ac.at
http://www.complang.tuwien.ac.at/anton/home.html
| |
| Hannah Schroeter 2005-05-18, 4:02 pm |
| Hello!
George Neuner <gneuner2@comcast.net> wrote:
>On 18 Feb 2005 22:49:12 -0500, hannah@schlund.de (Hannah Schroeter)
>wrote:
[color=darkred]
[color=darkred]
[color=darkred]
>Terminology? I didn't mean a function can be changed, I meant that
>its name can be re-bound to a different function (possibly eliminating
>all references to the first function).
You can't re-bind (global) names in *ML or Haskell. You can use different
parameters on a parameter position with a function type, you can change
reference cells in *ML/Haskell. I.e. you can use values with a function
type. They're usually represented as lexical closure, often represented
as a pointer to code, plus a pointer to the captured lexical environment,
or similar.
Not much hassle to compilers.
And in Lisp, yeah, you can dynamically compile code
(compile 'nil (lambda ...))
yields an object of type function. That object can also be represented
as closure (see above), where the code pointer points to a newly allocated
code block where the compile function wrote its object code (be it
byte code, be it native machine code).
The binding of global function names to the code is done using the
function cell of symbols in Common Lisp, which also references any
object of type function, i.e. you can use any closure there. Usually
the environment part of those closures are empty (e.g. if you use defun
not inside a lexical binding to create a function with a symbolic name).
>George
Kind regards,
Hannah.
|
|
|
|
|