Enhance [proc] for anonymous functions and closures

Hi,

I have filed the feature request below to the tracking system at
http://tcl.sourceforge.net/ and would like to have some feedback from
other users here as well...
How realistic is this, and how desireable? Is it possible at all, in a
certain time frame and what would this time frame be, approximately?

Asking as somebody who has no clue about interpreter implementation and
very few clue about compiler implementation...


Thanks,
Eckhard


----- Feature Request 1932618 -----
I would like to request that [proc] becomes a data type in addition to a
command, so that procs can be


- created on the fly, capturing the current environment and procedure stack
- created anonymously, e.g. [proc {} {...}] would return a new proc to
be used elsewhere.
- returned by other proc's
- given as arguments to other procs (including their arguments and the
environment in which they were created)

When this s possible, I think it would also be possible to have closures
in Tcl and better capabilities for functional programming - which would
enhance the language *a lot*

Is it possible to have [proc] as a Tcl_Obj* type?

I think this needs to be TIPed. See: wiki.tcl.tk/TIP

EL wrote:
> Hi,
>
> I have filed the feature request below to the tracking system at
> http://tcl.sourceforge.net/ and would like to have some feedback from
> other users here as well...
> How realistic is this, and how desireable? Is it possible at all, in a
> certain time frame and what would this time frame be, approximately?
>
> Asking as somebody who has no clue about interpreter implementation and
> very few clue about compiler implementation...
>
>
> Thanks,
> Eckhard
>
>
> ----- Feature Request 1932618 -----
> I would like to request that [proc] becomes a data type in addition to a
> command, so that procs can be
>
>
> - created on the fly, capturing the current environment and procedure stac
k
> - created anonymously, e.g. [proc {} {...}] would return a new proc to
> be used elsewhere.
> - returned by other proc's
> - given as arguments to other procs (including their arguments and the
> environment in which they were created)
>
> When this s possible, I think it would also be possible to have closures
> in Tcl and better capabilities for functional programming - which would
> enhance the language *a lot*
>
> Is it possible to have [proc] as a Tcl_Obj* type?


--
+--------------------------------+---------------------------------------+
| Gerald W. Lester                                                       |
|"The man who fights for his ideals is the man who is alive." - Cervantes|
+------------------------------------------------------------------------+

Gerald W. Lester schrieb:
> I think this needs to be TIPed. See: wiki.tcl.tk/TIP

I thought for TIP's it would be necessary to have a reference
implementation, or at least a more concrete specification how it could
be achieved?
That I don't have, unfortunately... the text in the request is the only
thing I can actually specify and I guess this is too vague for a TIP, or no?


Eckhard

EL wrote:
> Hi,
>
> I have filed the feature request below to the tracking system at
> http://tcl.sourceforge.net/ and would like to have some feedback from
> other users here as well...
> How realistic is this, and how desireable? Is it possible at all, in a
> certain time frame and what would this time frame be, approximately?
>
> Asking as somebody who has no clue about interpreter implementation and
> very few clue about compiler implementation...
>
>
> Thanks,
> Eckhard
>
>
> ----- Feature Request 1932618 -----
> I would like to request that [proc] becomes a data type in addition to a
> command, so that procs can be
>
>
> - created on the fly, capturing the current environment and procedure stac
k
> - created anonymously, e.g. [proc {} {...}] would return a new proc to
> be used elsewhere.
> - returned by other proc's
> - given as arguments to other procs (including their arguments and the
> environment in which they were created)
>
> When this s possible, I think it would also be possible to have closures
> in Tcl and better capabilities for functional programming - which would
> enhance the language *a lot*
>
> Is it possible to have [proc] as a Tcl_Obj* type?

have you looked at the new apply command in 8.5?
http://www.tcl.tk/man/tcl8.5/TclCmd/apply.htm
also check out the TIP that introduced it
http://www.tcl.tk/cgi-bin/tct/tip/194
it has some discussion of uses and how to  use it for
other FP constructs.

Bruce

EL wrote:
...
> ----- Feature Request 1932618 -----
> I would like to request that [proc] becomes a data type in addition to a
> command, so that procs can be
>
>
> - created on the fly, capturing the current environment and procedure stac
k
> - created anonymously, e.g. [proc {} {...}] would return a new proc to
> be used elsewhere.
> - returned by other proc's
> - given as arguments to other procs (including their arguments and the
> environment in which they were created)
>
> When this s possible, I think it would also be possible to have closures
> in Tcl and better capabilities for functional programming - which would
> enhance the language *a lot*

As Bruce Hartweg has pointed out, the [apply] command in 8.5 provides
first-class procs. Enhancing [proc] to allow a 2-arg case for a
constructor of such lambdas is a possibility, but I think it might cause
confusion[*]. I currently use:

proc lambda {params body} { list ::apply [list $params $body] }

(or a variation of this).

Closures are harder. See http://wiki.tcl.tk/17686 and
http://wiki.tcl.tk/20662 for a simple approach to creating immutable
closures (i.e., they capture the current values of variables rather than
the variables themselves). The problem with mutable variables is that
they have no natural string rep (e.g., if they are proc-local), so hard
to represent as a Tcl_Obj. The best I've come up with so far is to use
fully-qualified global/namespace variables as sort of references, but
you then need to deal with lifetime management.

[*] A generalisation of this idea would be to allow 0 or more name
arguments to proc, e.g. no-args = lambda, 1 arg = as now, many args =
namespace ensemble. See http://wiki.tcl.tk/20657

-- Neil

Neil Madden wrote:
...
> Closures are harder. See http://wiki.tcl.tk/17686 and
> http://wiki.tcl.tk/20662 for a simple approach to creating immutable
> closures (i.e., they capture the current values of variables rather than
> the variables themselves). The problem with mutable variables is that
> they have no natural string rep (e.g., if they are proc-local), so hard
> to represent as a Tcl_Obj. The best I've come up with so far is to use
> fully-qualified global/namespace variables as sort of references, but
> you then need to deal with lifetime management.

Sorry for the double post, but I should have mentioned that another
approach to mutable closures, used in the dictutils package, is to pass
around a lambda and a dictionary representing the environment
separately. When the lambda is executed it does something like a [dict
with], which allows the proc to update the dictionary and the new
version can then be passed on. It requires cooperation from the caller
of the lambda, but otherwise is a decent solution.

-- Neil

EL wrote:
> Gerald W. Lester schrieb: 
>
> I thought for TIP's it would be necessary to have a reference
> implementation, or at least a more concrete specification how it could
> be achieved?
> That I don't have, unfortunately... the text in the request is the only
> thing I can actually specify and I guess this is too vague for a TIP, or
> no?

No you do not need it -- but it sure helps.


--
+--------------------------------+---------------------------------------+
| Gerald W. Lester                                                       |
|"The man who fights for his ideals is the man who is alive." - Cervantes|
+------------------------------------------------------------------------+

EL wrote:

> I would like to request that [proc] becomes a data type in addition to a
> command, so that procs can be
> - created on the fly, capturing the current environment and procedure stac
k
> - created anonymously, [...]
> - returned by other proc's
> - given as arguments to other procs [...]
> Is it possible to have [proc] as a Tcl_Obj* type?
> [... earlier ...]
> How realistic is this, and how desireable? Is it possible at all, in a
> certain time frame and what would this time frame be, approximately?

In Tcl: quite desirable (you're not the first to ask
for this set of features), but not very realistic
(which is why it hasn't been implemented yet).

See TIP#187 ("Procedures as values") and TIP#196 ("Tcl
commands as values"), for some earlier attempts at this,
and TIP#194 ("Procedures as values via 'apply'"),
which is what finally worked.

TIP#194 does *most* of what you ask for -- anonymous
procedures that can be passed to and returned from
other procedures -- but it can't capture the current
environment or procedure stack.

Several strategies have been proposed to implement
various aspects of lexical scoping and capturing
continuations.  None have been workable.  Some
were fundamentally incompatible with Tcl's EIAS semantics;
others were unworkable with Tcl's current implementation;
and the rest have been feasible but deemed not worth
the effort.  Tcl just ain't Scheme.

However, Tcl does have pretty good support for the
"point-free" or "combinator-based" style of higher-order
functional programming; informally this is "Procedures
as Values via Command Prefixes".  No TIP# for this one,
since Tcl has supported it pretty much all along
(although TIP#174 ("Math operators as commands"), TIP#112
("Ensembles are Namespaces are Commands"), and TIP#293
("Splat operator") have a nice synergy with this technique
that makes it work even better).

With this style you don't need lexical scope or call/cc,
so the fact that they're impossible in Tcl doesn't matter
all that much.  And if you don't need full-blown
Scheme-style "real" lambda expressions, TIP#194's
"fake" lambdas are often suitable.


--Joe English

EL wrote:
> I thought for TIP's it would be necessary to have a reference
> implementation, or at least a more concrete specification how it could
> be achieved?

No, but we won't (usually) vote on one until there is an implementation.

Donal.

On 3 Apr., 07:46, Joe English <jengl...@flightlab.com> wrote:

Thanks a lot for the TIP hints. I think TIP#187 comes pretty close to
my feature request. (Salvatore Sanfilippo also implemented the sugar
macro system, which I think is also a great enhancement of the Tcl
language).

> Several strategies have been proposed to implement
> various aspects of lexical scoping and capturing
> continuations.  None have been workable.  Some
> were fundamentally incompatible with Tcl's EIAS semantics;
> others were unworkable with Tcl's current implementation;
> and the rest have been feasible but deemed not worth
> the effort.  Tcl just ain't Scheme.

Continuing the ideas of TIP#187... I guess that the "lambda" command
could have clientData associated in a Tcl_Obj*. Wouldn't it be
possible to capture variables and their current values at execution
time in the environment where the lambda is created - and associate
appropriate variables in the lambda's body before this is executed?
This could maybe be a way to implement lexical scoping?

Just an idea..


Eckhard