Lisp - sb-thread programming tutorial?

Author

sb-thread programming tutorial?

David Steuber

2004-04-28, 3:34 am

I've glanced through the SBCL manual for things outside the ANSI
spec. For instance, I am interested in threaded programming using
sb-thread. While I am sure that apropos and describe are good for
reference purposes (or at least brief reminders), I am looking for a
gentler introduction to threaded programming with SBCL's sb-thread.

I've done simple threaded programming with the Win32 API and Java, but
that was a while back. I know in general about things like
synchronization/serialization, race hazards, and deadlocks. What I
don't know is how these are handled with sb-thread.

To quote a line from the SBCL manual:

"Dynamic bindings to symbols are per-thread. Signal handlers are
per-thread."

Although I have not seen how Lisp does signal handling, it is nice to
know that the handler will be called in the same thread that the
signal was raised. That at least means that you can pretend the
thread is its own little world. I'm not quite sure about the dynamic
bindings. My interpretation is that

(defvar *foo* "bar")

means that every thread has *foo* with the initial value of "bar". If
a thread does

(setf *foo* "baz")

then /that/ thread and /only/ that thread has *foo* bound to "baz".
If this is the case, then that is great. I have a sort of thread
local storage thing going on for free.

That then leads me to wonder how to create a shareable object such as
a hash table (or an a-list) that has serialized access from all the
threads that need to touch it in some way.

Then there are other threading issues like waiting on objects (Java
style) and other scheduling issues. For example, if thread creation
has too much overhead for a given purpose, I would want to pre-spawn a
number of threads and have them sleeping in a queue to be woken up as
needed to handle a request and then go back into the queue when
finished rather than exiting.

I still consider myself very much a Lisp beginner (neolisper?). I
kind of skipped through macros and clos in Graham's ACL book. I
figured I could come back to that stuff. I am hoping that I have
enough ground work to handle seperate concepts such as threads and
also sockets and other things.

--
I wouldn't mind the rat race so much if it wasn't for all the damn cats.

Thomas F. Burdick

2004-04-28, 4:37 am

David Steuber <david@david-steuber.com> writes:

> To quote a line from the SBCL manual:
>
> "Dynamic bindings to symbols are per-thread. Signal handlers are
> per-thread."
>
> Although I have not seen how Lisp does signal handling, it is nice to
> know that the handler will be called in the same thread that the
> signal was raised. That at least means that you can pretend the
> thread is its own little world. I'm not quite sure about the dynamic
> bindings. My interpretation is that
>
> (defvar *foo* "bar")
>
> means that every thread has *foo* with the initial value of "bar". If
> a thread does
>
> (setf *foo* "baz")
>
> then /that/ thread and /only/ that thread has *foo* bound to "baz".
> If this is the case, then that is great. I have a sort of thread
> local storage thing going on for free.

No no no! What you're doing with SETF is, as its name implies,
setting the variable. Binding is different, it creates a new variable
(name->value mapping). LET and LAMBDA bind variables. You do get
thread-local storage, but with dynamic binding; so:

(defvar *foo* "global")

(let ((*foo* "thread-local"))
...)

In the body of the LET, and all the code paths called from there,
*FOO* has the value "thread-local". Not only that, but LET creates a
new binding, so any setting of *FOO* affects that binding, not the
global one. No matter what, that dynamic contour cannot affect the
global value. If *FOO* isn't bound, you access its global binding,
which would be shared by all threads.

--
/|_ .-----------------------.
,' .\ / | No to Imperialist war |
,--' _,' | Wage class war! |
/ / `-----------------------'
( -. |
| ) |
(`-. '--.)
`. )----'

Gabor Melis

2004-04-28, 7:32 am

David Steuber <david@david-steuber.com> wrote in message news:<87n04w7fpv.fsf@david-steuber.com>...
> I've glanced through the SBCL manual for things outside the ANSI
> spec. For instance, I am interested in threaded programming using
> sb-thread. While I am sure that apropos and describe are good for
> reference purposes (or at least brief reminders), I am looking for a
> gentler introduction to threaded programming with SBCL's sb-thread.
>
> I've done simple threaded programming with the Win32 API and Java, but
> that was a while back. I know in general about things like
> synchronization/serialization, race hazards, and deadlocks. What I
> don't know is how these are handled with sb-thread.
>
> To quote a line from the SBCL manual:
>
> "Dynamic bindings to symbols are per-thread. Signal handlers are
> per-thread."
>
> Although I have not seen how Lisp does signal handling, it is nice to
> know that the handler will be called in the same thread that the
> signal was raised. That at least means that you can pretend the
> thread is its own little world. I'm not quite sure about the dynamic
> bindings. My interpretation is that
>
> (defvar *foo* "bar")
>
> means that every thread has *foo* with the initial value of "bar". If
> a thread does
>
> (setf *foo* "baz")
>
> then /that/ thread and /only/ that thread has *foo* bound to "baz".
> If this is the case, then that is great. I have a sort of thread
> local storage thing going on for free.

My impression was that (setf *foo* "baz") modifies the global binding
of *foo* and since the global binding is shared by threads it is
suitable for inter-thread communication. However, if *foo* is rebound
with let in a thread then that binding is local to that particular
thread. Is that right?

Gabor

Tim Bradshaw

2004-04-28, 10:48 am

David Steuber <david@david-steuber.com> wrote in message news:<87n04w7fpv.fsf@david-steuber.com>...
that
>
> (defvar *foo* "bar")
>
> means that every thread has *foo* with the initial value of "bar". If
> a thread does
>
> (setf *foo* "baz")
>

That's unlikely to be the case. What *is* likely to be the case is
that if you *bind* a special in a thread, then that binding is
per-thread. If you don't bind it, then the above assignment will
probably alter the global binding, or the parent thread's binding, or
something like that.

Some threading implementations (Allegro? I'm not sure) also have a set
of default special bindings which are established on thread creation,
which you might be able to add to.

--tim

Duane Rettig

2004-04-28, 1:51 pm

tfb+google@tfeb.org (Tim Bradshaw) writes:

> David Steuber <david@david-steuber.com> wrote in message news:<87n04w7fpv.fsf@david-steuber.com>...
> that
>
> That's unlikely to be the case. What *is* likely to be the case is
> that if you *bind* a special in a thread, then that binding is
> per-thread. If you don't bind it, then the above assignment will
> probably alter the global binding, or the parent thread's binding, or
> something like that.
>
> Some threading implementations (Allegro? I'm not sure) also have a set
> of default special bindings which are established on thread creation,
> which you might be able to add to.

Yes. See

http://www.franz.com/support/docume...-bindings_s.htm

--
Duane Rettig duane@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182

David Steuber

2004-04-28, 6:57 pm

tfb@famine.OCF.Berkeley.EDU (Thomas F. Burdick) writes:

> No no no! What you're doing with SETF is, as its name implies,

I keep getting binding and setting mixed up. I'll get it after enough
hammering on my skull.

> setting the variable. Binding is different, it creates a new variable
> (name->value mapping). LET and LAMBDA bind variables. You do get
> thread-local storage, but with dynamic binding; so:
>
> (defvar *foo* "global")
>
> (let ((*foo* "thread-local"))
> ...)
>
> In the body of the LET, and all the code paths called from there,
> *FOO* has the value "thread-local". Not only that, but LET creates a
> new binding, so any setting of *FOO* affects that binding, not the
> global one. No matter what, that dynamic contour cannot affect the
> global value. If *FOO* isn't bound, you access its global binding,
> which would be shared by all threads.

Ok, so (defvar *foo* "global") is shared and access needs to be
protected with a mutex. Or is CL clever enough to provide mutexed
access for me?

How about this example (from Graham as best I can recall it):

(let ((count 0))
(defun stamp () (1+ count))
(defun reset () (setf count 0)))

Each thread should have its own copy of count for the purposes of
calling stamp and reset, correct? Or are things not so simple?

--
I wouldn't mind the rat race so much if it wasn't for all the damn cats.

David Steuber

2004-04-29, 12:29 am

David Steuber <david@david-steuber.com> writes:

> (let ((count 0))
> (defun stamp () (1+ count))
> (defun reset () (setf count 0)))

Actually, I think that should be (defun stamp () (incf count)).

--
I wouldn't mind the rat race so much if it wasn't for all the damn cats.

Thomas F. Burdick

2004-04-29, 12:29 am

David Steuber <david@david-steuber.com> writes:

> tfb@famine.OCF.Berkeley.EDU (Thomas F. Burdick) writes:
>
>
> I keep getting binding and setting mixed up. I'll get it after enough
> hammering on my skull.

What should eventually click is that binding creates new variables,
and setting gives new values to existing variables. A variable is
just a mapping from a name to a value.

Where it gets a little more complex is there are two types of
variables: lexical variables, and dynmaic variables. Lexical
variables (ie, the mapping from a name to a place to store/retrieve
values) exist only at compile time. Dynamic variables exist at
runtime.

> Ok, so (defvar *foo* "global") is shared and access needs to be
> protected with a mutex. Or is CL clever enough to provide mutexed
> access for me?

Nope, you gotta do it yourself.

> How about this example (from Graham as best I can recall it):
>
> (let ((count 0))
> (defun stamp ()
(incf count))
> (defun reset () (setf count 0)))
>
> Each thread should have its own copy of count for the purposes of
> calling stamp and reset, correct? Or are things not so simple?

Things are quite simple here, but that's a lexical variable named
count. So, it's resolved by the compiler, into a single place at
compile time. For thread-local variables, you want to create a
binding at runtime, sometime after the spawning of the thread. That's
exactly what dynamic binding does.

--
/|_ .-----------------------.
,' .\ / | No to Imperialist war |
,--' _,' | Wage class war! |
/ / `-----------------------'
( -. |
| ) |
(`-. '--.)
`. )----'

Fred Gilham

2004-04-29, 12:29 am

David Steuber <david@david-steuber.com> writes:

> How about this example (from Graham as best I can recall it):
>
> (let ((count 0))
> (defun stamp () (1+ count))
> (defun reset () (setf count 0)))
>
> Each thread should have its own copy of count for the purposes of
> calling stamp and reset, correct? Or are things not so simple?

No. The values of lexical variables are always obtained from the
lexically apparent bindings. That's why they're called lexical
variables.

Note that the code above probably didn't do what you meant it to do;
you want

(let ((count 0))
(defun stamp () (incf count)) ;; <<<<----
(defun reset () (setf count 0)))

instead.

It might help to think as follows.

A dynamic variable obeys a stack protocol; it gets its binding by
looking up the call stack to find a stack frame where it is bound.
But since threads can be considered to have cloned stacks, they each
have their own copy of the binding (or memory location) of the dynamic
variable.

A lexical variable, on the other hand, obeys a "lexical protocol",
which means that you can find the applicable binding for the variable
by looking at the enclosing binding forms. The innermost binding form
that binds the variable in question is the one that creates the
binding for that variable, regardless of the call stack. You can even
think of it as creating a pointer to a memory location, and
hard-coding the pointer into the code that references the variable.

So in the above example, you can think of a pointer to the memory
location 1234 which holds the value of count. So both STAMP and RESET
have the number 1234, the address where the value of count is stored,
hard-coded into them so they will always reference that same memory
location.

This isn't exactly what really happens, but like I said, you can think
of it that way.

Here's some test code that runs under CMUCL that will illustrate what
is going on with closures and threads:

(let ((count 0))
(defun stamp () (incf count))
(defun reset () (setf count 0)))

(defun stamp-and-print (pid time)
(loop
(format t "Pid ~A: Count is ~A~%" pid (stamp))
(sleep time)))

(defun reset-it ()
(loop
(reset)
(sleep 10)))

(defun test-it ()
(mp:make-process #'(lambda () (stamp-and-print 1 (random 2))))
(mp:make-process #'(lambda () (stamp-and-print 2 (random 5))))
(mp:make-process #'reset-it))

If you run (test-it) you'll get something like the following:

Pid 2: Count is 1
Pid 1: Count is 2
Pid 1: Count is 3
Pid 1: Count is 4
Pid 1: Count is 5
Pid 2: Count is 6
Pid 1: Count is 7
Pid 1: Count is 8
Pid 1: Count is 9
Pid 1: Count is 10
Pid 2: Count is 11
Pid 1: Count is 12
Pid 1: Count is 13
Pid 1: Count is 1
Pid 1: Count is 2
Pid 2: Count is 3
Pid 1: Count is 4
Pid 1: Count is 5
Pid 1: Count is 6
Pid 1: Count is 7
Pid 2: Count is 8
Pid 1: Count is 9
Pid 1: Count is 10
Pid 1: Count is 11
Pid 1: Count is 12
Pid 2: Count is 1
Pid 1: Count is 2
Pid 1: Count is 3
Pid 1: Count is 4
Pid 1: Count is 5
Pid 2: Count is 6
Pid 1: Count is 7
Pid 1: Count is 8
Pid 1: Count is 9
Pid 1: Count is 10
Pid 2: Count is 11
Pid 1: Count is 12
Pid 1: Count is 13
Pid 1: Count is 1

etc.

--
Fred Gilham gilham@csl.sri.com
[My tutors] got bored sooner than I, and laid down a general rule
that all statements about languages had to be in a higher level
language. I thereupon asked in what level of language that rule was
formulated. I got a very bad report. -- J. R. Lucas

Fred Gilham

2004-04-29, 3:01 am

As another example, here's the special variable version with of the
previous example with thread-local bindings. Notice that there seem
to be three separate instances of the *count* variable --- the
instance that (reset) accesses doesn't show up anywhere, but the point
is that the reset-it process has no effect on the other two processes.

(defvar *count* 0)

(defun stamp () (incf *count*))
(defun reset () (setf *count* 0))

(defun stamp-and-print (pid time)
(loop
(format t "Pid ~A: Count is ~A~%" pid (stamp))
(sleep time)))

(defun reset-it ()
(loop
(reset)
(sleep 10)))

(defun test-it ()
(mp:make-process #'(lambda () (stamp-and-print 1 2)) :initial-bindings '((*count* . 0)))
(mp:make-process #'(lambda () (stamp-and-print 2 5)) :initial-bindings '((*count* . 0)))
(mp:make-process #'reset-it))

Here's the run:

Pid 2: Count is 1
Pid 1: Count is 1
Pid 1: Count is 2
Pid 1: Count is 3
Pid 2: Count is 2
Pid 1: Count is 4
Pid 1: Count is 5
Pid 2: Count is 3
Pid 1: Count is 6
Pid 1: Count is 7
Pid 1: Count is 8
Pid 2: Count is 4
Pid 1: Count is 9
Pid 1: Count is 10
Pid 2: Count is 5
Pid 1: Count is 11
Pid 1: Count is 12
Pid 1: Count is 13
Pid 2: Count is 6
Pid 1: Count is 14
Pid 1: Count is 15
Pid 2: Count is 7
Pid 1: Count is 16
Pid 1: Count is 17
Pid 1: Count is 18
Pid 2: Count is 8
Pid 1: Count is 19
Pid 1: Count is 20
Pid 2: Count is 9
Pid 1: Count is 21
Pid 1: Count is 22
Pid 1: Count is 23
Pid 2: Count is 10
Pid 1: Count is 24
Pid 1: Count is 25
Pid 2: Count is 11
Pid 1: Count is 26
Pid 1: Count is 27
Pid 1: Count is 28
Pid 2: Count is 12
Pid 1: Count is 29
Pid 1: Count is 30
Pid 2: Count is 13
Pid 1: Count is 31
Pid 1: Count is 32
Pid 1: Count is 33
Pid 2: Count is 14
Pid 1: Count is 34
Pid 1: Count is 35
Pid 2: Count is 15
Pid 1: Count is 36
Pid 1: Count is 37
Pid 1: Count is 38
Pid 2: Count is 16
Pid 1: Count is 39

--
Fred Gilham gilham@csl.sri.com
....the War between the States established...this principle, that the
federal government is, through its courts, the final judge of its own
powers. -- Woodrow Wilson, explaining who will watch the watchers.

Arthur Lemmens

2004-04-29, 8:11 am

Fred Gilham <gilham@snapdragon.csl.sri.com> wrote:

> A dynamic variable obeys a stack protocol; it gets its binding by
> looking up the call stack to find a stack frame where it is bound.
> But since threads can be considered to have cloned stacks, they each
> have their own copy of the binding (or memory location) of the dynamic
> variable.
>
> A lexical variable, on the other hand, obeys a "lexical protocol",
> which means that you can find the applicable binding for the variable
> by looking at the enclosing binding forms. The innermost binding form
> that binds the variable in question is the one that creates the
> binding for that variable, regardless of the call stack.

One way of looking at this is that the binding of a dynamic variable
is determined by the run-time stack, whereas the binding of a lexical
variable is determined by the compile-time stack.

--
Arthur

Sponsored Links