Home > Archive > Functional > May 2007 > kicking my servers all day long...
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
kicking my servers all day long...
|
|
|
| I'm writing a distributed, finite differences heat diffusion
simulation in Erlang. My idea about how to do it goes like this:
a) break the big grid into many little grids
b) assign the grids to individual servers, and connect each server
to those
servers with adjacent grids.
c) for each time step, have the servers evolve their state and
then spawn a
a new server with the updated data
I have sample code which models (c) as updating an int every 50
milliseconds -- it's posted on pastie:
http://pastie.caboo.se/62924
Although (c) is conceptually simple, I'm concerned it may be a source
of evil performance problems -- I have to spawn bazillions of servers,
over and over and over again! Is there another way to do it? What are
some other approaches to distributed, finite differences computing in
Erlang?
| |
| Jon Harrop 2007-05-19, 10:05 pm |
| jsnX wrote:
> What are some other approaches to distributed, finite differences
> computing in Erlang?
This is OT but, if this is uniform subdivision then F# might be a good
alternative because it sports very fast array performance and parallelism.
--
Dr Jon D Harrop, Flying Frog Consultancy
The F#.NET Journal
http://www.ffconsultancy.com/produc...journal/?usenet
| |
|
| On May 19, 10:52 am, Jon Harrop <j...@ffconsultancy.com> wrote:
> jsnX wrote:
>
> This is OT but, if this is uniform subdivision then F# might be a good
> alternative because it sports very fast array performance and parallelism.
Well, I'd have the same problem in F#, wouldn't I? How is F# for
distributed apps, anyhow?
| |
| Jon Harrop 2007-05-19, 10:05 pm |
| jsnX wrote:
> Well, I'd have the same problem in F#, wouldn't I?
I suspect you wouldn't phrase your solution in terms of spawning servers.
That sounds like a very purely functional approach to a problem that may be
much better suited to an impure solution.
> How is F# for distributed apps, anyhow?
I've heard of some work on this, using Alchemi. I've only done shared memory
concurrency myself, and it works beautifully. :-)
--
Dr Jon D Harrop, Flying Frog Consultancy
The F#.NET Journal
http://www.ffconsultancy.com/produc...journal/?usenet
| |
|
| On May 19, 2:40 pm, Jon Harrop <j...@ffconsultancy.com> wrote:
> jsnX wrote:
>
> I suspect you wouldn't phrase your solution in terms of spawning servers.
> That sounds like a very purely functional approach to a problem that may be
> much better suited to an impure solution.
> ...
> I've only done shared memory concurrency myself, and it works beautifully.
Well, it *is* a purely functional approach -- I figured any kind of
'shared memory' approach would be of no value in a distributed
environment. Could you please outline an alternative approach to this
problem, in the language of your choice?
| |
| Joachim Durchholz 2007-05-19, 10:05 pm |
| jsnX schrieb:
> I'm writing a distributed, finite differences heat diffusion
> simulation in Erlang. My idea about how to do it goes like this:
> a) break the big grid into many little grids
> b) assign the grids to individual servers, and connect each server
> to those
> servers with adjacent grids.
> c) for each time step, have the servers evolve their state and
> then spawn a
> a new server with the updated data
Not sure what exactly you mean with "server". Seems to map to "real
hardware" in (a) and (b), and to "process" in (c).
Continuing on that assumption, I'd slightly change (c): don't respawn
the process, have each process keep a clock and a current state.
Submit a timestamp with each message that distributes knowledge about
temperature data. Have each receiving process accept only messages that
have the "right" timestamp values, leaving other messages intact (these
might relate to moments that are farther down into the future).
You can easily have tens of thousands of processes in Erlang, so it's a
reasonable approach to associate an Erlang process to each grid node,
and have each process communicate with the grid neighbours. Just don't
create and destroy nodes - while the Erlang runtime should be optimized
for more process creation and destruction than the average process
library, it's probably not designed for processes that process just a
very small number of processes before dieing.
Regards,
Jo
| |
|
| On May 19, 3:34 pm, Joachim Durchholz <j...@durchholz.org> wrote:
> jsnX schrieb:
>
>
> Not sure what exactly you mean with "server". Seems to
> map to "real hardware" in (a) and (b), and to "process"
> in (c).
Actually, these 'servers' are all processes. I'd run one 'server' per
'node' (in Erlang, a 'node' is a running instance of the VM) and then
I'd distribute my 'nodes' over several machines -- though when
testing, I'll keep them all on one machine.
> Continuing on that assumption, I'd slightly change (c): don't
> respawn the process, have each process keep a clock and a
> current state.
The clock is a good idea -- I hadn't thought of that. As for the the
current state, that's all fine well and good, but what do I do when
the state changes? Erlang does not admit mutable variables.
I seem to be getting a lot of comments suggesting I need shared
memory, state variables and similar things -- whereas I came to
comp.lang.functional to find out the FP way to do this problem, not
necessarily the most widely accepted way.
| |
| Joachim Durchholz 2007-05-20, 4:13 am |
| jsnX schrieb:
>
> The clock is a good idea -- I hadn't thought of that. As for the the
> current state, that's all fine well and good, but what do I do when
> the state changes? Erlang does not admit mutable variables.
It does. There are per-process variables that can be mutated.
There's also a persistent database called Mnesia which gives you even
more mutable storage.
I don't think that you need these though. IIRC Erlang's process model is
"process the message and send any messages, then call the next
incarnation of the process-receiving function"; the process-receiving
function then gets the process state in the form of parameters.
> I seem to be getting a lot of comments suggesting I need shared
> memory, state variables and similar things -- whereas I came to
> comp.lang.functional to find out the FP way to do this problem, not
> necessarily the most widely accepted way.
Erlang is hybrid: Process stuff is referentially intransparent, but
there's usually a function that computes the next process state from the
current one, and that function uses just referentially transparent code.
The referentially intransparent things that you do in Erlang are (in
decreasing order of commonality):
* Receive and send messages
* Access Mnesia
* Access a process variable
* (Probably some other things I don't remember now)
Note that I haven't used Erlang for a while now, so details in the above
may be missing or inaccurate.
Regards,
Jo
| |
| Jon Harrop 2007-05-20, 7:06 pm |
| jsnX wrote:
> Well, it *is* a purely functional approach -- I figured any kind of
> 'shared memory' approach would be of no value in a distributed
> environment. Could you please outline an alternative approach to this
> problem, in the language of your choice?
Joachim basically beat me to the punch. I was going to say, rather than
respawning a replacement, just make the necessary internal state mutable
and update it. You should be able to maintain the appearance of pure
functionality from the outside.
--
Dr Jon D Harrop, Flying Frog Consultancy
The F#.NET Journal
http://www.ffconsultancy.com/produc...journal/?usenet
| |
|
|
|
|
|
|
|