Fortran - Parallel Programing P.P.

Author

Parallel Programing P.P.

Filippos Louis

2005-04-01, 8:57 am

Hello all

I am new in FORTRAN programming and I heard somewhere about parallel
programming! I have written a program that is quite slow (seismic modeling)
and I wonder what can P.P. do for my program and how difficult is to modify
my code to be a parallel code(?). Sorry for my English!

Thank you in advance for your consideration to this matter.

--

Richard Edgar

2005-04-01, 8:57 am

Filippos Louis wrote:

> I am new in FORTRAN programming and I heard somewhere about parallel
> programming! I have written a program that is quite slow (seismic modeling)
> and I wonder what can P.P. do for my program and how difficult is to modify
> my code to be a parallel code(?). Sorry for my English!

There are three common ways to achieve this. In order of increasing
difficulty (but better performance _if done right_):

1) See if your compiler has autoparallelisation
2) Use OpenMP to parallelise your code
3) Use MPI to parallelise your code

Option 1 is obviously the easiest - flick a switch, and you have a
parallel code. However, the performance is unlikely to be especially
good. It relies on the compiler being smart, and even if the compiler is
smart, it still has to be cautious. If the compiler isn't _certain_ that
a section of code can be run in parallel, then it will be forced to run
it in serial.

Option 2 is a bit more difficult, although I don't think it's
particularly hard. OpenMP is a set of compiler directives which let you
say "run this bit in parallel, sharing these variables." This is often a
good way to get modest speed ups on an existing code. However, it
requires shared memory machines, and nCPU > 4 tend to get very expensive
very fast.

Option 3 is the most difficult, but when done properly, is likely to
give the best performance (a fact not unrelated to it being the most
difficult to do....). MPI relies on the programmer explicitly passing
data between computational nodes (hence avoiding the need for shared
memory, making large CPU count clusters cheap). It is probably best to
design a code to be MPI from the start (although others may weigh in
with their own opinions on that).

In all cases though, your best starting point is a fast serial code.
Throwing processors at a problem isn't going to help if your basic code
does silly things (e.g. are all arrays accessed in a cache coherent
manner whenever possible? Have you minimised the number of branches in
the tightest loops?). Are you sure that you've done all you can to speed
up the serial version. I would caution against trying to pull too many
tricks to speed things up since
a) Optimisers are fairly smart, and if you try to be smarter you may
confuse them
b) What works on one architecture may not work on another (or even a
different compiler on the same architecture)
c) Some optimisations may force the code to be serial

Sorry I can't be more helpful, but you've asked a very general question
with very little about the specifics of your code. If you are new to
Fortran (maybe even to programming itself?), then I really wouldn't
worry about parallelising the code yet. Learn how to get the serial
version working faster first (e.g. use the profiler to tell you which
sections of the code take the most time).

HTH,

Richard

Phillip Helbig---remove CLOTHES to reply

2005-04-01, 8:57 am

In article <d2jbdf$bn2$1@news.su.se>, Richard Edgar <rge21@astro.su.se>
writes:

> Filippos Louis wrote:
>
>
> There are three common ways to achieve this. In order of increasing
> difficulty (but better performance _if done right_):
>
> 1) See if your compiler has autoparallelisation
> 2) Use OpenMP to parallelise your code
> 3) Use MPI to parallelise your code

What about HPF?

Richard Edgar

2005-04-01, 3:59 pm

Phillip Helbig---remove CLOTHES to reply wrote:

>
> What about HPF?

I had had the impression that that was losing out, caught between OpenMP
and MPI. However, a bit of searching suggests that that may not be the
case. I should probably have mentioned it (although I have never used it).

For completeness, there is also PVM, but I have no experience with that
at all.

Richard

Bart Vandewoestyne

2005-04-01, 8:57 pm

In article <d2jbdf$bn2$1@news.su.se>, Richard Edgar wrote:
>
> (e.g. use the profiler to tell you which sections of the
> code take the most time).

Wile we're at it... I'm currently using NagWare's f95 and g95 to compile
my code. What profiler tools can I use with these compilers?

Regards,
Bart

--
"Share what you know. Learn what you don't."

Ian Brown

2005-04-02, 8:57 am

>>>>I am new in FORTRAN programming and I heard somewhere about parallel
>
> I had had the impression that that was losing out, caught between OpenMP
> and MPI. However, a bit of searching suggests that that may not be the
> case. I should probably have mentioned it (although I have never used it).
>
> For completeness, there is also PVM, but I have no experience with that at
> all.

If the OP has a research project of interest to mankind, then one of
the most powerful 'parallel programming' techinques is to get others
to run variations of his model on their processors, Boinc's probably
the best known option: http://boinc.berkeley.edu/ they say it will
work with fortran.

Ian.

Richard Edgar

2005-04-02, 8:57 am

Ian Brown wrote:

> If the OP has a research project of interest to mankind, then one of
> the most powerful 'parallel programming' techinques is to get others
> to run variations of his model on their processors, Boinc's probably
> the best known option: http://boinc.berkeley.edu/ they say it will
> work with fortran.

True.... I should also have mentioned that the best speed up is obtained
by running two copies of the program, each with slightly different
initial conditions... 100% efficiency every time :-)

Richard

Stig Kildegård Andersen

2005-04-06, 12:21 pm

> True.... I should also have mentioned that the best speed up is obtained
> by running two copies of the program, each with slightly different initial
> conditions...

Sometimes, but certainly not always - sometimes two copies of a program can
benefit from sharing data during the computations. When, as You mention, two
programs are run with slightly different input parameters one could, for
instance, imagine determining by trial and error at run time if it is
beneficial to share expensive Jacobians for a Newton-Raphson solver between
the two or more concurrent programs. If work on an application where this
works great.

For this special case you could also make a sequential program that works on
several instances of your problem simultanously and share data between the
instances. But then you loose the ability to run the multiple instances of
your problem on more than one processor.

But all of this is of no use if the OP is interested in speeding up the
computations of a single solution.

I know nothing about seismic modelling but if a large part of the CPU-time
is spent solving large systems of linear equations or some similarly
standard task, then maybe the program can be succesfully parallelised simply
by substituting some sequential subroutines in the program with parallel
library subroutines? This could be the most desirable way to parallelise the
program since the OP mentions that He is new to programming in Fortran and
has only heard about parallel programming.

Kind Regards,
Stig Kildegård

Sponsored Links