Fortran - Alternatives for EXTERNAL?

Author

Alternatives for EXTERNAL?

Carlie J. Coats

2005-09-14, 7:00 pm

I'm trying to port an older F77 atmospheric-aerosol code into a new
meteorology/atmospheric chemistry model with rather strict (F90) coding
rules, one of which is: NO COMMONs.

The old code is structured as a "box model" (one call per met-model grid
cell per time step), with a driver subroutine that stores data for that
call in various COMMONs, and then calls a complex (> 28000 LOC, 5 levels
deep) hierarchy of subroutines that act upon the data in those COMMONs.

In the new code, one must be able to call this "box model" in parallel,
so the following structure suggests itself:

SUBROUTINE AERO
...
<variables from COMMONs turned into local variables>
...
CONTAINS
<hierarchy of worker routines>
END SUBROUTINE AERO

The problem is that there are a pair of third-level routines, two of
whose arguments are two routines selected from a set of 24 fourth-level
routines... and in this structure, these fourth-level routines can't
be declared as EXTERNAL, e.g.:

SUBROUTINE FOO2
EXTERNAL CALCQ1A, CALCQ2A, CALCQ3A, CALCQ4
...
IF ( ... )
CALL CALCMDRH (RH, DRMI1, DRNH4HS4, CALCQ1A, CALCQ2A)
...
ELSE IF ( ... )
CALL CALCMDRH (RH, DRMI1, DRNH4HS4, CALCQ1A, CALCQ3A)
...
ELSE
CALL CALCMDRH (RH, DRMI1, DRNH4HS4, CALCQ1A, CALCQ4)
...
END SUBROUTINE FOO2
...

SUBROUTINE CALCMDRH (RHI, RHDRY, RHLIQ, DRYCASE, LIQCASE)
EXTERNAL DRYCASE, LIQCASE
...

What alternatives do I have? ...or how do I declare CALCQ1A, etc.

--

Carlie J. Coats, Jr. carlie.coats@baronams.com
Environmental Modeling Center
Baron Advanced Meteorological Systems, LLC.

Richard E Maine

2005-09-14, 7:00 pm

In article <26ZVe.5366$XO6.1403@trnddc03>,
"Carlie J. Coats" <carlie@jyarborough.com> wrote:

> In the new code, one must be able to call this "box model" in parallel,
> so the following structure suggests itself:
>
> SUBROUTINE AERO
> ...
> <variables from COMMONs turned into local variables>

.....

I might well misunderstand what's going on here, but I don't
particularly see how this facilitates calling "in parallel". Perhaps
that's because I misunderstand what you mean by the "in parallel".
Before seeing this proposed structure, I presumed that you meant that
multiple copies of the model might simultaneously be active, perhaps not
literally parallel processing, but each copy having saved state that
needed to be maintained.

In that case, one approach is to take the variables previously in COMMON
and make them all components of a derived type. Then you can have
multiple variables of that derived type. A di

vantage of that is that
the code gets "wordier" because you need to tack on something like X%
(if, say, X is the appropriate variable or maybe a pointer to it) in
front of every reference.

But this is almost entirely about data structure and very about the
subroutine structure you are showing, so I might well have completely
missed the point of your comment about "parallel".

> and in this structure, these fourth-level routines can't
> be declared as EXTERNAL, e.g.:
>
> SUBROUTINE FOO2
> EXTERNAL CALCQ1A, CALCQ2A, CALCQ3A, CALCQ4
> ...
> IF ( ... )
> CALL CALCMDRH (RH, DRMI1, DRNH4HS4, CALCQ1A, CALCQ2A)
> ...
....
> What alternatives do I have? ...or how do I declare CALCQ1A, etc.

I was a bit

as to what you mean by "can't be declared as
external" but I think I got it figured out. You can't declare them as
external because they are internal. Declaring them as external would be
a lie. In fact, if I see what is going on, declaring them as external
should compile just fine... but it will compile as a reference to an
external procedure of that name. You don't have such an external
procedure (internal is not the same thing as external), so the program
will probably fail to link when it can't find one.

Your problem is more than just declaration. The standard doesn't allow
you to pass an internal procedure as an actual argument. Period. No
declaration is going to fix that. (I'll pass on explaining why this
rule, though I'll note that allowing this has been discussed several
times. I think there might be a compiler that does it as an extension
now, but it is definitely nonportable today).

I suggest that one fix is to put this stuff in a module. (But then,
that's a good fix for lots of things. :-)) Instead of having your
variables be local variables in subroutine aero, have them be module
variables. Then instead of having all the worker procedures be internal
to aero, make them module procedures (and make aero one also). You can
pass a module procedure as an actual argument. And you don't declare it
at all. The compiler already knows about the module procedures; if you
try something like an external declaration, you'll be back in the mode
of making the compiler look for a nonexistent external procedure (module
is not the same thing as external). Just do it with no declaration at
all.

But this module approach addresses only the bit about how to organize
the procedures. If you were hoping to facilitate parallelism by making
the data local in aero, and possibly then forcing stack allocation of
the local variables, my suggestion to move it into a module will
probably mess you up. You aren't likely to be able to force stack
allocation of module variables. In that case, I suggest you still stick
with the module, but also go with the above-mention data as components
of an object. That will allow you to actually make the multiple copies
explicit in the code rather than depending on implementation behavior
like stack allocation.

--
Richard Maine | Good judgment comes from experience;
email: my first.last at org.domain | experience comes from bad judgment.
org: nasa, domain: gov | -- Mark Twain

Steve Lionel

2005-09-14, 7:00 pm

On Wed, 14 Sep 2005 17:29:34 GMT, "Carlie J. Coats" <carlie@jyarborough.com>
wrote:

>I'm trying to port an older F77 atmospheric-aerosol code into a new
>meteorology/atmospheric chemistry model with rather strict (F90) coding
>rules, one of which is: NO COMMONs.
>
>The old code is structured as a "box model" (one call per met-model grid
>cell per time step), with a driver subroutine that stores data for that
>call in various COMMONs, and then calls a complex (> 28000 LOC, 5 levels
>deep) hierarchy of subroutines that act upon the data in those COMMONs.

Sounds like a job for module variables. Declare the variables in a module,
use the module everywhere you would have included the common, and nothing else
really needs to change.

Steve Lionel
Software Products Division
Intel Corporation
Nashua, NH

User communities for Intel Software Development Products
http://softwareforums.intel.com/
Intel Fortran Support
http://developer.intel.com/software/products/support/

Michael Metcalf

2005-09-14, 7:00 pm

"Carlie J. Coats" <carlie@jyarborough.com> wrote in message
news:26ZVe.5366$XO6.1403@trnddc03...
>
> The problem is that there are a pair of third-level routines, two of
> whose arguments are two routines selected from a set of 24 fourth-level
> routines... and in this structure, these fourth-level routines can't
> be declared as EXTERNAL, e.g.:
>

I assume that is a style restriction as it's allowed in the standard. You
could put the external procedures into a module that is USEd where
appropriate (MR&C, Section 5.12).

Regards,

Mike Metcalf

Carlie J. Coats

2005-09-14, 7:00 pm

Richard E Maine wrote:
> In article <26ZVe.5366$XO6.1403@trnddc03>,
> "Carlie J. Coats" <carlie@jyarborough.com> wrote:
>
>
>
>
> ....
>
> I might well misunderstand what's going on here, but I don't
> particularly see how this facilitates calling "in parallel".
[snip...]
> But this module approach addresses only the bit about how to organize
> the procedures. If you were hoping to facilitate parallelism by making
> the data local in aero, and possibly then forcing stack allocation of
> the local variables, my suggestion to move it into a module will
> probably mess you up....

I guess I didn't state my problem clearly -- this last is the idea:
the new calling model is a 3-D-grid met model that is parallelized
on the columns/rows/levels of that grid, and that needs to call AERO
on every cell of that 3-D grid in parallel.

I had hoped to make the promiscuously-shared but single-cell COMMON
variables of AERO's worker-routine hierarchy into (stack-allocated)
local variables in AERO, and the worker routines into internal
subroutines, and be done with it. But there's this indirect-call
mess, with about 4000 lines of the 24 callees, 500 lines of the two
routines that do the indirect calls (and that have the callees as
arguments), and 3000 lines of the 35 routines that call them.

Completely re-writing these messy 7500 lines of code is way beyond
the budget for the project ;-(

Does anyone have any better ideas?

Thanks!

-- Carlie

Richard E Maine

2005-09-14, 7:00 pm

In article <HzZVe.3618$Ap4.395518@twister.nyc.rr.com>,
"Michael Metcalf" <michaelmetcalf@compuserve.com> wrote:

> "Carlie J. Coats" <carlie@jyarborough.com> wrote in message
> news:26ZVe.5366$XO6.1403@trnddc03...
>
> I assume that is a style restriction as it's allowed in the standard.

That was my first thought. But on about my third reading I figured out
(I think) that the reason it isn't allowed is that the procedures in
question are internal instead of external.

> You
> could put the external procedures into a module that is USEd where
> appropriate (MR&C, Section 5.12).

But that's still a good fix IMO.

--
Richard Maine | Good judgment comes from experience;
email: my first.last at org.domain | experience comes from bad judgment.
org: nasa, domain: gov | -- Mark Twain

Richard E Maine

2005-09-14, 7:00 pm

In article <3u_Ve.1248$si2.24@trnddc06>,
"Carlie J. Coats" <carlie@jyarborough.com> wrote:

> I had hoped to make the promiscuously-shared but single-cell COMMON
> variables of AERO's worker-routine hierarchy into (stack-allocated)
> local variables in AERO,

Ah. I wasn't too far off then.

> Completely re-writing these messy 7500 lines of code is way beyond
> the budget for the project ;-(

7500 lines isn't too bad. Anyway.... I think my previously mentioned
derived-type object idea can work for this, and isn't nearly as
compiler-dependent as assuming you can force stack allocation of the
local variables.

You do need code changes to make the derived type idea work, but the
code changes don't constitute "rewriting". All you need is to insert x%
in front of every appropriate variable reference (and deal with lines
getting longer as a result). That might be a lot of insertions, but it
is nothing like rewriting. And it can be easily automated, at least if
your variable names are descriptive enough that you aren't likely to
find the same characters as a substring in something else.

Once you have all the variables in question in a single derived type
object, there are lots of ways to go from there. For example, you can
then reasonably pass that single object as an argument if that helps,
whereas you probably would not have found it reasonable to pass a
zillion individual items that way.

--
Richard Maine | Good judgment comes from experience;
email: my first.last at org.domain | experience comes from bad judgment.
org: nasa, domain: gov | -- Mark Twain

Brooks Moses

2005-09-14, 7:00 pm

Carlie J. Coats wrote:
> I had hoped to make the promiscuously-shared but single-cell COMMON
> variables of AERO's worker-routine hierarchy into (stack-allocated)
> local variables in AERO, and the worker routines into internal
> subroutines, and be done with it. But there's this indirect-call
> mess, with about 4000 lines of the 24 callees, 500 lines of the two
> routines that do the indirect calls (and that have the callees as
> arguments), and 3000 lines of the 35 routines that call them.
>
> Completely re-writing these messy 7500 lines of code is way beyond
> the budget for the project ;-(

I think that Richard's idea should not require completely re-writing the
entire 7500 lines, or anything close to it.

First, how many common variables are there? At what levels of the
process do those variables get changed?

I would recommend the following:

* Create a derived type that contains all of the variables currently in
common.

* In the top function, define a variable called COMMONDATA of this
derived type. Put the required information in it.

* To each function or subroutine call, add COMMONDATA to the end (or,
alternately, the beginning) of the list of arguments. If you add it to
the beginning, this can be accomplished by a simple search and replace
for each function name, which is only 61 discrete operations -- or, if
you've got a regex serach-and-replace functionality and do this to all
CALL statements, only 1 discrete operation.

* To each function or subroutine definition, add COMMONDATA to the
appropriate list of arguments, and add a declaration for it. Again,
this only means adding 122 lines, most of which are a repeated pasting
of the same thing over and over.

Then, you have two alternatives.

First, you can write some blocks of code to pack the existing COMMON
variables into COMMONDATA before function calls (if they've been
modified), and to unpack them after. You only need to write each block
once; they look like this:

! Begin COMMONDATA packing:
COMMONDATA.myvara = myvara
COMMONDATA.myvarb = myvarb
...
! End COMMONDATA packing:

! Begin COMMONDATA unpacking:
myvara = COMMONDATA.myvara
myvarb = COMMONDATA.myvarb
...
! End COMMONDATA unpacking:

Then, you copy and paste the unpacking blocks immediately after the
variable declarations of each function, and the packing blocks
immediately before any function call where the variables might have been
modified from the versions in COMMONDATA.

Alternately, depending on how you prefer to take your refactoring risks,
you can instead simply do a search-and-replace through the entire code,
replacing myvara with COMMONDATA.myvara, and so forth. This runs a bit
of risk with regards to line-lengths, but that should be trivial to do
an automated check for, and converting the code from fixed-form to
free-form at the same time (which can be done automatically) will buy
you plenty of extra line length so that won't be a problem.

If the variables never get modified, the former idea is probably a bit
safer to implement (though maybe a bit slower, if the compiler doesn't
optimize away the extra variables). The latter idea should end up with
cleaner and more maintainable code, though, and is probably safer if the
variables do get modified, as there's no risk of them getting out of
sync in the two versions.

- Brooks

--
The "bmoses-nospam" address is valid; no unmunging needed.

Carlie J. Coats

2005-09-14, 7:00 pm

Thanks, Richard and Brooks!

The "single variable of a derived type" idea is one that I think
will work for me.

It will be *much* easier to chase a 127-entry derived type through
this code than it would have been to chase the data flow analysis
through the whole system... or even just the bottom-most 7500+ lines
of it.

-- Carlie

> "Carlie J. Coats" <carlie@jyarborough.com> wrote:
>
>
>
>
> Ah. I wasn't too far off then.
>
>
>
>
> 7500 lines isn't too bad. Anyway.... I think my previously mentioned
> derived-type object idea can work for this, and isn't nearly as
> compiler-dependent as assuming you can force stack allocation of the
> local variables.
>
> You do need code changes to make the derived type idea work, but the
> code changes don't constitute "rewriting". All you need is to insert x%
> in front of every appropriate variable reference (and deal with lines
> getting longer as a result). That might be a lot of insertions, but it
> is nothing like rewriting. And it can be easily automated, at least if
> your variable names are descriptive enough that you aren't likely to
> find the same characters as a substring in something else.
>
> Once you have all the variables in question in a single derived type
> object, there are lots of ways to go from there. For example, you can
> then reasonably pass that single object as an argument if that helps,
> whereas you probably would not have found it reasonable to pass a
> zillion individual items that way.

Richard E Maine

2005-09-14, 7:00 pm

In article <43287EFB.5040100@cits1.stanford.edu>,
Brooks Moses <bmoses-nospam@cits1.stanford.edu> wrote:

> COMMONDATA.myvara = myvara
..........^.......

For standard conformance, % please instead of a period.

Otherwise, yes, this fleshes out "my" idea (which isn't at all original
to me; all I can take credit for is noticing that this seemed to be an
instance of a problem looking for that well-known idea).

--
Richard Maine | Good judgment comes from experience;
email: my first.last at org.domain | experience comes from bad judgment.
org: nasa, domain: gov | -- Mark Twain

Pierre Asselin

2005-09-15, 6:59 pm

Carlie J. Coats <carlie@jyarborough.com> wrote:

In addition to the excellent suggestions you received in this
thread, I would add:

*) Create a small test problem that exercises all the code
but runs quickly --because you will be running it a lot.
*) If possible, use code coverage tools. These will tell you
how many times each line of code was executed, so you can
confirm that your test problem goes through all the code.
Ask in your group about code coverage, maybe someone already
knows. If not, you'll have to stop and read a few compiler
manuals.
*) Run your test problem, check that the results make sense
and save the results.

Do not begin work until you have a test problem. When you do,

*) Start hacking the code: module, derived type, etc.
You'll go through a long phase where it won't even compile,
so keep fixing the typos until it does.
*) If possible, make your changes in stages, one subsystem at
a time.
*) Run the test problem and confirm that the results haven't
changed. Do this every time you have a version that compiles.
If the results are different or if the program crashes,
you introduced a bug in your last round of conversions.
Find it and fix it.
*) When all the code is converted, save it as a milestone.
Then start working on the parallelized version.

--
pa at panix dot com

Carlie J. Coats

2005-09-15, 6:59 pm

Pierre Asselin wrote:
> Carlie J. Coats <carlie@jyarborough.com> wrote:
>
> In addition to the excellent suggestions you received in this
> thread, I would add:
>
> *) Create a small test problem that exercises all the code
> but runs quickly --because you will be running it a lot.

If you go back to the top, you'll find that there was over 28000
lines of code, of which 7500 were directly involved with this issue
of indirect calls to EXTERNALs, either as caller, callee, or arguments.

Not exactly "small" ;-(

> *) If possible, use code coverage tools. These will tell you
> how many times each line of code was executed, so you can
> confirm that your test problem goes through all the code.
> Ask in your group about code coverage, maybe someone already
> knows. If not, you'll have to stop and read a few compiler
> manuals.
> *) Run your test problem, check that the results make sense
> and save the results.

Got that already.

> Do not begin work until you have a test problem. When you do,
>
> *) Start hacking the code: module, derived type, etc.
> You'll go through a long phase where it won't even compile,
> so keep fixing the typos until it does.

Done that already. That's when I encountered this mess with
EXTERNAL in what I wanted to be CONTAINed routines.

> *) If possible, make your changes in stages, one subsystem at
> a time.

The original is a just-barely-f77 loads-of-global-variables mess;
"one subsystem at a time" doesn't exist yet.

> *) Run the test problem and confirm that the results haven't
> changed. Do this every time you have a version that compiles.
> If the results are different or if the program crashes,
> you introduced a bug in your last round of conversions.
> Find it and fix it.
> *) When all the code is converted, save it as a milestone.
> Then start working on the parallelized version.

You missed the point here -- the point is to make it thread-safe,
so that a met driver parallelized on its (3-D) set of grid cells
can call it safely in parallel, each call being the aerosol calculation
for one of the grid cells.

But thanks for trying!

-- Carlie

Walter Spector

2005-09-15, 6:59 pm

"Carlie J. Coats" wrote:
> ... the point is to make it thread-safe,
> so that a met driver parallelized on its (3-D) set of grid cells
> can call it safely in parallel, each call being the aerosol calculation
> for one of the grid cells.

What parallel model are you using?

If you are using OpenMP, you can simply place the global variables at module
scope, then declare them to be 'task private'.

Walt
(w6ws att earthlinkk dott nett)

Walter Spector

2005-09-15, 6:59 pm

Walter Spector wrote:
> ...
> If you are using OpenMP, you can simply place the global variables at module
> scope, then declare them to be 'task private'.
^^^^thread

Walt
(w6ws att earthlinkk dott nett)

J. F. Cornwall

2005-09-15, 6:59 pm

Carlie J. Coats wrote:
> Pierre Asselin wrote:
>
>
>
> If you go back to the top, you'll find that there was over 28000
> lines of code, of which 7500 were directly involved with this issue
> of indirect calls to EXTERNALs, either as caller, callee, or arguments.
>
> Not exactly "small" ;-(
>

A small test *problem*, not test *program*... I.e create a small data
set to run your program against (at least that's my interpretation of
Pierre's comment).

Jim

Brooks Moses

2005-09-15, 6:59 pm

Richard E Maine wrote:
> In article <43287EFB.5040100@cits1.stanford.edu>,
> Brooks Moses <bmoses-nospam@cits1.stanford.edu> wrote:
>
> ..........^.......
>
> For standard conformance, % please instead of a period.

Argh, right. Thank you!

- Brooks

--
The "bmoses-nospam" address is valid; no unmunging needed.

David Flower

2005-09-16, 3:57 am

Employing strict coding standards (including NO COMMONS)...

>
> Completely re-writing these messy 7500 lines of code is way beyond
> the budget for the project ;-(
>

>
> Thanks!
>
> -- Carlie

It seems to me that your problem is not technical, but managerial.

Substantially re-writing a large program cannot be done on the cheap

Dave Flower

Pierre Asselin

2005-09-16, 6:59 pm

Carlie J. Coats <carlie@jyarborough.com> wrote:

> If you go back to the top, you'll find that there was over 28000
> lines of code, of which 7500 were directly involved with this issue
> of indirect calls to EXTERNALs, either as caller, callee, or arguments.

> Not exactly "small" ;-(

Small *problem*, not small *program*. Oops, Jim Cornwall already
said that.

> [ ... ]
> You missed the point here -- the point is to make it thread-safe,

No, I got that part. I'm just saying you want it thread-safe *and
equivalent to the original*.

> But thanks for trying!

It seems you are already following good practice. But if you ask
"is there an easy way to hack thread safety into a dusty deck" then
that's easy: the answer is "no". Your choices are to dump the
code and rewrite it from scratch, or hack thread safety into it
the hard way, or run it serial.

Oops. Fire alarm. Bye now.

--
pa at panix dot com

Pierre Asselin

2005-09-16, 6:59 pm

Pierre Asselin <pa@see.signature.invalid> wrote:
> Carlie J. Coats <carlie@jyarborough.com> wrote:

[color=darkred]
> [ ... ]
> Oops. Fire alarm. Bye now.

Okay, where was I. Right. If you choose to hack thread safety
into the thing, you have to rewrite *every* subroutine that's called
through a dummy argument to take one more argument than before, of
a derived type defined in a module that the subroutine USEs.
Replace every variable that used to be in COMMON by a reference to
the new argument, i.e. toto <-- X%toto. You should then be able
to comment out the COMMON statement in the rewritten routine and
get it to compile.

You also rewrite every caller to declare or allocate or recieve a
variable of the derived type and make sure it is stuffed with the
right initial values before passing it on to the rewritten routines.

It seems to me that you can do this in pieces, removing the COMMON
from one subroutine at a time, but I may be wrong. You may have
to recursively rewrite the callers to get rid of their copy of the
COMMON statement too; this is extra work but if it lets you do an
incremental refactoring it's probably worth it. You get a sequence
of modifications, each of which is functionally equivalent to the
preceding one, and the last of which is thread-safe. Just run your
test case every chance you get so you catch errors early.

Whether this approach is feasible depends on the quality of
the dust in the deck. I can't tell from here.

--
pa at panix dot com

Sponsored Links