Home > Archive > Fortran > March 2006 > .mod generation question
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
.mod generation question
|
|
|
|
g95 only touches the .mod files during compilation if the content of
the .mod changes. Since the .mod files contain all interface
definitions (and similar info), but don't depend on the procedure
bodies, this provides a easy way to avoid compilation cascades. I'm
wondering if anybody knows of other compilers that also implement this
feature. That would give us an additional reason to start using this
feature in our build system.
Thanks,
Joost
| |
| robert.corbett@sun.com 2006-03-28, 4:02 am |
|
Joost wrote:
> g95 only touches the .mod files during compilation if the content of
> the .mod changes. Since the .mod files contain all interface
> definitions (and similar info), but don't depend on the procedure
> bodies, this provides a easy way to avoid compilation cascades. I'm
> wondering if anybody knows of other compilers that also implement this
> feature. That would give us an additional reason to start using this
> feature in our build system.
We tried that at Sun. It was not popular with users who had makefile
dependencies for their .mod files on the source files that contained
the modules. The problem was that if the .mod file wasn't updated
when the source file was compiled, the .mod file would be out-of-date
with respect to the source file. The source file would be recompiled
with each and every make until the user manually deleted the .mod file.
On the first make after that, the recompilation cascade would occur.
Sun's makefile expert recently suggested a workable way to avoid
recompilation cascades that does not have the problem of
unnecessarily recompiling the source files that contain modules.
It can be implemented for many existing compilers, but it requires
writing the makefiles in what is currently an unusual and somewhat
awkward way. Unfortunately, because of differences in the ways
various compilers generate .mod files, the makefiles must differ from
comiler to compiler.
Bob Corbett
| |
|
|
robert.corbett@sun.com wrote:
>
> We tried that at Sun. It was not popular with users who had makefile
> dependencies for their .mod files on the source files that contained
> the modules. The problem was that if the .mod file wasn't updated
> when the source file was compiled, the .mod file would be out-of-date
> with respect to the source file. The source file would be recompiled
> with each and every make until the user manually deleted the .mod file.
> On the first make after that, the recompilation cascade would occur.
>
I'm not sure I understand that completely. I have the following example
Makefile:
FC=g95
FCFLAGS=-O2 -march=opteron -c
OBJ=m1.o m2.o main.o
EXE=prog.x
$(EXE): $(OBJ)
$(FC) -o $(EXE) $(OBJ)
%.o: %.f90
$(FC) $(FCFLAGS) $<
clean:
rm -f $(OBJ) $(EXE)
# generated with 'g95 -M'
m1.o m1.mod: m1.f90
m2.o m2.mod: ./m1.mod m2.f90
main.o: ./m2.mod ./m1.mod main.f90
this seems to do the right thing, even though m1.mod depends on m1.f90.
However, I'm really not a make expert, so it could be that I've
overlooked things.
Joost
| |
| Rich Townsend 2006-03-28, 4:02 am |
| Joost wrote:
> robert.corbett@sun.com wrote:
>
>
>
> I'm not sure I understand that completely. I have the following example
> Makefile:
>
> FC=g95
> FCFLAGS=-O2 -march=opteron -c
> OBJ=m1.o m2.o main.o
> EXE=prog.x
>
> $(EXE): $(OBJ)
> $(FC) -o $(EXE) $(OBJ)
>
> %.o: %.f90
> $(FC) $(FCFLAGS) $<
>
> clean:
> rm -f $(OBJ) $(EXE)
>
> # generated with 'g95 -M'
> m1.o m1.mod: m1.f90
> m2.o m2.mod: ./m1.mod m2.f90
> main.o: ./m2.mod ./m1.mod main.f90
>
> this seems to do the right thing, even though m1.mod depends on m1.f90.
> However, I'm really not a make expert, so it could be that I've
> overlooked things.
>
> Joost
>
OK, but what happens if you revise ml.f90 in such a way that the
interfaces in the .mod files don't change? Then, each time you run make,
it will see the .mod files as being older than ml.f90, and try to remake
them. But since the interfaces haven't changed, g95 will not update them
-- the timestamps will remain stale. So you end up with the compiler
trying to remake stuff all the time. I think that this is Bob's point.
cheers,
Rich
| |
| glen herrmannsfeldt 2006-03-28, 4:02 am |
| Rich Townsend wrote:
(snip on make and .mod files)
> OK, but what happens if you revise ml.f90 in such a way that the
> interfaces in the .mod files don't change? Then, each time you run make,
> it will see the .mod files as being older than ml.f90, and try to remake
> them. But since the interfaces haven't changed, g95 will not update them
> -- the timestamps will remain stale. So you end up with the compiler
> trying to remake stuff all the time. I think that this is Bob's point.
And that the solution is compiler dependent.
If it is just updating the timestamp, after the compile command put
a touch command to update the timestamp on the mod file.
-- glen
| |
|
| This is apparently not the case with gnu make. This is a sample session
with the above makefile (vi just changes some function body):
vondele@pcihopt1:~/g95> rm *.o *.mod
vondele@pcihopt1:~/g95> make
g95 -O2 -march=opteron -c m1.f90
g95 -O2 -march=opteron -c m2.f90
g95 -O2 -march=opteron -c main.f90
g95 -o prog.x m1.o m2.o main.o
vondele@pcihopt1:~/g95> vi m1.f90
vondele@pcihopt1:~/g95> make
g95 -O2 -march=opteron -c m1.f90
g95 -o prog.x m1.o m2.o main.o
vondele@pcihopt1:~/g95> make
make: `prog.x' is up to date.
vondele@pcihopt1:~/g95> ls -tr m1.*
m1.mod m1.f90 m1.o
vondele@pcihopt1:~/g95> make
make: `prog.x' is up to date.
in particular notice that even though m1.mod is older than m1.f90
prog.x is 'up to date', and there is no recompilation. I guess it is
sufficient that the m1.o is older than m1.f90.
Joost
| |
|
| I start understanding the problem, I think, looks like I'm in trouble
with the naive Makefile above as the .mod might not be valid as long as
the .o is not valid. In order to work fine I need a 'silly' rule like:
%.mod: %.o
echo $< > /dev/null
which is always executed if the .mod is older than .o. I start seeing
some of the mess, but I'm wondering if there is no clean solution. I'll
leave it to the makefile experts though.
Joost
| |
| Bil.Kleb@gmail.com 2006-03-28, 7:02 pm |
| Interesting, my Thunderbird didn't show this article in
news:comp.lang.fortran,
posting via Google groups... Either my newsfeed is broken, or the
leading "."
in the subject line confuses it?
Joost wrote:
> g95 only touches the .mod files during compilation if the content of
> the .mod changes. Since the .mod files contain all interface
> definitions (and similar info), but don't depend on the procedure
> bodies, this provides a easy way to avoid compilation cascades.
I asked Andy for this last December, and he implemented it within a
day!
> I'm wondering if anybody knows of other compilers that also implement this
> feature.
I haven't found any, but it sure would be nice...
> That would give us an additional reason to start using this
> feature in our build system.
We use it with our make build system by adding a module target
borrowed from Stern and Grimwood[1]:
%.mod:
@if [[ ! -s $@ ]] ; then \
printf "\n\t%s\n\t%s\n\n" \
"$@ doesn't exist or is of zero size." \
"Deleting $< -- Restart MAKE and try again" 1>&2 ; \
$(RM) -f $< $(@)* ; \
exit 1; \
fi
Regards,
--
Bil Kleb
http://fun3d.larc.nasa.gov
[1]http://www.theochem.uwa.edu.au/fortran/recompile/article
| |
|
| I'm getting this output from that target:
~/g95> make -j 8
m1.f90.mod doesn't exist or is of zero size.
Deleting -- Restart MAKE and try again
make: *** [m1.f90.mod] Error 1
But I guess the only problem I'm having right now that I'm missing
dependencies for the .mod file on the corresponding .o file.
Joost
| |
| robert.corbett@sun.com 2006-03-28, 7:02 pm |
|
Joost wrote:
> I start seeing
> some of the mess, but I'm wondering if there is no clean solution. I'll
> leave it to the makefile experts though.
I too doubted that a clean solution existed without requiring changes
to make. However, as I said in my earlier posting, Sun's makefile
expert came up with a solution. It isn't as simple as I would like,
but it works. When I have the time, I'll try to write it up.
Bob Corbett
| |
| Joe Krahn 2006-03-29, 7:02 pm |
| Joost wrote:
> g95 only touches the .mod files during compilation if the content of
> the .mod changes. Since the .mod files contain all interface
> definitions (and similar info), but don't depend on the procedure
> bodies, this provides a easy way to avoid compilation cascades. I'm
> wondering if anybody knows of other compilers that also implement this
> feature. That would give us an additional reason to start using this
> feature in our build system.
>
> Thanks,
>
> Joost
>
The compilation cascade is one of the things I have always considered a
significant flaw in the design of modules. That compiler feature is a
nice idea to avoid recompiling when a procedure changes without changing
the interface. It is useful enough that it would be worth simulating the
feature using 'diff' to see if the .mod files actually changed, assuming
there is no compile-time stamp stuck in there.
I will give this idea a try. Perhaps the best way to implement it is to
have a "build .mod file" and an "include .mod file". This way, the build
file time stamp is used to test freshness versus the module's source,
and the include file is used to test freshness versus source files that
use the .mod file.
Joe
| |
| Joe Krahn 2006-03-29, 7:02 pm |
| Joe Krahn wrote:
> Joost wrote:
>
>
> The compilation cascade is one of the things I have always considered a
> significant flaw in the design of modules. That compiler feature is a
> nice idea to avoid recompiling when a procedure changes without changing
> the interface. It is useful enough that it would be worth simulating the
> feature using 'diff' to see if the .mod files actually changed, assuming
> there is no compile-time stamp stuck in there.
>
> I will give this idea a try. Perhaps the best way to implement it is to
> have a "build .mod file" and an "include .mod file". This way, the build
> file time stamp is used to test freshness versus the module's source,
> and the include file is used to test freshness versus source files that
> use the .mod file.
>
> Joe
After a bit of fiddling, the idea of 'build' and 'include' copies of a
..mod file may not be easy for some compilers due to the variety of .mod
file handling. However, I did find one useful feature in gnu make:
order-only prerequisites. These specify build-order rules without
specifying time-stamp dependencies. This is probably what is needed for
specifying the relationship between a .mod file and the source file that
creates it.
This may be a feature specific to Gnu make, but why use anything else?
Maybe a good preliminary question is: are most modern Fortran developers
using Gnu make? With the complexity of Fortran module files, it seems
like the only good choice for Makefile-based development.
Joe
| |
| James Giles 2006-03-29, 7:02 pm |
| Joe Krahn wrote:
....
> The compilation cascade is one of the things I have always considered
> a significant flaw in the design of modules. That compiler feature is
> a nice idea to avoid recompiling when a procedure changes without
> changing the interface. [...]
I used to think so when modules first came out. I posted a
series of articles recommending that the standard should
clarify the definition of "public parts" of a module so
that a programmer could easily recognize when change
would require recompilation of USEs and when that would
not occur. I was told emphatically that implementations
should be free to optimize module procedures any way
they like and that any clarification of "public parts" would
constitute an undesirable limitation on such implementations
and that recompilation was a small price to pay for the
extra optimization.
Well, they convinced me. I subsequently beleved that *any*
change to a module should require recompilation of USEs, but
that the compilers should provide interprocedural optimization
as compensation for that inconvenience. Then people told
me that's too much of a hardship and that users should have
confidence that any changes other than to the "interfaces"
of the modules (whatever those are - it's not defined except
when INTERFACE blocks are used) doesn't cause recompilation
cascades. Seems like of those original people in the old days
that still participate in the newgroup now - they pretty much
all opposed my new position. :-(
Well, they've convinced me again: that Fortran's failure to
separate specifications from implementations (like most
other languages with any sort of modules do) was a mistake
all along. Allowing an external procedure to USE a module
that contains interfaces for the procedure itself (with the
obvious meaning that the interface must match the actual
definition) would correct the problem. Then you would
have unambiguous control: if you want independent compilation,
you write externals and USE interface blocks to check the
interfaces. If you want aggressive interprocedural optimization,
you put the procedures into the modules and suffer recompilation
cascades when you change them.
(I like language features to have clear definitions, and I like
programmers using those features to be in explicit control
of important aspects of those features.)
--
J. Giles
"I conclude that there are two ways of constructing a software
design: One way is to make it so simple that there are obviously
no deficiencies and the other way is to make it so complicated
that there are no obvious deficiencies." -- C. A. R. Hoare
| |
| Steven G. Kargl 2006-03-29, 10:00 pm |
| In article <K1FWf.148$9r.43@mencken.net.nih.gov>,
Joe Krahn <krahn-at-niehs.nih.gov@x.y.z> writes:
> Joe Krahn wrote:
>
> This may be a feature specific to Gnu make, but why use anything else?
> Maybe a good preliminary question is: are most modern Fortran developers
> using Gnu make? With the complexity of Fortran module files, it seems
> like the only good choice for Makefile-based development.
>
No, I don't use Gnu make for my projects. I use make(1) as
distributed by FreeBSD (aka bmake). In working with gmake,
I've converted a few of my nice, clean, Makefiles to use
gmake and boy are the new Makefiles ugly contortions of logic.
--
Steve
http://troutmask.apl.washington.edu/~kargl/
| |
| Richard Maine 2006-03-29, 10:00 pm |
| Steven G. Kargl <kargl@troutmask.apl.washington.edu> wrote:
>
> No, I don't use Gnu make for my projects...
Me neither. I keep my Make usage as simple as possible so that it will
work with a broad variety of makes. My code gets built by various users
on many different platforms and I don't need the extra complication of
having to dictate to them what versions of make they have to have
installed. Sometimes it is tempting to use "fancy" make features, but I
don't.
--
Richard Maine | Good judgement comes from experience;
email: last name at domain . net | experience comes from bad judgement.
domain: summertriangle | -- Mark Twain
| |
| Pierre Asselin 2006-03-30, 10:00 pm |
| James Giles <jamesgiles@worldnet.att.net> wrote:
> [ ... ] Allowing an external procedure to USE a module
> that contains interfaces for the procedure itself (with the
> obvious meaning that the interface must match the actual
> definition) would correct the problem. Then you would
> have unambiguous control: if you want independent compilation,
> you write externals and USE interface blocks to check the
> interfaces. If you want aggressive interprocedural optimization,
> you put the procedures into the modules and suffer recompilation
> cascades when you change them.
Hear, hear !
--
pa at panix dot com
| |
| Joe Krahn 2006-03-30, 10:00 pm |
| Steven G. Kargl wrote:
> In article <K1FWf.148$9r.43@mencken.net.nih.gov>,
> Joe Krahn <krahn-at-niehs.nih.gov@x.y.z> writes:
>
>
>
> No, I don't use Gnu make for my projects. I use make(1) as
> distributed by FreeBSD (aka bmake). In working with gmake,
> I've converted a few of my nice, clean, Makefiles to use
> gmake and boy are the new Makefiles ugly contortions of logic.
>
I had not tried bmake; it seems to be a more 'sane' enhancement of
traditional BSD make. Many of the Gnu make features are overly
complicated, but there are a lot of good basic features as well. I'm
surprised that the Gnu version is an ugly contortion. Maybe it's because
Gnu-make has a lot of complicated examples in the documentation.
In any case, it seems that the safest thing to do is to assume that
people use a standard BSD make, and use an external tool to generate the
make rules instead of relying on fancy make features.
In that case, can I at least assume that everybody has Perl available? I
sometimes get the idea that a nice Perl script would be better than
trying to use make at all for a robust multi-platform/multi-compiler
build tool. If not, at least it's a good tool for creating plain BSD
Makefiles.
Joe
|
|
|
|
|