Fortran - 30 Miles of Bad Code (FORTRAN->C)

Author

30 Miles of Bad Code (FORTRAN->C)

ibm@svpal.org

2005-05-05, 3:59 pm

We have been blessed with a large S/W dump from a vendor.
The s/w is really old crufty FORTRAN which isn't supported
in our operational environment ( Winders Ecks-Pee + a proprietary
component based real-time architecture ).
The vendor is not required to do anything under the terms of their
contract, sigh...

Looks like we'll have to convert the code to C at least.

We've tried "f2c" and some store-bought packages but they all
have issues with the code in question ( did I mention it was ugly,
even by spaghetti code standards ). If this was just a one
shot thing we might go through and manually fix the several hundred
problems before converting but this is liable to be a recurring issue.

Any ideas on what we might do to mitigate the recurrent effort.
Tools?
Nuking the vendor in question?

My first instinct would be to bite the bullet and go through the
code fixing and documenting fixes for the hundreds of errors we
see. Some of the rrros are likely to be consequences of a single
oops but an initial analysis looks like most of the crud is distinct
in nature.

TIA

Tim Prince

2005-05-05, 3:59 pm

Madhusudan Singh

2005-05-05, 3:59 pm

ibm@svpal.org wrote:

> We have been blessed with a large S/W dump from a vendor.
> The s/w is really old crufty FORTRAN which isn't supported
> in our operational environment ( Winders Ecks-Pee + a proprietary
> component based real-time architecture ).
> The vendor is not required to do anything under the terms of their
> contract, sigh...

One might wonder which "genius" signed that "contract" (as I understand the
term, both parties are usually supposed to either do something and / or
desist from doing something else). A caveat emptor kind of sale does not
jibe well with the definition of a contract agreed to between two sane
parties.

>
> Looks like we'll have to convert the code to C at least.
>
> We've tried "f2c" and some store-bought packages but they all
> have issues with the code in question ( did I mention it was ugly,
> even by spaghetti code standards ). If this was just a one
> shot thing we might go through and manually fix the several hundred
> problems before converting but this is liable to be a recurring issue.

I really sympathise with your situation. A problematic OS + negligent vendor
and now poorly written code.

>
> Any ideas on what we might do to mitigate the recurrent effort.
> Tools?
> Nuking the vendor in question?

That would be a start. Followed perhaps (depending on your requirements) by
shifting to a more friendly and dependable OS (Linux / BSD come to mind)
which have all the tools you need, obviating the need to mess around with
cygwin, or tweaking a toy OS to do serious work. Imagine having done all
this work and then one of windoze' blessings (aka viruses, etc.) comes
calling. One of my former coworkers once lost his Master's thesis (written
in Word) to a virus, 3 days before it was due. Not an experience any
reasonable person should have to go through. I would suggest the above, no
matter whether you were working with C or fortran.

And also hiring a fortran programmer (in case you are disinclined to do the
task yourself - I have worked with old undocumented code written by someone
else, so I know what the experience is like - not something I would care to
repeat) to translate the code into non-prehistoric fortran (maybe fortran
95).

Ben Barrowes

2005-05-05, 8:58 pm

Are the errors you see in trying to compile the fortran or when trying
to compile the C after conversion?

Getting it working in fortran would be a good start. Then there are
several code beautifiers/modernizers/spaghetti code restructurers
around. One that I have used is plusfort (spag). From their manual:

2.1.1 What does it do?
SPAG is a multi-purpose tool for analyzing and improving Fortran
programs. It combines restructuring and re-formatting with translation
to Fortran 95, and both static and dynamic analysis in a single powerful
package. But SPAG offers facilities which go beyond the sum of these
parts. For example:
* SPAG can restructure 'rats-nest' Fortran 66 to modern and
structured Fortran (77 or 95), or apply a final polish to well written
modern code. Progress can be monitored using 'Before and After'
complexity metrics.
etc...

After getting better f95, you can then attack the conversion problem.

ibm@svpal.org wrote:
> We have been blessed with a large S/W dump from a vendor.
> The s/w is really old crufty FORTRAN which isn't supported
> in our operational environment ( Winders Ecks-Pee + a proprietary
> component based real-time architecture ).
> The vendor is not required to do anything under the terms of their
> contract, sigh...
>
> Looks like we'll have to convert the code to C at least.
>
> We've tried "f2c" and some store-bought packages but they all
> have issues with the code in question ( did I mention it was ugly,
> even by spaghetti code standards ). If this was just a one
> shot thing we might go through and manually fix the several hundred
> problems before converting but this is liable to be a recurring issue.
>
> Any ideas on what we might do to mitigate the recurrent effort.
> Tools?
> Nuking the vendor in question?
>
> My first instinct would be to bite the bullet and go through the
> code fixing and documenting fixes for the hundreds of errors we
> see. Some of the rrros are likely to be consequences of a single
> oops but an initial analysis looks like most of the crud is distinct
> in nature.
>
> TIA
>

J DUDLEY

2005-05-05, 8:58 pm

<ibm@svpal.org> wrote in message
news:1115303630.835480.122710@g14g2000cwa.googlegroups.com...
> We have been blessed with a large S/W dump from a vendor.
> The s/w is really old crufty FORTRAN which isn't supported
> in our operational environment ( Winders Ecks-Pee + a proprietary
> component based real-time architecture ).
> The vendor is not required to do anything under the terms of their
> contract, sigh...
>
> Looks like we'll have to convert the code to C at least.
>
> We've tried "f2c" and some store-bought packages but they all
> have issues with the code in question ( did I mention it was ugly,
> even by spaghetti code standards ). If this was just a one
> shot thing we might go through and manually fix the several hundred
> problems before converting but this is liable to be a recurring issue.
>
> Any ideas on what we might do to mitigate the recurrent effort.
> Tools?
> Nuking the vendor in question?
>
> My first instinct would be to bite the bullet and go through the
> code fixing and documenting fixes for the hundreds of errors we
> see. Some of the rrros are likely to be consequences of a single
> oops but an initial analysis looks like most of the crud is distinct
> in nature.
>
> TIA
>

You don't say what in the code is causing the problems. However, many
Fortran compilers can accommodate such old code - with Visual Fortran there
is an option to switch on Fortran 66 interpretation of DO loops and. more
importantly, extended-DO loops. I had code that would jump out of a DO loop
to an error handler, then jump back again. Something like (forgive me if I
get the syntax slightly wrong - this is not my natural way of error
handling!)

DO 10, I = 1, 100
READ(10, *, ERR = 600) (X(J,I), J = 1, 10)
10 CONTINUE
....

600 PRINT *, 'Error reading line', I
GOTO 10

.....

In the event, I rewrote the relevant lines to not need extended DO

DO 10, I = 1, 100
READ(10, *, ERR = 600) (X(J,I), I = 1, 10)
GOTO 10
600 PRINT *, 'Error reading line', I
10 CONTINUE

Tedious. But gets rid of the main FORTRAN 66 impediment to compiling the
code. All the FORTRAN 66 code I have ever worked with has assumed (1) That
DO loops will have at least one pass, and (2) that that is now what is
wanted, so they explicitly code a check that is now implicit with FORTRAN 77
and later. To relieve the boredom of the extended DO loop stuff I would
remove the checks if it was close by the code I was cleaning up.

As another poster has noted, there are tools that will automate this for
you, and SPAG sounds good. What would be nice, and SPAG used not to do this,
would be something that also identifies where array syntax can be used -
this I find a great help in cleaning up code.

I'd also go along with another poster - if you can't satisfy yourself with a
clean Fortran compile, what confidence would you have that the C translation
was valid?

beliavsky@aol.com

2005-05-05, 8:58 pm

J DUDLEY wrote:

> As another poster has noted, there are tools that will automate this
for
> you, and SPAG sounds good. What would be nice, and SPAG used not to
do this,
> would be something that also identifies where array syntax can be
used -
> this I find a great help in cleaning up code.

I am interesed in this too, but the only tool I know of, VAST/77to90
http://www.crescentbaysoftware.com/vast_77to90.html , costs $16,000 for
unlimited use. Googling "vast 77 90" in this ng brings up a detailed
review by John Prentice posted in 1993.

I wonder how hard it would be to write a tool that does some
vectorizations. Just replacing the ubiquitious F77 construct

xsum = 0.0
do i=1,n
xsum = xsum + x(i)
end do

with

xsum = sum(x)

would be of some of value.

epc8@juno.com

2005-05-06, 4:00 am

kia

2005-05-06, 4:00 am

"ibm@svpal.org" wrote:
>
> Any ideas on what we might do to mitigate the recurrent effort.
> Tools?
> Nuking the vendor in question?

No, *hire* their contract lawyer and *fire* yours!
As for tools, check

http://cobalt-blue.com

They've been around awhile, they're reasonably priced and have Fortran
structuring tools and Fortran to ANSI C translator.

Jan Vorbrüggen

2005-05-06, 3:59 pm

> I wonder how hard it would be to write a tool that does some
> vectorizations. Just replacing the ubiquitious F77 construct
>
> xsum = 0.0
> do i=1,n
> xsum = xsum + x(i)
> end do
>
> with
>
> xsum = sum(x)
>
> would be of some of value.

For many compilers I've seen, this produces code that performes exactly the
same.

Jan

beliavsky@aol.com

2005-05-06, 3:59 pm

Jan Vorbr=FCggen wrote:
>
> For many compilers I've seen, this produces code that performes
exactly the
> same.
>
> Jan

I also have observed that. I want a source-to-source vectorizer that
makes a code shorter, not necessarily faster.

jon

2005-05-06, 3:59 pm

What?!!!
"lost his Master's thesis (written in Word) to a virus, 3 days before
it was due."
How did Microsoft and a virus retroactively prevent this unfortunate
person from making backup copies of the most important document in this
person's life? Or is this some sort of Darwinism preventing those who
can't do a modicum of risk analysis from obtaining degrees? Oh, well,
it does sound more inventive than "The dog ate my homework."

Madhusudan Singh

2005-05-06, 8:57 pm

jon wrote:

> What?!!!
> "lost his Master's thesis (written in Word) to a virus, 3 days before
> it was due."
> How did Microsoft and a virus retroactively prevent this unfortunate
> person from making backup copies of the most important document in this
> person's life? Or is this some sort of Darwinism preventing those who
> can't do a modicum of risk analysis from obtaining degrees? Oh, well,
> it does sound more inventive than "The dog ate my homework."

As a matter of fact, he did make backups of his thesis. Its been a while,
but I remember he had one copy on a floppy (which I tried to help him
retrieve his thesis from - all in vain) and another on his machine at home.

Making backups does not help if your backup copies are infected too (we
never were able to determine whether the floppy went bad at the same time
or whether the file carried a virus).

Of course, my pity for his plight was tinged with a little bit of enjoyment
at his expense, because a few months before that, I had made my own
transition from windoze (the only time in my personal experience when the
windoze 95 blue screens of death have done any good to anyone) to HP-UX and
then finally Linux (RH 4.x something)), and had been unsuccessfully egging
him to do the same.

No, Microsoft did not prevent him from making backups, but did ensure that
all his backups were to no avail.

J DUDLEY

2005-05-06, 8:57 pm

<beliavsky@aol.com> wrote in message
news:1115383394.556263.181870@g14g2000cwa.googlegroups.com...

Jan Vorbrüggen wrote:
>
> For many compilers I've seen, this produces code that performes
exactly the
> same.
>
> Jan

I also have observed that. I want a source-to-source vectorizer that
makes a code shorter, not necessarily faster.

JD:

Exactly! It is to simplify future maintenance, by reducing code clutter.

Colin Watters

2005-05-06, 8:57 pm

> Any ideas on what we might do to mitigate the recurrent effort.
> Tools?
>

How about posting a sample of the code so we can get some idea of what
you're up against?

--
Qolin

Email: my qname at domain
Domain: qomputing dot demon dot co dot uk

Gib Bogle

2005-05-09, 3:57 am

Arjen Markus

2005-05-09, 8:58 am

beliavsky@aol.com wrote:
>
> J DUDLEY wrote:
>
> for
> do this,
> used -
>
> I am interesed in this too, but the only tool I know of, VAST/77to90
> http://www.crescentbaysoftware.com/vast_77to90.html , costs $16,000 for
> unlimited use. Googling "vast 77 90" in this ng brings up a detailed
> review by John Prentice posted in 1993.
>
> I wonder how hard it would be to write a tool that does some
> vectorizations. Just replacing the ubiquitious F77 construct
>
> xsum = 0.0
> do i=1,n
> xsum = xsum + x(i)
> end do
>
> with
>
> xsum = sum(x)
>
> would be of some of value.

You could use a little "regular expression" to identify this
kind of code. While UNIX tools like grep do not seem to support
bckreferences, a language like Tcl or Perl does. If you scan
the files in question with a regular expression like:

regexp {(\w+) *= *\1} $line full sum

(Tcl-syntax)

you can easily find the lines that involve a pattern like the
above.

(I agree that decyphering and designing regular expressions
is not always easy, but it is something that can be learned :))

The next step would be to convert the code to use SUM()....

Regards,

Arjen

jon

2005-05-09, 4:00 pm

His was a costly lesson. I still need reminders from time to time. I
once worked an 8 hour evening shift connected to a Boeing Computer
Services CDC mainframe via a 300 bps modem from work in Colorado. I was
just about to save and log off when the session terminated
unexpectedly. In that environment, nothing was saved to disk unless
specifically saved during the session. I couldn't recover, so I spent
another shift to redo all the work and was there when the morning shift
arrived. I think every other command was a save during that session!

James Giles

2005-05-09, 8:58 pm

Madhusudan Singh wrote:
....
> No, Microsoft did not prevent him from making backups, but did
> ensure that all his backups were to no avail.

Of course, UNIX is about as secure as windows. That is, not at
all. The people involved in trying to produce genuinely secure
systems have already made it plain that backward compatibility
with either UNIX or Windows is impossible. Not just difficult:
impossible. If UNIX were installed on as many systems as
Windows and Windows on as few as UNIX, then UNIX would
be the preferred target of malware and Windows would seem
comparatively safe. It would be similarly false conclusion
to suppose that Windows was really safer though.

--
J. Giles

"I conclude that there are two ways of constructing a software
design: One way is to make it so simple that there are obviously
no deficiencies and the other way is to make it so complicated
that there are no obvious deficiencies." -- C. A. R. Hoare

Madhusudan Singh

2005-05-10, 3:59 am

James Giles wrote:

> Madhusudan Singh wrote:
> ...
>
> Of course, UNIX is about as secure as windows. That is, not at
> all. The people involved in trying to produce genuinely secure
> systems have already made it plain that backward compatibility
> with either UNIX or Windows is impossible. Not just difficult:
> impossible. If UNIX were installed on as many systems as
> Windows and Windows on as few as UNIX, then UNIX would
> be the preferred target of malware and Windows would seem
> comparatively safe. It would be similarly false conclusion
> to suppose that Windows was really safer though.
>

Even if a unix system was targetted with viruses and a user stupidly allowed
it to run, the maximum damage a user could do would be to remove his files,
or set up a program to send spam. Not enough to bring down the system.
Unlike windoze where a lot of user level tasks need "root" access.

The real danger to a unix system lies in having insecure ports open, and
having vulnerable programs listening to those ports. Different unix flavors
have different paradigms regarding this. Many Linux flavors default to a
lot of useless servers running without a firewall. OTOH, systems like
OpenBSD and FreeBSD default to almost every server being switched off. As
far as this side of security is concerned, there is no substitute for an
alert admin and a proper firewall. However, unlike windoze, these are tasks
that are easy to do and cost nothing in terms of money. The reason being
that unices were fully networked OSes long before they captured large
portions of the server market and increasingly significant portions of the
desktop/notebook market. Windoze OTOH, from its DOS days, was best run on a
stand alone machine. M$ introduced protocols like winbind, etc. that are
inherently more insecure than ssh. Even in those long past days, windoze
had its share of viruses. Dark Avenger anyone ?

The architectures of the two OSes are very different. An ordinary user on a
windoze machine has far more privileges than a corresponding user on a unix
machine. Hence, the far greater damage potential. Secure windoze networking
is, and always has been, an afterthought. So, statements like the ones you
made above just do not stand closer scrutiny.

Anyways, that will be my last post on this subthread. Its getting very OT.

James Giles

2005-05-10, 8:58 am

Madhusudan Singh wrote:
> James Giles wrote:
>
>
> Even if a unix system was targetted with viruses and a user
> stupidly allowed it to run, the maximum damage a user could do
> would be to remove his files, or set up a program to send spam. Not
> enough to bring down the system. Unlike windoze where a lot of user
> level tasks need "root" access.

You are living in a fantasy world. Unix is pretty insecure. The
last I saw a demonstration, a malicious attacker needed less than
ten minutes to get root privilege from an unprivileged account.

Unix security doesn't even meet the lowest classification of
security certification:

The CAPP (Controlled Access Protection Profile) provides for a
level of protection which is appropriate for an assumed non-hostile
and well-managed user community requiring protection against
threats of inadvertent or casual attempts to breach the system security.
The profile is not intended to be applicable to circumstances in which
protection is required against determined attempts by hostile and well
funded attackers to breach system security. The CAPP does not fully
address the threats posed by malicious system development or
administrative personnel.

As one observer said:

Translating that into colloquial English:

Don't hook this to the Internet, don't run e-mail, don't install
software unless you can 100% trust the developer, and if anybody
who works for you turns out to be out to get you you are toast.

And no Unix version is certified at this level. Unix has as many
holes as Windows. Neither is a very good system. Not secure,
reliable, or user friendly. I'm convinced that Microsoft decided
they didn't need any of those characteristics have a successful
system by observing the success of Unix.

--
J. Giles

"I conclude that there are two ways of constructing a software
design: One way is to make it so simple that there are obviously
no deficiencies and the other way is to make it so complicated
that there are no obvious deficiencies." -- C. A. R. Hoare

Madhusudan Singh

2005-05-10, 3:59 pm

NG followup shifted to comp.os.linux.misc as this very OT in c.l.f.
JG : Please read on, and respond in c.o.l.m or comp.os.linux.advocacy.
For others reading this in c.l.f., please ignore.

James Giles wrote:

>
> You are living in a fantasy world. Unix is pretty insecure. The
> last I saw a demonstration, a malicious attacker needed less than
> ten minutes to get root privilege from an unprivileged account.

Let me put it this way - I don't believe you. Unix passwords in /etc/passwd
are fairly strongly encrypted, and even if some dumb admin allowed (a
password generator for instance) to repeatedly try passwords without limit
or interval, I think it would a lot longer than 10 minutes to crack a well
chosen password.

As to obtaining privileges, an unprivilieged user, who is not a member of
groups like "root", "admin", "staff" or "wheel", cannot do much where
his/her native group ("users" usually) does not have write and execute
permissions.

>
> Unix security doesn't even meet the lowest classification of
> security certification:

Then I am curious about this classification system and a list of OSes that
meet the lowest classification or higher (if any).

A URL would be very useful.

>
> The CAPP (Controlled Access Protection Profile) provides for a
> level of protection which is appropriate for an assumed non-hostile
> and well-managed user community requiring protection against
> threats of inadvertent or casual attempts to breach the system security.
> The profile is not intended to be applicable to circumstances in which
> protection is required against determined attempts by hostile and well
> funded attackers to breach system security. The CAPP does not fully
> address the threats posed by malicious system development or
> administrative personnel.

No OS can defend against a malicious root or system software developer !!

>
> As one observer said:
>
> Translating that into colloquial English:
>
> Don't hook this to the Internet, don't run e-mail, don't install
> software unless you can 100% trust the developer, and if anybody
> who works for you turns out to be out to get you you are toast.

If you have a malicious root, the above is pointless.

>
> And no Unix version is certified at this level. Unix has as many
> holes as Windows. Neither is a very good system. Not secure,

Linux holes tend to be found (eyeball count) and fixed long before they
become a widespread problem. Windoze holes remain unfixed for a very long
time comparatively, costing the poor saps that use the OS lost time, money,
productivity, etc.

> reliable, or user friendly. I'm convinced that Microsoft decided
> they didn't need any of those characteristics have a successful
> system by observing the success of Unix.

They had no choice. Networking was an afterthought in DOS / win 3.1. Getting
an elephant to fly, IMO.

And returning to your initial bromide about Linux not being as targetted as
windoze, most servers run Linux/BSD. By your logic, most of them would be
compromised (or in a fraction at least as high as compromised windoze
servers).

Your "argument" so far seems to be something along these lines :

An airplane cannot take you instantaneously from, say, Atlanta to Los
Angeles. Neither can an Amtrak train. "Ergo", they are equally slow.

You can always set security standards so ridiculously high so that they are
of no practical value, and every OS extant fails to meet them. That does
not mean that you can ignore their significant differences.

ejko123@yahoo.com

2005-05-13, 4:10 pm

James Giles wrote:

> Unix security doesn't even meet the lowest classification of
> security certification:

Not quite true. SGI's Trusted IRIX meets Orange Book B1
standards and Trusted XENIX meets B2 - see e.g.,
www.radium.ncsc.mil/tpep/epl/entrie...L-92-001-A.html
Whether you want to call these "Unix" or "Unix-like" is another
matter.

(And that's my last o.t. post for today...)

--Eric

Sponsored Links