Fortran - writing about 100 colomns of real data into text file

Author

writing about 100 colomns of real data into text file

Bart Vandewoestyne

2005-06-08, 8:58 pm

I am trying to write about 100 columns of real data into a text file,
but somehow I get an 'Aborted' message when my write statement is
executed.

I've decided to test if there were some limitations on the number of
columns I could write, and apparently the following program works fine:

! This program demonstrates that it is possible to use *a lot* of columns with
! data when writing data-output to a textfile.

program test_max_write_columns

implicit none

integer, parameter :: dp = &
selected_real_kind(2*precision(1.0))
integer :: cols
real(kind=dp), dimension(:), allocatable :: myvar
character(len=100) :: formatstring

open(unit=20, file="test_max_write_columns.dat", &
status="replace", access="sequential", action="write")

print *, "How many columns of real data do you want?"
read *, cols

allocate(myvar(cols))
call random_number(myvar)

! Build up formatstring dynamically with 'cols' columns.
write(unit=formatstring, fmt="(A, I0, A)") "(", cols, "ES25.15e2)"

! Write to file and see if we can get that many columns...
write(unit=20, fmt=formatstring) myvar

close(unit=20)

deallocate(myvar)

end program test_max_write_columns

If I enter for example that I want 104 columns, then i cleanly get a .dat file
containing 104 columns of real data-values.

Now I have a bigger application in which i basically does the same. The write
statement at which my program gets aborted is the following:

write(unit=unit_number, fmt=formatstring) 2**i-1, &
mymeans(i), &
minval(myrel_errs(i,:)), maxval(myrel_errs(i,:)), &
myvariances(i), &
myrel_errs(i,:)

Formatstring gets built as follows:

write(unit=formatstring, fmt="(A, I3, A)") "(I10, ", nbruns+4, "ES25.15e2)"

but apparently things fail when I use that format string in the first write
statement: it works when nb_runs is 36, but it fails when nb_runs is 37. If
i change the format-descriptor to for example ES10.5e2 it works for larger
values than 37.

So my main guess is that it is a problem which is related to the
format-descriptor or so, but then I don't understand why my little test-program
*can* in fact write 104 columns, and this code from my application *can't*...

Does anybody have any suggestions on what could be wrong or things to look for?

Thanks,
Bart

--
"Share what you know. Learn what you don't."

FJRA

2005-06-08, 8:58 pm

When I need many columns to be written, I use ADVANCE='NO' and write
each column in a different statement. Maybe it helps.

Richard E Maine

2005-06-08, 8:58 pm

In article <1118257533.895792@seven.kulnet.kuleuven.ac.be>,
Bart Vandewoestyne <MyFirstName.MyLastName@telenet.be> wrote:

> I am trying to write about 100 columns of real data into a text file,
> but somehow I get an 'Aborted' message when my write statement is
> executed.
>
> I've decided to test if there were some limitations on the number of
> columns I could write, and apparently the following program works fine:
[example elided]

Ok. Note that compilers are allowed to place limits on record size.
Including a maxrec= specifier in the OPEN can help, though the compiler
may also have a limit on the allowable size of maxrec.

However, the fact that your test program ran ok argues against that as
the problem. It doesn't absolutely prove it, but argues against it. I do
note that you didn't show the OPEN from the "real" program. That can be
relevant. I'm guessing that it is probably ok, but that is only a guess
on my part.

>
> write(unit=unit_number, fmt=formatstring) 2**i-1, &
> mymeans(i), &
> minval(myrel_errs(i,:)), maxval(myrel_errs(i,:)), &
> myvariances(i), &
> myrel_errs(i,:)
>
> Formatstring gets built as follows:
>
> write(unit=formatstring, fmt="(A, I3, A)") "(I10, ", nbruns+4,
> "ES25.15e2)"
>
> but apparently things fail when I use that format string in the first write
> statement: it works when nb_runs is 36, but it fails when nb_runs is 37. If
> i change the format-descriptor to for example ES10.5e2 it works for larger
> values than 37.

Hmm. I don't see the connection between nbruns and the number of data
items in the write statement. I suppose that is probably ok because I
don't see why changing the ES25.15e2 to ES10.5e2 would change that...

Oh. Hold on a sec. What is the declaration of formatstring? Any chance
that the character string is one character too short for the ES25.15e2
form? Hard to say what the effects of truncating the last character
would be, but it just might possibly happen to work as long as the
number of items was small enough.

--
Richard Maine | Good judgment comes from experience;
email: my first.last at org.domain | experience comes from bad judgment.
org: nasa, domain: gov | -- Mark Twain

Bart Vandewoestyne

2005-06-08, 8:58 pm

In article <nospam-4B37C8.12272608062005@news.supernews.com>, Richard E Maine wrote:
>
> Ok. Note that compilers are allowed to place limits on record size.
> Including a maxrec= specifier in the OPEN can help, though the compiler
> may also have a limit on the allowable size of maxrec.
>
> However, the fact that your test program ran ok argues against that as
> the problem. It doesn't absolutely prove it, but argues against it.

This is why I wrote this simple test-program. Apparently, my compiler
can handle such cases... (I'm using the F-compiler for this, with as
only option the -O3 option)

> I do
> note that you didn't show the OPEN from the "real" program. That can be
> relevant. I'm guessing that it is probably ok, but that is only a guess
> on my part.

This is the OPEN from the "real" program:

open(unit=unit_number, file=filename, &
iostat=ios, status="replace", access="sequential", action="write")

Now I have for example ES9.2e2 and i can use at least nb_runs=100. So depending on the value of this edit-descriptor, my program gets aborted or not...
[color=darkred]
> Hmm. I don't see the connection between nbruns and the number of data
> items in the write statement. I suppose that is probably ok because I
> don't see why changing the ES25.15e2 to ES10.5e2 would change that...

There are actually nb_runs + 5 columns in my data file. The first column is
an integer value, and the other nb_runs+4 columns are reals of a certain
real kind that i want to write out saving as much as precision as i can.

> Oh. Hold on a sec. What is the declaration of formatstring?

character(len=50) :: formatstring

> Any chance
> that the character string is one character too short for the ES25.15e2
> form? Hard to say what the effects of truncating the last character
> would be, but it just might possibly happen to work as long as the
> number of items was small enough.

I guess len=50 should be enough in my case. I've PRINTed format string right
before it's used and it looks ok.

I really don't understand why i can write 100 colmns with my example test
program, but not in my other program... they both practically do the same, but
still the "real" program aborts if the width from my edit descriptor gets too
big.

Any other things I could check?

Regards,
Bart

--
"Share what you know. Learn what you don't."

Bart Vandewoestyne

2005-06-08, 8:58 pm

glen herrmannsfeldt

2005-06-08, 8:58 pm

Bart Vandewoestyne wrote:

> I am trying to write about 100 columns of real data into a text file,
> but somehow I get an 'Aborted' message when my write statement is
> executed.

(snip)

> ! Build up formatstring dynamically with 'cols' columns.
> write(unit=formatstring, fmt="(A, I0, A)") "(", cols, "ES25.15e2)"

While there are cases that require dynamic format strings, this
(usually) isn't one of them. If you put the largest number you
could expect, (or that your compiler allows), such as
"(I10,9999ES25.15e2)" and note that there aren't many printers
that will print lines that long. There is no penalty for extra
format items at the end that are not used.

(snip)

> Formatstring gets built as follows:

> write(unit=formatstring, fmt="(A, I3, A)") "(I10, ", nbruns+4, "ES25.15e2)"

> but apparently things fail when I use that format string in the first write
> statement: it works when nb_runs is 36, but it fails when nb_runs is 37. If
> i change the format-descriptor to for example ES10.5e2 it works for larger
> values than 37.

It seems to fail on the change from a 1010 character record to a 1035
character record, which looks suspiciously like a 1024 character
boundary. Is it possible that your real program sets a record length
limit on the OPEN statement?

-- glen

Richard E Maine

2005-06-08, 8:58 pm

In article <1118263050.107161@seven.kulnet.kuleuven.ac.be>,
Bart Vandewoestyne <MyFirstName.MyLastName@telenet.be> wrote:

> In article <nospam-4B37C8.12272608062005@news.supernews.com>, Richard E Maine
> wrote:
....[color=darkred]
> The test-program compiled with g95 so apparently g95 can handle that
> case, but when I compile my test-program with the F-compiler, i get the
> following message:
>
> Buffer overflow on output
> Program terminated by fatal I/O error
> Aborted

I suggest trying the specifier in the OPEN statement, except that I
"spelled" it incorrectly above. It is recl= instead of maxrec=. I had
also noticed that you seemed to be hitting it at about 1024 characters,
as glenn commented, and that did seem like a plausible default record
size limit. But the reported success of the test case threw me off
track. I suggest trying recl=some-sufficiently-large-number in the OPEN.

>Is the ADVANCE=NO option as suggested by
> another poster the way to go?

I'll not say that it couldn't work.... but I wouldn't expect it to.

Advance='no' is convenient for when you want or need to use multiple
write statements. But it is still writing to a sequential formatted
file. The fundamental file characteristics (like record length limits)
should not be any different. I use the word "should" a lot here. It is
possible that the implementation might happen to work in such a way that
the effective limit is different, but I wouldn't expect it.

The f2003 access='stream' is a different matter. In fact, avoiding
record length limitations has been cited as a reason for doing the
formatted version of access='stream'. (I don't fully understand why it
would help, and I recall someone here - Giles I think - emphasizing that
it shouldn't... but whether it should or not, that *WAS* advanced as an
argument). Anyway, you don't have f2003 yet.

--
Richard Maine | Good judgment comes from experience;
email: my first.last at org.domain | experience comes from bad judgment.
org: nasa, domain: gov | -- Mark Twain

glen herrmannsfeldt

2005-06-09, 3:58 am

Richard E Maine wrote:
(snip)

> The f2003 access='stream' is a different matter. In fact, avoiding
> record length limitations has been cited as a reason for doing the
> formatted version of access='stream'. (I don't fully understand why it
> would help, and I recall someone here - Giles I think - emphasizing that
> it shouldn't... but whether it should or not, that *WAS* advanced as an
> argument). Anyway, you don't have f2003 yet.

Well, the record oriented I/O is from the beginnings of Fortran,
allocate a buffer (or two) the length of the record (print line) fill
it, and then write it out. Each WRITE always starts with an empty
buffer.

Stream is the name C uses for its I/O system, sort of like Fortran
I/O UNITs. For text streams, C makes no guarantee that there isn't a
record length limit, and there likely will be if the underlying file
system has one. For unix systems, the file system does not have a
record structure, and bytes can be written independent of any record
length. My interpretation of access='stream' would be a C stream.
(Especially if the C library is used for I/O.) The C fopen function
can open a stream in text or binary mode. Some things depend on which
mode is used.

Using C files with the IBM VSE, MVS, or CMS systems can be somewhat
complicated. Files are written with fixed length records, or with
variable length records with a length indication at the beginning.
The C/370 library can even convert to/from ASA carriage control
characters for text files. I expect access='stream' to have similar
complications under those systems.

The DEC/Compaq/HP VMS system also has a record oriented file system,
but I believe it also has the ability to do stream files without record
marks.

-- glen

jamesgiles@att.net

2005-06-09, 3:58 am

glen herrmannsfeldt wrote:
> Richard E Maine wrote:
> (snip)
>
>
> Well, the record oriented I/O is from the beginnings of Fortran,
> allocate a buffer (or two) the length of the record (print line) fill
> it, and then write it out. Each WRITE always starts with an empty
> buffer.

There is no feature of Formatted I/O that requires that model.
The buffer needs to be at least as long as the amount of a given
record read or written by a single READ or WRITE statement in
order for the tab edit descriptors (particularly the tab left
and absolute tab specifiers) to work correctly. But that's a
requirement for Fortran 2003's stream I/O too. *If* stream I/O
had disallowed the use of the left tab and absolute tab specifiers
in the format, that would have constituted a significant change.
Even with the tab specifiers, arbitrary length records should
still be possible using multiple READ or WRITE statements and
non-advancing I/O - even without the stream feature.

The only difference I can find between stream I/O and non-stream
I/O in F2003 is that stream I/O requires the programmer to handle
record marks manually. But, since stream I/O doesn't provide
any control over what the record mark is, that manual handling
confers no new capabilities to the programmer, just a burden.

> [...] My interpretation of access='stream' would be a C stream.

But that *isn't* the F2003 interpretation. Any analogy to C is
probably misleading.

--
J. Giles

FJRA

2005-06-09, 3:58 am

Richard E Maine schrieb:
> In article <1118263050.107161@seven.kulnet.kuleuven.ac.be>,
> Bart Vandewoestyne <MyFirstName.MyLastName@telenet.be> wrote:
>
.....
>
> I'll not say that it couldn't work.... but I wouldn't expect it to.
>
> Advance='no' is convenient for when you want or need to use multiple
> write statements. But it is still writing to a sequential formatted
> file. The fundamental file characteristics (like record length limits)
> should not be any different. I use the word "should" a lot here. It is
> possible that the implementation might happen to work in such a way that
> the effective limit is different, but I wouldn't expect it.
>
I just tried opening with recl= some value, and at least in gfortran,
you get the runtime error:
Fortran runtime error: End of record
when you try to write more columns. I know that in CVF, recl is set to
2147483647 or 2147483640 or some other value (depends on the other
open options), when it's not specified.
Remaking your open should work.

Something else, I see the use of "I0" in the format to make the
formatstring, is this use allowed by the standard (I talk about the
zero width, which means in CVF and gfortran automatic width)?

Richard Maine

2005-06-09, 3:58 am

"FJRA" <a19980403@pucp.edu.pe> writes:

> Something else, I see the use of "I0" in the format to make the
> formatstring, is this use allowed by the standard (I talk about the
> zero width, which means in CVF and gfortran automatic width)?

Yes. That is an f95 feature. It was one of the few new features in f95.

--
Richard Maine
email: my last name at domain
domain: summertriangle dot net

Bart Vandewoestyne

2005-06-09, 8:57 am

In article <cMGdnTk6kJb3_zrfRVn-uQ@comcast.com>, glen herrmannsfeldt wrote:
>
> It seems to fail on the change from a 1010 character record to a 1035
> character record, which looks suspiciously like a 1024 character
> boundary.

Yep, this 1024 boundary was indeed the problem. I'm using the NAGWare
f95 compiler and according to the manpage it has a default maximum
record length of 1024 for formatted output.

> Is it possible that your real program sets a record length
> limit on the OPEN statement?

I didn't set 'recl' in my OPEN statement, so I was using the default of
1024. The reason why my testprogram *did* work with g95 is because it
has a default recl of 50000000.

Adding 'recl=<whatever i want>' solves my problem.

Thanks for helping me figuring this out guys!

Regards,
Bart

--
"Share what you know. Learn what you don't."

Bart Vandewoestyne

2005-06-09, 8:57 am

In article <nospam-941B16.15472608062005@news.supernews.com>, Richard E Maine wrote:
>
> I suggest trying the specifier in the OPEN statement, except that I
> "spelled" it incorrectly above. It is recl= instead of maxrec=. I had
> also noticed that you seemed to be hitting it at about 1024 characters,
> as glenn commented, and that did seem like a plausible default record
> size limit. But the reported success of the test case threw me off
> track. I suggest trying recl=some-sufficiently-large-number in the OPEN.

This is indeed the (portable!) solution to my problem.

One more question from my side: does the standard specify a limit on the value
that can be specified as recl? Or is it dependent on the compiler, operating
system, filesystem,...?
(sorry, i do not have a copy of the official Fortran standard...)

>
> I'll not say that it couldn't work.... but I wouldn't expect it to.
>
> Advance='no' is convenient for when you want or need to use multiple
> write statements. But it is still writing to a sequential formatted
> file. The fundamental file characteristics (like record length limits)
> should not be any different. I use the word "should" a lot here. It is
> possible that the implementation might happen to work in such a way that
> the effective limit is different, but I wouldn't expect it.

You are right. When I tried adding advance="no" to my write statements,
i still got stuck on this boundary of 1024.

Again, thanks for your help and suggestions!

Bart

--
"Share what you know. Learn what you don't."

Jan Vorbrüggen

2005-06-09, 8:57 am

Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-Trace: individual.net pSFXcVFz5rtIfYVPSo+14wiSo/lylZRe6wKijvdq2hDc9MtJ8p
User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206)
X-Accept-Language: de-DE, de, en-us, en
In-Reply-To: <1118308297.856219@seven.kulnet.kuleuven.ac.be>
Xref: number1.nntp.dca.giganews.com comp.lang.fortran:132762

> One more question from my side: does the standard specify a limit on the value
> that can be specified as recl? Or is it dependent on the compiler, operating
> system, filesystem,...?

THe limits in the standard are to be understood as guarantees, not as limits.
There is, I believe, no limit for the record length. Usually, such a limit
is set by the RTL or even the OS. For instance, if you want to write a for-
matted record to a normal text file on VMS, that record cannot be longer
than 32766 characters.

Jan

Bart Vandewoestyne

2005-06-09, 3:58 pm

In article <3gqidrFdrmhiU2@individual.net>, Jan Vorbrüggen wrote:
>
> THe limits in the standard are to be understood as guarantees, not as limits.
> There is, I believe, no limit for the record length.

Just being curious, i've tried to look this up in the September 23, 2002
committee draft 'ISO/IEC 1539-1' but i could not find anything
concerning the guaranteed/maximum record lengths.

Where exactly in the standards (or my committee draft) is this
described?

Regards,
Bart

--
"Share what you know. Learn what you don't."

jamesgiles@att.net

2005-06-09, 3:58 pm

Bart Vandewoestyne wrote:
> In article <3gqidrFdrmhiU2@individual.net>, Jan Vorbr=FCggen wrote:
imits.[color=darkred]
>
> Just being curious, i've tried to look this up in the September 23, 2002
> committee draft 'ISO/IEC 1539-1' but i could not find anything
> concerning the guaranteed/maximum record lengths.

There is always the statement in section 1.4 (Exclusions) to the
effect that the standard doesn't specify what size or complexity
exceeds a given implementation's limits. Implementations are
free to choose any limits on size or complexity they want as long
as they meet or exceed the few things the standard *does* specify.

For example, there is not actually any limit on the length of a
line of source code, or on the number of continuation lines that
an implementation may allow. There *is* a limit on conforming
programs: they must not rely on more than a certain number of
characters per line, or on more than a certain number of continuation
lines. These, in a subtle way, express minimum requirements on
implementations. The usual way this is interpreted is that all
implementations must be able to handle at least 132 character
lines and at least (in f2003) as many as 255 continuation lines.

This subtle logic applies to the the case in question in a reverse
sort of way. Since the standard doesn't give a limit (on programs)
for how long an I/O record may be, the implementation may select
any limit it likes. The limit can be zero (which it would be for
external files in an embedded implementation with no I/O support
at all). This discussion of limits applies to all I/O, including
the new stream I/O - which can also have implementation defined
limits on record lengths.

--=20
J=2E Giles

Richard E Maine

2005-06-09, 3:58 pm

In article <1118319092.180055@seven.kulnet.kuleuven.ac.be>,
Bart Vandewoestyne <MyFirstName.MyLastName@telenet.be> wrote:

> In article <3gqidrFdrmhiU2@individual.net>, Jan Vorbrüggen wrote:
>
> Just being curious, i've tried to look this up in the September 23, 2002
> committee draft 'ISO/IEC 1539-1' but i could not find anything
> concerning the guaranteed/maximum record lengths.
>
> Where exactly in the standards (or my committee draft) is this
> described?

A slight confusion here. Jan's statement was probably intended a bit
more generally, about other limits specified by the standard. The
standard doesn't specify one at all for record lengths; it is completely
processor-dependent.

In practice, in the case of recl, some compilers have modest default
values for recl, but I don't off-hand know of any cases where the
compiler won't let you set recl to a value as large as the operating
system and file system can handle (often 2gb or so, though some might be
less and some more). Umm, I take it back. There are 32-bit compilers
that can't handle records... or even files... larger than 2 or 4 gB even
when the OS and file system can.

On the general question of limits specified by the standard (not the
recl one, which the standard says nothing about), I think it is critical
to keep in mind the distinction between limits on programs and limits on
compilers; in fact, the two are often quite opposite sides of the same
question. It gets confusing if you talk about a limit without saying who
the limit applies to. For example...

The f95 standard limits the programmer to using arrays of rank no
greater than 7. This means that the compiler must support ranks of at
least 7. It does not limit the compiler from supporting ranks greater
than 7. Or, to phrase it as implied by Jan, the programmer is guaranteed
that he can use ranks of at least 7; he might be able to use ranks
greater than 7, but that would be a compiler-dependent extension.

--
Richard Maine | Good judgment comes from experience;
email: my first.last at org.domain | experience comes from bad judgment.
org: nasa, domain: gov | -- Mark Twain

glen herrmannsfeldt

2005-06-09, 8:58 pm

Richard E Maine wrote:

(snip)

> In practice, in the case of recl, some compilers have modest default
> values for recl, but I don't off-hand know of any cases where the
> compiler won't let you set recl to a value as large as the operating
> system and file system can handle (often 2gb or so, though some might be
> less and some more). Umm, I take it back. There are 32-bit compilers
> that can't handle records... or even files... larger than 2 or 4 gB even
> when the OS and file system can.

I believe that IBM MVS and VM/CMS, using the traditional BSAM access
method, still have the 32760 limit from OS/360 for fixed length record,
and 32752 for variable length records.

Too many control blocks store them in two bytes, and someone very early
used signed arithmetic for what should have been an unsigned value.
(S/360 didn't have so much support for unsigned arithmetic as it could
have.)

It might be that in some cases you can get to 65536 or so, but not
in most cases.

-- glen

glen herrmannsfeldt

2005-06-09, 8:58 pm

jamesgiles@att.net wrote:

> glen herrmannsfeldt wrote:

(snip)

[color=darkred]
> There is no feature of Formatted I/O that requires that model.
> The buffer needs to be at least as long as the amount of a given
> record read or written by a single READ or WRITE statement in
> order for the tab edit descriptors (particularly the tab left
> and absolute tab specifiers) to work correctly. But that's a
> requirement for Fortran 2003's stream I/O too.

I wasn't trying to say that it was required, but that tended to be
the popular model in the early years of Fortran. Going from OS/360
to PDP-10/TOPS-10 made this pretty obvious to me at the time.

> *If* stream I/O
> had disallowed the use of the left tab and absolute tab specifiers
> in the format, that would have constituted a significant change.
> Even with the tab specifiers, arbitrary length records should
> still be possible using multiple READ or WRITE statements and
> non-advancing I/O - even without the stream feature.

I had forgotten about T. That could definitely complicate
STREAM files, and I don't see any indication in the standard
(as far as the version I have) regarding STREAM and T.

> The only difference I can find between stream I/O and non-stream
> I/O in F2003 is that stream I/O requires the programmer to handle
> record marks manually. But, since stream I/O doesn't provide
> any control over what the record mark is, that manual handling
> confers no new capabilities to the programmer, just a burden.

It seems that RECL= is not allowed for STREAM files, yet they
still have a record structure. Until they make machines with
an infinite amount of memory, that would seem to me to be
a complication.

[color=darkred]
> But that *isn't* the F2003 interpretation. Any analogy to C is
> probably misleading.

I think you are right, but I am still not convinced that it will
actually happen, especially with the number of implementations
that use the C library for Fortran I/O. If the idea of STREAM
is that you can do I/O without worrying about record limits,
yet the system has to buffer an infinite amount of data,
something will break somewhere.

-- glen

jamesgiles@att.net

2005-06-09, 8:58 pm

glen herrmannsfeldt wrote:
> jamesgiles@att.net wrote:
....
> I had forgotten about T. That could definitely complicate
> STREAM files, and I don't see any indication in the standard
> (as far as the version I have) regarding STREAM and T.

There isn't any. So, the only conclusion possible is that,
like all other format specifiers, it's allowed for formatted
streams. It's not as bad you migh think. The tab specifiers
cannot tab to the left of the beginning of the current *partial*
record when using non-advancing I/O. So, if you begin a WRITE
statement with the file positioned in the middle of an output
record, the T1 format specifier moves to that location, not
to the beginning of the whole record. And TLn is limited to
tabbing no further left than that initial location for the
WRITE also.

Arbitrary length records should be fairly easy using non-
advancing I/O. There are no insurmountable problems there.
But, the problems that exist are the *same* whether stream
or non-stream I/O is to be used. For on thing, it means
that you can't process an arbitrarily long record with a
*single* READ or WRITE statement because the implementation
must allow for tabs in the format. I suppose the implementation
could special case situations where it can verify that tab will
not occur (like list-directed, or in implementations where the
format is completely parsed and analyzed before any data is
transferred).

Still, you could easily process a long record with a loop
around READ or WRITE statements using non-advancing I/O.
Presumably that's the intent of the committee.

--
J. Giles

Richard E Maine

2005-06-09, 8:58 pm

In article <nMWdnS1ydbRAOjXfRVn-qw@comcast.com>,
glen herrmannsfeldt <gah@ugcs.caltech.edu> wrote:

> jamesgiles@att.net wrote:
>
[color=darkred]
>
>
> I think you are right, but I am still not convinced that it will
> actually happen, especially with the number of implementations
> that use the C library for Fortran I/O.

Well, I'm confident that it is wrong to say that *ANY* analogy to C is
misleading for stream files. While the Fortran standard doesn't come out
and say "these are C stream files", that was most definitely a model
used in the development. I know because I was the one who first
formalized the proposal (though it went through quite a bit of work
later - my initial proposal had only stream unformatted, which I still
think is the most important of the stream functionalities - it *DOES*
have things that you couldn't do before... and it doesn't have any funny
issues of record structure at all.)

The proposal was explicitly adopted as a C interop feature. In fact, I
had been a little unsure that I'd be able to "sell" it because the
general topics for the f2003 feature set had supposedly already been
decided. The planed feature set did include a general topic of C
interop, but none of the stuff discussed previously had related to C
files. Turns out that my concerns about "selling" it were unfounded, the
main complaints being that I hadn't gone far enough (namely that I
hadn't included formatted stream).

C matters were explicitly discussed regularly in the development of
stream. It is true that there were also other matters discussed and that
the C tie-in isn't explicitly mentioned in the standard (at least I
don't see it at a quick glance - might have missed a spot, though). So
the one can't say definitively that F2003 stream, files are defined to
be C stream files. But I cannot agree with a statement that any analogy
to C stream files is misleading.

--
Richard Maine | Good judgment comes from experience;
email: my first.last at org.domain | experience comes from bad judgment.
org: nasa, domain: gov | -- Mark Twain

glen herrmannsfeldt

2005-06-09, 8:58 pm

jamesgiles@att.net wrote:

>
> glen herrmannsfeldt wrote:

>
>
> There isn't any. So, the only conclusion possible is that,
> like all other format specifiers, it's allowed for formatted
> streams. It's not as bad you migh think. The tab specifiers
> cannot tab to the left of the beginning of the current *partial*
> record when using non-advancing I/O.

WRITE(1,1) 0,(IRAND(X),I=1,2000000000),2000000000
1 FORMAT(2000000001I12,T1,I12)

--glen

jamesgiles@att.net

2005-06-09, 8:58 pm

Richard E Maine wrote:
....
> [...] But I cannot agree with a statement that any analogy
> to C stream files is misleading.

Well one difference is that people have been comparing C's binary
streams to Fortran's unformatted I/O, and C's text streams to Fortran's
Formatted I/O. That is not a valid analogy. In C, the text/binary
distinction describes a difference in the external structure of the
data to be processed. In Fortran, the formatted/unformatted
distinction
describes a difference in *how* the data is to be processed. I know
(because I wrote a working library that did it) that you can do
formatted I/O from/to a file with no record structure at all. That
would be a C binary file. I know (because I wrote a working library
that did it) that you can do unformatted I/O from/to an ASCII text
file.

Perhaps I should have said that "not all such analogies are valid",
or perhaps "some of the analogies are misleading"?

--
J. Giles

jamesgiles@att.net

2005-06-09, 8:58 pm

glen herrmannsfeldt wrote:
> jamesgiles@att.net wrote:
>
>
>
> WRITE(1,1) 0,(IRAND(X),I=1,2000000000),2000000000
> 1 FORMAT(2000000001I12,T1,I12)

Yes, there are things you can't do. I said it's not as bad
you think. I did't say there weren't problems. As I've already
pointed out, processing very long records would, in general,
require multiple READ or WRITE statements and non-advancing
I/O. In this case, it would also require that you know in
advance how mant data items there were so you don't have to
go back and insert it.

--
J. Giles

Richard E Maine

2005-06-09, 8:58 pm

In article <1118354486.585478.71160@g14g2000cwa.googlegroups.com>,
"jamesgiles@att.net" <jamesgiles@att.net> wrote:

> Perhaps I should have said that "not all such analogies are valid",
> or perhaps "some of the analogies are misleading"?

Those seem like much safer statements.

In fact, I'd be a fool to claim otherwise. You aren't going to catch me
claiming, for example that all the analogies are valid or that none of
them could possibly be misleading. I have more faith than that in the
"inventiveness" of people in making analogies. :-)

--
Richard Maine | Good judgment comes from experience;
email: my first.last at org.domain | experience comes from bad judgment.
org: nasa, domain: gov | -- Mark Twain

jamesgiles@att.net

2005-06-09, 8:58 pm

glen herrmannsfeldt wrote:
> jamesgiles@att.net wrote:
....
> [...] It's not as bad you migh think. The tab specifiers
>
> WRITE(1,1) 0,(IRAND(X),I=1,2000000000),2000000000
> 1 FORMAT(2000000001I12,T1,I12)

Perhaps I should have added that there is a compromise
solution. It requires no changes to the standard, but
relies on the "usual cop-out". What an impementation's
document could say is that it guarantees that you can
tab left by, say, 16K characters from the rightmost point
in the record that you processed so far. If you try to
tab further to the left than that, the implementation
will stop with a message essentially to the effect that
"as permitted by section 1.4 of the Fortran standard, this
implementation has a limit on the size/complexity of your
program - in this case it doesn't allow tabbing further
than 16K characters to the left of the rightmost part of
the record processed thus far."

This could be implemented using double buffering and 32K
of total memory. It would then permit arbitrary length
records to be processed with a single READ or WRITE
statement, but with the stated limit on the ability to
tab to the left.

Given that tabbing was a feature that you've already
admitted to have forgotten about, this comprimise solution
is likely to be quite satisfactory for most of your needs.
Whether anyone else even notices that the limit exists
is an interesting question. I have no intuition on how
frequently tab is used, much less whether it's an important
feature to those that process very long records.

--
J. Giles

Sponsored Links