NEW_LINE()

I just noticed that the new standard (F2003) overspecifies
formatted stream I/O by always interpreting the record
structure of the file instead of just passing the characters
found straight through.  For example, if a file happens to
contain (as text data) something that the I/O library would
treat as a record mark if the file were opened as a non-stream
formatted file, the I/O library consumes that data and passes
through one character that matches the result of the NEW_LINE
intrinsic.  Or, if your output data contains (as text data) one
of those NEW_LINE characters, that data will not be written
and an implementation dependent record mark will be output
instead.

I guess someone pointed this out to me before, but I tought
the NEW_LINE intrinsic was merely informative (and easy
to ignore).  I didn't realize that the stream I/O itself didn't
move the data unchanged.

Well, seems we still need to add a new feature to support *real*
stream I/O.

--
J. Giles

"I conclude that there are two ways of constructing a software
design: One way is to make it so simple that there are obviously
no deficiencies and the other way is to make it so complicated
that there are no obvious deficiencies."   --  C. A. R. Hoare

"James Giles" <jamesgiles@worldnet.att.net> writes:

> I just noticed that the new standard (F2003) overspecifies
> formatted stream I/O by always interpreting the record
> structure of the file instead of just passing the characters
> found straight through. [describes what he means]
> Well, seems we still need to add a new feature to support *real*
> stream I/O.

A few notes.

1. This is basically modelled directly after C, stream I/O having been
introduced under the guise of a C interop feature.

Please note that I'm *NOT* necesssarily saying that I like it.
I'm just pointing out where this came from.

2. Unformatted streams are "real" in the sense that the data passes
through completely transparently. Initially, unformatted streams
are all that I proposed, but other people also wanted formatted
ones.

You could even use unformatted streams to make formatted files
on pretty much all current systems.  It is system-dependent
in that you'd have to add the right newline characters (and
even more so in that the file system has to be one where adding
control characters to the end of each record is how to make a
formatted file - but that's how most current systems are, whether
I like it or not).  Anyway, on current systems, one could build
up other things based on unformatted streams.

--
Richard Maine                       |  Good judgment comes from experience;
email: my first.last at org.domain  |  experience comes from bad judgment.
org: nasa, domain: gov              |        -- Mark Twain

Richard E Maine wrote:
> "James Giles" <jamesgiles@worldnet.att.net> writes:
> 
>
> A few notes.
>
> 1. This is basically modelled directly after C, stream I/O having been
>    introduced under the guise of a C interop feature.
>
>    Please note that I'm *NOT* necesssarily saying that I like it.
>    I'm just pointing out where this came from.

I can't find a corresponding requirement in the C standard.
I think *some* I/O functions in may C do something similar
(it's hard to tell), but that it's independent of whether formatting
is applied or not.  §5.2.1 of the C standard (I'm actually
looking at a draft of C99 dated Aug. 3, 1998, WG14/N843)
says that the execution character set contains a new-line
character, but doesn't say this character is the result of a
conversion of any implementation dependent file convention
(though it does explicitly say that for the new-line in the
source file character set - they regard the character set used
for source files and that of the execution environment as
possibly distinct).  I take that to mean that such a conversion
is not intended or allowed.  I might be wrong.

> 2. Unformatted streams are "real" in the sense that the data passes
>    through completely transparently. Initially, unformatted streams
>    are all that I proposed, but other people also wanted formatted
>    ones.
>
>    You could even use unformatted streams to make formatted files
>    on pretty much all current systems.  It is system-dependent
>    in that you'd have to add the right newline characters (and
>    even more so in that the file system has to be one where adding
>    control characters to the end of each record is how to make a
>    formatted file - but that's how most current systems are, whether
>    I like it or not).  Anyway, on current systems, one could build
>    up other things based on unformatted streams.

Ok.  So the upshot of this is that stream I/O is only *really*
stream I/O if you open it for unformatted and then do your
format conversions on internal files based on the buffers you
transmit unformatted.  As I said, the standard overspecified:
too much stuff when the really needed functionality is much
smaller and simpler.

--
J. Giles

"I conclude that there are two ways of constructing a software
design: One way is to make it so simple that there are obviously
no deficiencies and the other way is to make it so complicated
that there are no obvious deficiencies."   --  C. A. R. Hoare

James Giles wrote:

> Richard E Maine wrote:
 
 

I don't know about real stream I/O.  IBM's MVS (and related systems),
VM/CMS and DEC/Compaq/HP's VMS (in some record formats) don't use
control characters to terminate records.  They use either fixed length
records, or variable length records containing a block header with the
block length.

In the case of C, the library converts between newline terminated
lines and the record structure of the file system.
 
 
 

> I can't find a corresponding requirement in the C standard.
> I think *some* I/O functions in may C do something similar
> (it's hard to tell), but that it's independent of whether formatting
> is applied or not.  §5.2.1 of the C standard (I'm actually
> looking at a draft of C99 dated Aug. 3, 1998, WG14/N843)
> says that the execution character set contains a new-line
> character, but doesn't say this character is the result of a
> conversion of any implementation dependent file convention
> (though it does explicitly say that for the new-line in the
> source file character set - they regard the character set used
> for source files and that of the execution environment as
> possibly distinct).  I take that to mean that such a conversion
> is not intended or allowed.  I might be wrong.

Well, C makes a distinction between text files and binary files,
though the two are the same under unix-like systems.  It doesn't
have to do with the use of formatting, but with conversin of newline
characters.  Under DOS/Windows on writing a text file each newline
character is converted to two characters, CR and LF (X'0d' and X'0a')
and converted back on input.  A file opened in binary does not have
the conversions done.

 
 

Except ones like I described above.

> Ok.  So the upshot of this is that stream I/O is only *really*
> stream I/O if you open it for unformatted and then do your
> format conversions on internal files based on the buffers you
> transmit unformatted.  As I said, the standard overspecified:
> too much stuff when the really needed functionality is much
> smaller and simpler.

How about Fortran implementations of getchar(), putchar(),
printf() and scanf()?

(Just kidding.)

-- glen

glen herrmannsfeldt wrote:
...
> I don't know about real stream I/O.  IBM's MVS (and related systems),
> VM/CMS and DEC/Compaq/HP's VMS (in some record formats) don't use
> control characters to terminate records.  They use either fixed length
> records, or variable length records containing a block header with the
> block length.
>
> In the case of C, the library converts between newline terminated
> lines and the record structure of the file system.

I just read the appropriate parts of the C standard and I find no
mention of that.  The description of fscanf, for example, says
nothing about it.  Nor, can I see any reason (in any language)
to use stream I/O on text files, instead of record based I/O,
if the data on the stream is modified in that way.  It eliminates
the whole value of the concept.

--
J. Giles

"I conclude that there are two ways of constructing a software
design: One way is to make it so simple that there are obviously
no deficiencies and the other way is to make it so complicated
that there are no obvious deficiencies."   --  C. A. R. Hoare

James Giles wrote:
>
> glen herrmannsfeldt wrote:
> ... 
>
> I just read the appropriate parts of the C standard and I find no
> mention of that.  The description of fscanf, for example, says
> nothing about it.  Nor, can I see any reason (in any language)
> to use stream I/O on text files, instead of record based I/O,
> if the data on the stream is modified in that way.  It eliminates
> the whole value of the concept.

In the C 'fopen' library routine, one can add the letter 'b' to the
file mode to turn on 'binary' mode.  The default is 'text' mode.
In my copy of C89 (November 9, 1987 draft) this is described in
§4.9.2.

As Glen mentioned, this has little meaning on unix-like systems.
But on other systems can be quite important.  I first encountered
this in the C compiler for the long-defunct Cray-1 Operating System
(COS).  It used control words to delineate records - even for simple
text datasets.  So one had to set the 'b' in the mode to get a true
binary stream.

Walt
-...-
Walt Spector
(w6ws at earthlink dot net)

Walter Spector wrote:
> James Giles wrote: 
>
> In the C 'fopen' library routine, one can add the letter 'b' to the
> file mode to turn on 'binary' mode.  The default is 'text' mode.
> In my copy of C89 (November 9, 1987 draft) this is described in
> §4.9.2.


OK.  Now I see it.  But, as I said before, that means the decision
is independent of whether you use formatting on the file or not.
As Fortran 2003's implementation of streams works, it introduces
no functionality that non-advancing formatted I/O doesn't already
provide in F95.

--
J. Giles

"I conclude that there are two ways of constructing a software
design: One way is to make it so simple that there are obviously
no deficiencies and the other way is to make it so complicated
that there are no obvious deficiencies."   --  C. A. R. Hoare

On 01 Oct 2004 15:27:45 -0700, Richard E Maine <nospam@see.signature>
wrote:


>A few notes.
>
>1. This is basically modelled directly after C, stream I/O having been
>   introduced under the guise of a C interop feature.

That is unfortunate.  Why is there a fetish about interoperating with
C?  Why not a push to interoperate with BASIC, COBOL, Lisp, etc.?
Even more, since C came after Fortran (as well as a bunch of other
languages), why does it not fall on the C community to ensure
interoperation with everyone else?

>   Please note that I'm *NOT* necesssarily saying that I like it.
>   I'm just pointing out where this came from.

Your place in heaven is ensured. :-)

>2. Unformatted streams are "real" in the sense that the data passes
>   through completely transparently. Initially, unformatted streams
>   are all that I proposed, but other people also wanted formatted
>   ones.

The committee is fairly democratically run, isn't it?  And its members
tend to be nice to each other?  That, unfortunately, leads to bloated
and ambiguous results.

>   You could even use unformatted streams to make formatted files
>   on pretty much all current systems.  It is system-dependent
>   in that you'd have to add the right newline characters (and
[snip]

I've done that often enough.  As long as there is unformatted stream,
you can do whatever you want.  Your program prepares exactly the bits
you want to go out, and unformatted stream delivers those.

I'm not sure I want to hear the reasoning behind formatted stream.

Ken Plotkin

> I'm not sure I want to hear the reasoning behind formatted stream.

You do know that the FTP protocol makes the distinction between
transferring a file in binary or in ASCII mode, do you not? This
is exactly the same distinction.

Jan

> I just noticed that the new standard (F2003) overspecifies
> formatted stream I/O by always interpreting the record
> structure of the file instead of just passing the characters
> found straight through.  For example, if a file happens to
> contain (as text data) something that the I/O library would
> treat as a record mark if the file were opened as a non-stream
> formatted file, the I/O library consumes that data and passes
> through one character that matches the result of the NEW_LINE
> intrinsic.  Or, if your output data contains (as text data) one
> of those NEW_LINE characters, that data will not be written
> and an implementation dependent record mark will be output
> instead.

And that is exactly as it should be. For instance, it makes it
easy for Steve Lionel to add a new CONVERT option to the OPEN
statement that will allow a program to transparently read or
write a text file for Mac, Windows or Unix use, as the case may
be. C does not hide the implementation-dependant NEW_LINE behind
this curtain.

Jan