Unix Programming - Write a struct to a FIFo

Author

Write a struct to a FIFo

erik.dassi@gmail.com

2006-05-13, 4:01 am

hi, How could I write a struct to a Fifo under linux ? I have a struct
named Packet with some stuff in it, and if possible I would like to
write it to a Fifo... thanks, erik

Barry Margolin

2006-05-13, 8:01 am

In article <1147504493.277667.55780@j73g2000cwa.googlegroups.com>,
erik.dassi@gmail.com wrote:

> hi, How could I write a struct to a Fifo under linux ? I have a struct
> named Packet with some stuff in it, and if possible I would like to
> write it to a Fifo... thanks, erik

write(Fifo_fd, &Packet, sizeof Packet);

WTP?

--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***

SM Ryan

2006-05-13, 7:01 pm

Barry Margolin <barmar@alum.mit.edu> wrote:
# In article <1147504493.277667.55780@j73g2000cwa.googlegroups.com>,
# erik.dassi@gmail.com wrote:
#
# > hi, How could I write a struct to a Fifo under linux ? I have a struct
# > named Packet with some stuff in it, and if possible I would like to
# > write it to a Fifo... thanks, erik
#
# write(Fifo_fd, &Packet, sizeof Packet);

Also fwrite with stdio. However writing and reading structs is a
bad idea because the C compiler is allowed to add and remove
at whim bytes in the struct not in any field. It's safer (and
more painful) to write a serialiser/deserialiser for the struct
or else make absolutely sure you can control exactly how the
compiler allocates structs on every system you are going to use.

--
SM Ryan http://www.rawbw.com/~wyrmwif/
Quit killing people. That's high profile.

Gordon Burditt

2006-05-13, 7:01 pm

># write(Fifo_fd, &Packet, sizeof Packet);
>
>Also fwrite with stdio.

Maybe not fwrite with stdio.

If multiple writers are writing structs to the same FIFO with write(),
and the size involved is within reasonable limits, you are guaranteed
atomic writes (no pieces of one struct appearing between pieces of
another struct as seen by the reader). With fwrite() there is
no such guarantee.

Gordon L. Burditt

Barry Margolin

2006-05-13, 7:01 pm

In article <126bs9sqlguks5f@corp.supernews.com>,
SM Ryan <wyrmwif@tango-sierra-oscar-foxtrot-tango.fake.org> wrote:

> Barry Margolin <barmar@alum.mit.edu> wrote:
> # In article <1147504493.277667.55780@j73g2000cwa.googlegroups.com>,
> # erik.dassi@gmail.com wrote:
> #
> # > hi, How could I write a struct to a Fifo under linux ? I have a struct
> # > named Packet with some stuff in it, and if possible I would like to
> # > write it to a Fifo... thanks, erik
> #
> # write(Fifo_fd, &Packet, sizeof Packet);
>
> Also fwrite with stdio. However writing and reading structs is a
> bad idea because the C compiler is allowed to add and remove
> at whim bytes in the struct not in any field. It's safer (and
> more painful) to write a serialiser/deserialiser for the struct
> or else make absolutely sure you can control exactly how the
> compiler allocates structs on every system you are going to use.

Since a FIFO only allows communication within the same system, that
shouldn't be a problem. However, your concern WOULD be valid for
writing structures across a network socket.

--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***

Kurtis D. Rader

2006-05-13, 10:02 pm

On Sat, 13 May 2006 20:18:58 -0400, Barry Margolin wrote:

> In article <126bs9sqlguks5f@corp.supernews.com>,
>
> Since a FIFO only allows communication within the same system, that
> shouldn't be a problem. However, your concern WOULD be valid for writing
> structures across a network socket.

Just to pick nits....

For the neophytes (which does not include Mr. Margolin) this is
theoretically an issue even when using a FIFO, or more generally any two
programs running on the same system that are exchanging structures. That's
because compiler options and pragmas (and other things?) can affect the
padding inserted in a structure. Therefore you cannot assume any two
programs can safely exchange arbitrary structures. It is only safe when
exchanging structures between the same program, compiled with the same
options, on the same CPU architecture.

Note that I say the above as someone who has earned their living as a
UNIX support specialist for 15+ years and has been asked by a customer
to diagnose this very issue. The customer felt that because the two
programs were running on the same system it was safe to exchange binary
data between them. They did not consider that the two programs had been
compiled by different compilers and options.

Casper H.S. Dik

2006-05-14, 4:01 am

"Kurtis D. Rader" <krader@skepticism.us> writes:

>For the neophytes (which does not include Mr. Margolin) this is
>theoretically an issue even when using a FIFO, or more generally any two
>programs running on the same system that are exchanging structures. That's
>because compiler options and pragmas (and other things?) can affect the
>padding inserted in a structure. Therefore you cannot assume any two
>programs can safely exchange arbitrary structures. It is only safe when
>exchanging structures between the same program, compiled with the same
>options, on the same CPU architecture.

Or when running a 32 bit and 64 bit program at either end of the pipe.

>Note that I say the above as someone who has earned their living as a
>UNIX support specialist for 15+ years and has been asked by a customer
>to diagnose this very issue. The customer felt that because the two
>programs were running on the same system it was safe to exchange binary
>data between them. They did not consider that the two programs had been
>compiled by different compilers and options.

In principle, different compiler options should not change structure
layout unless the compiler has options which break ABI conformance;
this is rare. (E.g., a compiler option which allows packing of structures
would make it impossible to use standard system headers *unless* they
bracketed all structure declarations with the appropriate "unpack"
pragma's)

Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

SM Ryan

2006-05-14, 4:01 am

Barry Margolin <barmar@alum.mit.edu> wrote:

# Since a FIFO only allows communication within the same system, that
# shouldn't be a problem. However, your concern WOULD be valid for
# writing structures across a network socket.

Knowing is better than guessing. What should really be available
is a way to specify the exact layout of a C struct, and let the
compiler handle the serialisation and deserialisation.

Cobol had this back in 1965.

--
SM Ryan http://www.rawbw.com/~wyrmwif/
TEMPORARILY CLOSED
BE OPENED AFTER FIRST PERIOD

Casper H.S. Dik

2006-05-14, 4:01 am

SM Ryan <wyrmwif@tango-sierra-oscar-foxtrot-tango.fake.org> writes:

>Barry Margolin <barmar@alum.mit.edu> wrote:

># Since a FIFO only allows communication within the same system, that
># shouldn't be a problem. However, your concern WOULD be valid for
># writing structures across a network socket.

>Knowing is better than guessing. What should really be available
>is a way to specify the exact layout of a C struct, and let the
>compiler handle the serialisation and deserialisation.

Cobol, of course, is the Kitchen Sink of langauges; in C this was
always left to particular libraries. Several ways of serializing
data in C are available, e.g., stdio (use printf/scanf) or Sun's
XDR (RFC 1014[1987] and 1832[1995]) which coupled with the "rpcgen"
compiler and the serialization routines allow for much of the same,
using C libraries and generated C.

Casper

Barry Margolin

2006-05-14, 7:04 pm

In article <pan.2006.05.14.02.17.13.66439@skepticism.us>,
"Kurtis D. Rader" <krader@skepticism.us> wrote:

> On Sat, 13 May 2006 20:18:58 -0400, Barry Margolin wrote:
>
>
> Just to pick nits....
>
> For the neophytes (which does not include Mr. Margolin) this is
> theoretically an issue even when using a FIFO, or more generally any two
> programs running on the same system that are exchanging structures. That's
> because compiler options and pragmas (and other things?) can affect the
> padding inserted in a structure. Therefore you cannot assume any two
> programs can safely exchange arbitrary structures. It is only safe when
> exchanging structures between the same program, compiled with the same
> options, on the same CPU architecture.

This is virtually never an issue except when communicating between
different machines. If it were, it would also affect reading binary
files (e.g. /etc/utmp), or passing structures between applications and
libraries or the kernel (do you know what compiler and options were used
when Sun compiled Solaris?).

So unless you go out of your way to force your structures to be laid out
in an unusual way (something like "#pragma packed"), the compiler should
conform to the system's ABI for structure layout, and the result should
be compatible with all other programs, libraries, and the kernel.

--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***

erik.dassi@gmail.com

2006-05-15, 4:14 am

I guess I should do :
Packet *p;
read(fifo_fd, p, sizeof(Packet));
in order to read back the struct from the fifo....
correct me if I'm wrong... I'm quite unexperienced on this things. :)
thank you very much, erik

Barry Margolin

2006-05-15, 10:01 pm

In article <1147686484.512998.125950@i39g2000cwa.googlegroups.com>,
erik.dassi@gmail.com wrote:

> I guess I should do :
> Packet *p;
> read(fifo_fd, p, sizeof(Packet));
> in order to read back the struct from the fifo....
> correct me if I'm wrong... I'm quite unexperienced on this things. :)
> thank you very much, erik

Since read() doesn't always return the full length of data you request,
you have to call it in a loop until you get all sizeof Packet bytes.

--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***

Kurtis D. Rader

2006-05-16, 10:01 pm

On Sun, 14 May 2006 07:37:34 +0000, Casper H. S. Dik wrote:

> In principle, different compiler options should not change structure
> layout unless the compiler has options which break ABI conformance; this
> is rare. (E.g., a compiler option which allows packing of structures
> would make it impossible to use standard system headers *unless* they
> bracketed all structure declarations with the appropriate "unpack"
> pragma's)

Correct. And that is exactly what happened in the case I had to resolve.
One of the programs in question had been built with "packed" structures
to save space. Thus making it incompatible with another program it had to
exchange data with. So you and Mr. Margolin are certainly correct that
in practice the problem I describe won't happen. But it's amazing how
many ways customers can find to shoot themselves in the foot :-)

Case in point. Just a few w

s ago I was called by a a well known defense
contractor to help them with a problem they were having exchanging binary
structures between a little endian and big endian system. Rather than
properly serialize the data in an architecture nuetral format they wanted a
compiler option that would reverse the natural byte, and bit, order used by
the compiler on one of the systems! Thankfully no such compiler flag exists.

Sponsored Links