| Kamal R. Prasad 2004-12-20, 3:59 pm |
| From: Paul Pluzhnikov (ppluzhnikov-nsp@charter.net)
Subject: Re: loss of data on pipes
View this article only
Newsgroups: comp.unix.programmer
Date: 2004-12-16 21:51:53 PST
"Adrian Wong" <atwong@gmail.com> writes:
> Does this sound right?
>No. It sounds like a bug in your program.
> I didnt think pipes would be "lossy"?
>They aren't.
They aren't lossy -if you mean that they should be writing out all
that they *manage* to read in.
> Shouldnt the child processes printing to
> a pipe be blocked if the pipe is full, i.e. the
> whole setup has it's own throttling mechanism?
>Yes.
The best thing to do is to write() in a loop, till all of the bytes
are sent out. I mean, write() returns the no. of bytes actually
written -which you use to determine how much more needs to be written.
> BTW I am interested in any other solution for
> the mangled output problem besides writing to
> regular files.
>The trivial solution is to use write(2) instead of printf(3) in
>the child output. So long as the buffer you write() is smaller
>then PIPE_BUF (32768 on AIX-5.1), your writes will be atomic and no
>"mangling" will happen.
Some unix'es have an upper limit of 4KB. If you are communicating
small amounts of data between related processes -pipes are indeed the
preferred means. For larger amounts, try shared memory or mmap()'ing a
file wherein the child writes to it, and the parent reads in and
deletes it. Both of these mechanisms will require a synchronization
primitive to coordinate between child & parent.
You may want to post some pseudo code for people to point out the
mistake.
regards
-kamal
|