| Fredderic 2008-03-29, 4:49 am |
| On Fri, 28 Mar 2008 18:08:29 -0700 (PDT),
vitick@gmail.com wrote:
> On Mar 28, 5:32 pm, Alan Anderson <arand...@insightbb.com> wrote:
> But, nevertheless, lets attempt to go after a solution that does not
> use threads.
Last time I did this kind of thing manually (rather a while back), I
used a pair of [fileevent] handlers that represent "copy" and "wait"
modes of operation. (With, of course, non-blocking input and output.)
Start in "copy" mode; set up a writable [fileevent] on the outbound
stream, and when it fires, read a chunk of data from the source stream
and write it out. This will dump data as fast as we can to the output
stream, but would spin like crazy if there's no input to read. This is
optimised for the case where the source is faster then the destination,
for example, dumping a local file to a network socket.
When we get a zero-byte read from the input (not eof), you kill the
writable handler, and set up a readable [fileevent] on the source
stream instead. If you can tell how much data's being buffered on the
outgoing stream, then read and dump the data available and switch back
to the writable handler if you notice that a block or so of data is
just sitting in the outbound buffer (this supports the case where the
outbound stream is the faster one). If you can't, then just switch
right away to avoid buffering the whole source stream in memory. Either
way doing the usual copy-and-write will make sure there's as much data
available to be written as we've got.
It might have sub-optimal performance where small data chunks are
coming though erratically, where allowing data to collect in the output
buffers for a second or two can sometimes be useful. On the other
hand, if you're copying an interractive stream, then pushing the data
through regardless is a good idea. (This is a distinction that [fcopy]
doesn't seem to make.)
There are other algorithms, I'm sure, but this one is fairly simple,is
as generic as I have ever needed, and has served me well in the past.
Of course, if you know you'll always be reading from a file, or always
writing to a file, then the whole thing becomes a lot simpler because
you can get away with just one of the two handlers, and none of the
code to switch between them.
> How do you write a file to socket copy procedure that notifies me
> about it's progress every so many bytes and does not block while
> doing so.
If the [fileevent] handlers take an extra argument which is a notifier
function to call, similar to [fcopy]s -command, then just call it with
the current amount of data called, and it's up to that function to do
what it needs to do without blocking too long. Simply setting a
progress widget percentage shouldn't take too long.
> How about if I want to control how fast the copy progresses without
> blocking other events?
That's very difficult to do properly, in my experience... The easiest
way is to just keep a count of the current data transferred, and a
recurring timer that subtracts some fixed amount at fixed intervals
(optionally clipping if it goes negative). Then you stop the process
if the sent data exceeds some threshold, and restart it again from the
timer if it was stopped.
> Or, lets say, theoretically, that I want to stream a music or a video
> file where the user can fast forward and go back, without blocking
> and without sending the entire file over.
You're looking at something much more specialised there. For
starters, how will you know when the user has elected to rewind or
fast-forward the stream? Block-based protocols that request a specific
portion of the file to be sent at a time work well there, as can any
download protocol with a "resume at location" option. The client can
just cut the connection, and "resume" from a different arbitrary
point. It's still way beyond the realms of a generic [fcopy].
> And also, how about a long running procedure that doesn't block?
If it blocks, you're doing something wrong. That's what event driven
programming was invented for. There should only be one event loop, and
it runs from start to finish. Busy-loops are bad, use a timer.
Infinite loops are bad, use an idle timer. Polling is bad, use
[fileevent] instead.
Fredderic
|