Home > Archive > PERL POE > December 2006 > Buffering in PoCo::Child
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Buffering in PoCo::Child
|
|
| Mike Schroeder 2006-12-14, 7:17 pm |
| We have a POE app that reads multiple input files, reformats them and
the posts them to various binaries wrapped with PoCo::Child that in turn
load the data into our underlying system. The problem we are running
into is that PoCo::Child seems to be buffering two or three records at a
time. Sometimes we have interdependencies between records going into
various loaders, so we need these to be unbuffered.
In looking at PoCo::Child, I see how it uses POE::Wheel::Run, which in
turn uses POE::Driver::SysRW and that Driver::SysRW has a flush() method.
Do I need to patch PoCo::Child's write() to flush() after each write()?
Or is there some setting or a better way to get unbuffered writes to
PoCo::Child?
Thanks in advance for any advice.
| |
| Nicholas Perez 2006-12-14, 7:17 pm |
| It really depends on how you are processing the records and sending
them upstream. Could it be your Filter is doing the buffering?
On 12/14/06, Mike Schroeder <MikeSchroeder@donorware.com> wrote:
> We have a POE app that reads multiple input files, reformats them and
> the posts them to various binaries wrapped with PoCo::Child that in turn
> load the data into our underlying system. The problem we are running
> into is that PoCo::Child seems to be buffering two or three records at a
> time. Sometimes we have interdependencies between records going into
> various loaders, so we need these to be unbuffered.
>
> In looking at PoCo::Child, I see how it uses POE::Wheel::Run, which in
> turn uses POE::Driver::SysRW and that Driver::SysRW has a flush() method.
>
> Do I need to patch PoCo::Child's write() to flush() after each write()?
> Or is there some setting or a better way to get unbuffered writes to
> PoCo::Child?
>
>
> Thanks in advance for any advice.
>
| |
| Mike Schroeder 2006-12-14, 7:17 pm |
| Nicholas Perez wrote:
> It really depends on how you are processing the records and sending
> them upstream. Could it be your Filter is doing the buffering?
I'm just using the stock PoCo::Child, which uses POE::Filter::Line -- we
are just writing pipe-delimited lines -- nothing fancy.
[color=darkred]
>
> On 12/14/06, Mike Schroeder <MikeSchroeder@donorware.com> wrote:
| |
| Rocco Caputo 2006-12-15, 7:17 pm |
| You'll need to patch it. POE::Wheel::ReadWrite wasn't originally
designed for unbuffered writes. There's a patch to add it, but I
haven't included it because it fails one of POE's tests. You can
find the patch in POE's queue at rt.cpan.org.
--
Rocco Caputo - rcaputo@pobox.com
put() calls aren't buffered in filters. They are however buffered in
the driver, and they're flushed
On Dec 14, 2006, at 10:26, Mike Schroeder wrote:
> We have a POE app that reads multiple input files, reformats them
> and the posts them to various binaries wrapped with PoCo::Child
> that in turn load the data into our underlying system. The problem
> we are running into is that PoCo::Child seems to be buffering two
> or three records at a time. Sometimes we have interdependencies
> between records going into various loaders, so we need these to be
> unbuffered.
> In looking at PoCo::Child, I see how it uses POE::Wheel::Run, which
> in turn uses POE::Driver::SysRW and that Driver::SysRW has a flush
> () method.
>
> Do I need to patch PoCo::Child's write() to flush() after each write
> ()? Or is there some setting or a better way to get unbuffered
> writes to PoCo::Child?
>
>
> Thanks in advance for any advice.
|
|
|
|
|