PERL POE - using poe for web application server

Author

using poe for web application server

Manfred Meier

2006-04-29, 4:09 am

Hello list,

I'm searching for a tool to solve one of my problems. Perhaps somebody
has a hint. The situation is: (on linux)

- the software is for calculating routes in public transport networks
for special purposes,
- people put the names of origin and destination in their webbrowser
into a form and submit this form,
- using the CGI mechanism apache calls a (existing) perl program, that
calculates the route,

That all is running fine. The perl program uses C subroutines over a
swig interface. The C subroutines read the graph data from disk and
perform a path finding algorithm.

The problem is that each user request results in reading the whole data
for the routing graph (can be 200 MB or more). I would like to make this
faster. So I think basically I need a daemon which once reads the data
and then listens to requests from the cgi-program.

I'm trying to find out if poe or mod-perl or fastcgi are possible
solutions. Can somebody give a hint, if poe makes sense here.

Many thanks
Manfred Meier

Mathieu Longtin

2006-04-29, 7:08 pm

If your data is a C structure, and you may want to load it
in shared memory, that way every CGI can access it.

Or you can use POE to run an HTTP server, which loads the
data on startup, and calls the C subroutine to answer each
call.

Performance wise, the problem with POE is its not
multi-threaded, so you won't take advantage of a
multi-processor/multi-core system. You can probably use
some pooling and have multiple servers running, and use the
shared memory thing so not every process has its own copy
of the 200MB data.

-Mathieu

--- Manfred Meier <m.meier@spiekermann.de> wrote:

> Hello list,
>
> I'm searching for a tool to solve one of my problems.
> Perhaps somebody
> has a hint. The situation is: (on linux)
>
> - the software is for calculating routes in public
> transport networks
> for special purposes,
> - people put the names of origin and destination in their
> webbrowser
> into a form and submit this form,
> - using the CGI mechanism apache calls a (existing) perl
> program, that
> calculates the route,
>
> That all is running fine. The perl program uses C
> subroutines over a
> swig interface. The C subroutines read the graph data
> from disk and
> perform a path finding algorithm.
>
> The problem is that each user request results in reading
> the whole data
> for the routing graph (can be 200 MB or more). I would
> like to make this
> faster. So I think basically I need a daemon which once
> reads the data
> and then listens to requests from the cgi-program.
>
> I'm trying to find out if poe or mod-perl or fastcgi are
> possible
> solutions. Can somebody give a hint, if poe makes sense
> here.
>
> Many thanks
> Manfred Meier
>
>
>
>
>
>
>
>
>
>

________________________________________
__________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

Al Tobey

2006-05-01, 7:12 pm

On Sat, 2006-04-29 at 11:06 +0200, Manfred Meier wrote:
> Hello list,
>
> I'm searching for a tool to solve one of my problems. Perhaps somebody
> has a hint. The situation is: (on linux)
>
> - the software is for calculating routes in public transport networks
> for special purposes,
> - people put the names of origin and destination in their webbrowser
> into a form and submit this form,
> - using the CGI mechanism apache calls a (existing) perl program, that
> calculates the route,
>
> That all is running fine. The perl program uses C subroutines over a
> swig interface. The C subroutines read the graph data from disk and
> perform a path finding algorithm.
>
> The problem is that each user request results in reading the whole data
> for the routing graph (can be 200 MB or more). I would like to make this
> faster. So I think basically I need a daemon which once reads the data
> and then listens to requests from the cgi-program.
>
> I'm trying to find out if poe or mod-perl or fastcgi are possible
> solutions. Can somebody give a hint, if poe makes sense here.

Short answer: POE is probably a both overkill and underkill at the same
time. 200mb would still cost you quite a bit to transfer even over a
unix domain socket. It might work apache proxied the entire CGI
request to a POE socket that returned the entire page. As mentioned in
another post, you could run into new throughput problems there.

You could create a singleton class in mod_perl that would cache the data
per-subprocess then play MaxRequestsPerChild to control memory
consumption. I think there is a chapter on singletons in Damian
Conway's Perl Best Practices, which is a good buy anyways.

Another module to check is Cache::Cache which I believe can use shared
memory to share the data between apache subprocesses.

There is also a fairly slick system developed by the LiveJournal people
intended to work with MySQL that does shared memory and inter-host
caching.

http://search.cpan.org/author/DCLIN.../Cache/Cache.pm
http://search.cpan.org/~abw/Class-S...03/Singleton.pm
http://www.livejournal.com/code/

-Al Tobey

>
> Many thanks
> Manfred Meier
>
>
>
>
>
>
>
>
>
>

** ** ** PRIVILEGED AND CONFIDENTIAL ** ** **
This email transmission contains privileged and confidential information intended only for the use of the individual or entity named above. Any unauthorized review, use, disclosure or distribution is prohibited and may be a violation of law. If you are
not the intended recipient or a person responsible for delivering this message to an intended recipient, please delete the email and immediately notify the sender via the email return address or mailto:postmaster@priority-health.com. Thank you.

- end -

Sponsored Links