Home > Archive > PHP Language > June 2007 > PHP parser XML
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
|
|
|
| Hello Folks,
My environment is : FreeBSD 6 , apache 1.3.7 and PHP5
I want to parse a big XML file (39MB) . In my php.ini I set memory_limit to
64M but I still have a error message followed :
Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to
allocate 57 bytes) ...
Your helps would be appreciated .
Thanks
CK
| |
| Christoph Burschka 2007-05-25, 3:59 am |
| CK schrieb:
> Hello Folks,
> My environment is : FreeBSD 6 , apache 1.3.7 and PHP5
> I want to parse a big XML file (39MB) . In my php.ini I set memory_limit to
> 64M but I still have a error message followed :
> Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to
> allocate 57 bytes) ...
>
> Your helps would be appreciated .
> Thanks
>
> CK
The parsing process likely requires more memory than just the size of
the file. I've had similar problems when reading or processing other
files that were significantly above half the memory limit.
You could try to optimize the memory usage - perhaps split the file up
in chunks in temp files and process them one at a time, rather than read
the entire file into memory. Not all XML files can be split like this,
but those that have a long list of identical entities in series (say, an
RSS feed with hundreds of items) can be. You'd just need to add the
opening and closing tags to each chunk.
--
cb
| |
| Roy Kaldung 2007-05-25, 3:59 am |
| CK wrote:
> Hello Folks,
> My environment is : FreeBSD 6 , apache 1.3.7 and PHP5
> I want to parse a big XML file (39MB) . In my php.ini I set memory_limit to
> 64M but I still have a error message followed :
> Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to
> allocate 57 bytes) ...
>
> Your helps would be appreciated .
> Thanks
Hi,
try to parse your XML with an SAX Parser
instead a DOM parser (-> )if possible. A Dom parser builds a complete
tree of the XML file in memory, a SAX parser is 'event driven'. For
event driven parsing see http://www.php.net/manual/en/ref.xml.php
hth,
Roy
| |
|
| Christoph Burschka wrote:
> CK schrieb:
>
> The parsing process likely requires more memory than just the size of
> the file. I've had similar problems when reading or processing other
> files that were significantly above half the memory limit.
>
> You could try to optimize the memory usage - perhaps split the file up
> in chunks in temp files and process them one at a time, rather than read
> the entire file into memory. Not all XML files can be split like this,
> but those that have a long list of identical entities in series (say, an
> RSS feed with hundreds of items) can be. You'd just need to add the
> opening and closing tags to each chunk.
>
> --
> cb
Thanks for your input . But my XML file can not be split .
| |
|
| Roy Kaldung wrote:
> CK wrote:
>
> Hi,
>
> try to parse your XML with an SAX Parser
> instead a DOM parser (-> )if possible. A Dom parser builds a complete
> tree of the XML file in memory, a SAX parser is 'event driven'. For
> event driven parsing see http://www.php.net/manual/en/ref.xml.php
>
> hth,
> Roy
Thank you . I tried some examples from
http://www.php.net/manual/en/ref.xml.php but my XML file is really too
big . Do you know is some free XML parser out there on the net can display
the structure of XML file ? I visit MagicParser.com but we have to buy.
| |
|
|
"CK" <nospams@nospams.com> wrote in message
news:oEC5i.24262$gk7.118002@wagner.videotron.net...
> Roy Kaldung wrote:
>
memory_limit[color=darkred]
>
> Thank you . I tried some examples from
> http://www.php.net/manual/en/ref.xml.php but my XML file is really too
> big . Do you know is some free XML parser out there on the net can display
> the structure of XML file ? I visit MagicParser.com but we have to buy.
>
If you can't find anything for PHP, you might check the Perl CPAN site to
see if someone might already have a module you could use. Depending what you
needed maybe "XML" or "XML::Simple".
Since you're running FreeBSD, it might be worth chekcing ports too.
Tom
--
Basic Accounts $5.95 / mo. or $39.95 / yr.
http://newsguy.com/overview.htm
| |
|
| Tom wrote:
>
> "CK" <nospams@nospams.com> wrote in message
> news:oEC5i.24262$gk7.118002@wagner.videotron.net...
> memory_limit
>
> If you can't find anything for PHP, you might check the Perl CPAN site to
> see if someone might already have a module you could use. Depending what
> you needed maybe "XML" or "XML::Simple".
>
> Since you're running FreeBSD, it might be worth chekcing ports too.
>
> Tom
Thanks Tom,
I tried simplexml_load_file from http://ca.php.net/simplexml_load_file .
It seems to take time to parse my XML file.
Let me explain to you my problem :
I have to download to my serveur every day a big XML file (about 40MB ).
The content of this XML file is about the location,products, description,
price, etc. . I have to write a PHP program to do the search (with
selection criteria) about the products into this file .
The time to parse this XML file is about 15 secs . And I do not want that
ervery one who visit my page would wait at least 15 secs . So that I want
is to load only once this file into an objet according to
simplexml_load_file ( $xml_product
=simplexml_load_file('myXMLfile.xml') ) .
Do you know how to define that objet globally so that every one who visit my
page does not have to read or load the XML file ?
| |
| Roy Kaldung 2007-05-25, 6:58 pm |
| CK schrieb:
> Tom wrote:
>
>
> Thanks Tom,
>
> I tried simplexml_load_file from http://ca.php.net/simplexml_load_file .
> It seems to take time to parse my XML file.
> Let me explain to you my problem :
> I have to download to my serveur every day a big XML file (about 40MB ).
> The content of this XML file is about the location,products, description,
> price, etc. . I have to write a PHP program to do the search (with
> selection criteria) about the products into this file .
> The time to parse this XML file is about 15 secs . And I do not want that
> ervery one who visit my page would wait at least 15 secs . So that I want
> is to load only once this file into an objet according to
> simplexml_load_file ( $xml_product
> =simplexml_load_file('myXMLfile.xml') ) .
> Do you know how to define that objet globally so that every one who visit my
> page does not have to read or load the XML file ?
You should try do put the parsing task into a cron job right after the
download of the file and putting the relevant contents into a database
for the search. It's probably better to spend one time 15 secs or more
than for each single request.
Roy
| |
|
|
|
|
|
|
|