Home > Archive > PERL Beginners > June 2007 > DBI, postgresql and large table
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
DBI, postgresql and large table
|
|
| Andrej Kastrin 2007-06-28, 3:59 am |
| Dear all,
I need to process postgresql table with DBI module. The script below
works well for small tables, but when I want to process larger tables
(100000 rows) out-of-memory occurs.
Any suggestion? Thanks in advance,
Andrej
use strict;
use warnings;
use DBI;
my $dbh = DBI->connect("DBI:Pg:dbname=medline;host=localhost",
"postgres", "secret", {'RaiseError' => 1});
my $sth = $dbh->prepare("SELECT text_a, text_b FROM tmp_table");
$sth->execute();
while (my $ref = $sth->fetchrow_hashref()) {
my $field_a = $ref->{'text_a'};
my $field_b = $ref->{'text_b'};
print "$field_a \t $field_b\n";
}
| |
| Chas Owens 2007-06-28, 3:59 am |
| On 6/27/07, Andrej Kastrin <andrej.kastrin@siol.net> wrote:
> Dear all,
>
> I need to process postgresql table with DBI module. The script below
> works well for small tables, but when I want to process larger tables
> (100000 rows) out-of-memory occurs.
>
> Any suggestion? Thanks in advance,
> Andrej
>
> use strict;
> use warnings;
> use DBI;
>
> my $dbh = DBI->connect("DBI:Pg:dbname=medline;host=localhost",
> "postgres", "secret", {'RaiseError' => 1});
> my $sth = $dbh->prepare("SELECT text_a, text_b FROM tmp_table");
> $sth->execute();
>
> while (my $ref = $sth->fetchrow_hashref()) {
> my $field_a = $ref->{'text_a'};
> my $field_b = $ref->{'text_b'};
> print "$field_a \t $field_b\n";
> }
* When does it fail and what is the exact error message?
* Have you tried to execute this command in an SQL editor (psql if I
remember correctly)?
* How much memory do you have on that box?
| |
| Andrej Kastrin 2007-06-28, 3:59 am |
| Chas Owens wrote:
> On 6/27/07, Andrej Kastrin <andrej.kastrin@siol.net> wrote:
>
> * When does it fail and what is the exact error message?
> * Have you tried to execute this command in an SQL editor (psql if I
> remember correctly)?
> * How much memory do you have on that box?
>
1. The error message: DBD::Pg::st execute failed: out of memory for
query result
2. Everything works fine, when I execute the command in psql
3. 4GB RAM on Linux box
Andrej
|
|
|
|
|