PERL Beginners - unfamiliar array reference

Author

unfamiliar array reference

petelink1@yahoo.com

2007-08-18, 10:02 pm

I'm reading the book Programming the Perl DBI to start work with
databases. The DBI portion I'm getting OK, but I find the reference
here to @$ confusing. (This code is based from the book.)

my $sth1 = $dbh->prepare("select 'SCM_ORDERS_OUT', message_index,
message_no, message_event, time_stamp, data from log");

$sth1->execute();

my @stash;

while ($array_ref = $sth1->fetchrow_arrayref) {
push @stash, [ @$array_ref ]; ##copy array contents
}

###dump stash contents
foreach $array_ref ( @stash ) {
print "Row: @$array_ref\n";
}

The rows print out fine, so it "works", but I don't understand how. I
need to be able to work with specific items in the output, such as the
column "data". How would I reference it, using this example?

Peter Link

Chas Owens

2007-08-18, 10:02 pm

On 8/17/07, petelink1@yahoo.com <petelink1@yahoo.com> wrote:
> I'm reading the book Programming the Perl DBI to start work with
> databases. The DBI portion I'm getting OK, but I find the reference
> here to @$ confusing. (This code is based from the book.)
snip

It is funny that you use the word reference, because $array_ref is a
reference to an array (hence its name). There are several ways to
access the array referenced by $array_ref. If you want a specific
element you can say $array_ref->[0] if you want access to the whole
array you can say @$array_ref. You can read more about reverences in
perldoc perlreftut (also available as
http://perldoc.perl.org/perlreftut.html).

Gunnar Hjalmarsson

2007-08-18, 10:02 pm

petelink1@yahoo.com wrote:
> I'm reading the book Programming the Perl DBI to start work with
> databases. The DBI portion I'm getting OK, but I find the reference
> here to @$ confusing. (This code is based from the book.)
>
> my $sth1 = $dbh->prepare("select 'SCM_ORDERS_OUT', message_index,
> message_no, message_event, time_stamp, data from log");
>
> $sth1->execute();
>
> my @stash;
>
> while ($array_ref = $sth1->fetchrow_arrayref) {
> push @stash, [ @$array_ref ]; ##copy array contents
> }
>
> ###dump stash contents
> foreach $array_ref ( @stash ) {
> print "Row: @$array_ref\n";
> }
>
> The rows print out fine, so it "works", but I don't understand how.

@stash is an "Array of Arrays", so to start with I think you should
study "perldoc perllol".

> I need to be able to work with specific items in the output, such as
> the column "data". How would I reference it, using this example?

foreach my $row ( @stash) {
print "Data: $row->[5]\n";
}

--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl

Mr. Shawn H. Corey

2007-08-18, 10:02 pm

petelink1@yahoo.com wrote:
> I'm reading the book Programming the Perl DBI to start work with
> databases. The DBI portion I'm getting OK, but I find the reference
> here to @$ confusing. (This code is based from the book.)
>
> my $sth1 = $dbh->prepare("select 'SCM_ORDERS_OUT', message_index,
> message_no, message_event, time_stamp, data from log");
>
> $sth1->execute();
>
> my @stash;
>
> while ($array_ref = $sth1->fetchrow_arrayref) {
> push @stash, [ @$array_ref ]; ##copy array contents
> }
>
> ###dump stash contents
> foreach $array_ref ( @stash ) {
> print "Row: @$array_ref\n";
> }
>
> The rows print out fine, so it "works", but I don't understand how. I
> need to be able to work with specific items in the output, such as the
> column "data". How would I reference it, using this example?

The term '@$' is a de-reference of a reference.

To be exact, $array_ref holds a reference to an array. To access the array you need to de-reference it. There are two ways to doing it. The following are equivalent.

@$array_ref
@{ $array_ref }

You can use these anywhere you use an array:

push @$array_ref, $some_data;
if( grep { /$pattern/ } @{ $array_ref } ){
...
}

--
Just my 0.00000002 million dollars worth,
Shawn

"For the things we have to learn before we can do them, we learn by doing them."
Aristotle

Paul Lalli

2007-08-18, 10:02 pm

On Aug 17, 5:07 pm, peteli...@yahoo.com wrote:
> I'm reading the book Programming the Perl DBI to start work with
> databases. The DBI portion I'm getting OK, but I find the reference
> here to @$ confusing. (This code is based from the book.)
>
> my $sth1 = $dbh->prepare("select 'SCM_ORDERS_OUT', message_index,
> message_no, message_event, time_stamp, data from log");
>
> $sth1->execute();
>
> my @stash;
>
> while ($array_ref = $sth1->fetchrow_arrayref) {
> push @stash, [ @$array_ref ]; ##copy array contents

Is that seriously from the book? UGH. Just declare $array_ref in the
proper scope, and there's no need to take a reference to a dereference
of the reference.

while (my $array_ref = $sth1->fetchrow_arrayref) {
push @stash, $array_ref;
}

>
> }
>
> ###dump stash contents
> foreach $array_ref ( @stash ) {
> print "Row: @$array_ref\n";
>
> }
>
> The rows print out fine, so it "works", but I don't understand how. I
> need to be able to work with specific items in the output, such as the
> column "data". How would I reference it, using this example?

In this specific example, "data" is the sixth column in the SELECT
query. You would therefore access it by $array_ref->[5].

I would generally recommend you use the fetchrrow_hashref method
rather than fetchrow_arrayref, so that you can refer to the column by
name. This will prevent bugs in your code if you later change the
query to add or remove columns. With fetchrow_hashref, the return
value is a reference to a hash rather than to an array:

while (my $hash_ref = $sth1->fetchrow_hashref()) {
print $hash_ref->{data} . "\n";
}

I agree with other posters - you should read:
perldoc perlref
perldoc perlreftut
perldoc perllol
perldoc perldsc

Paul Lalli

Randal L. Schwartz

2007-08-18, 10:02 pm

>>>>> "Paul" == Paul Lalli <mritty@gmail.com> writes:
[color=darkred]

Paul> Is that seriously from the book? UGH. Just declare $array_ref in the
Paul> proper scope, and there's no need to take a reference to a dereference
Paul> of the reference.

Yes there is.

Paul> while (my $array_ref = $sth1->fetchrow_arrayref) {
Paul> push @stash, $array_ref;
Paul> }

Your code breaks on older versions of DBI, for which the book was written.
In older versions of DBI, the value of $array_ref would have been
the same for each row returned. There was this clause in "perldoc DBI":

Note that the same array reference is returned for each fetch, so don't
store the reference and then use it after a later fetch. Also, the elements
of the array are also reused for each row, so take care if you want to take
a reference to an element.

Yes, DBI has matured since the book, but I wouldn't jump to conclusions about
the validity of the code in the book. After all, Tim Bunce co-wrote the book,
and there's nobody on the planet who knows more about DBI.

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

Paul Lalli

2007-08-19, 7:58 am

On Aug 18, 10:03 pm, mer...@stonehenge.com (Randal L. Schwartz) wrote:
>
> Paul> Is that seriously from the book? UGH. Just declare $array_ref in the
> Paul> proper scope, and there's no need to take a reference to a dereference
> Paul> of the reference.
>
> Yes there is.
>
> Paul> while (my $array_ref = $sth1->fetchrow_arrayref) {
> Paul> push @stash, $array_ref;
> Paul> }
>
> Your code breaks on older versions of DBI, for which the book was written.
> In older versions of DBI, the value of $array_ref would have been
> the same for each row returned. There was this clause in "perldoc DBI":
>
> Note that the same array reference is returned for each fetch, so don't
> store the reference and then use it after a later fetch. Also, the elements
> of the array are also reused for each row, so take care if you want to take
> a reference to an element.

Yeesh. I guess I'm fortunate to have only dealt with newer versions
of DBI.pm. :-)

Thanks for the history lesson.

Paul Lalli

petelink1@yahoo.com

2007-08-24, 4:01 am

On Aug 18, 7:58 pm, mri...@gmail.com (Paul Lalli) wrote:

snip

>
> Is that seriously from the book? UGH. Just declare $array_ref in the
> proper scope, and there's no need to take a reference to a dereference
> of the reference.
>
> while (my $array_ref = $sth1->fetchrow_arrayref) {
> push @stash, $array_ref;
>
> }

Great - this worked fine.

>
> In this specific example, "data" is the sixth column in the SELECT
> query. You would therefore access it by $array_ref->[5].
>

No luck here, I could not retrieve anything (example code below).

> I would generally recommend you use the fetchrrow_hashref method
> rather than fetchrow_arrayref, so that you can refer to the column by
> name. This will prevent bugs in your code if you later change the
> query to add or remove columns. With fetchrow_hashref, the return
> value is a reference to a hash rather than to an array:
>
> while (my $hash_ref = $sth1->fetchrow_hashref()) {
> print $hash_ref->{data} . "\n";
>
> }

I was able to replicate this and also do a push to an array, but
again, I could not retrieve rows.

> I agree with other posters - you should read:
> perldoc perlref
> perldoc perlreftut
> perldoc perllol
> perldoc perldsc
>
> Paul Lalli

Thanks to all who suggested these perldocs - the perlreftut and
perllol were especially helpful.

I've learned more about references and also about scoping of variables
- now I'm doing "use strict" and it took a while to make it work
here.
A side question: why is it necessary to declare "our @variable" more
than once in a program?

The main problem is that I still can't retrieve data at will.
See below:

First, as an array - I tried commented-out options one at a time.
Msg is error message from Perl.

$sth1->execute();

while (my $array_ref = $sth1->fetchrow_arrayref) {
push ((our @stash), $array_ref); ##copy array contents
}

#use Data::Dumper 'Dumper';
# print Dumper our @stash ; #Msg: $VAR19586 = $VAR1;

###dump stash contents
foreach my $array_ref (our @stash ) {
# print "Row: $array_ref->[5]\n"; #Msg:Use of uninitialized value in
concatenation
# print $array_ref->[5]; #Msg: Use of uninitialized value in
concatenation
print "Row: @$array_ref\n"; #works, prints entire rows (from book)
}

foreach my $row (our @stash) {
# print "DATA: $row->[5]\n"; #Msg: Use of uninitialized value in
concatenation or string
# print "DATA: @$row->[5]\n"; #Msg: Use of uninitialized value in join
or string
#}

Now as a hash, which is what I would prefer, for the reasons stated by
Paul.

$sth1->execute();

while (my $hash_ref = $sth1->fetchrow_hashref()) {
#print $hash_ref->{data} . "\n"; # works, prints data
push our @stash, [ %$hash_ref ]; #works, copies hash contents
}

#use Data::Dumper 'Dumper';
# print Dumper our @stash; #showed good results (see below)

foreach my $hash_ref (our @stash ) {
#print "Row: %$hash_ref\n"; #Msg: Row: %ARRAY(0x13f88d44)
#print "Row: $hash_ref\n"; #Msg: Row: %ARRAY(0x13f88d44)
#print %$hash_ref . "\n"; #Msg: 'Can't coerce array into hash at'
}

from hash Data Dumper:
$VAR790 = [
'time_stamp',
'Mon Mar 05 2007 11:07:11',
'message_event',
'MESSAGE OUT',
'message_no',
'12101589',
'data',
' MSH|^~\\&|HUB|HOSP.MED.XXXX.EDU|PM3.0|HOSP.MED.XXXX.EDU|200
',A|AA|9}|MESSAGE ACCEPTED||
'message_index',
'36314470'
];
$VAR791 = [
'time_stamp',
'Mon Mar 05 2007 11:07:11',
etc.....

What am I doing wrong?
Thanks for all the help.

Peter Link

Paul Lalli

2007-08-24, 7:01 pm

On Aug 23, 7:13 pm, peteli...@yahoo.com wrote:
> On Aug 18, 7:58 pm, mri...@gmail.com (Paul Lalli) wrote:
> I've learned more about references and also about scoping of variables
> - now I'm doing "use strict" and it took a while to make it work here.
> A side question: why is it necessary to declare "our @variable" more
> than once in a program?

'our', like 'my', is lexically scoped. Its effects are terminated
when the innermost enclosing block ends. So if you're using the same
package variable in two different blocks, you have to use 'our' in
each of them:

package Foo;
{
our $bar;
#for the duration of this scope, you can use $Foo::bar by just
saying $bar
}
#now scope of 'our' has ended, must fully qualify $Foo::bar again
{
#new scope, must still say $Foo::bar
our $bar;
#Can now call $Foo:: bar as just $bar again
}

> The main problem is that I still can't retrieve data at will.
> See below:
>
> First, as an array - I tried commented-out options one at a time.
> Msg is error message from Perl.
>
> $sth1->execute();
>
> while (my $array_ref = $sth1->fetchrow_arrayref) {
> push ((our @stash), $array_ref); ##copy array contents
> }
>
> #use Data::Dumper 'Dumper';
> # print Dumper our @stash ; #Msg: $VAR19586 = $VAR1;

Your syntax for 'our' is confusing at best. Just declare it once, at
the very top of your program. From the output however, it looks like
you have one of the older versions of DBI that Randal mentioned in his
post. That is, you cannot do what I suggested. You have to make an
explicit copy.

my @stash; #there's no reason this has to be 'our' to begin with.
while (my $array_ref = $sth1->fetchrow_arrayref) {
push @stash, [ @{$array_ref} ];
}
print Dumper(\@stash);

> ###dump stash contents
> foreach my $array_ref (our @stash ) {

Again. Stop using 'our' all over the place. Declare your variables
once, at the beginning of your program, if it's being used throughout
the program.

> # print "Row: $array_ref->[5]\n"; #Msg:Use of uninitialized value in
> concatenation
> # print $array_ref->[5]; #Msg: Use of uninitialized value in
> concatenation
> print "Row: @$array_ref\n"; #works, prints entire rows (from book)

And what is the output of that? If $array_ref->[5] is undefined, then
you do not have six elements in your result arrays, which means you're
using a different query than the one you originally posted, and you'll
have to adjust your code accordingly.

> Now as a hash, which is what I would prefer, for the reasons stated by
> Paul.
>
> $sth1->execute();
>
> while (my $hash_ref = $sth1->fetchrow_hashref()) {
> #print $hash_ref->{data} . "\n"; # works, prints data
> push our @stash, [ %$hash_ref ]; #works, copies hash contents

I wouldn't say "works" here. I would say nothing more than "doesn't
generate compiler errors". This doesn't do what you think it does.
This creates a reference to an *array* that contains the keys and
values of your hash. You don't want to do that. You want to create a
reference to a *hash*. To do that, we use { } instead of [ ]

push @stash, { %{$hash_ref} };

>
> }
>
> #use Data::Dumper 'Dumper';
> # print Dumper our @stash; #showed good results (see below)
>
> foreach my $hash_ref (our @stash ) {
> #print "Row: %$hash_ref\n"; #Msg: Row: %ARRAY(0x13f88d44)
> #print "Row: $hash_ref\n"; #Msg: Row: %ARRAY(0x13f88d44)
> #print %$hash_ref . "\n"; #Msg: 'Can't coerce array into hash at'

Right. All these messages are because you put array refs into @stash
rather than hash refs.

Paul Lalli

Chas Owens

2007-08-24, 7:01 pm

On 8/24/07, Paul Lalli <mritty@gmail.com> wrote:
snip
> 'our', like 'my', is lexically scoped. Its effects are terminated
> when the innermost enclosing block ends. So if you're using the same
> package variable in two different blocks, you have to use 'our' in
> each of them:
snip
> Your syntax for 'our' is confusing at best. Just declare it once, at
> the very top of your program.
snip
> Again. Stop using 'our' all over the place. Declare your variables
> once, at the beginning of your program, if it's being used throughout
> the program.
snip

I can't say that I agree with the advice to declare all of your our
variables once at the top of your program. To my way of thinking this
defeats the purpose of having the our function*. I have a
once-per-function rule for our; if a function needs to use a global
variable, then it has to request it. This means that when you come
back to the code a year later, and are looking at just that one
function, you can easily see that the variable is special (i.e. used
by many functions). That said, the code you were referring to was
definitely overusing our. It doesn't need to be attached to every
reference to the variable.

* yes, the variable is still visible to all packages in the same file
without having to be fully qualified, but I consider that aspect of
our a misfeature.

Mr. Shawn H. Corey

2007-08-24, 7:01 pm

Paul Lalli wrote:
> 'our', like 'my', is lexically scoped. Its effects are terminated
> when the innermost enclosing block ends. So if you're using the same
> package variable in two different blocks, you have to use 'our' in
> each of them:

'our' is not lexically scoped; it is package scoped. Once you declare a variable using 'our' you may use it anywhere in the package. Think of it as an alias to the fully-qualified variable. A variable declare via 'our' in one package is independent of
a variable of the same name in another package.

See `perldoc -f our` for more details.

#!/usr/bin/perl

use strict;
use warnings;

package Foo;
our $bar = 'Foo's bar'; # $bar is an alias to $Foo::bar
print "$bar\n";

sub bar {
print "Foo::bar sub: $bar\n";
}

package main;
our $bar = 'main's bar'; # $bar is an alias to $main::bar or $::bar;
print "$bar\n";

print "$Foo::bar\n";
print "$main::bar\n";
print "$::bar\n";

Foo::bar();

__END__

--
Just my 0.00000002 million dollars worth,
Shawn

"For the things we have to learn before we can do them, we learn by doing them."
Aristotle

"If you think Terrans are comprehensible, you don't understand them."
Great Fang Talphon

Chas Owens

2007-08-24, 7:01 pm

On 8/24/07, Mr. Shawn H. Corey <shawnhcorey@magma.ca> wrote:
> Paul Lalli wrote:
>
> 'our' is not lexically scoped; it is package scoped. Once you declare a
> variable using 'our' you may use it anywhere in the package. Think
> of it as an alias to the fully-qualified variable. A variable declare via
> 'our' in one package is independent of a variable of the same name
> in another package.

Nope, it is lexically scoped.

from perldoc -f our
our associates a simple name with a package variable in the current
package for use within the current scope. When use strict 'vars' is
in effect, our lets you use declared global variables without
qualifying them with package names, within the lexical scope of the
our declaration. In this way our differs from use vars , which is
package scoped.

Paul Lalli

2007-08-24, 7:01 pm

On Aug 24, 9:37 am, chas.ow...@gmail.com (Chas Owens) wrote:
> On 8/24/07, Paul Lalli <mri...@gmail.com> wrote:
> snip> 'our', like 'my', is lexically scoped. Its effects are terminated
> snip
> snip
>
> snip
>
> I can't say that I agree with the advice to declare all of your our
> variables once at the top of your program. To my way of thinking this
> defeats the purpose of having the our function*. I have a
> once-per-function rule for our; if a function needs to use a global
> variable, then it has to request it.

I tend to agree. However, this poster was not using functions. He
was simply declaring 'our' every time he saw the variable. That's why
I told him to move the declaration of 'our' to the top.

The reason he "had to" keep declaring 'our' is that he was only
declaring it in the insanely small scope of the actual expression.
There is no need for that. Just declare it before the big block of
code that uses it.

Paul Lalli

Paul Lalli

2007-08-24, 7:01 pm

On Aug 24, 9:55 am, shawnhco...@magma.ca (Mr. Shawn H. Corey) wrote:
> Paul Lalli wrote:
>
> 'our' is not lexically scoped; it is package scoped.

No. Package variables are package scoped. 'our' is lexically
scoped. The effects of 'our' are gone as soon as the lexical scope in
which it was used are gone. The package variable, of course, remains.

> Once you declare a variable using 'our' you may use it anywhere in the
> package.

Incorrect. Once you declare a variable using 'our', you may use its
shorthand name anywhere in the lexical scope of that declaration. You
can ALWAYS use the fully-qualified name anywhere, regardless of
strict, regardless of 'our'.

$ perl -le'
use strict;
$main::foo = "hello world";
print $main::foo;
{
our $foo;
print $foo;
}
print $foo;
'
Variable "$foo" is not imported at -e line 9.
Global symbol "$foo" requires explicit package name at -e line 9.
Execution of -e aborted due to compilation errors.

> See `perldoc -f our` for more details.

You may wish to read that yourself...
perldoc -f our
An "our" declaration declares a global variable that
will be visible across its entire lexical scope,
even across package boundaries.

>
> #!/usr/bin/perl
>
> use strict;
> use warnings;
>
> package Foo;
> our $bar = 'Foo's bar'; # $bar is an alias to $Foo::bar

Yes, but only for the LEXICAL SCOPE, which in this case is the entire
file, since there was no enclosing block.

Paul Lalli

petelink1@yahoo.com

2007-08-25, 7:59 am

On Aug 24, 9:03 am, mri...@gmail.com (Paul Lalli) wrote:

snip

>
> 'our', like 'my', is lexically scoped. Its effects are terminated
> when the innermost enclosing block ends. So if you're using the same
> package variable in two different blocks, you have to use 'our' in
> each of them:
>

Yes, that was a big revelation. I wasn't aware that declaring our in
a block made a difference. I thought that our meant it could be
declared once and used anywhere no matter where it was declared.

snip

> Your syntax for 'our' is confusing at best. Just declare it once, at
> the very top of your program.

Yes, I started declaring our at the beginning. Much clearer.

> From the output however, it looks like
> you have one of the older versions of DBI that Randal mentioned in his
> post. That is, you cannot do what I suggested. You have to make an
> explicit copy.
>
> my @stash; #there's no reason this has to be 'our' to begin with.
> while (my $array_ref = $sth1->fetchrow_arrayref) {
> push @stash, [ @{$array_ref} ];}
>

No, the version of DBI that I am using is very current so apparently
the DBI book is current on that issue also. Using your code above
(same as book) worked fine.

> Again. Stop using 'our' all over the place. Declare your variables
> once, at the beginning of your program, if it's being used throughout
> the program.

Yep, got it the first time.

>
> And what is the output of that? If $array_ref->[5] is undefined, then
> you do not have six elements in your result arrays, which means you're
> using a different query than the one you originally posted, and you'll
> have to adjust your code accordingly.
>

Yes, when I was experimenting I unknowingly deleted the first string
in my SQL, which made the reference to [5] nonsensical. Sorry.

> I wouldn't say "works" here. I would say nothing more than "doesn't
> generate compiler errors". This doesn't do what you think it does.
> This creates a reference to an *array* that contains the keys and
> values of your hash. You don't want to do that. You want to create a
> reference to a *hash*. To do that, we use { } instead of [ ]
>
> push @stash, { %{$hash_ref} };
>

Changed my code to this:
while (my $hash_ref = $sth1->fetchrow_hashref()) {
push @stash, { %$hash_ref }; #works, copies hash contents
}

> Right. All these messages are because you put array refs into @stash
> rather than hash refs.

Thanks again for all your help. I still need to digest all of the
posts regarding scope, but I understand it better than two days ago.

--pete link

Chas Owens

2007-08-25, 7:59 am

On 8/24/07, petelink1@yahoo.com <petelink1@yahoo.com> wrote:
snip
> No, the version of DBI that I am using is very current so apparently
> the DBI book is current on that issue also. Using your code above
> (same as book) worked fine.
snip

Ah, I knew there was a reason beyond readability that I preferred
fetchrow_hashref:

from perldoc DBI
"fetchrow_arrayref"
$ary_ref = $sth->fetchrow_arrayref;
$ary_ref = $sth->fetch; # alias
snip
Note that the same array reference is returned for each fetch, so
don't store the reference and then use it after a later fetch.
Also, the elements of the array are also reused for each row, so
take care if you want to take a reference to an element. See also
"bind_columns".
snip
"fetchrow_hashref"
$hash_ref = $sth->fetchrow_hashref;
$hash_ref = $sth->fetchrow_hashref($name);
snip
By default a reference to a new hash is returned for each row. It
is likely that a future version of the DBI will support an
attribute which will enable the same hash to be reused for each
row. This will give a significant performance boost, but it won't
be enabled by default because of the risk of breaking old code.

Paul Lalli

2007-08-25, 7:59 am

On Aug 25, 6:09 am, chas.ow...@gmail.com (Chas Owens) wrote:

> Ah, I knew there was a reason beyond readability that I preferred
> fetchrow_hashref:
>
> from perldoc DBI
> "fetchrow_arrayref"
> Note that the same array reference is returned for each fetch,

> "fetchrow_hashref"
> By default a reference to a new hash is returned for each row. It

Aha!! So maybe that's why I never knew about this feature. I only
ever use fetchrow_hashref, so I never got bit by the reusing of array
references.

Thanks very much for pointing that out, Chas.

Paul Lalli

Sponsored Links