Is glob a lazy iterator in a scalar context?
I really like the glob
often overlooked functionality of the iterator.
What I'm not clear about is whether it computes / loads the entire list into memory even when used in a scalar context. In this code example, the loop while
does not print anything to the screen (4 ** 24 is not a small number):
use strict;
use warnings;
use feature 'say';
my $opt = "{A,B,C,D}" x 24;
say while glob $opt;
Things I've tried / observed:
-
I tried to rule out the possibility of buffering being the problem by writing
1 while glob $opt;
, but it still gets executed forever and I'm done Ctrl+C. -
Checking my memory usage with Windows Task Manager doesn't seem to make any difference when running the script.
-
Execution
perl -MO=Deparse
only confirms that itglob
is being used in a scalar context, without indicating anything about memory usage. -
Running it
perl -MO=Concise
leads to the following output, which I don't know how to decode:m <@> leave[1 ref] vKP/REFC ->(end) 1 <0> enter ->2 2 <;> nextstate(main 49 -:5) v:%,*,&,{,x*,x&,x$,$,469762048 ->3 7 <2> sassign vKS/2 ->8 5 <2> repeat[t2] sK/2 ->6 3 <$> const[PV "{A,B,C,D}"] s ->4 4 <$> const[IV 24] s ->5 6 <0> padsv[$opt:49,74] sRM*/LVINTRO ->7 8 <;> nextstate(main 74 -:6) v:%,*,&,{,x*,x&,x$,$,469762048 ->9 l <@> leave vK* ->m 9 <0> enter v ->a - <1> null vKP/1 ->l g <|> and(other->h) vK/1 ->l f <1> defined sK/1 ->g e <2> sassign sK/2 ->f c <@> glob[t5] sK/1 ->d - <0> ex-pushmark s ->a a <0> padsv[$opt:49,74] s ->b b <#> gv[*_GEN_0] s ->c - <1> ex-rv2sv sKRM*/3 ->e d <#> gvsv[*_] s ->e - <@> lineseq vK ->- j <@> say vK ->k h <0> pushmark s ->i - <1> ex-rv2sv sK/3 ->j i <#> gvsv[*_] s ->j k <0> unstack v ->a - syntax OK
I am running ActivePerl 5.16.3.
source to share
When the glob pattern contains curly braces, a recursive function is called to expand the curly braces. See the ext/File-Glob/bsd_glob.c
Perl * source:
/*
* Expand recursively a glob {} pattern. When there is no more expansion
* invoke the standard globbing routine to glob the rest of the magic
* characters
*/
static int
globexp1(const Char *pattern, glob_t *pglob)
{
const Char* ptr = pattern;
int rv;
/* Protect a single {}, for find(1), like csh */
if (pattern[0] == BG_LBRACE && pattern[1] == BG_RBRACE && pattern[2] == BG_EOS)
return glob0(pattern, pglob);
while ((ptr = (const Char *) g_strchr((Char *) ptr, BG_LBRACE)) != NULL)
if (!globexp2(ptr, pattern, pglob, &rv))
return rv;
return glob0(pattern, pglob);
}
My C is rusty, but when I run your script in gdb and set a breakpoint on that function, it hits many times, many times, suggesting very deep recursion and, by and large, illogical behavior.Hopefully someone who is more familiar with Perl internals, can develop.
* The above is from version 5.20.2.
source to share
It doesn't seem lazy when you observe the delay between the first call to glob and the first print of the result. This is confirmed by the time it takes to execute the globe in a scalar context and not in a loop (in my example, I used a lower exponent so that the delay is still noticeable, but not too long):
use strict;
use warnings;
use feature 'say';
use Time::HiRes;
$| = 1;
my $opt = "{A,B,C,D}" x 10;
say Time::HiRes::time;
my $x = glob $opt;
say $x;
say Time::HiRes::time;
say while glob $opt;
say Time::HiRes::time;
source to share