little quiz

Look at this code which prints the first ten positive integers:

--->
#!/usr/bin/perl

use 5.010;

sub f1
{
state $v;
++$v;
say $v;
}

f1 for (1..10);
<---

Can you rewrite f1 without using 'state'?
You may only modify f1 and you may not use external code etc...:

--->
#!/usr/bin/perl

use 5.010;

sub f1
{
# write only here...
}

f1 for (1..10);
<---

Kiuhnm

Quoth Kiuhnm <kiuhnm03.4t.yahoo.it>:
> Look at this code which prints the first ten positive integers:
>
> --->
> #!/usr/bin/perl
>
> use 5.010;
>
> sub f1
> {
> state $v;
> ++$v;
> say $v;
> }
>
> f1 for (1..10);
> <---
>
> Can you rewrite f1 without using 'state'?
> You may only modify f1 and you may not use external code etc...:

TIMTOWTDI:

sub f1 {
++$v;
say $v;
}

(did you mean to forbid this?)

sub f1 {
my $v if 0;
++$v;
say $v;
}

(the warning only became mandatory under 5.12, you only required 5.10;
this was the semi-official way of doing 'state' before 5.10)

sub f1 {
my $v;
*f1 = sub {
++$v;
say $v;
};
f1();
}

(though this is just the same as the first example, really: I'm still
storing the state in the stash, just a bit more obscurely)

sub f1 {
BEGIN {
my $v;
sub f2 {
++$v;
say $v;
}
}
f2();
}

(this is nearly clean, though the BEGIN block isn't really part of f1)

sub f1 {
my $v;
sub f2 {
++$v;
say $v;
}
f2();
}

(and in fact you can skip the BEGIN, since you didn't use warnings)

sub f1 {
open X, "<", \"\n";
<X>;
say $.;
}

sub f1 {
"1 2 3 4 5 6 7 8 9 10 " =~ /(\d+) /g;
say $1;
}

Ben

On 3/4/2012 16:15, Ben Morrow wrote:
> Quoth Kiuhnm<kiuhnm03.4t.yahoo.it>:
>> Look at this code which prints the first ten positive integers:
>>
>> --->
>> #!/usr/bin/perl
>>
>> use 5.010;
>>
>> sub f1
>> {
>> state $v;
>> ++$v;
>> say $v;
>> }
>>
>> f1 for (1..10);
>> <---
>>
>> Can you rewrite f1 without using 'state'?
>> You may only modify f1 and you may not use external code etc...:
>
> TIMTOWTDI:
>
> sub f1 {
> ++$v;
> say $v;
> }
>
> (did you mean to forbid this?)

Yep.

> sub f1 {
> my $v if 0;
> ++$v;
> say $v;
> }
>
> (the warning only became mandatory under 5.12, you only required 5.10;
> this was the semi-official way of doing 'state' before 5.10)

That's new to me.
Same thing without the warning:

sub f1 {
my $v if my $y;
++$v;
say $v;
}

I didn't even know that
my $v = 7;
my $v = 0;
was possible.
I didn't know that "my" was an operator.

> sub f1 {
> my $v;
> *f1 = sub {
> ++$v;
> say $v;
> };
> f1();
> }
>
> (though this is just the same as the first example, really: I'm still
> storing the state in the stash, just a bit more obscurely)
>
> sub f1 {
> BEGIN {
> my $v;
> sub f2 {
> ++$v;
> say $v;
> }
> }
> f2();
> }
>
> (this is nearly clean, though the BEGIN block isn't really part of f1)
>
> sub f1 {
> my $v;
> sub f2 {
> ++$v;
> say $v;
> }
> f2();
> }

That's my solution! I was surprised when I saw that named sub
definitions are evaluated just once.
BTW, I don't like that f2 is visible from outside of f1.

> (and in fact you can skip the BEGIN, since you didn't use warnings)
>
> sub f1 {
> open X, "<", \"\n";
> <X>;
> say $.;
> }

Clever, but then why not

sub f1 {
say 1+$.++;
}

> sub f1 {
> "1 2 3 4 5 6 7 8 9 10 " =~ /(\d+) /g;
> say $1;
> }

I didn't know I could use =~ with the /g option in scalar context:
while ("1 2 3 4 5 6 7 8 9 10 " =~ /(\d+) /g) {
say $1;
}
Is it an idiom or a little trick?

Kiuhnm

Kiuhnm <kiuhnm03.4t.yahoo.it> writes:
> Look at this code which prints the first ten positive integers:
>
> --->
> #!/usr/bin/perl
>
> use 5.010;
>
> sub f1
> {
> state $v;
> ++$v;
> say $v;
> }
>
> f1 for (1..10);
> <---
>
> Can you rewrite f1 without using 'state'?
> You may only modify f1 and you may not use external code etc...:
>
> --->
> #!/usr/bin/perl
>
> use 5.010;
>
> sub f1
> {
> # write only here...
> }
>
> f1 for (1..10);
> <---

sub f1 {
print("$_\n");
}

On Sun, 04 Mar 2012 13:19:29 +0100, Kiuhnm wrote:

> Look at this code which prints the first ten positive integers:
>
> --->
> #!/usr/bin/perl
>
> use 5.010;
>
> sub f1
> {
> state $v;
> ++$v;
> say $v;
> }
>
> f1 for (1..10);
> <---
>
> Can you rewrite f1 without using 'state'? You may only modify f1 and you
> may not use external code etc...:
>

#!/usr/bin/perl

use 5.010;

sub f1
{
say $_;
}

f1 for (1..10);

M4

On 3/4/2012 20:23, Martijn Lievaart wrote:
> On Sun, 04 Mar 2012 13:19:29 +0100, Kiuhnm wrote:
>
>> Look at this code which prints the first ten positive integers:
>>
>> --->
>> #!/usr/bin/perl
>>
>> use 5.010;
>>
>> sub f1
>> {
>> state $v;
>> ++$v;
>> say $v;
>> }
>>
>> f1 for (1..10);
>> <---
>>
>> Can you rewrite f1 without using 'state'? You may only modify f1 and you
>> may not use external code etc...:
>>
>
> #!/usr/bin/perl
>
> use 5.010;
>
> sub f1
> {
> say $_;
> }
>
> f1 for (1..10);
>
> M4

I should have written "[...]external code, external variables, etc...",
where "external" means not declared inside f1.

Kiuhnm

On 03/04/2012 05:21 PM, Kiuhnm wrote:
> On 3/4/2012 20:23, Martijn Lievaart wrote:
>> On Sun, 04 Mar 2012 13:19:29 +0100, Kiuhnm wrote:
>>
>>> Look at this code which prints the first ten positive integers:
>>>
>>> --->
>>> #!/usr/bin/perl
>>>
>>> use 5.010;
>>>
>>> sub f1
>>> {
>>> state $v;
>>> ++$v;
>>> say $v;
>>> }
>>>
>>> f1 for (1..10);
>>> <---
>>>
>>> Can you rewrite f1 without using 'state'? You may only modify f1 and you
>>> may not use external code etc...:
>>>
>>
>> #!/usr/bin/perl
>>
>> use 5.010;
>>
>> sub f1
>> {
>> say $_;
>> }
>>
>> f1 for (1..10);
>>
>> M4
>
> I should have written "[...]external code, external variables, etc...",
> where "external" means not declared inside f1.

oh, declared?

sub f1
{
say our $_;
}

On 3/5/2012 0:00, sreservoir wrote:
> On 03/04/2012 05:21 PM, Kiuhnm wrote:
[...]
>> I should have written "[...]external code, external variables, etc...",
>> where "external" means not declared inside f1.
>
> oh, declared?
>
> sub f1
> {
> say our $_;
> }

Ops... I meant declared outside f1!

Kiuhnm

Quoth Kiuhnm <kiuhnm03.4t.yahoo.it>:
> On 3/4/2012 16:15, Ben Morrow wrote:
>
> > sub f1 {
> > my $v if 0;
> > ++$v;
> > say $v;
> > }
> >
> > (the warning only became mandatory under 5.12, you only required 5.10;
> > this was the semi-official way of doing 'state' before 5.10)
>
> That's new to me.

It was always accidental that this worked at all: it's to do with the
details of when the compile-time and run-time parts of 'my' happen, and
which conditionals get constant-folded at compile time. It is
sufficiently useful that it was promoted to a proper supported feature
with 'state'.

> Same thing without the warning:
>
> sub f1 {
> my $v if my $y;
> ++$v;
> say $v;
> }

I think the lack of warning is not intentional: that is, I think any
change to core perl which caused my example to stop working would do the
same to yours, it's just not so visible at compile time that that is the
case.

>
> I didn't even know that
> my $v = 7;
> my $v = 0;
> was possible.

Those are separate variables which happen to have the same names. It's
exactly the same situation as

my $v = 7;
{
my $v = 0;
}

except the scope of the second $v extends all the way to the end of the
scope of the first.

> I didn't know that "my" was an operator.

I'm not sure what you mean by that... it has a runtime effect,
certainly.

> > sub f1 {
> > my $v;
> > sub f2 {
> > ++$v;
> > say $v;
> > }
> > f2();
> > }
>
> That's my solution! I was surprised when I saw that named sub
> definitions are evaluated just once.
> BTW, I don't like that f2 is visible from outside of f1.

Those two things go together: the sub definition is only evaluated once
*because* you're defining a globally-visible named sub. What you were
expecting is 'my sub', which doesn't exist yet (and very possibly never
will: it seems it's harder to get right than it appears).

It's no worse, really, than sub definitions being visible outside the
file they were defined in. They are global objects living in the symbol
table, so their names aren't lexically scoped at all (modulo tricks like
namespace::clean...).

> > (and in fact you can skip the BEGIN, since you didn't use warnings)
> >
> > sub f1 {
> > open X, "<", \"\n";
> > <X>;
> > say $.;
> > }
>
> Clever, but then why not
>
> sub f1 {
> say 1+$.++;
> }

Well, yes, but that's just the same as using an ordinary global. I was
trying to find some other place to store global state.

My initial thought was that $. counted *output* lines (I don't have
occasion to use it much) which would make 'say $.' count all on its own.
Alas, no...

> > sub f1 {
> > "1 2 3 4 5 6 7 8 9 10 " =~ /(\d+) /g;
> > say $1;
> > }
>
> I didn't know I could use =~ with the /g option in scalar context:
> while ("1 2 3 4 5 6 7 8 9 10 " =~ /(\d+) /g) {
> say $1;
> }
> Is it an idiom or a little trick?

It's absolutely intentional that it works that way, if that's what you
mean. Of course, the intent was to use it in while(), the way you did,
rather than on successive calls to a sub.

Implicit iterators like that are an important part of Perl: m//g, s///g,
each(), scalar-<>, scalar-.., and ?? all have implicit state stashed
away somewhere. Keeping track of where that state is kept is often
rather important, and is one of the reasons those operators are not used
as much as they used to be.

Ben

On 3/5/2012 0:39, Ben Morrow wrote:
> Quoth Kiuhnm<kiuhnm03.4t.yahoo.it>:
[...]
>> Same thing without the warning:
>>
>> sub f1 {
>> my $v if my $y;
>> ++$v;
>> say $v;
>> }
>
> I think the lack of warning is not intentional: that is, I think any
> change to core perl which caused my example to stop working would do the
> same to yours, it's just not so visible at compile time that that is the
> case.

Much in Perl is not intentional
Do you remember my post about given...when?
Perl's syntax is so complex that suprises are bound to come up.
I've seen my share of hacks and tricks when doing metaprogramming in
C++. The Turing-completeness of templates was totally unexpected.

>> I didn't even know that
>> my $v = 7;
>> my $v = 0;
>> was possible.
>
> Those are separate variables which happen to have the same names. It's
> exactly the same situation as
>
> my $v = 7;
> {
> my $v = 0;
> }
>
> except the scope of the second $v extends all the way to the end of the
> scope of the first.
>
>> I didn't know that "my" was an operator.
>
> I'm not sure what you mean by that... it has a runtime effect,
> certainly.

In many languages, those are just directives to the compiler. Their
effect is entirely determined at compile-time.

>>> sub f1 {
>>> my $v;
>>> sub f2 {
>>> ++$v;
>>> say $v;
>>> }
>>> f2();
>>> }
>>
>> That's my solution! I was surprised when I saw that named sub
>> definitions are evaluated just once.
>> BTW, I don't like that f2 is visible from outside of f1.
>
> Those two things go together: the sub definition is only evaluated once
> *because* you're defining a globally-visible named sub. What you were
> expecting is 'my sub', which doesn't exist yet (and very possibly never
> will: it seems it's harder to get right than it appears).
>
> It's no worse, really, than sub definitions being visible outside the
> file they were defined in. They are global objects living in the symbol
> table, so their names aren't lexically scoped at all (modulo tricks like
> namespace::clean...).

Will Perl 6 correct these issues? I understand it will break code anyway.

>>> sub f1 {
>>> "1 2 3 4 5 6 7 8 9 10 " =~ /(\d+) /g;
>>> say $1;
>>> }
>>
>> I didn't know I could use =~ with the /g option in scalar context:
>> while ("1 2 3 4 5 6 7 8 9 10 " =~ /(\d+) /g) {
>> say $1;
>> }
>> Is it an idiom or a little trick?
>
> It's absolutely intentional that it works that way, if that's what you
> mean. Of course, the intent was to use it in while(), the way you did,
> rather than on successive calls to a sub.
>
> Implicit iterators like that are an important part of Perl: m//g, s///g,
> each(), scalar-<>, scalar-.., and ?? all have implicit state stashed
> away somewhere. Keeping track of where that state is kept is often
> rather important, and is one of the reasons those operators are not used
> as much as they used to be.

Good, I know two implicit iterators out of six...

Kiuhnm

Quoth Kiuhnm <kiuhnm03.4t.yahoo.it>:
> On 3/5/2012 0:39, Ben Morrow wrote:
> > Quoth Kiuhnm<kiuhnm03.4t.yahoo.it>:
> [...]
> >> Same thing without the warning:
> >>
> >> sub f1 {
> >> my $v if my $y;
> >> ++$v;
> >> say $v;
> >> }
> >
> > I think the lack of warning is not intentional: that is, I think any
> > change to core perl which caused my example to stop working would do the
> > same to yours, it's just not so visible at compile time that that is the
> > case.
>
> Much in Perl is not intentional
> Do you remember my post about given...when?
> Perl's syntax is so complex that suprises are bound to come up.

Yup. If you want Scheme, you know where to find it .

> >> I didn't even know that
> >> my $v = 7;
> >> my $v = 0;
> >> was possible.
<snip>
> >> I didn't know that "my" was an operator.
> >
> > I'm not sure what you mean by that... it has a runtime effect,
> > certainly.
>
> In many languages, those are just directives to the compiler. Their
> effect is entirely determined at compile-time.

perl (5) doesn't allocate a new lexical pad for each iteration of a
block, instead each 'my' registers a destructor on the scope which will
clear the variable out or replace it if it's still got refs. This is
done solely as an efficiency measure, though it does have a few
Perl-visible side effects (as we've discussed before).

Over and above that, Perl will allow you to declare a variable which
masks another of the same name in the same scope, which some languages
won't. (Though it will give you a warning.) I think this is mostly a
consequence of not keeping the compile-time lexical structure at
runtime, which means that the pad already has to be able to handle
having two variables called '$v' scoped over a slightly different set of
statements.

> > Those two things go together: the sub definition is only evaluated once
> > *because* you're defining a globally-visible named sub. What you were
> > expecting is 'my sub', which doesn't exist yet (and very possibly never
> > will: it seems it's harder to get right than it appears).
> >
> > It's no worse, really, than sub definitions being visible outside the
> > file they were defined in. They are global objects living in the symbol
> > table, so their names aren't lexically scoped at all (modulo tricks like
> > namespace::clean...).
>
> Will Perl 6 correct these issues? I understand it will break code anyway.

Having global subs is a feature, not a bug. Having *only* global subs is
a bug; AIUI Perl 6 has proper lexically-scoped named subs.

Ben

On 3/5/2012 17:50, Ben Morrow wrote:
> Quoth Kiuhnm<kiuhnm03.4t.yahoo.it>:
>> On 3/5/2012 0:39, Ben Morrow wrote:
>>> Quoth Kiuhnm<kiuhnm03.4t.yahoo.it>:
>> [...]
>>>> Same thing without the warning:
>>>>
>>>> sub f1 {
>>>> my $v if my $y;
>>>> ++$v;
>>>> say $v;
>>>> }
>>>
>>> I think the lack of warning is not intentional: that is, I think any
>>> change to core perl which caused my example to stop working would do the
>>> same to yours, it's just not so visible at compile time that that is the
>>> case.
>>
>> Much in Perl is not intentional
>> Do you remember my post about given...when?
>> Perl's syntax is so complex that suprises are bound to come up.
>
> Yup. If you want Scheme, you know where to find it .

One of the few languages with a 50-page spec!

>>>> I didn't even know that
>>>> my $v = 7;
>>>> my $v = 0;
>>>> was possible.
> <snip>
>>>> I didn't know that "my" was an operator.
>>>
>>> I'm not sure what you mean by that... it has a runtime effect,
>>> certainly.
>>
>> In many languages, those are just directives to the compiler. Their
>> effect is entirely determined at compile-time.
>
> perl (5) doesn't allocate a new lexical pad for each iteration of a
> block, instead each 'my' registers a destructor on the scope which will
> clear the variable out or replace it if it's still got refs. This is
> done solely as an efficiency measure, though it does have a few
> Perl-visible side effects (as we've discussed before).
>
> Over and above that, Perl will allow you to declare a variable which
> masks another of the same name in the same scope, which some languages
> won't. (Though it will give you a warning.) I think this is mostly a
> consequence of not keeping the compile-time lexical structure at
> runtime, which means that the pad already has to be able to handle
> having two variables called '$v' scoped over a slightly different set of
> statements.
>
>>> Those two things go together: the sub definition is only evaluated once
>>> *because* you're defining a globally-visible named sub. What you were
>>> expecting is 'my sub', which doesn't exist yet (and very possibly never
>>> will: it seems it's harder to get right than it appears).
>>>
>>> It's no worse, really, than sub definitions being visible outside the
>>> file they were defined in. They are global objects living in the symbol
>>> table, so their names aren't lexically scoped at all (modulo tricks like
>>> namespace::clean...).
>>
>> Will Perl 6 correct these issues? I understand it will break code anyway.
>
> Having global subs is a feature, not a bug. Having *only* global subs is
> a bug; AIUI Perl 6 has proper lexically-scoped named subs.

But named sub are always global. Why should I use a coderef just to make
a sub non-global?

Kiuhnm

Quoth Kiuhnm <kiuhnm03.4t.yahoo.it>:
> On 3/5/2012 17:50, Ben Morrow wrote:
> > Quoth Kiuhnm<kiuhnm03.4t.yahoo.it>:
> >>
> >> Will Perl 6 correct these issues? I understand it will break code anyway.
> >
> > Having global subs is a feature, not a bug. Having *only* global subs is
> > a bug; AIUI Perl 6 has proper lexically-scoped named subs.
>
> But named sub are always global. Why should I use a coderef just to make
> a sub non-global?

Because (and only because) noone's written the bits of code to make 'my
sub' work yet. There are maybe three or four people in the world who
understand all the subtleties of perl's lexical pads well enough to
implement something like this; IIRC the last time one of them (Dave
Mitchell) looked at the problem he said it was more complicated than it
appeared.

*However*, there are some neat tricks you can do with the symbol table,
if you're willing to get your hands a bit dirty:

package Foo;

no strict; # obviously this would be scoped in practice
use B::Hooks::EndOfScope;

sub import {
my $pkg = caller;

*{"$pkg\::foo"} = sub { "foo" };

on_scope_end {
my $old = *{"$pkg\::foo"};
my $stash = \%{"$pkg::"};

delete $stash->{foo};
# this will autoviv a new glob
my $new = *{"$pkg\::foo"};

*$old{SCALAR} and *$new = *$old{SCALAR};
*$old{ARRAY} and *$new = *$old{ARRAY};
*$old{HASH} and *$new = *$old{HASH};
*$old{IO} and *$new = *$old{IO};
# skip FORMAT since it's buggy in some perls
};
}

Now, when you say

{
use Foo;
say foo;
}
say foo;

the first call to foo calls the installed sub, but the second gives you
an undefined sub error. Also, if you say

use Foo;
say foo;
say main->foo;

the sub call calls the installed sub, but the method call gives an
undefined method error.

This was (as usual) quite accidental, but is extremely useful. It
happens because perl resolves sub calls as far as the relevant glob at
compile time, so if you replace that glob in the symbol table you can
put a different sub (or no sub at all) in its place without affecting
already-compiled code.

(The reason perl only resolves as far as the glob, and not right down to
the CV, is to allow subs to be redefined. So a simple

undef &{"$pkg\::foo"};
or
*{"$pkg\::foo"} = sub { "bar" };

*would* affect the already-compiled code, since it leaves the same glob
in the stash and only affects its CV entry.)

This idea is what is behind namespace::clean and the other namespace::*
modules, which hide the nasty internals behind a cleaner interface.

Ben

On 3/5/2012 23:40, Ben Morrow wrote:
> Quoth Kiuhnm<kiuhnm03.4t.yahoo.it>:
>> On 3/5/2012 17:50, Ben Morrow wrote:
>>> Quoth Kiuhnm<kiuhnm03.4t.yahoo.it>:
>>>>
>>>> Will Perl 6 correct these issues? I understand it will break code anyway.
>>>
>>> Having global subs is a feature, not a bug. Having *only* global subs is
>>> a bug; AIUI Perl 6 has proper lexically-scoped named subs.
>>
>> But named sub are always global. Why should I use a coderef just to make
>> a sub non-global?
>
> Because (and only because) noone's written the bits of code to make 'my
> sub' work yet. There are maybe three or four people in the world who
> understand all the subtleties of perl's lexical pads well enough to
> implement something like this; IIRC the last time one of them (Dave
> Mitchell) looked at the problem he said it was more complicated than it
> appeared.
>
> *However*, there are some neat tricks you can do with the symbol table,
> if you're willing to get your hands a bit dirty:
>
> package Foo;
>
> no strict; # obviously this would be scoped in practice
> use B::Hooks::EndOfScope;
>
> sub import {
> my $pkg = caller;
>
> *{"$pkg\::foo"} = sub { "foo" };
>
> on_scope_end {
> my $old = *{"$pkg\::foo"};
> my $stash = \%{"$pkg::"};
>
> delete $stash->{foo};
> # this will autoviv a new glob
> my $new = *{"$pkg\::foo"};
>
> *$old{SCALAR} and *$new = *$old{SCALAR};
> *$old{ARRAY} and *$new = *$old{ARRAY};
> *$old{HASH} and *$new = *$old{HASH};
> *$old{IO} and *$new = *$old{IO};
> # skip FORMAT since it's buggy in some perls
> };
> }
>
> Now, when you say
>
> {
> use Foo;
> say foo;
> }
> say foo;
>
> the first call to foo calls the installed sub, but the second gives you
> an undefined sub error. Also, if you say
>
> use Foo;
> say foo;
> say main->foo;
>
> the sub call calls the installed sub, but the method call gives an
> undefined method error.
>
> This was (as usual) quite accidental, but is extremely useful. It
> happens because perl resolves sub calls as far as the relevant glob at
> compile time, so if you replace that glob in the symbol table you can
> put a different sub (or no sub at all) in its place without affecting
> already-compiled code.
>
> (The reason perl only resolves as far as the glob, and not right down to
> the CV, is to allow subs to be redefined. So a simple
>
> undef&{"$pkg\::foo"};
> or
> *{"$pkg\::foo"} = sub { "bar" };
>
> *would* affect the already-compiled code, since it leaves the same glob
> in the stash and only affects its CV entry.)
>
> This idea is what is behind namespace::clean and the other namespace::*
> modules, which hide the nasty internals behind a cleaner interface.

I'm sure you can do that kinds of things with some C compilers as well.
Just find an appropriate vulnerability and then inject some code in the
compiler using your source code as a vector...

Kiuhnm

In <4f55036b$0$1393$4fafbaef@reader1.news.tin.it>, on 03/05/2012
at 07:18 PM, Kiuhnm <kiuhnm03.4t.yahoo.it> said:

>But named sub are always global.

In Perl; 5. That's not true of all languages, especially not those
descended from Algol 60.

--
Shmuel (Seymour J.) Metz, SysProg and JOAT <http://patriot.net/~shmuel>

Unsolicited bulk E-mail subject to legal action. I reserve the
right to publicly post or ridicule any abusive E-mail. Reply to
domain Patriot dot net user shmuel+news to contact me. Do not
reply to spamtrap@library.lspace.org