Discussion:
[perl #53976] [CAGE] Remove tools/dev/ops_renum.mak
(too old to reply)
Will Coleda
2008-05-11 03:22:53 UTC
Permalink
# New Ticket Created by Will Coleda
# Please include the string: [perl #53976]
# in the subject line of all future correspondence about this issue.
# <URL: http://rt.perl.org/rt3/Ticket/Display.html?id=53976 >


This make file isn't preprocessed like the standard root.in: it
assumes perl is in your path, and redefines the list of OPSFILES.

Let's move the step into the core makefile, and add another
--maintainer tweak like the one for lex/yacc - either run the real
command in maintainer mode, or set the command to be 'echo' in
non-maintainer mode. Then we can eliminate this other makefile.

Feedback welcome.
--
Will "Coke" Coleda
Chromatic
2008-05-11 06:42:45 UTC
Permalink
Post by Will Coleda
This make file isn't preprocessed like the standard root.in: it
assumes perl is in your path, and redefines the list of OPSFILES.
Let's move the step into the core makefile, and add another
--maintainer tweak like the one for lex/yacc - either run the real
command in maintainer mode, or set the command to be 'echo' in
non-maintainer mode. Then we can eliminate this other makefile.
Feedback welcome.
+1, provided we set the dependency properly such that it doesn't always
regenerate the files.
--
Nobody on a *gardening* website, for example, would seriously suggest
that CMM has anything to do with software quality.
- chromatic
James Keenan via RT
2008-06-21 14:16:58 UTC
Permalink
Post by Will Coleda
This make file isn't preprocessed like the standard root.in: it
assumes perl is in your path, and redefines the list of OPSFILES.
Let me pose some questions:

1. Who actually uses this program? Under what circumstances?

2. Must exist this program as a makefile? After all, all it does is to
provide a list of arguments[1] to a specialized use of one of the build
tools programs:

perl tools/build/ops2pm.pl --renum $(OPS_FILES)

Peeking in to tools/build/ops2pm.pl, we see that the '--renum' option is
one of those edge cases where a program invoked by 'make' is used
outside the normal context of 'make':

my $self = Parrot::Ops2pm::Utils->new(
{
argv => [@ARGV],
nolines => $flagref->{nolines},
renum => $flagref->{renum},
moddir => "lib/Parrot/OpLib",
module => "core.pm",
inc_dir => "include/parrot/oplib",
inc_f => "ops.h",
script => "tools/build/ops2pm.pl",
}
);

$self->prepare_ops();

if ( $flagref->{renum} ) {
$self->renum_op_map_file();
exit 0;
}

And if we further peer into Parrot::Ops2pm::Utils, we see that
renum_op_map_file() is described as follows:

"When F<tools/build/ops2pm.pl> is called with the C<--renum> option, this
method is triggered, after which F<ops2pm.pl> exits. Consequently, this
is the only Parrot::Ops2pm::Utils method which is I<not> a stepping
stone on the path to building F<lib/Parrot/OpLib/core.pm>."

So both tools/dev/ops_renum.mak and ops2pm.pl --renum seem to be Parrot
developers tools -- not intrinsic components of 'make'. My
recommendation -- once we have answered question (1) above -- would be:

(a) pull renum_op_map_file() out of Parrot::Ops2pm::Utils and into a
subclass -- the better to emphasize that it's not part of the regular
'make' process;

(b) pull the '--renum' option out of tools/build/ops2pm.pl;

(c) replace tools/dev/ops_renum.mak with a pure Perl program which does
what 'tools/build/ops2pm.pl --renum' does now; and

(d) only at this point determine whether we need to gussy this up as a
'make' target.

Thank you very much.
kid51


[1] 'make' invokes 'ops2pm.pl' once:

/usr/local/bin/perl tools/build/ops2pm.pl src/ops/core.ops
src/ops/bit.ops src/ops/cmp.ops src/ops/debug.ops
src/ops/experimental.ops src/ops/io.ops src/ops/math.ops
src/ops/object.ops src/ops/pic.ops src/ops/pmc.ops src/ops/set.ops
src/ops/stm.ops src/ops/string.ops src/ops/sys.ops src/ops/var.ops

core.ops is called first, then all the other .ops files in that
directory *except* 'obscure.ops' are called in alphabetical order.

tools/dev/ops_renum.mak calls the same list *with the exception of*
'experimental.ops.' I am not clear on why 'make' and
'tools/dev/ops_renum.mak' call different lists of '.ops' files -- and
I'm not sure why we retain a file called 'obscure.ops' in the same
directory as all our other essential '.ops' files.
Chromatic
2008-06-21 17:40:29 UTC
Permalink
Post by James Keenan via RT
Post by Will Coleda
This make file isn't preprocessed like the standard root.in: it
assumes perl is in your path, and redefines the list of OPSFILES.
1. Who actually uses this program? Under what circumstances?
Anyone who adds or removes a (non-experimental) Parrot op.
Post by James Keenan via RT
2. Must exist this program as a makefile?
Not at all. As far as I know, the only reason it's a makefile is history.
Post by James Keenan via RT
So both tools/dev/ops_renum.mak and ops2pm.pl --renum seem to be Parrot
developers tools -- not intrinsic components of 'make'. My
(a) pull renum_op_map_file() out of Parrot::Ops2pm::Utils and into a
subclass -- the better to emphasize that it's not part of the regular
'make' process;
I'm not sure a subclass is strictly necessary, but that won't hurt.
Post by James Keenan via RT
(b) pull the '--renum' option out of tools/build/ops2pm.pl;
(c) replace tools/dev/ops_renum.mak with a pure Perl program which does
what 'tools/build/ops2pm.pl --renum' does now; and
(d) only at this point determine whether we need to gussy this up as a
'make' target.
It would be convenient to have a make target which renumbers ops. Every time
I've added or removed an op, I've had to go source diving to figure out where
and how to invoke this makefile.
Post by James Keenan via RT
/usr/local/bin/perl tools/build/ops2pm.pl src/ops/core.ops
src/ops/bit.ops src/ops/cmp.ops src/ops/debug.ops
src/ops/experimental.ops src/ops/io.ops src/ops/math.ops
src/ops/object.ops src/ops/pic.ops src/ops/pmc.ops src/ops/set.ops
src/ops/stm.ops src/ops/string.ops src/ops/sys.ops src/ops/var.ops
core.ops is called first, then all the other .ops files in that
directory *except* 'obscure.ops' are called in alphabetical order.
tools/dev/ops_renum.mak calls the same list *with the exception of*
'experimental.ops.' I am not clear on why 'make' and
'tools/dev/ops_renum.mak' call different lists of '.ops' files -- and
I'm not sure why we retain a file called 'obscure.ops' in the same
directory as all our other essential '.ops' files.
None of the ops in experimental.ops get formal op numbers, as far as I can
tell. Thus they don't need renumbering.

I don't know what, if anything, uses obscure ops.

-- c
Bob Rogers
2008-06-21 18:36:21 UTC
Permalink
From: chromatic <***@wgz.org>
Date: Sat, 21 Jun 2008 10:40:29 -0700

On Saturday 21 June 2008 07:16:58 James Keenan via RT wrote:

. . .
Post by James Keenan via RT
(b) pull the '--renum' option out of tools/build/ops2pm.pl;
(c) replace tools/dev/ops_renum.mak with a pure Perl program which does
what 'tools/build/ops2pm.pl --renum' does now; and
(d) only at this point determine whether we need to gussy this up as a
'make' target.
It would be convenient to have a make target which renumbers ops. Every time
I've added or removed an op, I've had to go source diving to figure out where
and how to invoke this makefile.

When I do so, I get the following error:

hole in ops.num before #1186 at lib/Parrot/Ops2pm/Utils.pm line 330, <$op> line 1206.

If this message told how to fix it, that would reduce the amount of
esoteric knowledge required; you wouldn't even need to know to look in
the makefile. So if it were me, I would remove the makefile target, and
change Parrot::Ops2pm::Utils to point to the new script.

-- Bob Rogers
http://rgrjr.dyndns.org/
Will Coleda
2008-06-22 00:57:48 UTC
Permalink
Post by Chromatic
Post by James Keenan via RT
Post by Will Coleda
This make file isn't preprocessed like the standard root.in: it
assumes perl is in your path, and redefines the list of OPSFILES.
1. Who actually uses this program? Under what circumstances?
Anyone who adds or removes a (non-experimental) Parrot op.
Post by James Keenan via RT
2. Must exist this program as a makefile?
Not at all. As far as I know, the only reason it's a makefile is history.
I would rather it existed as a build step in the regular makefile.
Perhaps something that was only triggered if you were running in
--maintainer mode, as I have suggested elsewhere.
Post by Chromatic
Post by James Keenan via RT
So both tools/dev/ops_renum.mak and ops2pm.pl --renum seem to be Parrot
developers tools -- not intrinsic components of 'make'. My
(a) pull renum_op_map_file() out of Parrot::Ops2pm::Utils and into a
subclass -- the better to emphasize that it's not part of the regular
'make' process;
I'm not sure a subclass is strictly necessary, but that won't hurt.
Post by James Keenan via RT
(b) pull the '--renum' option out of tools/build/ops2pm.pl;
(c) replace tools/dev/ops_renum.mak with a pure Perl program which does
what 'tools/build/ops2pm.pl --renum' does now; and
(d) only at this point determine whether we need to gussy this up as a
'make' target.
It would be convenient to have a make target which renumbers ops. Every time
I've added or removed an op, I've had to go source diving to figure out where
and how to invoke this makefile.
Post by James Keenan via RT
/usr/local/bin/perl tools/build/ops2pm.pl src/ops/core.ops
src/ops/bit.ops src/ops/cmp.ops src/ops/debug.ops
src/ops/experimental.ops src/ops/io.ops src/ops/math.ops
src/ops/object.ops src/ops/pic.ops src/ops/pmc.ops src/ops/set.ops
src/ops/stm.ops src/ops/string.ops src/ops/sys.ops src/ops/var.ops
core.ops is called first, then all the other .ops files in that
directory *except* 'obscure.ops' are called in alphabetical order.
tools/dev/ops_renum.mak calls the same list *with the exception of*
'experimental.ops.' I am not clear on why 'make' and
'tools/dev/ops_renum.mak' call different lists of '.ops' files -- and
I'm not sure why we retain a file called 'obscure.ops' in the same
directory as all our other essential '.ops' files.
None of the ops in experimental.ops get formal op numbers, as far as I can
tell. Thus they don't need renumbering.
In ages past, I believe this was setup so that we would have a place
we could put ops that would allow us to add and subtract ops without
impacting our declared op list in between major releases.
Post by Chromatic
I don't know what, if anything, uses obscure ops.
While there is plenty of room for obscure ops in parrot's bytecode, I
would recommend that we move these particular ops over to methods on
the Float pmc and retire the obscure.ops file; unless nothing uses
them, in which case I suggest we just rip them out.
Post by Chromatic
-- c
--
Will "Coke" Coleda
James Keenan via RT
2008-06-22 17:05:05 UTC
Permalink
On Sat Jun 21 07:16:56 2008, ***@verizon.net wrote:
[snip]
Post by James Keenan via RT
(a) pull renum_op_map_file() out of Parrot::Ops2pm::Utils and into a
subclass -- the better to emphasize that it's not part of the regular
'make' process;
(b) pull the '--renum' option out of tools/build/ops2pm.pl;
(c) replace tools/dev/ops_renum.mak with a pure Perl program which does
what 'tools/build/ops2pm.pl --renum' does now; and
(d) only at this point determine whether we need to gussy this up as a
'make' target.
Please review the patch attached, which represents the current state of
development in the 'opsrenum' branch in the repository. The patch
implements (a), (b) and (c) above.

As for (d), I limited myself to establishing a 'renumberops' target in
config/gen/makefiles/root.in. I then reconfigured and called 'make
renumberops'. It renumbered the ops, because, after handling
src/ops/core.ops, it handled the remaining .ops files (including
experimental.ops but excluding obscure.opc) in alphabetical order. In
contrast, the older tools/dev/ops_renum.mak for some reason had stm.ops
out of alphabetical order.

One thing I don't understand: In config/gen/makefiles/root.in, I see
the following variable assignment:

OPS_FILES = @ops@ $(GEN_OPSFILES) # line 455

There currently are no generated .ops files (I think):

GEN_OPSFILES = # line 210

So OPS_FILES holds @***@. But I don't understand where @ops@ is
defined/assigned to.

This patch passes all existing tests, but since I never used
tools/dev/ops_renum.mak myself, I can't guarantee that that covers all
bases/cases. I would recommend that those who are concerned about this
RT do a checkout of the 'opsrenum' branch, add/subtract some ops, and
then see if both tools/dev/opsrenumber.pl and make renumberops DTRT.

Thank you very much.

kid51
James Keenan via RT
2008-06-22 17:30:57 UTC
Permalink
Post by James Keenan via RT
One thing I don't understand: In config/gen/makefiles/root.in, I see
GEN_OPSFILES = # line 210
defined/assigned to.
I found it: config/auto/ops.pm.
Will Coleda
2008-07-01 00:02:01 UTC
Permalink
On Sun, Jun 22, 2008 at 1:05 PM, James Keenan via RT
Post by James Keenan via RT
[snip]
Post by James Keenan via RT
(a) pull renum_op_map_file() out of Parrot::Ops2pm::Utils and into a
subclass -- the better to emphasize that it's not part of the regular
'make' process;
(b) pull the '--renum' option out of tools/build/ops2pm.pl;
(c) replace tools/dev/ops_renum.mak with a pure Perl program which does
what 'tools/build/ops2pm.pl --renum' does now; and
(d) only at this point determine whether we need to gussy this up as a
'make' target.
Please review the patch attached, which represents the current state of
development in the 'opsrenum' branch in the repository. The patch
implements (a), (b) and (c) above.
As for (d), I limited myself to establishing a 'renumberops' target in
config/gen/makefiles/root.in. I then reconfigured and called 'make
renumberops'. It renumbered the ops, because, after handling
src/ops/core.ops, it handled the remaining .ops files (including
experimental.ops but excluding obscure.opc) in alphabetical order. In
contrast, the older tools/dev/ops_renum.mak for some reason had stm.ops
out of alphabetical order.
One thing I don't understand: In config/gen/makefiles/root.in, I see
GEN_OPSFILES = # line 210
defined/assigned to.
This patch passes all existing tests, but since I never used
tools/dev/ops_renum.mak myself, I can't guarantee that that covers all
bases/cases. I would recommend that those who are concerned about this
RT do a checkout of the 'opsrenum' branch, add/subtract some ops, and
then see if both tools/dev/opsrenumber.pl and make renumberops DTRT.
Thank you very much.
kid51
First, thanks for tackling this. I'd honestly like to incorporate this
target into the normal build chain so we don't have to manually run it
anymore, but eliminating the custom make file is definitely a pre-req
for that.

Second, two notes about the branch:

In a fresh checkout, if I 'make renumberops' with no local
modifications, src/ops/ops.num changes.
If I rename the op store_lex to barf_lex, and run 'make renumberops',
the opcode barf_lex doesn't show up in src/ops/ops.num. (But,
store_lex vanishes)

Regards.
--
Will "Coke" Coleda
James Keenan via RT
2008-07-01 02:31:22 UTC
Permalink
Post by Will Coleda
In a fresh checkout, if I 'make renumberops' with no local
modifications, src/ops/ops.num changes.
With my non-fresh branch, I could not confirm this.
Post by Will Coleda
If I rename the op store_lex to barf_lex, and run 'make renumberops',
the opcode barf_lex doesn't show up in src/ops/ops.num. (But,
store_lex vanishes)
Confirmed. Back to the drawing board.
James Keenan via RT
2008-07-02 11:03:40 UTC
Permalink
Post by Will Coleda
In a fresh checkout, if I 'make renumberops' with no local
modifications, src/ops/ops.num changes.
If I rename the op store_lex to barf_lex, and run 'make renumberops',
the opcode barf_lex doesn't show up in src/ops/ops.num. (But,
store_lex vanishes)
I have found the code which causes (at least the) 2nd problem Coke
describes and know how to fix it. See the opsrenum branch.

But whether my fix is correct or not depends on two questions:

(1) Is the following passage from the inline comments in src/ops/ops.num
still valid?

# once an opcode is added to this file it should never be
# removed. Opcodes that are in here but that have no corresponding
# function backing them (because, for example, they've been deleted,
# which shouldn't ever happen once we hit production) should be mapped
# by the ops processing programs to an exception op

If it *is* valid, then any renaming of an opcode should leave that op's
*current* name and number in number.ops unchanged. The *new* name for
the opcode should presumably be pushed on to the end of the list.

But this runs counter to the generally programming principle that we
should get rid of old, unused code. Why is this case different?

Which leads to the next question:

(2) Why do we maintain number.ops as a *static file of source code at
all*? Why is it in the repository? If a given Parrot developer, for a
given instance of Parrot configuration and build, renumbers the opcodes
(either because the list of opcodes has changed or just for the hell of
it), why must that renumbering be inflicted on all other Parrot
developers -- which it would be as the result of 'svn ci'?

Thank you very much.
kid51
Jerry Gay
2008-07-02 13:24:35 UTC
Permalink
On Wed, Jul 2, 2008 at 4:03 AM, James Keenan via RT
Post by James Keenan via RT
Post by Will Coleda
In a fresh checkout, if I 'make renumberops' with no local
modifications, src/ops/ops.num changes.
If I rename the op store_lex to barf_lex, and run 'make renumberops',
the opcode barf_lex doesn't show up in src/ops/ops.num. (But,
store_lex vanishes)
I have found the code which causes (at least the) 2nd problem Coke
describes and know how to fix it. See the opsrenum branch.
(1) Is the following passage from the inline comments in src/ops/ops.num
still valid?
# once an opcode is added to this file it should never be
# removed. Opcodes that are in here but that have no corresponding
# function backing them (because, for example, they've been deleted,
# which shouldn't ever happen once we hit production) should be mapped
# by the ops processing programs to an exception op
If it *is* valid, then any renaming of an opcode should leave that op's
*current* name and number in number.ops unchanged. The *new* name for
the opcode should presumably be pushed on to the end of the list.
But this runs counter to the generally programming principle that we
should get rid of old, unused code. Why is this case different?
(2) Why do we maintain number.ops as a *static file of source code at
all*? Why is it in the repository? If a given Parrot developer, for a
given instance of Parrot configuration and build, renumbers the opcodes
(either because the list of opcodes has changed or just for the hell of
it), why must that renumbering be inflicted on all other Parrot
developers -- which it would be as the result of 'svn ci'?
Thank you very much.
kid51
opcodes are the very heart of bytecode. if the opcodes change, then
the bytecode is invalid. if opcode #1 (noop) was changed to something
like (delete_harddisk), and parrot is rebuilt, than any existing
bytecode (.pbc file) that contains a noop instruction will instead
perform delete_harddisk. that's *never* the right thing to do--it
means bytecode can never be shared reliably across parrot builds.
portable bytecode is one of the main goals of parrot.

therefore, in an attempt to keep bytecode compatible across versions
of parrot, opcodes can never be deleted. instead, if opcodes are
deprecated, their function bodies should throw an exception explaining
that the opcode is no longer supported.

like the comment says, this will be strictly enforced after parrot
1.0, but is a good rule to follow now as well. the design team
reserves the right to make architectural changes that impact the
opcode numbers, of course, until we have a production-ready product.

~jerry
James Keenan via RT
2008-07-02 16:17:46 UTC
Permalink
Post by Jerry Gay
therefore, in an attempt to keep bytecode compatible across versions
of parrot, opcodes can never be deleted. instead, if opcodes are
deprecated, their function bodies should throw an exception explaining
that the opcode is no longer supported.
What purpose remains, then, for either tools/dev/ops_renum.mak or my
alternative, tools/dev/opsrenumber.pl? Is such a program only intended
to provide a number for newly added opcodes?

kid51
James Keenan via RT
2008-07-10 22:38:11 UTC
Permalink
Post by James Keenan via RT
Post by Jerry Gay
therefore, in an attempt to keep bytecode compatible across versions
of parrot, opcodes can never be deleted. instead, if opcodes are
deprecated, their function bodies should throw an exception explaining
that the opcode is no longer supported.
What purpose remains, then, for either tools/dev/ops_renum.mak or my
alternative, tools/dev/opsrenumber.pl? Is such a program only intended
to provide a number for newly added opcodes?
Still hoping for some feedback on the question above.

Thanks.

kid51
James Keenan via RT
2008-08-09 13:33:46 UTC
Permalink
Post by James Keenan via RT
What purpose remains, then, for either tools/dev/ops_renum.mak or my
alternative, tools/dev/opsrenumber.pl? Is such a program only intended
to provide a number for newly added opcodes?
kid51
We haven't had any response to this question in the last month. If we
don't in the next few days, then I'm going to mark this ticket as
stalled and focus my attention elsewhere.

Thank you very much.
kid51
Chromatic
2008-08-09 17:30:52 UTC
Permalink
Post by James Keenan via RT
Post by James Keenan via RT
What purpose remains, then, for either tools/dev/ops_renum.mak or my
alternative, tools/dev/opsrenumber.pl? Is such a program only intended
to provide a number for newly added opcodes?
We haven't had any response to this question in the last month. If we
don't in the next few days, then I'm going to mark this ticket as
stalled and focus my attention elsewhere.
Hm, I missed the question the first time. The program renumbers opcodes when
adding or removing one or more opcodes. (Removing is less likely as time
goes by, but we reserve the right to do so for the forseeable future.)

-- c
James Keenan via RT
2008-08-09 22:50:01 UTC
Permalink
Post by James Keenan via RT
Post by James Keenan via RT
What purpose remains, then, for either tools/dev/ops_renum.mak or
my
Post by James Keenan via RT
alternative, tools/dev/opsrenumber.pl? Is such a program only
intended
Post by James Keenan via RT
to provide a number for newly added opcodes?
Hm, I missed the question the first time. The program renumbers opcodes when
adding or removing one or more opcodes. (Removing is less likely as time
goes by, but we reserve the right to do so for the forseeable future.)
Hmm ... it appears to me that particle took a harder line earlier in
this RT against removal: "in an attempt to keep bytecode compatible
across versions of parrot, opcodes can never be deleted."

Do we have a disagreement there? If not, I'd like to know whether my
replacement script DTRT or needs fixing ('cause I'm more confused than
when i started).

Thank you very much.
kid51
Jerry Gay
2008-08-10 07:46:08 UTC
Permalink
On Sat, Aug 9, 2008 at 3:50 PM, James Keenan via RT
Post by James Keenan via RT
Post by James Keenan via RT
Post by James Keenan via RT
What purpose remains, then, for either tools/dev/ops_renum.mak or
my
Post by James Keenan via RT
alternative, tools/dev/opsrenumber.pl? Is such a program only
intended
Post by James Keenan via RT
to provide a number for newly added opcodes?
Hm, I missed the question the first time. The program renumbers opcodes when
adding or removing one or more opcodes. (Removing is less likely as time
goes by, but we reserve the right to do so for the forseeable future.)
Hmm ... it appears to me that particle took a harder line earlier in
this RT against removal: "in an attempt to keep bytecode compatible
across versions of parrot, opcodes can never be deleted."
Do we have a disagreement there? If not, I'd like to know whether my
replacement script DTRT or needs fixing ('cause I'm more confused than
when i started).
let me clarify: after 1.0, in order to allow bytecode to be work
across parrot releases (where there may be new or deprecated opcodes),
opcode numbers can never be reused. before parrot is production ready
(before 1.0) we have not made any promise of bytecode compatibility
across releases, other than that it'd be great if we can do it (which
means we can reuse opcode numbers if we want to, but i still find it
unlikely as it's counter-cultural.

i don't see any disagreement between myself and chromatic.
~jerry
James Keenan via RT
2008-08-14 02:39:16 UTC
Permalink
Okay, thanks for getting back to me. Could you read the inline comments
I have inserted into Parrot::OpsRenumber::renum_op_map_file()?

Do the comments accurately reflect what needs to happen re opcode
renumbering both now and at 1.0?

You can read them here:

http://svn.perl.org/parrot/branches/opsrenum/lib/Parrot/OpsRenumber.pm

Thank you very much.

kid51
Chromatic
2008-08-14 05:13:23 UTC
Permalink
Post by James Keenan via RT
Okay, thanks for getting back to me. Could you read the inline comments
I have inserted into Parrot::OpsRenumber::renum_op_map_file()?
Do the comments accurately reflect what needs to happen re opcode
renumbering both now and at 1.0?
Yes, they're accurate.

-- c
James Keenan via RT
2008-08-15 02:24:53 UTC
Permalink
Thanks. Am close to submitting a patch. Anyone who wants to kibitz
should check out the 'opsrenum' branch from SVN.
Will Coleda
2008-07-02 14:27:29 UTC
Permalink
Post by Will Coleda
On Sun, Jun 22, 2008 at 1:05 PM, James Keenan via RT
Post by James Keenan via RT
[snip]
Post by James Keenan via RT
(a) pull renum_op_map_file() out of Parrot::Ops2pm::Utils and into a
subclass -- the better to emphasize that it's not part of the regular
'make' process;
(b) pull the '--renum' option out of tools/build/ops2pm.pl;
(c) replace tools/dev/ops_renum.mak with a pure Perl program which does
what 'tools/build/ops2pm.pl --renum' does now; and
(d) only at this point determine whether we need to gussy this up as a
'make' target.
Please review the patch attached, which represents the current state of
development in the 'opsrenum' branch in the repository. The patch
implements (a), (b) and (c) above.
As for (d), I limited myself to establishing a 'renumberops' target in
config/gen/makefiles/root.in. I then reconfigured and called 'make
renumberops'. It renumbered the ops, because, after handling
src/ops/core.ops, it handled the remaining .ops files (including
experimental.ops but excluding obscure.opc) in alphabetical order. In
contrast, the older tools/dev/ops_renum.mak for some reason had stm.ops
out of alphabetical order.
One thing I don't understand: In config/gen/makefiles/root.in, I see
GEN_OPSFILES = # line 210
defined/assigned to.
This patch passes all existing tests, but since I never used
tools/dev/ops_renum.mak myself, I can't guarantee that that covers all
bases/cases. I would recommend that those who are concerned about this
RT do a checkout of the 'opsrenum' branch, add/subtract some ops, and
then see if both tools/dev/opsrenumber.pl and make renumberops DTRT.
Thank you very much.
kid51
First, thanks for tackling this. I'd honestly like to incorporate this
target into the normal build chain so we don't have to manually run it
anymore, but eliminating the custom make file is definitely a pre-req
for that.
In a fresh checkout, if I 'make renumberops' with no local
modifications, src/ops/ops.num changes.
If I rename the op store_lex to barf_lex, and run 'make renumberops',
the opcode barf_lex doesn't show up in src/ops/ops.num. (But,
store_lex vanishes)
Having just tried this in a different branch for a different reason, I
found I had to manually add the opcodes to ops.num. Without checking
the code, it seems like it doesn't scan for added opcode, merely
deleted ones.

Regards.
Post by Will Coleda
Regards.
--
Will "Coke" Coleda
--
Will "Coke" Coleda
James Keenan via RT
2008-08-17 03:39:35 UTC
Permalink
The patch attached, diff.trunk.opsrenum.txt, is an improved solution to
the problem posed by Coke in the OP of this RT. Here are its features,
working from the surface (e.g., file name changes) down to the method
that does the renumbering of opcodes.

1. Renames lib/Parrot/Ops2pm/Utils.pm to lib/Parrot/Ops2pm.pm. Renames
the directory where the tests for this package are found from
t/tools/ops2pmutils/ to t/tools/ops2pm. Makes appropriate changes in
names within files.

2. Takes the method that did -- and continues to do -- the rewriting of
the ops.num file, renum_op_map_file(), out of lib/Parrot/Ops2pm.pm and
places it in its own module, lib/Parrot/OpsRenumber.pm. This is done so
that the methods that are used in tools/build/ops2pm.pl are found in one
package while the method used in tools/dev/opsrenumber.pl is placed in a
package of its own.

3. Two methods that are used in both lib/Parrot/Ops2pm.pm and
lib/Parrot/OpsRenumber.pm are extracted and placed in a module of their
own: lib/Parrot/Ops2pm/Base.pm. Those methods are new() (the
constructor) and prepare_ops() (which invokes Parrot::OpsFile, which
does the actual parsing of ops files).

4. Parrot::OpsRenumber::renum_op_map_file() has been revised so that it
will behave properly before Parrot 1.0 -- when deletion of opcodes is
still permitted -- and after 1.0 -- when no deletions are permitted.
This flexibility is attained by providing it with the Parrot major
version as an argument. This before-and-after functionality is tested
in a heavily revised t/tools/ops2pm/05-renum_op_map_file.t. Several new
sample files used in testing have been added to the distribution.

5. The name of the program that invokes Parrot::OpsRenumber is changed
from tools/dev/ops_renum.mak to tools/dev/opsrenumber.pl. It is, after
all, a Perl 5 program, so we provide it with a file name extension
consistent with other programs in tools/dev/.

6. A new makefile target, renumberops, invokes tools/dev/opsrenumber.pl.

Please review, particularly lib/Parrot/OpsRenumber.pm and
t/tools/ops2pm/05-renum_op_map_file.t, which is where most of the really
new stuff is found and/or demonstrated. This patch probably does not do
what Coke requested in the OP, but I don't actually know how to do that.
But this refactoring should get us closer to that objective.

Thank you very much.
kid51
Will Coleda
2008-08-23 02:20:49 UTC
Permalink
On Sat, Aug 16, 2008 at 11:39 PM, James Keenan via RT
Post by James Keenan via RT
The patch attached, diff.trunk.opsrenum.txt, is an improved solution to
the problem posed by Coke in the OP of this RT. Here are its features,
working from the surface (e.g., file name changes) down to the method
that does the renumbering of opcodes.
1. Renames lib/Parrot/Ops2pm/Utils.pm to lib/Parrot/Ops2pm.pm. Renames
the directory where the tests for this package are found from
t/tools/ops2pmutils/ to t/tools/ops2pm. Makes appropriate changes in
names within files.
2. Takes the method that did -- and continues to do -- the rewriting of
the ops.num file, renum_op_map_file(), out of lib/Parrot/Ops2pm.pm and
places it in its own module, lib/Parrot/OpsRenumber.pm. This is done so
that the methods that are used in tools/build/ops2pm.pl are found in one
package while the method used in tools/dev/opsrenumber.pl is placed in a
package of its own.
3. Two methods that are used in both lib/Parrot/Ops2pm.pm and
lib/Parrot/OpsRenumber.pm are extracted and placed in a module of their
own: lib/Parrot/Ops2pm/Base.pm. Those methods are new() (the
constructor) and prepare_ops() (which invokes Parrot::OpsFile, which
does the actual parsing of ops files).
So far, all generic refactoring which is fine.
Post by James Keenan via RT
4. Parrot::OpsRenumber::renum_op_map_file() has been revised so that it
will behave properly before Parrot 1.0 -- when deletion of opcodes is
still permitted -- and after 1.0 -- when no deletions are permitted.
This flexibility is attained by providing it with the Parrot major
version as an argument. This before-and-after functionality is tested
in a heavily revised t/tools/ops2pm/05-renum_op_map_file.t. Several new
sample files used in testing have been added to the distribution.
I know you were implementing Jerry's requirement here, and that's good.

I do wonder what the plan is for opcodes that are marked deprecated
(which we can do today.) ; once they go past deprecated (before 1.0,
we'd just delete them), what will happen when they are executed? will
we add a :removed pragma on the opcode definition that will simply
throw an exception when they are invoked? Will we just silently
translate them to a noop?

What about the case when we experimentally add an opcode between
official releases, but back it out before the next release? What
counts as an official release? How are we going to number the
intervening releases? Do we have a version indicator to mark as
stable/unstable/experimental/Don't Touch This Without A Hazmat Suit?

What about experimental.ops? is that going to survive with its special
treatment into post-1.0?

I apologize for not asking these questions immediately following
Jerry's original post, but I thought I had until version 0.99 to think
about it. =-)

These questions asked... I'm don't think they block application of the
patch at all; I think these are all questions that should be answered
in a new docs/project/release_strategy.pod which can then spawn TODOs
as necessary.
Post by James Keenan via RT
5. The name of the program that invokes Parrot::OpsRenumber is changed
from tools/dev/ops_renum.mak to tools/dev/opsrenumber.pl. It is, after
all, a Perl 5 program, so we provide it with a file name extension
consistent with other programs in tools/dev/.
6. A new makefile target, renumberops, invokes tools/dev/opsrenumber.pl.
Please review, particularly lib/Parrot/OpsRenumber.pm and
t/tools/ops2pm/05-renum_op_map_file.t, which is where most of the really
new stuff is found and/or demonstrated.
I don't have any questions regarding these tests that are unique to
these particular tests.
Post by James Keenan via RT
This patch probably does not do
what Coke requested in the OP, but I don't actually know how to do that.
That is lost in the mists of time, but I'm pretty sure I was looking
for a way to make the dependencies smarter here so we didn't have to
remember to run it. That probably conflicts with Jerry's post-1.0
strictures, however, so I'm not entirely sure that it's relevant
anymore. However, the fact that it's a standard perl script now
instead of a unixy makefile...
Post by James Keenan via RT
But this refactoring should get us closer to that objective.
...Exactly.

VMNWIHTEM[0]: The copyright in a lot of the new files seems to be
2007. Should it be 2008?
Post by James Keenan via RT
Thank you very much.
kid51
Thank you for digging into this.

[0] Very Minor Nit Which I Hate To Even Mention
--
Will "Coke" Coleda
Jerry Gay
2008-08-23 05:10:32 UTC
Permalink
Post by Will Coleda
On Sat, Aug 16, 2008 at 11:39 PM, James Keenan via RT
Post by James Keenan via RT
4. Parrot::OpsRenumber::renum_op_map_file() has been revised so that it
will behave properly before Parrot 1.0 -- when deletion of opcodes is
still permitted -- and after 1.0 -- when no deletions are permitted.
This flexibility is attained by providing it with the Parrot major
version as an argument. This before-and-after functionality is tested
in a heavily revised t/tools/ops2pm/05-renum_op_map_file.t. Several new
sample files used in testing have been added to the distribution.
I know you were implementing Jerry's requirement here, and that's good.
I do wonder what the plan is for opcodes that are marked deprecated
(which we can do today.) ; once they go past deprecated (before 1.0,
we'd just delete them), what will happen when they are executed? will
we add a :removed pragma on the opcode definition that will simply
throw an exception when they are invoked? Will we just silently
translate them to a noop?
What about the case when we experimentally add an opcode between
official releases, but back it out before the next release? What
counts as an official release? How are we going to number the
intervening releases? Do we have a version indicator to mark as
stable/unstable/experimental/Don't Touch This Without A Hazmat Suit?
What about experimental.ops? is that going to survive with its special
treatment into post-1.0?
I apologize for not asking these questions immediately following
Jerry's original post, but I thought I had until version 0.99 to think
about it. =-)
These questions asked... I'm don't think they block application of the
patch at all; I think these are all questions that should be answered
in a new docs/project/release_strategy.pod which can then spawn TODOs
as necessary.
agreed. we still have time to address these questions. it would be
nice if they weren't lost, though. i suggest a wiki page rather than a
mess of rt tickets for better tracking at this point. seems to me that
the wiki approach will allow us to collect and organize these thoughts
as they occur, and serve as a solid draft when we formalize them
later.
Post by Will Coleda
Post by James Keenan via RT
5. The name of the program that invokes Parrot::OpsRenumber is changed
from tools/dev/ops_renum.mak to tools/dev/opsrenumber.pl. It is, after
all, a Perl 5 program, so we provide it with a file name extension
consistent with other programs in tools/dev/.
6. A new makefile target, renumberops, invokes tools/dev/opsrenumber.pl.
Please review, particularly lib/Parrot/OpsRenumber.pm and
t/tools/ops2pm/05-renum_op_map_file.t, which is where most of the really
new stuff is found and/or demonstrated.
I don't have any questions regarding these tests that are unique to
these particular tests.
it strikes me that the makefile target and the script name should be
the same. there's still too much irregularity in parrot's naming
conventions across the board--opcodes, pir syntax, makefiles, scripts,
directories, filenames, c functions, etc.

we'll concentrate on this in earnest as we approach our first general
availability release (whether we call that release 1.0 or 2009R1 or
"Ball of Mud"), however there's no reason to hold off. if anybody sees
anything that looks a bit out of place (e.g. dynops vs. dynoplibs)
take note of it by opening a CAGE ticket in rt. it may take some
project team input to settle upon the preferred naming convention, but
implementing that work can be done by any contributor and applied by
any committer.

~jerry
James Keenan via RT
2008-08-23 12:41:20 UTC
Permalink
Post by Jerry Gay
Post by James Keenan via RT
6. A new makefile target, renumberops, invokes
tools/dev/opsrenumber.pl.
Post by Jerry Gay
it strikes me that the makefile target and the script name should be
the same.
Changed to 'make opsrenumber'.
James Keenan via RT
2008-08-23 15:08:03 UTC
Permalink
Patch committed to trunk, with small corrections, in r30478.
James Keenan via RT
2008-08-24 15:06:32 UTC
Permalink
Post by Will Coleda
This make file isn't preprocessed like the standard root.in: it
assumes perl is in your path, and redefines the list of OPSFILES.
Let's move the step into the core makefile, and add another
--maintainer tweak like the one for lex/yacc - either run the real
command in maintainer mode, or set the command to be 'echo' in
non-maintainer mode. Then we can eliminate this other makefile.
Feedback welcome.
Coke,

I'm going to give this ticket back to you so that if you want to do the
above makefile modification, you can. Otherwise, you can simply close
the ticket.

Thank you very much.
kid51
James Keenan via RT
2009-04-22 02:31:48 UTC
Permalink
There is no more tools/dev/ops_renum.mak. Can we close this ticket?
The only reason I did not previously close it was Coke's expression of a
desire to pull this into the main Makefile. But I'll let him open a TT
for that. Marking ticket resolved.

bacek, thanks for poking on this.

kid51

Loading...