(j3.2006) (j3.2005) Re: question on token replacement/concatenation
Tue Jan 23 05:04:05 EST 2007
>>>> 3 %% .14159 %% E %% + %% 0
>> I've asked this before, but I don't remember the answer. What sort of
>> lexical token is the "E"? It doesn't seem to match anything in the list.
> Before macro token concatenation, it's a name.
> <letter> [ <digit> | <letter> | _ ]...
> After macro token concatenation, the 5 tokens ( "digits", "decimal",
> "name", "operator", "digits" ) combine to form one token, so the "E"
> no longer maps to a whole token but is merely one character in a token.
Dick Hendrickson said:
> OK, but this sort of limits how we can use the %% to build up things.
> The syntax for "names", and the other lexical tokens, is fairly
> restrictive. The two cases I think might be hard to do are
> building up structure names and adding kinds to constants.
> Suppose I want to build up component names, so that sometimes I
> refer to A%B and sometimes to A. My niave attempt would be
> to get A, I'd invoke the macro with p1 as a null macro argument
> for A%B, I'd try to set p1 to %B.
> But, I can't do that because %B isn't a name, nor any other
> lexical token.
There does not seem to be a problem here.
You are not trying to build up a token: A%B is three tokens, not one,
so in any case you do not want to concatenate anything.
In other words, just
does what you are trying to do with A%%p1.
> Similarly for constants, if I sometimes want to postfix a kind parameter
> I'd try
> where "kind" would sometimes be null and other times be _4 or
> _workingprecision. Again, I can't do this.
Ugh. Yes, we have a problem here, but the problem is because of the
problematic way we have defined 3.14_WP as being one token instead of
three. Tokens are meant to be the basic lexical building blocks - and
this just isn't (WP has to be a name, with associated semantics etc.,
which means we're treating it as a token in its own right).
Actually, I had spotted this before (many years before) and it has been
somewhat of a sore point with users who want to write things like
3.14 _ WP
for readability reasons (in fact the NAG compiler used to allow that until
someone else complained that we didn't detect the spaces inside the "token"
> So, is it worth trying to add some sort of "except leading underscores
> and %s are allowed in the names" condition to the concatenation
> rules? Or are these cases too small to worry about this time?
> Or, have I misunderstand something?
W.r.t. 3.14_WP, I think that the right fix is to correct our faulty
definition of a token. If we want that to have no "user effect" (i.e.
the compiler writers not to have to change anything) we can forbid
blanks in between the tokens of a literal constant. Seems pretty
straightforward to do. (If the other vendors want to allow blanks
around the underscore that's fine by me too.)
We could call a literal constant without a kind parameter an "unkinded"
literal constant... we already use them quite a lot in i/o for example.
........................Malcolm Cohen (malcolm at nag-j.co.jp), Nihon NAG, Tokyo.
More information about the J3