(j3.2006) (SC22WG5.3656) [ukfortran] FW: Notes on WG5 coarray papers

[N.M. Maclaren]

This isn't a full response, and doesn't represent the BSI, but I am
sending my initial reactions to a selection of points; Most of them
would be easier to discuss face-to-face, as they are not simple.



> 1.2 VOLATILE coarrays (8.5.1, para6):
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
>  
> ---------------------------------------------------------------------------- 
> NOTE 8.29a A scalar coarray that is volatile and of type default integer, 
> default real, or default logical is 'atomic' in the sense that read 
> accesses from any other than those altering its value will obtain either 
> the value previous to an alteration, or the value after an alteration. It 
> remains the programmer's responsibility to prevent race conditions for 
> such volatile coarray references by suitable formulation of the algorithm 
> (ref. to example in Note 8.38). 
> ----------------------------------------------------------------------------

This NOTE goes to the heart of the problem.  There are two separate
concepts involved, corresponding to your two sentences:

    Whether accesses are atomic.  The current wording isn't enough,
because arrays are not a separate type from their elements in Fortran
(unlike modern languages).  However, that's the 'easy' one :-(

    Whether volatile coarray accesses cause any ordering, beyond that
caused by normal object accesses and image control statements.  And
that's the killer.

Roughly, in current Fortran, accesses are grouped into totally ordered
sets, in the mathematical sense.  End of story.  The sets are those that
have no ordering defined by the language (e.g. those within an
expression).  That fails as soon as we introduce parallelism, where
there can be at most a partial ordering and more usually only a
pre-ordering.

Fortran COULD say that the behaviour is AS IF there is single execution
order, but that is sequential consistency.  That is foul to implement on
most systems and inefficient on all large ones, for non-handshaking accesses
like VOLATILE coarrays.  Neither Intel nor AMD guarantee it even at the
hardware level, so it wouldn't be reliable even on them unless special
action was taken.

And, currently, it doesn't.  It doesn't say anything.

> In any case, we have here an extension of the semantics of VOLATILE for 
> the indicated coarray objects, as compared to the original definition in 
> 1.1. This additional property is needed for the code in Note 8.38 to 
> work. Note that formally the VOLATILE attribute is only exploited on 
> image Q in that example. Due to the additional semantics, a reliable 
> implementation for commodity clusters will probably incur an even larger 
> overhead than normal VOLATILE objects.

Yes, I agree - with one exception.  Where does it say that in normative
wording?


> 2. Comments on paper N1745:
> ===========================
> 
> In section 1, the author advocates three possibilities for a better
> approach. ...
> 
> The second one, atomic datatypes and operations, is in effect what is
> already there (see 1.2 above). ...

The issue is not the atomicity but the ordering, and Fortran defines
ordering only for statements, not accesses within statements. [ Please let's
not start the impure function 'debate' yet again! ]

> The VOLATILE attribute would then still be advantageous since it 
> provides the effect of an object-specific SYNC MEMORY, saving on overhead 
> if many other memory operations are outstanding (an additional SYNC 
> MEMORY would otherwise be required within the spin loop of Note 8.38!).

Grrk.  As they necessarily occur within an expression, that implies an
ordering of expression evaluation and object access within it, which
Fortran doesn't have, in general!

The point about a proper atomic interface is that it would use subroutines,
which can be invoked only as complete statements, and hence are ordered.

> In section 2, some examples are provided to illustrate inconsistencies
> in the standard.

As Malcolm says, arrays are just collections of scalars.  It would be
trivial to rewrite most of my array-using examples with purely scalar
access, and one of the facilities that don't imply ordering, such as DO
CONCURRENT.

> Example 2.4 ("Protected Context") is non-conforming since the cited
> restriction is violated by the object in question being VOLATILE within
> the scope of the DO loop. This would apply even if the CASE(1) statements
> were not present.

Malcolm has addressed this, but there is an extra wrinkle.  Is there a
difference in meaning between "be redefined" and "become redefined"?  In
formal English, there is - but, in the Fortran standard?  It makes a BIG
difference for VOLATILE.

> Finally, the reference to the C++ standardization efforts with respect
> to the memory model (actually at the end of N1744) appears to be relevant
> for shared memory processing, but not necessarily for segmented memory
> processing.

I can assure you that it is!



> 3. Concerning Paper N1744:
> ==========================
> 
> Section 1 (Sequential Consistency):
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> The author seems to believe that in absence of the user defining a 
> suitable synchronization sequence the processor should impose one. This 
> appears contrary to the spirit of coarray programming which intends to 
> minimize the constraints as far as possible to achieve improved 
> performance.

Absolutely NOT!  My belief is that the standard should specify, clearly
and explicitly, what ordering properties are defined, and what ones are not.
N1744 is asking for clarification of points on which there is scope for more
than one reasonable interpretation.

Actually, it is YOU who are assuming it!  Unless I am much mistaken,
sequential consistency is mathematically equivalent to there being a
partial ordering.  And, if your interpretation of SYNC MEMORY is correct,
then segments are partially ordered.  I have no problem with this!


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679