[J3] Questions about DO CONCURRENT and locality

Fri Jul 3 20:23:10 EDT 2020

I am not sure of the question here.
If the question is "can all DO
CONCURRENT loops be
parallelized?", the answer is
clearly "no".  The Fortran 2008
and 2018 standards specify that
the iterations of a DO
CONCURRENT loop can be
executed in any order.  They do
not specify that the
executions of the iterations of
a DO CONCURRENT loop can
be intermingled.  Each iteration
is executed to completion
before the next iteration begins.

The "as if" rule allows a loop to
be parallelized if the result is the
same as if it were executed
sequentially.  The rule applies to
all loops, not just DO
CONCURRENT loops.  A
DO CONCURRENT loop allows a
processor to make assumptions
about a loop that make it more
likely to be parallelizable, but it
does not ensure that the loop is
parallelizable.

I sympathize with those who
think DO CONCURRENT loops
are intended to be parallelizable.
I thought so too before I joined
the committee.

Robert Corbett

> On Jul 3, 2020, at 4:26 PM, Steve Lionel via J3 <j3 at mailman.j3-fortran.org> wrote:
> 
> Ok - does the programmer adding DEFAULT(NONE) address Peter's issues, or is there more to be done? If it does address them, is it worth adding a note, along the lines of what Daniel wrote, to the section on DO CONCURRENT to help the reader?
> 
> And can someone tell me if anyone wrote to Peter (who it seems was not at m218) explaining this?
> 
> Steve
> 
>> On 7/3/2020 5:49 PM, Daniel C Chen via J3 wrote:
>> Bill is right. If users specify DEFAULT(NONE), they are required to specify locality for each variable that 
>> appears in the DO CONCURRENT construct. 
>> 
>> As long as all the variables that appears in the DO CONCURRENT construct have explicit locality,
>> the processor doesn't need to exhaust the algorithm to figure out which one is shared/local,
>> there is no problem with parallelisation and extra impact to the performance. 
>> Otherwise, the original problem remains.
>> 
>> Daniel
>> 
>> XL Fortran Development, Fortran Standard Representative
>> IBM Toronto Software Lab
>> Phone: 905-413-3056 
>> Tie: 969-3056 
>> Email: cdchen at ca.ibm.com
>> http://www.ibm.com/software/awdtools/fortran/xlfortran
>> 
>> <graycol.gif>Bill Long via J3 ---2020-07-03 04:41:14 PM---> On Jul 3, 2020, at 2:59 PM, Daniel C Chen via J3 <j3 at mailman.j3-fortran.org> wrote: >
>> 
>> From: Bill Long via J3 <j3 at mailman.j3-fortran.org>
>> To: General J3 interest list <j3 at mailman.j3-fortran.org>
>> Cc: Bill Long <longb at cray.com>
>> Date: 2020-07-03 04:41 PM
>> Subject: [EXTERNAL] Re: [J3] Questions about DO CONCURRENT and locality
>> Sent by: "J3" <j3-bounces at mailman.j3-fortran.org>
>> 
>> 
>> 
>> 
>> 
>> > On Jul 3, 2020, at 2:59 PM, Daniel C Chen via J3 <j3 at mailman.j3-fortran.org> wrote:
>> > 
>> > This is a known issue that was first discussed in https://j3-fortran.org/doc/year/15/15-150.txt  
>> > The problem is that the wording in F2008 makes it almost impossible for the processor to 
>> > determine if an entity that appears in a DO CONCURRENT construct is shared or local. 
>> > I said "almost" because there is a way to implement it that requires a lot of 
>> > book keeping and copy-in/copy-out, which pretty much destroy the performance
>> > this feature is supposed to bring.
>> 
>> Right.  The main problem comes when a variable definition is protected by a condition, usually an IF statement.  It is difficult to tell whether this constitutes a previous definition before a subsequent reference outside the IF construct. But the 2008 rules, determination of local versus shared depended on such ordering of definition/reference.  This is part of why we have locality specs now.   The F2008 implementation was substantially complicated because of this. 
>> > 
>> > This finding is also the reason we added localities for DO CONCURRENT in F2018 to
>> > allow users to specify the locality by themselves.
>> > However, due to the concerns of backward compatibility, F2018 didn't add "default"
>> > locality even though it was requested at the time. 
>> > As the result, there is still a hole in the standard that makes 
>> > determination of a SHARED variable very difficult. It will completely kill the performance
>> > if a processor has to implement it). 
>> 
>> If I recall the discussion, allowing DEFAULT(SHARED) would basically force the user to declare all of the local  variables as LOCAL or LOCAL_INIT to have the loop make logical sense.   Similarly for DEFAULT(LOCAL). Providing only DEFAULT(NONE) as a mechanism for requiring declaration of ALL the variables seemed less error-prone.  (OpenMP does allow DEFAULT(SHARED) and includes a long list of reasons that cause a variable to be automatically private.  Which is probably broken.)
>> 
>> Cheers,
>> Bill
>> 
>> 
>> 
>> > 
>> > Daniel
>> > 
>> > XL Fortran Development, Fortran Standard Representative
>> > IBM Toronto Software Lab
>> > Phone: 905-413-3056 
>> > Tie: 969-3056 
>> > Email: cdchen at ca.ibm.com
>> > http://www.ibm.com/software/awdtools/fortran/xlfortran
>> > 
>> > <graycol.gif>Steve Lionel via J3 ---2020-07-03 03:31:48 PM---In https://j3-fortran.org/doc/year/19/19-134.txt 
>> > 
>> > From: Steve Lionel via J3 <j3 at mailman.j3-fortran.org>
>> > To: fortran standards email list for J3 <j3 at mailman.j3-fortran.org>
>> > Cc: Steve Lionel <steve at stevelionel.com>
>> > Date: 2020-07-03 03:31 PM
>> > Subject: [EXTERNAL] [J3] Questions about DO CONCURRENT and locality
>> > Sent by: "J3" <j3-bounces at mailman.j3-fortran.org>
>> > 
>> > 
>> > 
>> > 
>> > In https://j3-fortran.org/doc/year/19/19-134.txt   Peter Klausler asked 
>> > whether the words in the standard actually enabled parallelization of DO 
>> > CONCURRENT. I see that the paper was not taken up at 218 but I wasn't 
>> > aware of any discussion of the topic then or at later meetings. It came 
>> > up again today in discussions at the (online) FortranCon hosted by the 
>> > University of Zurich (which has been great so far - almost over.)
>> > 
>> > As I don't pretend to be a parallelization expert, I'd be interested in 
>> > hearing others' thoughts on the issues Peter raised. Thanks.
>> > 
>> > Steve
>> > 
>> > 
>> > 
>> > 
>> 
>> Bill Long                                                                                 longb at cray.com
>> Principal Engineer, Fortran Technical Support &   voice:  651-605-9024
>> Bioinformatics Software Development                      fax:  651-605-9143
>> Cray, a Hewlett Packard Enterprise company/ 2131 Lindau Lane/  Suite 1000/  Bloomington, MN  55425
>> 
>> 
>> 
>> 
>> 
>> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.j3-fortran.org/pipermail/j3/attachments/20200703/7d5bada7/attachment.htm>