[Coarray-ts] Teams

John Reid John.Reid at stfc.ac.uk
Tue May 29 11:20:39 EDT 2012


Dear all,

There are now less than 4 weeks to go before we meet in Toronto. We need 
to work on coarray requirements. J3 has made a start with 12-136r2, but 
more work is needed IMHO. Note that time was very limited at the Feb. J3 
meeting because of the need to give priority to the interop TS.

Let's start with teams. 12-136r2 says

"Teams provide a capability to restrict the image set of remote memory
references, coarray allocations, and synchronizations to a subset of
all the images of the program.  This simplifies code for some
applications that involve segregated activities (parts of a climate
model, for example). Teams also provide a mechanism for limiting
activity to a subset of the computer system that might result in
better performance within the team (such as within a local SMP
domain)."

I would like to expand this a bit. How's this?

"Teams provide a capability to restrict the image set of remote memory
references, coarray allocations, and synchronizations to a subset of
all the images of the program. This simplifies writing programs that
involve segregated activities (parts of a climate model, for example)
that might be more easily be written independently or may have
already been written as independent programs. Teams also provide
a mechanism for subdividing the computation for the sake of better
performance (such as within local SMP domains). Finally, teams provide
the capability to run routines (such as library routines) that have
developed and tested without any team syntax on a subset of the
images of a program."

The simple team feature described in N1858 is not adequate since the 
cosubscripts map onto the image indices of the set of all images rather 
than those of the team. When code that has been developed without teams 
is run on several teams, the cosubscripts will need to be changed for 
each team. We are led to this team requirement:

T1: When a subprogram without any team syntax is called on
     images executing as a team, it should execute on those images as
     if the program contained no other images. This has the following
     implications:
     1. Image indices shall be relative to the team, starting at 1 and
        ending with the number of images in the team.
     2. Collective activities, such as SYNC ALL, allocation and
        deallocation of coarrays, collective subroutine execution, and
        inquiry intrinsics such as THIS_IMAGE and NUM_IMAGES shall be
        relative to the team.

I think it is generally agreed that any team will have been formed as a 
part of a larger "parent" team. We need to consider whether there is a 
requirement for access to other variables of the parent. This could be 
done by naming each team and extending the coarray syntax in some way, 
such as
       array[i,j]@team
where the cosubscript mapping is that of the parent team. If we go this 
way, we would need to extend synchronization features, too. This was not 
part of the 12-136r2 proposal, which has these bullet points:

- An image is always a member of some team, and a member of only one
   team at a time.
- Access to variables on images outside the current team is not
   permitted.

This leads to these alternative team requirements

T2a: While an image executes a statement it shall be a member of one
     and only one team. Access to variables on images outside the
     current team is not permitted.

T2b: While an image executes a statement it shall be a full member of
     one and only one team, but also have access through new syntax to
     the data and synchronization features of its ancestor teams.

For the formation of teams, 12-136r2 has this bullet point:

- New teams can be formed with a new statement (SPLIT TEAM, or FORM
   TEAM) that defines the specified teams. The aggregate number of
   images in the teams shall equal the number of images in the current
   team. The new teams are composed of images with consecutive image
   numbers in the current team. A team variable cannot be defined other
   than by execution of the statement used to form teams.

The requirement "The new teams are composed of images with consecutive 
image numbers in the current team." seems too restrictive to me. It 
would be awkward for 2- and 3-d applications. I suggest this team 
requirement.

T3: It should be possible to split a team into mutually exclusive
     subsets that are themselves teams. This should be dynamic in
     order to allow different groupings of images during different
     stages of execution. It is desirable to have a
     compact mechanism for specifying a large number of teams each of
     the same number of images. Further mechanisms are not needed.

For changing teams, 12-136r2 has this bullet point:

- A construct is provided to specify a new current team for the
   executing image. Possibilities are a WITH TEAM ... END WITH TEAM
   construct, or a SELECT TEAM ... END SELECT construct. ....

which is rather too explicit for a requirement. How's this?

T4: There shall be a mechanism for changing the current team, involving
     the synchronization of all members of the old team and the
     synchronization of all members of the new team. It should be via a
     construct so that it is apparent (both to the system and the
     programmer) where team execution begins and ends.

To identify teams, how about

T5: There shall be a type for variables identifying teams (probably
     a derived type defined in the intrinsic module ISO_FORTRAN_ENV).

We also need:

T6: In order for the feature to coexist with MPI, there needs to be
     a mechanism to find the image index relative to the set of all
     images of the program.

This might best be done by adding an optional argument to IMAGE_INDEX
that specifies the team.

There is an issue with allocatable arrays that are allocated within a 
team. Can these be supported in symmetric memory or are they are like 
allocatable components of a coarray? They can be supported in memory 
that is symmetric for the team provided they are all deallocated when 
execution reverts to the parent team. So we have two alternative
requirements

T7a. It should be possible to support allocatable arrays that are 
allocated within a team in memory that is symmetric for the team.
T7b. It need not be possible to support allocatable arrays that are 
allocated within a team in memory that is symmetric for the team.


Cheers,

John.




More information about the Coarray-ts mailing list