(j3.2006) A short comment on a small part of 17-201r1

[Bill Long]

> On Oct 17, 2017, at 8:45 PM, Malcolm Cohen <malcolm at nag-j.co.jp> wrote:
> 
> 201r1 states:
>  
>    In particular, the image index
>    when executing in the team is different for every image. Because
>    NEW_INDEX= is permitted in FORM TEAM, there is no simple way to
>    compute the image index based on other data in the team variable,
>    short of including a full translation table in each images' team
>    variable value. Because a full translation table uses an unscalable
>    amount of memory, the image index will almost certainly be included in
>   the data that makes up the team variable value for each image.
>  
> The first sentence is empty or uninteresting; of course every image has its own unique image index.
>  
> The second sentence is misstated badly.  The image index determined

The image index OF this image is computed by FORM TEAM (or assigned by PMI at startup for the initial team).  That is the value that needs to be stored locally because the scheme for recomputing it on the fly is not practical. 


> by a set of cosubscripts is calculated based ONLY on the cosubscript values and cobounds.  So this must be talking about something different, like perhaps how to calculate the ?global address? of an index ? which depending on the transport mechanism could be anything from a memory offset (in a pure shared-memory machine) to a process number (on a distributed-memory machine with a single o.s. running the whole set) to an IP address.

Or, more generally, the virtual PE number that is returned by the PMI libraries at program launch.  You would likely store your own PE number somewhere in the runtime space, though maybe not in the team variable, since it is the same value no matter what team you are executing in.  However, computing the PE number for a given image in your team given its image index would also be needed.  For a simple mapping by FORM TEAM (or the very likely simple on for the initial team) the translation can be done with a formula.  For a general reassignment of images, you would probably resort to a table. But not in the normal case, since tables can get large.
>  
> However, so there is going to be some kind of translation table which turns the image index into whatever handle is being used to do the communication to that image.
>  
> It is *obviously incorrect* to state that this translation table is necessarily not going to be present on each image in a team.  On the contrary, how else are you going to do a coarray access to another image in the team ? are you going to poll each member of the team to find out which one is, say, image 77?  How do you even known what images are in the team without the table?  Or maybe you?re planning to broadcast to every image whether it?s in the team or no?  It is far from obvious that those techniques, which waste bandwidth and increase latency, are superior to having a table to map the image index to the ?handle?.

None of this is specific to teams.  You would have the same need in the initial team, in which you could execute the whole program. 
>  
> There are legitimate cases for having some information in the team variable that differs from one image to the next, such as handles to use for balanced synchronisations,

Yes. The information of where you are in the ?reduction tree? is different for each image. We discussed this, but the simpler example of the image index for the executing image seemed sufficient. 

> but the basic info for accessing coarrays on other images in the team is not such a case.  (And the cases I?m aware of offhand are in general re-calculatable anyway, so if the user uses the ?wrong? copy of the team variable, doing that would be a small performance hit, not a major implementation obstacle.)

The image-to-PE conversion is not the issue here.  But having the wrong reduction tree information (particularly the same values as another image) could be a disaster, leading to a hang. 

Cheers,
Bill


>  
> Now maybe I?m misunderstanding what information the paper is talking about, but on the face of it the paper is simply wrong in this basic claim.
>  
> Cheers,
> --
> .....................Malcolm.
>  
> _______________________________________________
> J3 mailing list
> J3 at mailman.j3-fortran.org
> http://mailman.j3-fortran.org/mailman/listinfo/j3

Bill Long                                                                       longb at cray.com
Principal Engineer, Fortran Technical Support &   voice:  651-605-9024
Bioinformatics Software Development                      fax:  651-605-9143
Cray Inc./ 2131 Lindau Lane/  Suite 1000/  Bloomington, MN  55425