(j3.2006) Comments to 10-166 (early coarray TR draft)

[Bill Long]

Hi Tobias,


On 4/1/11 11:58 AM, Tobias Burnus wrote:
> Hi all,
>
> admittedly, it is probably a bad timing as everyone is interested in TR
> 29113 and not other work items. However, I happened to have time to
> glance at 10-166 (draft of coarray TR, dated 2010/02/18).
>
> First, I spotted an "ALL STOP" which should be an ERROR STOP (in A.1.1).

This is due to the age of the paper.  ERROR STOP was originally ALL STOP.

>
> Secondly, I miss a possibility to broadcast values to all (or to a
> team); unless I have missed something even with TR one has still to do do:
>

I showed the TR proposals to our applications and benchmark teams and 
got a lot of (disapproving) feedback regarding the collective choices. 
Most of the ones listed were just extrapolations of current reduction 
routines in the standard.  The majority were deemed not worth the 
effort.  Based on the frequency of appearance of the related MPI 
reductions in actual codes, they asked for this list instead:

co_sum
co_bcast
co_min, co_max
co_reduce

where co_reduce is a general routine that has a procedure argument to 
specify the actual reduction operation.   Co_reduce would cover the few 
percent of actual reductions not covered by the other 4.  When we get to 
the stage of considering the Coarray TR, I intend to submit a paper 
arguing for this change.

>     if (this_image()==1) then
>       ! READ input file
>       ! Distribute values:
>       do image = 2, num_images()
>         z[image] = z
>       end do
>     end do
>     SYNC ALL
>
> (Or in the "IF" a "SYNC IMAGES(*)" and in ELSE a "SYNC IMAGES(1)"). I
> think sending the value to each other image, image by image, is rather
> slow if many images are involved. (Assume a calculation on 6k Blue Gene
> processors or using the full 294,912 processors of the HPC system 600
> metres from here.) On such systems, sending the configuration can then
> take a significant amount of the total computation time. That time is
> wasted especially as there is a dedicated collective network with
> one-to-all broadcast functionality.
>
> For reductions, the draft provides the most important ones. However, I
> see again some unneeded communication as: "A collective subroutine is
> one that is invoked on a team of images to perform a calculation on
> those images and which assigns the value of the result on all of them"
> (4.1.1). While that is often the desired result, one frequently needs
> the result only at one image. Coming again back to calculation on a

Indeed, this is also the feedback from the applications team.  All of 
the proposed replacement collectives above (except co_bcast) have an 
optional argument specifying the one image where the result is defined. 
  If the optional argument is not supplied, the result goes to all 
participating images.


> many-processor system: Doing the collective operations in a tree-like
> manner and sending it to a single reduction-master image is faster than
> collecting it on all systems - especially since there is a barrier (team
> synchronization) after the reduction, which could be avoided on all but
> the one image which is interested in the reduction.
>

Right. This is the same argument I got internally.

Cheers,
Bill




> Tobias
>
> PS: It would be nice if someone could save the three comments such that
> they can be discussed, when the topic comes up again after TR 29113.
>
> PPS: I hope I have found the latest draft.
> _______________________________________________
> J3 mailing list
> J3 at j3-fortran.org
> http://j3-fortran.org/mailman/listinfo/j3

-- 
Bill Long                                           longb at cray.com
Fortran Technical Support    &                 voice: 651-605-9024
Bioinformatics Software Development            fax:   651-605-9142
Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101