(j3.2006) Integration of co-arrays with the intrinsic shift functions

[Craig Rasmussen]

On Jul 12, 2007, at 3:54 PM, Aleksandar Donev wrote:

> On Thursday 12 July 2007 13:15, Craig Rasmussen wrote:
>
>> I have identified at least one code at Los Alamos that implements
>> global shift functions (nearly identical to the co_cshift and
>> co_eoshift intrinsics proposed in the paper) in MPI.
> I haven't thought much about circular shifts. A comment that may be  
> relevant:
> You say "Array variables (without the
> explicit index notation) naturally allow the programmer to think of a
> co-array as a global data structure with data-parallel operations
> applied across all co-array elements on all images."
>
> If one does want to look at co-arrays as a global structure (which  
> we do not
> as far as the standard is concerned),

Hum, I used to want to think of them "as a" global structure (in  
spite of the Fairfax agreement) but in terms of thinking about  
integration with cshift, I now view them only as "parts" of a virtual  
global structure.  I think this more faithful to the standard.

> they should be thought of as "arrays"
> of "global rank" that is the sum of the rank and co-rank (even this  
> is a bit
> of a stretch as the arrays can have "holes"). So a co-array  
> declared (2,3)[4]
> can be thought of as an array declared with (2,3,4). Any co_shift  
> et al.
> operations should, imho, be consistent with such a view and the  
> existing
> shift intrinsics.

I'm going to have to disagree with you here Aleks.

As a scientist/programmer/software architect, I start out with an N  
dimensional array by design, and then that array gets distributed  
across images, but the rank of the array doesn't change in doing  
this.  So your co-array A(2,3)[4] is a virtual V(2,12) array.  If it  
were declared A(2,3)[2,2], then it would be a virtual V(4,6) array.

I think this view is entirely consistent with Cray's Chapel design  
regarding distributed domains.  The local is our image and the  
distribution is the co-dimension specification [4] or [2,2].  In any  
case, I'm visiting Cray next week to discuss these issues with the  
Chapel team.

In considering this, I'm excited about the possibility that co-arrays  
may turn out to be much more powerful than I had originally thought.   
But we must absolutely get these ideas correct or co-arrays could be  
forever crippled.

Regards,
Craig