# (j3.2006) Integration of co-arrays with the intrinsic shift functions

Craig Rasmussen crasmussen
Thu Jul 12 19:16:07 EDT 2007

```On Jul 12, 2007, at 3:54 PM, Aleksandar Donev wrote:

> On Thursday 12 July 2007 13:15, Craig Rasmussen wrote:
>
>> I have identified at least one code at Los Alamos that implements
>> global shift functions (nearly identical to the co_cshift and
>> co_eoshift intrinsics proposed in the paper) in MPI.
> I haven't thought much about circular shifts. A comment that may be
> relevant:
> You say "Array variables (without the
> explicit index notation) naturally allow the programmer to think of a
> co-array as a global data structure with data-parallel operations
> applied across all co-array elements on all images."
>
> If one does want to look at co-arrays as a global structure (which
> we do not
> as far as the standard is concerned),

Hum, I used to want to think of them "as a" global structure (in
spite of the Fairfax agreement) but in terms of thinking about
integration with cshift, I now view them only as "parts" of a virtual
global structure.  I think this more faithful to the standard.

> they should be thought of as "arrays"
> of "global rank" that is the sum of the rank and co-rank (even this
> is a bit
> of a stretch as the arrays can have "holes"). So a co-array
> declared (2,3)[4]
> can be thought of as an array declared with (2,3,4). Any co_shift
> et al.
> operations should, imho, be consistent with such a view and the
> existing
> shift intrinsics.

I'm going to have to disagree with you here Aleks.

As a scientist/programmer/software architect, I start out with an N
dimensional array by design, and then that array gets distributed
across images, but the rank of the array doesn't change in doing
this.  So your co-array A(2,3)[4] is a virtual V(2,12) array.  If it
were declared A(2,3)[2,2], then it would be a virtual V(4,6) array.

I think this view is entirely consistent with Cray's Chapel design
regarding distributed domains.  The local is our image and the
distribution is the co-dimension specification [4] or [2,2].  In any
case, I'm visiting Cray next week to discuss these issues with the
Chapel team.

In considering this, I'm excited about the possibility that co-arrays
may turn out to be much more powerful than I had originally thought.
But we must absolutely get these ideas correct or co-arrays could be
forever crippled.

Regards,
Craig

```