(j3.2006) Lessons to be learned from HPF

[Craig Rasmussen]

On Jul 17, 2007, at 5:11 PM, Bill Long wrote:

>
>
> Craig Rasmussen wrote:
>>
>> To start off, I'll define a couple of terms and then present one
>> possible global view of a co-array declaration.  The definitions have
>> been borrowed from the very thorough paper, "An Approach to Data
>> Distributions in Chapel" by Diaconescu and Zima.
>>
>>
>
> Chapel's domain concept could be a basis for enhancements to  
> Fortran in
> the future.  However, I'd like to see some practical experience first.
> HPF jumped into this area and hung itself with a bad design.  Domains
> would layer on top of our current arrays and co-arrays.  Nothing is  
> lost
> by moving ahead with the current co-array design and then  
> integrating a
> coherent domain feature in the future.  There is certainly not time to
> start this (very large) feature for f08.
>

I agree with Bill here.  We should learn from Chapel and HPF.  While  
major changes should wait until later, it is absolutely critical that  
we think now about how a "global view" of arrays would look, so that  
it can be added later in a natural manner without looking like a  
hack.  I'm particular worried about the co-array collectives as they  
now stand (but more about that later).

I spend two days last week at Cray talking to the Chapel team (Cray's  
answer to why parallel programming is so difficult and how it can be  
helped with a high-level parallel programming language).  I was  
particularly interested in what Fortran could learn from the Chapel  
experience and what we might steal for the future.

Why did HPF fail?  Suggestions from Cray's Chapel team [and my  
comments]:

1. High performance not delivered quickly enough?
    [This won't be a problem on Cray hardware, but may be a problem  
on more common clusters, when compared to MPI.]

2. Lack of a portable performance/execution model and semantics?
    [Not a problem for co-arrays, as the execution model is well  
defined and very evident (maybe too much so) to the programmer.]

3. Inability to drop to lower levels of implementation?
   [Co-arrays are the low level ("local array view") so this is not a  
problem.  Hopefully we can add a "global array view" is a natural  
way.  From my examination of data parallel constructs (where, shifts,  
array notation, ...) the combination of low and high level should be  
extremely powerful.]

4. Lack of rich abstractions (sparse arrays, irregular data structures)?
   [This is both a problem and not a problem for co-arrays.  Co- 
arrays don't have the rich abstractions but they do allow the  
programmer to build them from low-level constructs.]

5. Lack of supporting tools (debuggers, performance analysis tools)?
   [Debuggers?  Will TotalView work with co-arrays?  Shouldn't be a  
problem given a local co-array can be treated like an array.]

6. Finite # of distributions, no ability for user to define their own?
   [This is where co-arrays is lacking; they have none.  To disagree  
with Bill, I think we could add one (standard block distribution to  
work with shift intrinsics and matmult) without difficulty and leave  
more complicated and user defined ones for the future.  Note that the  
problem with HPF is not that they had distributions, it was that they  
didn't have enough.]

7. Single level of data parallelism too restrictive?
   [I'm not sure what this means but I assume co-arrays will suffer  
the same problem.  If it means a flat image hierarchy, then co-arrays  
do have that problem.  We are already seeing machines at LANL deeper  
memory and processor hierarchies (RoadRunner).]

Regards,
Craig