(j3.2006) (SC22WG5.4032) Late in the day question

[Malcolm Cohen]

Robert Corbett wrote:
>
> In any case, the FORALL construct in
> Fortran 95 and FORTRAN 2003 is largely the same as that of HPF 2.0.
>   
HPF copied theirs from Fortran 8X, copied from CM Fortran IIRC.


FORALL as written makes sense when (very crudely) parallel evaluation 
trumps array temporaries.  This was at least semi-true for various 
massively parallel machines in the 1980s which had 100s or 1000s of 
nodes each of which with significant local storage; as long as the array 
temps fitted in the local storage you could get 1000-fold speedup over 
the DO loops.  (The array temps being the size of the iteration space, 
not the size of the data being worked on.)  Once the iteration space got 
too large though, this did not work so well.

Note that these machines had pretty good communication facilities for 
the time; in other distributed-memory systems the communications 
overheads are the limiting factor rather than the parallelising.  (Hmm, 
that's not expressed very well, what I'm trying to get across is that 
the focus on the normal distributed-memory system is on partitioning the 
data sets, which is maybe helped a bit by FORALL but not really enough.)

It's quite possible that FORALL might once again give good performance, 
if we get to systems with many hundreds of nodes each of which has 
significant local memory - and we have programs that can use its 
semantics, and that the parallel hardware is not yet maxed out by our 
other parallel facilities.  That's a lot of if's.

> In my work at Sun, I have had to explain what FORALL does to many users.
> A lot of them thought that FORALL was a parallel DO.
>
>   
Some of the users I ran into seemed to think that it was a normal DO 
"that ran faster" without any specific reason other than "it's a new 
feature, therefore it must run faster".  Mostly those users didn't even 
have dual processors so parallelism wasn't even on the map!

Cheers,
-- 
........................Malcolm Cohen, Nihon NAG, Tokyo.