(j3.2006) Inter-program communication using I/O statements

[Van Snyder]

We've been discussing how to prevent optimizers from moving stuff across
MPI_wait calls.

I suggested that a better long-term strategy is to provide hooks so
programs can intercept I/O at a connection-by-connection level, not a
type-by-type level that the current DIO supports, and then use the
existing machinery for ASYNCHRONOUS.

I started working through the details of this.  It wouldn't be a huge
project, but it's too big to get into the current standard.  Then I had
what looks to be a better idea.

When I run several programs, even different ones, on my SMP Linux
workstation, they can communicate through fifo's.  Unfortunately, this
doesn't work from one workstation to another, even with a shared file
system.  I didn't log into one of our clusters to test whether it would
work from one node to another, but I assume it wouldn't.

This brings to mind, however, another possibility:

Processors could interpret the FILE= specifier in an OPEN statement and
decide to open a connection that would work between programs on a SMP
computer (maybe using a FIFO), or node-to-node in a cluster (maybe using
MPI), or between workstations in Chile and Antarctica (maybe using
SMTP).  The processor could use whatever transport method it liked,
perhaps choosing a different transport method depending upon the
relationship between the guy doing the opening and what is being opened.
Some processors might let the user control which transport is selected
using an environment variable.  Others might encode it in the FILE=
string, say "http://192.168.50.250:7237".

If the FILE= specifier isn't good enough, the standard could provide
another specifier in the OPEN statement that is for sending extra info
to the RTL, with no defined semantics.  Its value should be a character
string.  Processors could do what they want with it, including setting
up broadcasts.

Assuming one has a processor-dependent routine to form a cluster, one
could do something like

  character(len=:), allocatable :: Cluster(:)
  integer :: WhoAmI ! Index of me in Cluster after forming the cluster
  call form_cluster ( cluster, whoAmI )
  do concurrent ( integer :: i = 1, size(cluster) )
    open ( 100+i, file=cluster(whoAmI), &
         &        action=merge('read ', 'write', i==whoAmI) )
  end do

Then Q can communicate with P using I/O statements for unit 100+P,
including unformatted asynchronous I/O.

This seems to be much easier than MPI, especially when one needs to send
a structure.  It also seems simpler than worrying about how to cheat the
optimizer.  The machinery we put in place for ASYNCHRONOUS already does
that, so we shouldn't need to invent another way to do it.

-- 
Van Snyder                    |  What fraction of Americans believe 
Van.Snyder at jpl.nasa.gov       |  Wrestling is real and NASA is fake?
Any alleged opinions are my own and have not been approved or
disapproved by JPL, CalTech, NASA, the President, or anybody else.