(j3.2006) Another application for coroutines

Van Snyder van.snyder
Mon Feb 1 04:29:31 EST 2010


Craig Dedo wrote:
>
> Van and Everyone Else:
>
>             Unfortunately for Van, this particular situation does not 
> seem to make a strong argument for coroutines.  To me, it does not 
> seem worthwhile to develop a complex new feature when good quality 
> solutions are already available, using features that are already in 
> the language.
>

Perhaps Craig didn't read the first word of the subject.

>             It seems to me that you have already hit upon the best 
> solution.  Use allocatable arrays and put them into a module.  If the 
> program is not straining the memory resources of the computer you are 
> using, then also use the SAVE attribute.  In order to avoid the 
> problem of mis-matched allocations and deallocations, write a 
> procedure to do all of the allocations at once and a second procedure 
> to do all of the deallocations at once.  Properly written, they should 
> be mirror images of each other and it should be easy to compare 
> whether an array is mentioned in one and not in the other.
>

I'm not as dense as Craig's explanation suggests.  I have two choices:  
(1) use the SAVE attribute if the same arrays are to be used for 
hundreds of invocations with different path lengths, or (2) put the 
variables in a subroutine with internal subroutines, and put the loop 
over paths in the outer subroutine.

Automatic variables in a coroutine have an implicit "save until return, 
even if suspend intervenes" attribute, which is exactly what I need.

>  
>
>             Yes, a procedure of 3784 source lines of code (SLOC) is 
> overdue for refactoring.  In fact, a procedure of this length is 
> almost begging for the presence of several (or many) serious defects.  
> The best research shows that the defect rate (i.e., # defects / KSLOC) 
> is constant below 200 executable statements per procedure.  The defect 
> rate increases linearly above 200 executable statements per 
> procedure.  Thus, a 1 KSLOC procedure has a defect rate 5 times that 
> of a 200 SLOC procedure, a 2 KSLOC procedure 10 times, and a 3.8 KSLOC 
> procedure 19 times.
>
>  
>
>             So for this reason alone, refactoring is very strongly 
> recommended.
>

I'm not as dense as Craig suggests.  It is already factored into dozens 
of internal subroutines.  At least part of the problem is that  the 
module procedure is so big.

>  
>
>             Then there is the issue of human understandability.  A 
> procedure of 3784 SLOC is highly unlikely to be understood by mere 
> mortals.  Even a super-genius would have a hard time understanding all 
> of it.  The best research shows that understanding is at its best at 
> 200 SLOC or less.  As length increases over 200 SLOC, understanding 
> goes down.  You might notice a similarity with what I wrote 2 
> paragraphs back.  Yes, understanding and defect rates are inversely 
> and strongly correlated.  As understanding goes down, defect rates go up.
>

Automatic arrays require only declaration.  My practice is to declare 
each one in a separate statement with a comment about its usage.  
Explicit allocation and deallocation requires at least two more 
statements.  In our case, we have a department standard that the status 
of every allocation and deallocation is checked.  This either requires 
another statement per allocation and deallocation, or encapsulating the 
allocation and checking in one routine, and the deallocation and 
checking in another one.  I can't (yet) use the distinction between 
allocatable and pointer attributes for generic resolution, so I need two 
sets of generics, one for pointers and one for allocatables.  Not using 
automatic variables increases the bulk of the part of the program that 
creates these variables by a factor of three or five.

>             There is a second understandability issue.  You mention an 
> ?already-long calling sequence?.  This suggests that you have a very 
> lengthy argument list.  If there are more than 7 arguments in the 
> argument list, then understanding will also go down.  This is because 
> human short-term memory is limited to around 7 items, i.e., the 
> average person can keep track of 7 items at one time.  Above that, 
> confusion starts to set in and understanding starts to degrade.  
> Another good place for using module data.
>

OK, I have mixing ratio, temperature, absorption cross section per 
molecule, total absorption cross section, incremental optical depth, 
transmissivity, the derivatives of each of these things with respect to 
mixing ratio, temperature, spectral line centers, spectral line widths, 
sensitivity of spectral line widths with respect to temperature, ..., 
all of this on a coarse grid (for rectangular quadrature) and a fine 
grid (for Gaussian quadrature).  Add to that arrays that specify where 
Gaussian quadrature is needed instead of where rectangular quadrature is 
needed, and the magnetic field.  That's just the arrays that depend upon 
path length.  Then there are the frequencies, the filter shapes, the 
antenna patterns, which chemical species are modeled line-by-line and 
which ones channel-by-channel, the pressure grid, the relationship 
between pressure and height (so it doesn't have to be recalculated from 
temperature and pressure using hydrostatic equilibrium on every 
invocation), and the output radiances and Jacobians, convolved with the 
filter shapes and antenna patterns.  This list is from memory, so I'm 
sure there are more.  Anyway, it's way beyond seven, without much hope 
of reducing the list below seven.

Some problems are inherently complicated.

>  
>
>             Hope this helps.  Please let me know how things work out.  
> Please feel free to contact me at any time with any questions or 
> concerns that you may have.  I am looking forward to hearing from you 
> soon.
>

It was no help whatsoever.

>  
>
> Sincerely,
>
> *Craig T. Dedo*
>
> 17130 W. Burleigh Place
>
> P. O. Box 423                         Mobile Phone:  (414) 412-5869
>
> Brookfield, WI   53008-0423    E-mail:  <craig at ctdedo.com 
> <mailto:craig at ctdedo.com>>
>
> USA
>
> Linked-In:  http://www.linkedin.com/in/craigdedo
>
>  
>
> -----Original Message-----
> From: j3-bounces at j3-fortran.org [mailto:j3-bounces at j3-fortran.org] On 
> Behalf Of Van Snyder
> Sent: Friday, January 29, 2010 15:17
> To: j3
> Subject: (j3.2006) Another application for coroutines
>
>  
>
> During the 2008 requirements phase, I advocated coroutines to ease 
> "reverse communication" in library codes that need access to user code.
>
>  
>
> I've recently encountered another circumstance where a coroutine would 
> be helpful.
>
>  
>
> The program of my current responsibility integrates the radiative 
> transfer equation through the limb of the Earth's atmosphere, from 
> space to our instrument.  The instrument's antenna points at about 70 
> angles over a period of about 26 seconds.  Therefore, each path 
> through the atmosphere is of a different length.
>
>  
>
> The procedure that does the integration has about three dozen arrays 
> with at least one dimension that depends upon the path length, and 
> upon the discretization of the path (which is determined by input data).
>
>  
>
> To avoid a trip to the allocator (and deallocator) for each pointing, 
> we allocate these arrays once for each scan, for the path of longest 
> length, as automatic arrays.
>
>  
>
> For a variety of reasons, only a few having to do with these arrays, 
> this subroutine has become enormous (3784 lines).  I'd like to break 
> it up.  So as not to add three dozen arrays to an already-long calling 
> sequence, I'd like to continue to access them either locally or 
> through host association.
>
>  
>
> An obvious way to handle this is to convert them from automatic to 
> allocatable, either put them in a module or make them save variables, 
> and add the appropriate explicit allocations and deallocations.  This 
> risks failing to deallocate a new one someday, if I have to add one 
> (which I've had to do several times).
>
>  
>
> One advantage of automatic variables, compared to allocatable ones, is 
> that you can't forget to deallocate them (which I would have to do to 
> get rid of them if they're save variables).  Another potential 
> advantage is that the processor might make one trip to the allocator 
> to get them all in one gulp, then calculate its descriptors.  I don't 
> know whether this actually would be an advantage, or if so whether any 
> processor exploits it.
>
>  
>
> If these variables were automatic variables in a coroutine, it could 
> be called to get them created and to do some preliminary calculations 
> necessary for all paths.  Then it would suspend instead of returning, 
> thereby preserving its activation record.  It would be resumed for 
> each path, and would finally return after the last path, at which time 
> the arrays would be deallocated.
>
>  
>
> _______________________________________________
>
> J3 mailing list
>
> J3 at j3-fortran.org
>
> http://j3-fortran.org/mailman/listinfo/j3
>
>  
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://j3-fortran.org/pipermail/j3/attachments/20100201/c9ca4ecd/attachment-0001.htm>



More information about the J3 mailing list