[J3] [EXTERNAL] Re: Custom Memory Allocator

[Reuben D. Budiardja]

Hi Steve,

On 11/15/2019 03:37 PM, Steve Lionel via J3 wrote:
> No, there is no way to do this and if you try playing games with the 
> descriptor you may be in for big trouble later when the Fortran run-time 
> wants to deallocate the memory, as it isn't using the C descriptor for that.

Well, presumably whatever custom allocator will also have a deallocator 
to deallocate, so it won't be using Fortran runtime. But yes, care must 
be taken so that it doesn't get to the point that the runtime needs to 
deallocate.

But out of curiosity, I am not sure what you meant by "it isn't using C 
descriptor", would a Fortran runtime not also update the C descriptor 
consistently when it allocate / deallocate, and using the same 
information that's there (e.g. the base address)?

> My advice is that if you want to use a custom allocator, use POINTER and 
> CFI_establish (or C_F_POINTER on the Fortran side.) Admittedly you will 
> be giving up some of the advantages of allocatables.

Yes, using C_F_POINTER where the C_PTR points to the address allocated 
by the custom allocator is what we are doing. But we do want to avoid 
giving up the advantages of allocatable. And some OpenMP runtime 
surprisingly works better with Fortran allocatable vs pointer. Hence the 
question.

I'd very much like to avoid playing game in the non-standard way. Thus 
my wondering if a proposal for a mechanism to do this in the standard 
would be a good idea.

Thanks,
Reuben

> Steve
> 
> On Fri, Nov 15, 2019 at 2:30 PM Reuben D. Budiardja via J3 
> <j3 at mailman.j3-fortran.org <mailto:j3 at mailman.j3-fortran.org>> wrote:
> 
>     Hello,
> 
>     I am wondering if there is a standard-conformance way to allocate
>     allocatable array with a custom memory allocator, e.g. via third-party
>     provided C-library.
> 
>     At first I looked into passing the allocatable into a bind(c)
>     subroutine
>     and use the C descriptor from the C side. But sec 18.6 and 18.7 prevent
>     using any custom allocator other than using CFI_allocate().
> 
>     Real-world use case examples: we need to use CUDA memory allocator
>     cudaMallocManaged() to use CUDA Unified Memory, and cudaHostAlloc() to
>     use page-locked memory on the system.
> 
>     I suppose the hack-ish way is to swap the base_address member of C
>     descriptor to the address given cudaMallocManaged() (and free-up the
>     original address from CFI_allocate()).
> 
>     I wondered if for further enhancement on C interoperability we may
>     consider allowing custom allocator into the standard.
> 
>     I'd be interested in any feedback.
> 
>     Best,
>     Reuben
>