(j3.2006) Reading C descriptors from different vendors
Bill Long
longb
Tue Nov 24 13:33:18 EST 2015
On Nov 24, 2015, at 10:52 AM, Rafik Zurob <rzurob at ca.ibm.com> wrote:
> Hi Bill
>
> I think the difference is that module symbol files are only needed at
> compilation time, not execution. Ideally, I think one would want to
Module information is needed at compile time and at link time (for the data and executable code in the module). The pgas library is needed only at link time. Unless you are willing to pay the performance price for dynamic linking. On top of the additional hit for using accessor functions to get component values.
> install one library package on the system that is used by every one. i.e.
> Don't ship a custom library in the compiler package. Instead, use a
The pgas library is just as much an integral part of the compiler as the library that contains the routines that implement READ and WRITE statements, or the intrinsic procedures. I don?t see why there should be different treatment.
> separate package that is independently distributed. I'm using
> OpenCoarrays only as an example here. In theory, there could be any
> number of future third party libraries in the same boat. It wouldn't be
> feasible to create custom versions of everything.
>
> As for MPI, if there was a way of making C descriptors interoperable with
> each other, one could ship a single MPI library per platform (e.g. Linux
> x86) along with the mpi module in source form. Vendors on that platform
> would then only have to compile the module file, e.g. at compiler
> installation or configuration time. My point here is that we're forcing
> vendors and third party libraries to customize for each compiler when we
> don't really have to.
The third-party libraries don?t customize anything at all, assuming the sources are available. If they are compiler-vendor-supported (something like hdf5, for example), then the vendor compiles it once for each supported compiler and ships the lib* file as part of the programming environment. If there is no vendor support, the user downloads and compiles the library. One assumes with optimization and the IPO hooks so that when the user builds a code that uses this library, all the expected optimizations happen.
Cheers,
Bill
>
> Regards
>
> Rafik
>
>
>
> From: Bill Long <longb at cray.com>
> To: fortran standards email list for J3 <j3 at mailman.j3-fortran.org>
> Date: 24/11/2015 08:30 AM
> Subject: Re: (j3.2006) Reading C descriptors from different vendors
> Sent by: j3-bounces at mailman.j3-fortran.org
>
>
>
> I guess I don?t see how this is any different from the ?problem" of not
> having a uniform format for Fortran module information. Vendors compile
> MPI libraries separately for each compiler they support (at least the
> parts that depend on vendor-specific memory layouts). I would assume the
> same would be true for OpenCoarrays. If you are using OpenCoarrays as the
> runtime for parallel processing with Fortran compiler X, then the
> opencoarrays library, pre-compiled by you, should be part of the compiler
> X package. If you compile the library with the companion compiler to X,
> there is no problem.
>
> Cheers,
> Bill
>
>
> On Nov 24, 2015, at 6:01 AM, Rafik Zurob <rzurob at ca.ibm.com> wrote:
>
>> Hello
>>
>> As you know, TS29113 does not specify the offsets of the rank, type,
>> attribute, or dim fields in the C descriptor. C descriptors are also
>> allowed to contain a vendor-specific area of unspecified length in the
>> middle of the descriptor. The descriptors also don't contain any field
> to
>> identify the vendor. This makes it impossible to convert between
>> descriptors from different vendors at run time. I raised this issue
> once
>> before when TS29113 was under development, but couldn't provide a strong
>
>> case for binary compatibility. As was pointed out to me back then, the
> TS
>> only provides source portability. I think I now have an example that
>> would benefit from better binary compatibility: Third party libraries
>> like OpenCoarrays have APIs that take descriptors. OpenCoarrays
> currently
>> takes gfortran descriptors but plans to move to C descriptors when
>> gfortran moves as well. For other vendors to be able to call the
>> OpenCoarray API, there needs to be a way of letting the API understand
> the
>> vendors' C descriptors. I think it's undesirable to hardcode for each
> API
>> what C descriptor format needs to be generated to call the API.
>>
>> I have several ideas on how to address this:
>>
>> 1. Add new functions to get the offsets of the fields whose offsets are
>> not specified by TS29113:
>> int CFI_rank_offset() that returns the offset in bytes of the
> rank
>> field relative to the start of the C descriptor.
>> int CFI_type_offset() that returns the offset in bytes of the
> type
>> field relative to the start of the C descriptor.
>> int CFI_attribute_offset() that returns the offset in bytes of
> the
>> attribute field relative to the start of the C descriptor.
>> int CFI_dim_offset() that returns the offset in bytes of the dim
>> array relative to the start of the C descriptor.
>>
>> 2. Add new functions to return pointers to the fields whose offsets are
>> not specified by TS29113:
>> CFI_rank_t *CFI_rank(CFI_desc_t *) that returns a pointer to the
>> rank field of a given C descriptor
>> CFI_type_t *CFI_type(CFI_desc_t *) that returns a pointer to the
>> type field of a given C descriptor
>> CFI_attribute_t *CFI_attribute(CFI_desc_t *) that returns a
>> pointer to the attribute field of a given C descriptor
>> CFI_dim_t *CFI_dim(CFI_desc_t *) that returns a pointer to the
> dim
>> field of a given C descriptor
>>
>> 3. Define a typedef CFI_min_cdesc_t that maps to a C descriptor with no
>> vendor-specific area. Provide a conversion function to convert from
>> CFI_cdesc_t to CFI_min_cdesc_t:
>> int CFI_get_min_cdesct_t(CFI_min_cdesc_t *, const CFI_cdesc_t *);
>>
>> 4. Go for broke and redesign the C descriptor: Specify the offsets of
> the
>> rank, type, and attribute fields. Add a new field (with known offset)
>> containing the size of the vendor-specific section. If the
>> vendor-specific section size is non-zero, the first 4 bytes of it must
> be
>> a four-character vendor identifier. Define a macro (with known value)
>> CFI_VERSION_F2015. C descriptors that follow this format must have a
>> version field with a value such that ((version & CFI_VERSION_F2015) ==
>> CFI_VERSION_F2015).
>>
>> 5. Other ideas?
>>
>> I believe we should address this in Fortran 2015 since it's the first
>> version to incorporate TS29113.
>>
>> Regards
>>
>> Rafik
>>
>> _______________________________________________
>> J3 mailing list
>> J3 at mailman.j3-fortran.org
>> http://mailman.j3-fortran.org/mailman/listinfo/j3
>
> Bill Long longb at cray.com
> Fortran Technical Support & voice:
> 651-605-9024
> Bioinformatics Software Development fax: 651-605-9142
> Cray Inc./ Cray Plaza, Suite 210/ 380 Jackson St./ St. Paul, MN 55101
>
>
> _______________________________________________
> J3 mailing list
> J3 at mailman.j3-fortran.org
> http://mailman.j3-fortran.org/mailman/listinfo/j3
>
>
>
>
> _______________________________________________
> J3 mailing list
> J3 at mailman.j3-fortran.org
> http://mailman.j3-fortran.org/mailman/listinfo/j3
Bill Long longb at cray.com
Fortran Technical Support & voice: 651-605-9024
Bioinformatics Software Development fax: 651-605-9142
Cray Inc./ Cray Plaza, Suite 210/ 380 Jackson St./ St. Paul, MN 55101
More information about the J3
mailing list