[J3] [EXTERNAL] [BULK] Re: Question to compiler developers about compilation speed

[Clune, Thomas L. (GSFC-6101)]

Again – one of the first things we tried was to place all of the source files and TMPDIR onto SSD and we saw very little change in performance.

This suggests that I/O is not the bottleneck, unless it is metadata contention on our shared filesystem.    I would think the latter would also be small because we are not talking about lots of files here in the grand scheme and metadata access ought to be 2-3 orders of magnitude faster than what we are seeing.   I suppose if the compiler is opening and closing the same file over and over again then metadata might be the culprit.



  *   Tom

From: J3 <j3-bounces at mailman.j3-fortran.org> on behalf of Damian Rouson via J3 <j3 at mailman.j3-fortran.org>
Date: Wednesday, October 30, 2024 at 11:12 AM
To: General J3 interest list <j3 at mailman.j3-fortran.org>
Cc: Damian Rouson <rouson at lbl.gov>
Subject: Re: [J3] [EXTERNAL] [BULK] Re: Question to compiler developers about compilation speed
CAUTION: This email originated from outside of NASA.  Please take care when clicking links or opening attachments.  Use the "Report Message" button to report suspicious messages to the NASA SOC.


This sounds like a task for a parallel profiling tool such as TAU. I wonder if anyone has done parallel performance analysis of a build system — could be an interesting paper.

As was mentioned already in this thread, I too wondered whether I/O might be the bottleneck with lots of small files.

D

On Wed, Oct 30, 2024 at 07:52 Thomas König via J3 <j3 at mailman.j3-fortran.org<mailto:j3 at mailman.j3-fortran.org>> wrote:
Am 30.10.24 um 14:20 schrieb Clune, Thomas L. (GSFC-6101) via J3:
> I had wondered if arranging the compiler itself to be on SSD could make
> the difference.   This is relatively easy to try with some compilers
> (NAG in particular), but not so much with Intel which is our work horse.

That would help for the first startup, but after the first
compilation, I would expect the relevant parts of the
compiler to be cached in memory, so any difference would
be small.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.j3-fortran.org/pipermail/j3/attachments/20241030/bbcffa1d/attachment.htm>