GNU bug report logs - #38112
guile's recompilation does not play well with release builds

Message received at submit <at> debbugs.gnu.org:

Received: (at submit) by debbugs.gnu.org; 7 Nov 2019 23:34:29 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Nov 07 18:34:29 2019
Received: from localhost ([127.0.0.1]:44125 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1iSrIL-0006cK-GQ
	for submit <at> debbugs.gnu.org; Thu, 07 Nov 2019 18:34:29 -0500
Received: from lists.gnu.org ([209.51.188.17]:35516)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <slyfox@HIDDEN>) id 1iSrIJ-0006cC-F5
 for submit <at> debbugs.gnu.org; Thu, 07 Nov 2019 18:34:28 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:52205)
 by lists.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <slyfox@HIDDEN>) id 1iSrIG-00055Q-W7
 for bug-guile@HIDDEN; Thu, 07 Nov 2019 18:34:27 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_50,RCVD_IN_DNSWL_HI,
 URIBL_BLOCKED autolearn=disabled version=3.3.2
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <slyfox@HIDDEN>) id 1iSrIF-0002lD-Bb
 for bug-guile@HIDDEN; Thu, 07 Nov 2019 18:34:24 -0500
Received: from smtp.gentoo.org ([140.211.166.183]:38114)
 by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <slyfox@HIDDEN>) id 1iSrIF-0002jK-5o
 for bug-guile@HIDDEN; Thu, 07 Nov 2019 18:34:23 -0500
Received: from sf (tunnel547699-pt.tunnel.tserv1.lon2.ipv6.he.net
 [IPv6:2001:470:1f1c:3e6::2])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (No client certificate requested) (Authenticated sender: slyfox)
 by smtp.gentoo.org (Postfix) with ESMTPSA id 266C234CA3A
 for <bug-guile@HIDDEN>; Thu,  7 Nov 2019 23:34:19 +0000 (UTC)
Date: Thu, 7 Nov 2019 23:34:17 +0000
From: Sergei Trofimovich <slyfox@HIDDEN>
To: bug-guile@HIDDEN
Subject: guile's recompilation does not play well with release builds
Message-ID: <20191107233417.5be99aab@sf>
X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; x86_64-pc-linux-gnu)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
 recognized.
X-Received-From: 140.211.166.183
X-Spam-Score: -1.6 (-)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.6 (--)

I'll start from a failure where I found it out.

What I observed:
  I had guix-0.13 installed systemwide. Then I attempted to install
  guix-0.14 by unpacking/building it Bbuild failed with obscure errors
  about missing required functions

Why I think it failed:
  When guix-0.13 was built and installed among other things my system
  got compiled file as:
    '/usr/lib64/guile/2.2/site-ccache/guix/modules.go'

  When I unpacked guix-0.14 it's source file timestamps were preserved
  by unpack process. Specifically 'guix/modules.scm' was older than system's
  '/usr/lib64/guile/2.2/site-ccache/guix/modules.go'

  As a result guix build attempted to pull in parts of 0.13 release into 0.14
  build process.

  It looks like the main problem is in libguile/load.c:compiled_is_fresh():
    https://git.savannah.gnu.org/gitweb/?p=guile.git;a=blob;f=libguile/load.c;h=e95c36db1d670f0c7091aba080e21deb53bbab4f;hb=HEAD#l558

   567   if (source_mtime.tv_sec < compiled_mtime.tv_sec
   568       || (source_mtime.tv_sec == compiled_mtime.tv_sec
   569           && source_mtime.tv_nsec <= compiled_mtime.tv_nsec))
   570     compiled_is_newer = 1;

  Here 'mtime(.scm) < mtime(.go)' is enough to avoid recompilation.

The workaround:
  Currently Gentoo just 'touch'es source tarball at unpacking time to guarantee
  the rebuild as:
    https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=651f557f9302d29aa0de5ea10ef4e575b686fb21

  It is unfortunate and would require changing every package.

  The build failure is easily reproducible. I can try to craft artificial example
  to ease debugging if needed.

Probable fix:
  I think guile should embed/check more data about source file to hit the cache.
  The examples would be:
  1. size of source file
  2. hash of source file contents
  3. hash of parsed source file
  4. use 'mtime(.scm) == mtime(.go)' as a cache hit hint (would require faking
    mtime on a .go file)

Thank you!

-- 

  Sergei