GNU logs - #74425, boring messages

Message sent to guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:

Subject: [bug#74425] [PATCH] ui: Include channels in load path before searching for commands.
Resent-From: Carlo Zancanaro <carlo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN
Resent-Date: Mon, 18 Nov 2024 22:32:02 +0000
Resent-Message-ID: <handler.74425.B.173196911225233 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 74425 <at> debbugs.gnu.org
Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>, Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
From: Carlo Zancanaro <carlo@HIDDEN>
Date: Tue, 19 Nov 2024 09:26:37 +1100
Message-ID: <eb0ba012b1cafc05b8438eb99a9c736237031242.1731968797.git.carlo@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Received-SPF: pass client-ip=45.77.50.64; envelope-from=carlo@HIDDEN;
 helo=voltorb.zancanaro.id.au
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.3 (-)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)

* guix/ui.scm (extension-directories): Add channel directories to load path,
compiled load path, and returned list of directories.

Co-authored-by: Ludovic Courtès <ludo@HIDDEN>
Change-Id: I3063cbf7164a065b2c6c2a5c6473df79ce8cbe9b
---

This patch allows extensions in channels to be found by the Guix
CLI. Most notably, this means that a script can define commands by
defining an appropriate (guix scripts X) module, with a guix-X
function.

This is a slight modification to a diff that Ludovic sent through IRC.

I've tested that it works when I use "guix time-machine". Guix is able
to find a command that I define in a channel.

I haven't specifically tested that it finds commands when run through
"guix pull", but I have tested that this change does not break "guix
pull".

 guix/ui.scm | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/guix/ui.scm b/guix/ui.scm
index 447550635c..a702b6f3f8 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -19,6 +19,7 @@
 ;;; Copyright © 2018 Steve Sprang <scs@HIDDEN>
 ;;; Copyright © 2022 Taiju HIGASHI <higashi@HIDDEN>
 ;;; Copyright © 2022 Liliana Marie Prikler <liliana.prikler@HIDDEN>
+;;; Copyright © 2024 Carlo Zancanaro <carlo@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -73,6 +74,7 @@ (define-module (guix ui)                       ;import in user interfaces only
   #:use-module (srfi srfi-31)
   #:use-module (srfi srfi-34)
   #:use-module (srfi srfi-35)
+  #:use-module (srfi srfi-71)
   #:autoload   (ice-9 ftw)  (scandir)
   #:use-module (ice-9 match)
   #:use-module (ice-9 format)
@@ -2192,9 +2194,16 @@ (define* (command-files #:optional directory)

 (define (extension-directories)
   "Return the list of directories containing Guix extensions."
-  (filter file-exists?
-          (parse-path
-           (getenv "GUIX_EXTENSIONS_PATH"))))
+  (define package-path-entries
+    ;; We need to resolve this lazily, because even using an #:autoload is too
+    ;; much and breaks compilation during "guix pull".
+    (module-ref (resolve-module `(guix describe))
+                (symbol-append 'package-path-entries)))
+  (let ((scm-path go-path (package-path-entries)))
+    (set! %load-path (append %load-path scm-path))
+    (set! %load-compiled-path (append %load-compiled-path go-path))
+    (filter file-exists?
+            (parse-path (getenv "GUIX_EXTENSIONS_PATH") scm-path))))

 (define (commands)
   "Return the list of commands, alphabetically sorted."

base-commit: 23cbbe6860782c5d4a0ba599ea1cda0642e91661
-- 
2.46.0

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Carlo Zancanaro <carlo@HIDDEN>
Subject: bug#74425: Acknowledgement ([PATCH] ui: Include channels in load
 path before searching for commands.)
Message-ID: <handler.74425.B.173196911225233.ack <at> debbugs.gnu.org>
References: <eb0ba012b1cafc05b8438eb99a9c736237031242.1731968797.git.carlo@HIDDEN>
X-Gnu-PR-Message: ack 74425
X-Gnu-PR-Package: guix-patches
X-Gnu-PR-Keywords: patch
Reply-To: 74425 <at> debbugs.gnu.org
Date: Mon, 18 Nov 2024 22:32:02 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

As you requested using X-Debbugs-CC, your message was also forwarded to
  Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>,=
 Ludovic Court=C3=A8s <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, =
Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@tobias=
.gr>
(after having been given a bug report number, if it did not have one).

Your message has been sent to the package maintainer(s):
 guix-patches@HIDDEN

If you wish to submit further information on this problem, please
send it to 74425 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
74425: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D74425
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to guix-patches@HIDDEN:

Subject: [bug#74425] [PATCH] ui: Include channels in load path before searching for commands.
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Thu, 21 Nov 2024 12:15:02 +0000
Resent-Message-ID: <handler.74425.B74425.173219125628983 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Carlo Zancanaro <carlo@HIDDEN>
Cc: Josselin Poiret <dev@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>, 74425 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
In-Reply-To: <eb0ba012b1cafc05b8438eb99a9c736237031242.1731968797.git.carlo@HIDDEN>
 (Carlo Zancanaro's message of "Tue, 19 Nov 2024 09:26:37 +1100")
References: <eb0ba012b1cafc05b8438eb99a9c736237031242.1731968797.git.carlo@HIDDEN>
Date: Thu, 21 Nov 2024 13:14:00 +0100
Message-ID: <87jzcwss1j.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

Hello,

Carlo Zancanaro <carlo@HIDDEN> skribis:

> * guix/ui.scm (extension-directories): Add channel directories to load pa=
th,
> compiled load path, and returned list of directories.
>
> Co-authored-by: Ludovic Court=C3=A8s <ludo@HIDDEN>
> Change-Id: I3063cbf7164a065b2c6c2a5c6473df79ce8cbe9b

Nice, thanks for looking into it!

>  (define (extension-directories)
>    "Return the list of directories containing Guix extensions."
> -  (filter file-exists?
> -          (parse-path
> -           (getenv "GUIX_EXTENSIONS_PATH"))))
> +  (define package-path-entries
> +    ;; We need to resolve this lazily, because even using an #:autoload =
is too
> +    ;; much and breaks compilation during "guix pull".
> +    (module-ref (resolve-module `(guix describe))
> +                (symbol-append 'package-path-entries)))
> +  (let ((scm-path go-path (package-path-entries)))
> +    (set! %load-path (append %load-path scm-path))
> +    (set! %load-compiled-path (append %load-compiled-path go-path))
> +    (filter file-exists?
> +            (parse-path (getenv "GUIX_EXTENSIONS_PATH") scm-path))))

I think you could do:

  #:autoload (guix describe) (package-path-entries)

instead of doing the trick above.

Two things:

  1. The =E2=80=98%load-path=E2=80=99 and =E2=80=98%load-compiled-path=E2=
=80=99 modifications can now be
     removed from (gnu packages), after checking that GUIX_PACKAGE_PATH
     still takes precedence.

  2. The performance impact of this change must be tested, in particular
     the startup time of =E2=80=98guix=E2=80=99 commands since they=E2=80=
=99ll now be loading
     (guix channels) & co. unconditionally.

     One way to do that might be:

       PROFILE=3D$(guix time-machine -q --url=3D/path/to/checkout)
       guix shell time -- time $PROFILE/bin/guix build --help
       strace -c $PROFILE/bin/guix build --help
=20=20=20=20=20
Could you give it a try?

Ludo=E2=80=99.

Message sent to guix-patches@HIDDEN:

Subject: [bug#74425] [PATCH] ui: Include channels in load path before searching for commands.
Resent-From: Carlo Zancanaro <carlo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Thu, 21 Nov 2024 23:26:01 +0000
Resent-Message-ID: <handler.74425.B74425.173223152719255 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Cc: Josselin Poiret <dev@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>, 74425 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
From: Carlo Zancanaro <carlo@HIDDEN>
In-Reply-To: <87jzcwss1j.fsf@HIDDEN> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Thu, 21 Nov 2024 13:14:00 +0100")
References: <eb0ba012b1cafc05b8438eb99a9c736237031242.1731968797.git.carlo@HIDDEN>
 <87jzcwss1j.fsf@HIDDEN>
Date: Fri, 22 Nov 2024 10:25:15 +1100
Message-ID: <87iksgxj8k.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

On Thu, Nov 21 2024, Ludovic Court=C3=A8s wrote:
> I think you could do:
>
>   #:autoload (guix describe) (package-path-entries)
>
> instead of doing the trick above.

I tried that, and it breaks "guix pull" (as mentioned in the comment in
the code). I don't have the specific error easily accessible, but I
think it was "no code for module: (git)". I assume it was an issue with
how build-program in (build-self) puts things together, but I wasn't
able to investigate further than that.

>   1. The =E2=80=98%load-path=E2=80=99 and =E2=80=98%load-compiled-path=E2=
=80=99 modifications can now be
>      removed from (gnu packages), after checking that GUIX_PACKAGE_PATH
>      still takes precedence.

With this change, (extension-directories) does not include the paths in
GUIX_PACKAGE_PATH. Perhaps it should, but I wasn't sure given the
distinction between GUIX_PACKAGE_PATH and GUIX_EXTENSIONS_PATH.

Do we want to run commands from GUIX_PACKAGE_PATH? Or load packages from
GUIX_EXTENSIONS_PATH? At the moment I think the code says "no" to both.

My other concern with removing the code in (gnu packages) is that it may
break code that is using Guix as a library. That is, something like
this:

--8<---------------cut here---------------start------------->8---
guix shell guix guile -- guile -c '(use-modules (gnu packages)) (pk (find-p=
ackages-by-name "guile"))'
--8<---------------cut here---------------end--------------->8---

might not add the right things to the load path.

That said, the current patch will add duplicate entries to the load path
when using the Guix CLI, which also isn't good. Perhaps this needs to be
factored out into a single place where we can keep track of whether
we've already added to the load paths.

>   2. The performance impact of this change must be tested, in particular
>      the startup time of =E2=80=98guix=E2=80=99 commands since they=E2=80=
=99ll now be loading
>      (guix channels) & co. unconditionally.
>
>      One way to do that might be:
>
>        PROFILE=3D$(guix time-machine -q --url=3D/path/to/checkout)
>        guix shell time -- time $PROFILE/bin/guix build --help
>        strace -c $PROFILE/bin/guix build --help
>=20=20=20=20=20=20
> Could you give it a try?

Sure!

With the change in my patch, I'm seeing:

strace: number of calls and errors is roughly the same both with and
without my patch.

time: all the numbers look pretty similar with and without my patch.

I haven't done any statistical analysis of the various values recorded
by strace and time, but the numbers don't vary *too* much between runs,
so I'm fairly confident this isn't having too much of an effect.

I also tested leaving off the "-q" in the time machine call (so my other
channels were also included), but this didn't make any meaningful
difference to the numbers when running "guix build --help".

Carlo

Message received at control <at> debbugs.gnu.org:

Received: (at control) by debbugs.gnu.org; 26 Dec 2024 21:33:30 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Dec 26 16:33:30 2024
Received: from localhost ([127.0.0.1]:42507 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1tQvU2-0003Kn-14
	for submit <at> debbugs.gnu.org; Thu, 26 Dec 2024 16:33:30 -0500
Received: from eggs.gnu.org ([209.51.188.92]:52870)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1tQvTz-0003KY-Jz
 for control <at> debbugs.gnu.org; Thu, 26 Dec 2024 16:33:28 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1tQvTu-0004Ui-Da
 for control <at> debbugs.gnu.org; Thu, 26 Dec 2024 16:33:22 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-version:Subject:From:To:Date:in-reply-to:
 references; bh=TWJ5NoGYY+oKn2Q3Z7Ko2+7HQiX0PKERvoMkC1te5VI=; b=NNBi3AvsiLoSuX
 APO5DEu+AUu5A++8HPgA95rJWBbBTaHz5ypvTBq+CdM6EO9aLL3FAg5J8IbPVlwdSBQKJ9AT3nmGM
 cExCFcKlsFXQIavShKtU3qUhc1G3XpoQfiSGF0LP9FC+u0dFw6Klmhl8aGbO5ABgX1zz/gwgYlnwS
 qqEeSbiBVCYXjsOO6d4PmkoL8zfowNgTF+WccH/mEPOjPvG37RpBgO6G680EvGXVtCjqkuhnOtnQ7
 yf+ApAbgVxLPlj1N1sWk9zvDxexIqzVxJNShGAnQvOPf9neNLJhIrofhAvnCXDbgnrudqc0U7m3w8
 pxVtSPORe6U2jN7f03nw==;
Date: Thu, 26 Dec 2024 22:33:17 +0100
Message-Id: <87msgiyvs2.fsf@HIDDEN>
To: control <at> debbugs.gnu.org
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
Subject: control message for bug #74633
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: control
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

merge 74633 74425
quit

Last modified: Sun, 12 Jan 2025 05:45:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.