Received: (at 74253) by debbugs.gnu.org; 25 Nov 2024 09:13:47 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Nov 25 04:13:47 2024 Received: from localhost ([127.0.0.1]:37857 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1tFVAB-0004Yt-9x for submit <at> debbugs.gnu.org; Mon, 25 Nov 2024 04:13:47 -0500 Received: from mail3-relais-sop.national.inria.fr ([192.134.164.104]:13839) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludovic.courtes@HIDDEN>) id 1tFVA9-0004Yf-BH for 74253 <at> debbugs.gnu.org; Mon, 25 Nov 2024 04:13:46 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=inria.fr; s=dc; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version:content-transfer-encoding; bh=zSnJsaPWrFqVnfy6OoTl7bjLQZEe0gmbDtodmHb0IjU=; b=ZKSk1VHVft87Pnd/i1CT6PWtgPlYVD3tiiTIt8zo3NcZUBWBCz45wfcp 1qgydVZUrliVwJHJSiL5TYspR46f9iYx3PfuEQ1sJ7c4rM2/YPkbArsP8 HOS/7KIq81grJCh+t/vdhYgaAv91NnPM65LcITWsnRvngUs/uEtkhpbRS k=; Authentication-Results: mail3-relais-sop.national.inria.fr; dkim=none (message not signed) header.i=none; spf=SoftFail smtp.mailfrom=ludovic.courtes@HIDDEN; dmarc=fail (p=none dis=none) d=inria.fr X-IronPort-AV: E=Sophos;i="6.12,182,1728943200"; d="scan'208";a="102528313" Received: from unknown (HELO ribbon) ([193.50.110.122]) by mail3-relais-sop.national.inria.fr with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Nov 2024 10:13:39 +0100 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludovic.courtes@HIDDEN> To: Efraim Flashner <efraim@HIDDEN> Subject: Re: [bug#74253] [PATCH] transformations: Add multituned-package. In-Reply-To: <Z0IgZDbD7fZM8mn4@3900XT> (Efraim Flashner's message of "Sat, 23 Nov 2024 20:35:16 +0200") References: <d3571383007eb50c7156d1ef88f9fb159b3fbb3d.1731062591.git.efraim@HIDDEN> <87serwu4iq.fsf@HIDDEN> <Z0IgZDbD7fZM8mn4@3900XT> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Quintidi 5 Frimaire an 233 de la =?utf-8?Q?R=C3=A9vo?= =?utf-8?Q?lution=2C?= jour du Cochon X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 25 Nov 2024 10:13:38 +0100 Message-ID: <87o723fzgd.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 74253 Cc: Josselin Poiret <dev@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>, Christopher Baines <guix@HIDDEN>, 74253 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Hello, Efraim Flashner <efraim@HIDDEN> skribis: > On Tue, Nov 12, 2024 at 11:31:57AM +0100, Ludovic Court=C3=A8s wrote: [...] >> Should that be a package transformation though? Could we instead have a >> build system trick or the =E2=80=98multituned-package=E2=80=99 procedure= exposed so >> build the package several times and fill in lib/glibc-hwcaps? > > I figured (guix transformations) worked well since it was using an > existing transformation to generate the different variants. > > I'm not sure what you mean by a build system trick, like build the > package multiple times in one like with x265? Yes, that sort of thing, or basically what you have here but provided as a build phase. Now, it=E2=80=99s probably easier to implement it the way you did than as a build phase, because how exactly to rebuild a package with the right optimizations is very dependent on details about its build system. > This exposes the multituned-package procedure (looks like I mistyped in > the commit message) which takes a package and can be just added on top > of an existing package definition, like hidden-package can. Yes, but I think the key ideal with hwcaps is that, contrary to what =E2=80=98--tune=E2=80=99 does, we=E2=80=99d provide several optimized versi= ons upfront. In that sense, we=E2=80=99d statically choose to =E2=80=9Cmulti-tune=E2=80=9D certa= in packages. >> That way, packagers would explicitly choose this technique for select >> packages, which would then no longer need the =E2=80=98tunable?=E2=80=99= property. > > Something like a flag in the build-system like #:tests? is? Or like a > package property? Either that, or write (define p' (multituned-package p)) etc. The key point being: packagers would be the one deciding that. So hmm, maybe the way forward is to keep the =E2=80=98multituned-package=E2= =80=99, but just not expose it as a package transformation. Also, we might want to move tuning support to (guix cpu-tuning) or similar, which is the module that packagers would import (rather than (guix transformations)). WDYT? Ludo=E2=80=99.
guix-patches@HIDDEN:bug#74253; Package guix-patches.
Full text available.Received: (at 74253) by debbugs.gnu.org; 23 Nov 2024 18:36:27 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Nov 23 13:36:27 2024 Received: from localhost ([127.0.0.1]:59118 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1tEuzb-0006aR-5J for submit <at> debbugs.gnu.org; Sat, 23 Nov 2024 13:36:27 -0500 Received: from mail-wm1-f46.google.com ([209.85.128.46]:45489) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <efraim.flashner@HIDDEN>) id 1tEuzY-0006aA-Sp for 74253 <at> debbugs.gnu.org; Sat, 23 Nov 2024 13:36:25 -0500 Received: by mail-wm1-f46.google.com with SMTP id 5b1f17b1804b1-4316f3d3c21so28141015e9.3 for <74253 <at> debbugs.gnu.org>; Sat, 23 Nov 2024 10:36:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732386919; x=1732991719; darn=debbugs.gnu.org; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date:sender:from:to :cc:subject:date:message-id:reply-to; bh=Px4nHyXQOazETJwBvPcJhoJPHcovYqJp7QOnW/dTLSc=; b=JL1Rg7akh5Q2DgCoymJIvJ40EANxlaU4TjTewm9dgtWi/IJkA+1tHiDrML6bSi+TT9 4qwh69c8UfgLhNopIYJpUtmayKecDHck6CV9V1VBCaxfyyD0mH11zk91HtjFLlE+hwsA MmX5w8MLgPBFsS0jUR1XVknEd/kJJMSZQUoPAu5FEgMAAk1OhQRkKotZvyX7m9KkBERn JeoFCl3y9HecZpJ+2k5z2rPX4NMBtMVQ0MaFriJ4xD5LEnf6o1jQOKni6zLNBJL1ei8r E7yyP4qe3x9UrI9ZZgigb/oRTgC62bz8VQ36hc5S1bQSAt6qV9dABm8+ayrICgDqYOFC tLSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732386919; x=1732991719; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Px4nHyXQOazETJwBvPcJhoJPHcovYqJp7QOnW/dTLSc=; b=TKxMaZSVf9DauA6FcLG+r0Z+UqaDTynbsjnDYrba3K3/4khEjQTDCUnjKV9J6qsgeV KcWPm3X8aIv4kGXrC6RpiLOZykNv1dR3FzPc7evs91eDD1hiSvU/bc/Gr+kaioLdEwBe pndZ0kHQzkggTffMRPNO29wQodfHFAttXRzyOxBZTcIxG9CYU6E7NFg57wkgPX9zPYID 9jjOoeXjV68k44ygrCmFNkeSpK5e3TUrOZnQQzLs123yQME5wWaHvUUeZngePaOWr2mi EEc8wpzAaOtyn3Uxi0I50dcvPlUebmLedNQH/7OKwc/ZOCuYyCCOCSv0ePVMs66VXhBL 4prA== X-Gm-Message-State: AOJu0YwivM57y6i43713fGCpR9I8Qr/5dvc+prs/vYw674iDZVszF2Og EG490DLjWAU2SbGR/T6lrxLl3ECO2qOhP4Yw4xMH554Xw6ylW9Us X-Gm-Gg: ASbGncuuzvh/7HVgEWqUE46OF5KqENHWbtuDLn7seUEdi0F2+C3dkGUdvVU1NPBkTkt EU5HJ0Ctj/5cR9Ld7jIQXIxHKmWVYaV1B6HEGXfDbeurz4hgDnh2XlB2JSqRcf1xhEUPovwWzIG jltsSJgJS/nme7OSIykA4mL8uiIdPLVVT7SuBxKh+I02rfJeTtBR25hzq3kqLB3fRDkFEpV6NCQ nFq/Gl9PU8ybftO0wJXZWgEpHF62kS6sSzaN0tvOanVgpKDkw== X-Google-Smtp-Source: AGHT+IHWydGgr3WnWB+RSdLritDmO2LETy7tDDD/I80oz02YNQzXbCGq1jEEMLnjq7iZP87GeNmtRg== X-Received: by 2002:a05:600c:358d:b0:431:59b2:f0d1 with SMTP id 5b1f17b1804b1-433ce41802amr65234135e9.4.1732386918732; Sat, 23 Nov 2024 10:35:18 -0800 (PST) Received: from localhost ([188.120.129.43]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-433b01e0584sm131264995e9.5.2024.11.23.10.35.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Nov 2024 10:35:17 -0800 (PST) Date: Sat, 23 Nov 2024 20:35:16 +0200 From: Efraim Flashner <efraim@HIDDEN> To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludovic.courtes@HIDDEN> Subject: Re: [bug#74253] [PATCH] transformations: Add multituned-package. Message-ID: <Z0IgZDbD7fZM8mn4@3900XT> Mail-Followup-To: Efraim Flashner <efraim@HIDDEN>, Ludovic =?utf-8?Q?Court=C3=A8s?= <ludovic.courtes@HIDDEN>, 74253 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN> References: <d3571383007eb50c7156d1ef88f9fb159b3fbb3d.1731062591.git.efraim@HIDDEN> <87serwu4iq.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="MX4Kectl+38D19Hi" Content-Disposition: inline In-Reply-To: <87serwu4iq.fsf@HIDDEN> X-PGP-Key-ID: 0x41AAE7DCCA3D8351 X-PGP-Key: https://flashner.co.il/~efraim/efraim_flashner.asc X-PGP-Fingerprint: A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 74253 Cc: Josselin Poiret <dev@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>, Christopher Baines <guix@HIDDEN>, 74253 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -0.7 (/) --MX4Kectl+38D19Hi Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Nov 12, 2024 at 11:31:57AM +0100, Ludovic Court=C3=A8s wrote: > Hi, >=20 > Efraim Flashner <efraim@HIDDEN> skribis: >=20 > > * guix/transformations.scm (package-tuned-for-psabi, > > multituned-package): New variables. > > > > Change-Id: I09ac7ae9fc2bcd9aa712b3c30fef807bc7d55895 > > --- > > > > This allows wrapping a package definition in multituned-package, ie: > > > > (define-public opus > > (multituned-package > > (package > > ...))) > > > > I'm not sure where to go with this patch from here. This will provide > > the psabi libraries for x86_64 and powerpc64le so they get most of the > > benefits from tuning for the architecture but without needing to specify > > which architecture to tune for. It should also provide a nice boost for > > guix packs and docker images and the like. > > > > The downside with using this by default is the larger package size due > > to the extra versions of the libraries, and if it is used then the > > regular --tune is disabled for that package. > > > > I think adding it as a '--tune=3Dgeneric' or '--tune=3Dpsabi' would be a > > nice way to use it. >=20 > Should that be a package transformation though? Could we instead have a > build system trick or the =E2=80=98multituned-package=E2=80=99 procedure = exposed so > build the package several times and fill in lib/glibc-hwcaps? I figured (guix transformations) worked well since it was using an existing transformation to generate the different variants. I'm not sure what you mean by a build system trick, like build the package multiple times in one like with x265? This exposes the multituned-package procedure (looks like I mistyped in the commit message) which takes a package and can be just added on top of an existing package definition, like hidden-package can. > That way, packagers would explicitly choose this technique for select > packages, which would then no longer need the =E2=80=98tunable?=E2=80=99 = property. Something like a flag in the build-system like #:tests? is? Or like a package property? > The question becomes: how would we choose which packages is eligible to > this technique as opposed to =E2=80=98--tune=E2=80=99? Intuitively, I wo= uld use that > for general-purpose packages like =E2=80=98opus=E2=80=99, but keep =E2=80= =98--tune=E2=80=99 for more > niche/scientific packages. Unfortunately I think just like with choosing what to do with tunable? we end up in the same spot deciding somewhat arbitrarily what to provide the psabi options for and what not to. > WDYT? >=20 > Thanks, > Ludo=E2=80=99. Looking more at the patch I've remembered that I've only taken care of the gnu- and cmake- build-systems. I was going to add about tuning the package on other architectures, like aarch64, but it seems I fixed that already. If it were just the configure-flags and the post-installation removal of files then I could see trying to make something that could be inserted manually into specific packages, but the whole thing rests on the tuned-package procedure actually producing libraries tuned for the different psABIs. --=20 Efraim Flashner <efraim@HIDDEN> =D7=90=D7=A4=D7=A8=D7=99=D7=9D = =D7=A4=D7=9C=D7=A9=D7=A0=D7=A8 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --MX4Kectl+38D19Hi Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmdCIF0ACgkQQarn3Mo9 g1EJlBAAsDrZYDFCOHNOdJbvN0AZg6aMrTGoAKKTJ2ElwT300Y76STETfw0EZvKm vriX82SWf8/FWmkvSlyI+xjKvVbWW3wYL9he7/TjSIt0Un2JJrox31t598ug/qsI KQ55TczsdJlv4P7yA5rmNliMHB6ac1KC9c6ZFi8cuLHrk7KLZjXOgpCDK0a+bGkv /ZZNKbCevC2pmQ34A8XxyLFuQtnMdkVPVOq0WS0hv9dsmu79v0xLcKPAQOBSY97w FZkSu9ZqQ/EYxR2v2gxzAeQCYhAoAj+UnskXq2dtjDGKlOawfKwuCB5hvKDMQTpL duKQcNM9qDL7u88M3WZ3MSvQAR+n7vrqq7kMb0etuzspJLbYY+rtTbuVFEtmBX3t /jClJFfkemTyAnCp56D4w6c4m6OFcvl9d4USxlimRsOk03nF7tQ2oXMY+6tmdHWX Eip1d/E+Gk043mZtXkLuP0PIVDZlkJjH+iuibq+Nuj0F4edORTH2i7a1MCTUzFpO 8+rHJs9ytexZ5kC8yz8DYrZlicgU9Of1a0RcRva9wHWW+R5EpwRhr3X0HG/EmqVi VZbtbsvL1SLnUZVEMuO3wgiJ+yzqg6nzjTuRSjiAU6MlZ10DK39tKiKNzJUOojdl 9rtdioVxWsgU8njqcMdeTCHryDY+1mg66OZnLiGYENitP/chp5o= =Qlmk -----END PGP SIGNATURE----- --MX4Kectl+38D19Hi--
guix-patches@HIDDEN:bug#74253; Package guix-patches.
Full text available.Received: (at 74253) by debbugs.gnu.org; 12 Nov 2024 10:32:07 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Nov 12 05:32:07 2024 Received: from localhost ([127.0.0.1]:60740 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1tAoBr-0007s4-0b for submit <at> debbugs.gnu.org; Tue, 12 Nov 2024 05:32:07 -0500 Received: from mail2-relais-roc.national.inria.fr ([192.134.164.83]:46045) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludovic.courtes@HIDDEN>) id 1tAoBo-0007rU-1z for 74253 <at> debbugs.gnu.org; Tue, 12 Nov 2024 05:32:05 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=inria.fr; s=dc; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version:content-transfer-encoding; bh=2X0aJIhjSOQaDXSrt/1w1NkIwCdMA3JPBTlrX0HWnbI=; b=vQ/bTgGQom/F/7urqCSGnEmdSCbj+rfR1qUEfUaVqhXR+bJ7cjRZoqTV ydcjiRh+Y88elY6KY1KQUcX2/eFsGAsk6KyK3AkqVXhlPXpx7FQPGSjoi JPSXN9l2DBBpE6hABr9t+BxYE6NuTlt3y/ogNR07wwOJgUeZhlQmaCfdO 4=; Authentication-Results: mail2-relais-roc.national.inria.fr; dkim=none (message not signed) header.i=none; spf=SoftFail smtp.mailfrom=ludovic.courtes@HIDDEN; dmarc=fail (p=none dis=none) d=inria.fr X-IronPort-AV: E=Sophos;i="6.12,147,1728943200"; d="scan'208";a="193324239" Received: from unknown (HELO ribbon) ([193.50.110.122]) by mail2-relais-roc.national.inria.fr with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Nov 2024 11:31:57 +0100 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludovic.courtes@HIDDEN> To: Efraim Flashner <efraim@HIDDEN> Subject: Re: [bug#74253] [PATCH] transformations: Add multituned-package. In-Reply-To: <d3571383007eb50c7156d1ef88f9fb159b3fbb3d.1731062591.git.efraim@HIDDEN> (Efraim Flashner's message of "Fri, 8 Nov 2024 12:44:15 +0200") References: <d3571383007eb50c7156d1ef88f9fb159b3fbb3d.1731062591.git.efraim@HIDDEN> Date: Tue, 12 Nov 2024 11:31:57 +0100 Message-ID: <87serwu4iq.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 74253 Cc: Josselin Poiret <dev@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>, Christopher Baines <guix@HIDDEN>, 74253 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Hi, Efraim Flashner <efraim@HIDDEN> skribis: > * guix/transformations.scm (package-tuned-for-psabi, > multituned-package): New variables. > > Change-Id: I09ac7ae9fc2bcd9aa712b3c30fef807bc7d55895 > --- > > This allows wrapping a package definition in multituned-package, ie: > > (define-public opus > (multituned-package > (package > ...))) > > I'm not sure where to go with this patch from here. This will provide > the psabi libraries for x86_64 and powerpc64le so they get most of the > benefits from tuning for the architecture but without needing to specify > which architecture to tune for. It should also provide a nice boost for > guix packs and docker images and the like. > > The downside with using this by default is the larger package size due > to the extra versions of the libraries, and if it is used then the > regular --tune is disabled for that package. > > I think adding it as a '--tune=3Dgeneric' or '--tune=3Dpsabi' would be a > nice way to use it. Should that be a package transformation though? Could we instead have a build system trick or the =E2=80=98multituned-package=E2=80=99 procedure ex= posed so build the package several times and fill in lib/glibc-hwcaps? That way, packagers would explicitly choose this technique for select packages, which would then no longer need the =E2=80=98tunable?=E2=80=99 pr= operty. The question becomes: how would we choose which packages is eligible to this technique as opposed to =E2=80=98--tune=E2=80=99? Intuitively, I woul= d use that for general-purpose packages like =E2=80=98opus=E2=80=99, but keep =E2=80= =98--tune=E2=80=99 for more niche/scientific packages. WDYT? Thanks, Ludo=E2=80=99.
guix-patches@HIDDEN:bug#74253; Package guix-patches.
Full text available.
Received: (at submit) by debbugs.gnu.org; 8 Nov 2024 10:44:35 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Nov 08 05:44:35 2024
Received: from localhost ([127.0.0.1]:51248 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1t9MTi-00080G-Kq
for submit <at> debbugs.gnu.org; Fri, 08 Nov 2024 05:44:35 -0500
Received: from lists.gnu.org ([209.51.188.17]:59458)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <efraim.flashner@HIDDEN>) id 1t9MTd-000803-Ej
for submit <at> debbugs.gnu.org; Fri, 08 Nov 2024 05:44:32 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <efraim.flashner@HIDDEN>)
id 1t9MTc-0006XR-Dt
for guix-patches@HIDDEN; Fri, 08 Nov 2024 05:44:28 -0500
Received: from mail-wm1-x333.google.com ([2a00:1450:4864:20::333])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
(Exim 4.90_1) (envelope-from <efraim.flashner@HIDDEN>)
id 1t9MTa-0006Ba-CV
for guix-patches@HIDDEN; Fri, 08 Nov 2024 05:44:28 -0500
Received: by mail-wm1-x333.google.com with SMTP id
5b1f17b1804b1-43161c0068bso17477845e9.1
for <guix-patches@HIDDEN>; Fri, 08 Nov 2024 02:44:25 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1731062664; x=1731667464; darn=gnu.org;
h=content-transfer-encoding:mime-version:message-id:date:subject:cc
:to:from:sender:from:to:cc:subject:date:message-id:reply-to;
bh=ZE+YnHjYRfo9BFqJaCKkYU0joU4rkR5LbNyaosUECDY=;
b=WuefaQSYs8A/XmrmKbCxi1JOV62/SG1L+NQmDuXsBTteOX4Wxd1CruE7Av+85vcM9W
EouJMr496+NOmidSCtggNP3fKc6K4oJMPL3ET4NBqmrpM2EW5K0xKlxxC41OFNtZ5U2g
6VYvWYdNVt6Y2bqC6TH1WNdWcmSbhStmiURLABxLCGSn9049dVaafaxWA1QrV50gFM1Q
UNNV6Yb1qBnFjkE7AgHWaShvWvl99z5Mx5RjJC7WLkk/NfAwU9/az5CBoRL0p5oFhIGY
8MTVRWz8mwb/kx/h0e0DZ5xi2PdDGsbVG24hef8D2LrxqPmnBhgTlfs9VRYFKX/1pxGN
WIBA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1731062664; x=1731667464;
h=content-transfer-encoding:mime-version:message-id:date:subject:cc
:to:from:sender:x-gm-message-state:from:to:cc:subject:date
:message-id:reply-to;
bh=ZE+YnHjYRfo9BFqJaCKkYU0joU4rkR5LbNyaosUECDY=;
b=mmSDP055/KYESq7RQivN9zYTHzYsnslLcuEj2IVW6OIrO0Wg2QJuJpYbopUehSzuUf
ie7qFmk8RhI6IefSi8a2s2o6Flg4SyDcRMniNJtNHq7QuSlyVof+Kd/5rpF7wRL7nEXY
3JDboL69uNpDDSBQZFS+rdtoO7BH52Nc8iIOpJqZ4jYTe72KdjQ32d12HTGD0IQo/N/k
F5JAShDOMKmbaF5o0SJyWnltPH+b1tnT9BcgZwCb0o5InoiSptBlLEmM/wW4d0Tm+05a
mKljnusdXZi8Bu82mnWeJw6pNvBqY6+bjgIaiTX4yHR04f75yYrGAkXAKyXv22W5lpY7
U+nQ==
X-Gm-Message-State: AOJu0Ywd7gv7uCsGaV2sitoaylSHPsxDRrdxx8qh24oySqC0WlTBuoNS
kK22PqWy3RVDuIr94y2cr7t4cyLbzFkeU1zneYkt8vLvPYi/iYy/80W4kg==
X-Google-Smtp-Source: AGHT+IHRGSIXm1U25llVHu0fRj4QUN6ahMdbTjIQUC8gscZ+sZ8Jb/B+XYvbmBr+jtj2Z0/uzpaGRA==
X-Received: by 2002:a05:600d:b:b0:432:bb4d:cd63 with SMTP id
5b1f17b1804b1-432bb4dce85mr1564255e9.18.1731062664093;
Fri, 08 Nov 2024 02:44:24 -0800 (PST)
Received: from localhost ([141.226.162.35]) by smtp.gmail.com with ESMTPSA id
5b1f17b1804b1-432aa70a1e9sm97724085e9.29.2024.11.08.02.44.23
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Fri, 08 Nov 2024 02:44:23 -0800 (PST)
From: Efraim Flashner <efraim@HIDDEN>
To: guix-patches@HIDDEN
Subject: [PATCH] transformations: Add multituned-package.
Date: Fri, 8 Nov 2024 12:44:15 +0200
Message-ID: <d3571383007eb50c7156d1ef88f9fb159b3fbb3d.1731062591.git.efraim@HIDDEN>
X-Mailer: git-send-email 2.46.0
MIME-Version: 1.0
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Received-SPF: pass client-ip=2a00:1450:4864:20::333;
envelope-from=efraim.flashner@HIDDEN; helo=mail-wm1-x333.google.com
X-Spam_score_int: -16
X-Spam_score: -1.7
X-Spam_bar: -
X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.001,
FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249,
RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
SPF_PASS=-0.001 autolearn=no autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.1 (-)
X-Debbugs-Envelope-To: submit
Cc: Efraim Flashner <efraim@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.1 (--)
* guix/transformations.scm (package-tuned-for-psabi,
multituned-package): New variables.
Change-Id: I09ac7ae9fc2bcd9aa712b3c30fef807bc7d55895
---
This allows wrapping a package definition in multituned-package, ie:
(define-public opus
(multituned-package
(package
...)))
I'm not sure where to go with this patch from here. This will provide
the psabi libraries for x86_64 and powerpc64le so they get most of the
benefits from tuning for the architecture but without needing to specify
which architecture to tune for. It should also provide a nice boost for
guix packs and docker images and the like.
The downside with using this by default is the larger package size due
to the extra versions of the libraries, and if it is used then the
regular --tune is disabled for that package.
I think adding it as a '--tune=generic' or '--tune=psabi' would be a
nice way to use it.
guix/transformations.scm | 75 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 75 insertions(+)
diff --git a/guix/transformations.scm b/guix/transformations.scm
index ea8b7a08443..4787e016b5d 100644
--- a/guix/transformations.scm
+++ b/guix/transformations.scm
@@ -60,6 +60,7 @@ (define-module (guix transformations)
tunable-package?
tuned-package
+ multituned-package
show-transformation-options-help
transformation-option-key?
@@ -634,6 +635,80 @@ (define (tuned-package p micro-architecture)
;; call 'tuned-package' again on this one.
,@(alist-delete 'tunable? (package-properties p))))))
+(define (package-tuned-for-psabi p psabi)
+ (let ((base (tuned-package p psabi)))
+ (package/inherit base
+ (name (string-append (package-name base) "-" psabi))
+ (arguments
+ (substitute-keyword-arguments (package-arguments base)
+ ((#:configure-flags flags #~'())
+ #~(append
+ (list
+ #$@(if (eq? (build-system-name (package-build-system p)) ; not base
+ (quote cmake-build-system))
+ #~((string-append "-DCMAKE_INSTALL_LIBDIR=lib/glibc-hwcaps/"
+ #$psabi))
+ #~((string-append "--libdir=" #$output
+ "/lib/glibc-hwcaps/" #$psabi))))
+ #$flags))
+ ((#:phases phases #~%standard-phases)
+ #~(modify-phases #$phases
+ (add-after 'install 'remove-extra-files
+ (lambda _
+ (for-each (lambda (dir)
+ (when (file-exists? (string-append #$output dir))
+ (delete-file-recursively
+ (string-append #$output dir))))
+ (list (string-append "/lib/glibc-hwcaps/"
+ #$psabi "/cmake")
+ (string-append "/lib/glibc-hwcaps/"
+ #$psabi "/pkgconfig")
+ "/bin" "/etc" "/include" "/libexec"
+ "/sbin" "/share" "/var")))))))))))
+
+(define (multituned-package p)
+ (package/inherit p
+ (arguments
+ (substitute-keyword-arguments (package-arguments p)
+ ((#:phases phases #~%standard-phases)
+ (if (or (target-x86-64?)
+ (target-ppc64le?))
+ #~(modify-phases #$phases
+ (add-after 'install 'install-optimized-libraries
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (let ((hwcaps "/lib/glibc-hwcaps/"))
+ (for-each
+ (lambda (psabi)
+ (copy-recursively
+ (string-append
+ (assoc-ref inputs (string-append
+ #$(package-name p) "-" psabi))
+ hwcaps psabi)
+ (string-append #$output hwcaps psabi)))
+ #$(cond ((target-x86-64?)
+ #~(list "x86-64-v2" "x86-64-v3" "x86-64-v4"))
+ ((target-ppc64le?)
+ #~(list "power9" "power10"))
+ (#t #~'())))))))
+ phases))))
+ (inputs
+ (cond ((target-x86-64?)
+ (modify-inputs (package-inputs p)
+ (append (package-tuned-for-psabi p "x86-64-v2")
+ (package-tuned-for-psabi p "x86-64-v3")
+ (package-tuned-for-psabi p "x86-64-v4"))))
+ ((target-ppc64le?)
+ (modify-inputs (package-inputs p)
+ (append (package-tuned-for-psabi p "power9")
+ (package-tuned-for-psabi p "power10"))))
+ (#t (package-inputs p))))
+ ;; With the addition of the psABIs this package should not be tuned.
+ (properties
+ (if (or (target-x86-64?)
+ (target-ppc64le?))
+ '((alist-delete 'tunable? (package-properties p)))
+ (package-properties p)))))
+
(define (tunable-package? package)
"Return true if package PACKAGE is \"tunable\"--i.e., if tuning it for the
host CPU is worthwhile."
base-commit: 2a6d96425eea57dc6dd48a2bec16743046e32e06
--
Efraim Flashner <efraim@HIDDEN> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
Efraim Flashner <efraim@HIDDEN>:guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN.
Full text available.guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:bug#74253; Package guix-patches.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.