Received: (at 75045) by debbugs.gnu.org; 24 Dec 2024 10:42:44 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Dec 24 05:42:44 2024
Received: from localhost ([127.0.0.1]:59638 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1tQ2NA-00009w-1j
for submit <at> debbugs.gnu.org; Tue, 24 Dec 2024 05:42:44 -0500
Received: from eggs.gnu.org ([209.51.188.92]:36120)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1tQ2N6-00009d-9M
for 75045 <at> debbugs.gnu.org; Tue, 24 Dec 2024 05:42:42 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
id 1tQ2Ks-0000S1-Sz; Tue, 24 Dec 2024 05:40:22 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To:
From; bh=55p6zUpTXy0yyvL3ZLv7MqnbwCIJL6VEk3fWg7KSDTA=; b=GhlDPQCiI1VfedkZeXa7
Vi6m14cRRxXje2wt0bSRDphzI2x2jOr0qkkT1JrOUV58YXsrOpNo0ulDExkQEVelD9O92SXtCmF0A
Vha3/K3XmCeBVDpsHh5pAlCbcJ+FqVdxPG5ENNRJWWJY4awm9LGjBX4u86VDRofrgPfhZqItqVVRm
n6uk4nep9uOg/NNc6RdxQY7dgZtcwD17eeSEnWva+Yo7o8gTvsd+P7yoWRRI+WMDjtBLFWJ71HH7s
ZPHzsBkB/ErMQWwq8aRugIFpGM/4YnTT7944gi3hEbI0om9diqZCid5lV6IaokSUL1uJ35tgFJSNs
bjB+QAB0Wu997A==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Giacomo Leidi <goodoldpaul@HIDDEN>
Subject: Re: [bug#75045] [PATCH] services: restic-backup: Implement as a
Shepherd timer.
In-Reply-To: <f52cde358b609d18f43bf62f1dfe63835c1a57b9.1734950765.git.goodoldpaul@HIDDEN>
(Giacomo Leidi's message of "Mon, 23 Dec 2024 11:46:05 +0100")
References: <f52cde358b609d18f43bf62f1dfe63835c1a57b9.1734950765.git.goodoldpaul@HIDDEN>
Date: Tue, 24 Dec 2024 11:40:19 +0100
Message-ID: <87ikr95ppo.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 75045
Cc: Maxim Cournoyer <maxim.cournoyer@HIDDEN>, 75045 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Hi Giacomo,
Giacomo Leidi <goodoldpaul@HIDDEN> skribis:
> This patch implements restic backup with Shepherd services. It is
> supposed not to break any existing setup.
>
> * gnu/services/backup.scm (restic-backup-job): Add Shepherd
> configuration options;
> (restic-backup-job->mcron-job): Replace with...;
> (restic-job-log-file): New procedure;
> (restic-backup-job->shepherd-service): New procedure;
> (restic-backup-activation): New procedure;
> (restic-backup-service-type): Replace mcron with Shepherd extension and a=
dd
> activation extension hook.
> * doc/guix.texi: Document it.
>
> Change-Id: I66de3b6a1cb6177f9e4ee0c2acf3013ecbcdd338
Woo, nice!
As mentioned in <https://issues.guix.gnu.org/74860>, I think we should
postpone a little bit since these new features won=E2=80=99t work for peopl=
e who
haven=E2=80=99t rebooted into Shepherd 1.0, and they=E2=80=99ll get possibl=
y confusing
messages when reconfiguring.
How we should postpone, I=E2=80=99m not sure. The conservative approach wo=
uld
be to wait until after the next Guix release, but that doesn=E2=80=99t sound
reasonable=E2=80=A6 One month after the Shepherd upgrade, which would be
Jan. 9th? It=E2=80=99s common for servers to have much longer uptimes thou=
gh,
plus there=E2=80=99s end-of-year vacation here.
> + (log-file
> + (maybe-string)
> + "The file system path to the log file for this job. By default the f=
ile will
> +have the name of the job and be under @code{/var/log/restic-backup}.")
Rather @file, with the =E2=80=9C.log=E2=80=9D suffix too, if I=E2=80=99m no=
t mistaken.
> + (max-duration
> + (maybe-number)
> + "The maximum duration in seconds that a job may last. Past
> +@code{max-duration} seconds, the job will forcefully terminated.")
s/will/is/
> + (shepherd-service (provision `(,(string->symbol
> + (string-append name "-job"))))
I would tend to not add the =E2=80=9C-job=E2=80=9D suffix, but that=E2=80=
=99s a matter of taste!
> + (command
> + (list
> + (string-append #+bash-minimal "/bin/bash")
> + "-l" "-c"
> + (string-append "restic-guix backup " #$name))
Why go through bash? Would it be possible to execute =E2=80=98restic-guix=
=E2=80=99
directly?
> (description
> - "This service configures @code{mcron} jobs for running =
backups
> + "This service configures @code{Shepherd} timers for run=
ning backups
> with @code{restic}.")))
You can drop @code here.
guix-patches@HIDDEN:bug#75045; Package guix-patches.
Full text available.paul <goodoldpaul@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.
Received: (at submit) by debbugs.gnu.org; 23 Dec 2024 10:46:46 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Dec 23 05:46:46 2024
Received: from localhost ([127.0.0.1]:53469 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1tPfxV-0006EU-KK
for submit <at> debbugs.gnu.org; Mon, 23 Dec 2024 05:46:46 -0500
Received: from lists.gnu.org ([209.51.188.17]:40722)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <goodoldpaul@HIDDEN>) id 1tPfxS-0006EK-V4
for submit <at> debbugs.gnu.org; Mon, 23 Dec 2024 05:46:44 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <goodoldpaul@HIDDEN>)
id 1tPfxS-0006tn-PF
for guix-patches@HIDDEN; Mon, 23 Dec 2024 05:46:42 -0500
Received: from confino.investici.org ([93.190.126.19])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <goodoldpaul@HIDDEN>)
id 1tPfxP-0003gD-NF
for guix-patches@HIDDEN; Mon, 23 Dec 2024 05:46:42 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=autistici.org;
s=stigmate; t=1734950794;
bh=rSJVA8TCLbx9MDrPZX5JM4ksnkPmSsxTJSgSPHPdU70=;
h=From:To:Cc:Subject:Date:From;
b=VxYTW+YUFQxlKnOcEpMYG/o8/WTqizDDm2HYIybcfuYoS214q+8ayxeN6LOoR7ETW
UhocFXvXz/2JmunYjPC3Qx0SluDRkOAoFwBZDR9MJxrxyobDYlXiOUXXYCO33/ZOHf
/qTBRTSL4f1Wlv9nYbeqseMFN55P5J0iE7MWU0J0=
Received: from mx1.investici.org (unknown [127.0.0.1])
by confino.investici.org (Postfix) with ESMTP id 4YGvpL679Kz10wk;
Mon, 23 Dec 2024 10:46:34 +0000 (UTC)
Received: from [93.190.126.19] (mx1.investici.org [93.190.126.19])
(Authenticated sender: goodoldpaul@HIDDEN) by localhost (Postfix) with
ESMTPSA id 4YGvpL53vBz10wf; Mon, 23 Dec 2024 10:46:34 +0000 (UTC)
From: Giacomo Leidi <goodoldpaul@HIDDEN>
To: guix-patches@HIDDEN
Subject: [PATCH] services: restic-backup: Implement as a Shepherd timer.
Date: Mon, 23 Dec 2024 11:46:05 +0100
Message-ID: <f52cde358b609d18f43bf62f1dfe63835c1a57b9.1734950765.git.goodoldpaul@HIDDEN>
X-Mailer: git-send-email 2.46.0
MIME-Version: 1.0
X-Debbugs-Cc: Ludovic Courtès <ludo@HIDDEN>, Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Content-Transfer-Encoding: 8bit
Received-SPF: pass client-ip=93.190.126.19;
envelope-from=goodoldpaul@HIDDEN; helo=confino.investici.org
X-Spam_score_int: -27
X-Spam_score: -2.8
X-Spam_bar: --
X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001,
RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001,
SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.4 (-)
X-Debbugs-Envelope-To: submit
Cc: Giacomo Leidi <goodoldpaul@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.4 (--)
This patch implements restic backup with Shepherd services. It is
supposed not to break any existing setup.
* gnu/services/backup.scm (restic-backup-job): Add Shepherd
configuration options;
(restic-backup-job->mcron-job): Replace with...;
(restic-job-log-file): New procedure;
(restic-backup-job->shepherd-service): New procedure;
(restic-backup-activation): New procedure;
(restic-backup-service-type): Replace mcron with Shepherd extension and add
activation extension hook.
* doc/guix.texi: Document it.
Change-Id: I66de3b6a1cb6177f9e4ee0c2acf3013ecbcdd338
---
doc/guix.texi | 36 +++++++++----
gnu/services/backup.scm | 114 ++++++++++++++++++++++++++++++++++------
2 files changed, 123 insertions(+), 27 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 57030102ca..f77b765933 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -41987,19 +41987,16 @@ Miscellaneous Services
"/etc/guix/signing-key.sec"))))))))))
@end lisp
-Each @code{restic-backup-job} translates to an mcron job which sets the
+Each @code{restic-backup-job} translates to a Shepherd timer which sets the
@env{RESTIC_PASSWORD} environment variable by reading the first line of
@code{password-file} and runs @command{restic backup}, creating backups
using rclone of all the files listed in the @code{files} field.
-The @code{restic-backup-service-type} installs as well @code{restic-guix}
-to the system profile, a @code{restic} utility wrapper that allows for easier
-interaction with the Guix configured backup jobs. For example the following
-could be used to instantaneusly trigger a backup for the above shown
-configuration, without waiting for the scheduled job:
+The @code{restic-backup-service-type} provides the ability to instantaneously
+trigger a backup with the @code{trigger} Shepherd action:
@example
-restic-guix backup remote-ftp
+sudo herd trigger remote-ftp-job
@end example
@c %start of fragment
@@ -42030,6 +42027,22 @@ Miscellaneous Services
@item @code{user} (default: @code{"root"}) (type: string)
The user used for running the current job.
+@item @code{group} (default: @code{"root"}) (type: string)
+The group used for running the current job.
+
+@item @code{log-file} (type: maybe-string)
+The file system path to the log file for this job. By default the file will
+have the name of the job and be under @code{/var/log/restic-backup}.
+
+@item @code{max-duration} (type: maybe-number)
+The maximum duration in seconds that a job may last. Past
+@code{max-duration} seconds, the job will forcefully terminated.
+
+@item @code{wait-for-termination?} (default: @code{#f}) (type: boolean)
+Wait until the job has finished before considering executing it again;
+otherwise, perform it strictly on every occurrence of event, at the risk of
+having multiple instances running concurrently.
+
@item @code{repository} (type: string)
The restic repository target of this job.
@@ -42042,9 +42055,12 @@ Miscellaneous Services
for the current job.
@item @code{schedule} (type: gexp-or-string)
-A string or a gexp that will be passed as time specification in the
-mcron job specification (@pxref{Syntax, mcron job specifications,,
-mcron,GNU@tie{}mcron}).
+A string or a gexp representing the frequency of the backup. Gexp must
+evaluate to @code{calendar-event} records or to strings. Strings must contain
+Vixie cron date lines.
+
+@item @code{requirement} (default: @code{'()}) (type: list-of-symbols)
+The list of Shepherd services that this backup job depends upon.
@item @code{files} (default: @code{'()}) (type: list-of-lowerables)
The list of files or directories to be backed up. It must be a list of
diff --git a/gnu/services/backup.scm b/gnu/services/backup.scm
index 555e9fc959..fc8934873b 100644
--- a/gnu/services/backup.scm
+++ b/gnu/services/backup.scm
@@ -18,9 +18,10 @@
(define-module (gnu services backup)
#:use-module (gnu packages backup)
+ #:use-module (gnu packages bash)
#:use-module (gnu services)
#:use-module (gnu services configuration)
- #:use-module (gnu services mcron)
+ #:use-module (gnu services shepherd)
#:use-module (guix build-system copy)
#:use-module (guix gexp)
#:use-module ((guix licenses)
@@ -33,11 +34,16 @@ (define-module (gnu services backup)
restic-backup-job-fields
restic-backup-job-restic
restic-backup-job-user
+ restic-backup-job-group
+ restic-backup-job-log-file
+ restic-backup-job-max-duration
+ restic-backup-job-wait-for-termination?
restic-backup-job-name
restic-backup-job-repository
restic-backup-job-password-file
restic-backup-job-schedule
restic-backup-job-files
+ restic-backup-job-requirement
restic-backup-job-verbose?
restic-backup-job-extra-flags
@@ -64,6 +70,12 @@ (define (lowerable? value)
(define list-of-lowerables?
(list-of lowerable?))
+(define list-of-symbols?
+ (list-of symbol?))
+
+(define-maybe string)
+(define-maybe number)
+
(define-configuration/no-serialization restic-backup-job
(restic
(package restic)
@@ -71,6 +83,22 @@ (define-configuration/no-serialization restic-backup-job
(user
(string "root")
"The user used for running the current job.")
+ (group
+ (string "root")
+ "The group used for running the current job.")
+ (log-file
+ (maybe-string)
+ "The file system path to the log file for this job. By default the file will
+have the name of the job and be under @code{/var/log/restic-backup}.")
+ (max-duration
+ (maybe-number)
+ "The maximum duration in seconds that a job may last. Past
+@code{max-duration} seconds, the job will forcefully terminated.")
+ (wait-for-termination?
+ (boolean #f)
+ "Wait until the job has finished before considering executing it again;
+otherwise, perform it strictly on every occurrence of event, at the risk of
+having multiple instances running concurrently.")
(name
(string)
"A string denoting a name for this job.")
@@ -84,9 +112,12 @@ (define-configuration/no-serialization restic-backup-job
current job.")
(schedule
(gexp-or-string)
- "A string or a gexp that will be passed as time specification in the mcron
-job specification (@pxref{Syntax, mcron job specifications,, mcron,
-GNU@tie{}mcron}).")
+ "A string or a gexp representing the frequency of the backup. Gexp must
+evaluate to @code{calendar-event} records or to strings. Strings must contain
+Vixie cron date lines.")
+ (requirement
+ (list-of-symbols '())
+ "The list of Shepherd services that this backup job depends upon.")
(files
(list-of-lowerables '())
"The list of files or directories to be backed up. It must be a list of
@@ -175,16 +206,56 @@ (define (restic-guix jobs)
(main (command-line)))))
-(define (restic-backup-job->mcron-job config)
- (let ((user
- (restic-backup-job-user config))
- (schedule
- (restic-backup-job-schedule config))
- (name
- (restic-backup-job-name config)))
- #~(job #$schedule
- #$(string-append "restic-guix backup " name)
- #:user #$user)))
+(define (restic-job-log-file job)
+ (let ((name (restic-backup-job-name job))
+ (log-file (restic-backup-job-log-file job)))
+ (if (maybe-value-set? log-file)
+ log-file
+ (string-append "/var/log/restic-backup/" name ".log"))))
+
+(define (restic-backup-job->shepherd-service config)
+ (let ((schedule (restic-backup-job-schedule config))
+ (name (restic-backup-job-name config))
+ (user (restic-backup-job-user config))
+ (group (restic-backup-job-group config))
+ (max-duration (restic-backup-job-max-duration config))
+ (wait-for-termination? (restic-backup-job-wait-for-termination? config))
+ (log-file (restic-job-log-file config))
+ (requirement (restic-backup-job-requirement config)))
+ (shepherd-service (provision `(,(string->symbol
+ (string-append name "-job"))))
+ (requirement
+ `(user-processes file-systems ,@requirement))
+ (documentation
+ "Run @code{restic} backed backups on a regular basis.")
+ (modules '((shepherd service timer)))
+ (start
+ #~(make-timer-constructor
+ (if (string? #$schedule)
+ (cron-string->calendar-event #$schedule)
+ #$schedule)
+ (command
+ (list
+ (string-append #+bash-minimal "/bin/bash")
+ "-l" "-c"
+ (string-append "restic-guix backup " #$name))
+ #:user #$user
+ #:group #$group
+ #:environment-variables
+ (list
+ (string-append
+ "HOME=" (passwd:dir (getpwnam #$user)))))
+ #:log-file #$log-file
+ #:wait-for-termination? #$wait-for-termination?
+ #:max-duration #$(and (maybe-value-set? max-duration)
+ max-duration)))
+ (stop
+ #~(make-timer-destructor))
+ (actions (list (shepherd-action
+ (name 'trigger)
+ (documentation "Manually trigger a backup,
+without waiting for the scheduled time.")
+ (procedure #~trigger-timer)))))))
(define (restic-guix-wrapper-package jobs)
(package
@@ -212,15 +283,24 @@ (define restic-backup-service-profile
(restic-guix-wrapper-package jobs))
'())))
+(define (restic-backup-activation config)
+ #~(for-each
+ (lambda (log-file)
+ (mkdir-p (dirname log-file)))
+ (list #$@(map restic-job-log-file
+ (restic-backup-configuration-jobs config)))))
+
(define restic-backup-service-type
(service-type (name 'restic-backup)
(extensions
(list
+ (service-extension activation-service-type
+ restic-backup-activation)
(service-extension profile-service-type
restic-backup-service-profile)
- (service-extension mcron-service-type
+ (service-extension shepherd-root-service-type
(lambda (config)
- (map restic-backup-job->mcron-job
+ (map restic-backup-job->shepherd-service
(restic-backup-configuration-jobs
config))))))
(compose concatenate)
@@ -232,5 +312,5 @@ (define restic-backup-service-type
jobs)))))
(default-value (restic-backup-configuration))
(description
- "This service configures @code{mcron} jobs for running backups
+ "This service configures @code{Shepherd} timers for running backups
with @code{restic}.")))
base-commit: 2743faebb2893f65fb29a5cfd55c72a66a2b98a9
--
2.46.0
Giacomo Leidi <goodoldpaul@HIDDEN>:ludo@HIDDEN, maxim.cournoyer@HIDDEN, guix-patches@HIDDEN.
Full text available.ludo@HIDDEN, maxim.cournoyer@HIDDEN, guix-patches@HIDDEN:bug#75045; Package guix-patches.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.