GNU bug report logs - #68417
scm_from_stringn throws error regardless of failed conversion handler

Message received at submit <at> debbugs.gnu.org:

Received: (at submit) by debbugs.gnu.org; 13 Jan 2024 07:13:09 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jan 13 02:13:09 2024
Received: from localhost ([127.0.0.1]:38249 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1rOYCZ-0002pU-Ee
	for submit <at> debbugs.gnu.org; Sat, 13 Jan 2024 02:13:08 -0500
Received: from lists.gnu.org ([2001:470:142::17]:40948)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <nathan.whetsell@HIDDEN>) id 1rOMwb-000722-3Y
 for submit <at> debbugs.gnu.org; Fri, 12 Jan 2024 14:11:55 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <nathan.whetsell@HIDDEN>)
 id 1rOMwX-00025j-Ck
 for bug-guile@HIDDEN; Fri, 12 Jan 2024 14:11:49 -0500
Received: from mail-qk1-x72d.google.com ([2607:f8b0:4864:20::72d])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.90_1) (envelope-from <nathan.whetsell@HIDDEN>)
 id 1rOMwV-0000W5-Ah; Fri, 12 Jan 2024 14:11:48 -0500
Received: by mail-qk1-x72d.google.com with SMTP id
 af79cd13be357-783195e57c7so444911185a.2; 
 Fri, 12 Jan 2024 11:11:45 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1705086704; x=1705691504; darn=gnu.org;
 h=to:message-id:subject:date:mime-version:content-transfer-encoding
 :from:from:to:cc:subject:date:message-id:reply-to;
 bh=S4uCDZhDXA0ZpSB4XH1AF9WQ7CeWAZy49bdBG8qdq4Q=;
 b=msD+8PoHq4TwMgNG2cj4p/zNyID1fxITs69+DSYuzIL2ts0orknzk8z3MZC0GwNlUf
 56Ke4k+gXKQXm24cmaKUE8G2BgbcvEteTWiHfPIcClfH/ABTVrW1J3Y1+7cOtCncw08M
 XBoa3wwG0Lk7hwEm3q6WSlhlY6OfAecjLU6/SwuLmqjHzmvv2KTOE+yoBWEDZHEfFz8i
 KNtQWSojkb8ESUENORTy9GpVyrEPl79zkJ50eP4K3LqKSvTytXxO/7wmb2hKyKRJoyti
 DSTCcAQDcNAAxzHb47HcNCGUSQrSVM/R54/tiiCSADVnlCjHyviYY8MOuOZgmi+B+osZ
 gXBw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1705086704; x=1705691504;
 h=to:message-id:subject:date:mime-version:content-transfer-encoding
 :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
 bh=S4uCDZhDXA0ZpSB4XH1AF9WQ7CeWAZy49bdBG8qdq4Q=;
 b=lXKL9TG2jva9JzkU4vWp+lL/8rA0faP0Tx+1uDWeRSZE/TBwzW3wjBmDXiK2miVlkC
 scx+3Y01t93tfRfe2g/CaDmWQ2/0kKJ8ITh+HQmCl01ZeL7tMTxXk5c9ufv48cpEPSkT
 0apDgggb1mlSqsJEINxvaQpTAkJHc7cTj46kSi/PGPY5ubP5qCd6HeCCT7lqJJcCuHGx
 zPY5kfsW5/Pfh/IjxP0hAuMA3nAWrgZWhrTx+0sv5RM8fdS600t6ALMn2URxNK+XHR2S
 6SXl/4snCdMiGr1HEabUnUQmRvWrT5JifiqY6w8e19twvzSvR6D25SNgeSPLyhbkQXWK
 I4fw==
X-Gm-Message-State: AOJu0Yw5CKuN27ZsV94ye5o2MW43Wv7nUxVjhvqsczfI7SwcM8OtqX/Q
 Szw5wE1rFgWYuNKPjJbYeciTvfDV1O4=
X-Google-Smtp-Source: AGHT+IH9HxcZqiCVFyb2YuGEdqzYAFeuAW2UwzVCM6NS8EMeFI9c/lJSwreywPxAonkPvgEWnYscDQ==
X-Received: by 2002:a05:620a:13e9:b0:783:25f9:170 with SMTP id
 h9-20020a05620a13e900b0078325f90170mr1891755qkl.80.1705086703907; 
 Fri, 12 Jan 2024 11:11:43 -0800 (PST)
Received: from smtpclient.apple (pool-68-132-223-133.nycmny.fios.verizon.net.
 [68.132.223.133]) by smtp.gmail.com with ESMTPSA id
 f14-20020a37ad0e000000b00781b8f4c89asm1252730qkm.43.2024.01.12.11.11.43
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Fri, 12 Jan 2024 11:11:43 -0800 (PST)
From: Nate Whetsell <nathan.whetsell@HIDDEN>
Content-Type: text/plain;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\))
Date: Fri, 12 Jan 2024 14:11:33 -0500
Subject: scm_from_stringn throws error regardless of failed conversion handler
Message-Id: <6CA63905-4ABF-4958-8B52-2F13EF9955C8@HIDDEN>
To: bug-guile@HIDDEN
X-Mailer: Apple Mail (2.3731.700.6)
Received-SPF: pass client-ip=2607:f8b0:4864:20::72d;
 envelope-from=nathan.whetsell@HIDDEN; helo=mail-qk1-x72d.google.com
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001,
 T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: submit
X-Mailman-Approved-At: Sat, 13 Jan 2024 02:13:05 -0500
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

Here is a test program that defines a Guile extension to convert invalid =
UTF-8 bytes to a string using each of the failed conversion handlers =
described at

=
https://www.gnu.org/software/guile/manual/html_node/Conversion-to_002ffrom=
-C.html#index-scm_005ft_005fstring_005ffailed_005fconversion_005fhandler

```
#include <libguile.h>

static char str[] =3D {
  208, 151, 208, 178, 208, 184, 209, 135, 208, 176, 208, 185, 208, 189, =
208, 184, 208, 102, 97, 109, 105, 108, 121
};

SCM
error_test (void)
{
  return scm_from_stringn (str, sizeof(str), "UTF-8", =
SCM_FAILED_CONVERSION_ERROR);
}

SCM
question_mark_test (void)
{
  return scm_from_stringn (str, sizeof(str), "UTF-8", =
SCM_FAILED_CONVERSION_QUESTION_MARK);
}

SCM
escape_test (void)
{
  return scm_from_stringn (str, sizeof(str), "UTF-8", =
SCM_FAILED_CONVERSION_ESCAPE_SEQUENCE);
}

void
init_test ()
{
  scm_c_define_gsubr ("conversion-error-test", 0, 0, 0, error_test);
  scm_c_define_gsubr ("conversion-question-mark-test", 0, 0, 0, =
question_mark_test);
  scm_c_define_gsubr ("conversion-escape-test", 0, 0, 0, escape_test);
}
```

If I compile this program using

```
gcc $(pkg-config --cflags --libs guile-3.0) -shared -o =
libguile-conversion-test.so -fPIC test.c
```

load the resulting extension into Guile, and run the functions, all of =
them throw errors. I would expect SCM_FAILED_CONVERSION_ERROR to throw =
an error, of course, and SCM_FAILED_CONVERSION_ESCAPE_SEQUENCE is =
documented as being irrelevant to converting C strings to Scheme. =
However, I would not expect using SCM_FAILED_CONVERSION_QUESTION_MARK to =
throw a conversion error.

Here is the output I get from Guile:

```
GNU Guile 3.0.9
Copyright (C) 1995-2023 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (load-extension "./libguile-conversion-test.so" =
"init_test")
scheme@(guile-user)> (conversion-error-test)
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Throw to key `decoding-error' with args `("scm_from_utf8_stringn" "input =
locale conversion error" 0 #vu8(208 151 208 178 208 184 209 135 208 176 =
208 185 208 189 208 184 208 102 97 109 105 108 121))'.

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,q
scheme@(guile-user)> (conversion-question-mark-test)
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Throw to key `decoding-error' with args `("scm_from_stringn" "input =
locale conversion error" 92 #vu8(208 151 208 178 208 184 209 135 208 176 =
208 185 208 189 208 184 208 102 97 109 105 108 121))'.

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,q
scheme@(guile-user)> (conversion-escape-test)
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Throw to key `decoding-error' with args `("scm_from_stringn" "input =
locale conversion error" 92 #vu8(208 151 208 178 208 184 209 135 208 176 =
208 185 208 189 208 184 208 102 97 109 105 108 121))'.

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
```

Note that it is possible to convert those bytes to a string without an =
error using bytevector->string, which is documented as the Scheme =
interface to scm_from_stringn:

=
https://www.gnu.org/software/guile/manual/html_node/Conversion-to_002ffrom=
-C.html#index-scm_005ffrom_005fstringn

```
scheme@(guile-user)> (use-modules (ice-9 iconv))
scheme@(guile-user)> (bytevector->string #vu8(208 151 208 178 208 184 =
209 135 208 176 208 185 208 189 208 184 208 102 97 109 105 108 121) =
"UTF-8" 'substitute)
$1 =3D "=D0=97=D0=B2=D0=B8=D1=87=D0=B0=D0=B9=D0=BD=D0=B8=EF=BF=BDfamily"
```=