From sven at anderson.de Fri May 3 00:26:21 2024 From: sven at anderson.de (Sven Anderson) Date: Fri, 3 May 2024 00:26:21 +0200 Subject: Fix for BMI2 detection for Intel CPUs in fat builds Message-ID: Hi, Due to a bug in the BMI2 detection in mpn/x86_64/fat/fat.c (missing load of the correct CPUID register) the codepath with the BMI2 and ADX instructions (_coreihwl and _coreibwl suffix) is never being used in fat binaries. Fixing it gave me a large performance boost. See attached patch. (Please reply also directly to me, I'm not subscribed to the list.) Best regards, Sven -------------- next part -------------- A non-text attachment was scrubbed... Name: libgmp-fat.patch Type: application/octet-stream Size: 656 bytes Desc: not available URL: From vincent at vinc17.net Tue May 14 09:52:17 2024 From: vincent at vinc17.net (Vincent Lefevre) Date: Tue, 14 May 2024 09:52:17 +0200 Subject: GMP repository: "make" failure if yacc is not available Message-ID: <20240514075217.GK3047@qaa.vinc17.org> I'm trying to build GMP from its repository (18472:1040c6303455) on a new machine, and "make" gives the following error: make[3]: Entering directory '/home/vinc17/software/gmp/demos/calc' test -f calc.c || /bin/sh ../../ylwrap calc.y y.tab.c calc.c y.tab.h `echo calc.c | sed -e s/cc$/hh/ -e s/cpp$/hpp/ -e s/cxx$/hxx/ -e s/c++$/h++/ -e s/c$/h/` y.output calc.output -- yacc -d ../../ylwrap: 176: yacc: not found make[3]: *** [Makefile:459: calc.c] Error 127 Is yacc a new requirement? If it is, it should be checked by the configure script. Note that neither yacc nor bison is mentioned as needed by "README.HG", in case this is specific to the use of the hg repository. -- Vincent Lef?vre - Web: 100% accessible validated (X)HTML - Blog: Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon) From nisse at lysator.liu.se Wed May 15 08:05:11 2024 From: nisse at lysator.liu.se (=?utf-8?Q?Niels_M=C3=B6ller?=) Date: Wed, 15 May 2024 08:05:11 +0200 Subject: GMP repository: "make" failure if yacc is not available In-Reply-To: <20240514075217.GK3047@qaa.vinc17.org> (Vincent Lefevre's message of "Tue, 14 May 2024 09:52:17 +0200") References: <20240514075217.GK3047@qaa.vinc17.org> Message-ID: Vincent Lefevre writes: > Is yacc a new requirement? If it is, it should be checked by the > configure script. It's not a requirement when building from a release tar file, since then the files generated by bison are already included. > Note that neither yacc nor bison is mentioned as needed by "README.HG", > in case this is specific to the use of the hg repository. You're right that bison should be listed in README.HG. Regards, /Niels -- Niels M?ller. PGP key CB4962D070D77D7FCB8BA36271D8F1FF368C6677. Internet email is subject to wholesale government surveillance. From tg at gmplib.org Wed May 15 21:00:11 2024 From: tg at gmplib.org (=?utf-8?Q?Torbj=C3=B6rn_Granlund?=) Date: Wed, 15 May 2024 21:00:11 +0200 Subject: Fix for BMI2 detection for Intel CPUs in fat builds In-Reply-To: (Sven Anderson's message of "Fri, 3 May 2024 00:26:21 +0200") References: Message-ID: <868r0adhok.fsf@shell.gmplib.org> Thanks, your proposed patch has been committed. -- Torbj?rn Please encrypt, key id 0xC8601622 From albin.ahlback at gmail.com Thu May 16 14:38:28 2024 From: albin.ahlback at gmail.com (=?UTF-8?Q?Albin_Ahlb=C3=A4ck?=) Date: Thu, 16 May 2024 14:38:28 +0200 Subject: Reduced number of allocated limbs after calling mpz_remove Message-ID: Hello, Standing at the following commit: changeset: 18465:7ecb3b2beea1 tag: tip user: Niels M?ller date: Sun Feb 18 20:20:57 2024 +0100 summary: mini-gmp: Fix bug in gcdext canonicalization, and strengthen related tests. I have noticed that `mpz_remove` may reduce the number of allocated limbs after a call. I haven't looked into this too much, but I suppose it is due to the variable `x` being initialized in the routine, whose limbs I believe are allocated in `mpz_tdiv_q`, is then being swapped with `dest`. It is stated in the documentation that "mpz_t and mpq_t variables never reduce their allocated space.", which is sort of true given that `x` and `dest` are only being swapped, but that requires the user to know what the internals are doing. Is this the expected behavior? Have I overlooked something perhaps? Please let me know if you need more information. Best, Albin From nisse at lysator.liu.se Mon May 20 09:06:35 2024 From: nisse at lysator.liu.se (=?utf-8?Q?Niels_M=C3=B6ller?=) Date: Mon, 20 May 2024 09:06:35 +0200 Subject: Reduced number of allocated limbs after calling mpz_remove In-Reply-To: ("Albin =?utf-8?Q?Ahlb=C3=A4ck=22's?= message of "Thu, 16 May 2024 14:38:28 +0200") References: Message-ID: Albin Ahlb?ck writes: > It is stated in the documentation that "mpz_t and mpq_t variables > never reduce their allocated space.", which is sort of true given that > `x` and `dest` are only being swapped, but that requires the user to > know what the internals are doing. Whenever an mpz function is called with destination operant equal to one of the inputs, and the lower-level functions don't support in-place operation, mpz functions would need to either 1. allocate new storage for the result (violating the above property), or 2. allocate a copy of the inputs. It looks like mpz_remove does option (1), while, e.g., mpz_tdiv_qr follows option (2). I would prefer to give the implementation a bit more freedom. And mini-gmp follows (1) rather aggressively, even when there is no argument overlap. So not sure if code or docs should be adjusted ("usually not" rather than "never"). Do you think the "never reduce their allocated space" property is important for your usecase, or for other GMP applications general? If the strong statement is kept, this should perhaps be mention as another execption in mini-gmp/README. Regards, /Niels -- Niels M?ller. PGP key CB4962D070D77D7FCB8BA36271D8F1FF368C6677. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Mon May 20 16:40:20 2024 From: nisse at lysator.liu.se (=?utf-8?Q?Niels_M=C3=B6ller?=) Date: Mon, 20 May 2024 16:40:20 +0200 Subject: Reduced number of allocated limbs after calling mpz_remove In-Reply-To: ("Albin =?utf-8?Q?Ahlb=C3=A4ck=22's?= message of "Mon, 20 May 2024 14:13:49 +0200") References: Message-ID: Albin Ahlb?ck writes: > In the project I help maintaining, we have started to rely on that mpz > never reduce the number of limbs. This is to keep a minimum number of > limbs allocated to 2, so that if one where to do a multiplication of > two limbs, the result is guaranteed to fit inside the mpz without > having to do a reallocation. Thanks for explaining. To me, it seems a bit brittle to rely in this. Could you use mpz_limbs_write (x, 2) for the functions that want a result area of at least two limbs? Added in gmp-6-0.0. There's also the mpz_realloc2 function, which does something related, but which may also shrink the allocation. > Since this indeed seems to be the exception, I am fine with either > documenting that this indeed is the exception (perhaps it should then > be stated under the section "Memory management" as well as the > docstring for `mpz_remove`), or fixing this in `mpz_remove`. I don't have a strong opinion on how to fix this discrepancy between docs and implementation. But I feel rather strongly that mini-gmp shpuldn't make this promise, so code that rely on it will not work with mini-gmp. Regards, /Niels -- Niels M?ller. PGP key CB4962D070D77D7FCB8BA36271D8F1FF368C6677. Internet email is subject to wholesale government surveillance. From marco.bodrato at tutanota.com Mon May 20 20:01:07 2024 From: marco.bodrato at tutanota.com (marco.bodrato at tutanota.com) Date: Mon, 20 May 2024 20:01:07 +0200 (CEST) Subject: Reduced number of allocated limbs after calling mpz_remove In-Reply-To: References: Message-ID: Ciao, 20 mag 2024, 16:40 da nisse at lysator.liu.se: > Albin Ahlb?ck writes: > >> In the project I help maintaining, we have started to rely on that mpz >> never reduce the number of limbs. This is to keep a minimum number of >> limbs allocated to 2 >> With lazy allocation, we moved toward the opposite direction, but having at least two limbs when at least one is allocated anyway might be a good idea. > Could you use mpz_limbs_write (x, 2) for the functions that want a > result area of at least two limbs? > They can, but surely they want an already allocated couple of limbs for speed reasons, calling _limbs_write every now and then might not be in the spirit of the optimization. >> Since this indeed seems to be the exception, I am fine with either >> documenting that this indeed is the exception (perhaps it should then >> be stated under the section "Memory management" as well as the >> docstring for `mpz_remove`), or fixing this in `mpz_remove`. >> Actually the manual (https://gmplib.org/manual/Memory-Management ) reads: "mpz_t and mpq_t variables never reduce their allocated space." The mis-behaviour of mpz_remove is my fault. When I experimented with that function I tested both the "never reduce" and the "may reduce" version, but when I did commit I forgot to push the correct one. https://gmplib.org/repo/gmp/rev/596470e0bc62 As you can see, the correct code is already there, but #if-ed out because WANT_ORIGINAL_DEST is not defined. > I don't have a strong opinion on how to fix this discrepancy between > docs and implementation. > I'll correct the _remove function, it's quite easy to do :-) > But I feel rather strongly that mini-gmp > shouldn't make this promise, so code that rely on it will not work with > mini-gmp. > I agree for the mini-gmp side. ?is, mb From nisse at lysator.liu.se Mon May 20 20:58:15 2024 From: nisse at lysator.liu.se (=?utf-8?Q?Niels_M=C3=B6ller?=) Date: Mon, 20 May 2024 20:58:15 +0200 Subject: Reduced number of allocated limbs after calling mpz_remove In-Reply-To: (marco bodrato's message of "Mon, 20 May 2024 20:01:07 +0200 (CEST)") References: Message-ID: marco.bodrato at tutanota.com writes: >> I don't have a strong opinion on how to fix this discrepancy between >> docs and implementation. > > I'll correct the _remove function, it's quite easy to do :-) Nice! To avoid regressions, it would be good to have tests for this. Could possibly go in tests/mpz/reuse.c, but looks like a fair amount of work to do properly. >> But I feel rather strongly that mini-gmp shouldn't make this promise, >> so code that rely on it will not work with mini-gmp. > > I agree for the mini-gmp side. Then it should be documented as an exception in mini-gmp/README. Suggested update below. Regards, /Niels --- a/mini-gmp/README Wed May 15 20:51:11 2024 +0200 +++ b/mini-gmp/README Mon May 20 20:53:53 2024 +0200 @@ -39,21 +39,24 @@ mini-gmp as a fallback when for some rea not desired as a dependency. The supported GMP subset of the mpn and mpz interfaces is declared in -mini-gmp.h, and implemented in mini-gmp.c. The implemented -functions are fully compatible with the corresponding GMP functions, -as specified in the GMP manual, with a few exceptions: +mini-gmp.h, and implemented in mini-gmp.c. The supported GMP subset of +the mpq layer is declared in mini-mpq.h, and implemented in +mini-mpq.c. The implemented functions are fully compatible with the +corresponding GMP functions, as specified in the GMP manual, with a +few exceptions: mpz_export and mpz_import support only NAILS = 0. + mini-gmp's handling of destinations operands does not satisfy the + documented property that 'mpz_t' and 'mpq_t' variables never reduce + their allocated space. + The performance target for mini-gmp is to be at most 10 times slower than the real GMP library, for numbers of size up to a few hundred bits. No asymptotically fast algorithms are included in mini-gmp, so it will be many orders of magnitude slower than GMP for very large numbers. -The supported GMP subset of the mpq layer is declared in mini-mpq.h, -and implemented in mini-mpq.c. - You should never "install" mini-gmp. Applications can either just #include mini-gmp.c (but then, beware that it defines several macros and functions outside of the advertised interface), and if needed -- Niels M?ller. PGP key CB4962D070D77D7FCB8BA36271D8F1FF368C6677. Internet email is subject to wholesale government surveillance. From albin.ahlback at gmail.com Mon May 20 14:13:49 2024 From: albin.ahlback at gmail.com (=?UTF-8?Q?Albin_Ahlb=C3=A4ck?=) Date: Mon, 20 May 2024 14:13:49 +0200 Subject: Reduced number of allocated limbs after calling mpz_remove In-Reply-To: References: Message-ID: Hejsan Niels, > So not sure if code or docs should be adjusted ("usually not" rather > than "never"). Do you think the "never reduce their allocated space" > property is important for your usecase, or for other GMP applications > general? If the strong statement is kept, this should perhaps be mention > as another execption in mini-gmp/README. In the project I help maintaining, we have started to rely on that mpz never reduce the number of limbs. This is to keep a minimum number of limbs allocated to 2, so that if one where to do a multiplication of two limbs, the result is guaranteed to fit inside the mpz without having to do a reallocation. Of course, the memory manager could check that recycled mpzs do have at least 2 limbs allocated, but that seems unnecessary since `mpz_remove` seems to be the exception. Since this indeed seems to be the exception, I am fine with either documenting that this indeed is the exception (perhaps it should then be stated under the section "Memory management" as well as the docstring for `mpz_remove`), or fixing this in `mpz_remove`. Best, Albin From albin.ahlback at gmail.com Mon May 20 17:33:03 2024 From: albin.ahlback at gmail.com (=?UTF-8?Q?Albin_Ahlb=C3=A4ck?=) Date: Mon, 20 May 2024 17:33:03 +0200 Subject: Reduced number of allocated limbs after calling mpz_remove In-Reply-To: References: Message-ID: On 5/20/24 4:40 PM, Niels M?ller wrote:> Could you use mpz_limbs_write (x, 2) for the functions that want a > result area of at least two limbs? Added in gmp-6-0.0. There's also the > mpz_realloc2 function, which does something related, but which may also > shrink the allocation. We do utilize the internal function mpz_realloc for almost everything as we mostly work with limbs instead of bits. Just to note, we have "patched" every call to mpz_remove so that the returning mpz does not have less limbs than we require. Best, Albin P.S. Looks like my previous email didn't make it to the list. From tg at gmplib.org Tue May 21 07:30:47 2024 From: tg at gmplib.org (=?utf-8?Q?Torbj=C3=B6rn_Granlund?=) Date: Tue, 21 May 2024 07:30:47 +0200 Subject: Reduced number of allocated limbs after calling mpz_remove In-Reply-To: ("Niels =?utf-8?Q?M?= =?utf-8?Q?=C3=B6ller=22's?= message of "Mon, 20 May 2024 16:40:20 +0200") References: Message-ID: <868r03ybnc.fsf@shell.gmplib.org> Niels M?ller writes: I don't have a strong opinion on how to fix this discrepancy between docs and implementation. But I feel rather strongly that mini-gmp shpuldn't make this promise, so code that rely on it will not work with mini-gmp. It is, in my opinion, not a bug in the code, nor is is a bug in the documentation. What we indended to say in the docs is that a variable which once contained something large then will occupy lots of RAM even if is later is used for only small things. (This in turn was an optimisation as trimming the allocation after each operaton would slow down GMP very, very much.) That's the only thing we should promise, as it has obvious exceptions. It is possible that the docs need clarification. Perhaps a "in most cases" can be inserted somewhere? -- Torbj?rn Please encrypt, key id 0xC8601622 From nisse at lysator.liu.se Tue May 21 08:17:35 2024 From: nisse at lysator.liu.se (=?utf-8?Q?Niels_M=C3=B6ller?=) Date: Tue, 21 May 2024 08:17:35 +0200 Subject: Reduced number of allocated limbs after calling mpz_remove In-Reply-To: <868r03ybnc.fsf@shell.gmplib.org> (=?utf-8?Q?=22Torbj=C3=B6rn?= Granlund"'s message of "Tue, 21 May 2024 07:30:47 +0200") References: <868r03ybnc.fsf@shell.gmplib.org> Message-ID: Torbj?rn Granlund writes: > It is possible that the docs need clarification. I think so, we should not write "never" if that's not what we mean. My understanding is that allocation grows as needed, and is not returned to the the system until mpz_clear. Except that allocation can shrink only as the effect of mpz_realloc2 (never called internally) or mpz_swap (called internally in few places). But we also shouldn't over-allocate things. It could perhaps be described like this: Each mpz output for a gmp function has an "expected size" depending on the sizes of the inputs. E.g., excpected size of the mpz_add output is the size of the largest input + 1, while actual size may be as small as zero, due to cancellation. On return, the allocation size of of an mpz_t output parameter should either be unchanged (available storage could be reused, without any reallocation, pointer would typically be unchanged too in this case), or at most a limb or two larger than the expected output size for the operation. E.g., mpz_powm internally needs storage for squares (twice the expected output size), but it shouldn't inflate the allocation of the output parameter to that size. To the GMP application, the allocation size of an mpz_t is then bounded by the largest "expected size" for any operation ever using that mpz_t as output (+ effects of any mpz_swap or mpz_realloc2 in the *application* code), which should be sufficient to get predictable memory usage. Regards, /Niels -- Niels M?ller. PGP key CB4962D070D77D7FCB8BA36271D8F1FF368C6677. Internet email is subject to wholesale government surveillance. From luc at inspiren.com Mon May 27 22:11:02 2024 From: luc at inspiren.com (Luc des Trois Maisons) Date: Mon, 27 May 2024 16:11:02 -0400 Subject: make check test failures on (MacOS 14.5, M3 pro, Xcode 15.3) Message-ID: Hello, This doesn't appear to be listed in the known build problems, or the Notes for particular systems. Apologies in advance if duplicate. When building GMP on an M3-based Macbook Pro with MacOS Sonoma 14.5, and Xcode 15.3 (15E204a), the majority of the test suite covered by make check appears to be failing: ============================================================================ Testsuite summary for GNU MP 6.2.1 ============================================================================ # TOTAL: 50 # PASS: 16 # SKIP: 0 # XFAIL: 0 # FAIL: 34 # XPASS: 0 # ERROR: 0 ============================================================================ See tests/mpn/test-suite.log Please report to gmp-bugs at gmplib.org, see https://gmplib.org/manual/Reporting-Bugs.html ============================================================================ As a previous step, I had built and installed GNU m4 v1.4.19 with the following configure invocation: ~/development/gcc/m4-1.4.19/configure --prefix=$HOME/development/gcc Attempting to build GMP as follows: % PATH=$HOME/development/gcc/bin:$PATH ~/development/gcc/gmp-6.2.1/configure CC=clang --prefix=$HOME/development/gcc % PATH=$HOME/development/gcc/bin:$PATH make -j4 % PATH=$HOME/development/gcc/bin:$PATH make check % PATH=$HOME/development/gcc/bin:$PATH ~/development/gcc/gmp-6.2.1/config.guess aarch64-apple-darwin23.5.0 % PATH=$HOME/development/gcc/bin:$PATH ~/development/gcc/gmp-6.2.1/configfsf.guess aarch64-apple-darwin23.5.0 % uname -a Darwin Statik.lan 23.5.0 Darwin Kernel Version 23.5.0: Wed May 1 20:13:18 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6030 arm64 % clang --version Apple clang version 15.0.0 (clang-1500.3.9.4) -------------- next part -------------- A non-text attachment was scrubbed... Name: configure.out Type: application/octet-stream Size: 34869 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test-suite.log.gz Type: application/x-gzip Size: 4944 bytes Desc: not available URL: From nisse at lysator.liu.se Tue May 28 13:29:44 2024 From: nisse at lysator.liu.se (=?utf-8?Q?Niels_M=C3=B6ller?=) Date: Tue, 28 May 2024 13:29:44 +0200 Subject: make check test failures on (MacOS 14.5, M3 pro, Xcode 15.3) In-Reply-To: (Luc des Trois Maisons's message of "Mon, 27 May 2024 16:11:02 -0400") References: Message-ID: Luc des Trois Maisons writes: > This doesn't appear to be listed in the known build problems, or the > Notes for particular systems. Apologies in advance if duplicate. My guess it's a known issue with Apple ARM systems and gmp-6.2.1 (see https://gmplib.org/#STATUS). Please try latest release, 6.3.0. Regards, /Niels -- Niels M?ller. PGP key CB4962D070D77D7FCB8BA36271D8F1FF368C6677. Internet email is subject to wholesale government surveillance. From albin.ahlback at gmail.com Tue May 28 13:33:25 2024 From: albin.ahlback at gmail.com (=?UTF-8?Q?Albin_Ahlb=C3=A4ck?=) Date: Tue, 28 May 2024 13:33:25 +0200 Subject: make check test failures on (MacOS 14.5, M3 pro, Xcode 15.3) In-Reply-To: References: Message-ID: Hello, You are not using the latest GMP version; it is a known issue for GMP 6.2.1 [1]. GMP 6.3.0 fixes this [2]. [1]: https://gmplib.org/#STATUS [2]: https://gmplib.org/gmp6.3 Best, Albin