This gives the caller more control about whether the state should
be cleaned (= should be considered secret). Moreover, it gives the
caller the possibility to clean a hash struct without finalizing it.
All of the invocations of secp256k1_memclear() operate on stack
memory and happen after the function is done with the memory object.
This commit replaces existing memset() invocations and also adds
secp256k1_memclear() to code locations where clearing was missing;
there is no guarantee that this commit covers all code locations
where clearing is necessary.
Co-Authored-By: isle2983 <isle2983@yahoo.com>
There are two uses of the secp256k1_fe_clear() function that are now separated
into these two functions in order to reflect the intent:
1) initializing the memory prior to being used -> converted to fe_set_int( . , 0 )
2) zeroing the memory after being used such that no sensitive data remains. ->
remains as fe_clear()
In the latter case, 'magnitude' and 'normalized' need to be overwritten when
VERIFY is enabled.
Co-Authored-By: isle2983 <isle2983@yahoo.com>
We rely on memset() and an __asm__ memory barrier where it's available or
on SecureZeroMemory() on Windows. The fallback implementation uses a
volatile function pointer to memset which the compiler is not clever
enough to optimize.
We should anyway prefer to use the predefined macros from <inttypes.h>.
If I haven't missed anything, this removes the last OS-specific #if,
leaving us only with compiler-specific #if(def)s.
096e3e23f6 ci: Update macOS image (Hennadii Stepanov)
Pull request description:
The macOS 12 GHA image has been deprecated since 2024-10-07. See: https://github.com/actions/runner-images/issues/10721.
Draft for now as `./libtool --mode=execute valgrind --error-exitcode=42 ./ctime_tests` fails.
ACKs for top commit:
real-or-random:
ACK 096e3e23f6
Tree-SHA512: 715e7d2638bb7161c756d3856ee7eb6826f2300ab215deb888f040881c6b8cddc311c206f90dd942844ee2e56247e8ca99078a229e80ef086c2a4fdd8937af9d
57eda3ba30 musig: ctimetests: fix _declassify range for generated nonce points (Sebastian Falbesoner)
Pull request description:
As noticed in https://github.com/bitcoin-core/secp256k1/pull/1614#discussion_r1796215582, the area marked as non-secret exceeds the nonce_pts array in the second iteration of the for loop. Fix that by passing the correct size to the _declassify call.
ACKs for top commit:
sipa:
utACK 57eda3ba30
real-or-random:
utACK 57eda3ba30
Tree-SHA512: ff8074e3d1078d66a52d08c661997856ff586b3b4564a865a75212b32fafd7906d58885371bd63005007fde554ebcad121ab66125abe4331cf0aac63fc018ed0
447334cb06 include: Avoid visibility("default") on Windows (Tim Ruffing)
Pull request description:
Fixes#1421. See code comments for rationale.
Related meta-bug: #1181. This reminds me that we should move forward with #1359.
ACKs for top commit:
fanquake:
ACK 447334cb06
hebasto:
ACK 447334cb06, tested on Ubuntu 24.04 using the following commands:
theuni:
ACK 447334cb06
Tree-SHA512: aaa47d88fd1b1f85c3e879a2b288c0eb3beebad0cc89e85f05d0b631f83e58d5a324fb441911970865eaa292f6820d03a1b516d6e8de37a87510e2082acc6e28
8be3839fb2 Remove unused scratch space from API (Jonas Nick)
Pull request description:
We had already merged this in #1305, but it was reverted before a release (#1311) because this change is not backwards compatible but at the time we only wanted to make a patch release in order to fix an actual issue.
Due to the musig module, the next release will increment the version number from 0.5.x to 0.6.0, so it would be a good time to remove the scratch space from the API.
ACKs for top commit:
sipa:
utACK 8be3839fb2
real-or-random:
utACK 8be3839fb2
Tree-SHA512: ecd6bc1d925992f9df8e26820388fc436bbb6bc5f250950edf00406f006ca0df52ab8cd56a1b7541e57af0682ddadf6d34bd638b27557d301a5dff6c327a5ebc
ef7ff03407 f can never equal -m (Russell O'Connor)
Pull request description:
In fact, before reaching this particular VERIFY_CHECK, we had already successfully passed through
VERIFY_CHECK(secp256k1_modinv64_mul_cmp_62(&f, len, &modinfo->modulus, -1) > 0); /* f > -modulus */
ensuring that f is not -m.
ACKs for top commit:
sipa:
ACK ef7ff03407
real-or-random:
utACK ef7ff03407
Tree-SHA512: a8a8dcbad4dff36b9c49e40e07b212312cbf915132aea008eab6ea61b35bddb6d7782229c2cc528fb404d05132482c602cad768414d76153bb425a3d23714fff
168c92011f build: allow enabling the musig module in cmake (Jonas Nick)
f411841a46 Add module "musig" that implements MuSig2 multi-signatures (BIP 327) (Jonas Nick)
0be79660f3 util: add constant-time is_zero_array function (Jonas Nick)
c8fbdb1b97 group: add ge_to_bytes_ext and ge_from_bytes_ext (Jonas Nick)
85e224dd97 group: add ge_to_bytes and ge_from_bytes (Jonas Nick)
Pull request description:
EDIT: based on #1518. Closes#1452. Most of the code is a copy from [libsecp256k1-zkp](https://github.com/BlockstreamResearch/secp256k1-zkp). The API added in this PR is identical with the exception of two modifications:
1. I removed the unused `scratch_space` argument from `secp256k1_musig_pubkey_agg`. This argument was intended to allow using `ecmult_multi` algorithms for key aggregation in the future. But at this point it's unclear whether the `scratch_space` object will remain in its current form (see #1302).
2. Support for adaptor signatures was removed and therefore the `adaptor` argument of `musig_nonce_process` was also removed.
In contrast to the module in libsecp256k1-zkp, the module is non-experimental. I slightly cleaned up parts of the module, adjusted the code to the new definition of the VERIFY_CHECK macro and applied some simplifications that were possible because the module is now in the upstream repo (`ge_from_bytes`, `ge_to_bytes`). You can follow the changes I made to the libsecp256k1-zkp module at https://github.com/jonasnick/secp256k1-zkp/commits/musig2-upstream/.
ACKs for top commit:
sipa:
reACK 168c92011f
real-or-random:
reACK 168c92011f
theStack:
re-ACK 168c92011f
Tree-SHA512: e3a599a8d5a466107b9a86f76582b8fb9dc87ec95416c784c3ef39d1c64686e6c739806ed6ba62c91793eb7fa418a6270cf999027ee7bd3dd85c67bc2c74f677
7c987ec89e cmake: Call `enable_testing()` unconditionally (Hennadii Stepanov)
6aa576515e cmake: Delete `CTest` module (Hennadii Stepanov)
Pull request description:
1. Delete `CTest` module.
The `CTest` module handles `CDash` integration, which we do not use. It is not required for testing functionality.
2. Clean up cases when to invoke `enable_testing()`
The `enable_testing()` command invocation is required for `add_test()` commands, which are used only for `{noverify_}tests`, `exhaustive_tests` and examples.
ACKs for top commit:
real-or-random:
utACK 7c987ec89e
theStack:
ACK 7c987ec89e
Tree-SHA512: f2628a4c4363b86f7c49d705df192e76368997e3f01cb736fcc272330bbf481af023d29d9faec17878d3bf74e4d265870cd93953406ae4612cd717811f22dd1a
292310fbb2 doc: fix typos in `secp256k1_ecdsa_{recoverable_,}signature` API description (Sebastian Falbesoner)
Pull request description:
This small PR fixes two small typos I noticed while looking at the pubkey recovery module (s/structured/structure/).
ACKs for top commit:
real-or-random:
utACK 292310fbb2
Tree-SHA512: 7abda2d5b56d810fd83c7de9b018818a28fd5969f669b53b9dc0e1034428feef63a3157714f45002185b94c6c35617cab21861fa56dba51883d1326b4a1cb033
fa67b6752d refactor: Use array initialization for unterminated strings (MarcoFalke)
e34b476730 ci: Bump GCC_SNAPSHOT_MAJOR to 15 (maflcko)
Pull request description:
Follow-up to https://github.com/bitcoin-core/secp256k1/pull/1313
Clang should silently follow the `main` devel branch, but GCC needs to be bumped manually.
ACKs for top commit:
hebasto:
ACK fa67b6752d, I have reviewed the code and it looks OK.
real-or-random:
utACK fa67b6752d
Tree-SHA512: e76371e5b1ff259ec501671872352c0d46d34a96aadae04e6ee37f9457308412e18010e724df667a15c3a85997a16da191f50cd3a01ee3f20d5f16b5893d179a
7057d3c9af ci: Silent Homebrew's noisy reinstall warnings (Hennadii Stepanov)
Pull request description:
Homebrew's warnings are quite noisy on the master branch:
This PR silents them to allow us to focus on any other CI infra warnings once they happen.
ACKs for top commit:
real-or-random:
utACK 7057d3c9af
Tree-SHA512: 4dbdb459e97cb5956219214fd7407ec52fc018dba24ae945337ad5c94df510472cfc708f9f67d4df5b8e983bf4e78b2efd56533f28113b3463a1397d4d7a032a
e2af491263 ci: Switch to the new default value of the precomputed table for signing (Hennadii Stepanov)
d94a9273f8 build: Adjust the default size of the precomputed table for signing (Hennadii Stepanov)
Pull request description:
This PR implements the [outcomes](https://github.com/bitcoin-core/secp256k1/issues/1549#issuecomment-2200559257) from today's IRC meeting:
1. The default size of the precomputed table for signing is now aligned with Bitcoin Core's [default](https://github.com/bitcoin/bitcoin/commit/a057869aa3c42457570765966cb66accb2375b13).
2. The default value in CI has been updated to reflect the new default.
ACKs for top commit:
sipa:
utACK e2af491263
real-or-random:
utACK e2af491263
Tree-SHA512: aa9db5bc2aec29a35a503a80617a4c096e9909648084fe1ce43b5dd7e74dd812e7642305bd5bc13eb581efc23f12904e200e13cb1a35955b773e05ab4f84be4e
9420eece24 cmake: Bump CMake minimum required version up to 3.16 (Hennadii Stepanov)
Pull request description:
Debian 10 [reached](https://wiki.debian.org/DebianReleases) EOL LTS yesterday, on 2024-06-30.
There no longer seem to be compelling reasons to maintain support for CMake 3.13.
The suggested minimum required version, CMake 3.16, is shipped with Ubuntu 20.04 LTS, which is [supported](https://wiki.ubuntu.com/Releases) until April 2025.
Debian 11 ships with CMake 3.18 (3.25 in backports). In [other](https://repology.org/project/cmake/versions) major distros and package managers, CMake versions are not older.
ACKs for top commit:
fanquake:
ACK 9420eece24 - If the only significant change is for some MSVC compat code, this doesn't seem super compelling, but seems ok either way.
real-or-random:
utACK 9420eece24
Tree-SHA512: 7066f053f9820f9ddeb27f9e8042edb67a31cd61e29eadff12b50d63bbe278f6ffd15f84035932bc4c337461a7eb342339ed62e46e100c5447106a702fad229c
b8fe33332b cmake: Fixed O3 replacement (Eduardo Menges Mattje)
Pull request description:
Old replacement of `O3` in `CMAKE_C_FLAGS_RELEASE` skip spaces, which is problematic. For instance, if `CMAKE_C_FLAGS_RELEASE = "-O3 -DFOO"`, regex will replace it with `-O2-DFOO`, which causes a compile error.
This patch changes this behavior, keeping whichever space exists between the flags.
If I may question, what is the rationale behind replacing `O3` with `O2`? Changing the user's flags is a bad practice overall, and I don't see how this replacement is beneficial.
ACKs for top commit:
hebasto:
re-ACK b8fe33332b.
Tree-SHA512: d84091eb594af695805784e87f49d38ac08a7cf42978c844383882196389a0d83fc5f72e6394887b9961cc3f96538962f4ee86ef3eeca2c96509c64428046c70
4d9645bee0 cmake: Remove "AUTO" value of `SECP256K1_ECMULT_GEN_KB` option (Hennadii Stepanov)
a06805ee74 cmake: Remove "AUTO" value of `SECP256K1_ECMULT_WINDOW_SIZE` option (Hennadii Stepanov)
26b94ee92a autotools: Remove "auto" value of `--with-ecmult-gen-kb` option (Hennadii Stepanov)
122dbaeb37 autotools: Remove "auto" value of `--with-ecmult-window` option (Hennadii Stepanov)
Pull request description:
"auto" implies that a value is being chosen based on build system introspection or host system capabilities. However, for the `--with-ecmult-window` and `--with-ecmult-gen-kb` options, the values "auto" are hardcoded, which might lead to confusion.
This PR replaces "auto" with more appropriate default values.
If Concept ACKed, I'll add equivalent commits for CMake.
ACKs for top commit:
sipa:
utACK 4d9645bee0
real-or-random:
utACK 4d9645bee0 good from my side, but let's see if we can get more (Concept) ACKs
Tree-SHA512: 9e68f73682c5310c68d2337594f13b99a52bfc365564e39df2e412b576635c90cccd2298406a4281f014916c4a1710e19c7390f05a4b0acbd09869bfb56f36ac
158f9e5eae cmake: Do not modify build types when integrating by downstream project (Hennadii Stepanov)
Pull request description:
The `CMAKE_BUILD_TYPE` and `CMAKE_CONFIGURATION_TYPES` must be managed by the downstream project.
Suggesting to review with `git diff -w`.
Fixes `std::out_of_range` exception from CMake in https://github.com/hebasto/bitcoin/pull/192 when running configuration step using "Ninja Multi-Config" generator:
```
$ cmake -B build -G "Ninja Multi-Config"
...
-- Configuring done (17.1s)
terminate called after throwing an instance of 'std::out_of_range'
what(): map::at
Aborted (core dumped)
```
Here are related discussions:
- https://discourse.cmake.org/t/uncaught-exception-when-trying-to-generate-a-project-using-ninja-multi-config/11051
- https://gitlab.kitware.com/cmake/cmake/-/issues/26064
ACKs for top commit:
real-or-random:
ACK 158f9e5eae
Tree-SHA512: b3040f40438d530f14b7e0f7d523e74b5843d88d250ff7955a99cc8c451feb9471a48134d1a89b3651b3f8195f91c17135c7b8a5d3ab092c8d35275b57743b8c
4706be2cd0 cmake: Reimplement `SECP256K1_APPEND_CFLAGS` using Bitcoin Core approach (Hennadii Stepanov)
c2764dbb99 cmake: Rename `SECP256K1_LATE_CFLAGS` to `SECP256K1_APPEND_CFLAGS` (Hennadii Stepanov)
Pull request description:
This PR address this https://github.com/hebasto/bitcoin/issues/239#issuecomment-2182713690:
> For consistency with libsecp256k1:
>
> > > Is this code block supposed to achieve the same as our `SECP256K1_LATE_CFLAGS` (implemented by a user-defined function `all_targets_add_compile_options`) in libsecp256k1?
> >
> >
> > It is. But this approach guaranties to override even options that are abstracted by CMake, for instance [#157 (comment)](https://github.com/hebasto/bitcoin/pull/157#issuecomment-2090465123).
>
> * If we agree that appending to rule variables is superior, should we also do this in libsecp256k1?
>
> * And/or should we rename the `SECP256K1_LATE_CFLAGS` variable to `APPEND_CFLAGS`?
ACKs for top commit:
real-or-random:
utACK 4706be2cd0
Tree-SHA512: 24603886c4d6ab4e31836a67d5759f9855a60c6c9d34cfc6fc4023bd309cd51c15d986ac0b77a434f9fdc6d5e97dcd3b8484d8f5ef5d8f840f47dc141de18084
f87a3589f4 cmake: Do not set `CTEST_TEST_TARGET_ALIAS` (Hennadii Stepanov)
Pull request description:
An alias for the "test" target can be confusing for the downstream project.
For instance, when integrating using `add_subdirectory(secp256k1 EXCLUDE_FROM_ALL)` (see https://github.com/hebasto/bitcoin/pull/192), test binaries are not being built by default. But the `check` alias target is exposed to the downstream project build system, which in turn fails:
```
$ make -C build check
...
Unable to find executable: /home/hebasto/git/bitcoin/build/src/secp256k1/src/exhaustive_tests
3/3 Test #3: exhaustive_tests .................***Not Run 0.00 sec
0% tests passed, 3 tests failed out of 3
Total Test time (real) = 0.03 sec
The following tests FAILED:
1 - noverify_tests (Not Run)
2 - tests (Not Run)
3 - exhaustive_tests (Not Run)
Errors while running CTest
...
```
This PR fixes this issue by deleting the `CTEST_TEST_TARGET_ALIAS` usage.
ACKs for top commit:
real-or-random:
utACK https://github.com/bitcoin-core/secp256k1/pull/1545/commits/f87a3589f4d3bfd7a398232971bdb1ff125b8e9d
Tree-SHA512: ccf3f30939cf1747471ea15260f7caa6dad3f510e5771245ecbfbef3cc0b0e7c8ac551519d0892bf2544c91467d8d67d2c6e6bc52f56c384b174b88bcf377d4a
ec4c002faa cmake: Simplify `PROJECT_IS_TOP_LEVEL` emulation (Hennadii Stepanov)
cae9a7ad14 cmake: Do not set emulated PROJECT_IS_TOP_LEVEL as cache variable (Hennadii Stepanov)
Pull request description:
As CMake's cache is a global database, modifying it within a project integrated with the `add_subdirectory()` command, which may also include using the `FetchContent` module, could potentially affect downstream projects and sibling ones.
ACKs for top commit:
real-or-random:
utACK ec4c002faa
theuni:
utACK ec4c002faa
Tree-SHA512: de2c8c583367028d06701f79fc5232b351622c8496d196aad8c22a1ec4e450af53e556a4f6526ed47250f818143a69a12f5fc8cc755c864510e67530dacde66e
e73f6f8fd9 tests: refactor: drop `secp256k1_` prefix from testrand.h functions (Sebastian Falbesoner)
0ee7453a99 tests: refactor: add `testutil_` prefix to testutil.h functions (Sebastian Falbesoner)
0c6bc76dcd tests: refactor: move `random_` helpers from tests.c to testutil.h (Sebastian Falbesoner)
0fef8479be tests: refactor: rename `random_field_element_magnitude` -> `random_fe_magnitude` (Sebastian Falbesoner)
59db007f0f tests: refactor: rename `random_group_element_...` -> `random_ge_...` (Sebastian Falbesoner)
Pull request description:
This PR is an attempt at tidying up test util functions, as suggested in #1491. The following changes are done:
* rename `_group_element...` functions to `_ge...`
* rename `_field_element...` functions to `_fe...`
* move `random_` helpers from tests.c to testutil.h (the alternative would be testrand.h, but to my understanding, this one is meant to contain the actual RNG implementation rather than helpers using it; happy to move the helpers there if that is preferred though)
* prefix testutil.h functions with `testutil_`
* prefix testrand.h functions with `testrand_` (this is currently done in a sloppy way by simply dropping the `secp256k1_` prefix, so some functions don't have the full prefix, like e.g. `testrand256`; naming suggestions welcome)
ACKs for top commit:
sipa:
utACK e73f6f8fd9
real-or-random:
utACK e73f6f8fd9
Tree-SHA512: c87a35a9f7f23d4bbb87a1ff0d40dd5fbd7d976719ca1027cad187ac44aa2db3ae887ac620639d2287c260e701a5963830b52048692d3e6b38b5eb6cdf17b854
f55703ba49 autotools: Delete unneeded compiler test (Hennadii Stepanov)
396e885886 autotools: Align MSan checking code with CMake's implementation (Hennadii Stepanov)
abde59f52d cmake: Report more compiler details in summary (Hennadii Stepanov)
7abf979a43 cmake: Disable `ctime_tests` if build with `-fsanitize=memory` (Hennadii Stepanov)
Pull request description:
Same as https://github.com/bitcoin-core/secp256k1/pull/1517, but for the CMakle build system.
The second commit improves the configure summary (similar to https://github.com/hebasto/bitcoin/pull/189.
ACKs for top commit:
real-or-random:
ACK f55703ba49
Tree-SHA512: 18190c062ae6e27d0ecbe7460cc22c960b25c0d35aa4b94f151d4b1c48f16e99fd5ecdfcb359784f95995292633d30d3d23b75a12be3aca5afffcc1e7e7daf31
ebfb82ee2f ci: Add job with -fsanitize-memory-param-retval (Tim Ruffing)
e1bef0961c configure: Move "experimental" warning to bottom (Tim Ruffing)
55e5d975db autotools: Disable eager MSan in ctime_tests (Tim Ruffing)
Pull request description:
This is the autotools solution for #1516.
Alternatively, we could have a full-blown `--enable-msan` option, but it's more work, and I'm not convinced that it's necessary or at least much better.
hebasto If you're Concept ACK, are you willing to work on an equivalent PR for CMake?
ACKs for top commit:
hebasto:
ACK ebfb82ee2f, tested on Ubuntu 24.04 with different clang versions (from 15 to 18) and different build configurations. CI changes look OK as well.
Tree-SHA512: c083d778fd50bd35c2e29b7fe0d92b98d912ee5ac7809ae73067d050a0d3c42b3483260f1286d0023cdb802a3c3006bf932ecf60ce81b942de1c9824374c0132
9554362b15 tests: call secp256k1_ecmult_multi_var with a non-NULL error callback (Nicolas Iooss)
Pull request description:
Hello,
This Pull Request fixes the issue reported in https://github.com/bitcoin-core/secp256k1/issues/1527. Function `secp256k1_ecmult_multi_var` expects to be called with a non-`NULL` `error_callback` parameter. Fix the invocation in `test_ecmult_accumulate` to do this. While at it, wrap the call in a `CHECK` macro to ensure it succeeds.
ACKs for top commit:
real-or-random:
utACK 9554362b15
siv2r:
ACK 9554362, I have also verified that other invocations of `ecmult_multi_var` (in tests) don’t use `NULL` for the error callback function argument.
Tree-SHA512: 6a9f6c10c575794da75f2254d6fbbc195de889c81a371ce35ab38e2e5483aa1e25ec0bcd5aa8d6a32a1493586f73430208a4bd0613e373571d2f04d63dbc4a1c
9f4c8cd730 cmake: Fix `check_arm32_assembly` when using as subproject (Hennadii Stepanov)
Pull request description:
When integrating libsecpk1 in a downstream project like this:
```cmake
set(SECP256K1_ASM arm32 CACHE STRING "" FORCE)
add_subdirectory(src/secp256k1)
```
it fails to configure:
```
CMake Error at /home/hebasto/git/bitcoin/build/check_arm32_assembly/CMakeFiles/CMakeTmp/CMakeLists.txt:21 (target_sources):
Cannot find source file:
/home/hebasto/git/bitcoin/cmake/source_arm32.s
CMake Error at /home/hebasto/git/bitcoin/build/check_arm32_assembly/CMakeFiles/CMakeTmp/CMakeLists.txt:20 (add_executable):
No SOURCES given to target: cmTC_d0f0b
CMake Error at src/secp256k1/cmake/CheckArm32Assembly.cmake:2 (try_compile):
Failed to generate test project build system.
Call Stack (most recent call first):
src/secp256k1/CMakeLists.txt:127 (check_arm32_assembly)
```
This PR fixes this issue, which was overlooked in https://github.com/bitcoin-core/secp256k1/pull/1304.
ACKs for top commit:
real-or-random:
utACK 9f4c8cd730
theuni:
utACK 9f4c8cd730
Tree-SHA512: 47d97ad0fb2e3779523c2111ea75906671a0fb3f50646e29dee195f53106ace69af5e4abc92c765f0eee6973528ce9195b94377d0157209230c958894d4049fb
dd695563e6 check-abi: explicitly provide public headers (Jonas Nick)
Pull request description:
Without this commit, the check-abi shell script outputs false positives because it consider some headers public that are actually not public.
ACKs for top commit:
real-or-random:
ACK https://github.com/bitcoin-core/secp256k1/pull/1524/commits/dd695563e68d605b837b7813ae7bfafbf866ddff
hebasto:
ACK dd695563e6, tested on Ubuntu 24.04.
Tree-SHA512: b26e61639061f5fbbdd47569ba04f91c627feeefc43ec3d529a3ac4012ab6487aa1904bd38100ed190dcaebdffe60895a8c99346720d5dee84a0c457ec3b6f94
f7f0184ba1 msan: notate more variable assignments from assembly code (Cory Fields)
a61339149f change inconsistent array param to pointer (Cory Fields)
Pull request description:
This was missed in 31ba404944 because older versions of clang did not complain about it. But clang-17, at least, does.
The array-as-a-param makes this annoying because `sizeof(l)` is not helpful. I'd be happy to change the size calculation if there are any better suggestions or strong preferences.
ACKs for top commit:
sipa:
utACK f7f0184ba1
real-or-random:
ACK f7f0184ba1 tests work fine with clang 17 and `./configure CFLAGS="-fsanitize=memory -fsanitize-memory-track-origins=2 -fno-omit-frame-pointer -g -O1 -fno-optimize-sibling-calls" CC=clang`
Tree-SHA512: 8ab22209ef322a10f500b123c82ae5e7141ae1da0e7a890cbf90bd7d2eb11f397db4ccfe15a1666f2f49228585cccbf5bec741effebd1e2c6012cb7ea1689675
a5e8ab2484 ci: Add sanitizer env variables to debug output (Tim Ruffing)
84a93de4d2 ci: Add workaround for ASLR bug in sanitizers (Tim Ruffing)
Pull request description:
Fixes#1506.
This also adds the sanitizer env variables to our debug output as suggested in the same issue.
ACKs for top commit:
sipa:
utACK a5e8ab2484
jonasnick:
ACK a5e8ab2484
Tree-SHA512: 5162d14eeec01e088c600ed77e21c5ffd4dec23327b7e81b5ecac59b7c535cac97cd7b7b744c767766036dfc6d9152a9933eb326cf4065d56c46e2ee858da662
31ba404944 msan: notate variable assignments from assembly code (Cory Fields)
e7ea32e30a msan: Add SECP256K1_CHECKMEM_MSAN_DEFINE which applies to memory sanitizer and not valgrind (Cory Fields)
Pull request description:
msan isn't smart enough to see that these are set without some help.
This was pointed out here: https://github.com/bitcoin-core/secp256k1/pull/1169#issuecomment-1370003449
With this commit, msan output is clean even with x86 asm turned on.
ACKs for top commit:
real-or-random:
utACK 31ba404944
hebasto:
re-ACK 31ba404944.
Tree-SHA512: c9c51fe542247e1e0a93f6d0063d119cf777ca8c1b7e9c8e45e168a2020dc503872eb2a78004725de81267a3ce78c923be1f8546fb92a3e95fc7ef034e5ba932
42f8c51402 cmake: Add `SECP256K1_LATE_CFLAGS` configure option (Hennadii Stepanov)
Pull request description:
This PR enables users to override compiler flags that have been set by the CMake-based build system, such as warning flags.
The Autotools-based build system has the same feature out-of-the-box.
See more details [here](https://github.com/bitcoin-core/secp256k1/issues/1235#issuecomment-1465330925).
Here are some examples of the new option usage:
```
cmake -S . -B build -DSECP256K1_LATE_CFLAGS="-Wno-extra -Wlong-long"
```
```
cmake -S . -B build -DSECP256K1_BUILD_EXAMPLES=ON -DSECP256K1_LATE_CFLAGS=-O1
cmake --build build
...
In function ‘secp256k1_ecmult_strauss_wnaf’,
inlined from ‘secp256k1_ecmult’ at /home/hebasto/git/secp256k1/src/ecmult_impl.h:353:5:
/home/hebasto/git/secp256k1/src/ecmult_impl.h:291:5: warning: ‘aux’ may be used uninitialized [-Wmaybe-uninitialized]
291 | secp256k1_ge_table_set_globalz(ECMULT_TABLE_SIZE(WINDOW_A) * no, state->pre_a, state->aux);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/hebasto/git/secp256k1/src/secp256k1.c:29:
/home/hebasto/git/secp256k1/src/ecmult_impl.h: In function ‘secp256k1_ecmult’:
/home/hebasto/git/secp256k1/src/group_impl.h:174:13: note: by argument 3 of type ‘const secp256k1_fe *’ to ‘secp256k1_ge_table_set_globalz’ declared here
174 | static void secp256k1_ge_table_set_globalz(size_t len, secp256k1_ge *a, const secp256k1_fe *zr) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/hebasto/git/secp256k1/src/secp256k1.c:30:
/home/hebasto/git/secp256k1/src/ecmult_impl.h:345:18: note: ‘aux’ declared here
345 | secp256k1_fe aux[ECMULT_TABLE_SIZE(WINDOW_A)];
| ^~~
...
```
Please note that in the last case providing `env CFLAGS=-O1` or `-DCMAKE_C_FLAGS=-O1` won't work.
ACKs for top commit:
real-or-random:
ACK 42f8c51402
Tree-SHA512: 2b152e420a4a8ffd5f67857de03ae5ba9f2223e535ac01a867c1025e0619180d8255fdd1e5fb8279b290f0a1c96bcc874043ef968fcd99b1ff4e13041a91b1e1
e6822678ea build: Error if required module explicitly off (Tim Ruffing)
89ec583ccf build: Clean up handling of module dependencies (Tim Ruffing)
Pull request description:
This is a cleanup which makes it easier to add further modules with dependencies, e.g., in #1452. The diff looks larger than it is because I also reordered the modules and made the order consistent between CMake and autotools.
(We noticed that the current logic could be improved in https://github.com/BlockstreamResearch/secp256k1-zkp/pull/275.)
ACKs for top commit:
jonasnick:
ACK e6822678ea
hebasto:
ACK e6822678ea.
Tree-SHA512: 040e791e5b5b9b8845a39632633a45ca759391455910bdefba2b7b77c6340e65df6eda18199ae2ad65c30ee2fc6630471437aec143c26fe09ae4c11409a37622
This also makes the order in which module options are processed
consistent between CMake and autotools (the reverse order of the listing
printed to stdout).
ba5d72d626 assumptions: Use new STATIC_ASSERT macro (Tim Ruffing)
e53c2d9ffc Require that sizeof(secp256k1_ge_storage) == 64 (Tim Ruffing)
d0ba2abbff util: Add STATIC_ASSERT macro (Tim Ruffing)
Pull request description:
This gets rid of an untested code path. Resolves https://github.com/bitcoin-core/secp256k1/issues/1352.
This is a bit opinionated in the sense that it adds a static assertion where it's needed in `secp256k1_pubkey_save` and `secp256k1_pubkey_load`. I think this is justified in this case. It helps the reviewer verify that these functions are correct.
See individual commit messages.
ACKs for top commit:
sipa:
utACK ba5d72d626
jonasnick:
ACK ba5d72d626
Tree-SHA512: 2553c0610b62bcda6d4ef26eb26b5b2e07acf723bcd299691a2d02da57af22b8763f63c2d4adb17d30de8825b6157be6e4f0398147854fbabdf8b865fb0b5c88
This also splits the big "&&" expression into separate expressions. If
we ever see an assertion fail, the error message will tell it precisely
which one failed.
This gets rid of an untested code path. Resolves#1352.
secp256k1_ge_storage is a struct with two secp256k1_fe_storage fields.
The C standard allows the compiler to add padding between the fields and
at the end of the struct, but no sane compiler in the end would do this:
The only reason to add padding is to ensure alignment, but such padding
is never necessary between two fields of the same type.
Similarly, secp256k1_fe_storage is a struct with a single array of
uintXX_t. No padding is allowed between array elements. Again, C allows
the compiler to insert padding at the end of the struct, but there's no
absolute reason to do so in this case.
For the uintXX_t itself, this guaranteed to have no padding bits, i.e.,
it's guaranteed to have exactly XX bits.
So I claim that for any existing compiler in the real world,
sizeof(secp256k1_ge_storage) == 64.
3928b7c383 doc: improve secp256k1_fe_set_b32_mod doc (Coding Enthusiast)
Pull request description:
As discussed in #1453
This only changes the `secp256k1_fe_impl_set_b32_mod` comment since I think `secp256k1_fe_set_b32_limit` doc is clear enough.
ACKs for top commit:
sipa:
ACK 3928b7c383
theStack:
ACK 3928b7c383
Tree-SHA512: ad62c1b72d6a487473b182e6aadc7765711385add8c6576bf15c2015db82721f19e3d635f7a29316c2ee7e3c73bc55e2cd4f46ec13180be93d6fe8641f47e7d2
e02f313b1f Add comment on length checks when parsing ECDSA sigs (Tim Ruffing)
Pull request description:
I claim the check can be removed but I don't want to touch this
stable and well-tested code.
On the way, we fix grammar in another comment.
ACKs for top commit:
sipa:
ACK e02f313b1f
RandyMcMillan:
ACK e02f313
Tree-SHA512: f82691a8f5db82a1e9683e52ce8e952ebd56b476a2817c5a876ce4638254b7b4ac93175318fb59598ed5532f33433951d75afea03724ef4419c3e1bd12ca8c20
0e5ea62207 CONTRIBUTING: add some coding and style conventions (Jonas Nick)
1a432cb982 README: update first sentence (Jonas Nick)
0922a047fb docs: move coverage report instructions to CONTRIBUTING (Jonas Nick)
76880e4015 Add CONTRIBUTING.md including scope and guidelines for new code (Jonas Nick)
Pull request description:
Following offline discussions, this PR documents the scope of the library and the requirements for adding new modules. I think this fixes most of #997. It also updates the README very slightly.
In addition, I added some coding conventions that I remembered explaining to new contributors in the past year. Even though it's far from exhaustive, I think this is an easy improvement to the CONTRIBUTING.md. Feel free to suggest more conventions.
ACKs for top commit:
sipa:
ACK 0e5ea62207
real-or-random:
ACK 0e5ea62207
Tree-SHA512: ffdbab22982fd632de92e81bd135f141ac86e24cc0dcfc0e1ae12b0d2a2e4f91377ab2c0cc440cb919889eaed8bfc1447b880fa1430fd771b956f2af0fe3766e
04af0ba162 Replace ge_equals_ge[,j] calls with group.h equality calls (Pieter Wuille)
60525f6c14 Add unit tests for group.h equality functions (Pieter Wuille)
a47cd97d51 Add group.h ge/gej equality functions (Pieter Wuille)
Pull request description:
This pull requests removes the test-only functions `ge_equals_ge` and `ge_equals_gej`, and replaces them with proper group.h functions `secp256k1_ge_eq_var` and `secp256k1_gej_eq_ge_var` (mimicking the existing `secp256k1_gej_eq_var` function).
This drops some of the arbitrary and undocumented magnitude restristrictions these functions have, makes them properly tested on their own, and makes their semantics cleaner (I'm always left checking whether `ge_equals_ge` does a `CHECK` internally or whether it returns a value...).
ACKs for top commit:
real-or-random:
utACK 04af0ba162
stratospher:
ACK 04af0ba.
Tree-SHA512: 49bc409ffa980144d1305c9389a846af45f0a97bfec19d016929056aa918c6a9f020dbe8549f5318fa8e6a4108621cc3cce60331aa0634f84619a1104d20a62a
bb4672342e remove VERIFY_SETUP define (Sebastian Falbesoner)
a3a3e11acd remove unneeded VERIFY_SETUP uses in ECMULT_CONST_TABLE_GET_GE macro (Sebastian Falbesoner)
a0fb68a2e7 introduce and use SECP256K1_SCALAR_VERIFY macro (Sebastian Falbesoner)
cf25c86d05 introduce and use SECP256K1_{FE,GE,GEJ}_VERIFY macros (Sebastian Falbesoner)
5d89bc031b remove superfluous `#ifdef VERIFY`/`#endif` preprocessor conditions (Sebastian Falbesoner)
c2688f8de9 redefine VERIFY_CHECK to empty in production (non-VERIFY) mode (Sebastian Falbesoner)
Pull request description:
As suggested in #1381, this PR reworks the policy for VERIFY_CHECK and when to use #ifdef VERIFY, by:
- redefining VERIFY_CHECK to empty in production (non-VERIFY) mode
- removing many then superflous #ifdef VERIFY blocks (if they exclusively contained VERIFY_CHECKs)
- introducing uppercase macros around verify_ functions and using them for better readabiliy
What is _not_ included yet is the proposed renaming from "_check" to "_assert":
> And while we're touching this anyway, we could consider renaming "check" to "assert", which is a more precise term. (In fact, if we redefine VERIFY_CHECK to be empty in production, we have almost reimplemented assert.h...)
This should be easy to achieve with simple search-and-replace (e.g. using sed), but I was hesitant as this would probably case annoying merge conflicts on some of the open PRs. Happy to add this if the rename if desired (#1381 didn't get any feedback about the renaming idea yet).
ACKs for top commit:
stratospher:
ACK bb46723.
real-or-random:
utACK bb4672342e
Tree-SHA512: 226ca609926dea638aa3bb537d29d4fac8b8302dcd9da35acf767ba9573e5221d2dae04ea26c15d80a50ed70af1ab0dca10642c21df7dbdda432fa237a5ef2cc
dcdda31f2c Tighten secp256k1_fe_mul_inner's VERIFY_BITS checks (Russell O'Connor)
8e2a5fe908 correct assertion for secp256k1_fe_mul_inner (roconnor-blockstream)
Pull request description:
Based on the surrounding asserts, 112 bits before this line, and 61 bits after this line, this assertion should be 113 bits. Notably the commensurate line in secp256k1_fe_sqr_inner is correctly assert to be 113 bits.
ACKs for top commit:
real-or-random:
ACK dcdda31f2c tested with asm disabled
Tree-SHA512: c35170e37d9a6d1413dd625032028129ab2eccee7da86697ab9641b68ad78efd7251953d51e7acaefd14888d3fd61877f9f05349c44f6fc0133ce9b3921b0e1a
1ddd76af0a bench: add --help option to bench_internal (Sebastian Falbesoner)
Pull request description:
While coming up with commands for running the benchmarks for issue https://github.com/bitcoin-core/secp256k1/issues/726#issuecomment-1824625653, I noticed that in contrast to `bench{_ecmult}`, `bench_internal` doesn't have a help option yet and figured it would be nice to have one. A comparable past PR is https://github.com/bitcoin-core/secp256k1/pull/1008. Benchmark categories appear in the same order as they are executed, the concrete benchmark names in parantheses per category are listed in alphabetical order.
ACKs for top commit:
real-or-random:
utACK 1ddd76af0a
siv2r:
ACK 1ddd76a, tested the `--help` option locally, and it works as expected.
Tree-SHA512: d117641a5f25a7cbf83881f3acceae99624528a0cbb2405efdbe1a3a2762b4d6b251392e954aaa32f6771069d31143743770fccafe198084c12258dedb0856fc
Widely available versions of GCC and Clang beat our field asm on -O2.
In particular, GCC 10.5.0, which is Bitcoin Core's current compiler
for official x86_64 builds, produces code that is > 20% faster for
fe_mul and > 10% faster for signature verification (see #726).
These are the alternatives to this PR:
We could replace our current asm with the fastest compiler output
that we can find. This is potentially faster, but it has multiple
drawbacks:
- It's more coding work because it needs detailed benchmarks (e.g.,
with many compiler/options).
- It's more review work because we need to deal with inline asm
(including clobbers etc.) and there's a lack of experts reviewers
in this area.
- It's not unlikely that we'll fall behind again in a few compiler
versions, and then we have to deal with this again, i.e., redo the
benchmarks. Given our history here, I doubt that we'll revolve
this timely.
We could change the default of the asm build option to off. But this
will also disable the scalar asm, which is still faster.
We could split the build option into two separate options for field
and scalar asm and only disable the field asm by default. But this
adds complexity to the build and to the test matrix.
My conclusion is that this PR gets the low-hanging fruit in terms of
performance. It simplifies our code significantly. It's clearly an
improvement, and it's very easy to review. Whether re-introducing
better asm (whether from a compiler or from CryptOpt) is worth the
hassle can be evaluated separately, and should not hold up this
improvement.
Solves #726.
33dc7e4d3e asm: add .note.GNU-stack section for non-exec stack (fanquake)
Pull request description:
With this in place, we no-longer see warnings like the following:
```bash
/usr/lib/gcc-cross/arm-linux-gnueabihf/12/../../../../arm-linux-gnueabihf/bin/ld: warning: field_10x26_arm.o: missing .note.GNU-stack section implies executable stack
/usr/lib/gcc-cross/arm-linux-gnueabihf/12/../../../../arm-linux-gnueabihf/bin/ld: NOTE: This behaviour is deprecated and will be removed in a future version of the linker
```
Should close#1434.
ACKs for top commit:
sipa:
utACK 33dc7e4d3e
real-or-random:
utACK 33dc7e4d3e
Tree-SHA512: f75ded8d971f54d1e871bcc4d815ba367b3e154eea2f18309ecaf9053e22f986bfffcf28418367f8055b65a5a0b245fee045adfcb63a2196df5e2f3aa6c97b89
10271356c8 Return temporaries to being unsigned in secp256k1_fe_sqr_inner (roconnor-blockstream)
Pull request description:
These temporaries seem to been inadvertently changed to signed during a refactoring. Generally, bit shifting is frowned upon for signed values.
ACKs for top commit:
sipa:
utACK 10271356c8
real-or-random:
utACK 10271356c8
Tree-SHA512: a9fefe4b146163209662cd435422beb3c9561eb9e83110454184f70df2292992f39ec1971143428e039a80cad2f6285db74de2f059e877ad8756ff739269b67a
8185e72d29 ci: Ignore internal errors in snapshot compilers (Hennadii Stepanov)
Pull request description:
It was discussed on today's IRC meeting.
ACKs for top commit:
real-or-random:
ACK 8185e72d29
Tree-SHA512: 0f41ca8303bd3d6efefcd3a544c7bd7dfcf464c57c779c876da4a77cacd262e6c963449d493fdf5a641b0d10b655c8c67fe8a147145b6533328d7bf5344313e1
355bbdf38a Add changelog entry for signed-digit ecmult_const algorithm (Pieter Wuille)
21f49d9bec Remove unused secp256k1_scalar_shr_int (Pieter Wuille)
115fdc7232 Remove unused secp256k1_wnaf_const (Pieter Wuille)
aa9f3a3c00 ecmult_const: add/improve tests (Jonas Nick)
4d16e90111 Signed-digit based ecmult_const algorithm (Pieter Wuille)
ba523be067 make SECP256K1_SCALAR_CONST reduce modulo exhaustive group order (Pieter Wuille)
2140da9cd5 Add secp256k1_scalar_half for halving scalars (+ tests/benchmarks). (Pieter Wuille)
Pull request description:
Using some insights learned from #1058, this replaces the fixed-wnaf ecmult_const algorithm with a signed-digit based one. Conceptually both algorithms are very similar, in that they boil down to summing precomputed odd multiples of the input points. Practically however, the new algorithm is simpler because it's just using scalar operations, rather than relying on wnaf machinery with skew terms to guarantee odd multipliers.
The idea is that we can compute $q \cdot A$ as follows:
* Let $s = f(q)$, for some function $f()$.
* Compute $(s_1, s_2)$ such that $s = s_1 + \lambda s_2$, using `secp256k1_scalar_lambda_split`.
* Let $v_1 = s_1 + 2^{128}$ and $v_2 = s_2 + 2^{128}$ (such that the $v_i$ are positive and $n$ bits long).
* Computing the result as $$\sum_{i=0}^{n-1} (2v_1[i]-1) 2^i A + \sum_{i=0}^{n-1} (2v_2[i]-1) 2^i \lambda A$$ where $x[i]$ stands for the *i*'th bit of $x$, so summing positive and negative powers of two times $A$, based on the bits of $v_1.$
The comments in `ecmult_const_impl.h` show that if $f(q) = (q + (1+\lambda)(2^n - 2^{129} - 1))/2 \mod n$, the result will equal $q \cdot A$.
This last step can be performed in groups of multiple bits at once, by looking up entries in a precomputed table of odd multiples of $A$ and $\lambda A$, and then multiplying by a power of two before proceeding to the next group.
The result is slightly faster (I measure ~2% speedup), but significantly simpler as it only uses scalar arithmetic to determine the table lookup values. The speedup is due to the fact that no skew corrections at the end are needed, and less overhead to determine table indices. The precomputed table sizes are also made independent from the `ecmult` ones, after observing that the optimal table size is bigger here (which also gives a small speedup).
ACKs for top commit:
jonasnick:
ACK 355bbdf38a
siv2r:
ACK 355bbdf
real-or-random:
ACK 355bbdf38a
Tree-SHA512: 13db572cb7f9be00bf0931c65fcd8bc8b5545be86a8c8700bd6a79ad9e4d9e5e79e7f763f92ca6a91d9717a355f8162204b0ea821b6ae99d58cb400497ddc656
fa4d6c76b6 ci/cirrus: Add native ARM64 persistent workers (MarcoFalke)
2262d0eaab ci/cirrus: Bring back skeleton .cirrus.yml without jobs (Tim Ruffing)
Pull request description:
ACKs for top commit:
real-or-random:
ACK fa4d6c76b6
hebasto:
re-ACK fa4d6c76b6, only last two commits have been squashed since my recent [review](https://github.com/bitcoin-core/secp256k1/pull/1426#pullrequestreview-1636119941).
Tree-SHA512: d1fee99d54a41a4126f7eb72695a56137c925dc9ce7cd692a60ea1262ac0789bbd6aa4e4dfc030f0d97d06aeeae0724a5f2d794a85ff533c6cf3cd215f6a4b7a
c45b7c4fbb refactor: introduce testutil.h (deduplicate `random_fe_`, `ge_equals_` helpers) (Sebastian Falbesoner)
dc5514144f tests: simplify `random_fe_non_zero` (remove loop limit and unneeded normalize) (Sebastian Falbesoner)
Pull request description:
`random_fe_non_zero` contains a loop iteration limit that ensures that we abort if `random_fe` ever yielded zero more than ten times in a row. This construct was first introduced in PR #19 (commit 09ca4f32) for random non-square field elements and was later refactored into the non-zero helper in PR #25 (commit 6d6102fe). The copy-over to the exhaustive tests happened recently in PR #1118 (commit 0f864207).
This case seems to be practically irrelevant and I'd argue for keeping things simple and removing it (which was already suggested in https://github.com/bitcoin-core/secp256k1/pull/1118#discussion_r1067259954); if there's really a worry that the test's random generator is heavily biased towards certain values or value ranges then there should consequently be checks at other places too (e.g. directly in `random_fe` for 256-bit values that repeatedly overflow, i.e. >= p).
Also, the _fe_normalize call is not needed and can be removed, as the result of `random_fe` is already normalized.
ACKs for top commit:
real-or-random:
utACK c45b7c4fbb
siv2r:
ACK `c45b7c4` (reviewed the changes and tests for both the commits passed locally).
Tree-SHA512: 4ffa66dd0b8392d7d0083a71e7b0682ad18f9261fd4ce8548c3059b497d3462db97e16114fded9787661ca447a877a27f5b996bd7d47e6f91c4454079d28a8ac
Tim Ruffing
