7040a20247 doc: fix sage code for deriving alternative generator H (Sebastian Falbesoner)
Pull request description:
The line calculating H (in particular, the expression `G.decode('hex')`) fails with the following error message on Sage 9.5:
```
AttributeError: 'str' object has no attribute 'decode'
```
Fix that by converting the hex-string to bytes using `bytes.fromhex`.
(Noticed while reviewing https://github.com/bitcoin/bitcoin/pull/30048 which picks this code snippet comment up.)
ACKs for top commit:
josibake:
ACK https://github.com/BlockstreamResearch/secp256k1-zkp/pull/292/commits/7040a2024795a7e3758c7ab604ab440652c9772f
real-or-random:
utACK 7040a20247
Tree-SHA512: 0a44f399b103c2f5840056d163c1483a1d4f032bc0f8d3822507ac6da9d567f46e36caa79c7f5016aebcc8827b79e9aec7ebdb4f21c3c0242dc6875be140f289
3a9b1d46a3 New Experimental Module: Incremental Half-Aggregation for Schnorr Signatures (Benedikt)
Pull request description:
Revisited PR #130 by jonasnick.
I am happy to hear your thoughts.
**Summary of changes compared to #130:**
- Address comments from rustyrussell
- Use tagged hash
- Compute hashes with common prefix by copying midstate
- Allow Incremental Aggregation and make code consistent with the [draft spec](https://github.com/BlockstreamResearch/cross-input-aggregation/blob/master/half-aggregation.mediawiki)
ACKs for top commit:
real-or-random:
ACK 3a9b1d46a3
Tree-SHA512: 27239033f8b28ecf87ea310b3dd5a19dbbe6fd07495db71ef7017f8f444ec25a12897087d1bea0a2e9c3df77d7f17c38b183d7fe768858da2180f26624add4aa
42f8c51402 cmake: Add `SECP256K1_LATE_CFLAGS` configure option (Hennadii Stepanov)
Pull request description:
This PR enables users to override compiler flags that have been set by the CMake-based build system, such as warning flags.
The Autotools-based build system has the same feature out-of-the-box.
See more details [here](https://github.com/bitcoin-core/secp256k1/issues/1235#issuecomment-1465330925).
Here are some examples of the new option usage:
```
cmake -S . -B build -DSECP256K1_LATE_CFLAGS="-Wno-extra -Wlong-long"
```
```
cmake -S . -B build -DSECP256K1_BUILD_EXAMPLES=ON -DSECP256K1_LATE_CFLAGS=-O1
cmake --build build
...
In function ‘secp256k1_ecmult_strauss_wnaf’,
inlined from ‘secp256k1_ecmult’ at /home/hebasto/git/secp256k1/src/ecmult_impl.h:353:5:
/home/hebasto/git/secp256k1/src/ecmult_impl.h:291:5: warning: ‘aux’ may be used uninitialized [-Wmaybe-uninitialized]
291 | secp256k1_ge_table_set_globalz(ECMULT_TABLE_SIZE(WINDOW_A) * no, state->pre_a, state->aux);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/hebasto/git/secp256k1/src/secp256k1.c:29:
/home/hebasto/git/secp256k1/src/ecmult_impl.h: In function ‘secp256k1_ecmult’:
/home/hebasto/git/secp256k1/src/group_impl.h:174:13: note: by argument 3 of type ‘const secp256k1_fe *’ to ‘secp256k1_ge_table_set_globalz’ declared here
174 | static void secp256k1_ge_table_set_globalz(size_t len, secp256k1_ge *a, const secp256k1_fe *zr) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/hebasto/git/secp256k1/src/secp256k1.c:30:
/home/hebasto/git/secp256k1/src/ecmult_impl.h:345:18: note: ‘aux’ declared here
345 | secp256k1_fe aux[ECMULT_TABLE_SIZE(WINDOW_A)];
| ^~~
...
```
Please note that in the last case providing `env CFLAGS=-O1` or `-DCMAKE_C_FLAGS=-O1` won't work.
ACKs for top commit:
real-or-random:
ACK 42f8c51402
Tree-SHA512: 2b152e420a4a8ffd5f67857de03ae5ba9f2223e535ac01a867c1025e0619180d8255fdd1e5fb8279b290f0a1c96bcc874043ef968fcd99b1ff4e13041a91b1e1
e6822678ea build: Error if required module explicitly off (Tim Ruffing)
89ec583ccf build: Clean up handling of module dependencies (Tim Ruffing)
Pull request description:
This is a cleanup which makes it easier to add further modules with dependencies, e.g., in #1452. The diff looks larger than it is because I also reordered the modules and made the order consistent between CMake and autotools.
(We noticed that the current logic could be improved in https://github.com/BlockstreamResearch/secp256k1-zkp/pull/275.)
ACKs for top commit:
jonasnick:
ACK e6822678ea
hebasto:
ACK e6822678ea.
Tree-SHA512: 040e791e5b5b9b8845a39632633a45ca759391455910bdefba2b7b77c6340e65df6eda18199ae2ad65c30ee2fc6630471437aec143c26fe09ae4c11409a37622
This also makes the order in which module options are processed
consistent between CMake and autotools (the reverse order of the listing
printed to stdout).
ba5d72d626 assumptions: Use new STATIC_ASSERT macro (Tim Ruffing)
e53c2d9ffc Require that sizeof(secp256k1_ge_storage) == 64 (Tim Ruffing)
d0ba2abbff util: Add STATIC_ASSERT macro (Tim Ruffing)
Pull request description:
This gets rid of an untested code path. Resolves https://github.com/bitcoin-core/secp256k1/issues/1352.
This is a bit opinionated in the sense that it adds a static assertion where it's needed in `secp256k1_pubkey_save` and `secp256k1_pubkey_load`. I think this is justified in this case. It helps the reviewer verify that these functions are correct.
See individual commit messages.
ACKs for top commit:
sipa:
utACK ba5d72d626
jonasnick:
ACK ba5d72d626
Tree-SHA512: 2553c0610b62bcda6d4ef26eb26b5b2e07acf723bcd299691a2d02da57af22b8763f63c2d4adb17d30de8825b6157be6e4f0398147854fbabdf8b865fb0b5c88
This also splits the big "&&" expression into separate expressions. If
we ever see an assertion fail, the error message will tell it precisely
which one failed.
This gets rid of an untested code path. Resolves#1352.
secp256k1_ge_storage is a struct with two secp256k1_fe_storage fields.
The C standard allows the compiler to add padding between the fields and
at the end of the struct, but no sane compiler in the end would do this:
The only reason to add padding is to ensure alignment, but such padding
is never necessary between two fields of the same type.
Similarly, secp256k1_fe_storage is a struct with a single array of
uintXX_t. No padding is allowed between array elements. Again, C allows
the compiler to insert padding at the end of the struct, but there's no
absolute reason to do so in this case.
For the uintXX_t itself, this guaranteed to have no padding bits, i.e.,
it's guaranteed to have exactly XX bits.
So I claim that for any existing compiler in the real world,
sizeof(secp256k1_ge_storage) == 64.
3928b7c383 doc: improve secp256k1_fe_set_b32_mod doc (Coding Enthusiast)
Pull request description:
As discussed in #1453
This only changes the `secp256k1_fe_impl_set_b32_mod` comment since I think `secp256k1_fe_set_b32_limit` doc is clear enough.
ACKs for top commit:
sipa:
ACK 3928b7c383
theStack:
ACK 3928b7c383
Tree-SHA512: ad62c1b72d6a487473b182e6aadc7765711385add8c6576bf15c2015db82721f19e3d635f7a29316c2ee7e3c73bc55e2cd4f46ec13180be93d6fe8641f47e7d2
e02f313b1f Add comment on length checks when parsing ECDSA sigs (Tim Ruffing)
Pull request description:
I claim the check can be removed but I don't want to touch this
stable and well-tested code.
On the way, we fix grammar in another comment.
ACKs for top commit:
sipa:
ACK e02f313b1f
RandyMcMillan:
ACK e02f313
Tree-SHA512: f82691a8f5db82a1e9683e52ce8e952ebd56b476a2817c5a876ce4638254b7b4ac93175318fb59598ed5532f33433951d75afea03724ef4419c3e1bd12ca8c20
0e5ea62207 CONTRIBUTING: add some coding and style conventions (Jonas Nick)
1a432cb982 README: update first sentence (Jonas Nick)
0922a047fb docs: move coverage report instructions to CONTRIBUTING (Jonas Nick)
76880e4015 Add CONTRIBUTING.md including scope and guidelines for new code (Jonas Nick)
Pull request description:
Following offline discussions, this PR documents the scope of the library and the requirements for adding new modules. I think this fixes most of #997. It also updates the README very slightly.
In addition, I added some coding conventions that I remembered explaining to new contributors in the past year. Even though it's far from exhaustive, I think this is an easy improvement to the CONTRIBUTING.md. Feel free to suggest more conventions.
ACKs for top commit:
sipa:
ACK 0e5ea62207
real-or-random:
ACK 0e5ea62207
Tree-SHA512: ffdbab22982fd632de92e81bd135f141ac86e24cc0dcfc0e1ae12b0d2a2e4f91377ab2c0cc440cb919889eaed8bfc1447b880fa1430fd771b956f2af0fe3766e
04af0ba162 Replace ge_equals_ge[,j] calls with group.h equality calls (Pieter Wuille)
60525f6c14 Add unit tests for group.h equality functions (Pieter Wuille)
a47cd97d51 Add group.h ge/gej equality functions (Pieter Wuille)
Pull request description:
This pull requests removes the test-only functions `ge_equals_ge` and `ge_equals_gej`, and replaces them with proper group.h functions `secp256k1_ge_eq_var` and `secp256k1_gej_eq_ge_var` (mimicking the existing `secp256k1_gej_eq_var` function).
This drops some of the arbitrary and undocumented magnitude restristrictions these functions have, makes them properly tested on their own, and makes their semantics cleaner (I'm always left checking whether `ge_equals_ge` does a `CHECK` internally or whether it returns a value...).
ACKs for top commit:
real-or-random:
utACK 04af0ba162
stratospher:
ACK 04af0ba.
Tree-SHA512: 49bc409ffa980144d1305c9389a846af45f0a97bfec19d016929056aa918c6a9f020dbe8549f5318fa8e6a4108621cc3cce60331aa0634f84619a1104d20a62a
bb4672342e remove VERIFY_SETUP define (Sebastian Falbesoner)
a3a3e11acd remove unneeded VERIFY_SETUP uses in ECMULT_CONST_TABLE_GET_GE macro (Sebastian Falbesoner)
a0fb68a2e7 introduce and use SECP256K1_SCALAR_VERIFY macro (Sebastian Falbesoner)
cf25c86d05 introduce and use SECP256K1_{FE,GE,GEJ}_VERIFY macros (Sebastian Falbesoner)
5d89bc031b remove superfluous `#ifdef VERIFY`/`#endif` preprocessor conditions (Sebastian Falbesoner)
c2688f8de9 redefine VERIFY_CHECK to empty in production (non-VERIFY) mode (Sebastian Falbesoner)
Pull request description:
As suggested in #1381, this PR reworks the policy for VERIFY_CHECK and when to use #ifdef VERIFY, by:
- redefining VERIFY_CHECK to empty in production (non-VERIFY) mode
- removing many then superflous #ifdef VERIFY blocks (if they exclusively contained VERIFY_CHECKs)
- introducing uppercase macros around verify_ functions and using them for better readabiliy
What is _not_ included yet is the proposed renaming from "_check" to "_assert":
> And while we're touching this anyway, we could consider renaming "check" to "assert", which is a more precise term. (In fact, if we redefine VERIFY_CHECK to be empty in production, we have almost reimplemented assert.h...)
This should be easy to achieve with simple search-and-replace (e.g. using sed), but I was hesitant as this would probably case annoying merge conflicts on some of the open PRs. Happy to add this if the rename if desired (#1381 didn't get any feedback about the renaming idea yet).
ACKs for top commit:
stratospher:
ACK bb46723.
real-or-random:
utACK bb4672342e
Tree-SHA512: 226ca609926dea638aa3bb537d29d4fac8b8302dcd9da35acf767ba9573e5221d2dae04ea26c15d80a50ed70af1ab0dca10642c21df7dbdda432fa237a5ef2cc
dcdda31f2c Tighten secp256k1_fe_mul_inner's VERIFY_BITS checks (Russell O'Connor)
8e2a5fe908 correct assertion for secp256k1_fe_mul_inner (roconnor-blockstream)
Pull request description:
Based on the surrounding asserts, 112 bits before this line, and 61 bits after this line, this assertion should be 113 bits. Notably the commensurate line in secp256k1_fe_sqr_inner is correctly assert to be 113 bits.
ACKs for top commit:
real-or-random:
ACK dcdda31f2c tested with asm disabled
Tree-SHA512: c35170e37d9a6d1413dd625032028129ab2eccee7da86697ab9641b68ad78efd7251953d51e7acaefd14888d3fd61877f9f05349c44f6fc0133ce9b3921b0e1a
1ddd76af0a bench: add --help option to bench_internal (Sebastian Falbesoner)
Pull request description:
While coming up with commands for running the benchmarks for issue https://github.com/bitcoin-core/secp256k1/issues/726#issuecomment-1824625653, I noticed that in contrast to `bench{_ecmult}`, `bench_internal` doesn't have a help option yet and figured it would be nice to have one. A comparable past PR is https://github.com/bitcoin-core/secp256k1/pull/1008. Benchmark categories appear in the same order as they are executed, the concrete benchmark names in parantheses per category are listed in alphabetical order.
ACKs for top commit:
real-or-random:
utACK 1ddd76af0a
siv2r:
ACK 1ddd76a, tested the `--help` option locally, and it works as expected.
Tree-SHA512: d117641a5f25a7cbf83881f3acceae99624528a0cbb2405efdbe1a3a2762b4d6b251392e954aaa32f6771069d31143743770fccafe198084c12258dedb0856fc
Widely available versions of GCC and Clang beat our field asm on -O2.
In particular, GCC 10.5.0, which is Bitcoin Core's current compiler
for official x86_64 builds, produces code that is > 20% faster for
fe_mul and > 10% faster for signature verification (see #726).
These are the alternatives to this PR:
We could replace our current asm with the fastest compiler output
that we can find. This is potentially faster, but it has multiple
drawbacks:
- It's more coding work because it needs detailed benchmarks (e.g.,
with many compiler/options).
- It's more review work because we need to deal with inline asm
(including clobbers etc.) and there's a lack of experts reviewers
in this area.
- It's not unlikely that we'll fall behind again in a few compiler
versions, and then we have to deal with this again, i.e., redo the
benchmarks. Given our history here, I doubt that we'll revolve
this timely.
We could change the default of the asm build option to off. But this
will also disable the scalar asm, which is still faster.
We could split the build option into two separate options for field
and scalar asm and only disable the field asm by default. But this
adds complexity to the build and to the test matrix.
My conclusion is that this PR gets the low-hanging fruit in terms of
performance. It simplifies our code significantly. It's clearly an
improvement, and it's very easy to review. Whether re-introducing
better asm (whether from a compiler or from CryptOpt) is worth the
hassle can be evaluated separately, and should not hold up this
improvement.
Solves #726.
33dc7e4d3e asm: add .note.GNU-stack section for non-exec stack (fanquake)
Pull request description:
With this in place, we no-longer see warnings like the following:
```bash
/usr/lib/gcc-cross/arm-linux-gnueabihf/12/../../../../arm-linux-gnueabihf/bin/ld: warning: field_10x26_arm.o: missing .note.GNU-stack section implies executable stack
/usr/lib/gcc-cross/arm-linux-gnueabihf/12/../../../../arm-linux-gnueabihf/bin/ld: NOTE: This behaviour is deprecated and will be removed in a future version of the linker
```
Should close#1434.
ACKs for top commit:
sipa:
utACK 33dc7e4d3e
real-or-random:
utACK 33dc7e4d3e
Tree-SHA512: f75ded8d971f54d1e871bcc4d815ba367b3e154eea2f18309ecaf9053e22f986bfffcf28418367f8055b65a5a0b245fee045adfcb63a2196df5e2f3aa6c97b89
10271356c8 Return temporaries to being unsigned in secp256k1_fe_sqr_inner (roconnor-blockstream)
Pull request description:
These temporaries seem to been inadvertently changed to signed during a refactoring. Generally, bit shifting is frowned upon for signed values.
ACKs for top commit:
sipa:
utACK 10271356c8
real-or-random:
utACK 10271356c8
Tree-SHA512: a9fefe4b146163209662cd435422beb3c9561eb9e83110454184f70df2292992f39ec1971143428e039a80cad2f6285db74de2f059e877ad8756ff739269b67a
8185e72d29 ci: Ignore internal errors in snapshot compilers (Hennadii Stepanov)
Pull request description:
It was discussed on today's IRC meeting.
ACKs for top commit:
real-or-random:
ACK 8185e72d29
Tree-SHA512: 0f41ca8303bd3d6efefcd3a544c7bd7dfcf464c57c779c876da4a77cacd262e6c963449d493fdf5a641b0d10b655c8c67fe8a147145b6533328d7bf5344313e1
355bbdf38a Add changelog entry for signed-digit ecmult_const algorithm (Pieter Wuille)
21f49d9bec Remove unused secp256k1_scalar_shr_int (Pieter Wuille)
115fdc7232 Remove unused secp256k1_wnaf_const (Pieter Wuille)
aa9f3a3c00 ecmult_const: add/improve tests (Jonas Nick)
4d16e90111 Signed-digit based ecmult_const algorithm (Pieter Wuille)
ba523be067 make SECP256K1_SCALAR_CONST reduce modulo exhaustive group order (Pieter Wuille)
2140da9cd5 Add secp256k1_scalar_half for halving scalars (+ tests/benchmarks). (Pieter Wuille)
Pull request description:
Using some insights learned from #1058, this replaces the fixed-wnaf ecmult_const algorithm with a signed-digit based one. Conceptually both algorithms are very similar, in that they boil down to summing precomputed odd multiples of the input points. Practically however, the new algorithm is simpler because it's just using scalar operations, rather than relying on wnaf machinery with skew terms to guarantee odd multipliers.
The idea is that we can compute $q \cdot A$ as follows:
* Let $s = f(q)$, for some function $f()$.
* Compute $(s_1, s_2)$ such that $s = s_1 + \lambda s_2$, using `secp256k1_scalar_lambda_split`.
* Let $v_1 = s_1 + 2^{128}$ and $v_2 = s_2 + 2^{128}$ (such that the $v_i$ are positive and $n$ bits long).
* Computing the result as $$\sum_{i=0}^{n-1} (2v_1[i]-1) 2^i A + \sum_{i=0}^{n-1} (2v_2[i]-1) 2^i \lambda A$$ where $x[i]$ stands for the *i*'th bit of $x$, so summing positive and negative powers of two times $A$, based on the bits of $v_1.$
The comments in `ecmult_const_impl.h` show that if $f(q) = (q + (1+\lambda)(2^n - 2^{129} - 1))/2 \mod n$, the result will equal $q \cdot A$.
This last step can be performed in groups of multiple bits at once, by looking up entries in a precomputed table of odd multiples of $A$ and $\lambda A$, and then multiplying by a power of two before proceeding to the next group.
The result is slightly faster (I measure ~2% speedup), but significantly simpler as it only uses scalar arithmetic to determine the table lookup values. The speedup is due to the fact that no skew corrections at the end are needed, and less overhead to determine table indices. The precomputed table sizes are also made independent from the `ecmult` ones, after observing that the optimal table size is bigger here (which also gives a small speedup).
ACKs for top commit:
jonasnick:
ACK 355bbdf38a
siv2r:
ACK 355bbdf
real-or-random:
ACK 355bbdf38a
Tree-SHA512: 13db572cb7f9be00bf0931c65fcd8bc8b5545be86a8c8700bd6a79ad9e4d9e5e79e7f763f92ca6a91d9717a355f8162204b0ea821b6ae99d58cb400497ddc656
b41caaafd2 bppp: replace memcmp in tests with secp256k1_memcmp_var (Jonas Nick)
6a3aae8f1d group_parse: use secp256k1_memcmp_var instead of memcmp (Jonas Nick)
e9d522fc64 ci: turn on -zkp modules in macos-native job (Jonas Nick)
c45b7c4fbb refactor: introduce testutil.h (deduplicate `random_fe_`, `ge_equals_` helpers) (Sebastian Falbesoner)
dc5514144f tests: simplify `random_fe_non_zero` (remove loop limit and unneeded normalize) (Sebastian Falbesoner)
d9d80fd155 ci: Bump major versions for docker actions (Hennadii Stepanov)
421d84855a ci: Align Autotools/CMake `CI_INSTALL` directory names (Hennadii Stepanov)
9f005c60d6 cmake: Install `libsecp256k1.pc` file (Hennadii Stepanov)
b0f7bfedc9 doc: Do not mention soname in CHANGELOG.md "ABI Compatibility" section (Hennadii Stepanov)
bd9d98d353 doc: Align documented scripts with CI ones (Hennadii Stepanov)
70303643cf tests: add CHECK_ERROR_VOID and use it in scratch tests (Jonas Nick)
f8d7ea68df tests: Replace counting_illegal_callbacks with CHECK_ILLEGAL_VOID (Jonas Nick)
a1d52e3e12 tests: remove unnecessary test in run_ec_pubkey_parse_test (Jonas Nick)
875b0ada25 tests: remove unnecessary set_illegal_callback (Jonas Nick)
9b118bc7fb release cleanup: bump version after 0.4.0 (Jonas Nick)
16339804c9 release: Prepare for 0.4.0 (Tim Ruffing)
d9a85065a9 changelog: Catch up in preparation of release (Tim Ruffing)
8659a01714 ci: Add `release` job (Hennadii Stepanov)
f9b38894ba ci: Update `actions/checkout` version (Hennadii Stepanov)
2635068abf ci/gha: Let MSan continue checking after errors in all jobs (Tim Ruffing)
e78c7b68eb ci/Dockerfile: Reduce size of Docker image further (Tim Ruffing)
2f0d3bbffb ci/Dockerfile: Warn if `ulimit -n` is too high when running Docker (Tim Ruffing)
4b8a647ad3 ci/gha: Add ARM64 QEMU jobs for clang and clang-snapshot (Tim Ruffing)
6ebe7d2bb3 ci/Dockerfile: Always use versioned clang packages (Tim Ruffing)
c223d7e33d ci: Switch macOS from Ventura to Monterey and add Valgrind (Hennadii Stepanov)
cce0456304 ci: Make repetitive command the default one (Hennadii Stepanov)
317a4c48f0 ci: Move `git config ...` to `run-in-docker-action` (Hennadii Stepanov)
676ed8f9cf ci: Move "C++ (public headers)" from Cirrus to GitHub Actions (Hennadii Stepanov)
61fc3a2dc8 ci: Move "C++ -fpermissive..." from Cirrus to GitHub Actions (Hennadii Stepanov)
d51fb0a533 ci: Move "MSan" from Cirrus to GitHub Actions (Hennadii Stepanov)
c22ac27529 ci: Move sanitizers task from Cirrus to GitHub Actions (Hennadii Stepanov)
ee1be62d84 ci: Use concurrency for pull requests only (Hennadii Stepanov)
fc3dea29ea ci: Move "ppc64le: Linux..." from Cirrus to GitHub Actions (Hennadii Stepanov)
7782dc8276 ci: Move "ARM64: Linux..." from Cirrus to GitHub Actions (Hennadii Stepanov)
0a16de671c ci: Move "ARM32: Linux..." from Cirrus to GitHub Actions (Hennadii Stepanov)
ea33914e00 ci: Move "s390x (big-endian): Linux..." from Cirrus to GitHub Actions (Hennadii Stepanov)
880be8af99 ci: Move "i686: Linux (Debian stable)" from Cirrus to GiHub Actions (Hennadii Stepanov)
e10878f58e ci, gha: Drop `driver-opts.network` input for `setup-buildx-action` (Hennadii Stepanov)
4ad4914bd1 ci, gha: Add `retry_builder` Docker image builder (Hennadii Stepanov)
6617a620d9 ci: Remove "x86_64: Linux (Debian stable)" task from Cirrus CI (Hennadii Stepanov)
03c9e6508c ci, gha: Add "x86_64: Linux (Debian stable)" GitHub Actions job (Hennadii Stepanov)
ad3e65d9fe ci: Remove GCC build files and sage to reduce size of Docker image (Tim Ruffing)
ef9fe959de ci: Drop no longer needed workaround (Hennadii Stepanov)
87d35f30c0 ci: Rename `cirrus.sh` to more general `ci.sh` (Hennadii Stepanov)
d6281dd008 ci: Remove Windows tasks from Cirrus CI (Hennadii Stepanov)
2b6f9cd546 ci, gha: Add Windows jobs based on Linux image (Hennadii Stepanov)
b0886fd35c ci, gha: Ensure only a single workflow processes `github.ref` at a time (Hennadii Stepanov)
d23da6d557 use secp256k1_scalar_verify checks (stratospher)
c7d0454932 add verification for scalars (stratospher)
ad152151b0 update max scalar in scalar_cmov_test and fix schnorrsig_verify exhaustive test (stratospher)
d78bec7001 ci: Remove Windows MSVC tasks from Cirrus CI (Hennadii Stepanov)
3545dc2b9b ci, gha: Run all MSVC tests on Windows natively (Hennadii Stepanov)
54058d16fe field: remove `secp256k1_fe_equal_var` (siv2r)
bb4efd6404 tests: remove unwanted `secp256k1_fe_normalize_weak` call (siv2r)
c2f6435802 ci: Add comment about switching macOS to M1 on GHA later (Tim Ruffing)
4a24fae0bc ci: Remove "arm64: macOS Ventura" task from Cirrus CI (Hennadii Stepanov)
8e54a346d2 ci, gha: Run "x86_64: macOS Ventura" job on GitHub Actions (Hennadii Stepanov)
747ada3587 test: Silent noisy clang warnings about Valgrind code on macOS x86_64 (Hennadii Stepanov)
d62db57427 ci: Use Homebrew's gcc in native macOS task (Hennadii Stepanov)
b7c685e74a Save _normalize_weak calls in group add methods (Peter Dettman)
c83afa66e0 Tighten group magnitude limits (Peter Dettman)
173e8d061a Implement current magnitude assumptions (Peter Dettman)
49afd2f5d8 Take use of _fe_verify_magnitude in field_impl.h (Sebastian Falbesoner)
4e9661fc42 Add _fe_verify_magnitude (no-op unless VERIFY is enabled) (Peter Dettman)
690b0fc05a add missing group element invariant checks (Sebastian Falbesoner)
9c91ea41b1 ci: Enable ellswift module where it's missing (Tim Ruffing)
db32a24761 ctimetests: Use new SECP256K1_CHECKMEM macros also for ellswift (Tim Ruffing)
8408dfdc4c Revert "ci: Run sage prover on CI" (Hennadii Stepanov)
c8d9914fb1 ci, gha: Run "SageMath prover" job on GitHub Actions (Hennadii Stepanov)
f1774e5ec4 ci, gha: Make MSVC job presentation more explicit (Hennadii Stepanov)
5ee039bb58 ci: Remove "Windows (VS 2022)" task from Cirrus CI (Hennadii Stepanov)
a2f7ccdecc ci: Run "Windows (VS 2022)" job on GitHub Actions (Hennadii Stepanov)
175db31149 ci: Drop no longer needed `PATH` variable update on Windows (Hennadii Stepanov)
116d2ab3df cmake: Set `ENVIRONMENT` property for examples on Windows (Hennadii Stepanov)
cef373997c cmake, refactor: Use helper function instead of interface library (Hennadii Stepanov)
a1bd4971d6 refactor: take use of `secp256k1_scalar_{zero,one}` constants (part 2) (Sebastian Falbesoner)
Pull request description:
[bitcoin-core/secp256k1#1391]: refactor: take use of `secp256k1_scalar_{zero,one}` constants (part 2)
[bitcoin-core/secp256k1#1290]: cmake: Set `ENVIRONMENT` property for examples on Windows
[bitcoin-core/secp256k1#1389]: ci: Run "Windows (VS 2022)" job on GitHub Actions
[bitcoin-core/secp256k1#1397]: ci: Remove "Windows (VS 2022)" task from Cirrus CI
[bitcoin-core/secp256k1#1399]: ci, gha: Run "SageMath prover" job on GitHub Actions
[bitcoin-core/secp256k1#1400]: ctimetests: Use new SECP256K1_CHECKMEM macros also for ellswift
[bitcoin-core/secp256k1#1348]: tighten group magnitude limits, save normalize_weak calls in group add methods (revival of #1032)
[bitcoin-core/secp256k1#1402]: ci: Use Homebrew'\''s gcc in native macOS task
[bitcoin-core/secp256k1#1274]: test: Silent noisy clang warnings about Valgrind code on macOS x86_64
[bitcoin-core/secp256k1#1394]: ci, gha: Run "x86_64: macOS Ventura" job on GitHub Actions
[bitcoin-core/secp256k1#1404]: ci: Remove "arm64: macOS Ventura" task from Cirrus CI
[bitcoin-core/secp256k1#1062]: Removes `_fe_equal_var`, and unwanted `_fe_normalize_weak` calls (in tests)
[bitcoin-core/secp256k1#1401]: ci, gha: Run all MSVC tests on Windows natively
[bitcoin-core/secp256k1#1373]: Add invariant checking for scalars
[bitcoin-core/secp256k1#1403]: ci, gha: Ensure only a single workflow processes `github.ref` at a time
[bitcoin-core/secp256k1#1398]: ci, gha: Add Windows jobs based on Linux image
[bitcoin-core/secp256k1#1405]: ci: Drop no longer needed workaround
[bitcoin-core/secp256k1#1396]: ci, gha: Add "x86_64: Linux (Debian stable)" GitHub Actions job
[bitcoin-core/secp256k1#1406]: ci, gha: Move more non-x86_64 tasks from Cirrus CI to GitHub Actions
[bitcoin-core/secp256k1#1410]: ci: Use concurrency for pull requests only
[bitcoin-core/secp256k1#1409]: ci: Move remained task from Cirrus to GitHub Actions
[bitcoin-core/secp256k1#1411]: ci: Make repetitive command the default one
[bitcoin-core/secp256k1#1412]: ci: Switch macOS from Ventura to Monterey and add Valgrind
[bitcoin-core/secp256k1#1414]: ci/gha: Add ARM64 QEMU jobs for clang and clang-snapshot
[bitcoin-core/secp256k1#1413]: ci: Add `release` job
[bitcoin-core/secp256k1#1415]: release: Prepare for 0.4.0
[bitcoin-core/secp256k1#1417]: release cleanup: bump version after 0.4.0
[bitcoin-core/secp256k1#1390]: tests: Replace counting_illegal_callbacks with CHECK_ILLEGAL_VOID
[bitcoin-core/secp256k1#1416]: doc: Align documented scripts with CI ones
[bitcoin-core/secp256k1#1422]: cmake: Install `libsecp256k1.pc` file
[bitcoin-core/secp256k1#1424]: ci: Bump major versions for docker actions
[bitcoin-core/secp256k1#1395]: tests: simplify `random_fe_non_zero` (remove loop limit and unneeded normalize)
This PR can be recreated with `./contrib/sync-upstream.sh -b master range ee7aaf213ea3eb42fc8960c7d178b5ffb286440f`.
Tip: Use `git show --remerge-diff` to show the changes manually added to the merge commit.' --web
- Replace fe_equal_var with fe_equal
- Use CHECK_ILLEGAL instead of CHECK/ecount
- Turn on secp256k1-zkp specific modules in CI
ACKs for top commit:
real-or-random:
ACK b41caaafd2
Tree-SHA512: e8994fa2590d6e277d60a130d3f11b0256dc7fbe67e8a2b85463d9849d3f9fbfccf366b17b3904f1974f8b7b27a24128d2a04a3337c1b8409d000d136714bf90
fa4d6c76b6 ci/cirrus: Add native ARM64 persistent workers (MarcoFalke)
2262d0eaab ci/cirrus: Bring back skeleton .cirrus.yml without jobs (Tim Ruffing)
Pull request description:
ACKs for top commit:
real-or-random:
ACK fa4d6c76b6
hebasto:
re-ACK fa4d6c76b6, only last two commits have been squashed since my recent [review](https://github.com/bitcoin-core/secp256k1/pull/1426#pullrequestreview-1636119941).
Tree-SHA512: d1fee99d54a41a4126f7eb72695a56137c925dc9ce7cd692a60ea1262ac0789bbd6aa4e4dfc030f0d97d06aeeae0724a5f2d794a85ff533c6cf3cd215f6a4b7a
c45b7c4fbb refactor: introduce testutil.h (deduplicate `random_fe_`, `ge_equals_` helpers) (Sebastian Falbesoner)
dc5514144f tests: simplify `random_fe_non_zero` (remove loop limit and unneeded normalize) (Sebastian Falbesoner)
Pull request description:
`random_fe_non_zero` contains a loop iteration limit that ensures that we abort if `random_fe` ever yielded zero more than ten times in a row. This construct was first introduced in PR #19 (commit 09ca4f32) for random non-square field elements and was later refactored into the non-zero helper in PR #25 (commit 6d6102fe). The copy-over to the exhaustive tests happened recently in PR #1118 (commit 0f864207).
This case seems to be practically irrelevant and I'd argue for keeping things simple and removing it (which was already suggested in https://github.com/bitcoin-core/secp256k1/pull/1118#discussion_r1067259954); if there's really a worry that the test's random generator is heavily biased towards certain values or value ranges then there should consequently be checks at other places too (e.g. directly in `random_fe` for 256-bit values that repeatedly overflow, i.e. >= p).
Also, the _fe_normalize call is not needed and can be removed, as the result of `random_fe` is already normalized.
ACKs for top commit:
real-or-random:
utACK c45b7c4fbb
siv2r:
ACK `c45b7c4` (reviewed the changes and tests for both the commits passed locally).
Tree-SHA512: 4ffa66dd0b8392d7d0083a71e7b0682ad18f9261fd4ce8548c3059b497d3462db97e16114fded9787661ca447a877a27f5b996bd7d47e6f91c4454079d28a8ac
9b118bc7fb release cleanup: bump version after 0.4.0 (Jonas Nick)
Pull request description:
based on #1415
ACKs for top commit:
sipa:
ACK 9b118bc7fb
hebasto:
ACK 9b118bc7fb
real-or-random:
ACK 9b118bc7fb
Tree-SHA512: 76df87c41bdc3379df4e88619645f5110010d7713ebe20bad3e7c99472bd62b90f4bd3c6b558ad5a23119acc4734e39383d96a9800e4a43dfadc086ef66fd0ab
8659a01714 ci: Add `release` job (Hennadii Stepanov)
f9b38894ba ci: Update `actions/checkout` version (Hennadii Stepanov)
Pull request description:
This PR introduces a new "Release" job that conducts sanity checks as defined in [`doc/release-process.md`](https://github.com/bitcoin-core/secp256k1/blob/master/doc/release-process.md#sanity-checks).
ACKs for top commit:
sipa:
ACK 8659a01714
real-or-random:
ACK 8659a01714
Tree-SHA512: 84e03fa07f8c41aec0f6d1ccb4ac3643e85d370ef7e388b335365deadb555f2d9ef7e5d80e1255a18e790a774e04ca66f265b9441402b183d4c535a97688f20f
2635068abf ci/gha: Let MSan continue checking after errors in all jobs (Tim Ruffing)
e78c7b68eb ci/Dockerfile: Reduce size of Docker image further (Tim Ruffing)
2f0d3bbffb ci/Dockerfile: Warn if `ulimit -n` is too high when running Docker (Tim Ruffing)
4b8a647ad3 ci/gha: Add ARM64 QEMU jobs for clang and clang-snapshot (Tim Ruffing)
6ebe7d2bb3 ci/Dockerfile: Always use versioned clang packages (Tim Ruffing)
Pull request description:
Solves one item in https://github.com/bitcoin-core/secp256k1/issues/1392.
This PR also has a few tweaks to the Dockerfile, see individual commits.
---
I'll follow up soon with a PR for ARM64/gcc. This will rely on Cirrus CI.
ACKs for top commit:
hebasto:
ACK 2635068abf.
Tree-SHA512: d290bdd8e8e2a2a2b6ccb1b25ecdc9662c51dab745068a98044b9abed75232d13cb9d2ddc2c63c908dcff6a12317f0c7a35db3288c57bc3b814793f7fce059fd
This commit switches to a new strategy to make sure we're installing the
most recent LLVM packages. Before this commit, we used the unversioned
LLVM packages (e.g., `clang` instead of `clang-18`), which are supposed
to provide the latest snapshot, but this is broken for arm64 [1],
which we want to add in a later PR.
Anyway, the new approach is cleaner because it does not require us to
fiddle with the installed `clang` package by removing a symlink.
[1] https://github.com/llvm/llvm-project/issues/64790
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
cce0456304 ci: Make repetitive command the default one (Hennadii Stepanov)
317a4c48f0 ci: Move `git config ...` to `run-in-docker-action` (Hennadii Stepanov)
Pull request description:
This PR addresses the https://github.com/bitcoin-core/secp256k1/pull/1409#discussion_r1301767281:
> couldn't we add this to `run-in-docker-action` to avoid duplication?
ACKs for top commit:
real-or-random:
ACK cce0456304
Tree-SHA512: 793cec3d99853a23e06680fe35a7cae2dc8771f22e3940c4a4f36357273164f2d92e196768118d146e956ffca9ef59d4e5f86e1cba5576ebfdc59343581c9924
676ed8f9cf ci: Move "C++ (public headers)" from Cirrus to GitHub Actions (Hennadii Stepanov)
61fc3a2dc8 ci: Move "C++ -fpermissive..." from Cirrus to GitHub Actions (Hennadii Stepanov)
d51fb0a533 ci: Move "MSan" from Cirrus to GitHub Actions (Hennadii Stepanov)
c22ac27529 ci: Move sanitizers task from Cirrus to GitHub Actions (Hennadii Stepanov)
Pull request description:
This PR concludes the migration from Cirrus CI to GitHub Actions.
ACKs for top commit:
real-or-random:
ACK 676ed8f9cf
Tree-SHA512: d8ea91a20297ff4c2c11a02c0a52f19413fde442c71e2e8b660737c09d62e55e4ae3f9bdbdeb0d967f6720a3dffa1566b7f94e0e32bcd490ac052819d5427c84
ee1be62d84 ci: Use concurrency for pull requests only (Hennadii Stepanov)
Pull request description:
This PR is an amendment for https://github.com/bitcoin-core/secp256k1/pull/1403.
It avoids skipping builds when some pushes were done consequentially.
From GitHub Actions [docs](https://docs.github.com/en/actions/using-jobs/using-concurrency):
> When a concurrent ... workflow is queued, if another ... workflow using the same concurrency group in the repository is in progress, the queued ... workflow will be pending. **Any previously pending ... workflow in the concurrency group will be canceled.**
No behavior change for pull requests.
Same as https://github.com/bitcoin/bitcoin/pull/28322.
ACKs for top commit:
real-or-random:
ACK ee1be62d84
Tree-SHA512: ce26378c3224a7647eb3b351d19e9308650ad49b933a68d37a0eca8516767a63f55580a33b02864caa539392c9aab0b4b54ecbace85bea6082bf533539a37c9c
fc3dea29ea ci: Move "ppc64le: Linux..." from Cirrus to GitHub Actions (Hennadii Stepanov)
7782dc8276 ci: Move "ARM64: Linux..." from Cirrus to GitHub Actions (Hennadii Stepanov)
0a16de671c ci: Move "ARM32: Linux..." from Cirrus to GitHub Actions (Hennadii Stepanov)
ea33914e00 ci: Move "s390x (big-endian): Linux..." from Cirrus to GitHub Actions (Hennadii Stepanov)
880be8af99 ci: Move "i686: Linux (Debian stable)" from Cirrus to GiHub Actions (Hennadii Stepanov)
Pull request description:
Move more non-x86_64 tasks from Cirrus CI to GitHub Actions.
Solves one item in https://github.com/bitcoin-core/secp256k1/issues/1392 partially.
ACKs for top commit:
real-or-random:
ACK fc3dea29ea but still waiting for Cirrus
Tree-SHA512: 9a910b3ee500aa34fc4db827f8b2a50bcfb637a9e59f4ad32545634772b397ce80b31a18723f4605dc42aa19a5632292943102099f7720f87de1da454da068b0
ef9fe959de ci: Drop no longer needed workaround (Hennadii Stepanov)
Pull request description:
The https://sourceware.org/bugzilla/show_bug.cgi?id=27008 bug has been resolved since libc 2.33.
Debian Bookworm has [libc](https://packages.debian.org/bookworm/libc6) 2.36.
I've separated this change from moving CI tasks to GitHub Actions intentionally.
ACKs for top commit:
real-or-random:
ACK ef9fe959de
Tree-SHA512: 4e8ce1232fcb581fa4700da75e5f63ff3da359416e5c5c1966f6aae079219fd697554db03d0b1729ea62cca42aae74bd36621a85d6ec7e4ee18e2c20b879cfa6
d23da6d557 use secp256k1_scalar_verify checks (stratospher)
c7d0454932 add verification for scalars (stratospher)
ad152151b0 update max scalar in scalar_cmov_test and fix schnorrsig_verify exhaustive test (stratospher)
Pull request description:
From #1360. This PR:
1. adds `secp256k1_scalar_verify` to make sure scalars are reduced mod the group order in VERIFY mode
2. uses `secp256k1_scalar_verify` in all the scalar functions except `secp256k1_scalar_clear`, `secp256k1_scalar_reduce_512`, `secp256k1_scalar_mul_512` and `secp256k1_scalar_*_var` functions in `scalar_low_impl.h`
ACKs for top commit:
real-or-random:
utACK d23da6d557
theStack:
Code-review ACK d23da6d557
Tree-SHA512: a371b319d948198c4038d35c9ea58f4b94de4dc312215e2b78a323c2acd4ae1355d97935c558b388774832d6d0058b97ff8ca50c3aab40b9ede5307760d0a505
747ada3587 test: Silent noisy clang warnings about Valgrind code on macOS x86_64 (Hennadii Stepanov)
Pull request description:
Since #1206, on macOS x86_64 with Valgrind installed, clang emits a massive amount of `-Wreserved-identifier` and `-Wreserved-macro-identifier` warnings from the `valgrind/valgrind.h` and `valgrind/memcheck.h` headers.
This PR prevents warnings emitted for the Valgrind code.
ACKs for top commit:
real-or-random:
utACK 747ada3587
Tree-SHA512: dd1b2b9db2d471939fdc30f9d8fd106a12f21ec5008ca98d8ebe3087d7ea352d564e8bbd0cec59a004e084af3a84d4680cb81f2ef6fe13cf164b7691e33f437d
b7c685e74a Save _normalize_weak calls in group add methods (Peter Dettman)
c83afa66e0 Tighten group magnitude limits (Peter Dettman)
173e8d061a Implement current magnitude assumptions (Peter Dettman)
49afd2f5d8 Take use of _fe_verify_magnitude in field_impl.h (Sebastian Falbesoner)
4e9661fc42 Add _fe_verify_magnitude (no-op unless VERIFY is enabled) (Peter Dettman)
690b0fc05a add missing group element invariant checks (Sebastian Falbesoner)
Pull request description:
This PR picks up #1032 by peterdettman. It's essentially a rebase on master; the original first commit (09dbba561fdb9d57a2cc9842ce041d9ba29a6189) which introduced group verification methods has mostly been replaced by PR #1299 (commit f20266722a) and what remains now is only adding a few missing checks at some places. The remaining commits are unchanged, though some (easy-to-solve) conflicts appeared through cherry-picking. The last commit which actually removes the `normalize_weak` calls is obviously the critical one and needs the most attention for review.
ACKs for top commit:
sipa:
utACK b7c685e74a
real-or-random:
ACK b7c685e74a
jonasnick:
ACK b7c685e74a
Tree-SHA512: f15167eff7ef6ed971c726a4d738de9a15be95b0c947d7e38329e7b16656202b7113497d36625304e784866349f2293f6f1d8cb97df35393af9ea465a4156da3
5bf57590bf bppp: Fix test for invalid sign byte again (Jonas Nick)
Pull request description:
The first byte provided to secp256k1_bppp_parse_one_of_points is allowed to be 0, 1, 2, or 3 since it encodes the Y coordinate of two points. In a previous fix we wrongly assumed it can only be 2 or 3.
ACKs for top commit:
real-or-random:
utACK 5bf57590bf
Tree-SHA512: 4dcc83a03691b8cb0cbe39da6e34311bde518e09fb02fb8648557eba435031eaee8acef9dc762e24c7d106b935b725981dcdaadbd1a3801026ee2b7b9bc34951
175db31149 ci: Drop no longer needed `PATH` variable update on Windows (Hennadii Stepanov)
116d2ab3df cmake: Set `ENVIRONMENT` property for examples on Windows (Hennadii Stepanov)
cef373997c cmake, refactor: Use helper function instead of interface library (Hennadii Stepanov)
Pull request description:
This PR simplifies running examples on Windows, because the DLL must reside either in the same folder where the executable is or somewhere in PATH.
It is an alternative to #1233.
ACKs for top commit:
real-or-random:
utACK 175db31149
Tree-SHA512: 8188018589a5bcf0179647a039cdafcce661dc103a70a5bb9e6b6f680b899332ba30b1e9ef5dad2a8c22c315d7794747e49d8cf2e391eebea21e3d8505ee334b
The test is supposed to create an invalid sign byte. Before this PR,
the generated sign byte could in fact be valid due to an overflow.
Co-authored-by: Jonas Nick <jonasd.nick@gmail.com>
b097a466c1 util: remove unused checked_realloc (Cory Fields)
Pull request description:
Usage was removed in 6fe50439 . This should be a NOOP.
Noticed when analyzing for zenbleed exposure: stdlib calls that aren't optimized away.
In this case realloc isn't making it into the final binary, but as far as I can tell this is completely dead code and should be dropped.
ACKs for top commit:
jonasnick:
ACK b097a466c1
real-or-random:
ACK b097a466c1
Tree-SHA512: d4249215eddd4035be2b50a8bb48b8a681abdab4ab41ca53f6c2a2507edfbc9ffa39ba22eb48e7da52f978e224198294495ce64f9d571d98c19283b20b82a63a
ea478beec6 musig: change test vector generation code shebang from python to python3 (Jonas Nick)
Pull request description:
The linter included in the Bitcoin Core and Elements test framework requires python3.
ACKs for top commit:
real-or-random:
utACK ea478beec6
Tree-SHA512: 0174e9d72529d1aa2c7a0542bc49a21af9163715961fa042da39c9bb857259f4c7e2b9be8e30b77c7b9b420574bd15f76308d807e22bdc590a91d249cca5ae86
The test is supposed to create an invalid sign byte. Before this PR,
the generated sign byte could in fact be valid due to an overflow.
Co-authored-by: Jonas Nick <jonasd.nick@gmail.com>
9b6a1c384d sync-upstream.sh: Fix position of "-b" option in reproduce command (Tim Ruffing)
Pull request description:
ACKs for top commit:
jonasnick:
utACK 9b6a1c384d
Tree-SHA512: 27e4a41bc9c8f10715623f669c97a511520753b23d24ae91d6d2144e54588da0769f97b1de78c87b7471b39e556b682b1c2910b2bf71f124fb77cbc9e446d5f8
c424e2fb43 ellswift: fix probabilistic test failure when swapping sides (Jonas Nick)
Pull request description:
Reported by jonatack in https://github.com/bitcoin/bitcoin/issues/28079.
When configured with `--disable-module-ecdh --enable-module-recovery`, then `./tests 64 81af32fd7ab8c9cbc2e62a689f642106` fails with
```
src/modules/ellswift/tests_impl.h:396: test condition failed: secp256k1_memcmp_var(share32_bad, share32a, 32) != 0
```
This tests verifies that changing the `party` bit of the `secp256k1_ellswift_xdh` function results in a different share. However, that's not the case when the secret keys of both parties are the same and this is actually what happens in the observed test failure. The keys can be equal in this test case because they are created by the `random_scalar_order_test` function whose output is not uniformly random (it's biased towards 0).
This commit restores the assumption that the secret keys differ.
ACKs for top commit:
sipa:
utACK c424e2fb43
real-or-random:
utACK c424e2fb43
Tree-SHA512: d1ab61473a77478f9aeffb21ad73e0bba478c90d8573c72ec89d2e0140434cc65c9d5f4d56e5f259931dc68fc1800695c6cd5d63d9cfce4c1c4d6744eeaa2028
Tim Ruffing
