e6433fb2c1 feat(persist): Add stage_and_commit to Persist (LLFourn)
0bee46e75b fix(store): Remove lifetime (LLFourn)
Pull request description:
Remove gratuitous use of lifetimes in the main persistence struct `Store`. Having lifetimes on this means that you have to keep the magic bytes alive longer than the database which is particularly offensive if you have to send the database to another thread. On top of that the bytes aren't even read.
ACKs for top commit:
evanlinjin:
ACK e6433fb2c1
Tree-SHA512: 7f6d9d60951a8ceaee30719d0771e15632c6fad0702294af15409c5df492669a07299874ef5ee34e3d75bdecbbd41df29bced3ff16b2360d5d5c7687ef677ffc
a4f28c079e chore: improve LocalChain::apply_header_connected_to doc (LLFourn)
8ec65f0b8e feat(example): add RPC wallet example (Vladimir Fomene)
a7d01dc39a feat(chain)!: make `IndexedTxGraph::apply_block_relevant` more efficient (志宇)
e0512acf94 feat(bitcoind_rpc)!: emissions include checkpoint and connected_to data (志宇)
8f2d4d9d40 test(chain): `LocalChain` test for update that is shorter than original (志宇)
9467cad55d feat(wallet): introduce block-by-block api (Vladimir Fomene)
d3e5095df1 feat(chain): add `apply_header..` methods to `LocalChain` (志宇)
2b61a122ff feat(chain): add `CheckPoint::from_block_ids` convenience method (志宇)
Pull request description:
### Description
Introduce block-by-block API for `bdk::Wallet`. A `wallet_rpc` example is added to demonstrate syncing `bdk::Wallet` with the `bdk_bitcoind_rpc` chain-source crate.
The API of `bdk_bitcoind_rpc::Emitter` is changed so the receiver knows how to connect to the block emitted.
### Notes to the reviewers
### Changelog notice
Added
* `Wallet` methods to apply full blocks (`apply_block` and `apply_block_connected_to`) and a method to apply a batch of unconfirmed transactions (`apply_unconfirmed_txs`).
* `CheckPoint::from_block_ids` convenience method.
* `LocalChain` methods to apply a block header (`apply_header` and `apply_header_connected_to`).
* Test to show that `LocalChain` can apply updates that are shorter than original. This will happen during reorgs if we sync wallet with `bdk_bitcoind_rpc::Emitter`.
Fixed
* `InsertTxError` now implements `std::error::Error`.
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
LLFourn:
self-ACK: a4f28c079e
evanlinjin:
ACK a4f28c079e
Tree-SHA512: e39fb65b4e69c0a6748d64eab12913dc9cfe5eb8355ab8fb68f60a37c3bb2e1489ddd8f2f138c6470135344f40e3dc671928f65d303fd41fb63f577b30895b60
In the example_cli we were not always committing (seemingly by mistake).
This then caused all the examples to have to compensate by manually
committing.
d33acc1466 Bump version to 1.0.0-alpha.4 (Steve Myers)
Pull request description:
### Description
Bump version to 1.0.0-alpha.4, also:
bdk_chain to 0.8.0
bdk_bitcoind_rpc to 0.3.0
bdk_electrum to 0.6.0
bdk_esplora to 0.6.0
bdk_file_store to 0.4.0
Fixes#1281.
### Notes to the reviewers
Since the version of bdk_chain changed all crates that depend on it also need to have their versions bumped.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
evanlinjin:
ACK d33acc1466
Tree-SHA512: ffb5c05b45e9d646ef796bd894353e4b58c3aae3dcf0921dfb9fcae45020ed4fc82274a7497494d6259bfffff9c2f97c530d2cbe6ace578babd04852af461f3a
d494f63d08 doc, example(bdk): fix derivation path in mnemonic_to_descriptors (vmammal)
Pull request description:
This will help avoid confusion with mainnet descriptors.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
danielabrozzoni:
ACK d494f63d08
Tree-SHA512: f0450d171bc53085204280495f2954e20f4b64a73cfbcc9a0c3be131c0b04ad53e936c83fa83c89fbd1ed1c4edd680e8437550453f75056049d36f84cae1df43
71fff1613d feat(chain): add txout methods to `KeychainTxOutIndex` (志宇)
83e7b7ec40 docs(chain): improve `KeychainTxOutIndex` docs (志宇)
9294e30943 docs(wallet): improve docs for unbounded spk iterator methods (志宇)
b74c2e2622 fix(wallet): use efficient peek address logic (志宇)
81aeaba48a feat(chain): add `SpkIterator::descriptor` method (志宇)
c7b47af72f refactor(chain)!: revamp `KeychainTxOutIndex` API (志宇)
705690ee8f feat(chain): make output of `SpkTxOutIndex::unused_spks` cloneable (志宇)
Pull request description:
Closes#1268
### Description
Previously `SpkTxOutIndex` methods can be called from `KeychainTxOutIndex` due to the `DeRef` implementation. However, the internal `SpkTxOut` will also contain lookahead spks resulting in an error-prone API.
`SpkTxOutIndex` methods are now not directly callable from `KeychainTxOutIndex`. Methods of `KeychainTxOutIndex` are renamed for clarity. I.e. methods that return an unbounded spk iter are prefixed with `unbounded`.
In addition to this, I also optimized the peek-address logic of `bdk::Wallet` using the optimized `<SpkIterator as Iterator>::nth` implementation.
### Notes to the reviewers
This is mostly refactoring, but can also be considered a bug-fix (as the API before was very problematic).
### Changelog notice
Changed
* Wallet's peek-address logic is optimized by making use of `<SpkIterator as Iterator>::nth`.
* `KeychainTxOutIndex` API is refactored to better differentiate between methods that return unbounded vs stored spks.
* `KeychainTxOutIndex` no longer directly exposes `SpkTxOutIndex` methods via `DeRef`. This was problematic because `SpkTxOutIndex` also contains lookahead spks which we want to hide.
Added
* `SpkIterator::descriptor` method which gets a reference to the internal descriptor.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
~* [ ] I've added tests for the new feature~
* [x] I've added docs for the new feature
#### Bugfixes:
* [x] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
ACKs for top commit:
danielabrozzoni:
ACK 71fff1613d
Tree-SHA512: f29c7d2311d0e81c4fe29b8f57c219c24db958194fad5de82bb6d42d562d37fd5d152be7ee03a3f00843be5760569ad29b848250267a548d7d15320fd5292a8f
`txouts` and `txouts_in_tx` are exposed from `SpkTxOutIndex`, but
modified to remove nested unions.
Add `keychain_outpoints_in_range` that iterates over outpoints of a
given keychain derivation range.
Changes the peek address logic to use the optimized `Iterator::nth`
implementation of `SpkIterator`.
Additionally, docs are added for panics that will occur when the caller
requests for addresses with out-of-bound derivation indices (BIP32).
Previously `SpkTxOutIndex` methods can be called from
`KeychainTxOutIndex` due to the `DeRef` implementation. However, the
internal `SpkTxOut` will also contain lookahead spks resulting in an
error-prone API.
`SpkTxOutIndex` methods are now not directly callable from
`KeychainTxOutIndex`. Methods of `KeychainTxOutIndex` are renamed for
clarity. I.e. methods that return an unbounded spk iter are prefixed
with `unbounded`.
Previously, `apply_block_relevant` used `batch_insert_relevant` which
allows inserting non-topologically-ordered transactions. However,
transactions from blocks are always ordered, so we can avoid looping
through block transactions twice (as done in `batch_insert_relevant`).
Additionally, `apply_block_relevant` now takes in a reference to a
`Block` instead of consuming the `Block`. This makes sense as typically
very few of the transactions in the block are inserted.
Previously, emissions are purely blocks + the block height. This means
emitted blocks can only connect to previous-adjacent blocks. Hence, sync
must start from genesis and include every block.
bf67519768 feat(chain): add `LocalChain::disconnect_from` method (志宇)
Pull request description:
Closes#1271
### Description
Add a method for disconnecting a chain of blocks starting from the given `BlockId`.
### Notes to the reviewers
I want to have this for https://github.com/utreexo/utreexod/pull/110
### Changelog notice
Added
* `LocalChain::disconnect_from` method to evict a chain of blocks starting from a given `BlockId`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
danielabrozzoni:
ACK bf67519768
Tree-SHA512: e6bd213b49b553355370994567722ad2c3460d11fcd81adc65a85e5d03822d3c38e4a4d7f967044991cf0187845467b67d035bf8904871f9fcc4ea61be761ef7
5b77942993 fix(example_electrum): init LocalChain from genesis (vmammal)
Pull request description:
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
<!-- Describe the purpose of this PR, what's being adding and/or fixed -->
This should fix#1252
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
evanlinjin:
ACK 5b77942993
Tree-SHA512: bf049edb2ba22abb949dfd4cb38608589287f3f374f397c82f778b59c21010daa80b63ad85c442c4ee00a2608f89a8d50447b0db85d15caac7fd0b4fd8956e1a
761189ab2b feat(chain): debug_assert non-wildcard desc. to only cache index 0 (志宇)
887e112e8f ref(chain): Refactor reveal_to_target (Daniela Brozzoni)
21d8875826 ref(chain): Refactor next_store_index (Daniela Brozzoni)
Pull request description:
### Description
As a part of #1203, I found myself having to modify `reveal_to_target`, but had some troubles understanding exactly what the method was doing.
This PR refactors `reveal_to_target` to be, in my opinion, a bit clearer. We now exist prematurely if `next_reveal_index` < `target_index`; this way, we don't need to keep track of `next_reveal_index`, as it would always be equal to `Some(target_index)`.
As a part of trying to understand `reveal_to_target` I had to read through `next_store_index`, I slightly improved it for clarity reasons as well.
### Changelog notice
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
evanlinjin:
ACK 761189ab2b
Tree-SHA512: 17f70bccbfa1cccf718ece0eeb61f4944c9108776cf1d606ef823ae9ff58bf52114750927ac99561644ece598f8131ee7f7605071d42c91e5e88e05035e74836
8f6dad76ef Bump bdk_esplora and bdk_file_store versions for 1.0.0-alpha.3 release (Steve Myers)
Pull request description:
### Description
These crates need to have their versions bumped because the version of bdk_chain was bumped to 0.7.0 with the 1.0.0-alpha.3 release.
bdk_esplora to 0.5.0
bdk_file_store to 0.3.0
After this PR is merged I'll publish new versions of these crates to crates.io.
### Notes to the reviewers
I should have done this as part of the 1.0.0-alpha.3 release process. This problem was found when @thunderbiscuit started updating the bdk-ffi project to alpha.3 and we got type mismatch errors for bdk_chain types used in bdk_esplora.
ACKs for top commit:
thunderbiscuit:
ACK 8f6dad76ef.
Tree-SHA512: 735094a5d78da082437118a38db085b12c26665c8cf8fd809d62dbe9a058ee04c59a165e454b3eb8baea4209330083a6a7bce8c303be1660f6593c80ccbe46b7
These crates need to have their versions bumped because the version of bdk_chain was bumped to 0.7.0 with the 1.0.0-alpha.3 release.
bdk_esplora to 0.5.0
bdk_file_store to 0.3.0
Simplify the `reveal_to_target` algorithm by exiting prematurely if
the `target_index` is already revealed.
Since the `reveal_to_index` variable was different from
`Some(target_index)` only if the target was already revealed, we can
getrid of the variable altogether.
105d70e974 ref(hwi): Move hwi out of bdk (Daniela Brozzoni)
Pull request description:
Fixes#872
### Description
This commit moves the `hardwaresigner` outside of bdk and inside `bdk_hwi`
### Notes to the reviewers
There are currently two issues with the code:
- `TransactionSigner` dictates that `sign_transaction` must return a `SignerError` - but being `SignerError` defined inside of bdk, we can't modify it to include an hwi specific error! I don't know how we could fix this (other than getting rid of the trait altogether :)); for now I just added a `SignerError::Generic` variant, lmk if you have better ideas!
- The hwi tests used the bdk utils to get a funded wallet for testing, which aren't available in `bdk_hwi`, which made me realize - maybe we should expose them so that we can use them across our crates, and also our users can use them to test their code?
For now, I just left the test commented.
### Changelog notice
- The old `hardwaresigner` module has been moved out of `bdk` and inside `bdk_hwi`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
~~* [ ] I've added tests for the new feature~~
* [x] I've added docs for the new feature
ACKs for top commit:
notmandatory:
reACK 105d70e974
Tree-SHA512: 9ae3cd035cc27bd7b2831e89c104f40771c6f165cb3bfe1a9052f820050fca7c19f4dd68f171c630a71a7d18ccdee5f94adbc497c6475bd257c2d01cc08109a1
003271117c Bump bdk version to 1.0.0-alpha.3 (Steve Myers)
Pull request description:
### Description
- Bump bdk version to 1.0.0-alpha.3
- Bump bdk_chain to 0.7.0
- Bump bdk_bitcoin_rpc to 0.2.0
- Bump bdk_electrum to 0.5.0
### Notes to the reviewers
### Changelog notice
See #1254
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
evanlinjin:
ACK 003271117c
Tree-SHA512: ac0756f52436880fe633e9ecb83f3d53f485ccfa89a3a89aa51ee4ba5da5cee87f507da69a9e1271f8aaf4425f65d04fb201ea9a4f64bce18f96039ea3548d61
8694624bd5 Bump `bip39` to v2.0 (Elias Rohrer)
Pull request description:
We previously bumped the `bip39` version to 2.0 [in the 0.2X release branch](https://github.com/bitcoindevkit/bdk/pull/875). Back then, bumping it in `master` was [assumed unnecessary](https://github.com/bitcoindevkit/bdk/pull/875#issuecomment-1483990088). It seems this was erroneous, as the current `1.0.1` dependency is present since https://github.com/bitcoindevkit/bdk/pull/793, which was merged before the bump in `release/0.27`.
Here, we therefore bump the crate version in `master` after all.
ACKs for top commit:
notmandatory:
ACK 8694624bd5
Tree-SHA512: a109219bc97bb8e965e8b10e72439aa898b710d1d1a154801ce499ad47475a6b23448d85e0de3f306f990573d1fccdae7d587ed41676a01f91d66a719782eae1
We previously bumped the `bip39` version to 2.0 in the 0.2X release
branch. Back then, bumping it in `master` was erroneously considered unnecessary.
Here, we therefore bump the crate version in `master` after all.
028caa9f8c fix(typos): existant -> existent (Jose Storopoli)
Pull request description:
### Description
These typos are blocking the Nix typo CI in #1257
### Notes to the reviewers
Blocking #1257
### Changelog notice
- fix: typos in documentation
-
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [ ] I've added docs for the new feature
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [ ] I'm linking the issue being fixed by this PR
ACKs for top commit:
notmandatory:
ACK 028caa9f8c
Tree-SHA512: 0bd91efd0eec55fdc537824435552c968858a5b827179b3f9f3f37785db3fa92d3e6f0c73023de0c506224c81217c402d5afa8a2f768fecaf6a3c8378226d184
a1d34afa24 feat(wallet)!: add `NonEmptyDatabase` variant to `NewError` (志宇)
Pull request description:
### Description
`NewError` is the error type when constructing a wallet with `Wallet::new`. We want this to return an error when the database already contains data (in which case, the caller should use `load` or `new_or_load`).
### Notes to the reviewers
This is cherry-picked from #1172 so that we can add it to the alpha.3 release.
### Changelog notice
Change
- Return `NonEmptyDatabase` error when constructing a wallet with `Wallet::new` if the file already contains data (in which case, the caller should use `load` or `new_or_load`).
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
evanlinjin:
ACK a1d34afa24
Tree-SHA512: 7c20171fa3d7dee5b1ac24f8a808781dbb0be0034951005e1e87acdf023123c01161e225b47b6d4484865889778c39549a3780f641227ddc0f84d1577d69f40a
`NewError` is the error type when constructing a wallet with
`Wallet::new`. We want this to return an error when the database already
contains data (in which case, the caller should use `load` or
`new_or_load`).
de54e710ed refactor(esplora_ext): rename scan_txs to sync and scan_txs_with_keychains to full_scan (Steve Myers)
95d34854f4 refactor(electrum_ext): rename scan_without_keychain to sync and scan to full_scan (Steve Myers)
Pull request description:
### Description
fixes#1112
Simple function renaming plus updated docs:
1. electrum_ext: rename functions `scan_without_keychain` to `sync` and `scan` to `full_scan`
2. esplora_ext: rename functions `scan_txs` to `sync` and `scan_txs_with_keychains` to `full_scan`
### Notes to the reviewers
The esplora_ext changes were partially fixed in #1070 but I renamed again so the functions match names ~~suggested in #1112~~ agreed on in discord poll, `sync` and `full_scan`.
### Changelog notice
Changed
- electrum_ext: rename functions scan_without_keychain to sync and scan to full_scan
- esplora_ext: rename functions scan_txs to sync and scan_txs_with_keychains to full_scan
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
Top commit has no ACKs.
Tree-SHA512: d34516ecc513a194b679f73a1260d0cbc3d12b6a2e162d822e7381da0b3250aff319e85ed2fadec506e36f95a78a5cd79d0ab972da2b02928c074be17664da08
1def76f1f1 chore: make clippy happy and bump clippy msrv (志宇)
c9467dcbb2 chore: improve documentation of lookahead (LLFourn)
bc796f412a fix(example): bitcoind_rpc_polling now initializes local_chain properly (志宇)
4fd539b647 feat(chain)!: `KeychainTxOutIndex` uses a universal lookahead (Antoine Poinsot)
Pull request description:
### Description
The `bdk::Wallet` is currently created without setting any lookahead value for the keychain. This implicitly makes it a lookahead of 0. As this is a high-level interface we should avoid footguns and aim for a reasonable default.
To fix this, we have also decided to change `KeychainTxOutIndex` to have a default lookahead. Additionally, we have simplified the `KeychainTxOutIndex` API to have a single `lookahead` that is ONLY set at construction `KeychainTxOutIndex::new(lookahead: u32) -> Self`. This avoids the footguns of having methods which allows the caller to decrease the `lookahead` (which will panic).
### Notes to the reviewers
~A way to set this value externally is introduced in #1172. This PR only aims to use a saner default than 0. `1_000` is the value used by the Bitcoin Core wallet, and that seems reasonable to me.~
Edit: we should NOT allow setting the `lookahead` value after-the-fact. Instead, the `lookahead` should be provided to the wallet's constructor.
@evanlinjin: I don't think additional tests are necessary as no additional features are added, and the surface area of the API is decreased. The original tests already thoroughly test the `lookahead` concept.
### Checklists
#### All Submissions:
*(This section was updated as this PR changed from being a simple setting to introducing a new method.)*
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
~* [ ] I've added tests~
* [x] I've added docs
ACKs for top commit:
LLFourn:
ACK 1def76f1f1
Tree-SHA512: b4c3be8a4f2ac4877cf3f05852147e7dd1daeb02d3bc40895f02fd2a58e584f1dc0735b524153ff0875380ac93c0b4c31e516873d7a9b0027fdbbb5fe7970ff2
