afbf83c8b0 chain(fix): conflict resolution for txs with same last_seen (Wei Chen)
Pull request description:
### Description
Fixes#1102. If a conflicting tx has the same `last_seen`, then we check lexicographical sorting of txids.
### Notes to the reviewers
The tests for this fix exist in the `TxTemplate` structure in #1064 which may need to be merged first.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
danielabrozzoni:
ACK afbf83c8b0
Tree-SHA512: 91b8fbff305b715247501b861ab7ea9e9d9ef99248b05d14e01aacf7e64ad7826f35773e8998cf421dbd04f663714026084c6e817ac5365bce4844c8ea3b7e3f
d5c87c49a8 ref(chain): Rename direct_conflicts_of_tx to... ...direct_conflitcs (Daniela Brozzoni)
009408d243 ci: Pin jobserver to 0.1.26 to keep MSRV (Daniela Brozzoni)
Pull request description:
Fixes#1138
### Changelog notice
- rename `TxGraph::direct_conflicts_of_tx` to `TxGraph::direct_conflicts`
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
evanlinjin:
ACK d5c87c49a8
Tree-SHA512: 10072c65025d5fcc5d90c61ecba1dad53bcd10929326ab18ddadb21322841203aa5861adc9c5f200bbc57b0fe59dd4859dc3672870a0ad343fd8a660fd8aa40e
85c62532a5 docs(bitcoind_rpc): better `Emitter::mempool` explanation (志宇)
b69c13ddf6 example_bitcoind_rpc: tweaks (志宇)
5f34df8489 bitcoind_rpc!: bring back `CheckPoint`s to `Emitter` (志宇)
57590e0a1f bitcoind_rpc: rm `BlockHash` from `Emitter::last_mempool_tip` (志宇)
6d4b33ef91 chain: split `IndexedTxGraph::insert_tx` into 3 methods (志宇)
4f5695d43a chain: improvements to `IndexedTxGraph` and `TxGraph` APIs (志宇)
150d6f8ab6 feat(example_bitcoind_rpc_polling): add example for RPC polling (志宇)
4f10463d9e test(bitcoind_rpc): add no_agreement_point test (志宇)
a73dac2d91 test(bitcoind_rpc): initial tests for `Emitter` (志宇)
bb7424d11d feat(bitcoind_rpc): introduce `bitcoind_rpc` crate (志宇)
240657b167 chain: add batch-insert methods for `IndexedTxGraph` (志宇)
43bc813c64 chain: add helper methods on `CheckPoint` (志宇)
b3db5ca9df feat(chain): add `AnchorFromBlockPosition` trait (志宇)
f795a43cc7 feat(example_cli): allow chain specific args in examples (志宇)
Pull request description:
### Description
This PR builds on top of #1034 and adds the `bitcoind_rpc` chain-src module and example.
### Notes to the reviewers
Don't merge this until #1034 is in!
### Changelog notice
* Add `bitcoind_rpc` chain-source module.
* Add `example_bitcoind_rpc` example module.
* Add `AnchorFromBlockPosition` trait which are for anchors that can be constructed from a given block, height and position in block.
* Add helper methods to `IndexedTxGraph` and `TxGraph` for batch operations and applying blocks directly.
* Add helper methods to `CheckPoint` for easier construction from a block `Header`.
### Checklists
* [x] Add test: we should detect when an initially-confirmed transaction is "unconfirmed" during a reorg.
* [x] Improve `example_bitcoind_rpc`: add `live` command.
* [x] Improve docs.
* [x] Reintroduce `CheckPoint`.
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
notmandatory:
Re ACK 85c62532a5
Tree-SHA512: 88dbafbebaf227b18c69f2ea884e3e586bf9c11e5e450eb4872ade1d1ccd5cf1e33ce9930a6f5aa918baa3e92add7503858b039b8c9d553a281ad6d833f08a49
b1461f05d0 fix(wallet_esplora): missing_heights uses the... ...graph update (Daniela Brozzoni)
Pull request description:
Fixes#1151.
When wallet_esplora_* was used to sync a wallet containing a transaction confirmed some time ago (more than 10-15 blocks ago), the transaction would be stuck in an "unconfirmed" state forever.
At the first scan time, `update_local_chain` would just fetch the last 10 to 15 blocks (depending on the server used), and `tx_graph.missing_heights` wouldn't return the tx's confirmation block as it was called on the original, non-updated tx_graph.
So, after the first scan, we would have a transaction in memory with an anchor that doesn't exist in our local_chain, and try_get_chain_position would return unconfirmed.
When scanning again, missing_heights would find the missing anchor, but `update_local_chain` wouldn't include it as it's older than ASSUME_FINAL_DEPTH.
The missing block would be downloaded every time, but never included in the local_chain, and the transaction would remain unconfirmed forever.
Here we call missing_heights on the updated graph, so that it can correctly return the anchor height, and `update_local_chain` can fetch it and include it in the chain.
### Notes to the reviewers
I'm not sure if this is the right approach, so I'm opening this PR to gather feedback. I still need to add tests, I'll do so once we decide if this is the right way to go or not.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [x] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
ACKs for top commit:
evanlinjin:
ACK b1461f05d0
Tree-SHA512: ba0cf85929644ee737dbc77e6afec662845532de0f120917aa6000ca8f5db79d0cb3971bd92285b5c5b5d26042b60b6c8536f50c9bd49615e31f5da28e80a509
* avoid holding mutex lock over io
* document `CHANNEL_BOUND` const
* use the `relevant` variant of `batch_insert_unconfirmed`
* print elapsed time in stdout for various updates
* `bdk_chain` dependency is added. In the future, we will introduce a
separate `bdk_core` crate to contain shared types.
* replace `Emitter::new` with `from_height` and `from_checkpoint`
* `from_height` emits from the given start height
* `from_checkpoint` uses the provided cp to find agreement point
* introduce logic that ensures emitted blocks can connect with
receiver's `LocalChain`
* in our rpc example, we can now `expect()` chain updates to always
since we are using checkpoints and receiving blocks in order
Instead of comparing the blockhash against the emitted_blocks map
to see whether the block is part of the emitter's best chain, we
reduce the `last_mempool_tip` height to the last agreement height
during the polling logic.
The benefits of this is we have tighter bounds for avoiding re-
emission. Also, it will be easier to replace `emitted_blocks` to
a `CheckPoint` (since we no longer rely on map lookup).
Instead of inserting anchors and seen_at timestamp in the same method,
we have three separate methods. This makes the API easier to understand
and makes `IndexedTxGraph` more consistent with the `TxGraph` API.
For `IndexedTxGraph`:
- Remove `InsertTxItem` type (this is too complex).
- `batch_insert_relevant` now uses a simple tuple `(&tx, anchors)`.
- `batch_insert` is now also removed, as the same functionality can be
done elsewhere.
- Add internal helper method `index_tx_graph_changeset` so we don't need
to create a seprate `TxGraph` update in each method.
- `batch_insert_<relevant>_unconfirmed` no longer takes in an option of
last_seen.
- `batch_insert_unconfirmed` no longer takes a reference of a
transaction (since we apply all transactions anyway, so there is no
need to clone).
For `TxGraph`:
- Add `batch_insert_unconfirmed` method.
* `CheckPoint::from_header` allows us to construct a checkpoint from
block header.
* `CheckPoint::into_update` transforms the cp into a
`local_chain::Update`.
...graph update
Fixes#1151.
When wallet_esplora_* was used to sync a wallet containing a transaction
confirmed some time ago (more than 10-15 blocks ago), the transaction would
be stuck in an "unconfirmed" state forever.
At the first scan time, `update_local_chain` would just fetch the last 10 to
15 blocks (depending on the server used), and `tx_graph.missing_heights`
wouldn't return the tx's confirmation block as it was called on the
original, non-updated tx_graph.
So, after the first scan, we would have a transaction in memory with an
anchor that doesn't exist in our local_chain, and try_get_chain_position
would return unconfirmed.
When scanning again, missing_heights would find the missing anchor, but
`update_local_chain` wouldn't include it as it's older than
ASSUME_FINAL_DEPTH.
The missing block would be downloaded every time, but never included in
the local_chain, and the transaction would remain unconfirmed forever.
Here we call missing_heights on the updated graph, so that it can
correctly return the anchor height, and `update_local_chain` can
fetch it and include it in the chain.
1db3f87a48 chore: pin `byteorder` and `webpki` to fix MSRV (志宇)
Pull request description:
* pin `byteorder` to 1.4.3
* pin `webpki` to 0.22.2
* it's tokio:1.33.0 that needs to be pinned to 1.29.1
* fix README MSRV comments
ACKs for top commit:
danielabrozzoni:
utACK 1db3f87a48
Tree-SHA512: 4c2dc009ea117dd7ca18c7a836eb2b169c68d6ccaf84f85bcc2d4a3bcb7b008ad99f1dadf933b20c9d6ca6cc761e89dbafa99f99e31776e291f480cd6a112b1d
e89cf5a16a refactor: use set_lookahead in set_lookahead_for_all (Vladimir Fomene)
Pull request description:
### Description
Use set_lookahead in set_lookahead_for_all.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
realeinherjar:
ACK: e89cf5a16a
evanlinjin:
ACK e89cf5a16a
Tree-SHA512: 02d226be7adcfd5e23ecb9d17539b0e089cb55ddf9c6de980155960f8181f2d3ea3ac9a93b2c1b7e0a8d4e4c821d92f65405852c696c9a367193a42d60c1aac6
6d601a7e88 test(chain): Add test for conflicting transactions (Daniela Brozzoni)
48ca95b541 test(chain): Add test for walk_ancestors (Daniela Brozzoni)
59a2403e28 test(chain): Introduce TxTemplate (Daniela Brozzoni)
6e511473a5 test(chain): add block_id! utility macro (Daniela Brozzoni)
62de55f12d fix(chain): Consider conflicting ancestors in... ...try_get_chain_pos (Daniela Brozzoni)
a3e8480ad9 doc(chain): Clarify direct_conflicts_of_tx's docs (Daniela Brozzoni)
4742d88ea3 feat(chain): Introduce TxAncestors, walk_ancestors (Daniela Brozzoni)
2f26eca607 fix(chain): TxDescendants performs a BFS (Daniela Brozzoni)
486e0e1437 doc(chain): Fix typos (Daniela Brozzoni)
Pull request description:
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
Fixes#1063.
This PR introduces a new `TxTemplate` struct to test different transaction conflict scenarios in `TxGraph`.
The following transaction conflict scenarios are tested:
- 2 unconfirmed txs with different last_seens conflict. The most recent tx should be the only tx that appears in the list methods.
- 3 unconfirmed txs with different last_seens conflict. The most recent tx should be the only tx that appears in the list methods.
- An unconfirmed tx U conflicts with a tx anchored in orphaned block O. O has higher last_seen. O should be the only tx that appears in the list methods.
- An unconfirmed tx U conflicts with a tx anchored in orphaned block O. U has higher last_seen. U should be the only tx that appears in the list methods.
- Multiple unconfirmed txs conflict with a confirmed tx. None of the unconfirmed txs should appear in the list methods.
- B and B' conflict. C spends B. B' is anchored in best chain. B and C should not appear in the list methods.
- B and B' conflict. C spends B. B is anchored in best chain. B' should not appear in the list methods.
- B and B' conflict. C spends both B and B'. C is impossible.
- B and B' conflict. C spends both B and B'. C is impossible. B' is confirmed.
- B and B' conflict. C spends both B and B'. C is impossible. D spends C.
These tests revealed that `TxGraph::walk_conflicts` was not checking ancestors of the root tx for conflicts. `TxGraph::walk_conflicts` has been refactored to check for conflicting ancestor transactions by using a new `TxAncestors` iterator in `TxGraph`.
### Changelog notice
- Introduced `tx_template` module
- Introduced `TxGraph::TxAncestors` iterator
- Refactored `TxGraph::walk_conflicts` to use `TxGraph::TxAncestors`
- Added `walk_ancestors` to `TxGraph`
### Checklists
All Submissions:
- [x] I've signed all my commits
- [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
- [x] I ran cargo fmt and cargo clippy before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
evanlinjin:
ACK 6d601a7e88
Tree-SHA512: ea151392874c4312233e4e10299579f4eee4a7100ae344b4d7f19994284b49c1e43f37338bed931d16e77326021166ea0b94d6de3ccf50a8fabb25139a8e69b4
...try_get_chain_pos
In try_get_chain_pos, when we notice that a transaction is not included
in the best chain, we check the transactions in mempool to find
conflicting ones, and decide based on that if our transaction is still
in mempool or has been dropped.
This commit adds a check for transactions conflicting with the
unconfirmed ancestors of our tx.
Co-authored-by: Wei Chen <wzc110@gmail.com>
f28e665c7d ci: update code_coverage workflow to use rust stable version (Steve Myers)
edfd4c236d fix: Add the pinning of time/cc in README (Daniela Brozzoni)
Pull request description:
Should have been in #1135, sorry.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
notmandatory:
ACK f28e665c7d
Tree-SHA512: e1c2c05f436da1fd59b9988628318c0563b2dbe85a5f4fb8aef4c35d7601c441a542e8a2c3591c02f3a05ad9456cfc2d3f5e8605064498a56a39766f5bcbc2e1
ea6411c685 ci: Pin time, cc in order to keep MSRV (Daniela Brozzoni)
Pull request description:
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
notmandatory:
ACK ea6411c685
Tree-SHA512: e2400bd4a1583db8503efa05279f1d97b136dfbe63ae92bc33817fa6763f7f3b8789acf9aef0f36bb8a0486539637eaa101f638d223354d3c6193f2ab2ebdfdb
