Bump version to 0.22.0

d7163c3a97 Update electrum-client to 0.11.0 (Alekos Filini)

Pull request description:

  ### Description

  Update electrum-client to 0.11.0

  ### Changelog notice

  - Updated `electrum-client` to `0.11.0`

  ### Checklists

  #### All Submissions:

  * [x] I've signed all my commits
  * [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
  * [x] I ran `cargo fmt` and `cargo clippy` before committing

  #### New Features:

  * [ ] I've added tests for the new feature
  * [ ] I've added docs for the new feature
  * [x] I've updated `CHANGELOG.md`

ACKs for top commit:
  danielabrozzoni:
    utACK d7163c3a97

Tree-SHA512: c8c295cc37f8b24e5b61447401df9044cdd9f92a577d82aa3fb9882f135245395651edff1b8710fe40897b8ce081344ac3cc98aefc7e37e22ffbbdcc419b5135

.github/ISSUE_TEMPLATE/minor_release.md

@@ -0,0 +1,89 @@
+---
+name: Minor Release
+about: Create a new minor release [for release managers only]
+title: 'Release MAJOR.MINOR+1.0'
+labels: 'release'
+assignees: ''
+
+---
+
+## Create a new minor release
+
+### Summary
+
+<--release summary to be used in announcements-->
+
+### Commit
+
+<--latest commit ID to include in this release-->
+
+### Changelog
+
+<--add notices from PRs merged since the prior release, see ["keep a changelog"]-->
+
+### Checklist 
+
+Release numbering must follow [Semantic Versioning]. These steps assume the current `master` 
+branch **development** version is *MAJOR.MINOR.0*.
+
+#### On the day of the feature freeze
+
+Change the `master` branch to the next MINOR+1 version:
+
+- [ ] Switch to the `master` branch.
+- [ ] Create a new PR branch called `bump_dev_MAJOR_MINOR+1`, eg. `bump_dev_0_22`.
+- [ ] Bump the `bump_dev_MAJOR_MINOR+1` branch to the next development MINOR+1 version.
+  - Change the `Cargo.toml` version value to `MAJOR.MINOR+1.0`.
+  - The commit message should be "Bump version to MAJOR.MINOR+1.0".
+- [ ] Create PR and merge the `bump_dev_MAJOR_MINOR+1` branch to `master`.
+  - Title PR "Bump version to MAJOR.MINOR+1.0".
+  
+Create a new release branch and release candidate tag:
+
+- [ ] Double check that your local `master` is up-to-date with the upstream repo.
+- [ ] Create a new branch called `release/MAJOR.MINOR+1` from `master`.
+- [ ] Add a tag to the `HEAD` commit in the `release/MAJOR.MINOR+1` branch.
+  - The tag name should be `vMAJOR.MINOR+1.0-RC.1`
+  - Use message "Release MAJOR.MINOR+1.0 RC.1".  
+  - Make sure the tag is signed, for extra safety use the explicit `--sign` flag.
+- [ ] Push the `release/MAJOR.MINOR` branch and new tag to the `bitcoindevkit/bdk` repo.
+  - Use `git push --tags` option to push the new `vMAJOR.MINOR+1.0-RC.1` tag.
+  
+If any issues need to be fixed before the *MAJOR.MINOR+1.0* version is released:
+
+- [ ] Merge fix PRs to the `master` branch.
+- [ ] Git cherry-pick fix commits to the `release/MAJOR.MINOR+1` branch.
+- [ ] Verify fixes in `release/MAJOR.MINOR+1` branch.
+- [ ] Add a tag to the `HEAD` commit in the `release/MAJOR.MINOR+1` branch.
+  - The tag name should be `vMAJOR.MINOR+1.0-RC.x+1`, where x is the current release candidate number.
+  - Use tag message "Release MAJOR.MINOR+1.0 RC.x+1".  
+  - Make sure the tag is signed, for extra safety use the explicit `--sign` flag.
+- [ ] Push the new tag to the `bitcoindevkit/bdk` repo.
+  - Use `git push --tags` option to push the new `vMAJOR.MINOR+1.0-RC.x+1` tag.
+
+#### On the day of the release
+
+Tag and publish new release:
+
+- [ ] Add a tag to the `HEAD` commit in the `release/MAJOR.MINOR+1` branch.
+  - The tag name should be `vMAJOR.MINOR+1.0`
+  - The first line of the tag message should be "Release MAJOR.MINOR+1.0".
+  - In the body of the tag message put a copy of the **Summary** and **Changelog** for the release.
+  - Make sure the tag is signed, for extra safety use the explicit `--sign` flag.
+- [ ] Wait for the CI to finish one last time.
+- [ ] Push the new tag to the `bitcoindevkit/bdk` repo.
+- [ ] Publish **all** the updated crates to crates.io.
+- [ ] Create the release on GitHub.
+  - Go to "tags", click on the dots on the right and select "Create Release".
+  - Set the title to `Release MAJOR.MINOR+1.0`.
+  - In the release notes body put the **Summary** and **Changelog**.
+  - Use the "+ Auto-generate release notes" button to add details from included PRs.
+  - Until we reach a `1.0.0` release check the "Pre-release" box.
+- [ ] Make sure the new release shows up on [crates.io] and that the docs are built correctly on [docs.rs].
+- [ ] Announce the release, using the **Summary**, on Discord, Twitter and Mastodon.
+- [ ] Celebrate 🎉
+
+[Semantic Versioning]: https://semver.org/
+[crates.io]: https://crates.io/crates/bdk
+[docs.rs]: https://docs.rs/bdk/latest/bdk
+["keep a changelog"]: https://keepachangelog.com/en/1.0.0/

.github/ISSUE_TEMPLATE/patch_release.md

@@ -0,0 +1,67 @@
+---
+name: Patch Release
+about: Create a new patch release [for release managers only]
+title: 'Release MAJOR.MINOR.PATCH+1'
+labels: 'release'
+assignees: ''
+
+---
+
+## Create a new patch release
+
+### Summary
+
+<--release summary to be used in announcements-->
+
+### Commit
+
+<--latest commit ID to include in this release-->
+
+### Changelog
+
+<--add notices from PRs merged since the prior release, see ["keep a changelog"]-->
+
+### Checklist  
+
+Release numbering must follow [Semantic Versioning]. These steps assume the current `master` 
+branch **development** version is *MAJOR.MINOR.PATCH*.
+
+### On the day of the patch release
+
+Change the `master` branch to the new PATCH+1 version:
+
+- [ ] Switch to the `master` branch.
+- [ ] Create a new PR branch called `bump_dev_MAJOR_MINOR_PATCH+1`, eg. `bump_dev_0_22_1`.
+- [ ] Bump the `bump_dev_MAJOR_MINOR` branch to the next development PATCH+1 version.
+  - Change the `Cargo.toml` version value to `MAJOR.MINOR.PATCH+1`.
+  - The commit message should be "Bump version to MAJOR.MINOR.PATCH+1".
+- [ ] Create PR and merge the `bump_dev_MAJOR_MINOR_PATCH+1` branch to `master`.
+  - Title PR "Bump version to MAJOR.MINOR.PATCH+1".
+
+Cherry-pick, tag and publish new PATCH+1 release:
+
+- [ ] Merge fix PRs to the `master` branch.
+- [ ] Git cherry-pick fix commits to the `release/MAJOR.MINOR` branch to be patched.
+- [ ] Verify fixes in `release/MAJOR.MINOR` branch.
+- [ ] Add a tag to the `HEAD` commit in the `release/MAJOR.MINOR` branch.
+  - The tag name should be `vMAJOR.MINOR.PATCH+1`
+  - The first line of the tag message should be "Release MAJOR.MINOR.PATCH+1".
+  - In the body of the tag message put a copy of the **Summary** and **Changelog** for the release.
+  - Make sure the tag is signed, for extra safety use the explicit `--sign` flag.
+- [ ] Wait for the CI to finish one last time.
+- [ ] Push the new tag to the `bitcoindevkit/bdk` repo.
+- [ ] Publish **all** the updated crates to crates.io.
+- [ ] Create the release on GitHub.
+  - Go to "tags", click on the dots on the right and select "Create Release".
+  - Set the title to `Release MAJOR.MINOR.PATCH+1`.
+  - In the release notes body put the **Summary** and **Changelog**.
+  - Use the "+ Auto-generate release notes" button to add details from included PRs.
+  - Until we reach a `1.0.0` release check the "Pre-release" box.
+- [ ] Make sure the new release shows up on [crates.io] and that the docs are built correctly on [docs.rs].
+- [ ] Announce the release, using the **Summary**, on Discord, Twitter and Mastodon.
+- [ ] Celebrate 🎉
+
+[Semantic Versioning]: https://semver.org/
+[crates.io]: https://crates.io/crates/bdk
+[docs.rs]: https://docs.rs/bdk/latest/bdk
+["keep a changelog"]: https://keepachangelog.com/en/1.0.0/

.github/pull_request_template.md

@@ -9,6 +9,11 @@
 <!-- In this section you can include notes directed to the reviewers, like explaining why some parts
 of the PR were done in a specific way -->
 
+### Changelog notice
+
+<!-- Notice the release manager should include in the release tag message changelog -->
+<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
+
 ### Checklists
 
 #### All Submissions:
@@ -21,7 +26,6 @@ of the PR were done in a specific way -->
 
 * [ ] I've added tests for the new feature
 * [ ] I've added docs for the new feature
-* [ ] I've updated `CHANGELOG.md`
 
 #### Bugfixes:
 

.github/workflows/code_coverage.yml

@@ -3,35 +3,53 @@ on: [push]
 name: Code Coverage
 
 jobs:
-
   Codecov:
     name: Code Coverage
     runs-on: ubuntu-latest
     env:
-      CARGO_INCREMENTAL: '0'
-      RUSTFLAGS: '-Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off'
-      RUSTDOCFLAGS: '-Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off'
+      RUSTFLAGS: "-Cinstrument-coverage"
+      RUSTDOCFLAGS: "-Cinstrument-coverage"
+      LLVM_PROFILE_FILE: "report-%p-%m.profraw"
 
     steps:
       - name: Checkout
         uses: actions/checkout@v2
+      - name: Install lcov tools
+        run: sudo apt-get install lcov -y
       - name: Install rustup
         run: curl https://sh.rustup.rs -sSf | sh -s -- -y
       - name: Set default toolchain
         run: rustup default nightly
       - name: Set profile
         run: rustup set profile minimal
+      - name: Add llvm tools
+        run: rustup component add llvm-tools-preview
       - name: Update toolchain
         run: rustup update
-      - name: Test
-        run: cargo test --features all-keys,compiler,esplora,ureq,compact_filters --no-default-features
-                    
-      - id: coverage
-        name: Generate coverage
-        uses: actions-rs/grcov@v0.1.5
-
-      - name: Upload coverage to Codecov
-        uses: codecov/codecov-action@v2
+      - name: Cache cargo
+        uses: actions/cache@v3
         with:
-          file: ${{ steps.coverage.outputs.report }}
-          directory: ./coverage/reports/
+          path: |
+            ~/.cargo/bin/
+            ~/.cargo/registry/index/
+            ~/.cargo/registry/cache/
+            ~/.cargo/git/db/
+          key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
+      - name: Install grcov
+        run: if [[ ! -e ~/.cargo/bin/grcov ]]; then cargo install grcov; fi
+      - name: Test
+        run: cargo test --features default,minimal,all-keys,compact_filters,key-value-db,compiler,sqlite,sqlite-bundled,test-electrum,verify,test-rpc
+      - name: Run grcov
+        run: mkdir coverage; grcov . --binary-path ./target/debug/ -s . -t lcov --branch --ignore-not-existing --ignore '/*' -o ./coverage/lcov.info
+      - name: Generate HTML coverage report
+        run: genhtml -o coverage-report.html ./coverage/lcov.info
+
+      - name: Coveralls upload
+        uses: coverallsapp/github-action@master
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+      - name: Upload artifact
+        uses: actions/upload-artifact@v2
+        with:
+          name: coverage-report
+          path: coverage-report.html

.github/workflows/cont_integration.yml

@@ -12,7 +12,7 @@ jobs:
         rust:
           - version: 1.60.0 # STABLE
             clippy: true
-          - version: 1.56.0 # MSRV
+          - version: 1.56.1 # MSRV
         features:
           - default
           - minimal
@@ -146,7 +146,7 @@ jobs:
       - run: sudo apt-get update || exit 1
       - run: sudo apt-get install -y libclang-common-10-dev clang-10 libc6-dev-i386 || exit 1
       - name: Set default toolchain
-        run: rustup default 1.56.0 # STABLE
+        run: rustup default 1.56.1 # STABLE
       - name: Set profile
         run: rustup set profile minimal
       - name: Add target wasm32
@@ -172,3 +172,32 @@ jobs:
         run: rustup update
       - name: Check fmt
         run: cargo fmt --all -- --config format_code_in_doc_comments=true --check
+
+  test_harware_wallet:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        rust:
+          - version: 1.60.0 # STABLE
+          - version: 1.56.1 # MSRV
+    steps:
+    - name: Checkout
+      uses: actions/checkout@v3
+    - name: Build simulator image
+      run: docker build -t hwi/ledger_emulator ./ci -f ci/Dockerfile.ledger
+    - name: Run simulator image
+      run: docker run --name simulator --network=host hwi/ledger_emulator &
+    - name: Install Python
+      uses: actions/setup-python@v4
+      with:
+        python-version: '3.9'
+    - name: Install python dependencies
+      run: pip install hwi==2.1.1 protobuf==3.20.1
+    - name: Set default toolchain
+      run: rustup default ${{ matrix.rust.version }}
+    - name: Set profile
+      run: rustup set profile minimal
+    - name: Update toolchain
+      run: rustup update
+    - name: Test
+      run: cargo test --features test-hardware-signer

CHANGELOG.md

@@ -1,10 +1,35 @@
 # Changelog
-All notable changes to this project will be documented in this file.
+All notable changes to this project prior to release **0.22.0** are documented in this file. Future
+changelog information can be found in each release's git tag and can be viewed with `git tag -ln100 "v*"`.
+Changelog info is also documented on the [GitHub releases](https://github.com/bitcoindevkit/bdk/releases)
+page. See [DEVELOPMENT_CYCLE.md](DEVELOPMENT_CYCLE.md) for more details.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [v0.21.0] - [v0.20.0]
+
+- Add `descriptor::checksum::get_checksum_bytes` method.
+- Add `Excess` enum to handle remaining amount after coin selection.
+- Move change creation from `Wallet::create_tx` to `CoinSelectionAlgorithm::coin_select`.
+- Change the interface of `SqliteDatabase::new` to accept any type that implement AsRef<Path>
+- Add the ability to specify which leaves to sign in a taproot transaction through `TapLeavesOptions` in `SignOptions`
+- Add the ability to specify whether a taproot transaction should be signed using the internal key or not, using `sign_with_tap_internal_key` in `SignOptions`
+- Consolidate params `fee_amount` and `amount_needed` in `target_amount` in `CoinSelectionAlgorithm::coin_select` signature.
+- Change the meaning of the `fee_amount` field inside `CoinSelectionResult`: from now on the `fee_amount` will represent only the fees asociated with the utxos in the `selected` field of `CoinSelectionResult`.
+- New `RpcBlockchain` implementation with various fixes.
+- Return balance in separate categories, namely `confirmed`, `trusted_pending`, `untrusted_pending` & `immature`.
+
+## [v0.20.0] - [v0.19.0]
+
+- New MSRV set to `1.56.1`
+- Fee sniping discouraging through nLockTime - if the user specifies a `current_height`, we use that as a nlocktime, otherwise we use the last sync height (or 0 if we never synced)
+- Fix hang when `ElectrumBlockchainConfig::stop_gap` is zero.
+- Set coin type in BIP44, BIP49, and BIP84 templates
+- Get block hash given a block height - A `get_block_hash` method is now defined on the `GetBlockHash` trait and implemented on every blockchain backend. This method expects a block height and returns the corresponding block hash.
+- Add `remove_partial_sigs` and `try_finalize` to `SignOptions`
+- Deprecate `AddressValidator`
+- Fix Electrum wallet sync potentially causing address index decrement - compare proposed index and current index before applying batch operations during sync.
 
 ## [v0.19.0] - [v0.18.0]
 
@@ -13,7 +38,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Unpinned tokio to `1`
 - Add traits to reuse `Blockchain`s across multiple wallets (`BlockchainFactory` and `StatelessBlockchain`).
 - Upgrade to rust-bitcoin `0.28`
-- If using the `sqlite-db` feature all cached wallet data is deleted due to a possible UTXO inconsistency, a wallet.sync will recreate it  
+- If using the `sqlite-db` feature all cached wallet data is deleted due to a possible UTXO inconsistency, a wallet.sync will recreate it
 - Update `PkOrF` in the policy module to become an enum
 - Add experimental support for Taproot, including:
   - Support for `tr()` descriptors with complex tapscript trees
@@ -24,13 +49,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [v0.18.0] - [v0.17.0]
 
-- Add `sqlite-bundled` feature for deployments that need a bundled version of sqlite, ie. for mobile platforms.
+- Add `sqlite-bundled` feature for deployments that need a bundled version of sqlite, i.e. for mobile platforms.
 - Added `Wallet::get_signers()`, `Wallet::descriptor_checksum()` and `Wallet::get_address_validators()`, exposed the `AsDerived` trait.
 - Deprecate `database::Database::flush()`, the function is only needed for the sled database on mobile, instead for mobile use the sqlite database.
 - Add `keychain: KeychainKind` to `wallet::AddressInfo`.
 - Improve key generation traits
 - Rename `WalletExport` to `FullyNodedExport`, deprecate the former.
-- Bump `miniscript` dependency version to `^6.1`. 
+- Bump `miniscript` dependency version to `^6.1`.
 
 ## [v0.17.0] - [v0.16.1]
 
@@ -51,6 +76,7 @@ To decouple the `Wallet` from the `Blockchain` we've made major changes:
 - Stop making a request for the block height when calling `Wallet:new`.
 - Added `SyncOptions` to capture extra (future) arguments to `Wallet::sync`.
 - Removed `max_addresses` sync parameter which determined how many addresses to cache before syncing since this can just be done with `ensure_addresses_cached`.
+- remove `flush` method from the `Database` trait.
 
 ## [v0.16.1] - [v0.16.0]
 
@@ -465,4 +491,5 @@ final transaction is created by calling `finish` on the builder.
 [v0.17.0]: https://github.com/bitcoindevkit/bdk/compare/v0.16.1...v0.17.0
 [v0.18.0]: https://github.com/bitcoindevkit/bdk/compare/v0.17.0...v0.18.0
 [v0.19.0]: https://github.com/bitcoindevkit/bdk/compare/v0.18.0...v0.19.0
-[unreleased]: https://github.com/bitcoindevkit/bdk/compare/v0.19.0...HEAD
+[v0.20.0]: https://github.com/bitcoindevkit/bdk/compare/v0.19.0...v0.20.0
+[v0.21.0]: https://github.com/bitcoindevkit/bdk/compare/v0.20.0...v0.21.0

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "bdk"
-version = "0.19.1-dev"
+version = "0.22.0"
 edition = "2018"
 authors = ["Alekos Filini <alekos.filini@gmail.com>", "Riccardo Casatta <riccardo@casatta.it>"]
 homepage = "https://bitcoindevkit.org"
@@ -22,9 +22,9 @@ rand = "^0.7"
 
 # Optional dependencies
 sled = { version = "0.34", optional = true }
-electrum-client = { version = "0.10", optional = true }
-rusqlite = { version = "0.25.3", optional = true }
-ahash = { version = "=0.7.4", optional = true }
+electrum-client = { version = "0.11", optional = true }
+rusqlite = { version = "0.27.0", optional = true }
+ahash = { version = "0.7.6", optional = true }
 reqwest = { version = "0.11", optional = true, default-features = false, features = ["json"] }
 ureq = { version = "~2.2.0", features = ["json"], optional = true }
 futures = { version = "0.3", optional = true }
@@ -33,6 +33,7 @@ rocksdb = { version = "0.14", default-features = false, features = ["snappy"], o
 cc = { version = ">=1.0.64", optional = true }
 socks = { version = "0.3", optional = true }
 lazy_static = { version = "1.4", optional = true }
+hwi = { version = "0.2.2", optional = true }
 
 bip39 = { version = "1.0.1", optional = true }
 bitcoinconsensus = { version = "0.19.0-3", optional = true }
@@ -61,6 +62,7 @@ key-value-db = ["sled"]
 all-keys = ["keys-bip39"]
 keys-bip39 = ["bip39"]
 rpc = ["bitcoincore-rpc"]
+hardware-signer = ["hwi"]
 
 # We currently provide mulitple implementations of `Blockchain`, all are
 # blocking except for the `EsploraBlockchain` which can be either async or
@@ -93,12 +95,13 @@ test-rpc = ["rpc", "electrsd/electrs_0_8_10", "electrsd/bitcoind_22_0", "test-bl
 test-rpc-legacy = ["rpc", "electrsd/electrs_0_8_10", "electrsd/bitcoind_0_20_0", "test-blockchains"]
 test-esplora = ["electrsd/legacy", "electrsd/esplora_a33e97e1", "electrsd/bitcoind_22_0", "test-blockchains"]
 test-md-docs = ["electrum"]
+test-hardware-signer = ["hardware-signer"]
 
 [dev-dependencies]
 lazy_static = "1.4"
 env_logger = "0.7"
 clap = "2.33"
-electrsd = "0.19.1"
+electrsd = "0.20"
 
 [[example]]
 name = "address_validator"
@@ -114,11 +117,11 @@ required-features = ["compiler"]
 [[example]]
 name = "rpcwallet"
 path = "examples/rpcwallet.rs"
-required-features = ["keys-bip39", "key-value-db", "rpc"]
+required-features = ["keys-bip39", "key-value-db", "rpc", "electrsd/bitcoind_22_0"]
 
 [workspace]
 members = ["macros"]
 [package.metadata.docs.rs]
-features = ["compiler", "electrum", "esplora", "ureq", "compact_filters", "rpc", "key-value-db", "sqlite", "all-keys", "verify"]
+features = ["compiler", "electrum", "esplora", "use-esplora-ureq", "compact_filters", "rpc", "key-value-db", "sqlite", "all-keys", "verify", "hardware-signer"]
 # defines the configuration attribute `docsrs`
 rustdoc-args = ["--cfg", "docsrs"]

DEVELOPMENT_CYCLE.md

@@ -1,46 +1,16 @@
 # Development Cycle
 
-This project follows a regular releasing schedule similar to the one [used by the Rust language](https://doc.rust-lang.org/book/appendix-07-nightly-rust.html). In short, this means that a new release is made at a regular cadence, with all the feature/bugfixes that made it to `master` in time. This ensures that we don't keep delaying releases waiting for "just one more little thing".
+This project follows a regular releasing schedule similar to the one [used by the Rust language]. In short, this means that a new release is made at a regular cadence, with all the feature/bugfixes that made it to `master` in time. This ensures that we don't keep delaying releases waiting for "just one more little thing".
+
+This project uses [Semantic Versioning], but is currently at MAJOR version zero (0.y.z) meaning it is still in initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable. Until we reach version `1.0.0` we will do our best to document any breaking API changes in the changelog info attached to each release tag.
 
 We decided to maintain a faster release cycle while the library is still in "beta", i.e. before release `1.0.0`: since we are constantly adding new features and, even more importantly, fixing issues, we want developers to have access to those updates as fast as possible. For this reason we will make a release **every 4 weeks**.
 
-Once the project will have reached a more mature state (>= `1.0.0`), we will very likely switch to longer release cycles of **6 weeks**.
+Once the project reaches a more mature state (>= `1.0.0`), we will very likely switch to longer release cycles of **6 weeks**.
 
 The "feature freeze" will happen **one week before the release date**. This means a new branch will be created originating from the `master` tip at that time, and in that branch we will stop adding new features and only focus on ensuring the ones we've added are working properly.
 
-```
-master:           - - - - * - - - * - - - - - - * - - - * ...
-                          |      /              |       |
-release/0.x.0:            * - - #               |       |
-                                                |      /
-release/0.y.0:                                  * - - #
-```
+To create a new release a release manager will create a new issue using the `Release` template and follow the template instructions.
 
-As soon as the release is tagged and published, the `release` branch will be merged back into `master` to update the version in the `Cargo.toml` to apply the new `Cargo.toml` version and all the other fixes made during the feature freeze window.
-
-## Making the Release
-
-What follows are notes and procedures that maintainers can refer to when making releases. All the commits and tags must be signed and, ideally, also [timestamped](https://github.com/opentimestamps/opentimestamps-client/blob/master/doc/git-integration.md).
-
-Pre-`v1.0.0` our "major" releases only affect the "minor" semver value. Accordingly, our "minor" releases will only affect the "patch" value.
-
-1. Create a new branch called `release/x.y.z` from `master`. Double check that your local `master` is up-to-date with the upstream repo before doing so.
-2. Make a commit on the release branch to bump the version to `x.y.z-rc.1`. The message should be "Bump version to x.y.z-rc.1".
-3. Push the new branch to `bitcoindevkit/bdk` on GitHub.
-4. During the one week of feature freeze run additional tests on the release branch.
-5. If a bug is found:
-    - If it's a minor issue you can just fix it in the release branch, since it will be merged back to `master` eventually
-    - For bigger issues you can fix them on `master` and then *cherry-pick* the commit to the release branch
-6. Update the changelog with the new release version.
-7. Update `src/lib.rs` with the new version (line ~43)
-8. On release day, make a commit on the release branch to bump the version to `x.y.z`. The message should be "Bump version to x.y.z".
-9. Add a tag to this commit. The tag name should be `vx.y.z` (for example `v0.5.0`), and the message "Release x.y.z". Make sure the tag is signed, for extra safety use the explicit `--sign` flag.
-10. Push the new commits to the upstream release branch, wait for the CI to finish one last time.
-11. Publish **all** the updated crates to crates.io.
-12. Make a new commit to bump the version value to `x.y.(z+1)-dev`. The message should be "Bump version to x.y.(z+1)-dev".
-13. Merge the release branch back into `master`.
-14. If the `master` branch contains any unreleased changes to the `bdk-macros` crate, change the `bdk` Cargo.toml `[dependencies]` to point to the local path (ie. `bdk-macros = { path = "./macros"}`)
-15. Create the release on GitHub: go to "tags", click on the dots on the right and select "Create Release". Then set the title to `vx.y.z` and write down some brief release notes.
-16. Make sure the new release shows up on crates.io and that the docs are built correctly on docs.rs.
-17. Announce the release on Twitter, Discord and Telegram.
-18. Celebrate :tada:
+[used by the Rust language]: https://doc.rust-lang.org/book/appendix-07-nightly-rust.html
+[Semantic Versioning]: https://semver.org/

README.md

@@ -11,9 +11,9 @@
     <a href="https://crates.io/crates/bdk"><img alt="Crate Info" src="https://img.shields.io/crates/v/bdk.svg"/></a>
     <a href="https://github.com/bitcoindevkit/bdk/blob/master/LICENSE"><img alt="MIT or Apache-2.0 Licensed" src="https://img.shields.io/badge/license-MIT%2FApache--2.0-blue.svg"/></a>
     <a href="https://github.com/bitcoindevkit/bdk/actions?query=workflow%3ACI"><img alt="CI Status" src="https://github.com/bitcoindevkit/bdk/workflows/CI/badge.svg"></a>
-    <a href="https://codecov.io/gh/bitcoindevkit/bdk"><img src="https://codecov.io/gh/bitcoindevkit/bdk/branch/master/graph/badge.svg"/></a>
+    <a href="https://coveralls.io/github/bitcoindevkit/bdk?branch=master"><img src="https://coveralls.io/repos/github/bitcoindevkit/bdk/badge.svg?branch=master"/></a>
     <a href="https://docs.rs/bdk"><img alt="API Docs" src="https://img.shields.io/badge/docs.rs-bdk-green"/></a>
-    <a href="https://blog.rust-lang.org/2020/08/27/Rust-1.56.0.html"><img alt="Rustc Version 1.56+" src="https://img.shields.io/badge/rustc-1.56%2B-lightgrey.svg"/></a>
+    <a href="https://blog.rust-lang.org/2021/11/01/Rust-1.56.1.html"><img alt="Rustc Version 1.56.1+" src="https://img.shields.io/badge/rustc-1.56.1%2B-lightgrey.svg"/></a>
     <a href="https://discord.gg/d7NkDKm"><img alt="Chat on Discord" src="https://img.shields.io/discord/753336465005608961?logo=discord"></a>
   </p>
 
@@ -95,6 +95,7 @@ use bdk::blockchain::ElectrumBlockchain;
 use bdk::electrum_client::Client;
 use bdk::wallet::AddressIndex::New;
 
+use bitcoin::base64;
 use bitcoin::consensus::serialize;
 
 fn main() -> Result<(), bdk::Error> {
@@ -131,6 +132,7 @@ fn main() -> Result<(), bdk::Error> {
 ```rust,no_run
 use bdk::{Wallet, SignOptions, database::MemoryDatabase};
 
+use bitcoin::base64;
 use bitcoin::consensus::deserialize;
 
 fn main() -> Result<(), bdk::Error> {
@@ -154,7 +156,7 @@ fn main() -> Result<(), bdk::Error> {
 
 ### Unit testing
 
-```
+```bash
 cargo test
 ```
 
@@ -162,7 +164,7 @@ cargo test
 
 Integration testing require testing features, for example:
 
-```
+```bash
 cargo test --features test-electrum
 ```
 

ci/Dockerfile.ledger

@@ -0,0 +1,9 @@
+# Taken from bitcoindevkit/rust-hwi
+FROM ghcr.io/ledgerhq/speculos
+
+RUN apt-get update
+RUN apt-get install wget -y
+RUN wget "https://github.com/LedgerHQ/speculos/blob/master/apps/nanos%23btc%232.1%231c8db8da.elf?raw=true" -O /speculos/btc.elf
+ADD automation.json /speculos/automation.json
+
+ENTRYPOINT ["python", "./speculos.py", "--automation", "file:automation.json", "--display", "headless", "--vnc-port", "41000", "btc.elf"]

ci/automation.json

@@ -0,0 +1,30 @@
+{
+    "version": 1,
+    "rules": [
+        {
+            "regexp": "Address \\(\\d/\\d\\)|Message hash \\(\\d/\\d\\)|Confirm|Fees|Review|Amount",
+            "actions": [
+                [ "button", 2, true ],
+                [ "button", 2, false ]
+            ]
+        },
+        {
+            "text": "Sign",
+            "conditions": [
+                [ "seen", false ]
+            ],
+            "actions": [
+                [ "button", 2, true ],
+                [ "button", 2, false ],
+                [ "setbool", "seen", true ]
+            ]
+        },
+        {
+            "regexp": "Approve|Sign|Accept",
+            "actions": [
+                [ "button", 3, true ],
+                [ "button", 3, false ]
+            ]
+        }
+    ]
+}

codecov.yaml

@@ -1,13 +0,0 @@
-coverage:
-  status:
-    project:
-      default:
-        target: auto
-        threshold: 1%
-        base: auto
-        informational: false
-    patch:
-      default:
-        target: auto
-        threshold: 100%
-        base: auto

examples/address_validator.rs

@@ -14,6 +14,7 @@ use std::sync::Arc;
 use bdk::bitcoin;
 use bdk::database::MemoryDatabase;
 use bdk::descriptor::HdKeyPaths;
+#[allow(deprecated)]
 use bdk::wallet::address_validator::{AddressValidator, AddressValidatorError};
 use bdk::KeychainKind;
 use bdk::Wallet;
@@ -25,6 +26,7 @@ use bitcoin::{Network, Script};
 
 #[derive(Debug)]
 struct DummyValidator;
+#[allow(deprecated)]
 impl AddressValidator for DummyValidator {
     fn validate(
         &self,
@@ -50,6 +52,7 @@ fn main() -> Result<(), bdk::Error> {
     let descriptor = "sh(and_v(v:pk(tpubDDpWvmUrPZrhSPmUzCMBHffvC3HyMAPnWDSAQNBTnj1iZeJa7BZQEttFiP4DS4GCcXQHezdXhn86Hj6LHX5EDstXPWrMaSneRWM8yUf6NFd/*),after(630000)))";
     let mut wallet = Wallet::new(descriptor, None, Network::Regtest, MemoryDatabase::new())?;
 
+    #[allow(deprecated)]
     wallet.add_address_validator(Arc::new(DummyValidator));
 
     wallet.get_address(New)?;

examples/rpcwallet.rs

@@ -103,7 +103,7 @@ fn main() -> Result<(), Box<dyn Error>> {
         auth: bitcoind_auth,
         network: Network::Regtest,
         wallet_name,
-        skip_blocks: None,
+        sync_params: None,
     };
 
     // Use the above configuration to create a RPC blockchain backend

src/blockchain/any.rs

@@ -120,6 +120,13 @@ impl GetTx for AnyBlockchain {
     }
 }
 
+#[maybe_async]
+impl GetBlockHash for AnyBlockchain {
+    fn get_block_hash(&self, height: u64) -> Result<BlockHash, Error> {
+        maybe_await!(impl_inner_method!(self, get_block_hash, height))
+    }
+}
+
 #[maybe_async]
 impl WalletSync for AnyBlockchain {
     fn wallet_sync<D: BatchDatabase>(

src/blockchain/compact_filters/mod.rs

@@ -260,6 +260,16 @@ impl GetTx for CompactFiltersBlockchain {
     }
 }
 
+impl GetBlockHash for CompactFiltersBlockchain {
+    fn get_block_hash(&self, height: u64) -> Result<BlockHash, Error> {
+        self.headers
+            .get_block_hash(height as usize)?
+            .ok_or(Error::CompactFilters(
+                CompactFiltersError::BlockHashNotFound,
+            ))
+    }
+}
+
 impl WalletSync for CompactFiltersBlockchain {
     #[allow(clippy::mutex_atomic)] // Mutex is easier to understand than a CAS loop.
     fn wallet_setup<D: BatchDatabase>(
@@ -536,6 +546,8 @@ pub enum CompactFiltersError {
     InvalidFilter,
     /// The peer is missing a block in the valid chain
     MissingBlock,
+    /// Block hash at specified height not found
+    BlockHashNotFound,
     /// The data stored in the block filters storage are corrupted
     DataCorruption,
 

src/blockchain/electrum.rs

@@ -25,6 +25,7 @@
 //! ```
 
 use std::collections::{HashMap, HashSet};
+use std::ops::Deref;
 
 #[allow(unused_imports)]
 use log::{debug, error, info, trace};
@@ -79,6 +80,14 @@ impl Blockchain for ElectrumBlockchain {
     }
 }
 
+impl Deref for ElectrumBlockchain {
+    type Target = Client;
+
+    fn deref(&self) -> &Self::Target {
+        &self.client
+    }
+}
+
 impl StatelessBlockchain for ElectrumBlockchain {}
 
 impl GetHeight for ElectrumBlockchain {
@@ -98,6 +107,13 @@ impl GetTx for ElectrumBlockchain {
     }
 }
 
+impl GetBlockHash for ElectrumBlockchain {
+    fn get_block_hash(&self, height: u64) -> Result<BlockHash, Error> {
+        let block_header = self.client.block_header(height as usize)?;
+        Ok(block_header.block_hash())
+    }
+}
+
 impl WalletSync for ElectrumBlockchain {
     fn wallet_setup<D: BatchDatabase>(
         &self,
@@ -108,7 +124,10 @@ impl WalletSync for ElectrumBlockchain {
         let mut block_times = HashMap::<u32, u32>::new();
         let mut txid_to_height = HashMap::<Txid, u32>::new();
         let mut tx_cache = TxCache::new(database, &self.client);
-        let chunk_size = self.stop_gap;
+
+        // Set chunk_size to the smallest value capable of finding a gap greater than stop_gap.
+        let chunk_size = self.stop_gap + 1;
+
         // The electrum server has been inconsistent somehow in its responses during sync. For
         // example, we do a batch request of transactions and the response contains less
         // tranascations than in the request. This should never happen but we don't want to panic.
@@ -144,21 +163,12 @@ impl WalletSync for ElectrumBlockchain {
 
                 Request::Conftime(conftime_req) => {
                     // collect up to chunk_size heights to fetch from electrum
-                    let needs_block_height = {
-                        let mut needs_block_height_iter = conftime_req
-                            .request()
-                            .filter_map(|txid| txid_to_height.get(txid).cloned())
-                            .filter(|height| block_times.get(height).is_none());
-                        let mut needs_block_height = HashSet::new();
-
-                        while needs_block_height.len() < chunk_size {
-                            match needs_block_height_iter.next() {
-                                Some(height) => needs_block_height.insert(height),
-                                None => break,
-                            };
-                        }
-                        needs_block_height
-                    };
+                    let needs_block_height = conftime_req
+                        .request()
+                        .filter_map(|txid| txid_to_height.get(txid).cloned())
+                        .filter(|height| block_times.get(height).is_none())
+                        .take(chunk_size)
+                        .collect::<HashSet<u32>>();
 
                     let new_block_headers = self
                         .client
@@ -328,6 +338,7 @@ mod test {
     use super::*;
     use crate::database::MemoryDatabase;
     use crate::testutils::blockchain_tests::TestClient;
+    use crate::testutils::configurable_blockchain_tests::ConfigurableBlockchainTester;
     use crate::wallet::{AddressIndex, Wallet};
 
     crate::bdk_blockchain_tests! {
@@ -383,6 +394,31 @@ mod test {
             .sync_wallet(&wallet, None, Default::default())
             .unwrap();
 
-        assert_eq!(wallet.get_balance().unwrap(), 50_000);
+        assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 50_000);
+    }
+
+    #[test]
+    fn test_electrum_with_variable_configs() {
+        struct ElectrumTester;
+
+        impl ConfigurableBlockchainTester<ElectrumBlockchain> for ElectrumTester {
+            const BLOCKCHAIN_NAME: &'static str = "Electrum";
+
+            fn config_with_stop_gap(
+                &self,
+                test_client: &mut TestClient,
+                stop_gap: usize,
+            ) -> Option<ElectrumBlockchainConfig> {
+                Some(ElectrumBlockchainConfig {
+                    url: test_client.electrsd.electrum_url.clone(),
+                    socks5: None,
+                    retry: 0,
+                    timeout: None,
+                    stop_gap: stop_gap,
+                })
+            }
+        }
+
+        ElectrumTester.run();
     }
 }

src/blockchain/esplora/mod.rs

@@ -209,4 +209,38 @@ mod test {
             "should inherit from value for 25"
         );
     }
+
+    #[test]
+    #[cfg(feature = "test-esplora")]
+    fn test_esplora_with_variable_configs() {
+        use crate::testutils::{
+            blockchain_tests::TestClient,
+            configurable_blockchain_tests::ConfigurableBlockchainTester,
+        };
+
+        struct EsploraTester;
+
+        impl ConfigurableBlockchainTester<EsploraBlockchain> for EsploraTester {
+            const BLOCKCHAIN_NAME: &'static str = "Esplora";
+
+            fn config_with_stop_gap(
+                &self,
+                test_client: &mut TestClient,
+                stop_gap: usize,
+            ) -> Option<EsploraBlockchainConfig> {
+                Some(EsploraBlockchainConfig {
+                    base_url: format!(
+                        "http://{}",
+                        test_client.electrsd.esplora_url.as_ref().unwrap()
+                    ),
+                    proxy: None,
+                    concurrency: None,
+                    stop_gap: stop_gap,
+                    timeout: None,
+                })
+            }
+        }
+
+        EsploraTester.run();
+    }
 }

src/blockchain/esplora/reqwest.rs

@@ -12,6 +12,7 @@
 //! Esplora by way of `reqwest` HTTP client.
 
 use std::collections::{HashMap, HashSet};
+use std::ops::Deref;
 
 use bitcoin::consensus::{deserialize, serialize};
 use bitcoin::hashes::hex::{FromHex, ToHex};
@@ -31,8 +32,9 @@ use crate::database::BatchDatabase;
 use crate::error::Error;
 use crate::FeeRate;
 
+/// Structure encapsulates Esplora client
 #[derive(Debug)]
-struct UrlClient {
+pub struct UrlClient {
     url: String,
     // We use the async client instead of the blocking one because it automatically uses `fetch`
     // when the target platform is wasm32.
@@ -101,6 +103,14 @@ impl Blockchain for EsploraBlockchain {
     }
 }
 
+impl Deref for EsploraBlockchain {
+    type Target = UrlClient;
+
+    fn deref(&self) -> &Self::Target {
+        &self.url_client
+    }
+}
+
 impl StatelessBlockchain for EsploraBlockchain {}
 
 #[maybe_async]
@@ -117,6 +127,14 @@ impl GetTx for EsploraBlockchain {
     }
 }
 
+#[maybe_async]
+impl GetBlockHash for EsploraBlockchain {
+    fn get_block_hash(&self, height: u64) -> Result<BlockHash, Error> {
+        let block_header = await_or_block!(self.url_client._get_header(height as u32))?;
+        Ok(block_header.block_hash())
+    }
+}
+
 #[maybe_async]
 impl WalletSync for EsploraBlockchain {
     fn wallet_setup<D: BatchDatabase>(
@@ -205,7 +223,6 @@ impl WalletSync for EsploraBlockchain {
         };
 
         database.commit_batch(batch_update)?;
-
         Ok(())
     }
 }

src/blockchain/esplora/ureq.rs

@@ -14,6 +14,7 @@
 use std::collections::{HashMap, HashSet};
 use std::io;
 use std::io::Read;
+use std::ops::Deref;
 use std::time::Duration;
 
 #[allow(unused_imports)]
@@ -33,8 +34,9 @@ use crate::database::BatchDatabase;
 use crate::error::Error;
 use crate::FeeRate;
 
+/// Structure encapsulates ureq Esplora client
 #[derive(Debug, Clone)]
-struct UrlClient {
+pub struct UrlClient {
     url: String,
     agent: Agent,
 }
@@ -88,7 +90,7 @@ impl Blockchain for EsploraBlockchain {
     }
 
     fn broadcast(&self, tx: &Transaction) -> Result<(), Error> {
-        let _txid = self.url_client._broadcast(tx)?;
+        self.url_client._broadcast(tx)?;
         Ok(())
     }
 
@@ -98,6 +100,14 @@ impl Blockchain for EsploraBlockchain {
     }
 }
 
+impl Deref for EsploraBlockchain {
+    type Target = UrlClient;
+
+    fn deref(&self) -> &Self::Target {
+        &self.url_client
+    }
+}
+
 impl StatelessBlockchain for EsploraBlockchain {}
 
 impl GetHeight for EsploraBlockchain {
@@ -112,6 +122,13 @@ impl GetTx for EsploraBlockchain {
     }
 }
 
+impl GetBlockHash for EsploraBlockchain {
+    fn get_block_hash(&self, height: u64) -> Result<BlockHash, Error> {
+        let block_header = self.url_client._get_header(height as u32)?;
+        Ok(block_header.block_hash())
+    }
+}
+
 impl WalletSync for EsploraBlockchain {
     fn wallet_setup<D: BatchDatabase>(
         &self,

src/blockchain/mod.rs

@@ -21,7 +21,7 @@ use std::ops::Deref;
 use std::sync::mpsc::{channel, Receiver, Sender};
 use std::sync::Arc;
 
-use bitcoin::{Transaction, Txid};
+use bitcoin::{BlockHash, Transaction, Txid};
 
 use crate::database::BatchDatabase;
 use crate::error::Error;
@@ -87,7 +87,7 @@ pub enum Capability {
 
 /// Trait that defines the actions that must be supported by a blockchain backend
 #[maybe_async]
-pub trait Blockchain: WalletSync + GetHeight + GetTx {
+pub trait Blockchain: WalletSync + GetHeight + GetTx + GetBlockHash {
     /// Return the set of [`Capability`] supported by this backend
     fn get_capabilities(&self) -> HashSet<Capability>;
     /// Broadcast a transaction
@@ -110,6 +110,13 @@ pub trait GetTx {
     fn get_tx(&self, txid: &Txid) -> Result<Option<Transaction>, Error>;
 }
 
+#[maybe_async]
+/// Trait for getting block hash by block height
+pub trait GetBlockHash {
+    /// fetch block hash given its height
+    fn get_block_hash(&self, height: u64) -> Result<BlockHash, Error>;
+}
+
 /// Trait for blockchains that can sync by updating the database directly.
 #[maybe_async]
 pub trait WalletSync {
@@ -187,7 +194,7 @@ This example shows how to sync multiple walles and return the sum of their balan
 # use bdk::database::*;
 # use bdk::wallet::*;
 # use bdk::*;
-fn sum_of_balances<B: BlockchainFactory>(blockchain_factory: B, wallets: &[Wallet<MemoryDatabase>]) -> Result<u64, Error> {
+fn sum_of_balances<B: BlockchainFactory>(blockchain_factory: B, wallets: &[Wallet<MemoryDatabase>]) -> Result<Balance, Error> {
     Ok(wallets
         .iter()
         .map(|w| -> Result<_, Error> {
@@ -359,6 +366,13 @@ impl<T: GetHeight> GetHeight for Arc<T> {
     }
 }
 
+#[maybe_async]
+impl<T: GetBlockHash> GetBlockHash for Arc<T> {
+    fn get_block_hash(&self, height: u64) -> Result<BlockHash, Error> {
+        maybe_await!(self.deref().get_block_hash(height))
+    }
+}
+
 #[maybe_async]
 impl<T: WalletSync> WalletSync for Arc<T> {
     fn wallet_setup<D: BatchDatabase>(

src/blockchain/script_sync.rs

@@ -5,6 +5,7 @@ returns associated transactions i.e. electrum.
 #![allow(dead_code)]
 use crate::{
     database::{BatchDatabase, BatchOperations, DatabaseUtils},
+    error::MissingCachedScripts,
     wallet::time::Instant,
     BlockTime, Error, KeychainKind, LocalUtxo, TransactionDetails,
 };
@@ -34,11 +35,12 @@ pub fn start<D: BatchDatabase>(db: &D, stop_gap: usize) -> Result<Request<'_, D>
     let scripts_needed = db
         .iter_script_pubkeys(Some(keychain))?
         .into_iter()
-        .collect();
+        .collect::<VecDeque<_>>();
     let state = State::new(db);
 
     Ok(Request::Script(ScriptReq {
         state,
+        initial_scripts_needed: scripts_needed.len(),
         scripts_needed,
         script_index: 0,
         stop_gap,
@@ -50,6 +52,7 @@ pub fn start<D: BatchDatabase>(db: &D, stop_gap: usize) -> Result<Request<'_, D>
 pub struct ScriptReq<'a, D: BatchDatabase> {
     state: State<'a, D>,
     script_index: usize,
+    initial_scripts_needed: usize, // if this is 1, we assume the descriptor is not derivable
     scripts_needed: VecDeque<Script>,
     stop_gap: usize,
     keychain: KeychainKind,
@@ -113,43 +116,71 @@ impl<'a, D: BatchDatabase> ScriptReq<'a, D> {
             self.script_index += 1;
         }
 
-        for _ in txids {
-            self.scripts_needed.pop_front();
-        }
+        self.scripts_needed.drain(..txids.len());
 
-        let last_active_index = self
+        // last active index: 0 => No last active
+        let last = self
             .state
             .last_active_index
             .get(&self.keychain)
-            .map(|x| x + 1)
-            .unwrap_or(0); // so no addresses active maps to 0
+            .map(|&l| l + 1)
+            .unwrap_or(0);
+        // remaining scripts left to check
+        let remaining = self.scripts_needed.len();
+        // difference between current index and last active index
+        let current_gap = self.script_index - last;
 
-        Ok(
-            if self.script_index > last_active_index + self.stop_gap
-                || self.scripts_needed.is_empty()
-            {
-                debug!(
-                    "finished scanning for transactions for keychain {:?} at index {}",
-                    self.keychain, last_active_index
-                );
-                // we're done here -- check if we need to do the next keychain
-                if let Some(keychain) = self.next_keychains.pop() {
-                    self.keychain = keychain;
-                    self.script_index = 0;
-                    self.scripts_needed = self
-                        .state
-                        .db
-                        .iter_script_pubkeys(Some(keychain))?
-                        .into_iter()
-                        .collect();
-                    Request::Script(self)
-                } else {
-                    Request::Tx(TxReq { state: self.state })
-                }
-            } else {
-                Request::Script(self)
-            },
-        )
+        // this is a hack to check whether the scripts are coming from a derivable descriptor
+        // we assume for non-derivable descriptors, the initial script count is always 1
+        let is_derivable = self.initial_scripts_needed > 1;
+
+        debug!(
+            "sync: last={}, remaining={}, diff={}, stop_gap={}",
+            last, remaining, current_gap, self.stop_gap
+        );
+
+        if is_derivable {
+            if remaining > 0 {
+                // we still have scriptPubKeys to do requests for
+                return Ok(Request::Script(self));
+            }
+
+            if last > 0 && current_gap < self.stop_gap {
+                // current gap is not large enough to stop, but we are unable to keep checking since
+                // we have exhausted cached scriptPubKeys, so return error
+                let err = MissingCachedScripts {
+                    last_count: self.script_index,
+                    missing_count: self.stop_gap - current_gap,
+                };
+                return Err(Error::MissingCachedScripts(err));
+            }
+
+            // we have exhausted cached scriptPubKeys and found no txs, continue
+        }
+
+        debug!(
+            "finished scanning for txs of keychain {:?} at index {:?}",
+            self.keychain, last
+        );
+
+        if let Some(keychain) = self.next_keychains.pop() {
+            // we still have another keychain to request txs with
+            let scripts_needed = self
+                .state
+                .db
+                .iter_script_pubkeys(Some(keychain))?
+                .into_iter()
+                .collect::<VecDeque<_>>();
+
+            self.keychain = keychain;
+            self.script_index = 0;
+            self.initial_scripts_needed = scripts_needed.len();
+            self.scripts_needed = scripts_needed;
+            return Ok(Request::Script(self));
+        }
+
+        // We have finished requesting txids, let's get the actual txs.
+        Ok(Request::Tx(TxReq { state: self.state }))
     }
 }
 
@@ -294,6 +325,8 @@ struct State<'a, D> {
     tx_missing_conftime: BTreeMap<Txid, TransactionDetails>,
     /// The start of the sync
     start_time: Instant,
+    /// Missing number of scripts to cache per keychain
+    missing_script_counts: HashMap<KeychainKind, usize>,
 }
 
 impl<'a, D: BatchDatabase> State<'a, D> {
@@ -305,6 +338,7 @@ impl<'a, D: BatchDatabase> State<'a, D> {
             tx_needed: BTreeSet::default(),
             tx_missing_conftime: BTreeMap::default(),
             start_time: Instant::new(),
+            missing_script_counts: HashMap::default(),
         }
     }
     fn into_db_update(self) -> Result<D::Batch, Error> {
@@ -314,6 +348,22 @@ impl<'a, D: BatchDatabase> State<'a, D> {
         let finished_txs = make_txs_consistent(&self.finished_txs);
         let observed_txids: HashSet<Txid> = finished_txs.iter().map(|tx| tx.txid).collect();
         let txids_to_delete = existing_txids.difference(&observed_txids);
+
+        // Ensure `last_active_index` does not decrement database's current state.
+        let index_updates = self
+            .last_active_index
+            .iter()
+            .map(|(keychain, sync_index)| {
+                let sync_index = *sync_index as u32;
+                let index_res = match self.db.get_last_index(*keychain) {
+                    Ok(Some(db_index)) => Ok(std::cmp::max(db_index, sync_index)),
+                    Ok(None) => Ok(sync_index),
+                    Err(err) => Err(err),
+                };
+                index_res.map(|index| (*keychain, index))
+            })
+            .collect::<Result<Vec<(KeychainKind, u32)>, _>>()?;
+
         let mut batch = self.db.begin_batch();
 
         // Delete old txs that no longer exist
@@ -377,8 +427,10 @@ impl<'a, D: BatchDatabase> State<'a, D> {
             batch.set_tx(finished_tx)?;
         }
 
-        for (keychain, last_active_index) in self.last_active_index {
-            batch.set_last_index(keychain, last_active_index as u32)?;
+        // apply index updates
+        for (keychain, new_index) in index_updates {
+            debug!("updating index ({}, {})", keychain.as_byte(), new_index);
+            batch.set_last_index(keychain, new_index)?;
         }
 
         info!(

src/database/any.rs

@@ -255,10 +255,6 @@ impl Database for AnyDatabase {
     fn increment_last_index(&mut self, keychain: KeychainKind) -> Result<u32, Error> {
         impl_inner_method!(AnyDatabase, self, increment_last_index, keychain)
     }
-
-    fn flush(&mut self) -> Result<(), Error> {
-        impl_inner_method!(AnyDatabase, self, flush)
-    }
 }
 
 impl BatchOperations for AnyBatch {

src/database/keyvalue.rs

@@ -166,16 +166,9 @@ macro_rules! impl_batch_operations {
         fn del_last_index(&mut self, keychain: KeychainKind) -> Result<Option<u32>, Error> {
             let key = MapKey::LastIndex(keychain).as_map_key();
             let res = self.remove(key);
-            let res = $process_delete!(res);
-
-            match res {
-                None => Ok(None),
-                Some(b) => {
-                    let array: [u8; 4] = b.as_ref().try_into().map_err(|_| Error::InvalidU32Bytes(b.to_vec()))?;
-                    let val = u32::from_be_bytes(array);
-                    Ok(Some(val))
-                }
-            }
+            $process_delete!(res)
+            .map(ivec_to_u32)
+            .transpose()
         }
 
         fn del_sync_time(&mut self) -> Result<Option<SyncTime>, Error> {
@@ -357,16 +350,7 @@ impl Database for Tree {
 
     fn get_last_index(&self, keychain: KeychainKind) -> Result<Option<u32>, Error> {
         let key = MapKey::LastIndex(keychain).as_map_key();
-        self.get(key)?
-            .map(|b| -> Result<_, Error> {
-                let array: [u8; 4] = b
-                    .as_ref()
-                    .try_into()
-                    .map_err(|_| Error::InvalidU32Bytes(b.to_vec()))?;
-                let val = u32::from_be_bytes(array);
-                Ok(val)
-            })
-            .transpose()
+        self.get(key)?.map(ivec_to_u32).transpose()
     }
 
     fn get_sync_time(&self) -> Result<Option<SyncTime>, Error> {
@@ -393,19 +377,17 @@ impl Database for Tree {
 
             Some(new.to_be_bytes().to_vec())
         })?
-        .map_or(Ok(0), |b| -> Result<_, Error> {
-            let array: [u8; 4] = b
-                .as_ref()
-                .try_into()
-                .map_err(|_| Error::InvalidU32Bytes(b.to_vec()))?;
-            let val = u32::from_be_bytes(array);
-            Ok(val)
-        })
+        .map_or(Ok(0), ivec_to_u32)
     }
+}
 
-    fn flush(&mut self) -> Result<(), Error> {
-        Ok(Tree::flush(self).map(|_| ())?)
-    }
+fn ivec_to_u32(b: sled::IVec) -> Result<u32, Error> {
+    let array: [u8; 4] = b
+        .as_ref()
+        .try_into()
+        .map_err(|_| Error::InvalidU32Bytes(b.to_vec()))?;
+    let val = u32::from_be_bytes(array);
+    Ok(val)
 }
 
 impl BatchDatabase for Tree {

src/database/memory.rs

@@ -449,10 +449,6 @@ impl Database for MemoryDatabase {
 
         Ok(*value)
     }
-
-    fn flush(&mut self) -> Result<(), Error> {
-        Ok(())
-    }
 }
 
 impl BatchDatabase for MemoryDatabase {
@@ -486,15 +482,23 @@ impl ConfigurableDatabase for MemoryDatabase {
 /// don't have `test` set.
 macro_rules! populate_test_db {
     ($db:expr, $tx_meta:expr, $current_height:expr$(,)?) => {{
+        $crate::populate_test_db!($db, $tx_meta, $current_height, (@coinbase false))
+    }};
+    ($db:expr, $tx_meta:expr, $current_height:expr, (@coinbase $is_coinbase:expr)$(,)?) => {{
         use std::str::FromStr;
-        use $crate::database::BatchOperations;
+        use $crate::database::SyncTime;
+        use $crate::database::{BatchOperations, Database};
         let mut db = $db;
         let tx_meta = $tx_meta;
         let current_height: Option<u32> = $current_height;
+        let mut input = vec![$crate::bitcoin::TxIn::default()];
+        if !$is_coinbase {
+            input[0].previous_output.vout = 0;
+        }
         let tx = $crate::bitcoin::Transaction {
             version: 1,
             lock_time: 0,
-            input: vec![],
+            input,
             output: tx_meta
                 .output
                 .iter()
@@ -508,10 +512,31 @@ macro_rules! populate_test_db {
         };
 
         let txid = tx.txid();
-        let confirmation_time = tx_meta.min_confirmations.map(|conf| $crate::BlockTime {
-            height: current_height.unwrap().checked_sub(conf as u32).unwrap(),
-            timestamp: 0,
-        });
+        // Set Confirmation time only if current height is provided.
+        // panics if `tx_meta.min_confirmation` is Some, and current_height is None.
+        let confirmation_time = tx_meta
+            .min_confirmations
+            .and_then(|v| if v == 0 { None } else { Some(v) })
+            .map(|conf| $crate::BlockTime {
+                height: current_height.expect("Current height is needed for testing transaction with min-confirmation values").checked_sub(conf as u32).unwrap() + 1,
+                timestamp: 0,
+            });
+
+        // Set the database sync_time.
+        // Check if the current_height is less than already known sync height, apply the max
+        // If any of them is None, the other will be applied instead.
+        // If both are None, this will not be set.
+        if let Some(height) = db.get_sync_time().unwrap()
+                                .map(|sync_time| sync_time.block_time.height)
+                                .max(current_height) {
+            let sync_time = SyncTime {
+                block_time: BlockTime {
+                    height,
+                    timestamp: 0
+                }
+            };
+            db.set_sync_time(sync_time).unwrap();
+        }
 
         let tx_details = $crate::TransactionDetails {
             transaction: Some(tx.clone()),

src/database/mod.rs

@@ -158,13 +158,6 @@ pub trait Database: BatchOperations {
     ///
     /// It should insert and return `0` if not present in the database
     fn increment_last_index(&mut self, keychain: KeychainKind) -> Result<u32, Error>;
-
-    #[deprecated(
-        since = "0.18.0",
-        note = "The flush function is only needed for the sled database on mobile, instead for mobile use the sqlite database."
-    )]
-    /// Force changes to be written to disk
-    fn flush(&mut self) -> Result<(), Error>;
 }
 
 /// Trait for a database that supports batch operations

src/database/sqlite.rs

@@ -8,6 +8,8 @@
 // <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option.
 // You may not use this file except in accordance with one or both of these
 // licenses.
+use std::path::Path;
+use std::path::PathBuf;
 
 use bitcoin::consensus::encode::{deserialize, serialize};
 use bitcoin::hash_types::Txid;
@@ -60,7 +62,7 @@ static MIGRATIONS: &[&str] = &[
 #[derive(Debug)]
 pub struct SqliteDatabase {
     /// Path on the local filesystem to store the sqlite file
-    pub path: String,
+    pub path: PathBuf,
     /// A rusqlite connection object to the sqlite database
     pub connection: Connection,
 }
@@ -68,9 +70,12 @@ pub struct SqliteDatabase {
 impl SqliteDatabase {
     /// Instantiate a new SqliteDatabase instance by creating a connection
     /// to the database stored at path
-    pub fn new(path: String) -> Self {
+    pub fn new<T: AsRef<Path>>(path: T) -> Self {
         let connection = get_connection(&path).unwrap();
-        SqliteDatabase { path, connection }
+        SqliteDatabase {
+            path: PathBuf::from(path.as_ref()),
+            connection,
+        }
     }
     fn insert_script_pubkey(
         &self,
@@ -891,10 +896,6 @@ impl Database for SqliteDatabase {
             }
         }
     }
-
-    fn flush(&mut self) -> Result<(), Error> {
-        Ok(())
-    }
 }
 
 impl BatchDatabase for SqliteDatabase {
@@ -912,7 +913,7 @@ impl BatchDatabase for SqliteDatabase {
     }
 }
 
-pub fn get_connection(path: &str) -> Result<Connection, Error> {
+pub fn get_connection<T: AsRef<Path>>(path: &T) -> Result<Connection, Error> {
     let connection = Connection::open(path)?;
     migrate(&connection)?;
     Ok(connection)

src/descriptor/checksum.rs

@@ -14,12 +14,10 @@
 //! This module contains a re-implementation of the function used by Bitcoin Core to calculate the
 //! checksum of a descriptor
 
-use std::iter::FromIterator;
-
 use crate::descriptor::DescriptorError;
 
-const INPUT_CHARSET: &str =  "0123456789()[],'/*abcdefgh@:$%{}IJKLMNOPQRSTUVWXYZ&+-.;<=>?!^_|~ijklmnopqrstuvwxyzABCDEFGH`#\"\\ ";
-const CHECKSUM_CHARSET: &str = "qpzry9x8gf2tvdw0s3jn54khce6mua7l";
+const INPUT_CHARSET: &[u8] = b"0123456789()[],'/*abcdefgh@:$%{}IJKLMNOPQRSTUVWXYZ&+-.;<=>?!^_|~ijklmnopqrstuvwxyzABCDEFGH`#\"\\ ";
+const CHECKSUM_CHARSET: &[u8] = b"qpzry9x8gf2tvdw0s3jn54khce6mua7l";
 
 fn poly_mod(mut c: u64, val: u64) -> u64 {
     let c0 = c >> 35;
@@ -43,15 +41,17 @@ fn poly_mod(mut c: u64, val: u64) -> u64 {
     c
 }
 
-/// Compute the checksum of a descriptor
-pub fn get_checksum(desc: &str) -> Result<String, DescriptorError> {
+/// Computes the checksum bytes of a descriptor
+pub fn get_checksum_bytes(desc: &str) -> Result<[u8; 8], DescriptorError> {
     let mut c = 1;
     let mut cls = 0;
     let mut clscount = 0;
-    for ch in desc.chars() {
+
+    for ch in desc.as_bytes() {
         let pos = INPUT_CHARSET
-            .find(ch)
-            .ok_or(DescriptorError::InvalidDescriptorCharacter(ch))? as u64;
+            .iter()
+            .position(|b| b == ch)
+            .ok_or(DescriptorError::InvalidDescriptorCharacter(*ch))? as u64;
         c = poly_mod(c, pos & 31);
         cls = cls * 3 + (pos >> 5);
         clscount += 1;
@@ -67,17 +67,18 @@ pub fn get_checksum(desc: &str) -> Result<String, DescriptorError> {
     (0..8).for_each(|_| c = poly_mod(c, 0));
     c ^= 1;
 
-    let mut chars = Vec::with_capacity(8);
+    let mut checksum = [0_u8; 8];
     for j in 0..8 {
-        chars.push(
-            CHECKSUM_CHARSET
-                .chars()
-                .nth(((c >> (5 * (7 - j))) & 31) as usize)
-                .unwrap(),
-        );
+        checksum[j] = CHECKSUM_CHARSET[((c >> (5 * (7 - j))) & 31) as usize];
     }
 
-    Ok(String::from_iter(chars))
+    Ok(checksum)
+}
+
+/// Compute the checksum of a descriptor
+pub fn get_checksum(desc: &str) -> Result<String, DescriptorError> {
+    // unsafe is okay here as the checksum only uses bytes in `CHECKSUM_CHARSET`
+    get_checksum_bytes(desc).map(|b| unsafe { String::from_utf8_unchecked(b.to_vec()) })
 }
 
 #[cfg(test)]
@@ -97,17 +98,12 @@ mod test {
 
     #[test]
     fn test_get_checksum_invalid_character() {
-        let sparkle_heart = vec![240, 159, 146, 150];
-        let sparkle_heart = std::str::from_utf8(&sparkle_heart)
-            .unwrap()
-            .chars()
-            .next()
-            .unwrap();
+        let sparkle_heart = unsafe { std::str::from_utf8_unchecked(&[240, 159, 146, 150]) };
         let invalid_desc = format!("wpkh(tprv8ZgxMBicQKsPdpkqS7Eair4YxjcuuvDPNYmKX3sCniCf16tHEVrjjiSXEkFRnUH77yXc6ZcwHHcL{}fjdi5qUvw3VDfgYiH5mNsj5izuiu2N/1/2/*)", sparkle_heart);
 
         assert!(matches!(
             get_checksum(&invalid_desc).err(),
-            Some(DescriptorError::InvalidDescriptorCharacter(invalid_char)) if invalid_char == sparkle_heart
+            Some(DescriptorError::InvalidDescriptorCharacter(invalid_char)) if invalid_char == sparkle_heart.as_bytes()[0]
         ));
     }
 }

src/descriptor/dsl.rs

@@ -839,7 +839,7 @@ mod test {
         }
     }
 
-    // - at least one of each "type" of operator; ie. one modifier, one leaf_opcode, one leaf_opcode_value, etc.
+    // - at least one of each "type" of operator; i.e. one modifier, one leaf_opcode, one leaf_opcode_value, etc.
     // - mixing up key types that implement IntoDescriptorKey in multi() or thresh()
 
     // expected script for pk and bare manually created

src/descriptor/error.rs

@@ -28,8 +28,8 @@ pub enum Error {
     /// Error while extracting and manipulating policies
     Policy(crate::descriptor::policy::PolicyError),
 
-    /// Invalid character found in the descriptor checksum
-    InvalidDescriptorCharacter(char),
+    /// Invalid byte found in the descriptor checksum
+    InvalidDescriptorCharacter(u8),
 
     /// BIP32 error
     Bip32(bitcoin::util::bip32::Error),

src/descriptor/mod.rs

@@ -134,13 +134,13 @@ impl IntoWalletDescriptor for (ExtendedDescriptor, KeyMap) {
 
         let check_key = |pk: &DescriptorPublicKey| {
             let (pk, _, networks) = if self.0.is_witness() {
-                let desciptor_key: DescriptorKey<miniscript::Segwitv0> =
+                let descriptor_key: DescriptorKey<miniscript::Segwitv0> =
                     pk.clone().into_descriptor_key()?;
-                desciptor_key.extract(secp)?
+                descriptor_key.extract(secp)?
             } else {
-                let desciptor_key: DescriptorKey<miniscript::Legacy> =
+                let descriptor_key: DescriptorKey<miniscript::Legacy> =
                     pk.clone().into_descriptor_key()?;
-                desciptor_key.extract(secp)?
+                descriptor_key.extract(secp)?
             };
 
             if networks.contains(&network) {

src/descriptor/template.rs

@@ -40,18 +40,19 @@ pub type DescriptorTemplateOut = (ExtendedDescriptor, KeyMap, ValidNetworks);
 /// use bdk::keys::{IntoDescriptorKey, KeyError};
 /// use bdk::miniscript::Legacy;
 /// use bdk::template::{DescriptorTemplate, DescriptorTemplateOut};
+/// use bitcoin::Network;
 ///
 /// struct MyP2PKH<K: IntoDescriptorKey<Legacy>>(K);
 ///
 /// impl<K: IntoDescriptorKey<Legacy>> DescriptorTemplate for MyP2PKH<K> {
-///     fn build(self) -> Result<DescriptorTemplateOut, DescriptorError> {
+///     fn build(self, network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
 ///         Ok(bdk::descriptor!(pkh(self.0))?)
 ///     }
 /// }
 /// ```
 pub trait DescriptorTemplate {
     /// Build the complete descriptor
-    fn build(self) -> Result<DescriptorTemplateOut, DescriptorError>;
+    fn build(self, network: Network) -> Result<DescriptorTemplateOut, DescriptorError>;
 }
 
 /// Turns a [`DescriptorTemplate`] into a valid wallet descriptor by calling its
@@ -62,7 +63,7 @@ impl<T: DescriptorTemplate> IntoWalletDescriptor for T {
         secp: &SecpCtx,
         network: Network,
     ) -> Result<(ExtendedDescriptor, KeyMap), DescriptorError> {
-        self.build()?.into_wallet_descriptor(secp, network)
+        self.build(network)?.into_wallet_descriptor(secp, network)
     }
 }
 
@@ -95,7 +96,7 @@ impl<T: DescriptorTemplate> IntoWalletDescriptor for T {
 pub struct P2Pkh<K: IntoDescriptorKey<Legacy>>(pub K);
 
 impl<K: IntoDescriptorKey<Legacy>> DescriptorTemplate for P2Pkh<K> {
-    fn build(self) -> Result<DescriptorTemplateOut, DescriptorError> {
+    fn build(self, _network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
         descriptor!(pkh(self.0))
     }
 }
@@ -130,7 +131,7 @@ impl<K: IntoDescriptorKey<Legacy>> DescriptorTemplate for P2Pkh<K> {
 pub struct P2Wpkh_P2Sh<K: IntoDescriptorKey<Segwitv0>>(pub K);
 
 impl<K: IntoDescriptorKey<Segwitv0>> DescriptorTemplate for P2Wpkh_P2Sh<K> {
-    fn build(self) -> Result<DescriptorTemplateOut, DescriptorError> {
+    fn build(self, _network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
         descriptor!(sh(wpkh(self.0)))
     }
 }
@@ -164,12 +165,12 @@ impl<K: IntoDescriptorKey<Segwitv0>> DescriptorTemplate for P2Wpkh_P2Sh<K> {
 pub struct P2Wpkh<K: IntoDescriptorKey<Segwitv0>>(pub K);
 
 impl<K: IntoDescriptorKey<Segwitv0>> DescriptorTemplate for P2Wpkh<K> {
-    fn build(self) -> Result<DescriptorTemplateOut, DescriptorError> {
+    fn build(self, _network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
         descriptor!(wpkh(self.0))
     }
 }
 
-/// BIP44 template. Expands to `pkh(key/44'/0'/0'/{0,1}/*)`
+/// BIP44 template. Expands to `pkh(key/44'/{0,1}'/0'/{0,1}/*)`
 ///
 /// Since there are hardened derivation steps, this template requires a private derivable key (generally a `xprv`/`tprv`).
 ///
@@ -193,21 +194,21 @@ impl<K: IntoDescriptorKey<Segwitv0>> DescriptorTemplate for P2Wpkh<K> {
 ///     MemoryDatabase::default()
 /// )?;
 ///
-/// assert_eq!(wallet.get_address(New)?.to_string(), "miNG7dJTzJqNbFS19svRdTCisC65dsubtR");
-/// assert_eq!(wallet.public_descriptor(KeychainKind::External)?.unwrap().to_string(), "pkh([c55b303f/44'/0'/0']tpubDDDzQ31JkZB7VxUr9bjvBivDdqoFLrDPyLWtLapArAi51ftfmCb2DPxwLQzX65iNcXz1DGaVvyvo6JQ6rTU73r2gqdEo8uov9QKRb7nKCSU/0/*)#xgaaevjx");
+/// assert_eq!(wallet.get_address(New)?.to_string(), "mmogjc7HJEZkrLqyQYqJmxUqFaC7i4uf89");
+/// assert_eq!(wallet.public_descriptor(KeychainKind::External)?.unwrap().to_string(), "pkh([c55b303f/44'/1'/0']tpubDCuorCpzvYS2LCD75BR46KHE8GdDeg1wsAgNZeNr6DaB5gQK1o14uErKwKLuFmeemkQ6N2m3rNgvctdJLyr7nwu2yia7413Hhg8WWE44cgT/0/*)#5wrnv0xt");
 /// # Ok::<_, Box<dyn std::error::Error>>(())
 /// ```
 pub struct Bip44<K: DerivableKey<Legacy>>(pub K, pub KeychainKind);
 
 impl<K: DerivableKey<Legacy>> DescriptorTemplate for Bip44<K> {
-    fn build(self) -> Result<DescriptorTemplateOut, DescriptorError> {
-        P2Pkh(legacy::make_bipxx_private(44, self.0, self.1)?).build()
+    fn build(self, network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+        P2Pkh(legacy::make_bipxx_private(44, self.0, self.1, network)?).build(network)
     }
 }
 
 /// BIP44 public template. Expands to `pkh(key/{0,1}/*)`
 ///
-/// This assumes that the key used has already been derived with `m/44'/0'/0'`.
+/// This assumes that the key used has already been derived with `m/44'/0'/0'` for Mainnet or `m/44'/1'/0'` for Testnet.
 ///
 /// This template requires the parent fingerprint to populate correctly the metadata of PSBTs.
 ///
@@ -240,12 +241,12 @@ impl<K: DerivableKey<Legacy>> DescriptorTemplate for Bip44<K> {
 pub struct Bip44Public<K: DerivableKey<Legacy>>(pub K, pub bip32::Fingerprint, pub KeychainKind);
 
 impl<K: DerivableKey<Legacy>> DescriptorTemplate for Bip44Public<K> {
-    fn build(self) -> Result<DescriptorTemplateOut, DescriptorError> {
-        P2Pkh(legacy::make_bipxx_public(44, self.0, self.1, self.2)?).build()
+    fn build(self, network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+        P2Pkh(legacy::make_bipxx_public(44, self.0, self.1, self.2)?).build(network)
     }
 }
 
-/// BIP49 template. Expands to `sh(wpkh(key/49'/0'/0'/{0,1}/*))`
+/// BIP49 template. Expands to `sh(wpkh(key/49'/{0,1}'/0'/{0,1}/*))`
 ///
 /// Since there are hardened derivation steps, this template requires a private derivable key (generally a `xprv`/`tprv`).
 ///
@@ -269,15 +270,15 @@ impl<K: DerivableKey<Legacy>> DescriptorTemplate for Bip44Public<K> {
 ///     MemoryDatabase::default()
 /// )?;
 ///
-/// assert_eq!(wallet.get_address(New)?.to_string(), "2N3K4xbVAHoiTQSwxkZjWDfKoNC27pLkYnt");
-/// assert_eq!(wallet.public_descriptor(KeychainKind::External)?.unwrap().to_string(), "sh(wpkh([c55b303f/49\'/0\'/0\']tpubDC49r947KGK52X5rBWS4BLs5m9SRY3pYHnvRrm7HcybZ3BfdEsGFyzCMzayi1u58eT82ZeyFZwH7DD6Q83E3fM9CpfMtmnTygnLfP59jL9L/0/*))#gsmdv4xr");
+/// assert_eq!(wallet.get_address(New)?.to_string(), "2N4zkWAoGdUv4NXhSsU8DvS5MB36T8nKHEB");
+/// assert_eq!(wallet.public_descriptor(KeychainKind::External)?.unwrap().to_string(), "sh(wpkh([c55b303f/49'/1'/0']tpubDDYr4kdnZgjjShzYNjZUZXUUtpXaofdkMaipyS8ThEh45qFmhT4hKYways7UXmg6V7het1QiFo9kf4kYUXyDvV4rHEyvSpys9pjCB3pukxi/0/*))#s9vxlc8e");
 /// # Ok::<_, Box<dyn std::error::Error>>(())
 /// ```
 pub struct Bip49<K: DerivableKey<Segwitv0>>(pub K, pub KeychainKind);
 
 impl<K: DerivableKey<Segwitv0>> DescriptorTemplate for Bip49<K> {
-    fn build(self) -> Result<DescriptorTemplateOut, DescriptorError> {
-        P2Wpkh_P2Sh(segwit_v0::make_bipxx_private(49, self.0, self.1)?).build()
+    fn build(self, network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+        P2Wpkh_P2Sh(segwit_v0::make_bipxx_private(49, self.0, self.1, network)?).build(network)
     }
 }
 
@@ -310,18 +311,18 @@ impl<K: DerivableKey<Segwitv0>> DescriptorTemplate for Bip49<K> {
 /// )?;
 ///
 /// assert_eq!(wallet.get_address(New)?.to_string(), "2N3K4xbVAHoiTQSwxkZjWDfKoNC27pLkYnt");
-/// assert_eq!(wallet.public_descriptor(KeychainKind::External)?.unwrap().to_string(), "sh(wpkh([c55b303f/49\'/0\'/0\']tpubDC49r947KGK52X5rBWS4BLs5m9SRY3pYHnvRrm7HcybZ3BfdEsGFyzCMzayi1u58eT82ZeyFZwH7DD6Q83E3fM9CpfMtmnTygnLfP59jL9L/0/*))#gsmdv4xr");
+/// assert_eq!(wallet.public_descriptor(KeychainKind::External)?.unwrap().to_string(), "sh(wpkh([c55b303f/49'/0'/0']tpubDC49r947KGK52X5rBWS4BLs5m9SRY3pYHnvRrm7HcybZ3BfdEsGFyzCMzayi1u58eT82ZeyFZwH7DD6Q83E3fM9CpfMtmnTygnLfP59jL9L/0/*))#gsmdv4xr");
 /// # Ok::<_, Box<dyn std::error::Error>>(())
 /// ```
 pub struct Bip49Public<K: DerivableKey<Segwitv0>>(pub K, pub bip32::Fingerprint, pub KeychainKind);
 
 impl<K: DerivableKey<Segwitv0>> DescriptorTemplate for Bip49Public<K> {
-    fn build(self) -> Result<DescriptorTemplateOut, DescriptorError> {
-        P2Wpkh_P2Sh(segwit_v0::make_bipxx_public(49, self.0, self.1, self.2)?).build()
+    fn build(self, network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+        P2Wpkh_P2Sh(segwit_v0::make_bipxx_public(49, self.0, self.1, self.2)?).build(network)
     }
 }
 
-/// BIP84 template. Expands to `wpkh(key/84'/0'/0'/{0,1}/*)`
+/// BIP84 template. Expands to `wpkh(key/84'/{0,1}'/0'/{0,1}/*)`
 ///
 /// Since there are hardened derivation steps, this template requires a private derivable key (generally a `xprv`/`tprv`).
 ///
@@ -345,15 +346,15 @@ impl<K: DerivableKey<Segwitv0>> DescriptorTemplate for Bip49Public<K> {
 ///     MemoryDatabase::default()
 /// )?;
 ///
-/// assert_eq!(wallet.get_address(New)?.to_string(), "tb1qedg9fdlf8cnnqfd5mks6uz5w4kgpk2pr6y4qc7");
-/// assert_eq!(wallet.public_descriptor(KeychainKind::External)?.unwrap().to_string(), "wpkh([c55b303f/84\'/0\'/0\']tpubDC2Qwo2TFsaNC4ju8nrUJ9mqVT3eSgdmy1yPqhgkjwmke3PRXutNGRYAUo6RCHTcVQaDR3ohNU9we59brGHuEKPvH1ags2nevW5opEE9Z5Q/0/*)#nkk5dtkg");
+/// assert_eq!(wallet.get_address(New)?.to_string(), "tb1qhl85z42h7r4su5u37rvvw0gk8j2t3n9y7zsg4n");
+/// assert_eq!(wallet.public_descriptor(KeychainKind::External)?.unwrap().to_string(), "wpkh([c55b303f/84'/1'/0']tpubDDc5mum24DekpNw92t6fHGp8Gr2JjF9J7i4TZBtN6Vp8xpAULG5CFaKsfugWa5imhrQQUZKXe261asP5koDHo5bs3qNTmf3U3o4v9SaB8gg/0/*)#6kfecsmr");
 /// # Ok::<_, Box<dyn std::error::Error>>(())
 /// ```
 pub struct Bip84<K: DerivableKey<Segwitv0>>(pub K, pub KeychainKind);
 
 impl<K: DerivableKey<Segwitv0>> DescriptorTemplate for Bip84<K> {
-    fn build(self) -> Result<DescriptorTemplateOut, DescriptorError> {
-        P2Wpkh(segwit_v0::make_bipxx_private(84, self.0, self.1)?).build()
+    fn build(self, network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+        P2Wpkh(segwit_v0::make_bipxx_private(84, self.0, self.1, network)?).build(network)
     }
 }
 
@@ -392,8 +393,8 @@ impl<K: DerivableKey<Segwitv0>> DescriptorTemplate for Bip84<K> {
 pub struct Bip84Public<K: DerivableKey<Segwitv0>>(pub K, pub bip32::Fingerprint, pub KeychainKind);
 
 impl<K: DerivableKey<Segwitv0>> DescriptorTemplate for Bip84Public<K> {
-    fn build(self) -> Result<DescriptorTemplateOut, DescriptorError> {
-        P2Wpkh(segwit_v0::make_bipxx_public(84, self.0, self.1, self.2)?).build()
+    fn build(self, network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+        P2Wpkh(segwit_v0::make_bipxx_public(84, self.0, self.1, self.2)?).build(network)
     }
 }
 
@@ -406,10 +407,19 @@ macro_rules! expand_make_bipxx {
                 bip: u32,
                 key: K,
                 keychain: KeychainKind,
+                network: Network,
             ) -> Result<impl IntoDescriptorKey<$ctx>, DescriptorError> {
                 let mut derivation_path = Vec::with_capacity(4);
                 derivation_path.push(bip32::ChildNumber::from_hardened_idx(bip)?);
-                derivation_path.push(bip32::ChildNumber::from_hardened_idx(0)?);
+
+                match network {
+                    Network::Bitcoin => {
+                        derivation_path.push(bip32::ChildNumber::from_hardened_idx(0)?);
+                    }
+                    _ => {
+                        derivation_path.push(bip32::ChildNumber::from_hardened_idx(1)?);
+                    }
+                }
                 derivation_path.push(bip32::ChildNumber::from_hardened_idx(0)?);
 
                 match keychain {
@@ -466,6 +476,40 @@ mod test {
     use miniscript::descriptor::{DescriptorPublicKey, DescriptorTrait, KeyMap};
     use miniscript::Descriptor;
 
+    // BIP44 `pkh(key/44'/{0,1}'/0'/{0,1}/*)`
+    #[test]
+    fn test_bip44_template_cointype() {
+        use bitcoin::util::bip32::ChildNumber::{self, Hardened};
+
+        let xprvkey = bitcoin::util::bip32::ExtendedPrivKey::from_str("xprv9s21ZrQH143K2fpbqApQL69a4oKdGVnVN52R82Ft7d1pSqgKmajF62acJo3aMszZb6qQ22QsVECSFxvf9uyxFUvFYQMq3QbtwtRSMjLAhMf").unwrap();
+        assert_eq!(Network::Bitcoin, xprvkey.network);
+        let xdesc = Bip44(xprvkey, KeychainKind::Internal)
+            .build(Network::Bitcoin)
+            .unwrap();
+
+        if let ExtendedDescriptor::Pkh(pkh) = xdesc.0 {
+            let path: Vec<ChildNumber> = pkh.into_inner().full_derivation_path().into();
+            let purpose = path.get(0).unwrap();
+            assert!(matches!(purpose, Hardened { index: 44 }));
+            let coin_type = path.get(1).unwrap();
+            assert!(matches!(coin_type, Hardened { index: 0 }));
+        }
+
+        let tprvkey = bitcoin::util::bip32::ExtendedPrivKey::from_str("tprv8ZgxMBicQKsPcx5nBGsR63Pe8KnRUqmbJNENAfGftF3yuXoMMoVJJcYeUw5eVkm9WBPjWYt6HMWYJNesB5HaNVBaFc1M6dRjWSYnmewUMYy").unwrap();
+        assert_eq!(Network::Testnet, tprvkey.network);
+        let tdesc = Bip44(tprvkey, KeychainKind::Internal)
+            .build(Network::Testnet)
+            .unwrap();
+
+        if let ExtendedDescriptor::Pkh(pkh) = tdesc.0 {
+            let path: Vec<ChildNumber> = pkh.into_inner().full_derivation_path().into();
+            let purpose = path.get(0).unwrap();
+            assert!(matches!(purpose, Hardened { index: 44 }));
+            let coin_type = path.get(1).unwrap();
+            assert!(matches!(coin_type, Hardened { index: 1 }));
+        }
+    }
+
     // verify template descriptor generates expected address(es)
     fn check(
         desc: Result<(Descriptor<DescriptorPublicKey>, KeyMap, ValidNetworks), DescriptorError>,
@@ -497,7 +541,7 @@ mod test {
             bitcoin::PrivateKey::from_wif("cTc4vURSzdx6QE6KVynWGomDbLaA75dNALMNyfjh3p8DRRar84Um")
                 .unwrap();
         check(
-            P2Pkh(prvkey).build(),
+            P2Pkh(prvkey).build(Network::Bitcoin),
             false,
             true,
             &["mwJ8hxFYW19JLuc65RCTaP4v1rzVU8cVMT"],
@@ -508,7 +552,7 @@ mod test {
         )
         .unwrap();
         check(
-            P2Pkh(pubkey).build(),
+            P2Pkh(pubkey).build(Network::Bitcoin),
             false,
             true,
             &["muZpTpBYhxmRFuCjLc7C6BBDF32C8XVJUi"],
@@ -522,7 +566,7 @@ mod test {
             bitcoin::PrivateKey::from_wif("cTc4vURSzdx6QE6KVynWGomDbLaA75dNALMNyfjh3p8DRRar84Um")
                 .unwrap();
         check(
-            P2Wpkh_P2Sh(prvkey).build(),
+            P2Wpkh_P2Sh(prvkey).build(Network::Bitcoin),
             true,
             true,
             &["2NB4ox5VDRw1ecUv6SnT3VQHPXveYztRqk5"],
@@ -533,7 +577,7 @@ mod test {
         )
         .unwrap();
         check(
-            P2Wpkh_P2Sh(pubkey).build(),
+            P2Wpkh_P2Sh(pubkey).build(Network::Bitcoin),
             true,
             true,
             &["2N5LiC3CqzxDamRTPG1kiNv1FpNJQ7x28sb"],
@@ -547,7 +591,7 @@ mod test {
             bitcoin::PrivateKey::from_wif("cTc4vURSzdx6QE6KVynWGomDbLaA75dNALMNyfjh3p8DRRar84Um")
                 .unwrap();
         check(
-            P2Wpkh(prvkey).build(),
+            P2Wpkh(prvkey).build(Network::Bitcoin),
             true,
             true,
             &["bcrt1q4525hmgw265tl3drrl8jjta7ayffu6jfcwxx9y"],
@@ -558,7 +602,7 @@ mod test {
         )
         .unwrap();
         check(
-            P2Wpkh(pubkey).build(),
+            P2Wpkh(pubkey).build(Network::Bitcoin),
             true,
             true,
             &["bcrt1qngw83fg8dz0k749cg7k3emc7v98wy0c7azaa6h"],
@@ -570,7 +614,7 @@ mod test {
     fn test_bip44_template() {
         let prvkey = bitcoin::util::bip32::ExtendedPrivKey::from_str("tprv8ZgxMBicQKsPcx5nBGsR63Pe8KnRUqmbJNENAfGftF3yuXoMMoVJJcYeUw5eVkm9WBPjWYt6HMWYJNesB5HaNVBaFc1M6dRjWSYnmewUMYy").unwrap();
         check(
-            Bip44(prvkey, KeychainKind::External).build(),
+            Bip44(prvkey, KeychainKind::External).build(Network::Bitcoin),
             false,
             false,
             &[
@@ -580,7 +624,7 @@ mod test {
             ],
         );
         check(
-            Bip44(prvkey, KeychainKind::Internal).build(),
+            Bip44(prvkey, KeychainKind::Internal).build(Network::Bitcoin),
             false,
             false,
             &[
@@ -597,7 +641,7 @@ mod test {
         let pubkey = bitcoin::util::bip32::ExtendedPubKey::from_str("tpubDDDzQ31JkZB7VxUr9bjvBivDdqoFLrDPyLWtLapArAi51ftfmCb2DPxwLQzX65iNcXz1DGaVvyvo6JQ6rTU73r2gqdEo8uov9QKRb7nKCSU").unwrap();
         let fingerprint = bitcoin::util::bip32::Fingerprint::from_str("c55b303f").unwrap();
         check(
-            Bip44Public(pubkey, fingerprint, KeychainKind::External).build(),
+            Bip44Public(pubkey, fingerprint, KeychainKind::External).build(Network::Bitcoin),
             false,
             false,
             &[
@@ -607,7 +651,7 @@ mod test {
             ],
         );
         check(
-            Bip44Public(pubkey, fingerprint, KeychainKind::Internal).build(),
+            Bip44Public(pubkey, fingerprint, KeychainKind::Internal).build(Network::Bitcoin),
             false,
             false,
             &[
@@ -623,7 +667,7 @@ mod test {
     fn test_bip49_template() {
         let prvkey = bitcoin::util::bip32::ExtendedPrivKey::from_str("tprv8ZgxMBicQKsPcx5nBGsR63Pe8KnRUqmbJNENAfGftF3yuXoMMoVJJcYeUw5eVkm9WBPjWYt6HMWYJNesB5HaNVBaFc1M6dRjWSYnmewUMYy").unwrap();
         check(
-            Bip49(prvkey, KeychainKind::External).build(),
+            Bip49(prvkey, KeychainKind::External).build(Network::Bitcoin),
             true,
             false,
             &[
@@ -633,7 +677,7 @@ mod test {
             ],
         );
         check(
-            Bip49(prvkey, KeychainKind::Internal).build(),
+            Bip49(prvkey, KeychainKind::Internal).build(Network::Bitcoin),
             true,
             false,
             &[
@@ -650,7 +694,7 @@ mod test {
         let pubkey = bitcoin::util::bip32::ExtendedPubKey::from_str("tpubDC49r947KGK52X5rBWS4BLs5m9SRY3pYHnvRrm7HcybZ3BfdEsGFyzCMzayi1u58eT82ZeyFZwH7DD6Q83E3fM9CpfMtmnTygnLfP59jL9L").unwrap();
         let fingerprint = bitcoin::util::bip32::Fingerprint::from_str("c55b303f").unwrap();
         check(
-            Bip49Public(pubkey, fingerprint, KeychainKind::External).build(),
+            Bip49Public(pubkey, fingerprint, KeychainKind::External).build(Network::Bitcoin),
             true,
             false,
             &[
@@ -660,7 +704,7 @@ mod test {
             ],
         );
         check(
-            Bip49Public(pubkey, fingerprint, KeychainKind::Internal).build(),
+            Bip49Public(pubkey, fingerprint, KeychainKind::Internal).build(Network::Bitcoin),
             true,
             false,
             &[
@@ -676,7 +720,7 @@ mod test {
     fn test_bip84_template() {
         let prvkey = bitcoin::util::bip32::ExtendedPrivKey::from_str("tprv8ZgxMBicQKsPcx5nBGsR63Pe8KnRUqmbJNENAfGftF3yuXoMMoVJJcYeUw5eVkm9WBPjWYt6HMWYJNesB5HaNVBaFc1M6dRjWSYnmewUMYy").unwrap();
         check(
-            Bip84(prvkey, KeychainKind::External).build(),
+            Bip84(prvkey, KeychainKind::External).build(Network::Bitcoin),
             true,
             false,
             &[
@@ -686,7 +730,7 @@ mod test {
             ],
         );
         check(
-            Bip84(prvkey, KeychainKind::Internal).build(),
+            Bip84(prvkey, KeychainKind::Internal).build(Network::Bitcoin),
             true,
             false,
             &[
@@ -703,7 +747,7 @@ mod test {
         let pubkey = bitcoin::util::bip32::ExtendedPubKey::from_str("tpubDC2Qwo2TFsaNC4ju8nrUJ9mqVT3eSgdmy1yPqhgkjwmke3PRXutNGRYAUo6RCHTcVQaDR3ohNU9we59brGHuEKPvH1ags2nevW5opEE9Z5Q").unwrap();
         let fingerprint = bitcoin::util::bip32::Fingerprint::from_str("c55b303f").unwrap();
         check(
-            Bip84Public(pubkey, fingerprint, KeychainKind::External).build(),
+            Bip84Public(pubkey, fingerprint, KeychainKind::External).build(Network::Bitcoin),
             true,
             false,
             &[
@@ -713,7 +757,7 @@ mod test {
             ],
         );
         check(
-            Bip84Public(pubkey, fingerprint, KeychainKind::Internal).build(),
+            Bip84Public(pubkey, fingerprint, KeychainKind::Internal).build(Network::Bitcoin),
             true,
             false,
             &[

src/doctest.rs

@@ -9,6 +9,6 @@
 // You may not use this file except in accordance with one or both of these
 // licenses.
 
-#[doc(include = "../README.md")]
+#[doc = include_str!("../README.md")]
 #[cfg(doctest)]
 pub struct ReadmeDoctests;

src/error.rs

@@ -13,7 +13,7 @@ use std::fmt;
 
 use crate::bitcoin::Network;
 use crate::{descriptor, wallet, wallet::address_validator};
-use bitcoin::OutPoint;
+use bitcoin::{OutPoint, Txid};
 
 /// Errors that can be thrown by the [`Wallet`](crate::wallet::Wallet)
 #[derive(Debug)]
@@ -125,6 +125,10 @@ pub enum Error {
     //DifferentDescriptorStructure,
     //Uncapable(crate::blockchain::Capability),
     //MissingCachedAddresses,
+    /// [`crate::blockchain::WalletSync`] sync attempt failed due to missing scripts in cache which
+    /// are needed to satisfy `stop_gap`.
+    MissingCachedScripts(MissingCachedScripts),
+
     #[cfg(feature = "electrum")]
     /// Electrum client error
     Electrum(electrum_client::Error),
@@ -145,6 +149,16 @@ pub enum Error {
     Rusqlite(rusqlite::Error),
 }
 
+/// Represents the last failed [`crate::blockchain::WalletSync`] sync attempt in which we were short
+/// on cached `scriptPubKey`s.
+#[derive(Debug)]
+pub struct MissingCachedScripts {
+    /// Number of scripts in which txs were requested during last request.
+    pub last_count: usize,
+    /// Minimum number of scripts to cache more of in order to satisfy `stop_gap`.
+    pub missing_count: usize,
+}
+
 impl fmt::Display for Error {
     fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
         write!(f, "{:?}", self)

src/lib.rs

@@ -43,10 +43,6 @@
 //! blockchain data and an [electrum](https://docs.rs/electrum-client/) blockchain client to
 //! interact with the bitcoin P2P network.
 //!
-//! ```toml
-//! bdk = "0.18.0"
-//! ```
-//!
 //! # Examples
 #![cfg_attr(
     feature = "electrum",
@@ -81,6 +77,7 @@ fn main() -> Result<(), bdk::Error> {
 //!
 //! ## Generate a few addresses
 //!
+//! ### Example
 //! ```
 //! use bdk::{Wallet};
 //! use bdk::database::MemoryDatabase;

src/testutils/blockchain_tests.rs

@@ -372,6 +372,7 @@ macro_rules! bdk_blockchain_tests {
             use $crate::blockchain::Blockchain;
             use $crate::database::MemoryDatabase;
             use $crate::types::KeychainKind;
+            use $crate::wallet::AddressIndex;
             use $crate::{Wallet, FeeRate, SyncOptions};
             use $crate::testutils;
 
@@ -453,7 +454,7 @@ macro_rules! bdk_blockchain_tests {
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
                 assert!(wallet.database().deref().get_sync_time().unwrap().is_some(), "sync_time hasn't been updated");
 
-                assert_eq!(wallet.get_balance().unwrap(), 50_000, "incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 50_000, "incorrect balance");
                 assert_eq!(wallet.list_unspent().unwrap()[0].keychain, KeychainKind::External, "incorrect keychain kind");
 
                 let list_tx_item = &wallet.list_transactions(false).unwrap()[0];
@@ -476,7 +477,7 @@ macro_rules! bdk_blockchain_tests {
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
 
-                assert_eq!(wallet.get_balance().unwrap(), 100_000, "incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 100_000, "incorrect balance");
                 assert_eq!(wallet.list_transactions(false).unwrap().len(), 2, "incorrect number of txs");
             }
 
@@ -485,7 +486,7 @@ macro_rules! bdk_blockchain_tests {
                 let (wallet, blockchain, descriptors, mut test_client) = init_single_sig();
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 0);
+                assert_eq!(wallet.get_balance().unwrap().get_total(), 0);
 
                 test_client.receive(testutils! {
                     @tx ( (@external descriptors, 0) => 50_000 )
@@ -493,8 +494,16 @@ macro_rules! bdk_blockchain_tests {
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
 
-                assert_eq!(wallet.get_balance().unwrap(), 50_000, "incorrect balance");
-                assert_eq!(wallet.list_transactions(false).unwrap().len(), 1, "incorrect number of txs");
+                assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 50_000, "incorrect balance");
+
+                test_client.receive(testutils! {
+                    @tx ( (@external descriptors, 0) => 50_000 ) (@confirmations 1)
+                });
+
+                wallet.sync(&blockchain, SyncOptions::default()).unwrap();
+
+                assert_eq!(wallet.get_balance().unwrap().confirmed, 100_000, "incorrect balance");
+                assert_eq!(wallet.list_transactions(false).unwrap().len(), 2, "incorrect number of txs");
             }
 
             #[test]
@@ -507,7 +516,7 @@ macro_rules! bdk_blockchain_tests {
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
 
-                assert_eq!(wallet.get_balance().unwrap(), 105_000, "incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 105_000, "incorrect balance");
                 assert_eq!(wallet.list_transactions(false).unwrap().len(), 1, "incorrect number of txs");
                 assert_eq!(wallet.list_unspent().unwrap().len(), 3, "incorrect number of unspents");
 
@@ -531,7 +540,7 @@ macro_rules! bdk_blockchain_tests {
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
 
-                assert_eq!(wallet.get_balance().unwrap(), 75_000, "incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 75_000, "incorrect balance");
                 assert_eq!(wallet.list_transactions(false).unwrap().len(), 2, "incorrect number of txs");
                 assert_eq!(wallet.list_unspent().unwrap().len(), 2, "incorrect number of unspent");
             }
@@ -545,14 +554,14 @@ macro_rules! bdk_blockchain_tests {
                 });
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 50_000);
+                assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 50_000);
 
                 test_client.receive(testutils! {
                     @tx ( (@external descriptors, 0) => 25_000 )
                 });
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 75_000, "incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 75_000, "incorrect balance");
             }
 
             #[test]
@@ -565,7 +574,7 @@ macro_rules! bdk_blockchain_tests {
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
 
-                assert_eq!(wallet.get_balance().unwrap(), 50_000, "incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 50_000, "incorrect balance");
                 assert_eq!(wallet.list_transactions(false).unwrap().len(), 1, "incorrect number of txs");
                 assert_eq!(wallet.list_unspent().unwrap().len(), 1, "incorrect unspent");
 
@@ -579,7 +588,7 @@ macro_rules! bdk_blockchain_tests {
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
 
-                assert_eq!(wallet.get_balance().unwrap(), 50_000, "incorrect balance after bump");
+                assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 50_000, "incorrect balance after bump");
                 assert_eq!(wallet.list_transactions(false).unwrap().len(), 1, "incorrect number of txs after bump");
                 assert_eq!(wallet.list_unspent().unwrap().len(), 1, "incorrect unspent after bump");
 
@@ -602,8 +611,7 @@ macro_rules! bdk_blockchain_tests {
                 });
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-
-                assert_eq!(wallet.get_balance().unwrap(), 50_000, "incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), 50_000, "incorrect balance");
                 assert_eq!(wallet.list_transactions(false).unwrap().len(), 1, "incorrect number of txs");
                 assert_eq!(wallet.list_unspent().unwrap().len(), 1, "incorrect number of unspents");
 
@@ -616,7 +624,7 @@ macro_rules! bdk_blockchain_tests {
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
 
-                assert_eq!(wallet.get_balance().unwrap(), 50_000, "incorrect balance after invalidate");
+                assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 50_000, "incorrect balance after invalidate");
 
                 let list_tx_item = &wallet.list_transactions(false).unwrap()[0];
                 assert_eq!(list_tx_item.txid, txid, "incorrect txid after invalidate");
@@ -634,7 +642,7 @@ macro_rules! bdk_blockchain_tests {
                 });
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 50_000, "incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 50_000, "incorrect balance");
 
                 let mut builder = wallet.build_tx();
                 builder.add_recipient(node_addr.script_pubkey(), 25_000);
@@ -645,12 +653,71 @@ macro_rules! bdk_blockchain_tests {
                 println!("{}", bitcoin::consensus::encode::serialize_hex(&tx));
                 blockchain.broadcast(&tx).unwrap();
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), details.received, "incorrect balance after send");
+                assert_eq!(wallet.get_balance().unwrap().trusted_pending, details.received, "incorrect balance after send");
+
+                test_client.generate(1, Some(node_addr));
+                wallet.sync(&blockchain, SyncOptions::default()).unwrap();
+
+                assert_eq!(wallet.get_balance().unwrap().confirmed, details.received, "incorrect balance after send");
 
                 assert_eq!(wallet.list_transactions(false).unwrap().len(), 2, "incorrect number of txs");
                 assert_eq!(wallet.list_unspent().unwrap().len(), 1, "incorrect number of unspents");
             }
 
+            // Syncing wallet should not result in wallet address index to decrement.
+            // This is critical as we should always ensure to not reuse addresses.
+            #[test]
+            fn test_sync_address_index_should_not_decrement() {
+                let (wallet, blockchain, _descriptors, mut test_client) = init_single_sig();
+
+                const ADDRS_TO_FUND: u32 = 7;
+                const ADDRS_TO_IGNORE: u32 = 11;
+
+                let mut first_addr_index: u32 = 0;
+
+                (0..ADDRS_TO_FUND + ADDRS_TO_IGNORE).for_each(|i| {
+                    let new_addr = wallet.get_address(AddressIndex::New).unwrap();
+
+                    if i == 0 {
+                        first_addr_index = new_addr.index;
+                    }
+                    assert_eq!(new_addr.index, i+first_addr_index, "unexpected new address index (before sync)");
+
+                    if i < ADDRS_TO_FUND {
+                        test_client.receive(testutils! {
+                            @tx ((@addr new_addr.address) => 50_000)
+                        });
+                    }
+                });
+
+                wallet.sync(&blockchain, SyncOptions::default()).unwrap();
+
+                let new_addr = wallet.get_address(AddressIndex::New).unwrap();
+                assert_eq!(new_addr.index, ADDRS_TO_FUND+ADDRS_TO_IGNORE+first_addr_index, "unexpected new address index (after sync)");
+            }
+
+            // Even if user does not explicitly grab new addresses, the address index should
+            // increment after sync (if wallet has a balance).
+            #[test]
+            fn test_sync_address_index_should_increment() {
+                let (wallet, blockchain, descriptors, mut test_client) = init_single_sig();
+
+                const START_FUND: u32 = 4;
+                const END_FUND: u32 = 20;
+
+                // "secretly" fund wallet via given range
+                (START_FUND..END_FUND).for_each(|addr_index| {
+                    test_client.receive(testutils! {
+                        @tx ((@external descriptors, addr_index) => 50_000)
+                    });
+                });
+
+                wallet.sync(&blockchain, SyncOptions::default()).unwrap();
+
+                let address = wallet.get_address(AddressIndex::New).unwrap();
+                assert_eq!(address.index, END_FUND, "unexpected new address index (after sync)");
+            }
+
             /// Send two conflicting transactions to the same address twice in a row.
             /// The coins should only be received once!
             #[test]
@@ -665,7 +732,7 @@ macro_rules! bdk_blockchain_tests {
                 });
 
                 wallet.sync(&blockchain, SyncOptions::default()).expect("sync");
-                assert_eq!(wallet.get_balance().unwrap(), 75_000, "incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), 75_000, "incorrect balance");
                 let target_addr = receiver_wallet.get_address($crate::wallet::AddressIndex::New).unwrap().address;
 
                 let tx1 = {
@@ -688,9 +755,8 @@ macro_rules! bdk_blockchain_tests {
 
                 blockchain.broadcast(&tx1).expect("broadcasting first");
                 blockchain.broadcast(&tx2).expect("broadcasting replacement");
-
                 receiver_wallet.sync(&blockchain, SyncOptions::default()).expect("syncing receiver");
-                assert_eq!(receiver_wallet.get_balance().expect("balance"), 49_000, "should have received coins once and only once");
+                assert_eq!(receiver_wallet.get_balance().expect("balance").untrusted_pending, 49_000, "should have received coins once and only once");
             }
 
             #[test]
@@ -716,7 +782,8 @@ macro_rules! bdk_blockchain_tests {
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
 
-                assert_eq!(wallet.get_balance().unwrap(), 100_000);
+                let balance = wallet.get_balance().unwrap();
+                assert_eq!(balance.untrusted_pending + balance.get_spendable(), 100_000);
             }
 
             #[test]
@@ -730,7 +797,7 @@ macro_rules! bdk_blockchain_tests {
                 });
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 50_000, "incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 50_000, "incorrect balance");
 
                 let tx_map = wallet.list_transactions(false).unwrap().into_iter().map(|tx| (tx.txid, tx)).collect::<std::collections::HashMap<_, _>>();
                 let details = tx_map.get(&received_txid).unwrap();
@@ -754,7 +821,7 @@ macro_rules! bdk_blockchain_tests {
                 });
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 50_000, "incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 50_000, "incorrect balance");
 
                 let mut builder = wallet.build_tx();
                 builder.add_recipient(node_addr.script_pubkey(), 25_000);
@@ -766,7 +833,7 @@ macro_rules! bdk_blockchain_tests {
                 blockchain.broadcast(&sent_tx).unwrap();
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), details.received, "incorrect balance after receive");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), details.received, "incorrect balance after receive");
 
                 // empty wallet
                 let wallet = get_wallet_from_descriptors(&descriptors);
@@ -797,7 +864,7 @@ macro_rules! bdk_blockchain_tests {
                 });
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 50_000, "incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 50_000, "incorrect balance");
 
                 let mut total_sent = 0;
                 for _ in 0..5 {
@@ -814,7 +881,7 @@ macro_rules! bdk_blockchain_tests {
                 }
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 50_000 - total_sent, "incorrect balance after chain");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), 50_000 - total_sent, "incorrect balance after chain");
 
                 // empty wallet
 
@@ -824,7 +891,7 @@ macro_rules! bdk_blockchain_tests {
                 test_client.generate(1, Some(node_addr));
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 50_000 - total_sent, "incorrect balance empty wallet");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), 50_000 - total_sent, "incorrect balance empty wallet");
 
             }
 
@@ -838,7 +905,7 @@ macro_rules! bdk_blockchain_tests {
                 });
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 50_000, "incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), 50_000, "incorrect balance");
 
                 let mut builder = wallet.build_tx();
                 builder.add_recipient(node_addr.script_pubkey().clone(), 5_000).enable_rbf();
@@ -847,8 +914,8 @@ macro_rules! bdk_blockchain_tests {
                 assert!(finalized, "Cannot finalize transaction");
                 blockchain.broadcast(&psbt.extract_tx()).unwrap();
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 50_000 - details.fee.unwrap_or(0) - 5_000, "incorrect balance from fees");
-                assert_eq!(wallet.get_balance().unwrap(), details.received, "incorrect balance from received");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), 50_000 - details.fee.unwrap_or(0) - 5_000, "incorrect balance from fees");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), details.received, "incorrect balance from received");
 
                 let mut builder = wallet.build_fee_bump(details.txid).unwrap();
                 builder.fee_rate(FeeRate::from_sat_per_vb(2.1));
@@ -857,8 +924,8 @@ macro_rules! bdk_blockchain_tests {
                 assert!(finalized, "Cannot finalize transaction");
                 blockchain.broadcast(&new_psbt.extract_tx()).unwrap();
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 50_000 - new_details.fee.unwrap_or(0) - 5_000, "incorrect balance from fees after bump");
-                assert_eq!(wallet.get_balance().unwrap(), new_details.received, "incorrect balance from received after bump");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), 50_000 - new_details.fee.unwrap_or(0) - 5_000, "incorrect balance from fees after bump");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), new_details.received, "incorrect balance from received after bump");
 
                 assert!(new_details.fee.unwrap_or(0) > details.fee.unwrap_or(0), "incorrect fees");
             }
@@ -873,7 +940,7 @@ macro_rules! bdk_blockchain_tests {
                 });
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 50_000, "incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), 50_000, "incorrect balance");
 
                 let mut builder = wallet.build_tx();
                 builder.add_recipient(node_addr.script_pubkey().clone(), 49_000).enable_rbf();
@@ -882,8 +949,8 @@ macro_rules! bdk_blockchain_tests {
                 assert!(finalized, "Cannot finalize transaction");
                 blockchain.broadcast(&psbt.extract_tx()).unwrap();
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 1_000 - details.fee.unwrap_or(0), "incorrect balance after send");
-                assert_eq!(wallet.get_balance().unwrap(), details.received, "incorrect received after send");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), 1_000 - details.fee.unwrap_or(0), "incorrect balance after send");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), details.received, "incorrect received after send");
 
                 let mut builder = wallet.build_fee_bump(details.txid).unwrap();
                 builder.fee_rate(FeeRate::from_sat_per_vb(5.1));
@@ -892,7 +959,7 @@ macro_rules! bdk_blockchain_tests {
                 assert!(finalized, "Cannot finalize transaction");
                 blockchain.broadcast(&new_psbt.extract_tx()).unwrap();
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 0, "incorrect balance after change removal");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), 0, "incorrect balance after change removal");
                 assert_eq!(new_details.received, 0, "incorrect received after change removal");
 
                 assert!(new_details.fee.unwrap_or(0) > details.fee.unwrap_or(0), "incorrect fees");
@@ -908,7 +975,7 @@ macro_rules! bdk_blockchain_tests {
                 });
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 75_000, "incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), 75_000, "incorrect balance");
 
                 let mut builder = wallet.build_tx();
                 builder.add_recipient(node_addr.script_pubkey().clone(), 49_000).enable_rbf();
@@ -917,7 +984,7 @@ macro_rules! bdk_blockchain_tests {
                 assert!(finalized, "Cannot finalize transaction");
                 blockchain.broadcast(&psbt.extract_tx()).unwrap();
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 26_000 - details.fee.unwrap_or(0), "incorrect balance after send");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), 26_000 - details.fee.unwrap_or(0), "incorrect balance after send");
                 assert_eq!(details.received, 1_000 - details.fee.unwrap_or(0), "incorrect received after send");
 
                 let mut builder = wallet.build_fee_bump(details.txid).unwrap();
@@ -928,7 +995,7 @@ macro_rules! bdk_blockchain_tests {
                 blockchain.broadcast(&new_psbt.extract_tx()).unwrap();
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
                 assert_eq!(new_details.sent, 75_000, "incorrect sent");
-                assert_eq!(wallet.get_balance().unwrap(), new_details.received, "incorrect balance after add input");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), new_details.received, "incorrect balance after add input");
             }
 
             #[test]
@@ -941,7 +1008,7 @@ macro_rules! bdk_blockchain_tests {
                 });
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 75_000, "incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), 75_000, "incorrect balance");
 
                 let mut builder = wallet.build_tx();
                 builder.add_recipient(node_addr.script_pubkey().clone(), 49_000).enable_rbf();
@@ -950,7 +1017,7 @@ macro_rules! bdk_blockchain_tests {
                 assert!(finalized, "Cannot finalize transaction");
                 blockchain.broadcast(&psbt.extract_tx()).unwrap();
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 26_000 - details.fee.unwrap_or(0), "incorrect balance after send");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), 26_000 - details.fee.unwrap_or(0), "incorrect balance after send");
                 assert_eq!(details.received, 1_000 - details.fee.unwrap_or(0), "incorrect received after send");
 
                 let mut builder = wallet.build_fee_bump(details.txid).unwrap();
@@ -963,7 +1030,7 @@ macro_rules! bdk_blockchain_tests {
                 blockchain.broadcast(&new_psbt.extract_tx()).unwrap();
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
                 assert_eq!(new_details.sent, 75_000, "incorrect sent");
-                assert_eq!(wallet.get_balance().unwrap(), 0, "incorrect balance after add input");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), 0, "incorrect balance after add input");
                 assert_eq!(new_details.received, 0, "incorrect received after add input");
             }
 
@@ -977,7 +1044,7 @@ macro_rules! bdk_blockchain_tests {
                 });
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 50_000, "incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 50_000, "incorrect balance");
 
                 let mut builder = wallet.build_tx();
                 let data = [42u8;80];
@@ -992,7 +1059,7 @@ macro_rules! bdk_blockchain_tests {
                 blockchain.broadcast(&tx).unwrap();
                 test_client.generate(1, Some(node_addr));
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 50_000 - details.fee.unwrap_or(0), "incorrect balance after send");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), 50_000 - details.fee.unwrap_or(0), "incorrect balance after send");
 
                 let tx_map = wallet.list_transactions(false).unwrap().into_iter().map(|tx| (tx.txid, tx)).collect::<std::collections::HashMap<_, _>>();
                 let _ = tx_map.get(&tx.txid()).unwrap();
@@ -1003,22 +1070,24 @@ macro_rules! bdk_blockchain_tests {
                 let (wallet, blockchain, _, mut test_client) = init_single_sig();
 
                 let wallet_addr = wallet.get_address($crate::wallet::AddressIndex::New).unwrap().address;
+                println!("wallet addr: {}", wallet_addr);
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 0, "incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().immature, 0, "incorrect balance");
 
                 test_client.generate(1, Some(wallet_addr));
 
-                #[cfg(feature = "rpc")]
-                {
-                    // rpc consider coinbase only when mature (100 blocks)
-                    let node_addr = test_client.get_node_address(None);
-                    test_client.generate(100, Some(node_addr));
-                }
-
-
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert!(wallet.get_balance().unwrap() > 0, "incorrect balance after receiving coinbase");
+
+                assert!(wallet.get_balance().unwrap().immature > 0, "incorrect balance after receiving coinbase");
+
+                // make coinbase mature (100 blocks)
+                let node_addr = test_client.get_node_address(None);
+                test_client.generate(100, Some(node_addr));
+                wallet.sync(&blockchain, SyncOptions::default()).unwrap();
+
+                assert!(wallet.get_balance().unwrap().confirmed > 0, "incorrect balance after maturing coinbase");
+
             }
 
             #[test]
@@ -1095,7 +1164,7 @@ macro_rules! bdk_blockchain_tests {
                 });
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 50_000, "wallet has incorrect balance");
+                assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 50_000, "wallet has incorrect balance");
 
                 // 4. Send 25_000 sats from test BDK wallet to test bitcoind node taproot wallet
 
@@ -1107,7 +1176,7 @@ macro_rules! bdk_blockchain_tests {
                 let tx = psbt.extract_tx();
                 blockchain.broadcast(&tx).unwrap();
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), details.received, "wallet has incorrect balance after send");
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), details.received, "wallet has incorrect balance after send");
                 assert_eq!(wallet.list_transactions(false).unwrap().len(), 2, "wallet has incorrect number of txs");
                 assert_eq!(wallet.list_unspent().unwrap().len(), 1, "wallet has incorrect number of unspents");
                 test_client.generate(1, None);
@@ -1213,12 +1282,12 @@ macro_rules! bdk_blockchain_tests {
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
 
                 let _ = test_client.receive(testutils! {
-                    @tx ( (@external descriptors, 0)   => 50_000 )
+                    @tx ( (@external descriptors, 0) => 50_000 )
                 });
 
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
 
-                assert_eq!(wallet.get_balance().unwrap(), 50_000);
+                assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 50_000);
 
                 let tx = {
                     let mut builder = wallet.build_tx();
@@ -1241,7 +1310,7 @@ macro_rules! bdk_blockchain_tests {
                     @tx ( (@external descriptors, 0) => 50_000 )
                 });
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 50_000);
+                assert_eq!(wallet.get_balance().unwrap().untrusted_pending, 50_000);
 
                 let tx = {
                     let mut builder = wallet.build_tx();
@@ -1262,7 +1331,7 @@ macro_rules! bdk_blockchain_tests {
                     @tx ( (@external descriptors, 0) => 50_000 ) ( @confirmations 6 )
                 });
                 wallet.sync(&blockchain, SyncOptions::default()).unwrap();
-                assert_eq!(wallet.get_balance().unwrap(), 50_000);
+                assert_eq!(wallet.get_balance().unwrap().get_spendable(), 50_000);
 
                 let ext_policy = wallet.policies(KeychainKind::External).unwrap().unwrap();
                 let int_policy = wallet.policies(KeychainKind::Internal).unwrap().unwrap();
@@ -1361,6 +1430,35 @@ macro_rules! bdk_blockchain_tests {
                 let finalized = wallet.sign(&mut psbt, Default::default()).unwrap();
                 assert_eq!(finalized, true);
             }
+
+            #[test]
+            fn test_get_block_hash() {
+                use bitcoincore_rpc::{ RpcApi };
+                use crate::blockchain::GetBlockHash;
+
+                // create wallet with init_wallet
+                let (_, blockchain, _descriptors, mut test_client) = init_single_sig();
+
+                let height = test_client.bitcoind.client.get_blockchain_info().unwrap().blocks as u64;
+                let best_hash = test_client.bitcoind.client.get_best_block_hash().unwrap();
+
+                // use get_block_hash to get best block hash and compare with best_hash above
+                let block_hash = blockchain.get_block_hash(height).unwrap();
+                assert_eq!(best_hash, block_hash);
+
+                // generate blocks to address
+                let node_addr = test_client.get_node_address(None);
+                test_client.generate(10, Some(node_addr));
+
+                let height = test_client.bitcoind.client.get_blockchain_info().unwrap().blocks as u64;
+                let best_hash = test_client.bitcoind.client.get_best_block_hash().unwrap();
+
+                let block_hash = blockchain.get_block_hash(height).unwrap();
+                assert_eq!(best_hash, block_hash);
+
+                // try to get hash for block that has not yet been created.
+                assert!(blockchain.get_block_hash(height + 1).is_err());
+            }
         }
     };
 

src/testutils/configurable_blockchain_tests.rs

@@ -0,0 +1,257 @@
+use bitcoin::Network;
+
+use crate::{
+    blockchain::ConfigurableBlockchain, database::MemoryDatabase, testutils, wallet::AddressIndex,
+    Wallet,
+};
+
+use super::blockchain_tests::TestClient;
+
+/// Trait for testing [`ConfigurableBlockchain`] implementations.
+pub trait ConfigurableBlockchainTester<B: ConfigurableBlockchain>: Sized {
+    /// Blockchain name for logging.
+    const BLOCKCHAIN_NAME: &'static str;
+
+    /// Generates a blockchain config with a given stop_gap.
+    ///
+    /// If this returns [`Option::None`], then the associated tests will not run.
+    fn config_with_stop_gap(
+        &self,
+        _test_client: &mut TestClient,
+        _stop_gap: usize,
+    ) -> Option<B::Config> {
+        None
+    }
+
+    /// Runs all avaliable tests.
+    fn run(&self) {
+        let test_client = &mut TestClient::default();
+
+        if self.config_with_stop_gap(test_client, 0).is_some() {
+            test_wallet_sync_with_stop_gaps(test_client, self);
+            test_wallet_sync_fulfills_missing_script_cache(test_client, self);
+            test_wallet_sync_self_transfer_tx(test_client, self);
+        } else {
+            println!(
+                "{}: Skipped tests requiring config_with_stop_gap.",
+                Self::BLOCKCHAIN_NAME
+            );
+        }
+    }
+}
+
+/// Test whether blockchain implementation syncs with expected behaviour given different `stop_gap`
+/// parameters.
+///
+/// For each test vector:
+/// * Fill wallet's derived addresses with balances (as specified by test vector).
+///    * [0..addrs_before]          => 1000sats for each address
+///    * [addrs_before..actual_gap] => empty addresses
+///    * [actual_gap..addrs_after]  => 1000sats for each address
+/// * Then, perform wallet sync and obtain wallet balance
+/// * Check balance is within expected range (we can compare `stop_gap` and `actual_gap` to
+///    determine this).
+fn test_wallet_sync_with_stop_gaps<T, B>(test_client: &mut TestClient, tester: &T)
+where
+    T: ConfigurableBlockchainTester<B>,
+    B: ConfigurableBlockchain,
+{
+    // Generates wallet descriptor
+    let descriptor_of_account = |account_index: usize| -> String {
+        format!("wpkh([c258d2e4/84h/1h/0h]tpubDDYkZojQFQjht8Tm4jsS3iuEmKjTiEGjG6KnuFNKKJb5A6ZUCUZKdvLdSDWofKi4ToRCwb9poe1XdqfUnP4jaJjCB2Zwv11ZLgSbnZSNecE/{account_index}/*)")
+    };
+
+    // Amount (in satoshis) provided to a single address (which expects to have a balance)
+    const AMOUNT_PER_TX: u64 = 1000;
+
+    // [stop_gap, actual_gap, addrs_before, addrs_after]
+    //
+    // [0]     stop_gap: Passed to [`ElectrumBlockchainConfig`]
+    // [1]   actual_gap: Range size of address indexes without a balance
+    // [2] addrs_before: Range size of address indexes (before gap) which contains a balance
+    // [3]  addrs_after: Range size of address indexes (after gap) which contains a balance
+    let test_vectors: Vec<[u64; 4]> = vec![
+        [0, 0, 0, 5],
+        [0, 0, 5, 5],
+        [0, 1, 5, 5],
+        [0, 2, 5, 5],
+        [1, 0, 5, 5],
+        [1, 1, 5, 5],
+        [1, 2, 5, 5],
+        [2, 1, 5, 5],
+        [2, 2, 5, 5],
+        [2, 3, 5, 5],
+    ];
+
+    for (account_index, vector) in test_vectors.into_iter().enumerate() {
+        let [stop_gap, actual_gap, addrs_before, addrs_after] = vector;
+        let descriptor = descriptor_of_account(account_index);
+
+        let blockchain = B::from_config(
+            &tester
+                .config_with_stop_gap(test_client, stop_gap as _)
+                .unwrap(),
+        )
+        .unwrap();
+
+        let wallet =
+            Wallet::new(&descriptor, None, Network::Regtest, MemoryDatabase::new()).unwrap();
+
+        // fill server-side with txs to specified address indexes
+        // return the max balance of the wallet (also the actual balance)
+        let max_balance = (0..addrs_before)
+            .chain(addrs_before + actual_gap..addrs_before + actual_gap + addrs_after)
+            .fold(0_u64, |sum, i| {
+                let address = wallet.get_address(AddressIndex::Peek(i as _)).unwrap();
+                test_client.receive(testutils! {
+                    @tx ( (@addr address.address) => AMOUNT_PER_TX )
+                });
+                sum + AMOUNT_PER_TX
+            });
+
+        // minimum allowed balance of wallet (based on stop gap)
+        let min_balance = if actual_gap > stop_gap {
+            addrs_before * AMOUNT_PER_TX
+        } else {
+            max_balance
+        };
+        let details = format!(
+            "test_vector: [stop_gap: {}, actual_gap: {}, addrs_before: {}, addrs_after: {}]",
+            stop_gap, actual_gap, addrs_before, addrs_after,
+        );
+        println!("{}", details);
+
+        // perform wallet sync
+        wallet.sync(&blockchain, Default::default()).unwrap();
+
+        let wallet_balance = wallet.get_balance().unwrap().get_total();
+        println!(
+            "max: {}, min: {}, actual: {}",
+            max_balance, min_balance, wallet_balance
+        );
+
+        assert!(
+            wallet_balance <= max_balance,
+            "wallet balance is greater than received amount: {}",
+            details
+        );
+        assert!(
+            wallet_balance >= min_balance,
+            "wallet balance is smaller than expected: {}",
+            details
+        );
+
+        // generate block to confirm new transactions
+        test_client.generate(1, None);
+    }
+}
+
+/// With a `stop_gap` of x and every x addresses having a balance of 1000 (for y addresses),
+/// we expect `Wallet::sync` to correctly self-cache addresses, so that the resulting balance,
+/// after sync, should be y * 1000.
+fn test_wallet_sync_fulfills_missing_script_cache<T, B>(test_client: &mut TestClient, tester: &T)
+where
+    T: ConfigurableBlockchainTester<B>,
+    B: ConfigurableBlockchain,
+{
+    // wallet descriptor
+    let descriptor = "wpkh([c258d2e4/84h/1h/0h]tpubDDYkZojQFQjht8Tm4jsS3iuEmKjTiEGjG6KnuFNKKJb5A6ZUCUZKdvLdSDWofKi4ToRCwb9poe1XdqfUnP4jaJjCB2Zwv11ZLgSbnZSNecE/200/*)";
+
+    // amount in sats per tx
+    const AMOUNT_PER_TX: u64 = 1000;
+
+    // addr constants
+    const ADDR_COUNT: usize = 6;
+    const ADDR_GAP: usize = 60;
+
+    let blockchain =
+        B::from_config(&tester.config_with_stop_gap(test_client, ADDR_GAP).unwrap()).unwrap();
+
+    let wallet = Wallet::new(descriptor, None, Network::Regtest, MemoryDatabase::new()).unwrap();
+
+    let expected_balance = (0..ADDR_COUNT).fold(0_u64, |sum, i| {
+        let addr_i = i * ADDR_GAP;
+        let address = wallet.get_address(AddressIndex::Peek(addr_i as _)).unwrap();
+
+        println!(
+            "tx: {} sats => [{}] {}",
+            AMOUNT_PER_TX,
+            addr_i,
+            address.to_string()
+        );
+
+        test_client.receive(testutils! {
+            @tx ( (@addr address.address) => AMOUNT_PER_TX )
+        });
+        test_client.generate(1, None);
+
+        sum + AMOUNT_PER_TX
+    });
+    println!("expected balance: {}, syncing...", expected_balance);
+
+    // perform sync
+    wallet.sync(&blockchain, Default::default()).unwrap();
+    println!("sync done!");
+
+    let balance = wallet.get_balance().unwrap().get_total();
+    assert_eq!(balance, expected_balance);
+}
+
+/// Given a `stop_gap`, a wallet with a 2 transactions, one sending to `scriptPubKey` at derivation
+/// index of `stop_gap`, and the other spending from the same `scriptPubKey` into another
+/// `scriptPubKey` at derivation index of `stop_gap * 2`, we expect `Wallet::sync` to perform
+/// correctly, so that we detect the total balance.
+fn test_wallet_sync_self_transfer_tx<T, B>(test_client: &mut TestClient, tester: &T)
+where
+    T: ConfigurableBlockchainTester<B>,
+    B: ConfigurableBlockchain,
+{
+    const TRANSFER_AMOUNT: u64 = 10_000;
+    const STOP_GAP: usize = 75;
+
+    let descriptor = "wpkh(tprv8i8F4EhYDMquzqiecEX8SKYMXqfmmb1Sm7deoA1Hokxzn281XgTkwsd6gL8aJevLE4aJugfVf9MKMvrcRvPawGMenqMBA3bRRfp4s1V7Eg3/*)";
+
+    let blockchain =
+        B::from_config(&tester.config_with_stop_gap(test_client, STOP_GAP).unwrap()).unwrap();
+
+    let wallet = Wallet::new(descriptor, None, Network::Regtest, MemoryDatabase::new()).unwrap();
+
+    let address1 = wallet
+        .get_address(AddressIndex::Peek(STOP_GAP as _))
+        .unwrap();
+    let address2 = wallet
+        .get_address(AddressIndex::Peek((STOP_GAP * 2) as _))
+        .unwrap();
+
+    test_client.receive(testutils! {
+        @tx ( (@addr address1.address) => TRANSFER_AMOUNT )
+    });
+    test_client.generate(1, None);
+
+    wallet.sync(&blockchain, Default::default()).unwrap();
+
+    let mut builder = wallet.build_tx();
+    builder.add_recipient(address2.script_pubkey(), TRANSFER_AMOUNT / 2);
+    let (mut psbt, details) = builder.finish().unwrap();
+    assert!(wallet.sign(&mut psbt, Default::default()).unwrap());
+    blockchain.broadcast(&psbt.extract_tx()).unwrap();
+
+    test_client.generate(1, None);
+
+    // obtain what is expected
+    let fee = details.fee.unwrap();
+    let expected_balance = TRANSFER_AMOUNT - fee;
+    println!("fee={}, expected_balance={}", fee, expected_balance);
+
+    // actually test the wallet
+    wallet.sync(&blockchain, Default::default()).unwrap();
+    let balance = wallet.get_balance().unwrap().get_total();
+    assert_eq!(balance, expected_balance);
+
+    // now try with a fresh wallet
+    let fresh_wallet =
+        Wallet::new(descriptor, None, Network::Regtest, MemoryDatabase::new()).unwrap();
+    fresh_wallet.sync(&blockchain, Default::default()).unwrap();
+    let fresh_balance = fresh_wallet.get_balance().unwrap().get_total();
+    assert_eq!(fresh_balance, expected_balance);
+}

src/testutils/mod.rs

@@ -14,6 +14,10 @@
 #[cfg(feature = "test-blockchains")]
 pub mod blockchain_tests;
 
+#[cfg(test)]
+#[cfg(feature = "test-blockchains")]
+pub mod configurable_blockchain_tests;
+
 use bitcoin::{Address, Txid};
 
 #[derive(Clone, Debug)]

src/types.rs

@@ -51,14 +51,44 @@ impl AsRef<[u8]> for KeychainKind {
 pub struct FeeRate(f32);
 
 impl FeeRate {
+    /// Create a new instance checking the value provided
+    ///
+    /// ## Panics
+    ///
+    /// Panics if the value is not [normal](https://doc.rust-lang.org/std/primitive.f32.html#method.is_normal) (except if it's a positive zero) or negative.
+    fn new_checked(value: f32) -> Self {
+        assert!(value.is_normal() || value == 0.0);
+        assert!(value.is_sign_positive());
+
+        FeeRate(value)
+    }
+
+    /// Create a new instance of [`FeeRate`] given a float fee rate in sats/kwu
+    pub fn from_sat_per_kwu(sat_per_kwu: f32) -> Self {
+        FeeRate::new_checked(sat_per_kwu / 250.0_f32)
+    }
+
+    /// Create a new instance of [`FeeRate`] given a float fee rate in sats/kvb
+    pub fn from_sat_per_kvb(sat_per_kvb: f32) -> Self {
+        FeeRate::new_checked(sat_per_kvb / 1000.0_f32)
+    }
+
     /// Create a new instance of [`FeeRate`] given a float fee rate in btc/kvbytes
+    ///
+    /// ## Panics
+    ///
+    /// Panics if the value is not [normal](https://doc.rust-lang.org/std/primitive.f32.html#method.is_normal) (except if it's a positive zero) or negative.
     pub fn from_btc_per_kvb(btc_per_kvb: f32) -> Self {
-        FeeRate(btc_per_kvb * 1e5)
+        FeeRate::new_checked(btc_per_kvb * 1e5)
     }
 
     /// Create a new instance of [`FeeRate`] given a float fee rate in satoshi/vbyte
-    pub const fn from_sat_per_vb(sat_per_vb: f32) -> Self {
-        FeeRate(sat_per_vb)
+    ///
+    /// ## Panics
+    ///
+    /// Panics if the value is not [normal](https://doc.rust-lang.org/std/primitive.f32.html#method.is_normal) (except if it's a positive zero) or negative.
+    pub fn from_sat_per_vb(sat_per_vb: f32) -> Self {
+        FeeRate::new_checked(sat_per_vb)
     }
 
     /// Create a new [`FeeRate`] with the default min relay fee value
@@ -78,7 +108,7 @@ impl FeeRate {
     }
 
     /// Return the value as satoshi/vbyte
-    pub fn as_sat_vb(&self) -> f32 {
+    pub fn as_sat_per_vb(&self) -> f32 {
         self.0
     }
 
@@ -89,7 +119,7 @@ impl FeeRate {
 
     /// Calculate absolute fee in Satoshis using size in virtual bytes.
     pub fn fee_vb(&self, vbytes: usize) -> u64 {
-        (self.as_sat_vb() * vbytes as f32).ceil() as u64
+        (self.as_sat_per_vb() * vbytes as f32).ceil() as u64
     }
 }
 
@@ -202,10 +232,12 @@ pub struct TransactionDetails {
     pub txid: Txid,
 
     /// Received value (sats)
+    /// Sum of owned outputs of this transaction.
     pub received: u64,
     /// Sent value (sats)
+    /// Sum of owned inputs of this transaction.
     pub sent: u64,
-    /// Fee value (sats) if available.
+    /// Fee value (sats) if confirmed.
     /// The availability of the fee depends on the backend. It's never `None` with an Electrum
     /// Server backend, but it could be `None` with a Bitcoin RPC node without txindex that receive
     /// funds while offline.
@@ -240,13 +272,130 @@ impl BlockTime {
     }
 }
 
+/// Balance differentiated in various categories
+#[derive(Serialize, Deserialize, Debug, PartialEq, Eq, Clone, Default)]
+pub struct Balance {
+    /// All coinbase outputs not yet matured
+    pub immature: u64,
+    /// Unconfirmed UTXOs generated by a wallet tx
+    pub trusted_pending: u64,
+    /// Unconfirmed UTXOs received from an external wallet
+    pub untrusted_pending: u64,
+    /// Confirmed and immediately spendable balance
+    pub confirmed: u64,
+}
+
+impl Balance {
+    /// Get sum of trusted_pending and confirmed coins
+    pub fn get_spendable(&self) -> u64 {
+        self.confirmed + self.trusted_pending
+    }
+
+    /// Get the whole balance visible to the wallet
+    pub fn get_total(&self) -> u64 {
+        self.confirmed + self.trusted_pending + self.untrusted_pending + self.immature
+    }
+}
+
+impl std::fmt::Display for Balance {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+        write!(
+            f,
+            "{{ immature: {}, trusted_pending: {}, untrusted_pending: {}, confirmed: {} }}",
+            self.immature, self.trusted_pending, self.untrusted_pending, self.confirmed
+        )
+    }
+}
+
+impl std::ops::Add for Balance {
+    type Output = Self;
+
+    fn add(self, other: Self) -> Self {
+        Self {
+            immature: self.immature + other.immature,
+            trusted_pending: self.trusted_pending + other.trusted_pending,
+            untrusted_pending: self.untrusted_pending + other.untrusted_pending,
+            confirmed: self.confirmed + other.confirmed,
+        }
+    }
+}
+
+impl std::iter::Sum for Balance {
+    fn sum<I: Iterator<Item = Self>>(iter: I) -> Self {
+        iter.fold(
+            Balance {
+                ..Default::default()
+            },
+            |a, b| a + b,
+        )
+    }
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;
 
     #[test]
     fn can_store_feerate_in_const() {
-        const _MY_RATE: FeeRate = FeeRate::from_sat_per_vb(10.0);
         const _MIN_RELAY: FeeRate = FeeRate::default_min_relay_fee();
     }
+
+    #[test]
+    #[should_panic]
+    fn test_invalid_feerate_neg_zero() {
+        let _ = FeeRate::from_sat_per_vb(-0.0);
+    }
+
+    #[test]
+    #[should_panic]
+    fn test_invalid_feerate_neg_value() {
+        let _ = FeeRate::from_sat_per_vb(-5.0);
+    }
+
+    #[test]
+    #[should_panic]
+    fn test_invalid_feerate_nan() {
+        let _ = FeeRate::from_sat_per_vb(f32::NAN);
+    }
+
+    #[test]
+    #[should_panic]
+    fn test_invalid_feerate_inf() {
+        let _ = FeeRate::from_sat_per_vb(f32::INFINITY);
+    }
+
+    #[test]
+    fn test_valid_feerate_pos_zero() {
+        let _ = FeeRate::from_sat_per_vb(0.0);
+    }
+
+    #[test]
+    fn test_fee_from_btc_per_kvb() {
+        let fee = FeeRate::from_btc_per_kvb(1e-5);
+        assert!((fee.as_sat_per_vb() - 1.0).abs() < f32::EPSILON);
+    }
+
+    #[test]
+    fn test_fee_from_sat_per_vbyte() {
+        let fee = FeeRate::from_sat_per_vb(1.0);
+        assert!((fee.as_sat_per_vb() - 1.0).abs() < f32::EPSILON);
+    }
+
+    #[test]
+    fn test_fee_default_min_relay_fee() {
+        let fee = FeeRate::default_min_relay_fee();
+        assert!((fee.as_sat_per_vb() - 1.0).abs() < f32::EPSILON);
+    }
+
+    #[test]
+    fn test_fee_from_sat_per_kvb() {
+        let fee = FeeRate::from_sat_per_kvb(1000.0);
+        assert!((fee.as_sat_per_vb() - 1.0).abs() < f32::EPSILON);
+    }
+
+    #[test]
+    fn test_fee_from_sat_per_kwu() {
+        let fee = FeeRate::from_sat_per_kwu(250.0);
+        assert!((fee.as_sat_per_vb() - 1.0).abs() < f32::EPSILON);
+    }
 }

src/wallet/address_validator.rs

@@ -100,6 +100,7 @@ impl std::error::Error for AddressValidatorError {}
 /// validator will be propagated up to the original caller that triggered the address generation.
 ///
 /// For a usage example see [this module](crate::address_validator)'s documentation.
+#[deprecated = "AddressValidator was rarely used. Address validation can occur outside of BDK"]
 pub trait AddressValidator: Send + Sync + fmt::Debug {
     /// Validate or inspect an address
     fn validate(
@@ -120,6 +121,7 @@ mod test {
 
     #[derive(Debug)]
     struct TestValidator;
+    #[allow(deprecated)]
     impl AddressValidator for TestValidator {
         fn validate(
             &self,
@@ -135,6 +137,7 @@ mod test {
     #[should_panic(expected = "InvalidScript")]
     fn test_address_validator_external() {
         let (mut wallet, _, _) = get_funded_wallet(get_test_wpkh());
+        #[allow(deprecated)]
         wallet.add_address_validator(Arc::new(TestValidator));
 
         wallet.get_address(New).unwrap();
@@ -144,6 +147,7 @@ mod test {
     #[should_panic(expected = "InvalidScript")]
     fn test_address_validator_internal() {
         let (mut wallet, descriptors, _) = get_funded_wallet(get_test_wpkh());
+        #[allow(deprecated)]
         wallet.add_address_validator(Arc::new(TestValidator));
 
         let addr = crate::testutils!(@external descriptors, 10);

src/wallet/hardwaresigner.rs

@@ -0,0 +1,64 @@
+// Bitcoin Dev Kit
+// Written in 2020 by Alekos Filini <alekos.filini@gmail.com>
+//
+// Copyright (c) 2020-2021 Bitcoin Dev Kit Developers
+//
+// This file is licensed under the Apache License, Version 2.0 <LICENSE-APACHE
+// or http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option.
+// You may not use this file except in accordance with one or both of these
+// licenses.
+
+//! HWI Signer
+//!
+//! This module contains a simple implementation of a Custom signer for rust-hwi
+
+use bitcoin::psbt::PartiallySignedTransaction;
+use bitcoin::secp256k1::{All, Secp256k1};
+use bitcoin::util::bip32::Fingerprint;
+
+use hwi::error::Error;
+use hwi::types::{HWIChain, HWIDevice};
+use hwi::HWIClient;
+
+use crate::signer::{SignerCommon, SignerError, SignerId, TransactionSigner};
+
+#[derive(Debug)]
+/// Custom signer for Hardware Wallets
+///
+/// This ignores `sign_options` and leaves the decisions up to the hardware wallet.
+pub struct HWISigner {
+    fingerprint: Fingerprint,
+    client: HWIClient,
+}
+
+impl HWISigner {
+    /// Create a instance from the specified device and chain
+    pub fn from_device(device: &HWIDevice, chain: HWIChain) -> Result<HWISigner, Error> {
+        let client = HWIClient::get_client(device, false, chain)?;
+        Ok(HWISigner {
+            fingerprint: device.fingerprint,
+            client,
+        })
+    }
+}
+
+impl SignerCommon for HWISigner {
+    fn id(&self, _secp: &Secp256k1<All>) -> SignerId {
+        SignerId::Fingerprint(self.fingerprint)
+    }
+}
+
+/// This implementation ignores `sign_options`
+impl TransactionSigner for HWISigner {
+    fn sign_transaction(
+        &self,
+        psbt: &mut PartiallySignedTransaction,
+        _sign_options: &crate::SignOptions,
+        _secp: &crate::wallet::utils::SecpCtx,
+    ) -> Result<(), SignerError> {
+        psbt.combine(self.client.sign_tx(psbt)?.psbt)
+            .expect("Failed to combine HW signed psbt with passed PSBT");
+        Ok(())
+    }
+}

src/wallet/signer.rs

@@ -58,6 +58,7 @@
 //!         &self,
 //!         psbt: &mut psbt::PartiallySignedTransaction,
 //!         input_index: usize,
+//!         _sign_options: &SignOptions,
 //!         _secp: &Secp256k1<All>,
 //!     ) -> Result<(), SignerError> {
 //!         self.device.hsm_sign_input(psbt, input_index)?;
@@ -158,6 +159,16 @@ pub enum SignerError {
     InvalidSighash,
     /// Error while computing the hash to sign
     SighashError(sighash::Error),
+    /// Error while signing using hardware wallets
+    #[cfg(feature = "hardware-signer")]
+    HWIError(hwi::error::Error),
+}
+
+#[cfg(feature = "hardware-signer")]
+impl From<hwi::error::Error> for SignerError {
+    fn from(e: hwi::error::Error) -> Self {
+        SignerError::HWIError(e)
+    }
 }
 
 impl From<sighash::Error> for SignerError {
@@ -241,6 +252,7 @@ pub trait InputSigner: SignerCommon {
         &self,
         psbt: &mut psbt::PartiallySignedTransaction,
         input_index: usize,
+        sign_options: &SignOptions,
         secp: &SecpCtx,
     ) -> Result<(), SignerError>;
 }
@@ -254,6 +266,7 @@ pub trait TransactionSigner: SignerCommon {
     fn sign_transaction(
         &self,
         psbt: &mut psbt::PartiallySignedTransaction,
+        sign_options: &SignOptions,
         secp: &SecpCtx,
     ) -> Result<(), SignerError>;
 }
@@ -262,10 +275,11 @@ impl<T: InputSigner> TransactionSigner for T {
     fn sign_transaction(
         &self,
         psbt: &mut psbt::PartiallySignedTransaction,
+        sign_options: &SignOptions,
         secp: &SecpCtx,
     ) -> Result<(), SignerError> {
         for input_index in 0..psbt.inputs.len() {
-            self.sign_input(psbt, input_index, secp)?;
+            self.sign_input(psbt, input_index, sign_options, secp)?;
         }
 
         Ok(())
@@ -287,6 +301,7 @@ impl InputSigner for SignerWrapper<DescriptorXKey<ExtendedPrivKey>> {
         &self,
         psbt: &mut psbt::PartiallySignedTransaction,
         input_index: usize,
+        sign_options: &SignOptions,
         secp: &SecpCtx,
     ) -> Result<(), SignerError> {
         if input_index >= psbt.inputs.len() {
@@ -346,7 +361,7 @@ impl InputSigner for SignerWrapper<DescriptorXKey<ExtendedPrivKey>> {
                 inner: derived_key.private_key,
             };
 
-            SignerWrapper::new(priv_key, self.ctx).sign_input(psbt, input_index, secp)
+            SignerWrapper::new(priv_key, self.ctx).sign_input(psbt, input_index, sign_options, secp)
         }
     }
 }
@@ -369,6 +384,7 @@ impl InputSigner for SignerWrapper<PrivateKey> {
         &self,
         psbt: &mut psbt::PartiallySignedTransaction,
         input_index: usize,
+        sign_options: &SignOptions,
         secp: &SecpCtx,
     ) -> Result<(), SignerError> {
         if input_index >= psbt.inputs.len() || input_index >= psbt.unsigned_tx.input.len() {
@@ -385,7 +401,10 @@ impl InputSigner for SignerWrapper<PrivateKey> {
         let x_only_pubkey = XOnlyPublicKey::from(pubkey.inner);
 
         if let SignerContext::Tap { is_internal_key } = self.ctx {
-            if is_internal_key && psbt.inputs[input_index].tap_key_sig.is_none() {
+            if is_internal_key
+                && psbt.inputs[input_index].tap_key_sig.is_none()
+                && sign_options.sign_with_tap_internal_key
+            {
                 let (hash, hash_ty) = Tap::sighash(psbt, input_index, None)?;
                 sign_psbt_schnorr(
                     &self.inner,
@@ -404,9 +423,18 @@ impl InputSigner for SignerWrapper<PrivateKey> {
                 let leaf_hashes = leaf_hashes
                     .iter()
                     .filter(|lh| {
-                        !psbt.inputs[input_index]
-                            .tap_script_sigs
-                            .contains_key(&(x_only_pubkey, **lh))
+                        // Removing the leaves we shouldn't sign for
+                        let should_sign = match &sign_options.tap_leaves_options {
+                            TapLeavesOptions::All => true,
+                            TapLeavesOptions::Include(v) => v.contains(lh),
+                            TapLeavesOptions::Exclude(v) => !v.contains(lh),
+                            TapLeavesOptions::None => false,
+                        };
+                        // Filtering out the leaves without our key
+                        should_sign
+                            && !psbt.inputs[input_index]
+                                .tap_script_sigs
+                                .contains_key(&(x_only_pubkey, **lh))
                     })
                     .cloned()
                     .collect::<Vec<_>>();
@@ -457,10 +485,10 @@ fn sign_psbt_ecdsa(
     hash_ty: EcdsaSighashType,
     secp: &SecpCtx,
 ) {
-    let sig = secp.sign_ecdsa(
-        &Message::from_slice(&hash.into_inner()[..]).unwrap(),
-        secret_key,
-    );
+    let msg = &Message::from_slice(&hash.into_inner()[..]).unwrap();
+    let sig = secp.sign_ecdsa(msg, secret_key);
+    secp.verify_ecdsa(msg, &sig, &pubkey.inner)
+        .expect("invalid or corrupted ecdsa signature");
 
     let final_signature = ecdsa::EcdsaSig { sig, hash_ty };
     psbt_input.partial_sigs.insert(pubkey, final_signature);
@@ -486,10 +514,10 @@ fn sign_psbt_schnorr(
         Some(_) => keypair, // no tweak for script spend
     };
 
-    let sig = secp.sign_schnorr(
-        &Message::from_slice(&hash.into_inner()[..]).unwrap(),
-        &keypair,
-    );
+    let msg = &Message::from_slice(&hash.into_inner()[..]).unwrap();
+    let sig = secp.sign_schnorr(msg, &keypair);
+    secp.verify_schnorr(&sig, msg, &XOnlyPublicKey::from_keypair(&keypair))
+        .expect("invalid or corrupted schnorr signature");
 
     let final_signature = schnorr::SchnorrSig { sig, hash_ty };
 
@@ -667,6 +695,48 @@ pub struct SignOptions {
     ///
     /// Defaults to `false` which will only allow signing using `SIGHASH_ALL`.
     pub allow_all_sighashes: bool,
+
+    /// Whether to remove partial_sigs from psbt inputs while finalizing psbt.
+    ///
+    /// Defaults to `true` which will remove partial_sigs after finalizing.
+    pub remove_partial_sigs: bool,
+
+    /// Whether to try finalizing psbt input after the inputs are signed.
+    ///
+    /// Defaults to `true` which will try fianlizing psbt after inputs are signed.
+    pub try_finalize: bool,
+
+    /// Specifies which Taproot script-spend leaves we should sign for. This option is
+    /// ignored if we're signing a non-taproot PSBT.
+    ///
+    /// Defaults to All, i.e., the wallet will sign all the leaves it has a key for.
+    pub tap_leaves_options: TapLeavesOptions,
+
+    /// Whether we should try to sign a taproot transaction with the taproot internal key
+    /// or not. This option is ignored if we're signing a non-taproot PSBT.
+    ///
+    /// Defaults to `true`, i.e., we always try to sign with the taproot internal key.
+    pub sign_with_tap_internal_key: bool,
+}
+
+/// Customize which taproot script-path leaves the signer should sign.
+#[derive(Debug, Clone, PartialEq)]
+pub enum TapLeavesOptions {
+    /// The signer will sign all the leaves it has a key for.
+    All,
+    /// The signer won't sign leaves other than the ones specified. Note that it could still ignore
+    /// some of the specified leaves, if it doesn't have the right key to sign them.
+    Include(Vec<taproot::TapLeafHash>),
+    /// The signer won't sign the specified leaves.
+    Exclude(Vec<taproot::TapLeafHash>),
+    /// The signer won't sign any leaf.
+    None,
+}
+
+impl Default for TapLeavesOptions {
+    fn default() -> Self {
+        TapLeavesOptions::All
+    }
 }
 
 #[allow(clippy::derivable_impls)]
@@ -676,6 +746,10 @@ impl Default for SignOptions {
             trust_witness_utxo: false,
             assume_height: None,
             allow_all_sighashes: false,
+            remove_partial_sigs: true,
+            try_finalize: true,
+            tap_leaves_options: TapLeavesOptions::default(),
+            sign_with_tap_internal_key: true,
         }
     }
 }
@@ -1006,6 +1080,7 @@ mod signers_container_tests {
         fn sign_transaction(
             &self,
             _psbt: &mut psbt::PartiallySignedTransaction,
+            _sign_options: &SignOptions,
             _secp: &SecpCtx,
         ) -> Result<(), SignerError> {
             Ok(())

src/wallet/tx_builder.rs

@@ -147,6 +147,8 @@ pub(crate) struct TxParams {
     pub(crate) add_global_xpubs: bool,
     pub(crate) include_output_redeem_witness_script: bool,
     pub(crate) bumping_fee: Option<PreviousFee>,
+    pub(crate) current_height: Option<u32>,
+    pub(crate) allow_dust: bool,
 }
 
 #[derive(Clone, Copy, Debug)]
@@ -515,7 +517,7 @@ impl<'a, D: BatchDatabase, Cs: CoinSelectionAlgorithm<D>, Ctx: TxBuilderContext>
         }
     }
 
-    /// Finish the building the transaction.
+    /// Finish building the transaction.
     ///
     /// Returns the [`BIP174`] "PSBT" and summary details about the transaction.
     ///
@@ -543,6 +545,30 @@ impl<'a, D: BatchDatabase, Cs: CoinSelectionAlgorithm<D>, Ctx: TxBuilderContext>
         self.params.rbf = Some(RbfValue::Value(nsequence));
         self
     }
+
+    /// Set the current blockchain height.
+    ///
+    /// This will be used to:
+    /// 1. Set the nLockTime for preventing fee sniping.
+    /// **Note**: This will be ignored if you manually specify a nlocktime using [`TxBuilder::nlocktime`].
+    /// 2. Decide whether coinbase outputs are mature or not. If the coinbase outputs are not
+    ///    mature at `current_height`, we ignore them in the coin selection.
+    ///    If you want to create a transaction that spends immature coinbase inputs, manually
+    ///    add them using [`TxBuilder::add_utxos`].
+    ///
+    /// In both cases, if you don't provide a current height, we use the last sync height.
+    pub fn current_height(&mut self, height: u32) -> &mut Self {
+        self.params.current_height = Some(height);
+        self
+    }
+
+    /// Set whether or not the dust limit is checked.
+    ///
+    /// **Note**: by avoiding a dust limit check you may end up with a transaction that is non-standard.
+    pub fn allow_dust(&mut self, allow_dust: bool) -> &mut Self {
+        self.params.allow_dust = allow_dust;
+        self
+    }
 }
 
 impl<'a, D: BatchDatabase, Cs: CoinSelectionAlgorithm<D>> TxBuilder<'a, D, Cs, CreateTx> {
@@ -574,6 +600,9 @@ impl<'a, D: BatchDatabase, Cs: CoinSelectionAlgorithm<D>> TxBuilder<'a, D, Cs, C
     /// difference is that it is valid to use `drain_to` without setting any ordinary recipients
     /// with [`add_recipient`] (but it is perfectly fine to add recipients as well).
     ///
+    /// If you choose not to set any recipients, you should either provide the utxos that the
+    /// transaction should spend via [`add_utxos`], or set [`drain_wallet`] to spend all of them.
+    ///
     /// When bumping the fees of a transaction made with this option, you probably want to
     /// use [`allow_shrinking`] to allow this output to be reduced to pay for the extra fees.
     ///
@@ -604,6 +633,7 @@ impl<'a, D: BatchDatabase, Cs: CoinSelectionAlgorithm<D>> TxBuilder<'a, D, Cs, C
     ///
     /// [`allow_shrinking`]: Self::allow_shrinking
     /// [`add_recipient`]: Self::add_recipient
+    /// [`add_utxos`]: Self::add_utxos
     /// [`drain_wallet`]: Self::drain_wallet
     pub fn drain_to(&mut self, script_pubkey: Script) -> &mut Self {
         self.params.drain_to = Some(script_pubkey);

src/wallet/utils.rs

@@ -144,7 +144,6 @@ mod test {
         SEQUENCE_LOCKTIME_TYPE_FLAG,
     };
     use crate::bitcoin::Address;
-    use crate::types::FeeRate;
     use std::str::FromStr;
 
     #[test]
@@ -164,24 +163,6 @@ mod test {
         assert!(!294.is_dust(&script_p2wpkh));
     }
 
-    #[test]
-    fn test_fee_from_btc_per_kb() {
-        let fee = FeeRate::from_btc_per_kvb(1e-5);
-        assert!((fee.as_sat_vb() - 1.0).abs() < 0.0001);
-    }
-
-    #[test]
-    fn test_fee_from_sats_vbyte() {
-        let fee = FeeRate::from_sat_per_vb(1.0);
-        assert!((fee.as_sat_vb() - 1.0).abs() < 0.0001);
-    }
-
-    #[test]
-    fn test_fee_default_min_relay_fee() {
-        let fee = FeeRate::default_min_relay_fee();
-        assert!((fee.as_sat_vb() - 1.0).abs() < 0.0001);
-    }
-
     #[test]
     fn test_check_nsequence_rbf_msb_set() {
         let result = check_nsequence_rbf(0x80000000, 5000);