Bump version to 1.0.0-alpha.1

* `bdk_chain/std` should also enable `miniscript/std`
* use the version of `hashbrown` that `bitcoin` and `miniscript` is
  using

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,26 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: 'bug'
+assignees: ''
+
+---
+
+**Describe the bug**  
+<!-- A clear and concise description of what the bug is. -->
+
+**To Reproduce**  
+<!-- Steps or code to reproduce the behavior. -->
+
+**Expected behavior**  
+<!-- A clear and concise description of what you expected to happen. -->
+
+**Build environment**  
+ - BDK tag/commit: <!-- e.g. v0.13.0, 3a07614 --> 
+ - OS+version: <!-- e.g. ubuntu 20.04.01, macOS 12.0.1, windows -->  
+ - Rust/Cargo version: <!-- e.g. 1.56.0 --> 
+ - Rust/Cargo target: <!-- e.g. x86_64-apple-darwin, x86_64-unknown-linux-gnu, etc. -->  
+
+**Additional context**  
+<!-- Add any other context about the problem here. --> 

.github/ISSUE_TEMPLATE/enhancement_request.md

@@ -0,0 +1,17 @@
+---
+name: Enhancement request
+about: Request a new feature or change to an existing feature
+title: ''
+labels: 'enhancement'
+assignees: ''
+
+---
+
+**Describe the enhancement**  
+<!-- A clear and concise description of what you would like added or changed. -->
+
+**Use case**  
+<!-- Tell us how you or others will use this new feature or change to an existing feature. --> 
+
+**Additional context**
+<!-- Add any other context about the enhancement here. --> 

.github/ISSUE_TEMPLATE/minor_release.md

@@ -0,0 +1,99 @@
+---
+name: Minor Release
+about: Create a new minor release [for release managers only]
+title: 'Release MAJOR.MINOR+1.0'
+labels: 'release'
+assignees: ''
+
+---
+
+## Create a new minor release
+
+### Summary
+
+<--release summary to be used in announcements-->
+
+### Commit
+
+<--latest commit ID to include in this release-->
+
+### Changelog
+
+<--add notices from PRs merged since the prior release, see ["keep a changelog"]-->
+
+### Checklist
+
+Release numbering must follow [Semantic Versioning]. These steps assume the current `master`
+branch **development** version is *MAJOR.MINOR.0*.
+
+#### On the day of the feature freeze
+
+Change the `master` branch to the next MINOR+1 version:
+
+- [ ] Switch to the `master` branch.
+- [ ] Create a new PR branch called `bump_dev_MAJOR_MINOR+1`, eg. `bump_dev_0_22`.
+- [ ] Bump the `bump_dev_MAJOR_MINOR+1` branch to the next development MINOR+1 version.
+  - Change the `Cargo.toml` version value to `MAJOR.MINOR+1.0`.
+  - Update the `CHANGELOG.md` file.
+  - The commit message should be "Bump version to MAJOR.MINOR+1.0".
+- [ ] Create PR and merge the `bump_dev_MAJOR_MINOR+1` branch to `master`.
+  - Title PR "Bump version to MAJOR.MINOR+1.0".
+
+Create a new release branch and release candidate tag:
+
+- [ ] Double check that your local `master` is up-to-date with the upstream repo.
+- [ ] Create a new branch called `release/MAJOR.MINOR+1` from `master`.
+- [ ] Bump the `release/MAJOR.MINOR+1` branch to `MAJOR.MINOR+1.0-rc.1` version.
+  - Change the `Cargo.toml` version value to `MAJOR.MINOR+1.0-rc.1`.
+  - The commit message should be "Bump version to MAJOR.MINOR+1.0-rc.1".
+- [ ] Add a tag to the `HEAD` commit in the `release/MAJOR.MINOR+1` branch.
+  - The tag name should be `vMAJOR.MINOR+1.0-rc.1`
+  - Use message "Release MAJOR.MINOR+1.0 rc.1".
+  - Make sure the tag is signed, for extra safety use the explicit `--sign` flag.
+- [ ] Push the `release/MAJOR.MINOR` branch and new tag to the `bitcoindevkit/bdk` repo.
+  - Use `git push --tags` option to push the new `vMAJOR.MINOR+1.0-rc.1` tag.
+
+If any issues need to be fixed before the *MAJOR.MINOR+1.0* version is released:
+
+- [ ] Merge fix PRs to the `master` branch.
+- [ ] Git cherry-pick fix commits to the `release/MAJOR.MINOR+1` branch.
+- [ ] Verify fixes in `release/MAJOR.MINOR+1` branch.
+- [ ] Bump the `release/MAJOR.MINOR+1` branch to `MAJOR.MINOR+1.0-rc.x+1` version.
+  - Change the `Cargo.toml` version value to `MAJOR.MINOR+1.0-rc.x+1`.
+  - The commit message should be "Bump version to MAJOR.MINOR+1.0-rc.x+1".
+- [ ] Add a tag to the `HEAD` commit in the `release/MAJOR.MINOR+1` branch.
+  - The tag name should be `vMAJOR.MINOR+1.0-rc.x+1`, where x is the current release candidate number.
+  - Use tag message "Release MAJOR.MINOR+1.0 rc.x+1".
+  - Make sure the tag is signed, for extra safety use the explicit `--sign` flag.
+- [ ] Push the new tag to the `bitcoindevkit/bdk` repo.
+  - Use `git push --tags` option to push the new `vMAJOR.MINOR+1.0-rc.x+1` tag.
+
+#### On the day of the release
+
+Tag and publish new release:
+
+- [ ] Bump the `release/MAJOR.MINOR+1` branch to `MAJOR.MINOR+1.0` version.
+  - Change the `Cargo.toml` version value to `MAJOR.MINOR+1.0`.
+  - The commit message should be "Bump version to MAJOR.MINOR+1.0".
+- [ ] Add a tag to the `HEAD` commit in the `release/MAJOR.MINOR+1` branch.
+  - The tag name should be `vMAJOR.MINOR+1.0`
+  - The first line of the tag message should be "Release MAJOR.MINOR+1.0".
+  - In the body of the tag message put a copy of the **Summary** and **Changelog** for the release.
+  - Make sure the tag is signed, for extra safety use the explicit `--sign` flag.
+- [ ] Wait for the CI to finish one last time.
+- [ ] Push the new tag to the `bitcoindevkit/bdk` repo.
+- [ ] Publish **all** the updated crates to crates.io.
+- [ ] Create the release on GitHub.
+  - Go to "tags", click on the dots on the right and select "Create Release".
+  - Set the title to `Release MAJOR.MINOR+1.0`.
+  - In the release notes body put the **Summary** and **Changelog**.
+  - Use the "+ Auto-generate release notes" button to add details from included PRs.
+  - Until we reach a `1.0.0` release check the "Pre-release" box.
+- [ ] Make sure the new release shows up on [crates.io] and that the docs are built correctly on [docs.rs].
+- [ ] Announce the release, using the **Summary**, on Discord, Twitter and Mastodon.
+- [ ] Celebrate 🎉
+
+[Semantic Versioning]: https://semver.org/
+[crates.io]: https://crates.io/crates/bdk
+[docs.rs]: https://docs.rs/bdk/latest/bdk
+["keep a changelog"]: https://keepachangelog.com/en/1.0.0/

.github/ISSUE_TEMPLATE/patch_release.md

@@ -0,0 +1,71 @@
+---
+name: Patch Release
+about: Create a new patch release [for release managers only]
+title: 'Release MAJOR.MINOR.PATCH+1'
+labels: 'release'
+assignees: ''
+
+---
+
+## Create a new patch release
+
+### Summary
+
+<--release summary to be used in announcements-->
+
+### Commit
+
+<--latest commit ID to include in this release-->
+
+### Changelog
+
+<--add notices from PRs merged since the prior release, see ["keep a changelog"]-->
+
+### Checklist
+
+Release numbering must follow [Semantic Versioning]. These steps assume the current `master`
+branch **development** version is *MAJOR.MINOR.PATCH*.
+
+### On the day of the patch release
+
+Change the `master` branch to the new PATCH+1 version:
+
+- [ ] Switch to the `master` branch.
+- [ ] Create a new PR branch called `bump_dev_MAJOR_MINOR_PATCH+1`, eg. `bump_dev_0_22_1`.
+- [ ] Bump the `bump_dev_MAJOR_MINOR` branch to the next development PATCH+1 version.
+  - Change the `Cargo.toml` version value to `MAJOR.MINOR.PATCH+1`.
+  - Update the `CHANGELOG.md` file.
+  - The commit message should be "Bump version to MAJOR.MINOR.PATCH+1".
+- [ ] Create PR and merge the `bump_dev_MAJOR_MINOR_PATCH+1` branch to `master`.
+  - Title PR "Bump version to MAJOR.MINOR.PATCH+1".
+
+Cherry-pick, tag and publish new PATCH+1 release:
+
+- [ ] Merge fix PRs to the `master` branch.
+- [ ] Git cherry-pick fix commits to the `release/MAJOR.MINOR` branch to be patched.
+- [ ] Verify fixes in `release/MAJOR.MINOR` branch.
+- [ ] Bump the `release/MAJOR.MINOR.PATCH+1` branch to `MAJOR.MINOR.PATCH+1` version.
+  - Change the `Cargo.toml` version value to `MAJOR.MINOR.MINOR.PATCH+1`.
+  - The commit message should be "Bump version to MAJOR.MINOR.PATCH+1".
+- [ ] Add a tag to the `HEAD` commit in the `release/MAJOR.MINOR` branch.
+  - The tag name should be `vMAJOR.MINOR.PATCH+1`
+  - The first line of the tag message should be "Release MAJOR.MINOR.PATCH+1".
+  - In the body of the tag message put a copy of the **Summary** and **Changelog** for the release.
+  - Make sure the tag is signed, for extra safety use the explicit `--sign` flag.
+- [ ] Wait for the CI to finish one last time.
+- [ ] Push the new tag to the `bitcoindevkit/bdk` repo.
+- [ ] Publish **all** the updated crates to crates.io.
+- [ ] Create the release on GitHub.
+  - Go to "tags", click on the dots on the right and select "Create Release".
+  - Set the title to `Release MAJOR.MINOR.PATCH+1`.
+  - In the release notes body put the **Summary** and **Changelog**.
+  - Use the "+ Auto-generate release notes" button to add details from included PRs.
+  - Until we reach a `1.0.0` release check the "Pre-release" box.
+- [ ] Make sure the new release shows up on [crates.io] and that the docs are built correctly on [docs.rs].
+- [ ] Announce the release, using the **Summary**, on Discord, Twitter and Mastodon.
+- [ ] Celebrate 🎉
+
+[Semantic Versioning]: https://semver.org/
+[crates.io]: https://crates.io/crates/bdk
+[docs.rs]: https://docs.rs/bdk/latest/bdk
+["keep a changelog"]: https://keepachangelog.com/en/1.0.0/

.github/ISSUE_TEMPLATE/summer_project.md

@@ -0,0 +1,77 @@
+---
+name: Summer of Bitcoin Project
+about: Template to suggest a new https://www.summerofbitcoin.org/ project.
+title: ''
+labels: 'summer-of-bitcoin'
+assignees: ''
+
+---
+
+<!--
+## Overview  
+
+Project ideas are scoped for a university-level student with a basic background in CS and bitcoin 
+fundamentals - achievable over 12-weeks. Below are just a few types of ideas: 
+
+ - Low-hanging fruit: Relatively short projects with clear goals; requires basic technical knowledge 
+   and minimal familiarity with the codebase.
+ - Core development: These projects derive from the ongoing work from the core of your development 
+   team. The list of features and bugs is never-ending, and help is always welcome.
+ - Risky/Exploratory: These projects push the scope boundaries of your development effort. They 
+   might require expertise in an area not covered by your current development team. They might take 
+   advantage of a new technology. There is a reasonable chance that the project might be less 
+   successful, but the potential rewards make it worth the attempt.
+ - Infrastructure/Automation: These projects are the code that your organization uses to get its 
+   development work done; for example, projects that improve the automation of releases, regression 
+   tests and automated builds. This is a category where a Summer of Bitcoin student can be really 
+   helpful, doing work that the development team has been putting off while they focus on core 
+   development.
+ - Quality Assurance/Testing: Projects that work on and test your project's software development 
+   process. Additionally, projects that involve a thorough test and review of individual PRs.
+ - Fun/Peripheral: These projects might not be related to the current core development focus, but 
+   create new innovations and new perspectives for your project.
+-->
+
+**Description**
+<!-- Description: 3-7 sentences describing the project background and tasks to be done. -->  
+
+**Expected Outcomes**  
+<!-- Short bullet list describing what is to be accomplished -->   
+
+**Resources**  
+<!-- 2-3 reading materials for candidate to learn about the repo, project, scope etc -->  
+<!-- Recommended reading such as a developer/contributor guide -->  
+<!-- [Another example a paper citation](https://arxiv.org/pdf/1802.08091.pdf) -->  
+<!-- [Another example an existing issue](https://github.com/opencv/opencv/issues/11013) -->  
+<!-- [An existing related module](https://github.com/opencv/opencv_contrib/tree/master/modules/optflow) -->  
+
+**Skills Required**  
+<!-- 3-4 technical skills that the candidate should know -->  
+<!-- hands on experience with git -->  
+<!-- mastery plus experience coding in C++ -->  
+<!-- basic knowledge in matrix and tensor computations, college course work in cryptography -->  
+<!-- strong mathematical background -->
+<!-- Bonus - has experience with React Native. Best if you have also worked with OSSFuzz -->
+
+**Mentor(s)**  
+<!-- names of mentor(s) for this project go here -->
+
+**Difficulty**  
+<!-- Easy, Medium, Hard -->
+
+**Competency Test (optional)**  
+<!-- 2-3 technical tasks related to the project idea or repository you’d like a candidate to 
+    perform in order to demonstrate competency, good first bugs, warm-up exercises -->
+<!-- ex. Read the instructions here to get Bitcoin core running on your machine -->
+<!-- ex. pick an issue labeled as “newcomer” in the repository, and send a merge request to the 
+    repository. You can also suggest some other improvement that we did not think of yet, or 
+    something that you find interesting or useful -->
+<!-- ex. fixes for coding style are usually easy to do, and are good issues for first time 
+    contributions for those learning how to interact with the project. After you are done with the 
+    coding style issue, try making a different contribution. -->
+<!-- ex. setup a full Debian packaging development environment and learn the basics of Debian 
+    packaging. Then identify and package the missing dependencies to package Specter Desktop -->
+<!-- ex. write a pull parser for CSV files. You'll be judged by the decisions to store the parser 
+    state and how flexible it is to wrap this parser in other scenarios. -->
+<!-- ex. Stretch Goal: Implement some basic metaprogram/app to prove you're very familiar with BDK. 
+    Be prepared to make adjustments as we judge your solution. -->

.github/pull_request_template.md

@@ -0,0 +1,34 @@
+<!-- You can erase any parts of this template not applicable to your Pull Request. -->
+
+### Description
+
+<!-- Describe the purpose of this PR, what's being adding and/or fixed -->
+
+### Notes to the reviewers
+
+<!-- In this section you can include notes directed to the reviewers, like explaining why some parts
+of the PR were done in a specific way -->
+
+### Changelog notice
+
+<!-- Notice the release manager should include in the release tag message changelog -->
+<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
+
+### Checklists
+
+#### All Submissions:
+
+* [ ] I've signed all my commits
+* [ ] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
+* [ ] I ran `cargo fmt` and `cargo clippy` before committing
+
+#### New Features:
+
+* [ ] I've added tests for the new feature
+* [ ] I've added docs for the new feature
+
+#### Bugfixes:
+
+* [ ] This pull request breaks the existing API
+* [ ] I've added tests to reproduce the issue which are now passing
+* [ ] I'm linking the issue being fixed by this PR

.github/workflows/audit.yml

@@ -0,0 +1,19 @@
+name: Audit
+
+on:
+  push:
+    paths:
+      - '**/Cargo.toml'
+      - '**/Cargo.lock'
+  schedule:
+    - cron: '0 0 * * 0' # Once per week
+
+jobs:
+
+  security_audit:
+    runs-on: ubuntu-20.04
+    steps:
+      - uses: actions/checkout@v2
+      - uses: actions-rs/audit-check@v1
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/code_coverage.yml

@@ -0,0 +1,56 @@
+on: [push, pull_request]
+
+name: Code Coverage
+
+jobs:
+  Codecov:
+    name: Code Coverage
+    runs-on: ubuntu-latest
+    env:
+      RUSTFLAGS: "-Cinstrument-coverage"
+      RUSTDOCFLAGS: "-Cinstrument-coverage"
+      LLVM_PROFILE_FILE: "./target/coverage/%p-%m.profraw"
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v2
+      - name: Install lcov tools
+        run: sudo apt-get install lcov -y
+      - name: Install Rust toolchain
+        uses: actions-rs/toolchain@v1
+        with:
+            toolchain: "1.65.0"
+            override: true
+            profile: minimal
+            components: llvm-tools-preview
+      - name: Rust Cache
+        uses: Swatinem/rust-cache@v2.2.1
+      - name: Install grcov
+        run: if [[ ! -e ~/.cargo/bin/grcov ]]; then cargo install grcov; fi
+      - name: Build simulator image
+        run: docker build -t hwi/ledger_emulator ./ci -f ci/Dockerfile.ledger
+      - name: Run simulator image
+        run: docker run --name simulator --network=host hwi/ledger_emulator &
+      - name: Install Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.9'
+      - name: Install python dependencies
+        run: pip install hwi==2.1.1 protobuf==3.20.1
+      - name: Test
+        run: cargo test --all-features
+      - name: Make coverage directory
+        run: mkdir coverage
+      - name: Run grcov
+        run: grcov . --binary-path ./target/debug/ -s . -t lcov --branch --ignore-not-existing --keep-only '**/crates/**' --ignore '**/tests/**' --ignore '**/examples/**' -o ./coverage/lcov.info
+      - name: Generate HTML coverage report
+        run: genhtml -o coverage-report.html --ignore-errors source ./coverage/lcov.info
+      - name: Coveralls upload
+        uses: coverallsapp/github-action@master
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+      - name: Upload artifact
+        uses: actions/upload-artifact@v2
+        with:
+          name: coverage-report
+          path: coverage-report.html

.github/workflows/cont_integration.yml

@@ -0,0 +1,125 @@
+on: [push, pull_request]
+
+name: CI
+
+jobs:
+
+  build-test:
+    name: Build and test
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        rust:
+          - version: stable
+            clippy: true
+          - version: 1.57.0 # MSRV
+        features:
+          - --no-default-features
+          - --all-features
+    steps:
+      - name: checkout
+        uses: actions/checkout@v2
+      - name: Install Rust toolchain
+        uses: actions-rs/toolchain@v1
+        with:
+            toolchain: ${{ matrix.rust.version }}
+            override: true
+            profile: minimal
+      - name: Rust Cache
+        uses: Swatinem/rust-cache@v2.2.1
+      - name: Build
+        run: cargo build ${{ matrix.features }}
+      - name: Test
+        run: cargo test ${{ matrix.features }}
+
+  check-no-std:
+    name: Check no_std
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v2
+      - name: Install Rust toolchain
+        uses: actions-rs/toolchain@v1
+        with:
+          toolchain: stable
+          override: true
+          profile: minimal
+          # target: "thumbv6m-none-eabi"
+      - name: Rust Cache
+        uses: Swatinem/rust-cache@v2.2.1
+      - name: Check bdk_chain
+        working-directory: ./crates/chain
+        # TODO "--target thumbv6m-none-eabi" should work but currently does not
+        run: cargo check --no-default-features --features bitcoin/no-std,miniscript/no-std,hashbrown
+      - name: Check bdk
+        working-directory: ./crates/bdk
+        # TODO "--target thumbv6m-none-eabi" should work but currently does not
+        run: cargo check --no-default-features --features bitcoin/no-std,miniscript/no-std,bdk_chain/hashbrown
+      - name: Check esplora
+        working-directory: ./crates/esplora
+        # TODO "--target thumbv6m-none-eabi" should work but currently does not
+        run: cargo check --no-default-features --features bitcoin/no-std,miniscript/no-std,bdk_chain/hashbrown
+
+  check-wasm:
+    name: Check WASM
+    runs-on: ubuntu-20.04
+    env:
+      CC: clang-10
+      CFLAGS: -I/usr/include
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v2
+        # Install a recent version of clang that supports wasm32
+      - run: wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add - || exit 1
+      - run: sudo apt-add-repository "deb http://apt.llvm.org/focal/ llvm-toolchain-focal-10 main" || exit 1
+      - run: sudo apt-get update || exit 1
+      - run: sudo apt-get install -y libclang-common-10-dev clang-10 libc6-dev-i386 || exit 1
+      - name: Install Rust toolchain
+        uses: actions-rs/toolchain@v1
+        with:
+            toolchain: stable
+            override: true
+            profile: minimal
+            target: "wasm32-unknown-unknown"
+      - name: Rust Cache
+        uses: Swatinem/rust-cache@v2.2.1
+      - name: Check bdk
+        working-directory: ./crates/bdk
+        run: cargo check --target wasm32-unknown-unknown --no-default-features --features bitcoin/no-std,miniscript/no-std,bdk_chain/hashbrown,dev-getrandom-wasm
+      - name: Check esplora
+        working-directory: ./crates/esplora
+        run: cargo check --target wasm32-unknown-unknown --no-default-features --features bitcoin/no-std,miniscript/no-std,bdk_chain/hashbrown,async
+
+  fmt:
+    name: Rust fmt
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v2
+      - name: Install Rust toolchain
+        uses: actions-rs/toolchain@v1
+        with:
+            toolchain: stable
+            override: true
+            profile: minimal
+            components: rustfmt
+      - name: Check fmt
+        run: cargo fmt --all -- --config format_code_in_doc_comments=true --check
+
+  clippy_check:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v1
+      - uses: actions-rs/toolchain@v1
+        with:
+            # we pin clippy instead of using "stable" so that our CI doesn't break
+            # at each new cargo release
+            toolchain: "1.67.0"
+            components: clippy
+            override: true
+      - name: Rust Cache
+        uses: Swatinem/rust-cache@v2.2.1
+      - uses: actions-rs/clippy-check@v1
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+          args: --all-features --all-targets -- -D warnings

.github/workflows/nightly_docs.yml

@@ -0,0 +1,57 @@
+name: Publish Nightly Docs
+
+on: [push, pull_request]
+
+jobs:
+  build_docs:
+    name: Build docs
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout sources
+        uses: actions/checkout@v2
+      - name: Set default toolchain
+        run: rustup default nightly-2022-12-14
+      - name: Set profile
+        run: rustup set profile minimal
+      - name: Update toolchain
+        run: rustup update
+      - name: Rust Cache
+        uses: Swatinem/rust-cache@v2.2.1
+      - name: Build docs
+        run: cargo doc --no-deps
+        env:
+          RUSTDOCFLAGS: '--cfg docsrs -Dwarnings'
+      - name: Upload artifact
+        uses: actions/upload-artifact@v2
+        with:
+          name: built-docs
+          path: ./target/doc/*
+
+  publish_docs:
+    name: 'Publish docs'
+    if: github.ref == 'refs/heads/master'
+    needs: [build_docs]
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout `bitcoindevkit.org`
+        uses: actions/checkout@v2
+        with:
+          ssh-key: ${{ secrets.DOCS_PUSH_SSH_KEY }}
+          repository: bitcoindevkit/bitcoindevkit.org
+          ref: master
+      - name: Create directories
+        run: mkdir -p ./docs/.vuepress/public/docs-rs/bdk/nightly
+      - name: Remove old latest
+        run: rm -rf ./docs/.vuepress/public/docs-rs/bdk/nightly/latest
+      - name: Download built docs
+        uses: actions/download-artifact@v1
+        with:
+          name: built-docs
+          path: ./docs/.vuepress/public/docs-rs/bdk/nightly/latest
+      - name: Configure git
+        run: git config user.email "github-actions@github.com" && git config user.name "github-actions"
+      - name: Commit
+        continue-on-error: true         # If there's nothing to commit this step fails, but it's fine
+        run: git add ./docs/.vuepress/public/docs-rs && git commit -m "Publish autogenerated nightly docs"
+      - name: Push
+        run: git push origin master

.gitignore

@@ -1,4 +1,6 @@
 /target
 Cargo.lock
+/.vscode
 
 *.swp
+.idea

.travis.yml

@@ -1,22 +0,0 @@
-language: rust
-rust:
-  - stable
-#  - 1.31.0
-#  - 1.22.0
-before_script:
-  - rustup component add rustfmt
-script:
-  - cargo fmt -- --check --verbose
-  - cargo test --verbose --all
-  - cargo build --verbose --all
-  - cargo build --verbose --no-default-features --features=minimal
-  - cargo build --verbose --no-default-features --features=minimal,esplora
-  - cargo build --verbose --no-default-features --features=key-value-db
-  - cargo build --verbose --no-default-features --features=electrum
-
-notifications:
-  email: false
-
-before_cache:
-  - rm -rf "$TRAVIS_HOME/.cargo/registry/src"
-cache: cargo

CHANGELOG.md

@@ -0,0 +1,645 @@
+# Changelog
+
+All notable changes to this project can be found here and in each release's git tag and can be viewed with `git tag -ln100 "v*"`. See also [DEVELOPMENT_CYCLE.md](DEVELOPMENT_CYCLE.md) for more details.
+
+Contributors do not need to change this file but do need to add changelog details in their PR descriptions. The person making the next release will collect changelog details from included PRs and edit this file prior to each release.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [v0.27.1]
+
+### Summary
+
+Fixes [RUSTSEC-2022-0090], this issue is only applicable if you are using the optional sqlite database feature.
+
+[RUSTSEC-2022-0090]: https://rustsec.org/advisories/RUSTSEC-2022-0090
+
+### Changed
+
+- Update optional sqlite dependency from 0.27.0 to 0.28.0. #867
+
+## [v0.27.0]
+
+### Summary
+
+A maintenance release with a bump in project MSRV to 1.57.0, updated dependence and a few developer oriented improvements. Improvements include  better error formatting, don't default to async/await for wasm32 and adding derived PartialEq and Eq on SyncTime.
+
+### Changed
+
+- Improve display error formatting #814
+- Don't default to use async/await on wasm32 #831
+- Project MSRV changed from 1.56.1 to 1.57.0 #842
+- Update rust-miniscript dependency to latest bug fix release 9.0 #844
+
+### Added
+
+- Derive PartialEq, Eq on SyncTime #837
+
+## [v0.26.0]
+
+### Summary
+
+This release improves Fulcrum electrum server compatibility and fixes public descriptor template key origin paths. We also snuck in small enhancements to configure the electrum client to validate the domain using SSL and sort TransactionDetails by block height and timestamp.
+  
+### Fixed
+  
+- Make electrum blockchain client `save_tx` function order independent to work with Fulcrum servers. #808
+- Fix wrong testnet key origin path in public descriptor templates. #818
+- Make README.md code examples compile without errors. #820
+
+### Changed
+
+- Bump `hwi` dependency to `0.4.0`. #825
+- Bump `esplora-client` dependency to `0.3` #830
+
+### Added
+  
+- For electrum blockchain client, allow user to configure whether to validate the domain using SSL. #805
+- Implement ordering for `TransactionDetails`. #812
+
+## [v0.25.0]
+
+### Summary
+
+This release fixes slow sync time and big script_pubkeys table with SQLite, the wallet rescan height for the FullyNodedExport and setting the network for keys in the KeyMap when using descriptor templates. Also added are new blockchain and mnemonic examples.
+  
+### Fixed
+  
+- Slow sync time and big script_pubkeys table with SQLite.
+- Wallet rescan height for the FullyNodedExport.
+- Setting the network for keys in the KeyMap when using descriptor templates.
+  
+### Added
+  
+- Examples for connecting to Esplora, Electrum Server, Neutrino and Bitcoin Core.
+- Example for using a mnemonic in a descriptors.
+
+## [v0.24.0]
+
+### Summary
+
+This release contains important dependency updates for `rust-bitcoin` to `0.29` and `rust-miniscript` to `8.0`, plus related crates that also depend on the latest version of `rust-bitcoin`. The release also includes a breaking change to the BDK signer which now produces low-R signatures by default, saving one byte. A bug was found in the `get_checksum` and `get_checksum_bytes` functions, which are now deprecated in favor of fixed versions called `calc_checksum` and `calc_checksum_bytes`. And finally a new `hardware-signer` features was added that re-exports the `hwi` crate, along with a new `hardware_signers.rs` example file.
+   
+### Changed
+
+- Updated dependency versions for `rust-bitcoin` to `0.29` and `rust-miniscript` to `8.0`, plus all related crates. @afilini #770
+- BDK Signer now produces low-R signatures by default, saving one byte. If you want to preserve the original behavior, set allow_grinding in the SignOptions to false. @vladimirfomene #779
+- Deprecated `get_checksum`and `get_checksum_bytes` due to bug where they calculates the checksum of a descriptor that already has a checksum.  Use `calc_checksum` and `calc_checksum_bytes` instead. @evanlinjin #765
+- Remove deprecated "address validators". @afilini #770
+  
+### Added
+
+- New `calc_checksum` and `calc_checksum_bytes`, replace deprecated `get_checksum` and `get_checksum_bytes`. @evanlinjin #765
+- Re-export the hwi crate when the feature hardware-signer is on.  @danielabrozzoni #758
+- New examples/hardware_signer.rs. @danielabrozzoni #758
+- Make psbt module public to expose PsbtUtils trait to downstream projects. @notmandatory #782
+
+## [v0.23.0]
+
+### Summary
+
+This release brings new utilities functions on PSBTs like `fee_amount()` and `fee_rate()` and migrates BDK to use our new external esplora client library.
+As always many bug fixes, docs and tests improvement are also included.
+
+### Changed
+
+- Update electrum-client to 0.11.0 by @afilini in https://github.com/bitcoindevkit/bdk/pull/737
+- Change configs for source-base code coverage by @wszdexdrf in https://github.com/bitcoindevkit/bdk/pull/708
+- Improve docs regarding PSBT finalization by @tnull in https://github.com/bitcoindevkit/bdk/pull/753
+- Update compiler example to a Policy example by @rajarshimaitra in https://github.com/bitcoindevkit/bdk/pull/730
+- Fix the release process by @afilini in https://github.com/bitcoindevkit/bdk/pull/754
+- Remove redundant duplicated keys check by @afilini in https://github.com/bitcoindevkit/bdk/pull/761
+- Remove genesis_block lazy initialization by @shobitb in https://github.com/bitcoindevkit/bdk/pull/756
+- Fix `Wallet::descriptor_checksum` to actually return the checksum by @evanlinjin in https://github.com/bitcoindevkit/bdk/pull/763
+- Use the esplora client crate by @afilini in https://github.com/bitcoindevkit/bdk/pull/764
+
+### Added
+
+- Run code coverage on every PR by @danielabrozzoni in https://github.com/bitcoindevkit/bdk/pull/747
+- Add psbt_signer.rs example by @notmandatory in https://github.com/bitcoindevkit/bdk/pull/744
+- Add fee_amount() and fee_rate() functions to PsbtUtils trait by @notmandatory in https://github.com/bitcoindevkit/bdk/pull/728
+- Add tests to improve coverage by @vladimirfomene in https://github.com/bitcoindevkit/bdk/pull/745
+- Enable signing taproot transactions with only `non_witness_utxos` by @afilini in https://github.com/bitcoindevkit/bdk/pull/757
+- Add datatype for is_spent sqlite column by @vladimirfomene in https://github.com/bitcoindevkit/bdk/pull/713
+- Add vscode filter to gitignore by @evanlinjin in https://github.com/bitcoindevkit/bdk/pull/762
+
+## [v0.22.0]
+
+### Summary
+
+This release brings support for hardware signers on desktop through the HWI library.
+It also includes fixes and improvements which are part of our ongoing effort of integrating
+BDK and LDK together.
+
+### Changed
+
+- FeeRate function name as_sat_vb to as_sat_per_vb. #678
+- Verify signatures after signing. #718
+- Dependency electrum-client to 0.11.0. #737
+
+### Added
+  
+- Functions to create FeeRate from sats/kvbytes and sats/kwu. #678
+- Custom hardware wallet signer HwiSigner in wallet::hardwaresigner module. #682
+- Function allow_dust on TxBuilder. #689
+- Implementation of Deref<Target=UrlClient> for EsploraBlockchain. #722
+- Implementation of Deref<Target=Client> for ElectrumBlockchain #705
+- Implementation of Deref<Target=Client> for RpcBlockchain. #731
+
+## [v0.21.0]
+
+- Add `descriptor::checksum::get_checksum_bytes` method.
+- Add `Excess` enum to handle remaining amount after coin selection.
+- Move change creation from `Wallet::create_tx` to `CoinSelectionAlgorithm::coin_select`.
+- Change the interface of `SqliteDatabase::new` to accept any type that implement AsRef<Path>
+- Add the ability to specify which leaves to sign in a taproot transaction through `TapLeavesOptions` in `SignOptions`
+- Add the ability to specify whether a taproot transaction should be signed using the internal key or not, using `sign_with_tap_internal_key` in `SignOptions`
+- Consolidate params `fee_amount` and `amount_needed` in `target_amount` in `CoinSelectionAlgorithm::coin_select` signature.
+- Change the meaning of the `fee_amount` field inside `CoinSelectionResult`: from now on the `fee_amount` will represent only the fees asociated with the utxos in the `selected` field of `CoinSelectionResult`.
+- New `RpcBlockchain` implementation with various fixes.
+- Return balance in separate categories, namely `confirmed`, `trusted_pending`, `untrusted_pending` & `immature`.
+
+## [v0.20.0]
+
+- New MSRV set to `1.56.1`
+- Fee sniping discouraging through nLockTime - if the user specifies a `current_height`, we use that as a nlocktime, otherwise we use the last sync height (or 0 if we never synced)
+- Fix hang when `ElectrumBlockchainConfig::stop_gap` is zero.
+- Set coin type in BIP44, BIP49, and BIP84 templates
+- Get block hash given a block height - A `get_block_hash` method is now defined on the `GetBlockHash` trait and implemented on every blockchain backend. This method expects a block height and returns the corresponding block hash.
+- Add `remove_partial_sigs` and `try_finalize` to `SignOptions`
+- Deprecate `AddressValidator`
+- Fix Electrum wallet sync potentially causing address index decrement - compare proposed index and current index before applying batch operations during sync.
+
+## [v0.19.0]
+
+- added `OldestFirstCoinSelection` impl to `CoinSelectionAlgorithm`
+- New MSRV set to `1.56`
+- Unpinned tokio to `1`
+- Add traits to reuse `Blockchain`s across multiple wallets (`BlockchainFactory` and `StatelessBlockchain`).
+- Upgrade to rust-bitcoin `0.28`
+- If using the `sqlite-db` feature all cached wallet data is deleted due to a possible UTXO inconsistency, a wallet.sync will recreate it
+- Update `PkOrF` in the policy module to become an enum
+- Add experimental support for Taproot, including:
+  - Support for `tr()` descriptors with complex tapscript trees
+  - Creation of Taproot PSBTs (BIP-371)
+  - Signing Taproot PSBTs (key spend and script spend)
+  - Support for `tr()` descriptors in the `descriptor!()` macro
+- Add support for Bitcoin Core 23.0 when using the `rpc` blockchain
+
+## [v0.18.0]
+
+- Add `sqlite-bundled` feature for deployments that need a bundled version of sqlite, i.e. for mobile platforms.
+- Added `Wallet::get_signers()`, `Wallet::descriptor_checksum()` and `Wallet::get_address_validators()`, exposed the `AsDerived` trait.
+- Deprecate `database::Database::flush()`, the function is only needed for the sled database on mobile, instead for mobile use the sqlite database.
+- Add `keychain: KeychainKind` to `wallet::AddressInfo`.
+- Improve key generation traits
+- Rename `WalletExport` to `FullyNodedExport`, deprecate the former.
+- Bump `miniscript` dependency version to `^6.1`.
+
+## [v0.17.0]
+
+- Removed default verification from `wallet::sync`. sync-time verification is added in `script_sync` and is activated by `verify` feature flag.
+- `verify` flag removed from `TransactionDetails`.
+- Add `get_internal_address` to allow you to get internal addresses just as you get external addresses.
+- added `ensure_addresses_cached` to `Wallet` to let offline wallets load and cache addresses in their database
+- Add `is_spent` field to `LocalUtxo`; when we notice that a utxo has been spent we set `is_spent` field to true instead of deleting it from the db.
+
+### Sync API change
+
+To decouple the `Wallet` from the `Blockchain` we've made major changes:
+
+- Removed `Blockchain` from Wallet.
+- Removed `Wallet::broadcast` (just use `Blockchain::broadcast`)
+- Deprecated `Wallet::new_offline` (all wallets are offline now)
+- Changed `Wallet::sync` to take a `Blockchain`.
+- Stop making a request for the block height when calling `Wallet:new`.
+- Added `SyncOptions` to capture extra (future) arguments to `Wallet::sync`.
+- Removed `max_addresses` sync parameter which determined how many addresses to cache before syncing since this can just be done with `ensure_addresses_cached`.
+- remove `flush` method from the `Database` trait.
+
+## [v0.16.1]
+
+- Pin tokio dependency version to ~1.14 to prevent errors due to their new MSRV 1.49.0
+
+## [v0.16.0]
+
+- Disable `reqwest` default features.
+- Added `reqwest-default-tls` feature: Use this to restore the TLS defaults of reqwest if you don't want to add a dependency to it in your own manifest.
+- Use dust_value from rust-bitcoin
+- Fixed generating WIF in the correct network format.
+
+## [v0.15.0]
+
+- Overhauled sync logic for electrum and esplora.
+- Unify ureq and reqwest esplora backends to have the same configuration parameters. This means reqwest now has a timeout parameter and ureq has a concurrency parameter.
+- Fixed esplora fee estimation.
+
+## [v0.14.0]
+
+- BIP39 implementation dependency, in `keys::bip39` changed from tiny-bip39 to rust-bip39.
+- Add new method on the `TxBuilder` to embed data in the transaction via `OP_RETURN`. To allow that a fix to check the dust only on spendable output has been introduced.
+- Update the `Database` trait to store the last sync timestamp and block height
+- Rename `ConfirmationTime` to `BlockTime`
+
+## [v0.13.0]
+
+- Exposed `get_tx()` method from `Database` to `Wallet`.
+
+## [v0.12.0]
+
+- Activate `miniscript/use-serde` feature to allow consumers of the library to access it via the re-exported `miniscript` crate.
+- Add support for proxies in `EsploraBlockchain`
+- Added `SqliteDatabase` that implements `Database` backed by a sqlite database using `rusqlite` crate.
+
+## [v0.11.0]
+
+- Added `flush` method to the `Database` trait to explicitly flush to disk latest changes on the db.
+
+## [v0.10.0]
+
+- Added `RpcBlockchain` in the `AnyBlockchain` struct to allow using Rpc backend where `AnyBlockchain` is used (eg `bdk-cli`)
+- Removed hard dependency on `tokio`.
+
+### Wallet
+
+- Removed and replaced `set_single_recipient` with more general `drain_to` and replaced `maintain_single_recipient` with `allow_shrinking`.
+
+### Blockchain
+
+- Removed `stop_gap` from `Blockchain` trait and added it to only `ElectrumBlockchain` and `EsploraBlockchain` structs.
+- Added a `ureq` backend for use when not using feature `async-interface` or target WASM. `ureq` is a blocking HTTP client.
+
+## [v0.9.0]
+
+### Wallet
+
+- Added Bitcoin core RPC added as blockchain backend
+- Added a `verify` feature that can be enable to verify the unconfirmed txs we download against the consensus rules
+
+## [v0.8.0]
+
+### Wallet
+- Added an option that must be explicitly enabled to allow signing using non-`SIGHASH_ALL` sighashes (#350)
+#### Changed
+`get_address` now returns an `AddressInfo` struct that includes the index and derefs to `Address`.
+
+## [v0.7.0]
+
+### Policy
+#### Changed
+Removed `fill_satisfaction` method in favor of enum parameter in `extract_policy` method
+
+#### Added
+Timelocks are considered (optionally) in building the `satisfaction` field
+
+### Wallet
+
+- Changed `Wallet::{sign, finalize_psbt}` now take a `&mut psbt` rather than consuming it.
+- Require and validate `non_witness_utxo` for SegWit signatures by default, can be adjusted with `SignOptions`
+- Replace the opt-in builder option `force_non_witness_utxo` with the opposite `only_witness_utxo`. From now on we will provide the `non_witness_utxo`, unless explicitly asked not to.
+
+## [v0.6.0]
+
+### Misc
+#### Changed
+- New minimum supported rust version is 1.46.0
+- Changed `AnyBlockchainConfig` to use serde tagged representation.
+
+### Descriptor
+#### Added
+- Added ability to analyze a `PSBT` to check which and how many signatures are already available
+
+### Wallet
+#### Changed
+- `get_new_address()` refactored to `get_address(AddressIndex::New)` to support different `get_address()` index selection strategies
+
+#### Added
+- Added `get_address(AddressIndex::LastUnused)` which returns the last derived address if it has not been used or if used in a received transaction returns a new address
+- Added `get_address(AddressIndex::Peek(u32))` which returns a derived address for a specified descriptor index but does not change the current index
+- Added `get_address(AddressIndex::Reset(u32))` which returns a derived address for a specified descriptor index and resets current index to the given value
+- Added `get_psbt_input` to create the corresponding psbt input for a local utxo.
+
+#### Fixed
+- Fixed `coin_select` calculation for UTXOs where `value < fee` that caused over-/underflow errors.
+
+## [v0.5.1]
+
+### Misc
+#### Changed
+- Pin `hyper` to `=0.14.4` to make it compile on Rust 1.45
+
+## [v0.5.0]
+
+### Misc
+#### Changed
+- Updated `electrum-client` to version `0.7`
+
+### Wallet
+#### Changed
+- `FeeRate` constructors `from_sat_per_vb` and `default_min_relay_fee` are now `const` functions
+
+## [v0.4.0]
+
+### Keys
+#### Changed
+- Renamed `DerivableKey::add_metadata()` to `DerivableKey::into_descriptor_key()`
+- Renamed `ToDescriptorKey::to_descriptor_key()` to `IntoDescriptorKey::into_descriptor_key()`
+#### Added
+- Added an `ExtendedKey` type that is an enum of `bip32::ExtendedPubKey` and `bip32::ExtendedPrivKey`
+- Added `DerivableKey::into_extended_key()` as the only method that needs to be implemented
+
+### Misc
+#### Removed
+- Removed the `parse_descriptor` example, since it wasn't demonstrating any bdk-specific API anymore.
+#### Changed
+- Updated `bitcoin` to `0.26`, `miniscript` to `5.1` and `electrum-client` to `0.6`
+#### Added
+- Added support for the `signet` network (issue #62)
+- Added a function to get the version of BDK at runtime
+
+### Wallet
+#### Changed
+- Removed the explicit `id` argument from `Wallet::add_signer()` since that's now part of `Signer` itself
+- Renamed `ToWalletDescriptor::to_wallet_descriptor()` to `IntoWalletDescriptor::into_wallet_descriptor()`
+
+### Policy
+#### Changed
+- Removed unneeded `Result<(), PolicyError>` return type for `Satisfaction::finalize()`
+- Removed the `TooManyItemsSelected` policy error (see commit message for more details)
+
+## [v0.3.0]
+
+### Descriptor
+#### Changed
+- Added an alias `DescriptorError` for `descriptor::error::Error`
+- Changed the error returned by `descriptor!()` and `fragment!()` to `DescriptorError`
+- Changed the error type in `ToWalletDescriptor` to `DescriptorError`
+- Improved checks on descriptors built using the macros
+
+### Blockchain
+#### Changed
+- Remove `BlockchainMarker`, `OfflineClient` and `OfflineWallet` in favor of just using the unit
+  type to mark for a missing client.
+- Upgrade `tokio` to `1.0`.
+
+### Transaction Creation Overhaul
+
+The `TxBuilder` is now created from the `build_tx` or `build_fee_bump` functions on wallet and the
+final transaction is created by calling `finish` on the builder.
+
+- Removed `TxBuilder::utxos` in favor of `TxBuilder::add_utxos`
+- Added `Wallet::build_tx` to replace `Wallet::create_tx`
+- Added `Wallet::build_fee_bump` to replace `Wallet::bump_fee`
+- Added `Wallet::get_utxo`
+- Added `Wallet::get_descriptor_for_keychain`
+
+### `add_foreign_utxo`
+
+- Renamed `UTXO` to `LocalUtxo`
+- Added `WeightedUtxo` to replace floating `(UTXO, usize)`.
+- Added `Utxo` enum to incorporate both local utxos and foreign utxos
+- Added `TxBuilder::add_foreign_utxo` which allows adding a utxo external to the wallet.
+
+### CLI
+#### Changed
+- Remove `cli.rs` module, `cli-utils` feature and `repl.rs` example; moved to new [`bdk-cli`](https://github.com/bitcoindevkit/bdk-cli) repository
+
+## [v0.2.0]
+
+### Project
+#### Added
+- Add CONTRIBUTING.md
+- Add a Discord badge to the README
+- Add code coverage github actions workflow
+- Add scheduled audit check in CI
+- Add CHANGELOG.md
+
+#### Changed
+- Rename the library to `bdk`
+- Rename `ScriptType` to `KeychainKind`
+- Prettify README examples on github
+- Change CI to github actions
+- Bump rust-bitcoin to 0.25, fix Cargo dependencies
+- Enable clippy for stable and tests by default
+- Switch to "mainline" rust-miniscript
+- Generate a different cache key for every CI job
+- Fix to at least bitcoin ^0.25.2
+
+#### Fixed
+- Fix or ignore clippy warnings for all optional features except compact_filters
+- Pin cc version because last breaks rocksdb build
+
+### Blockchain
+#### Added
+- Add a trait to create `Blockchain`s from a configuration
+- Add an `AnyBlockchain` enum to allow switching at runtime
+- Document `AnyBlockchain` and `ConfigurableBlockchain`
+- Use our Instant struct to be compatible with wasm
+- Make esplora call in parallel
+- Allow to set concurrency in Esplora config and optionally pass it in repl
+
+#### Fixed
+- Fix receiving a coinbase using Electrum/Esplora
+- Use proper type for EsploraHeader, make conversion to BlockHeader infallible
+- Eagerly unwrap height option, save one collect
+
+#### Changed
+- Simplify the architecture of blockchain traits
+- Improve sync
+- Remove unused varaint HeaderParseFail
+
+### CLI
+#### Added
+- Conditionally remove cli args according to enabled feature
+
+#### Changed
+- Add max_addresses param in sync
+- Split the internal and external policy paths
+
+### Database
+#### Added
+- Add `AnyDatabase` and `ConfigurableDatabase` traits
+
+### Descriptor
+#### Added
+- Add a macro to write descriptors from code
+- Add descriptor templates, add `DerivableKey`
+- Add ToWalletDescriptor trait tests
+- Add support for `sortedmulti` in `descriptor!`
+- Add ExtractPolicy trait tests
+- Add get_checksum tests, cleanup tests
+- Add descriptor macro tests
+
+#### Changes
+- Improve the descriptor macro, add traits for key and descriptor types
+
+#### Fixes
+- Fix the recovery of a descriptor given a PSBT
+
+### Keys
+#### Added
+- Add BIP39 support
+- Take `ScriptContext` into account when converting keys
+- Add a way to restrict the networks in which keys are valid
+- Add a trait for keys that can be generated
+- Fix entropy generation
+- Less convoluted entropy generation
+- Re-export tiny-bip39
+- Implement `GeneratableKey` trait for `bitcoin::PrivateKey`
+- Implement `ToDescriptorKey` trait for `GeneratedKey`
+- Add a shortcut to generate keys with the default options
+
+#### Fixed
+- Fix all-keys and cli-utils tests
+
+### Wallet
+#### Added
+- Allow to define static fees for transactions Fixes #137
+- Merging two match expressions for fee calculation
+- Incorporate RBF rules into utxo selection function
+- Add Branch and Bound coin selection
+- Add tests for BranchAndBoundCoinSelection::coin_select
+- Add tests for BranchAndBoundCoinSelection::bnb
+- Add tests for BranchAndBoundCoinSelection::single_random_draw
+- Add test that shwpkh populates witness_utxo
+- Add witness and redeem scripts to PSBT outputs
+- Add an option to include `PSBT_GLOBAL_XPUB`s in PSBTs
+- Eagerly finalize inputs
+
+#### Changed
+- Use collect to avoid iter unwrapping Options
+- Make coin_select take may/must use utxo lists
+- Improve `CoinSelectionAlgorithm`
+- Refactor `Wallet::bump_fee()`
+- Default to SIGHASH_ALL if not specified
+- Replace ChangeSpendPolicy::filter_utxos with a predicate
+- Make 'unspendable' into a HashSet
+- Stop implicitly enforcing manaul selection by .add_utxo
+- Rename DumbCS to LargestFirstCoinSelection
+- Rename must_use_utxos to required_utxos
+- Rename may_use_utxos to optional_uxtos
+- Rename get_must_may_use_utxos to preselect_utxos
+- Remove redundant Box around address validators
+- Remove redundant Box around signers
+- Make Signer and AddressValidator Send and Sync
+- Split `send_all` into `set_single_recipient` and `drain_wallet`
+- Use TXIN_DEFAULT_WEIGHT constant in coin selection
+- Replace `must_use` with `required` in coin selection
+- Take both spending policies into account in create_tx
+- Check last derivation in cache to avoid recomputation
+- Use the branch-and-bound cs by default
+- Make coin_select return UTXOs instead of TxIns
+- Build output lookup inside complete transaction
+- Don't wrap SignersContainer arguments in Arc
+- More consistent references with 'signers' variables
+
+#### Fixed
+- Fix signing for `ShWpkh` inputs
+- Fix the recovery of a descriptor given a PSBT
+
+### Examples
+#### Added
+- Support esplora blockchain source in repl
+
+#### Changed
+- Revert back the REPL example to use Electrum
+- Remove the `magic` alias for `repl`
+- Require esplora feature for repl example
+
+#### Security
+- Use dirs-next instead of dirs since the latter is unmantained
+
+## [0.1.0-beta.1] - 2020-09-08
+
+### Blockchain
+#### Added
+- Lightweight Electrum client with SSL/SOCKS5 support
+- Add a generalized "Blockchain" interface
+- Add Error::OfflineClient
+- Add the Esplora backend
+- Use async I/O in the various blockchain impls
+- Compact Filters blockchain implementation
+- Add support for Tor
+- Impl OnlineBlockchain for types wrapped in Arc
+
+### Database
+#### Added
+- Add a generalized database trait and a Sled-based implementation
+- Add an in-memory database
+
+### Descriptor
+#### Added
+- Wrap Miniscript descriptors to support xpubs
+- Policy and contribution
+- Transform a descriptor into its "public" version
+- Use `miniscript::DescriptorPublicKey`
+
+### Macros
+#### Added
+- Add a feature to enable the async interface on non-wasm32 platforms
+
+### Wallet
+#### Added
+- Wallet logic
+- Add `assume_height_reached` in PSBTSatisfier
+- Add an option to change the assumed current height
+- Specify the policy branch with a map
+- Add a few commands to handle psbts
+- Add hd_keypaths to outputs
+- Add a `TxBuilder` struct to simplify `create_tx()`'s interface
+- Abstract coin selection in a separate trait
+- Refill the address pool whenever necessary
+- Implement the wallet import/export format from FullyNoded
+- Add a type convert fee units, add `Wallet::estimate_fee()`
+- TxOrdering, shuffle/bip69 support
+- Add RBF and custom versions in TxBuilder
+- Allow limiting the use of internal utxos in TxBuilder
+- Add `force_non_witness_utxo()` to TxBuilder
+- RBF and add a few tests
+- Add AddressValidators
+- Add explicit ordering for the signers
+- Support signing the whole tx instead of individual inputs
+- Create a PSBT signer from an ExtendedDescriptor
+
+### Examples
+#### Added
+- Add REPL broadcast command
+- Add a miniscript compiler CLI
+- Expose list_transactions() in the REPL
+- Use `MemoryDatabase` in the compiler example
+- Make the REPL return JSON
+
+[0.1.0-beta.1]: https://github.com/bitcoindevkit/bdk/compare/96c87ea5...0.1.0-beta.1
+[v0.2.0]: https://github.com/bitcoindevkit/bdk/compare/0.1.0-beta.1...v0.2.0
+[v0.3.0]: https://github.com/bitcoindevkit/bdk/compare/v0.2.0...v0.3.0
+[v0.4.0]: https://github.com/bitcoindevkit/bdk/compare/v0.3.0...v0.4.0
+[v0.5.0]: https://github.com/bitcoindevkit/bdk/compare/v0.4.0...v0.5.0
+[v0.5.1]: https://github.com/bitcoindevkit/bdk/compare/v0.5.0...v0.5.1
+[v0.6.0]: https://github.com/bitcoindevkit/bdk/compare/v0.5.1...v0.6.0
+[v0.7.0]: https://github.com/bitcoindevkit/bdk/compare/v0.6.0...v0.7.0
+[v0.8.0]: https://github.com/bitcoindevkit/bdk/compare/v0.7.0...v0.8.0
+[v0.9.0]: https://github.com/bitcoindevkit/bdk/compare/v0.8.0...v0.9.0
+[v0.10.0]: https://github.com/bitcoindevkit/bdk/compare/v0.9.0...v0.10.0
+[v0.11.0]: https://github.com/bitcoindevkit/bdk/compare/v0.10.0...v0.11.0
+[v0.12.0]: https://github.com/bitcoindevkit/bdk/compare/v0.11.0...v0.12.0
+[v0.13.0]: https://github.com/bitcoindevkit/bdk/compare/v0.12.0...v0.13.0
+[v0.14.0]: https://github.com/bitcoindevkit/bdk/compare/v0.13.0...v0.14.0
+[v0.15.0]: https://github.com/bitcoindevkit/bdk/compare/v0.14.0...v0.15.0
+[v0.16.0]: https://github.com/bitcoindevkit/bdk/compare/v0.15.0...v0.16.0
+[v0.16.1]: https://github.com/bitcoindevkit/bdk/compare/v0.16.0...v0.16.1
+[v0.17.0]: https://github.com/bitcoindevkit/bdk/compare/v0.16.1...v0.17.0
+[v0.18.0]: https://github.com/bitcoindevkit/bdk/compare/v0.17.0...v0.18.0
+[v0.19.0]: https://github.com/bitcoindevkit/bdk/compare/v0.18.0...v0.19.0
+[v0.20.0]: https://github.com/bitcoindevkit/bdk/compare/v0.19.0...v0.20.0
+[v0.21.0]: https://github.com/bitcoindevkit/bdk/compare/v0.20.0...v0.21.0
+[v0.22.0]: https://github.com/bitcoindevkit/bdk/compare/v0.21.0...v0.22.0
+[v0.23.0]: https://github.com/bitcoindevkit/bdk/compare/v0.22.0...v0.23.0
+[v0.24.0]: https://github.com/bitcoindevkit/bdk/compare/v0.23.0...v0.24.0
+[v0.25.0]: https://github.com/bitcoindevkit/bdk/compare/v0.24.0...v0.25.0
+[v0.26.0]: https://github.com/bitcoindevkit/bdk/compare/v0.25.0...v0.26.0
+[v0.27.0]: https://github.com/bitcoindevkit/bdk/compare/v0.26.0...v0.27.0
+[v0.27.1]: https://github.com/bitcoindevkit/bdk/compare/v0.27.0...v0.27.1
+[Unreleased]: https://github.com/bitcoindevkit/bdk/compare/v0.27.1...HEAD

CONTRIBUTING.md

@@ -0,0 +1,118 @@
+Contributing to BDK
+==============================
+
+The BDK project operates an open contributor model where anyone is welcome to
+contribute towards development in the form of peer review, documentation,
+testing and patches.
+
+Anyone is invited to contribute without regard to technical experience,
+"expertise", OSS experience, age, or other concern. However, the development of
+cryptocurrencies demands a high-level of rigor, adversarial thinking, thorough
+testing and risk-minimization.
+Any bug may cost users real money. That being said, we deeply welcome people
+contributing for the first time to an open source project or picking up Rust while
+contributing. Don't be shy, you'll learn.
+
+Communications Channels
+-----------------------
+
+Communication about BDK happens primarily on the [BDK Discord](https://discord.gg/dstn4dQ).
+
+Discussion about code base improvements happens in GitHub [issues](https://github.com/bitcoindevkit/bdk/issues) and
+on [pull requests](https://github.com/bitcoindevkit/bdk/pulls).
+
+Contribution Workflow
+---------------------
+
+The codebase is maintained using the "contributor workflow" where everyone
+without exception contributes patch proposals using "pull requests". This
+facilitates social contribution, easy testing and peer review.
+
+To contribute a patch, the workflow is as follows:
+
+  1. Fork Repository
+  2. Create topic branch
+  3. Commit patches
+
+In general commits should be atomic and diffs should be easy to read.
+For this reason do not mix any formatting fixes or code moves with actual code
+changes. Further, each commit, individually, should compile and pass tests, in
+order to ensure git bisect and other automated tools function properly.
+
+When adding a new feature, thought must be given to the long term technical
+debt.
+Every new feature should be covered by functional tests where possible.
+
+When refactoring, structure your PR to make it easy to review and don't
+hesitate to split it into multiple small, focused PRs.
+
+The Minimal Supported Rust Version is 1.46 (enforced by our CI).
+
+Commits should cover both the issue fixed and the solution's rationale.
+These [guidelines](https://chris.beams.io/posts/git-commit/) should be kept in mind.
+
+To facilitate communication with other contributors, the project is making use
+of GitHub's "assignee" field. First check that no one is assigned and then
+comment suggesting that you're working on it. If someone is already assigned,
+don't hesitate to ask if the assigned party or previous commenters are still
+working on it if it has been awhile.
+
+Deprecation policy
+------------------
+
+Where possible, breaking existing APIs should be avoided. Instead, add new APIs and
+use [`#[deprecated]`](https://github.com/rust-lang/rfcs/blob/master/text/1270-deprecation.md)
+to discourage use of the old one.
+
+Deprecated APIs are typically maintained for one release cycle. In other words, an
+API that has been deprecated with the 0.10 release can be expected to be removed in the
+0.11 release. This allows for smoother upgrades without incurring too much technical
+debt inside this library.
+
+If you deprecated an API as part of a contribution, we encourage you to "own" that API
+and send a follow-up to remove it as part of the next release cycle.
+
+Peer review
+-----------
+
+Anyone may participate in peer review which is expressed by comments in the
+pull request. Typically reviewers will review the code for obvious errors, as
+well as test out the patch set and opine on the technical merits of the patch.
+PR should be reviewed first on the conceptual level before focusing on code
+style or grammar fixes.
+
+Coding Conventions
+------------------
+
+This codebase uses spaces, not tabs.
+Use `cargo fmt` with the default settings to format code before committing.
+This is also enforced by the CI.
+
+Security
+--------
+
+Security is a high priority of BDK; disclosure of security vulnerabilites helps
+prevent user loss of funds.
+
+Note that BDK is currently considered "pre-production" during this time, there
+is no special handling of security issues. Please simply open an issue on
+Github.
+
+Testing
+-------
+
+Related to the security aspect, BDK developers take testing very seriously.
+Due to the modular nature of the project, writing new functional tests is easy
+and good test coverage of the codebase is an important goal.
+Refactoring the project to enable fine-grained unit testing is also an ongoing
+effort.
+
+Going further
+-------------
+
+You may be interested by Jon Atacks guide on [How to review Bitcoin Core PRs](https://github.com/jonatack/bitcoin-development/blob/master/how-to-review-bitcoin-core-prs.md)
+and [How to make Bitcoin Core PRs](https://github.com/jonatack/bitcoin-development/blob/master/how-to-make-bitcoin-core-prs.md).
+While there are differences between the projects in terms of context and
+maturity, many of the suggestions offered apply to this project.
+
+Overall, have fun :)

Cargo.toml

@@ -1,63 +1,18 @@
-[package]
-name = "magical-bitcoin-wallet"
-version = "0.1.0"
-edition = "2018"
-authors = ["Riccardo Casatta <riccardo@casatta.it>", "Alekos Filini <alekos.filini@gmail.com>"]
-
-[dependencies]
-log = "^0.4"
-bitcoin = { version = "0.23", features = ["use-serde"] }
-miniscript = { version = "0.12" }
-serde = { version = "^1.0", features = ["derive"] }
-serde_json = { version = "^1.0" }
-base64 = "^0.11"
-async-trait = "0.1"
-
-# Optional dependencies
-sled = { version = "0.31.0", optional = true }
-electrum-client = { git = "https://github.com/MagicalBitcoin/rust-electrum-client.git", optional = true }
-reqwest = { version = "0.10", optional = true, features = ["json"] }
-futures = { version = "0.3", optional = true }
-clap = { version = "2.33", optional = true }
-
-[features]
-minimal = []
-compiler = ["miniscript/compiler"]
-default = ["key-value-db", "electrum"]
-electrum = ["electrum-client"]
-esplora = ["reqwest", "futures"]
-key-value-db = ["sled"]
-cli-utils = ["clap"]
-multiparty = []
-
-[dev-dependencies]
-tokio = { version = "0.2", features = ["macros"] }
-lazy_static = "1.4"
-rustyline = "6.0"
-dirs = "2.0"
-env_logger = "0.7"
-rand = "0.7"
-
-[[example]]
-name = "repl"
-required-features = ["cli-utils"]
-[[example]]
-name = "psbt"
-[[example]]
-name = "parse_descriptor"
-
-[[example]]
-name = "miniscriptc"
-path = "examples/compiler.rs"
-required-features = ["compiler"]
-
-[[example]]
-name = "multiparty"
-required-features = ["multiparty","compiler"]
-
-# Provide a more user-friendly alias for the REPL
-[[example]]
-name = "magic"
-path = "examples/repl.rs"
-required-features = ["cli-utils"]
+[workspace]
+members = [
+    "crates/bdk",
+    "crates/chain",
+    "crates/file_store",
+    "crates/electrum",
+    "crates/esplora",
+    "example-crates/example_cli",
+    "example-crates/example_electrum",
+    "example-crates/wallet_electrum",
+    "example-crates/wallet_esplora",
+    "example-crates/wallet_esplora_async",
+    "nursery/tmp_plan",
+    "nursery/coin_select"
+]
 
+[workspace.package]
+authors = ["Bitcoin Dev Kit Developers"]

DEVELOPMENT_CYCLE.md

@@ -0,0 +1,16 @@
+# Development Cycle
+
+This project follows a regular releasing schedule similar to the one [used by the Rust language]. In short, this means that a new release is made at a regular cadence, with all the feature/bugfixes that made it to `master` in time. This ensures that we don't keep delaying releases waiting for "just one more little thing".
+
+This project uses [Semantic Versioning], but is currently at MAJOR version zero (0.y.z) meaning it is still in initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable. Until we reach version `1.0.0` we will do our best to document any breaking API changes in the changelog info attached to each release tag.
+
+We decided to maintain a faster release cycle while the library is still in "beta", i.e. before release `1.0.0`: since we are constantly adding new features and, even more importantly, fixing issues, we want developers to have access to those updates as fast as possible. For this reason we will make a release **every 4 weeks**.
+
+Once the project reaches a more mature state (>= `1.0.0`), we will very likely switch to longer release cycles of **6 weeks**.
+
+The "feature freeze" will happen **one week before the release date**. This means a new branch will be created originating from the `master` tip at that time, and in that branch we will stop adding new features and only focus on ensuring the ones we've added are working properly.
+
+To create a new release a release manager will create a new issue using the `Release` template and follow the template instructions.
+
+[used by the Rust language]: https://doc.rust-lang.org/book/appendix-07-nightly-rust.html
+[Semantic Versioning]: https://semver.org/

LICENSE

@@ -1,21 +0,0 @@
-MIT License
-
-Copyright (c) 2020 Magical Bitcoin
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.

README.md

@@ -1,7 +1,54 @@
-# Magical Bitcoin Wallet
+# The Bitcoin Dev Kit
 
-A modern, lightweight, descriptor-based wallet written in Rust!
+<div align="center">
+  <h1>BDK</h1>
 
-## Getting Started
+  <img src="./static/bdk.png" width="220" />
 
-See the documentation at [magicalbitcoin.org](https://magicalbitcoin.org)
+  <p>
+    <strong>A modern, lightweight, descriptor-based wallet library written in Rust!</strong>
+  </p>
+
+  <p>
+    <a href="https://crates.io/crates/bdk"><img alt="Crate Info" src="https://img.shields.io/crates/v/bdk.svg"/></a>
+    <a href="https://github.com/bitcoindevkit/bdk/blob/master/LICENSE"><img alt="MIT or Apache-2.0 Licensed" src="https://img.shields.io/badge/license-MIT%2FApache--2.0-blue.svg"/></a>
+    <a href="https://github.com/bitcoindevkit/bdk/actions?query=workflow%3ACI"><img alt="CI Status" src="https://github.com/bitcoindevkit/bdk/workflows/CI/badge.svg"></a>
+    <a href="https://coveralls.io/github/bitcoindevkit/bdk?branch=master"><img src="https://coveralls.io/repos/github/bitcoindevkit/bdk/badge.svg?branch=master"/></a>
+    <a href="https://docs.rs/bdk"><img alt="API Docs" src="https://img.shields.io/badge/docs.rs-bdk-green"/></a>
+    <a href="https://blog.rust-lang.org/2021/12/02/Rust-1.57.0.html"><img alt="Rustc Version 1.57.0+" src="https://img.shields.io/badge/rustc-1.57.0%2B-lightgrey.svg"/></a>
+    <a href="https://discord.gg/d7NkDKm"><img alt="Chat on Discord" src="https://img.shields.io/discord/753336465005608961?logo=discord"></a>
+  </p>
+
+  <h4>
+    <a href="https://bitcoindevkit.org">Project Homepage</a>
+    <span> | </span>
+    <a href="https://docs.rs/bdk">Documentation</a>
+  </h4>
+</div>
+
+## About
+
+The `bdk` libraries aims to provide well engineered and reviewed components for Bitcoin based applications.
+It is built upon the excellent [`rust-bitcoin`] and [`rust-miniscript`] crates.
+
+> ⚠ The Bitcoin Dev Kit developers are in the process of releasing a `v1.0` which is a fundamental re-write of how the library works.
+> See for some background on this project: https://bitcoindevkit.org/blog/road-to-bdk-1/ (ignore the timeline 😁)
+> For a release timeline see the [`bdk_core_staging`] repo where a lot of the component work is being done. The plan is that everything in the `bdk_core_staging` repo will be moved into the `crates` directory here.
+
+## Architecture
+
+The project is split up into several crates in the `/crates` directory:
+
+- [`bdk`](./crates/bdk): Contains the central high level `Wallet` type that is built from the low-level mechanisms provided by the other components
+- [`chain`](./crates/chain): Tools for storing and indexing chain data
+- [`file_store`](./crates/file_store): A (experimental) persistence backend for storing chain data in a single file.
+- [`esplora`](./crates/esplora): Extends the [`esplora-client`] crate with methods to fetch chain data from an esplora HTTP server in the form that [`bdk_chain`] and `Wallet` can consume.
+- [`electrum`](./crates/electrum): Extends the [`electrum-client`] crate with methods to fetch chain data from an electrum server in the form that [`bdk_chain`] and `Wallet` can consume.
+
+Fully working examples of how to use these components are in `/example-crates`
+
+[`bdk_core_staging`]: https://github.com/LLFourn/bdk_core_staging
+[`rust-miniscript`]: https://github.com/rust-bitcoin/rust-miniscript
+[`rust-bitcoin`]: https://github.com/rust-bitcoin/rust-bitcoin
+[`esplora-client`]: https://docs.rs/esplora-client/0.3.0/esplora_client/
+[`electrum-client`]: https://docs.rs/electrum-client/0.13.0/electrum_client/

ci/Dockerfile.ledger

@@ -0,0 +1,9 @@
+# Taken from bitcoindevkit/rust-hwi
+FROM ghcr.io/ledgerhq/speculos
+
+RUN apt-get update
+RUN apt-get install wget -y
+RUN wget "https://github.com/LedgerHQ/speculos/blob/master/apps/nanos%23btc%232.1%231c8db8da.elf?raw=true" -O /speculos/btc.elf
+ADD automation.json /speculos/automation.json
+
+ENTRYPOINT ["python", "./speculos.py", "--automation", "file:automation.json", "--model", "nanos", "--display", "headless", "--vnc-port", "41000", "btc.elf"]

ci/automation.json

@@ -0,0 +1,30 @@
+{
+    "version": 1,
+    "rules": [
+        {
+            "regexp": "Address \\(\\d/\\d\\)|Message hash \\(\\d/\\d\\)|Confirm|Fees|Review|Amount",
+            "actions": [
+                [ "button", 2, true ],
+                [ "button", 2, false ]
+            ]
+        },
+        {
+            "text": "Sign",
+            "conditions": [
+                [ "seen", false ]
+            ],
+            "actions": [
+                [ "button", 2, true ],
+                [ "button", 2, false ],
+                [ "setbool", "seen", true ]
+            ]
+        },
+        {
+            "regexp": "Approve|Sign|Accept",
+            "actions": [
+                [ "button", 3, true ],
+                [ "button", 3, false ]
+            ]
+        }
+    ]
+}

ci/start-core.sh

@@ -0,0 +1,14 @@
+#!/usr/bin/env sh
+
+echo "Starting bitcoin node."
+mkdir $GITHUB_WORKSPACE/.bitcoin
+/root/bitcoind -regtest -server -daemon -datadir=$GITHUB_WORKSPACE/.bitcoin -fallbackfee=0.0002 -rpcallowip=0.0.0.0/0 -rpcbind=0.0.0.0 -blockfilterindex=1 -peerblockfilters=1
+
+echo "Waiting for bitcoin node."
+until /root/bitcoin-cli -regtest -datadir=$GITHUB_WORKSPACE/.bitcoin getblockchaininfo; do
+    sleep 1
+done
+/root/bitcoin-cli -regtest -datadir=$GITHUB_WORKSPACE/.bitcoin createwallet $BDK_RPC_WALLET
+echo "Generating 150 bitcoin blocks."
+ADDR=$(/root/bitcoin-cli -regtest -datadir=$GITHUB_WORKSPACE/.bitcoin -rpcwallet=$BDK_RPC_WALLET getnewaddress)
+/root/bitcoin-cli -regtest -datadir=$GITHUB_WORKSPACE/.bitcoin generatetoaddress 150 $ADDR

clippy.toml

@@ -0,0 +1 @@
+msrv="1.57.0"

crates/bdk/Cargo.toml

@@ -0,0 +1,65 @@
+[package]
+name = "bdk"
+homepage = "https://bitcoindevkit.org"
+version = "1.0.0-alpha.1"
+repository = "https://github.com/bitcoindevkit/bdk"
+documentation = "https://docs.rs/bdk"
+description = "A modern, lightweight, descriptor-based wallet library"
+keywords = ["bitcoin", "wallet", "descriptor", "psbt"]
+readme = "README.md"
+license = "MIT OR Apache-2.0"
+authors = ["Bitcoin Dev Kit Developers"]
+edition = "2021"
+rust-version = "1.57"
+
+[dependencies]
+log = "=0.4.18"
+rand = "^0.8"
+miniscript = { version = "9", features = ["serde"], default-features = false }
+bitcoin = { version = "0.29", features = ["serde", "base64", "rand"], default-features = false }
+serde = { version = "^1.0", features = ["derive"] }
+serde_json = { version = "^1.0" }
+bdk_chain = { path = "../chain", version = "0.5.0", features = ["miniscript", "serde"], default-features = false }
+
+# Optional dependencies
+hwi = { version = "0.5", optional = true, features = [ "use-miniscript"] }
+bip39 = { version = "1.0.1", optional = true }
+
+[target.'cfg(target_arch = "wasm32")'.dependencies]
+getrandom = "0.2"
+js-sys = "0.3"
+
+[features]
+default = ["std"]
+std = ["bitcoin/std", "miniscript/std", "bdk_chain/std"]
+compiler = ["miniscript/compiler"]
+all-keys = ["keys-bip39"]
+keys-bip39 = ["bip39"]
+hardware-signer = ["hwi"]
+test-hardware-signer = ["hardware-signer"]
+
+# This feature is used to run `cargo check` in our CI targeting wasm. It's not recommended
+# for libraries to explicitly include the "getrandom/js" feature, so we only do it when
+# necessary for running our CI. See: https://docs.rs/getrandom/0.2.8/getrandom/#webassembly-support
+dev-getrandom-wasm = ["getrandom/js"]
+
+[dev-dependencies]
+lazy_static = "1.4"
+env_logger = "0.7"
+# Move back to importing from rust-bitcoin once https://github.com/rust-bitcoin/rust-bitcoin/pull/1342 is released
+base64 = "^0.13"
+assert_matches = "1.5.0"
+
+[package.metadata.docs.rs]
+all-features = true
+rustdoc-args = ["--cfg", "docsrs"]
+
+[[example]]
+name = "mnemonic_to_descriptors"
+path = "examples/mnemonic_to_descriptors.rs"
+required-features = ["all-keys"]
+
+[[example]]
+name = "miniscriptc"
+path = "examples/compiler.rs"
+required-features = ["compiler"]

crates/bdk/LICENSE

@@ -0,0 +1,14 @@
+This software is licensed under [Apache 2.0](LICENSE-APACHE) or
+[MIT](LICENSE-MIT), at your option.
+
+Some files retain their own copyright notice, however, for full authorship
+information, see version control history.
+
+Except as otherwise noted in individual files, all files in this repository are
+licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+http://www.apache.org/licenses/LICENSE-2.0> or the MIT license <LICENSE-MIT or
+http://opensource.org/licenses/MIT>, at your option.
+
+You may not use, copy, modify, merge, publish, distribute, sublicense, and/or
+sell copies of this software or any files in this repository except in
+accordance with one or both of these licenses.

crates/bdk/LICENSE-APACHE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

crates/bdk/LICENSE-MIT

@@ -0,0 +1,16 @@
+Permission is hereby granted, free of charge, to any person obtaining a copy of
+this software and associated documentation files (the "Software"), to deal in
+the Software without restriction, including without limitation the rights to
+use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
+the Software, and to permit persons to whom the Software is furnished to do so,
+subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

crates/bdk/README.md

@@ -0,0 +1,227 @@
+<div align="center">
+  <h1>BDK</h1>
+
+  <img src="https://raw.githubusercontent.com/bitcoindevkit/bdk/master/static/bdk.png" width="220" />
+
+  <p>
+    <strong>A modern, lightweight, descriptor-based wallet library written in Rust!</strong>
+  </p>
+
+  <p>
+    <a href="https://crates.io/crates/bdk"><img alt="Crate Info" src="https://img.shields.io/crates/v/bdk.svg"/></a>
+    <a href="https://github.com/bitcoindevkit/bdk/blob/master/LICENSE"><img alt="MIT or Apache-2.0 Licensed" src="https://img.shields.io/badge/license-MIT%2FApache--2.0-blue.svg"/></a>
+    <a href="https://github.com/bitcoindevkit/bdk/actions?query=workflow%3ACI"><img alt="CI Status" src="https://github.com/bitcoindevkit/bdk/workflows/CI/badge.svg"></a>
+    <a href="https://coveralls.io/github/bitcoindevkit/bdk?branch=master"><img src="https://coveralls.io/repos/github/bitcoindevkit/bdk/badge.svg?branch=master"/></a>
+    <a href="https://docs.rs/bdk"><img alt="API Docs" src="https://img.shields.io/badge/docs.rs-bdk-green"/></a>
+    <a href="https://blog.rust-lang.org/2021/12/02/Rust-1.57.0.html"><img alt="Rustc Version 1.57.0+" src="https://img.shields.io/badge/rustc-1.57.0%2B-lightgrey.svg"/></a>
+    <a href="https://discord.gg/d7NkDKm"><img alt="Chat on Discord" src="https://img.shields.io/discord/753336465005608961?logo=discord"></a>
+  </p>
+
+  <h4>
+    <a href="https://bitcoindevkit.org">Project Homepage</a>
+    <span> | </span>
+    <a href="https://docs.rs/bdk">Documentation</a>
+  </h4>
+</div>
+
+## `bdk`
+
+The `bdk` crate provides the [`Wallet`](`crate::Wallet`) type which is a simple, high-level
+interface built from the low-level components of [`bdk_chain`]. `Wallet` is a good starting point
+for many simple applications as well as a good demonstration of how to use the other mechanisms to
+construct a wallet. It has two keychains (external and internal) which are defined by
+[miniscript descriptors][`rust-miniscript`] and uses them to generate addresses. When you give it
+chain data it also uses the descriptors to find transaction outputs owned by them. From there, you
+can create and sign transactions.
+
+For more information, see the [`Wallet`'s documentation](https://docs.rs/bdk/latest/bdk/wallet/struct.Wallet.html).
+
+### Blockchain data
+
+In order to get blockchain data for `Wallet` to consume, you have to put it into particular form.
+Right now this is [`KeychainScan`] which is defined in [`bdk_chain`].
+
+This can be created manually or from blockchain-scanning crates.
+
+**Blockchain Data Sources**
+
+* [`bdk_esplora`]: Grabs blockchain data from Esplora for updating BDK structures.
+* [`bdk_electrum`]: Grabs blockchain data from Electrum for updating BDK structures.
+
+**Examples**
+
+* [`example-crates/wallet_esplora`](https://github.com/bitcoindevkit/bdk/tree/master/example-crates/wallet_esplora)
+* [`example-crates/wallet_electrum`](https://github.com/bitcoindevkit/bdk/tree/master/example-crates/wallet_electrum)
+
+### Persistence
+
+To persist the `Wallet` on disk, `Wallet` needs to be constructed with a
+[`Persist`](https://docs.rs/bdk_chain/latest/bdk_chain/keychain/struct.KeychainPersist.html) implementation.
+
+**Implementations**
+
+* [`bdk_file_store`]: a simple flat-file implementation of `Persist`.
+
+**Example**
+
+```rust
+use bdk::{bitcoin::Network, wallet::{AddressIndex, Wallet}};
+
+fn main() {
+    // a type that implements `Persist`
+    let db = ();
+
+    let descriptor = "wpkh(tprv8ZgxMBicQKsPdy6LMhUtFHAgpocR8GC6QmwMSFpZs7h6Eziw3SpThFfczTDh5rW2krkqffa11UpX3XkeTTB2FvzZKWXqPY54Y6Rq4AQ5R8L/84'/0'/0'/0/*)";
+    let mut wallet = Wallet::new(descriptor, None, db, Network::Testnet).expect("should create");
+
+    // get a new address (this increments revealed derivation index)
+    println!("revealed address: {}", wallet.get_address(AddressIndex::New));
+    println!("staged changes: {:?}", wallet.staged());
+    // persist changes
+    wallet.commit().expect("must save");
+}
+```
+
+<!-- ### Sync the balance of a descriptor -->
+
+<!-- ```rust,no_run -->
+<!-- use bdk::Wallet; -->
+<!-- use bdk::blockchain::ElectrumBlockchain; -->
+<!-- use bdk::SyncOptions; -->
+<!-- use bdk::electrum_client::Client; -->
+<!-- use bdk::bitcoin::Network; -->
+
+<!-- fn main() -> Result<(), bdk::Error> { -->
+<!--     let blockchain = ElectrumBlockchain::from(Client::new("ssl://electrum.blockstream.info:60002")?); -->
+<!--     let wallet = Wallet::new( -->
+<!--         "wpkh([c258d2e4/84h/1h/0h]tpubDDYkZojQFQjht8Tm4jsS3iuEmKjTiEGjG6KnuFNKKJb5A6ZUCUZKdvLdSDWofKi4ToRCwb9poe1XdqfUnP4jaJjCB2Zwv11ZLgSbnZSNecE/0/*)", -->
+<!--         Some("wpkh([c258d2e4/84h/1h/0h]tpubDDYkZojQFQjht8Tm4jsS3iuEmKjTiEGjG6KnuFNKKJb5A6ZUCUZKdvLdSDWofKi4ToRCwb9poe1XdqfUnP4jaJjCB2Zwv11ZLgSbnZSNecE/1/*)"), -->
+<!--         Network::Testnet, -->
+<!--     )?; -->
+
+<!--     wallet.sync(&blockchain, SyncOptions::default())?; -->
+
+<!--     println!("Descriptor balance: {} SAT", wallet.get_balance()?); -->
+
+<!--     Ok(()) -->
+<!-- } -->
+<!-- ``` -->
+<!-- ### Generate a few addresses -->
+
+<!-- ```rust -->
+<!-- use bdk::Wallet; -->
+<!-- use bdk::wallet::AddressIndex::New; -->
+<!-- use bdk::bitcoin::Network; -->
+
+<!-- fn main() -> Result<(), bdk::Error> { -->
+<!--     let wallet = Wallet::new_no_persist( -->
+<!--         "wpkh([c258d2e4/84h/1h/0h]tpubDDYkZojQFQjht8Tm4jsS3iuEmKjTiEGjG6KnuFNKKJb5A6ZUCUZKdvLdSDWofKi4ToRCwb9poe1XdqfUnP4jaJjCB2Zwv11ZLgSbnZSNecE/0/*)", -->
+<!--         Some("wpkh([c258d2e4/84h/1h/0h]tpubDDYkZojQFQjht8Tm4jsS3iuEmKjTiEGjG6KnuFNKKJb5A6ZUCUZKdvLdSDWofKi4ToRCwb9poe1XdqfUnP4jaJjCB2Zwv11ZLgSbnZSNecE/1/*)"), -->
+<!--         Network::Testnet, -->
+<!--     )?; -->
+
+<!--     println!("Address #0: {}", wallet.get_address(New)); -->
+<!--     println!("Address #1: {}", wallet.get_address(New)); -->
+<!--     println!("Address #2: {}", wallet.get_address(New)); -->
+
+<!--     Ok(()) -->
+<!-- } -->
+<!-- ``` -->
+
+<!-- ### Create a transaction -->
+
+<!-- ```rust,no_run -->
+<!-- use bdk::{FeeRate, Wallet, SyncOptions}; -->
+<!-- use bdk::blockchain::ElectrumBlockchain; -->
+
+<!-- use bdk::electrum_client::Client; -->
+<!-- use bdk::wallet::AddressIndex::New; -->
+
+<!-- use base64; -->
+<!-- use bdk::bitcoin::consensus::serialize; -->
+<!-- use bdk::bitcoin::Network; -->
+
+<!-- fn main() -> Result<(), bdk::Error> { -->
+<!--     let blockchain = ElectrumBlockchain::from(Client::new("ssl://electrum.blockstream.info:60002")?); -->
+<!--     let wallet = Wallet::new_no_persist( -->
+<!--         "wpkh([c258d2e4/84h/1h/0h]tpubDDYkZojQFQjht8Tm4jsS3iuEmKjTiEGjG6KnuFNKKJb5A6ZUCUZKdvLdSDWofKi4ToRCwb9poe1XdqfUnP4jaJjCB2Zwv11ZLgSbnZSNecE/0/*)", -->
+<!--         Some("wpkh([c258d2e4/84h/1h/0h]tpubDDYkZojQFQjht8Tm4jsS3iuEmKjTiEGjG6KnuFNKKJb5A6ZUCUZKdvLdSDWofKi4ToRCwb9poe1XdqfUnP4jaJjCB2Zwv11ZLgSbnZSNecE/1/*)"), -->
+<!--         Network::Testnet, -->
+<!--     )?; -->
+
+<!--     wallet.sync(&blockchain, SyncOptions::default())?; -->
+
+<!--     let send_to = wallet.get_address(New); -->
+<!--     let (psbt, details) = { -->
+<!--         let mut builder = wallet.build_tx(); -->
+<!--         builder -->
+<!--             .add_recipient(send_to.script_pubkey(), 50_000) -->
+<!--             .enable_rbf() -->
+<!--             .do_not_spend_change() -->
+<!--             .fee_rate(FeeRate::from_sat_per_vb(5.0)); -->
+<!--         builder.finish()? -->
+<!--     }; -->
+
+<!--     println!("Transaction details: {:#?}", details); -->
+<!--     println!("Unsigned PSBT: {}", base64::encode(&serialize(&psbt))); -->
+
+<!--     Ok(()) -->
+<!-- } -->
+<!-- ``` -->
+
+<!-- ### Sign a transaction -->
+
+<!-- ```rust,no_run -->
+<!-- use bdk::{Wallet, SignOptions}; -->
+
+<!-- use base64; -->
+<!-- use bdk::bitcoin::consensus::deserialize; -->
+<!-- use bdk::bitcoin::Network; -->
+
+<!-- fn main() -> Result<(), bdk::Error> { -->
+<!--     let wallet = Wallet::new_no_persist( -->
+<!--         "wpkh([c258d2e4/84h/1h/0h]tprv8griRPhA7342zfRyB6CqeKF8CJDXYu5pgnj1cjL1u2ngKcJha5jjTRimG82ABzJQ4MQe71CV54xfn25BbhCNfEGGJZnxvCDQCd6JkbvxW6h/0/*)", -->
+<!--         Some("wpkh([c258d2e4/84h/1h/0h]tprv8griRPhA7342zfRyB6CqeKF8CJDXYu5pgnj1cjL1u2ngKcJha5jjTRimG82ABzJQ4MQe71CV54xfn25BbhCNfEGGJZnxvCDQCd6JkbvxW6h/1/*)"), -->
+<!--         Network::Testnet, -->
+<!--     )?; -->
+
+<!--     let psbt = "..."; -->
+<!--     let mut psbt = deserialize(&base64::decode(psbt).unwrap())?; -->
+
+<!--     let _finalized = wallet.sign(&mut psbt, SignOptions::default())?; -->
+
+<!--     Ok(()) -->
+<!-- } -->
+<!-- ``` -->
+
+## Testing
+
+### Unit testing
+
+```bash
+cargo test
+```
+
+## License
+
+Licensed under either of
+
+ * Apache License, Version 2.0
+   ([LICENSE-APACHE](LICENSE-APACHE) or <https://www.apache.org/licenses/LICENSE-2.0>)
+ * MIT license
+   ([LICENSE-MIT](LICENSE-MIT) or <https://opensource.org/licenses/MIT>)
+
+at your option.
+
+## Contribution
+
+Unless you explicitly state otherwise, any contribution intentionally submitted
+for inclusion in the work by you, as defined in the Apache-2.0 license, shall be
+dual licensed as above, without any additional terms or conditions.
+
+[`bdk_chain`]: https://docs.rs/bdk_chain/latest
+[`bdk_file_store`]: https://docs.rs/bdk_file_store/latest
+[`bdk_electrum`]: https://docs.rs/bdk_electrum/latest
+[`bdk_esplora`]: https://docs.rs/bdk_esplora/latest
+[`KeychainScan`]: https://docs.rs/bdk_chain/latest/bdk_chain/keychain/struct.KeychainScan.html
+[`rust-miniscript`]: https://docs.rs/miniscript/latest/miniscript/index.html

crates/bdk/examples/compiler.rs

@@ -0,0 +1,73 @@
+// Bitcoin Dev Kit
+// Written in 2020 by Alekos Filini <alekos.filini@gmail.com>
+//
+// Copyright (c) 2020-2021 Bitcoin Dev Kit Developers
+//
+// This file is licensed under the Apache License, Version 2.0 <LICENSE-APACHE
+// or http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option.
+// You may not use this file except in accordance with one or both of these
+// licenses.
+
+extern crate bdk;
+extern crate bitcoin;
+extern crate log;
+extern crate miniscript;
+extern crate serde_json;
+
+use std::error::Error;
+use std::str::FromStr;
+
+use log::info;
+
+use bitcoin::Network;
+use miniscript::policy::Concrete;
+use miniscript::Descriptor;
+
+use bdk::wallet::AddressIndex::New;
+use bdk::{KeychainKind, Wallet};
+
+/// Miniscript policy is a high level abstraction of spending conditions. Defined in the
+/// rust-miniscript library here  https://docs.rs/miniscript/7.0.0/miniscript/policy/index.html
+/// rust-miniscript provides a `compile()` function that can be used to compile any miniscript policy
+/// into a descriptor. This descriptor then in turn can be used in bdk a fully functioning wallet
+/// can be derived from the policy.
+///
+/// This example demonstrates the interaction between a bdk wallet and miniscript policy.
+
+fn main() -> Result<(), Box<dyn Error>> {
+    env_logger::init_from_env(
+        env_logger::Env::default().filter_or(env_logger::DEFAULT_FILTER_ENV, "info"),
+    );
+
+    // We start with a generic miniscript policy string
+    let policy_str = "or(10@thresh(4,pk(029ffbe722b147f3035c87cb1c60b9a5947dd49c774cc31e94773478711a929ac0),pk(025f05815e3a1a8a83bfbb03ce016c9a2ee31066b98f567f6227df1d76ec4bd143),pk(025625f41e4a065efc06d5019cbbd56fe8c07595af1231e7cbc03fafb87ebb71ec),pk(02a27c8b850a00f67da3499b60562673dcf5fdfb82b7e17652a7ac54416812aefd),pk(03e618ec5f384d6e19ca9ebdb8e2119e5bef978285076828ce054e55c4daf473e2)),1@and(older(4209713),thresh(2,pk(03deae92101c790b12653231439f27b8897264125ecb2f46f48278603102573165),pk(033841045a531e1adf9910a6ec279589a90b3b8a904ee64ffd692bd08a8996c1aa),pk(02aebf2d10b040eb936a6f02f44ee82f8b34f5c1ccb20ff3949c2b28206b7c1068))))";
+    info!("Compiling policy: \n{}", policy_str);
+
+    // Parse the string as a [`Concrete`] type miniscript policy.
+    let policy = Concrete::<String>::from_str(policy_str)?;
+
+    // Create a `wsh` type descriptor from the policy.
+    // `policy.compile()` returns the resulting miniscript from the policy.
+    let descriptor = Descriptor::new_wsh(policy.compile()?)?;
+
+    info!("Compiled into following Descriptor: \n{}", descriptor);
+
+    // Create a new wallet from this descriptor
+    let mut wallet = Wallet::new_no_persist(&format!("{}", descriptor), None, Network::Regtest)?;
+
+    info!(
+        "First derived address from the descriptor: \n{}",
+        wallet.get_address(New)
+    );
+
+    // BDK also has it's own `Policy` structure to represent the spending condition in a more
+    // human readable json format.
+    let spending_policy = wallet.policies(KeychainKind::External)?;
+    info!(
+        "The BDK spending policy: \n{}",
+        serde_json::to_string_pretty(&spending_policy)?
+    );
+
+    Ok(())
+}

crates/bdk/examples/mnemonic_to_descriptors.rs

@@ -0,0 +1,60 @@
+// Copyright (c) 2020-2021 Bitcoin Dev Kit Developers
+//
+// This file is licensed under the Apache License, Version 2.0 <LICENSE-APACHE
+// or http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option.
+// You may not use this file except in accordance with one or both of these
+// licenses.
+
+use bdk::bitcoin::secp256k1::Secp256k1;
+use bdk::bitcoin::util::bip32::DerivationPath;
+use bdk::bitcoin::Network;
+use bdk::descriptor;
+use bdk::descriptor::IntoWalletDescriptor;
+use bdk::keys::bip39::{Language, Mnemonic, WordCount};
+use bdk::keys::{GeneratableKey, GeneratedKey};
+use bdk::miniscript::Tap;
+use bdk::Error as BDK_Error;
+use std::error::Error;
+use std::str::FromStr;
+
+/// This example demonstrates how to generate a mnemonic phrase
+/// using BDK and use that to generate a descriptor string.
+fn main() -> Result<(), Box<dyn Error>> {
+    let secp = Secp256k1::new();
+
+    // In this example we are generating a 12 words mnemonic phrase
+    // but it is also possible generate 15, 18, 21 and 24 words
+    // using their respective `WordCount` variant.
+    let mnemonic: GeneratedKey<_, Tap> =
+        Mnemonic::generate((WordCount::Words12, Language::English))
+            .map_err(|_| BDK_Error::Generic("Mnemonic generation error".to_string()))?;
+
+    println!("Mnemonic phrase: {}", *mnemonic);
+    let mnemonic_with_passphrase = (mnemonic, None);
+
+    // define external and internal derivation key path
+    let external_path = DerivationPath::from_str("m/86h/0h/0h/0").unwrap();
+    let internal_path = DerivationPath::from_str("m/86h/0h/0h/1").unwrap();
+
+    // generate external and internal descriptor from mnemonic
+    let (external_descriptor, ext_keymap) =
+        descriptor!(tr((mnemonic_with_passphrase.clone(), external_path)))?
+            .into_wallet_descriptor(&secp, Network::Testnet)?;
+    let (internal_descriptor, int_keymap) =
+        descriptor!(tr((mnemonic_with_passphrase, internal_path)))?
+            .into_wallet_descriptor(&secp, Network::Testnet)?;
+
+    println!("tpub external descriptor: {}", external_descriptor);
+    println!("tpub internal descriptor: {}", internal_descriptor);
+    println!(
+        "tprv external descriptor: {}",
+        external_descriptor.to_string_with_secret(&ext_keymap)
+    );
+    println!(
+        "tprv internal descriptor: {}",
+        internal_descriptor.to_string_with_secret(&int_keymap)
+    );
+
+    Ok(())
+}

crates/bdk/examples/policy.rs

@@ -0,0 +1,66 @@
+// Bitcoin Dev Kit
+// Written in 2020 by Alekos Filini <alekos.filini@gmail.com>
+//
+// Copyright (c) 2020-2021 Bitcoin Dev Kit Developers
+//
+// This file is licensed under the Apache License, Version 2.0 <LICENSE-APACHE
+// or http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option.
+// You may not use this file except in accordance with one or both of these
+// licenses.
+
+extern crate bdk;
+extern crate env_logger;
+extern crate log;
+use std::error::Error;
+
+use bdk::bitcoin::Network;
+use bdk::descriptor::{policy::BuildSatisfaction, ExtractPolicy, IntoWalletDescriptor};
+use bdk::wallet::signer::SignersContainer;
+
+/// This example describes the use of the BDK's [`bdk::descriptor::policy`] module.
+///
+/// Policy is higher abstraction representation of the wallet descriptor spending condition.
+/// This is useful to express complex miniscript spending conditions into more human readable form.
+/// The resulting `Policy` structure  can be used to derive spending conditions the wallet is capable
+/// to spend from.
+///
+/// This example demos a Policy output for a 2of2 multisig between between 2 parties, where the wallet holds
+/// one of the Extend Private key.
+
+fn main() -> Result<(), Box<dyn Error>> {
+    env_logger::init_from_env(
+        env_logger::Env::default().filter_or(env_logger::DEFAULT_FILTER_ENV, "info"),
+    );
+
+    let secp = bitcoin::secp256k1::Secp256k1::new();
+
+    // The descriptor used in the example
+    // The form is "wsh(multi(2, <privkey>, <pubkey>))"
+    let desc = "wsh(multi(2,tprv8ZgxMBicQKsPdpkqS7Eair4YxjcuuvDPNYmKX3sCniCf16tHEVrjjiSXEkFRnUH77yXc6ZcwHHcLNfjdi5qUvw3VDfgYiH5mNsj5izuiu2N/1/*,tpubD6NzVbkrYhZ4XHndKkuB8FifXm8r5FQHwrN6oZuWCz13qb93rtgKvD4PQsqC4HP4yhV3tA2fqr2RbY5mNXfM7RxXUoeABoDtsFUq2zJq6YK/1/*))";
+
+    // Use the descriptor string to derive the full descriptor and a keymap.
+    // The wallet descriptor can be used to create a new bdk::wallet.
+    // While the `keymap` can be used to create a `SignerContainer`.
+    //
+    // The `SignerContainer` can sign for `PSBT`s.
+    // a bdk::wallet internally uses these to handle transaction signing.
+    // But they can be used as independent tools also.
+    let (wallet_desc, keymap) = desc.into_wallet_descriptor(&secp, Network::Testnet)?;
+
+    log::info!("Example Descriptor for policy analysis : {}", wallet_desc);
+
+    // Create the signer with the keymap and descriptor.
+    let signers_container = SignersContainer::build(keymap, &wallet_desc, &secp);
+
+    // Extract the Policy from the given descriptor and signer.
+    // Note that Policy is a wallet specific structure. It depends on the the descriptor, and
+    // what the concerned wallet with a given signer can sign for.
+    let policy = wallet_desc
+        .extract_policy(&signers_container, BuildSatisfaction::None, &secp)?
+        .expect("We expect a policy");
+
+    log::info!("Derived Policy for the descriptor {:#?}", policy);
+
+    Ok(())
+}

crates/bdk/src/descriptor/checksum.rs

@@ -0,0 +1,182 @@
+// Bitcoin Dev Kit
+// Written in 2020 by Alekos Filini <alekos.filini@gmail.com>
+//
+// Copyright (c) 2020-2021 Bitcoin Dev Kit Developers
+//
+// This file is licensed under the Apache License, Version 2.0 <LICENSE-APACHE
+// or http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option.
+// You may not use this file except in accordance with one or both of these
+// licenses.
+
+//! Descriptor checksum
+//!
+//! This module contains a re-implementation of the function used by Bitcoin Core to calculate the
+//! checksum of a descriptor
+
+use crate::descriptor::DescriptorError;
+use alloc::string::String;
+
+const INPUT_CHARSET: &[u8] = b"0123456789()[],'/*abcdefgh@:$%{}IJKLMNOPQRSTUVWXYZ&+-.;<=>?!^_|~ijklmnopqrstuvwxyzABCDEFGH`#\"\\ ";
+const CHECKSUM_CHARSET: &[u8] = b"qpzry9x8gf2tvdw0s3jn54khce6mua7l";
+
+fn poly_mod(mut c: u64, val: u64) -> u64 {
+    let c0 = c >> 35;
+    c = ((c & 0x7ffffffff) << 5) ^ val;
+    if c0 & 1 > 0 {
+        c ^= 0xf5dee51989
+    };
+    if c0 & 2 > 0 {
+        c ^= 0xa9fdca3312
+    };
+    if c0 & 4 > 0 {
+        c ^= 0x1bab10e32d
+    };
+    if c0 & 8 > 0 {
+        c ^= 0x3706b1677a
+    };
+    if c0 & 16 > 0 {
+        c ^= 0x644d626ffd
+    };
+
+    c
+}
+
+/// Computes the checksum bytes of a descriptor.
+/// `exclude_hash = true` ignores all data after the first '#' (inclusive).
+pub(crate) fn calc_checksum_bytes_internal(
+    mut desc: &str,
+    exclude_hash: bool,
+) -> Result<[u8; 8], DescriptorError> {
+    let mut c = 1;
+    let mut cls = 0;
+    let mut clscount = 0;
+
+    let mut original_checksum = None;
+    if exclude_hash {
+        if let Some(split) = desc.split_once('#') {
+            desc = split.0;
+            original_checksum = Some(split.1);
+        }
+    }
+
+    for ch in desc.as_bytes() {
+        let pos = INPUT_CHARSET
+            .iter()
+            .position(|b| b == ch)
+            .ok_or(DescriptorError::InvalidDescriptorCharacter(*ch))? as u64;
+        c = poly_mod(c, pos & 31);
+        cls = cls * 3 + (pos >> 5);
+        clscount += 1;
+        if clscount == 3 {
+            c = poly_mod(c, cls);
+            cls = 0;
+            clscount = 0;
+        }
+    }
+    if clscount > 0 {
+        c = poly_mod(c, cls);
+    }
+    (0..8).for_each(|_| c = poly_mod(c, 0));
+    c ^= 1;
+
+    let mut checksum = [0_u8; 8];
+    for j in 0..8 {
+        checksum[j] = CHECKSUM_CHARSET[((c >> (5 * (7 - j))) & 31) as usize];
+    }
+
+    // if input data already had a checksum, check calculated checksum against original checksum
+    if let Some(original_checksum) = original_checksum {
+        if original_checksum.as_bytes() != checksum {
+            return Err(DescriptorError::InvalidDescriptorChecksum);
+        }
+    }
+
+    Ok(checksum)
+}
+
+/// Compute the checksum bytes of a descriptor, excludes any existing checksum in the descriptor string from the calculation
+pub fn calc_checksum_bytes(desc: &str) -> Result<[u8; 8], DescriptorError> {
+    calc_checksum_bytes_internal(desc, true)
+}
+
+/// Compute the checksum of a descriptor, excludes any existing checksum in the descriptor string from the calculation
+pub fn calc_checksum(desc: &str) -> Result<String, DescriptorError> {
+    // unsafe is okay here as the checksum only uses bytes in `CHECKSUM_CHARSET`
+    calc_checksum_bytes_internal(desc, true)
+        .map(|b| unsafe { String::from_utf8_unchecked(b.to_vec()) })
+}
+
+// TODO in release 0.25.0, remove get_checksum_bytes and get_checksum
+// TODO in release 0.25.0, consolidate calc_checksum_bytes_internal into calc_checksum_bytes
+
+/// Compute the checksum bytes of a descriptor
+#[deprecated(
+    since = "0.24.0",
+    note = "Use new `calc_checksum_bytes` function which excludes any existing checksum in the descriptor string before calculating the checksum hash bytes. See https://github.com/bitcoindevkit/bdk/pull/765."
+)]
+pub fn get_checksum_bytes(desc: &str) -> Result<[u8; 8], DescriptorError> {
+    calc_checksum_bytes_internal(desc, false)
+}
+
+/// Compute the checksum of a descriptor
+#[deprecated(
+    since = "0.24.0",
+    note = "Use new `calc_checksum` function which excludes any existing checksum in the descriptor string before calculating the checksum hash. See https://github.com/bitcoindevkit/bdk/pull/765."
+)]
+pub fn get_checksum(desc: &str) -> Result<String, DescriptorError> {
+    // unsafe is okay here as the checksum only uses bytes in `CHECKSUM_CHARSET`
+    calc_checksum_bytes_internal(desc, false)
+        .map(|b| unsafe { String::from_utf8_unchecked(b.to_vec()) })
+}
+
+#[cfg(test)]
+mod test {
+    use super::*;
+    use crate::descriptor::calc_checksum;
+    use assert_matches::assert_matches;
+
+    // test calc_checksum() function; it should return the same value as Bitcoin Core
+    #[test]
+    fn test_calc_checksum() {
+        let desc = "wpkh(tprv8ZgxMBicQKsPdpkqS7Eair4YxjcuuvDPNYmKX3sCniCf16tHEVrjjiSXEkFRnUH77yXc6ZcwHHcLNfjdi5qUvw3VDfgYiH5mNsj5izuiu2N/1/2/*)";
+        assert_eq!(calc_checksum(desc).unwrap(), "tqz0nc62");
+
+        let desc = "pkh(tpubD6NzVbkrYhZ4XHndKkuB8FifXm8r5FQHwrN6oZuWCz13qb93rtgKvD4PQsqC4HP4yhV3tA2fqr2RbY5mNXfM7RxXUoeABoDtsFUq2zJq6YK/44'/1'/0'/0/*)";
+        assert_eq!(calc_checksum(desc).unwrap(), "lasegmfs");
+    }
+
+    // test calc_checksum() function; it should return the same value as Bitcoin Core even if the
+    // descriptor string includes a checksum hash
+    #[test]
+    fn test_calc_checksum_with_checksum_hash() {
+        let desc = "wpkh(tprv8ZgxMBicQKsPdpkqS7Eair4YxjcuuvDPNYmKX3sCniCf16tHEVrjjiSXEkFRnUH77yXc6ZcwHHcLNfjdi5qUvw3VDfgYiH5mNsj5izuiu2N/1/2/*)#tqz0nc62";
+        assert_eq!(calc_checksum(desc).unwrap(), "tqz0nc62");
+
+        let desc = "pkh(tpubD6NzVbkrYhZ4XHndKkuB8FifXm8r5FQHwrN6oZuWCz13qb93rtgKvD4PQsqC4HP4yhV3tA2fqr2RbY5mNXfM7RxXUoeABoDtsFUq2zJq6YK/44'/1'/0'/0/*)#lasegmfs";
+        assert_eq!(calc_checksum(desc).unwrap(), "lasegmfs");
+
+        let desc = "wpkh(tprv8ZgxMBicQKsPdpkqS7Eair4YxjcuuvDPNYmKX3sCniCf16tHEVrjjiSXEkFRnUH77yXc6ZcwHHcLNfjdi5qUvw3VDfgYiH5mNsj5izuiu2N/1/2/*)#tqz0nc26";
+        assert_matches!(
+            calc_checksum(desc),
+            Err(DescriptorError::InvalidDescriptorChecksum)
+        );
+
+        let desc = "pkh(tpubD6NzVbkrYhZ4XHndKkuB8FifXm8r5FQHwrN6oZuWCz13qb93rtgKvD4PQsqC4HP4yhV3tA2fqr2RbY5mNXfM7RxXUoeABoDtsFUq2zJq6YK/44'/1'/0'/0/*)#lasegmsf";
+        assert_matches!(
+            calc_checksum(desc),
+            Err(DescriptorError::InvalidDescriptorChecksum)
+        );
+    }
+
+    #[test]
+    fn test_calc_checksum_invalid_character() {
+        let sparkle_heart = unsafe { core::str::from_utf8_unchecked(&[240, 159, 146, 150]) };
+        let invalid_desc = format!("wpkh(tprv8ZgxMBicQKsPdpkqS7Eair4YxjcuuvDPNYmKX3sCniCf16tHEVrjjiSXEkFRnUH77yXc6ZcwHHcL{}fjdi5qUvw3VDfgYiH5mNsj5izuiu2N/1/2/*)", sparkle_heart);
+
+        assert_matches!(
+            calc_checksum(&invalid_desc),
+            Err(DescriptorError::InvalidDescriptorCharacter(invalid_char)) if invalid_char == sparkle_heart.as_bytes()[0]
+        );
+    }
+}

crates/bdk/src/descriptor/error.rs

@@ -0,0 +1,89 @@
+// Bitcoin Dev Kit
+// Written in 2020 by Alekos Filini <alekos.filini@gmail.com>
+//
+// Copyright (c) 2020-2021 Bitcoin Dev Kit Developers
+//
+// This file is licensed under the Apache License, Version 2.0 <LICENSE-APACHE
+// or http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option.
+// You may not use this file except in accordance with one or both of these
+// licenses.
+
+//! Descriptor errors
+
+use core::fmt;
+
+/// Errors related to the parsing and usage of descriptors
+#[derive(Debug)]
+pub enum Error {
+    /// Invalid HD Key path, such as having a wildcard but a length != 1
+    InvalidHdKeyPath,
+    /// The provided descriptor doesn't match its checksum
+    InvalidDescriptorChecksum,
+    /// The descriptor contains hardened derivation steps on public extended keys
+    HardenedDerivationXpub,
+
+    /// Error thrown while working with [`keys`](crate::keys)
+    Key(crate::keys::KeyError),
+    /// Error while extracting and manipulating policies
+    Policy(crate::descriptor::policy::PolicyError),
+
+    /// Invalid byte found in the descriptor checksum
+    InvalidDescriptorCharacter(u8),
+
+    /// BIP32 error
+    Bip32(bitcoin::util::bip32::Error),
+    /// Error during base58 decoding
+    Base58(bitcoin::util::base58::Error),
+    /// Key-related error
+    Pk(bitcoin::util::key::Error),
+    /// Miniscript error
+    Miniscript(miniscript::Error),
+    /// Hex decoding error
+    Hex(bitcoin::hashes::hex::Error),
+}
+
+impl From<crate::keys::KeyError> for Error {
+    fn from(key_error: crate::keys::KeyError) -> Error {
+        match key_error {
+            crate::keys::KeyError::Miniscript(inner) => Error::Miniscript(inner),
+            crate::keys::KeyError::Bip32(inner) => Error::Bip32(inner),
+            e => Error::Key(e),
+        }
+    }
+}
+
+impl fmt::Display for Error {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        match self {
+            Self::InvalidHdKeyPath => write!(f, "Invalid HD key path"),
+            Self::InvalidDescriptorChecksum => {
+                write!(f, "The provided descriptor doesn't match its checksum")
+            }
+            Self::HardenedDerivationXpub => write!(
+                f,
+                "The descriptor contains hardened derivation steps on public extended keys"
+            ),
+            Self::Key(err) => write!(f, "Key error: {}", err),
+            Self::Policy(err) => write!(f, "Policy error: {}", err),
+            Self::InvalidDescriptorCharacter(char) => {
+                write!(f, "Invalid descriptor character: {}", char)
+            }
+            Self::Bip32(err) => write!(f, "BIP32 error: {}", err),
+            Self::Base58(err) => write!(f, "Base58 error: {}", err),
+            Self::Pk(err) => write!(f, "Key-related error: {}", err),
+            Self::Miniscript(err) => write!(f, "Miniscript error: {}", err),
+            Self::Hex(err) => write!(f, "Hex decoding error: {}", err),
+        }
+    }
+}
+
+#[cfg(feature = "std")]
+impl std::error::Error for Error {}
+
+impl_error!(bitcoin::util::bip32::Error, Bip32);
+impl_error!(bitcoin::util::base58::Error, Base58);
+impl_error!(bitcoin::util::key::Error, Pk);
+impl_error!(miniscript::Error, Miniscript);
+impl_error!(bitcoin::hashes::hex::Error, Hex);
+impl_error!(crate::descriptor::policy::PolicyError, Policy);

crates/bdk/src/descriptor/mod.rs

@@ -0,0 +1,876 @@
+// Bitcoin Dev Kit
+// Written in 2020 by Alekos Filini <alekos.filini@gmail.com>
+//
+// Copyright (c) 2020-2021 Bitcoin Dev Kit Developers
+//
+// This file is licensed under the Apache License, Version 2.0 <LICENSE-APACHE
+// or http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option.
+// You may not use this file except in accordance with one or both of these
+// licenses.
+
+//! Descriptors
+//!
+//! This module contains generic utilities to work with descriptors, plus some re-exported types
+//! from [`miniscript`].
+
+use crate::collections::BTreeMap;
+use alloc::string::String;
+use alloc::vec::Vec;
+
+use bitcoin::util::bip32::{ChildNumber, DerivationPath, ExtendedPubKey, Fingerprint, KeySource};
+use bitcoin::util::{psbt, taproot};
+use bitcoin::{secp256k1, PublicKey, XOnlyPublicKey};
+use bitcoin::{Network, TxOut};
+
+use miniscript::descriptor::{
+    DefiniteDescriptorKey, DescriptorSecretKey, DescriptorType, InnerXKey, SinglePubKey,
+};
+pub use miniscript::{
+    descriptor::DescriptorXKey, descriptor::KeyMap, descriptor::Wildcard, Descriptor,
+    DescriptorPublicKey, Legacy, Miniscript, ScriptContext, Segwitv0,
+};
+use miniscript::{ForEachKey, MiniscriptKey, TranslatePk};
+
+use crate::descriptor::policy::BuildSatisfaction;
+
+pub mod checksum;
+#[doc(hidden)]
+pub mod dsl;
+pub mod error;
+pub mod policy;
+pub mod template;
+
+pub use self::checksum::calc_checksum;
+use self::checksum::calc_checksum_bytes;
+pub use self::error::Error as DescriptorError;
+pub use self::policy::Policy;
+use self::template::DescriptorTemplateOut;
+use crate::keys::{IntoDescriptorKey, KeyError};
+use crate::wallet::signer::SignersContainer;
+use crate::wallet::utils::SecpCtx;
+
+/// Alias for a [`Descriptor`] that can contain extended keys using [`DescriptorPublicKey`]
+pub type ExtendedDescriptor = Descriptor<DescriptorPublicKey>;
+
+/// Alias for a [`Descriptor`] that contains extended **derived** keys
+pub type DerivedDescriptor = Descriptor<DefiniteDescriptorKey>;
+
+/// Alias for the type of maps that represent derivation paths in a [`psbt::Input`] or
+/// [`psbt::Output`]
+///
+/// [`psbt::Input`]: bitcoin::util::psbt::Input
+/// [`psbt::Output`]: bitcoin::util::psbt::Output
+pub type HdKeyPaths = BTreeMap<secp256k1::PublicKey, KeySource>;
+
+/// Alias for the type of maps that represent taproot key origins in a [`psbt::Input`] or
+/// [`psbt::Output`]
+///
+/// [`psbt::Input`]: bitcoin::util::psbt::Input
+/// [`psbt::Output`]: bitcoin::util::psbt::Output
+pub type TapKeyOrigins = BTreeMap<bitcoin::XOnlyPublicKey, (Vec<taproot::TapLeafHash>, KeySource)>;
+
+/// Trait for types which can be converted into an [`ExtendedDescriptor`] and a [`KeyMap`] usable by a wallet in a specific [`Network`]
+pub trait IntoWalletDescriptor {
+    /// Convert to wallet descriptor
+    fn into_wallet_descriptor(
+        self,
+        secp: &SecpCtx,
+        network: Network,
+    ) -> Result<(ExtendedDescriptor, KeyMap), DescriptorError>;
+}
+
+impl IntoWalletDescriptor for &str {
+    fn into_wallet_descriptor(
+        self,
+        secp: &SecpCtx,
+        network: Network,
+    ) -> Result<(ExtendedDescriptor, KeyMap), DescriptorError> {
+        let descriptor = match self.split_once('#') {
+            Some((desc, original_checksum)) => {
+                let checksum = calc_checksum_bytes(desc)?;
+                if original_checksum.as_bytes() != checksum {
+                    return Err(DescriptorError::InvalidDescriptorChecksum);
+                }
+                desc
+            }
+            None => self,
+        };
+
+        ExtendedDescriptor::parse_descriptor(secp, descriptor)?
+            .into_wallet_descriptor(secp, network)
+    }
+}
+
+impl IntoWalletDescriptor for &String {
+    fn into_wallet_descriptor(
+        self,
+        secp: &SecpCtx,
+        network: Network,
+    ) -> Result<(ExtendedDescriptor, KeyMap), DescriptorError> {
+        self.as_str().into_wallet_descriptor(secp, network)
+    }
+}
+
+impl IntoWalletDescriptor for ExtendedDescriptor {
+    fn into_wallet_descriptor(
+        self,
+        secp: &SecpCtx,
+        network: Network,
+    ) -> Result<(ExtendedDescriptor, KeyMap), DescriptorError> {
+        (self, KeyMap::default()).into_wallet_descriptor(secp, network)
+    }
+}
+
+impl IntoWalletDescriptor for (ExtendedDescriptor, KeyMap) {
+    fn into_wallet_descriptor(
+        self,
+        secp: &SecpCtx,
+        network: Network,
+    ) -> Result<(ExtendedDescriptor, KeyMap), DescriptorError> {
+        use crate::keys::DescriptorKey;
+
+        struct Translator<'s, 'd> {
+            secp: &'s SecpCtx,
+            descriptor: &'d ExtendedDescriptor,
+            network: Network,
+        }
+
+        impl<'s, 'd>
+            miniscript::Translator<DescriptorPublicKey, miniscript::DummyKey, DescriptorError>
+            for Translator<'s, 'd>
+        {
+            fn pk(
+                &mut self,
+                pk: &DescriptorPublicKey,
+            ) -> Result<miniscript::DummyKey, DescriptorError> {
+                let secp = &self.secp;
+
+                let (_, _, networks) = if self.descriptor.is_taproot() {
+                    let descriptor_key: DescriptorKey<miniscript::Tap> =
+                        pk.clone().into_descriptor_key()?;
+                    descriptor_key.extract(secp)?
+                } else if self.descriptor.is_witness() {
+                    let descriptor_key: DescriptorKey<miniscript::Segwitv0> =
+                        pk.clone().into_descriptor_key()?;
+                    descriptor_key.extract(secp)?
+                } else {
+                    let descriptor_key: DescriptorKey<miniscript::Legacy> =
+                        pk.clone().into_descriptor_key()?;
+                    descriptor_key.extract(secp)?
+                };
+
+                if networks.contains(&self.network) {
+                    Ok(miniscript::DummyKey)
+                } else {
+                    Err(DescriptorError::Key(KeyError::InvalidNetwork))
+                }
+            }
+            fn sha256(
+                &mut self,
+                _sha256: &<DescriptorPublicKey as MiniscriptKey>::Sha256,
+            ) -> Result<miniscript::DummySha256Hash, DescriptorError> {
+                Ok(Default::default())
+            }
+            fn hash256(
+                &mut self,
+                _hash256: &<DescriptorPublicKey as MiniscriptKey>::Hash256,
+            ) -> Result<miniscript::DummyHash256Hash, DescriptorError> {
+                Ok(Default::default())
+            }
+            fn ripemd160(
+                &mut self,
+                _ripemd160: &<DescriptorPublicKey as MiniscriptKey>::Ripemd160,
+            ) -> Result<miniscript::DummyRipemd160Hash, DescriptorError> {
+                Ok(Default::default())
+            }
+            fn hash160(
+                &mut self,
+                _hash160: &<DescriptorPublicKey as MiniscriptKey>::Hash160,
+            ) -> Result<miniscript::DummyHash160Hash, DescriptorError> {
+                Ok(Default::default())
+            }
+        }
+
+        // check the network for the keys
+        self.0.translate_pk(&mut Translator {
+            secp,
+            network,
+            descriptor: &self.0,
+        })?;
+
+        Ok(self)
+    }
+}
+
+impl IntoWalletDescriptor for DescriptorTemplateOut {
+    fn into_wallet_descriptor(
+        self,
+        _secp: &SecpCtx,
+        network: Network,
+    ) -> Result<(ExtendedDescriptor, KeyMap), DescriptorError> {
+        struct Translator {
+            network: Network,
+        }
+
+        impl miniscript::Translator<DescriptorPublicKey, DescriptorPublicKey, DescriptorError>
+            for Translator
+        {
+            fn pk(
+                &mut self,
+                pk: &DescriptorPublicKey,
+            ) -> Result<DescriptorPublicKey, DescriptorError> {
+                // workaround for xpubs generated by other key types, like bip39: since when the
+                // conversion is made one network has to be chosen, what we generally choose
+                // "mainnet", but then override the set of valid networks to specify that all of
+                // them are valid. here we reset the network to make sure the wallet struct gets a
+                // descriptor with the right network everywhere.
+                let pk = match pk {
+                    DescriptorPublicKey::XPub(ref xpub) => {
+                        let mut xpub = xpub.clone();
+                        xpub.xkey.network = self.network;
+
+                        DescriptorPublicKey::XPub(xpub)
+                    }
+                    other => other.clone(),
+                };
+
+                Ok(pk)
+            }
+            miniscript::translate_hash_clone!(
+                DescriptorPublicKey,
+                DescriptorPublicKey,
+                DescriptorError
+            );
+        }
+
+        let (desc, keymap, networks) = self;
+
+        if !networks.contains(&network) {
+            return Err(DescriptorError::Key(KeyError::InvalidNetwork));
+        }
+
+        // fixup the network for keys that need it in the descriptor
+        let translated = desc.translate_pk(&mut Translator { network })?;
+        // ...and in the key map
+        let fixed_keymap = keymap
+            .into_iter()
+            .map(|(mut k, mut v)| {
+                match (&mut k, &mut v) {
+                    (DescriptorPublicKey::XPub(xpub), DescriptorSecretKey::XPrv(xprv)) => {
+                        xpub.xkey.network = network;
+                        xprv.xkey.network = network;
+                    }
+                    (_, DescriptorSecretKey::Single(key)) => {
+                        key.key.network = network;
+                    }
+                    _ => {}
+                }
+
+                (k, v)
+            })
+            .collect();
+
+        Ok((translated, fixed_keymap))
+    }
+}
+
+/// Wrapper for `IntoWalletDescriptor` that performs additional checks on the keys contained in the
+/// descriptor
+pub(crate) fn into_wallet_descriptor_checked<T: IntoWalletDescriptor>(
+    inner: T,
+    secp: &SecpCtx,
+    network: Network,
+) -> Result<(ExtendedDescriptor, KeyMap), DescriptorError> {
+    let (descriptor, keymap) = inner.into_wallet_descriptor(secp, network)?;
+
+    // Ensure the keys don't contain any hardened derivation steps or hardened wildcards
+    let descriptor_contains_hardened_steps = descriptor.for_any_key(|k| {
+        if let DescriptorPublicKey::XPub(DescriptorXKey {
+            derivation_path,
+            wildcard,
+            ..
+        }) = k
+        {
+            return *wildcard == Wildcard::Hardened
+                || derivation_path.into_iter().any(ChildNumber::is_hardened);
+        }
+
+        false
+    });
+    if descriptor_contains_hardened_steps {
+        return Err(DescriptorError::HardenedDerivationXpub);
+    }
+
+    // Run miniscript's sanity check, which will look for duplicated keys and other potential
+    // issues
+    descriptor.sanity_check()?;
+
+    Ok((descriptor, keymap))
+}
+
+#[doc(hidden)]
+/// Used internally mainly by the `descriptor!()` and `fragment!()` macros
+pub trait CheckMiniscript<Ctx: miniscript::ScriptContext> {
+    fn check_miniscript(&self) -> Result<(), miniscript::Error>;
+}
+
+impl<Ctx: miniscript::ScriptContext, Pk: miniscript::MiniscriptKey> CheckMiniscript<Ctx>
+    for miniscript::Miniscript<Pk, Ctx>
+{
+    fn check_miniscript(&self) -> Result<(), miniscript::Error> {
+        Ctx::check_global_validity(self)?;
+
+        Ok(())
+    }
+}
+
+/// Trait implemented on [`Descriptor`]s to add a method to extract the spending [`policy`]
+pub trait ExtractPolicy {
+    /// Extract the spending [`policy`]
+    fn extract_policy(
+        &self,
+        signers: &SignersContainer,
+        psbt: BuildSatisfaction,
+        secp: &SecpCtx,
+    ) -> Result<Option<Policy>, DescriptorError>;
+}
+
+pub(crate) trait XKeyUtils {
+    fn root_fingerprint(&self, secp: &SecpCtx) -> Fingerprint;
+}
+
+impl<T> XKeyUtils for DescriptorXKey<T>
+where
+    T: InnerXKey,
+{
+    fn root_fingerprint(&self, secp: &SecpCtx) -> Fingerprint {
+        match self.origin {
+            Some((fingerprint, _)) => fingerprint,
+            None => self.xkey.xkey_fingerprint(secp),
+        }
+    }
+}
+
+pub(crate) trait DescriptorMeta {
+    fn is_witness(&self) -> bool;
+    fn is_taproot(&self) -> bool;
+    fn get_extended_keys(&self) -> Vec<DescriptorXKey<ExtendedPubKey>>;
+    fn derive_from_hd_keypaths(
+        &self,
+        hd_keypaths: &HdKeyPaths,
+        secp: &SecpCtx,
+    ) -> Option<DerivedDescriptor>;
+    fn derive_from_tap_key_origins(
+        &self,
+        tap_key_origins: &TapKeyOrigins,
+        secp: &SecpCtx,
+    ) -> Option<DerivedDescriptor>;
+    fn derive_from_psbt_key_origins(
+        &self,
+        key_origins: BTreeMap<Fingerprint, (&DerivationPath, SinglePubKey)>,
+        secp: &SecpCtx,
+    ) -> Option<DerivedDescriptor>;
+    fn derive_from_psbt_input(
+        &self,
+        psbt_input: &psbt::Input,
+        utxo: Option<TxOut>,
+        secp: &SecpCtx,
+    ) -> Option<DerivedDescriptor>;
+}
+
+impl DescriptorMeta for ExtendedDescriptor {
+    fn is_witness(&self) -> bool {
+        matches!(
+            self.desc_type(),
+            DescriptorType::Wpkh
+                | DescriptorType::ShWpkh
+                | DescriptorType::Wsh
+                | DescriptorType::ShWsh
+                | DescriptorType::ShWshSortedMulti
+                | DescriptorType::WshSortedMulti
+        )
+    }
+
+    fn is_taproot(&self) -> bool {
+        self.desc_type() == DescriptorType::Tr
+    }
+
+    fn get_extended_keys(&self) -> Vec<DescriptorXKey<ExtendedPubKey>> {
+        let mut answer = Vec::new();
+
+        self.for_each_key(|pk| {
+            if let DescriptorPublicKey::XPub(xpub) = pk {
+                answer.push(xpub.clone());
+            }
+
+            true
+        });
+
+        answer
+    }
+
+    fn derive_from_psbt_key_origins(
+        &self,
+        key_origins: BTreeMap<Fingerprint, (&DerivationPath, SinglePubKey)>,
+        secp: &SecpCtx,
+    ) -> Option<DerivedDescriptor> {
+        // Ensure that deriving `xpub` with `path` yields `expected`
+        let verify_key = |xpub: &DescriptorXKey<ExtendedPubKey>,
+                          path: &DerivationPath,
+                          expected: &SinglePubKey| {
+            let derived = xpub
+                .xkey
+                .derive_pub(secp, path)
+                .expect("The path should never contain hardened derivation steps")
+                .public_key;
+
+            match expected {
+                SinglePubKey::FullKey(pk) if &PublicKey::new(derived) == pk => true,
+                SinglePubKey::XOnly(pk) if &XOnlyPublicKey::from(derived) == pk => true,
+                _ => false,
+            }
+        };
+
+        let mut path_found = None;
+
+        // using `for_any_key` should make this stop as soon as we return `true`
+        self.for_any_key(|key| {
+            if let DescriptorPublicKey::XPub(xpub) = key {
+                // Check if the key matches one entry in our `key_origins`. If it does, `matches()` will
+                // return the "prefix" that matched, so we remove that prefix from the full path
+                // found in `key_origins` and save it in `derive_path`. We expect this to be a derivation
+                // path of length 1 if the key is `wildcard` and an empty path otherwise.
+                let root_fingerprint = xpub.root_fingerprint(secp);
+                let derive_path = key_origins
+                    .get_key_value(&root_fingerprint)
+                    .and_then(|(fingerprint, (path, expected))| {
+                        xpub.matches(&(*fingerprint, (*path).clone()), secp)
+                            .zip(Some((path, expected)))
+                    })
+                    .and_then(|(prefix, (full_path, expected))| {
+                        let derive_path = full_path
+                            .into_iter()
+                            .skip(prefix.into_iter().count())
+                            .cloned()
+                            .collect::<DerivationPath>();
+
+                        // `derive_path` only contains the replacement index for the wildcard, if present, or
+                        // an empty path for fixed descriptors. To verify the key we also need the normal steps
+                        // that come before the wildcard, so we take them directly from `xpub` and then append
+                        // the final index
+                        if verify_key(
+                            xpub,
+                            &xpub.derivation_path.extend(derive_path.clone()),
+                            expected,
+                        ) {
+                            Some(derive_path)
+                        } else {
+                            log::debug!(
+                                "Key `{}` derived with {} yields an unexpected key",
+                                root_fingerprint,
+                                derive_path
+                            );
+                            None
+                        }
+                    });
+
+                match derive_path {
+                    Some(path) if xpub.wildcard != Wildcard::None && path.len() == 1 => {
+                        // Ignore hardened wildcards
+                        if let ChildNumber::Normal { index } = path[0] {
+                            path_found = Some(index);
+                            return true;
+                        }
+                    }
+                    Some(path) if xpub.wildcard == Wildcard::None && path.is_empty() => {
+                        path_found = Some(0);
+                        return true;
+                    }
+                    _ => {}
+                }
+            }
+
+            false
+        });
+
+        path_found.map(|path| self.at_derivation_index(path))
+    }
+
+    fn derive_from_hd_keypaths(
+        &self,
+        hd_keypaths: &HdKeyPaths,
+        secp: &SecpCtx,
+    ) -> Option<DerivedDescriptor> {
+        // "Convert" an hd_keypaths map to the format required by `derive_from_psbt_key_origins`
+        let key_origins = hd_keypaths
+            .iter()
+            .map(|(pk, (fingerprint, path))| {
+                (
+                    *fingerprint,
+                    (path, SinglePubKey::FullKey(PublicKey::new(*pk))),
+                )
+            })
+            .collect();
+        self.derive_from_psbt_key_origins(key_origins, secp)
+    }
+
+    fn derive_from_tap_key_origins(
+        &self,
+        tap_key_origins: &TapKeyOrigins,
+        secp: &SecpCtx,
+    ) -> Option<DerivedDescriptor> {
+        // "Convert" a tap_key_origins map to the format required by `derive_from_psbt_key_origins`
+        let key_origins = tap_key_origins
+            .iter()
+            .map(|(pk, (_, (fingerprint, path)))| (*fingerprint, (path, SinglePubKey::XOnly(*pk))))
+            .collect();
+        self.derive_from_psbt_key_origins(key_origins, secp)
+    }
+
+    fn derive_from_psbt_input(
+        &self,
+        psbt_input: &psbt::Input,
+        utxo: Option<TxOut>,
+        secp: &SecpCtx,
+    ) -> Option<DerivedDescriptor> {
+        if let Some(derived) = self.derive_from_hd_keypaths(&psbt_input.bip32_derivation, secp) {
+            return Some(derived);
+        }
+        if let Some(derived) = self.derive_from_tap_key_origins(&psbt_input.tap_key_origins, secp) {
+            return Some(derived);
+        }
+        if self.has_wildcard() {
+            // We can't try to bruteforce the derivation index, exit here
+            return None;
+        }
+
+        let descriptor = self.at_derivation_index(0);
+        match descriptor.desc_type() {
+            // TODO: add pk() here
+            DescriptorType::Pkh
+            | DescriptorType::Wpkh
+            | DescriptorType::ShWpkh
+            | DescriptorType::Tr
+                if utxo.is_some()
+                    && descriptor.script_pubkey() == utxo.as_ref().unwrap().script_pubkey =>
+            {
+                Some(descriptor)
+            }
+            DescriptorType::Bare | DescriptorType::Sh | DescriptorType::ShSortedMulti
+                if psbt_input.redeem_script.is_some()
+                    && &descriptor.explicit_script().unwrap()
+                        == psbt_input.redeem_script.as_ref().unwrap() =>
+            {
+                Some(descriptor)
+            }
+            DescriptorType::Wsh
+            | DescriptorType::ShWsh
+            | DescriptorType::ShWshSortedMulti
+            | DescriptorType::WshSortedMulti
+                if psbt_input.witness_script.is_some()
+                    && &descriptor.explicit_script().unwrap()
+                        == psbt_input.witness_script.as_ref().unwrap() =>
+            {
+                Some(descriptor)
+            }
+            _ => None,
+        }
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use alloc::string::ToString;
+    use core::str::FromStr;
+
+    use assert_matches::assert_matches;
+    use bitcoin::consensus::encode::deserialize;
+    use bitcoin::hashes::hex::FromHex;
+    use bitcoin::secp256k1::Secp256k1;
+    use bitcoin::util::{bip32, psbt};
+    use bitcoin::Script;
+
+    use super::*;
+    use crate::psbt::PsbtUtils;
+
+    #[test]
+    fn test_derive_from_psbt_input_wpkh_wif() {
+        let descriptor = Descriptor::<DescriptorPublicKey>::from_str(
+            "wpkh(02b4632d08485ff1df2db55b9dafd23347d1c47a457072a1e87be26896549a8737)",
+        )
+        .unwrap();
+        let psbt: psbt::PartiallySignedTransaction = deserialize(
+            &Vec::<u8>::from_hex(
+                "70736274ff010052010000000162307be8e431fbaff807cdf9cdc3fde44d7402\
+                 11bc8342c31ffd6ec11fe35bcc0100000000ffffffff01328601000000000016\
+                 001493ce48570b55c42c2af816aeaba06cfee1224fae000000000001011fa086\
+                 01000000000016001493ce48570b55c42c2af816aeaba06cfee1224fae010304\
+                 010000000000",
+            )
+            .unwrap(),
+        )
+        .unwrap();
+
+        assert!(descriptor
+            .derive_from_psbt_input(&psbt.inputs[0], psbt.get_utxo_for(0), &Secp256k1::new())
+            .is_some());
+    }
+
+    #[test]
+    fn test_derive_from_psbt_input_pkh_tpub() {
+        let descriptor = Descriptor::<DescriptorPublicKey>::from_str(
+            "pkh([0f056943/44h/0h/0h]tpubDDpWvmUrPZrhSPmUzCMBHffvC3HyMAPnWDSAQNBTnj1iZeJa7BZQEttFiP4DS4GCcXQHezdXhn86Hj6LHX5EDstXPWrMaSneRWM8yUf6NFd/10/*)",
+        )
+        .unwrap();
+        let psbt: psbt::PartiallySignedTransaction = deserialize(
+            &Vec::<u8>::from_hex(
+                "70736274ff010053010000000145843b86be54a3cd8c9e38444e1162676c00df\
+                 e7964122a70df491ea12fd67090100000000ffffffff01c19598000000000017\
+                 a91432bb94283282f72b2e034709e348c44d5a4db0ef8700000000000100f902\
+                 0000000001010167e99c0eb67640f3a1b6805f2d8be8238c947f8aaf49eb0a9c\
+                 bee6a42c984200000000171600142b29a22019cca05b9c2b2d283a4c4489e1cf\
+                 9f8ffeffffff02a01dced06100000017a914e2abf033cadbd74f0f4c74946201\
+                 decd20d5c43c8780969800000000001976a9148b0fce5fb1264e599a65387313\
+                 3c95478b902eb288ac02473044022015d9211576163fa5b001e84dfa3d44efd9\
+                 86b8f3a0d3d2174369288b2b750906022048dacc0e5d73ae42512fd2b97e2071\
+                 a8d0bce443b390b1fe0b8128fe70ec919e01210232dad1c5a67dcb0116d407e2\
+                 52584228ab7ec00e8b9779d0c3ffe8114fc1a7d2c80600000103040100000022\
+                 0603433b83583f8c4879b329dd08bbc7da935e4cc02f637ff746e05f0466ffb2\
+                 a6a2180f0569432c00008000000080000000800a000000000000000000",
+            )
+            .unwrap(),
+        )
+        .unwrap();
+
+        assert!(descriptor
+            .derive_from_psbt_input(&psbt.inputs[0], psbt.get_utxo_for(0), &Secp256k1::new())
+            .is_some());
+    }
+
+    #[test]
+    fn test_derive_from_psbt_input_wsh() {
+        let descriptor = Descriptor::<DescriptorPublicKey>::from_str(
+            "wsh(and_v(v:pk(03b6633fef2397a0a9de9d7b6f23aef8368a6e362b0581f0f0af70d5ecfd254b14),older(6)))",
+        )
+        .unwrap();
+        let psbt: psbt::PartiallySignedTransaction = deserialize(
+            &Vec::<u8>::from_hex(
+                "70736274ff01005302000000011c8116eea34408ab6529223c9a176606742207\
+                 67a1ff1d46a6e3c4a88243ea6e01000000000600000001109698000000000017\
+                 a914ad105f61102e0d01d7af40d06d6a5c3ae2f7fde387000000000001012b80\
+                 969800000000002200203ca72f106a72234754890ca7640c43f65d2174e44d33\
+                 336030f9059345091044010304010000000105252103b6633fef2397a0a9de9d\
+                 7b6f23aef8368a6e362b0581f0f0af70d5ecfd254b14ad56b20000",
+            )
+            .unwrap(),
+        )
+        .unwrap();
+
+        assert!(descriptor
+            .derive_from_psbt_input(&psbt.inputs[0], psbt.get_utxo_for(0), &Secp256k1::new())
+            .is_some());
+    }
+
+    #[test]
+    fn test_derive_from_psbt_input_sh() {
+        let descriptor = Descriptor::<DescriptorPublicKey>::from_str(
+            "sh(and_v(v:pk(021403881a5587297818fcaf17d239cefca22fce84a45b3b1d23e836c4af671dbb),after(630000)))",
+        )
+        .unwrap();
+        let psbt: psbt::PartiallySignedTransaction = deserialize(
+            &Vec::<u8>::from_hex(
+                "70736274ff0100530100000001bc8c13df445dfadcc42afa6dc841f85d22b01d\
+                 a6270ebf981740f4b7b1d800390000000000feffffff01ba9598000000000017\
+                 a91457b148ba4d3e5fa8608a8657875124e3d1c9390887f09c0900000100e002\
+                 0000000001016ba1bbe05cc93574a0d611ec7d93ad0ab6685b28d0cd80e8a82d\
+                 debb326643c90100000000feffffff02809698000000000017a914d9a6e8c455\
+                 8e16c8253afe53ce37ad61cf4c38c487403504cf6100000017a9144044fb6e0b\
+                 757dfc1b34886b6a95aef4d3db137e870247304402202a9b72d939bcde8ba2a1\
+                 e0980597e47af4f5c152a78499143c3d0a78ac2286a602207a45b1df9e93b8c9\
+                 6f09f5c025fe3e413ca4b905fe65ee55d32a3276439a9b8f012102dc1fcc2636\
+                 4da1aa718f03d8d9bd6f2ff410ed2cf1245a168aa3bcc995ac18e0a806000001\
+                 03040100000001042821021403881a5587297818fcaf17d239cefca22fce84a4\
+                 5b3b1d23e836c4af671dbbad03f09c09b10000",
+            )
+            .unwrap(),
+        )
+        .unwrap();
+
+        assert!(descriptor
+            .derive_from_psbt_input(&psbt.inputs[0], psbt.get_utxo_for(0), &Secp256k1::new())
+            .is_some());
+    }
+
+    #[test]
+    fn test_to_wallet_descriptor_fixup_networks() {
+        use crate::keys::{any_network, IntoDescriptorKey};
+
+        let secp = Secp256k1::new();
+
+        let xprv = bip32::ExtendedPrivKey::from_str("xprv9s21ZrQH143K3c3gF1DUWpWNr2SG2XrG8oYPpqYh7hoWsJy9NjabErnzriJPpnGHyKz5NgdXmq1KVbqS1r4NXdCoKitWg5e86zqXHa8kxyB").unwrap();
+        let path = bip32::DerivationPath::from_str("m/0").unwrap();
+
+        // here `to_descriptor_key` will set the valid networks for the key to only mainnet, since
+        // we are using an "xpub"
+        let key = (xprv, path.clone()).into_descriptor_key().unwrap();
+        // override it with any. this happens in some key conversions, like bip39
+        let key = key.override_valid_networks(any_network());
+
+        // make a descriptor out of it
+        let desc = crate::descriptor!(wpkh(key)).unwrap();
+        // this should convert the key that supports "any_network" to the right network (testnet)
+        let (wallet_desc, keymap) = desc
+            .into_wallet_descriptor(&secp, Network::Testnet)
+            .unwrap();
+
+        let mut xprv_testnet = xprv;
+        xprv_testnet.network = Network::Testnet;
+
+        let xpub_testnet = bip32::ExtendedPubKey::from_priv(&secp, &xprv_testnet);
+        let desc_pubkey = DescriptorPublicKey::XPub(DescriptorXKey {
+            xkey: xpub_testnet,
+            origin: None,
+            derivation_path: path,
+            wildcard: Wildcard::Unhardened,
+        });
+
+        assert_eq!(wallet_desc.to_string(), "wpkh(tpubD6NzVbkrYhZ4XtJzoDja5snUjBNQRP5B3f4Hyn1T1x6PVPxzzVjvw6nJx2D8RBCxog9GEVjZoyStfepTz7TtKoBVdkCtnc7VCJh9dD4RAU9/0/*)#a3svx0ha");
+        assert_eq!(
+            keymap
+                .get(&desc_pubkey)
+                .map(|key| key.to_public(&secp).unwrap()),
+            Some(desc_pubkey)
+        );
+    }
+
+    // test IntoWalletDescriptor trait from &str with and without checksum appended
+    #[test]
+    fn test_descriptor_from_str_with_checksum() {
+        let secp = Secp256k1::new();
+
+        let desc = "wpkh(tprv8ZgxMBicQKsPdpkqS7Eair4YxjcuuvDPNYmKX3sCniCf16tHEVrjjiSXEkFRnUH77yXc6ZcwHHcLNfjdi5qUvw3VDfgYiH5mNsj5izuiu2N/1/2/*)#tqz0nc62"
+            .into_wallet_descriptor(&secp, Network::Testnet);
+        assert!(desc.is_ok());
+
+        let desc = "wpkh(tprv8ZgxMBicQKsPdpkqS7Eair4YxjcuuvDPNYmKX3sCniCf16tHEVrjjiSXEkFRnUH77yXc6ZcwHHcLNfjdi5qUvw3VDfgYiH5mNsj5izuiu2N/1/2/*)"
+            .into_wallet_descriptor(&secp, Network::Testnet);
+        assert!(desc.is_ok());
+
+        let desc = "wpkh(tpubD6NzVbkrYhZ4XHndKkuB8FifXm8r5FQHwrN6oZuWCz13qb93rtgKvD4PQsqC4HP4yhV3tA2fqr2RbY5mNXfM7RxXUoeABoDtsFUq2zJq6YK/1/2/*)#67ju93jw"
+            .into_wallet_descriptor(&secp, Network::Testnet);
+        assert!(desc.is_ok());
+
+        let desc = "wpkh(tpubD6NzVbkrYhZ4XHndKkuB8FifXm8r5FQHwrN6oZuWCz13qb93rtgKvD4PQsqC4HP4yhV3tA2fqr2RbY5mNXfM7RxXUoeABoDtsFUq2zJq6YK/1/2/*)"
+            .into_wallet_descriptor(&secp, Network::Testnet);
+        assert!(desc.is_ok());
+
+        let desc = "wpkh(tprv8ZgxMBicQKsPdpkqS7Eair4YxjcuuvDPNYmKX3sCniCf16tHEVrjjiSXEkFRnUH77yXc6ZcwHHcLNfjdi5qUvw3VDfgYiH5mNsj5izuiu2N/1/2/*)#67ju93jw"
+            .into_wallet_descriptor(&secp, Network::Testnet);
+        assert_matches!(desc, Err(DescriptorError::InvalidDescriptorChecksum));
+
+        let desc = "wpkh(tprv8ZgxMBicQKsPdpkqS7Eair4YxjcuuvDPNYmKX3sCniCf16tHEVrjjiSXEkFRnUH77yXc6ZcwHHcLNfjdi5qUvw3VDfgYiH5mNsj5izuiu2N/1/2/*)#67ju93jw"
+            .into_wallet_descriptor(&secp, Network::Testnet);
+        assert_matches!(desc, Err(DescriptorError::InvalidDescriptorChecksum));
+    }
+
+    // test IntoWalletDescriptor trait from &str with keys from right and wrong network
+    #[test]
+    fn test_descriptor_from_str_with_keys_network() {
+        let secp = Secp256k1::new();
+
+        let desc = "wpkh(tprv8ZgxMBicQKsPdpkqS7Eair4YxjcuuvDPNYmKX3sCniCf16tHEVrjjiSXEkFRnUH77yXc6ZcwHHcLNfjdi5qUvw3VDfgYiH5mNsj5izuiu2N/1/2/*)"
+            .into_wallet_descriptor(&secp, Network::Testnet);
+        assert!(desc.is_ok());
+
+        let desc = "wpkh(tprv8ZgxMBicQKsPdpkqS7Eair4YxjcuuvDPNYmKX3sCniCf16tHEVrjjiSXEkFRnUH77yXc6ZcwHHcLNfjdi5qUvw3VDfgYiH5mNsj5izuiu2N/1/2/*)"
+            .into_wallet_descriptor(&secp, Network::Regtest);
+        assert!(desc.is_ok());
+
+        let desc = "wpkh(tpubD6NzVbkrYhZ4XHndKkuB8FifXm8r5FQHwrN6oZuWCz13qb93rtgKvD4PQsqC4HP4yhV3tA2fqr2RbY5mNXfM7RxXUoeABoDtsFUq2zJq6YK/1/2/*)"
+            .into_wallet_descriptor(&secp, Network::Testnet);
+        assert!(desc.is_ok());
+
+        let desc = "wpkh(tpubD6NzVbkrYhZ4XHndKkuB8FifXm8r5FQHwrN6oZuWCz13qb93rtgKvD4PQsqC4HP4yhV3tA2fqr2RbY5mNXfM7RxXUoeABoDtsFUq2zJq6YK/1/2/*)"
+            .into_wallet_descriptor(&secp, Network::Regtest);
+        assert!(desc.is_ok());
+
+        let desc = "sh(wpkh(02864bb4ad00cefa806098a69e192bbda937494e69eb452b87bb3f20f6283baedb))"
+            .into_wallet_descriptor(&secp, Network::Testnet);
+        assert!(desc.is_ok());
+
+        let desc = "sh(wpkh(02864bb4ad00cefa806098a69e192bbda937494e69eb452b87bb3f20f6283baedb))"
+            .into_wallet_descriptor(&secp, Network::Bitcoin);
+        assert!(desc.is_ok());
+
+        let desc = "wpkh(tprv8ZgxMBicQKsPdpkqS7Eair4YxjcuuvDPNYmKX3sCniCf16tHEVrjjiSXEkFRnUH77yXc6ZcwHHcLNfjdi5qUvw3VDfgYiH5mNsj5izuiu2N/1/2/*)"
+            .into_wallet_descriptor(&secp, Network::Bitcoin);
+        assert_matches!(desc, Err(DescriptorError::Key(KeyError::InvalidNetwork)));
+
+        let desc = "wpkh(tpubD6NzVbkrYhZ4XHndKkuB8FifXm8r5FQHwrN6oZuWCz13qb93rtgKvD4PQsqC4HP4yhV3tA2fqr2RbY5mNXfM7RxXUoeABoDtsFUq2zJq6YK/1/2/*)"
+            .into_wallet_descriptor(&secp, Network::Bitcoin);
+        assert_matches!(desc, Err(DescriptorError::Key(KeyError::InvalidNetwork)));
+    }
+
+    // test IntoWalletDescriptor trait from the output of the descriptor!() macro
+    #[test]
+    fn test_descriptor_from_str_from_output_of_macro() {
+        let secp = Secp256k1::new();
+
+        let tpub = bip32::ExtendedPubKey::from_str("tpubD6NzVbkrYhZ4XHndKkuB8FifXm8r5FQHwrN6oZuWCz13qb93rtgKvD4PQsqC4HP4yhV3tA2fqr2RbY5mNXfM7RxXUoeABoDtsFUq2zJq6YK").unwrap();
+        let path = bip32::DerivationPath::from_str("m/1/2").unwrap();
+        let key = (tpub, path).into_descriptor_key().unwrap();
+
+        // make a descriptor out of it
+        let desc = crate::descriptor!(wpkh(key)).unwrap();
+
+        let (wallet_desc, _) = desc
+            .into_wallet_descriptor(&secp, Network::Testnet)
+            .unwrap();
+        let wallet_desc_str = wallet_desc.to_string();
+        assert_eq!(wallet_desc_str, "wpkh(tpubD6NzVbkrYhZ4XHndKkuB8FifXm8r5FQHwrN6oZuWCz13qb93rtgKvD4PQsqC4HP4yhV3tA2fqr2RbY5mNXfM7RxXUoeABoDtsFUq2zJq6YK/1/2/*)#67ju93jw");
+
+        let (wallet_desc2, _) = wallet_desc_str
+            .into_wallet_descriptor(&secp, Network::Testnet)
+            .unwrap();
+        assert_eq!(wallet_desc, wallet_desc2)
+    }
+
+    #[test]
+    fn test_into_wallet_descriptor_checked() {
+        let secp = Secp256k1::new();
+
+        let descriptor = "wpkh(tpubD6NzVbkrYhZ4XHndKkuB8FifXm8r5FQHwrN6oZuWCz13qb93rtgKvD4PQsqC4HP4yhV3tA2fqr2RbY5mNXfM7RxXUoeABoDtsFUq2zJq6YK/0'/1/2/*)";
+        let result = into_wallet_descriptor_checked(descriptor, &secp, Network::Testnet);
+
+        assert_matches!(result, Err(DescriptorError::HardenedDerivationXpub));
+
+        let descriptor = "wsh(multi(2,tpubD6NzVbkrYhZ4XHndKkuB8FifXm8r5FQHwrN6oZuWCz13qb93rtgKvD4PQsqC4HP4yhV3tA2fqr2RbY5mNXfM7RxXUoeABoDtsFUq2zJq6YK/0/*,tpubD6NzVbkrYhZ4XHndKkuB8FifXm8r5FQHwrN6oZuWCz13qb93rtgKvD4PQsqC4HP4yhV3tA2fqr2RbY5mNXfM7RxXUoeABoDtsFUq2zJq6YK/0/*))";
+        let result = into_wallet_descriptor_checked(descriptor, &secp, Network::Testnet);
+
+        assert!(result.is_err());
+    }
+
+    #[test]
+    fn test_sh_wsh_sortedmulti_redeemscript() {
+        use miniscript::psbt::PsbtInputExt;
+
+        let secp = Secp256k1::new();
+
+        let descriptor = "sh(wsh(sortedmulti(3,tpubDEsqS36T4DVsKJd9UH8pAKzrkGBYPLEt9jZMwpKtzh1G6mgYehfHt9WCgk7MJG5QGSFWf176KaBNoXbcuFcuadAFKxDpUdMDKGBha7bY3QM/0/*,tpubDF3cpwfs7fMvXXuoQbohXtLjNM6ehwYT287LWtmLsd4r77YLg6MZg4vTETx5MSJ2zkfigbYWu31VA2Z2Vc1cZugCYXgS7FQu6pE8V6TriEH/0/*,tpubDE1SKfcW76Tb2AASv5bQWMuScYNAdoqLHoexw13sNDXwmUhQDBbCD3QAedKGLhxMrWQdMDKENzYtnXPDRvexQPNuDrLj52wAjHhNEm8sJ4p/0/*,tpubDFLc6oXwJmhm3FGGzXkfJNTh2KitoY3WhmmQvuAjMhD8YbyWn5mAqckbxXfm2etM3p5J6JoTpSrMqRSTfMLtNW46poDaEZJ1kjd3csRSjwH/0/*,tpubDEWD9NBeWP59xXmdqSNt4VYdtTGwbpyP8WS962BuqpQeMZmX9Pur14dhXdZT5a7wR1pK6dPtZ9fP5WR493hPzemnBvkfLLYxnUjAKj1JCQV/0/*,tpubDEHyZkkwd7gZWCTgQuYQ9C4myF2hMEmyHsBCCmLssGqoqUxeT3gzohF5uEVURkf9TtmeepJgkSUmteac38FwZqirjApzNX59XSHLcwaTZCH/0/*,tpubDEqLouCekwnMUWN486kxGzD44qVgeyuqHyxUypNEiQt5RnUZNJe386TKPK99fqRV1vRkZjYAjtXGTECz98MCsdLcnkM67U6KdYRzVubeCgZ/0/*)))";
+        let (descriptor, _) =
+            into_wallet_descriptor_checked(descriptor, &secp, Network::Testnet).unwrap();
+
+        let descriptor = descriptor.at_derivation_index(0);
+
+        let script = Script::from_str("5321022f533b667e2ea3b36e21961c9fe9dca340fbe0af5210173a83ae0337ab20a57621026bb53a98e810bd0ee61a0ed1164ba6c024786d76554e793e202dc6ce9c78c4ea2102d5b8a7d66a41ffdb6f4c53d61994022e886b4f45001fb158b95c9164d45f8ca3210324b75eead2c1f9c60e8adeb5e7009fec7a29afcdb30d829d82d09562fe8bae8521032d34f8932200833487bd294aa219dcbe000b9f9b3d824799541430009f0fa55121037468f8ea99b6c64788398b5ad25480cad08f4b0d65be54ce3a55fd206b5ae4722103f72d3d96663b0ea99b0aeb0d7f273cab11a8de37885f1dddc8d9112adb87169357ae").unwrap();
+
+        let mut psbt_input = psbt::Input::default();
+        psbt_input
+            .update_with_descriptor_unchecked(&descriptor)
+            .unwrap();
+
+        assert_eq!(psbt_input.redeem_script, Some(script.to_v0_p2wsh()));
+        assert_eq!(psbt_input.witness_script, Some(script));
+    }
+}

crates/bdk/src/descriptor/template.rs

@@ -0,0 +1,985 @@
+// Bitcoin Dev Kit
+// Written in 2020 by Alekos Filini <alekos.filini@gmail.com>
+//
+// Copyright (c) 2020-2021 Bitcoin Dev Kit Developers
+//
+// This file is licensed under the Apache License, Version 2.0 <LICENSE-APACHE
+// or http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option.
+// You may not use this file except in accordance with one or both of these
+// licenses.
+
+//! Descriptor templates
+//!
+//! This module contains the definition of various common script templates that are ready to be
+//! used. See the documentation of each template for an example.
+
+use bitcoin::util::bip32;
+use bitcoin::Network;
+
+use miniscript::{Legacy, Segwitv0, Tap};
+
+use super::{ExtendedDescriptor, IntoWalletDescriptor, KeyMap};
+use crate::descriptor::DescriptorError;
+use crate::keys::{DerivableKey, IntoDescriptorKey, ValidNetworks};
+use crate::wallet::utils::SecpCtx;
+use crate::{descriptor, KeychainKind};
+
+/// Type alias for the return type of [`DescriptorTemplate`], [`descriptor!`](crate::descriptor!) and others
+pub type DescriptorTemplateOut = (ExtendedDescriptor, KeyMap, ValidNetworks);
+
+/// Trait for descriptor templates that can be built into a full descriptor
+///
+/// Since [`IntoWalletDescriptor`] is implemented for any [`DescriptorTemplate`], they can also be
+/// passed directly to the [`Wallet`](crate::Wallet) constructor.
+///
+/// ## Example
+///
+/// ```
+/// use bdk::descriptor::error::Error as DescriptorError;
+/// use bdk::keys::{IntoDescriptorKey, KeyError};
+/// use bdk::miniscript::Legacy;
+/// use bdk::template::{DescriptorTemplate, DescriptorTemplateOut};
+/// use bitcoin::Network;
+///
+/// struct MyP2PKH<K: IntoDescriptorKey<Legacy>>(K);
+///
+/// impl<K: IntoDescriptorKey<Legacy>> DescriptorTemplate for MyP2PKH<K> {
+///     fn build(self, network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+///         Ok(bdk::descriptor!(pkh(self.0))?)
+///     }
+/// }
+/// ```
+pub trait DescriptorTemplate {
+    /// Build the complete descriptor
+    fn build(self, network: Network) -> Result<DescriptorTemplateOut, DescriptorError>;
+}
+
+/// Turns a [`DescriptorTemplate`] into a valid wallet descriptor by calling its
+/// [`build`](DescriptorTemplate::build) method
+impl<T: DescriptorTemplate> IntoWalletDescriptor for T {
+    fn into_wallet_descriptor(
+        self,
+        secp: &SecpCtx,
+        network: Network,
+    ) -> Result<(ExtendedDescriptor, KeyMap), DescriptorError> {
+        self.build(network)?.into_wallet_descriptor(secp, network)
+    }
+}
+
+/// P2PKH template. Expands to a descriptor `pkh(key)`
+///
+/// ## Example
+///
+/// ```
+/// # use bdk::bitcoin::{PrivateKey, Network};
+/// # use bdk::Wallet;
+/// # use bdk::wallet::AddressIndex::New;
+/// use bdk::template::P2Pkh;
+///
+/// let key =
+///     bitcoin::PrivateKey::from_wif("cTc4vURSzdx6QE6KVynWGomDbLaA75dNALMNyfjh3p8DRRar84Um")?;
+/// let mut wallet = Wallet::new_no_persist(P2Pkh(key), None, Network::Testnet)?;
+///
+/// assert_eq!(
+///     wallet.get_address(New).to_string(),
+///     "mwJ8hxFYW19JLuc65RCTaP4v1rzVU8cVMT"
+/// );
+/// # Ok::<_, Box<dyn std::error::Error>>(())
+/// ```
+pub struct P2Pkh<K: IntoDescriptorKey<Legacy>>(pub K);
+
+impl<K: IntoDescriptorKey<Legacy>> DescriptorTemplate for P2Pkh<K> {
+    fn build(self, _network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+        descriptor!(pkh(self.0))
+    }
+}
+
+/// P2WPKH-P2SH template. Expands to a descriptor `sh(wpkh(key))`
+///
+/// ## Example
+///
+/// ```
+/// # use bdk::bitcoin::{PrivateKey, Network};
+/// # use bdk::Wallet;
+/// use bdk::template::P2Wpkh_P2Sh;
+/// use bdk::wallet::AddressIndex;
+///
+/// let key =
+///     bitcoin::PrivateKey::from_wif("cTc4vURSzdx6QE6KVynWGomDbLaA75dNALMNyfjh3p8DRRar84Um")?;
+/// let mut wallet = Wallet::new_no_persist(P2Wpkh_P2Sh(key), None, Network::Testnet)?;
+///
+/// assert_eq!(
+///     wallet.get_address(AddressIndex::New).to_string(),
+///     "2NB4ox5VDRw1ecUv6SnT3VQHPXveYztRqk5"
+/// );
+/// # Ok::<_, Box<dyn std::error::Error>>(())
+/// ```
+#[allow(non_camel_case_types)]
+pub struct P2Wpkh_P2Sh<K: IntoDescriptorKey<Segwitv0>>(pub K);
+
+impl<K: IntoDescriptorKey<Segwitv0>> DescriptorTemplate for P2Wpkh_P2Sh<K> {
+    fn build(self, _network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+        descriptor!(sh(wpkh(self.0)))
+    }
+}
+
+/// P2WPKH template. Expands to a descriptor `wpkh(key)`
+///
+/// ## Example
+///
+/// ```
+/// # use bdk::bitcoin::{PrivateKey, Network};
+/// # use bdk::{Wallet};
+/// use bdk::template::P2Wpkh;
+/// use bdk::wallet::AddressIndex::New;
+///
+/// let key =
+///     bitcoin::PrivateKey::from_wif("cTc4vURSzdx6QE6KVynWGomDbLaA75dNALMNyfjh3p8DRRar84Um")?;
+/// let mut wallet = Wallet::new_no_persist(P2Wpkh(key), None, Network::Testnet)?;
+///
+/// assert_eq!(
+///     wallet.get_address(New).to_string(),
+///     "tb1q4525hmgw265tl3drrl8jjta7ayffu6jf68ltjd"
+/// );
+/// # Ok::<_, Box<dyn std::error::Error>>(())
+/// ```
+pub struct P2Wpkh<K: IntoDescriptorKey<Segwitv0>>(pub K);
+
+impl<K: IntoDescriptorKey<Segwitv0>> DescriptorTemplate for P2Wpkh<K> {
+    fn build(self, _network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+        descriptor!(wpkh(self.0))
+    }
+}
+
+/// P2TR template. Expands to a descriptor `tr(key)`
+///
+/// ## Example
+///
+/// ```
+/// # use bdk::bitcoin::{PrivateKey, Network};
+/// # use bdk::Wallet;
+/// # use bdk::wallet::AddressIndex::New;
+/// use bdk::template::P2TR;
+///
+/// let key =
+///     bitcoin::PrivateKey::from_wif("cTc4vURSzdx6QE6KVynWGomDbLaA75dNALMNyfjh3p8DRRar84Um")?;
+/// let mut wallet = Wallet::new_no_persist(P2TR(key), None, Network::Testnet)?;
+///
+/// assert_eq!(
+///     wallet.get_address(New).to_string(),
+///     "tb1pvjf9t34fznr53u5tqhejz4nr69luzkhlvsdsdfq9pglutrpve2xq7hps46"
+/// );
+/// # Ok::<_, Box<dyn std::error::Error>>(())
+/// ```
+pub struct P2TR<K: IntoDescriptorKey<Tap>>(pub K);
+
+impl<K: IntoDescriptorKey<Tap>> DescriptorTemplate for P2TR<K> {
+    fn build(self, _network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+        descriptor!(tr(self.0))
+    }
+}
+
+/// BIP44 template. Expands to `pkh(key/44'/{0,1}'/0'/{0,1}/*)`
+///
+/// Since there are hardened derivation steps, this template requires a private derivable key (generally a `xprv`/`tprv`).
+///
+/// See [`Bip44Public`] for a template that can work with a `xpub`/`tpub`.
+///
+/// ## Example
+///
+/// ```
+/// # use std::str::FromStr;
+/// # use bdk::bitcoin::{PrivateKey, Network};
+/// # use bdk::{Wallet,  KeychainKind};
+/// # use bdk::wallet::AddressIndex::New;
+/// use bdk::template::Bip44;
+///
+/// let key = bitcoin::util::bip32::ExtendedPrivKey::from_str("tprv8ZgxMBicQKsPeZRHk4rTG6orPS2CRNFX3njhUXx5vj9qGog5ZMH4uGReDWN5kCkY3jmWEtWause41CDvBRXD1shKknAMKxT99o9qUTRVC6m")?;
+/// let mut wallet = Wallet::new_no_persist(
+///     Bip44(key.clone(), KeychainKind::External),
+///     Some(Bip44(key, KeychainKind::Internal)),
+///     Network::Testnet,
+/// )?;
+///
+/// assert_eq!(wallet.get_address(New).to_string(), "mmogjc7HJEZkrLqyQYqJmxUqFaC7i4uf89");
+/// assert_eq!(wallet.public_descriptor(KeychainKind::External).unwrap().to_string(), "pkh([c55b303f/44'/1'/0']tpubDCuorCpzvYS2LCD75BR46KHE8GdDeg1wsAgNZeNr6DaB5gQK1o14uErKwKLuFmeemkQ6N2m3rNgvctdJLyr7nwu2yia7413Hhg8WWE44cgT/0/*)#5wrnv0xt");
+/// # Ok::<_, Box<dyn std::error::Error>>(())
+/// ```
+pub struct Bip44<K: DerivableKey<Legacy>>(pub K, pub KeychainKind);
+
+impl<K: DerivableKey<Legacy>> DescriptorTemplate for Bip44<K> {
+    fn build(self, network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+        P2Pkh(legacy::make_bipxx_private(44, self.0, self.1, network)?).build(network)
+    }
+}
+
+/// BIP44 public template. Expands to `pkh(key/{0,1}/*)`
+///
+/// This assumes that the key used has already been derived with `m/44'/0'/0'` for Mainnet or `m/44'/1'/0'` for Testnet.
+///
+/// This template requires the parent fingerprint to populate correctly the metadata of PSBTs.
+///
+/// See [`Bip44`] for a template that does the full derivation, but requires private data
+/// for the key.
+///
+/// ## Example
+///
+/// ```
+/// # use std::str::FromStr;
+/// # use bdk::bitcoin::{PrivateKey, Network};
+/// # use bdk::{Wallet,  KeychainKind};
+/// # use bdk::wallet::AddressIndex::New;
+/// use bdk::template::Bip44Public;
+///
+/// let key = bitcoin::util::bip32::ExtendedPubKey::from_str("tpubDDDzQ31JkZB7VxUr9bjvBivDdqoFLrDPyLWtLapArAi51ftfmCb2DPxwLQzX65iNcXz1DGaVvyvo6JQ6rTU73r2gqdEo8uov9QKRb7nKCSU")?;
+/// let fingerprint = bitcoin::util::bip32::Fingerprint::from_str("c55b303f")?;
+/// let mut wallet = Wallet::new_no_persist(
+///     Bip44Public(key.clone(), fingerprint, KeychainKind::External),
+///     Some(Bip44Public(key, fingerprint, KeychainKind::Internal)),
+///     Network::Testnet,
+/// )?;
+///
+/// assert_eq!(wallet.get_address(New).to_string(), "miNG7dJTzJqNbFS19svRdTCisC65dsubtR");
+/// assert_eq!(wallet.public_descriptor(KeychainKind::External).unwrap().to_string(), "pkh([c55b303f/44'/1'/0']tpubDDDzQ31JkZB7VxUr9bjvBivDdqoFLrDPyLWtLapArAi51ftfmCb2DPxwLQzX65iNcXz1DGaVvyvo6JQ6rTU73r2gqdEo8uov9QKRb7nKCSU/0/*)#cfhumdqz");
+/// # Ok::<_, Box<dyn std::error::Error>>(())
+/// ```
+pub struct Bip44Public<K: DerivableKey<Legacy>>(pub K, pub bip32::Fingerprint, pub KeychainKind);
+
+impl<K: DerivableKey<Legacy>> DescriptorTemplate for Bip44Public<K> {
+    fn build(self, network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+        P2Pkh(legacy::make_bipxx_public(
+            44, self.0, self.1, self.2, network,
+        )?)
+        .build(network)
+    }
+}
+
+/// BIP49 template. Expands to `sh(wpkh(key/49'/{0,1}'/0'/{0,1}/*))`
+///
+/// Since there are hardened derivation steps, this template requires a private derivable key (generally a `xprv`/`tprv`).
+///
+/// See [`Bip49Public`] for a template that can work with a `xpub`/`tpub`.
+///
+/// ## Example
+///
+/// ```
+/// # use std::str::FromStr;
+/// # use bdk::bitcoin::{PrivateKey, Network};
+/// # use bdk::{Wallet,  KeychainKind};
+/// # use bdk::wallet::AddressIndex::New;
+/// use bdk::template::Bip49;
+///
+/// let key = bitcoin::util::bip32::ExtendedPrivKey::from_str("tprv8ZgxMBicQKsPeZRHk4rTG6orPS2CRNFX3njhUXx5vj9qGog5ZMH4uGReDWN5kCkY3jmWEtWause41CDvBRXD1shKknAMKxT99o9qUTRVC6m")?;
+/// let mut wallet = Wallet::new_no_persist(
+///     Bip49(key.clone(), KeychainKind::External),
+///     Some(Bip49(key, KeychainKind::Internal)),
+///     Network::Testnet,
+/// )?;
+///
+/// assert_eq!(wallet.get_address(New).to_string(), "2N4zkWAoGdUv4NXhSsU8DvS5MB36T8nKHEB");
+/// assert_eq!(wallet.public_descriptor(KeychainKind::External).unwrap().to_string(), "sh(wpkh([c55b303f/49'/1'/0']tpubDDYr4kdnZgjjShzYNjZUZXUUtpXaofdkMaipyS8ThEh45qFmhT4hKYways7UXmg6V7het1QiFo9kf4kYUXyDvV4rHEyvSpys9pjCB3pukxi/0/*))#s9vxlc8e");
+/// # Ok::<_, Box<dyn std::error::Error>>(())
+/// ```
+pub struct Bip49<K: DerivableKey<Segwitv0>>(pub K, pub KeychainKind);
+
+impl<K: DerivableKey<Segwitv0>> DescriptorTemplate for Bip49<K> {
+    fn build(self, network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+        P2Wpkh_P2Sh(segwit_v0::make_bipxx_private(49, self.0, self.1, network)?).build(network)
+    }
+}
+
+/// BIP49 public template. Expands to `sh(wpkh(key/{0,1}/*))`
+///
+/// This assumes that the key used has already been derived with `m/49'/0'/0'` for Mainnet or `m/49'/1'/0'` for Testnet.
+///
+/// This template requires the parent fingerprint to populate correctly the metadata of PSBTs.
+///
+/// See [`Bip49`] for a template that does the full derivation, but requires private data
+/// for the key.
+///
+/// ## Example
+///
+/// ```
+/// # use std::str::FromStr;
+/// # use bdk::bitcoin::{PrivateKey, Network};
+/// # use bdk::{Wallet,  KeychainKind};
+/// # use bdk::wallet::AddressIndex::New;
+/// use bdk::template::Bip49Public;
+///
+/// let key = bitcoin::util::bip32::ExtendedPubKey::from_str("tpubDC49r947KGK52X5rBWS4BLs5m9SRY3pYHnvRrm7HcybZ3BfdEsGFyzCMzayi1u58eT82ZeyFZwH7DD6Q83E3fM9CpfMtmnTygnLfP59jL9L")?;
+/// let fingerprint = bitcoin::util::bip32::Fingerprint::from_str("c55b303f")?;
+/// let mut wallet = Wallet::new_no_persist(
+///     Bip49Public(key.clone(), fingerprint, KeychainKind::External),
+///     Some(Bip49Public(key, fingerprint, KeychainKind::Internal)),
+///     Network::Testnet,
+/// )?;
+///
+/// assert_eq!(wallet.get_address(New).to_string(), "2N3K4xbVAHoiTQSwxkZjWDfKoNC27pLkYnt");
+/// assert_eq!(wallet.public_descriptor(KeychainKind::External).unwrap().to_string(), "sh(wpkh([c55b303f/49'/1'/0']tpubDC49r947KGK52X5rBWS4BLs5m9SRY3pYHnvRrm7HcybZ3BfdEsGFyzCMzayi1u58eT82ZeyFZwH7DD6Q83E3fM9CpfMtmnTygnLfP59jL9L/0/*))#3tka9g0q");
+/// # Ok::<_, Box<dyn std::error::Error>>(())
+/// ```
+pub struct Bip49Public<K: DerivableKey<Segwitv0>>(pub K, pub bip32::Fingerprint, pub KeychainKind);
+
+impl<K: DerivableKey<Segwitv0>> DescriptorTemplate for Bip49Public<K> {
+    fn build(self, network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+        P2Wpkh_P2Sh(segwit_v0::make_bipxx_public(
+            49, self.0, self.1, self.2, network,
+        )?)
+        .build(network)
+    }
+}
+
+/// BIP84 template. Expands to `wpkh(key/84'/{0,1}'/0'/{0,1}/*)`
+///
+/// Since there are hardened derivation steps, this template requires a private derivable key (generally a `xprv`/`tprv`).
+///
+/// See [`Bip84Public`] for a template that can work with a `xpub`/`tpub`.
+///
+/// ## Example
+///
+/// ```
+/// # use std::str::FromStr;
+/// # use bdk::bitcoin::{PrivateKey, Network};
+/// # use bdk::{Wallet,  KeychainKind};
+/// # use bdk::wallet::AddressIndex::New;
+/// use bdk::template::Bip84;
+///
+/// let key = bitcoin::util::bip32::ExtendedPrivKey::from_str("tprv8ZgxMBicQKsPeZRHk4rTG6orPS2CRNFX3njhUXx5vj9qGog5ZMH4uGReDWN5kCkY3jmWEtWause41CDvBRXD1shKknAMKxT99o9qUTRVC6m")?;
+/// let mut wallet = Wallet::new_no_persist(
+///     Bip84(key.clone(), KeychainKind::External),
+///     Some(Bip84(key, KeychainKind::Internal)),
+///     Network::Testnet,
+/// )?;
+///
+/// assert_eq!(wallet.get_address(New).to_string(), "tb1qhl85z42h7r4su5u37rvvw0gk8j2t3n9y7zsg4n");
+/// assert_eq!(wallet.public_descriptor(KeychainKind::External).unwrap().to_string(), "wpkh([c55b303f/84'/1'/0']tpubDDc5mum24DekpNw92t6fHGp8Gr2JjF9J7i4TZBtN6Vp8xpAULG5CFaKsfugWa5imhrQQUZKXe261asP5koDHo5bs3qNTmf3U3o4v9SaB8gg/0/*)#6kfecsmr");
+/// # Ok::<_, Box<dyn std::error::Error>>(())
+/// ```
+pub struct Bip84<K: DerivableKey<Segwitv0>>(pub K, pub KeychainKind);
+
+impl<K: DerivableKey<Segwitv0>> DescriptorTemplate for Bip84<K> {
+    fn build(self, network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+        P2Wpkh(segwit_v0::make_bipxx_private(84, self.0, self.1, network)?).build(network)
+    }
+}
+
+/// BIP84 public template. Expands to `wpkh(key/{0,1}/*)`
+///
+/// This assumes that the key used has already been derived with `m/84'/0'/0'` for Mainnet or `m/84'/1'/0'` for Testnet.
+///
+/// This template requires the parent fingerprint to populate correctly the metadata of PSBTs.
+///
+/// See [`Bip84`] for a template that does the full derivation, but requires private data
+/// for the key.
+///
+/// ## Example
+///
+/// ```
+/// # use std::str::FromStr;
+/// # use bdk::bitcoin::{PrivateKey, Network};
+/// # use bdk::{Wallet,  KeychainKind};
+/// # use bdk::wallet::AddressIndex::New;
+/// use bdk::template::Bip84Public;
+///
+/// let key = bitcoin::util::bip32::ExtendedPubKey::from_str("tpubDC2Qwo2TFsaNC4ju8nrUJ9mqVT3eSgdmy1yPqhgkjwmke3PRXutNGRYAUo6RCHTcVQaDR3ohNU9we59brGHuEKPvH1ags2nevW5opEE9Z5Q")?;
+/// let fingerprint = bitcoin::util::bip32::Fingerprint::from_str("c55b303f")?;
+/// let mut wallet = Wallet::new_no_persist(
+///     Bip84Public(key.clone(), fingerprint, KeychainKind::External),
+///     Some(Bip84Public(key, fingerprint, KeychainKind::Internal)),
+///     Network::Testnet,
+/// )?;
+///
+/// assert_eq!(wallet.get_address(New).to_string(), "tb1qedg9fdlf8cnnqfd5mks6uz5w4kgpk2pr6y4qc7");
+/// assert_eq!(wallet.public_descriptor(KeychainKind::External).unwrap().to_string(), "wpkh([c55b303f/84'/1'/0']tpubDC2Qwo2TFsaNC4ju8nrUJ9mqVT3eSgdmy1yPqhgkjwmke3PRXutNGRYAUo6RCHTcVQaDR3ohNU9we59brGHuEKPvH1ags2nevW5opEE9Z5Q/0/*)#dhu402yv");
+/// # Ok::<_, Box<dyn std::error::Error>>(())
+/// ```
+pub struct Bip84Public<K: DerivableKey<Segwitv0>>(pub K, pub bip32::Fingerprint, pub KeychainKind);
+
+impl<K: DerivableKey<Segwitv0>> DescriptorTemplate for Bip84Public<K> {
+    fn build(self, network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+        P2Wpkh(segwit_v0::make_bipxx_public(
+            84, self.0, self.1, self.2, network,
+        )?)
+        .build(network)
+    }
+}
+
+/// BIP86 template. Expands to `tr(key/86'/{0,1}'/0'/{0,1}/*)`
+///
+/// Since there are hardened derivation steps, this template requires a private derivable key (generally a `xprv`/`tprv`).
+///
+/// See [`Bip86Public`] for a template that can work with a `xpub`/`tpub`.
+///
+/// ## Example
+///
+/// ```
+/// # use std::str::FromStr;
+/// # use bdk::bitcoin::{PrivateKey, Network};
+/// # use bdk::{Wallet,  KeychainKind};
+/// # use bdk::wallet::AddressIndex::New;
+/// use bdk::template::Bip86;
+///
+/// let key = bitcoin::util::bip32::ExtendedPrivKey::from_str("tprv8ZgxMBicQKsPeZRHk4rTG6orPS2CRNFX3njhUXx5vj9qGog5ZMH4uGReDWN5kCkY3jmWEtWause41CDvBRXD1shKknAMKxT99o9qUTRVC6m")?;
+/// let mut wallet = Wallet::new_no_persist(
+///     Bip86(key.clone(), KeychainKind::External),
+///     Some(Bip86(key, KeychainKind::Internal)),
+///     Network::Testnet,
+/// )?;
+///
+/// assert_eq!(wallet.get_address(New).to_string(), "tb1p5unlj09djx8xsjwe97269kqtxqpwpu2epeskgqjfk4lnf69v4tnqpp35qu");
+/// assert_eq!(wallet.public_descriptor(KeychainKind::External).unwrap().to_string(), "tr([c55b303f/86'/1'/0']tpubDCiHofpEs47kx358bPdJmTZHmCDqQ8qw32upCSxHrSEdeeBs2T5Mq6QMB2ukeMqhNBiyhosBvJErteVhfURPGXPv3qLJPw5MVpHUewsbP2m/0/*)#dkgvr5hm");
+/// # Ok::<_, Box<dyn std::error::Error>>(())
+/// ```
+pub struct Bip86<K: DerivableKey<Tap>>(pub K, pub KeychainKind);
+
+impl<K: DerivableKey<Tap>> DescriptorTemplate for Bip86<K> {
+    fn build(self, network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+        P2TR(segwit_v1::make_bipxx_private(86, self.0, self.1, network)?).build(network)
+    }
+}
+
+/// BIP86 public template. Expands to `tr(key/{0,1}/*)`
+///
+/// This assumes that the key used has already been derived with `m/86'/0'/0'` for Mainnet or `m/86'/1'/0'` for Testnet.
+///
+/// This template requires the parent fingerprint to populate correctly the metadata of PSBTs.
+///
+/// See [`Bip86`] for a template that does the full derivation, but requires private data
+/// for the key.
+///
+/// ## Example
+///
+/// ```
+/// # use std::str::FromStr;
+/// # use bdk::bitcoin::{PrivateKey, Network};
+/// # use bdk::{Wallet,  KeychainKind};
+/// # use bdk::wallet::AddressIndex::New;
+/// use bdk::template::Bip86Public;
+///
+/// let key = bitcoin::util::bip32::ExtendedPubKey::from_str("tpubDC2Qwo2TFsaNC4ju8nrUJ9mqVT3eSgdmy1yPqhgkjwmke3PRXutNGRYAUo6RCHTcVQaDR3ohNU9we59brGHuEKPvH1ags2nevW5opEE9Z5Q")?;
+/// let fingerprint = bitcoin::util::bip32::Fingerprint::from_str("c55b303f")?;
+/// let mut wallet = Wallet::new_no_persist(
+///     Bip86Public(key.clone(), fingerprint, KeychainKind::External),
+///     Some(Bip86Public(key, fingerprint, KeychainKind::Internal)),
+///     Network::Testnet,
+/// )?;
+///
+/// assert_eq!(wallet.get_address(New).to_string(), "tb1pwjp9f2k5n0xq73ecuu0c5njvgqr3vkh7yaylmpqvsuuaafymh0msvcmh37");
+/// assert_eq!(wallet.public_descriptor(KeychainKind::External).unwrap().to_string(), "tr([c55b303f/86'/1'/0']tpubDC2Qwo2TFsaNC4ju8nrUJ9mqVT3eSgdmy1yPqhgkjwmke3PRXutNGRYAUo6RCHTcVQaDR3ohNU9we59brGHuEKPvH1ags2nevW5opEE9Z5Q/0/*)#2p65srku");
+/// # Ok::<_, Box<dyn std::error::Error>>(())
+/// ```
+pub struct Bip86Public<K: DerivableKey<Tap>>(pub K, pub bip32::Fingerprint, pub KeychainKind);
+
+impl<K: DerivableKey<Tap>> DescriptorTemplate for Bip86Public<K> {
+    fn build(self, network: Network) -> Result<DescriptorTemplateOut, DescriptorError> {
+        P2TR(segwit_v1::make_bipxx_public(
+            86, self.0, self.1, self.2, network,
+        )?)
+        .build(network)
+    }
+}
+
+macro_rules! expand_make_bipxx {
+    ( $mod_name:ident, $ctx:ty ) => {
+        mod $mod_name {
+            use super::*;
+
+            pub(super) fn make_bipxx_private<K: DerivableKey<$ctx>>(
+                bip: u32,
+                key: K,
+                keychain: KeychainKind,
+                network: Network,
+            ) -> Result<impl IntoDescriptorKey<$ctx>, DescriptorError> {
+                let mut derivation_path = alloc::vec::Vec::with_capacity(4);
+                derivation_path.push(bip32::ChildNumber::from_hardened_idx(bip)?);
+
+                match network {
+                    Network::Bitcoin => {
+                        derivation_path.push(bip32::ChildNumber::from_hardened_idx(0)?);
+                    }
+                    _ => {
+                        derivation_path.push(bip32::ChildNumber::from_hardened_idx(1)?);
+                    }
+                }
+                derivation_path.push(bip32::ChildNumber::from_hardened_idx(0)?);
+
+                match keychain {
+                    KeychainKind::External => {
+                        derivation_path.push(bip32::ChildNumber::from_normal_idx(0)?)
+                    }
+                    KeychainKind::Internal => {
+                        derivation_path.push(bip32::ChildNumber::from_normal_idx(1)?)
+                    }
+                };
+
+                let derivation_path: bip32::DerivationPath = derivation_path.into();
+
+                Ok((key, derivation_path))
+            }
+            pub(super) fn make_bipxx_public<K: DerivableKey<$ctx>>(
+                bip: u32,
+                key: K,
+                parent_fingerprint: bip32::Fingerprint,
+                keychain: KeychainKind,
+                network: Network,
+            ) -> Result<impl IntoDescriptorKey<$ctx>, DescriptorError> {
+                let derivation_path: bip32::DerivationPath = match keychain {
+                    KeychainKind::External => vec![bip32::ChildNumber::from_normal_idx(0)?].into(),
+                    KeychainKind::Internal => vec![bip32::ChildNumber::from_normal_idx(1)?].into(),
+                };
+
+                let source_path = bip32::DerivationPath::from(vec![
+                    bip32::ChildNumber::from_hardened_idx(bip)?,
+                    match network {
+                        Network::Bitcoin => bip32::ChildNumber::from_hardened_idx(0)?,
+                        _ => bip32::ChildNumber::from_hardened_idx(1)?,
+                    },
+                    bip32::ChildNumber::from_hardened_idx(0)?,
+                ]);
+
+                Ok((key, (parent_fingerprint, source_path), derivation_path))
+            }
+        }
+    };
+}
+
+expand_make_bipxx!(legacy, Legacy);
+expand_make_bipxx!(segwit_v0, Segwitv0);
+expand_make_bipxx!(segwit_v1, Tap);
+
+#[cfg(test)]
+mod test {
+    // test existing descriptor templates, make sure they are expanded to the right descriptors
+
+    use alloc::{string::ToString, vec::Vec};
+    use core::str::FromStr;
+
+    use super::*;
+    use crate::descriptor::{DescriptorError, DescriptorMeta};
+    use crate::keys::ValidNetworks;
+    use assert_matches::assert_matches;
+    use miniscript::descriptor::{DescriptorPublicKey, KeyMap};
+    use miniscript::Descriptor;
+
+    // BIP44 `pkh(key/44'/{0,1}'/0'/{0,1}/*)`
+    #[test]
+    fn test_bip44_template_cointype() {
+        use bitcoin::util::bip32::ChildNumber::{self, Hardened};
+
+        let xprvkey = bitcoin::util::bip32::ExtendedPrivKey::from_str("xprv9s21ZrQH143K2fpbqApQL69a4oKdGVnVN52R82Ft7d1pSqgKmajF62acJo3aMszZb6qQ22QsVECSFxvf9uyxFUvFYQMq3QbtwtRSMjLAhMf").unwrap();
+        assert_eq!(Network::Bitcoin, xprvkey.network);
+        let xdesc = Bip44(xprvkey, KeychainKind::Internal)
+            .build(Network::Bitcoin)
+            .unwrap();
+
+        if let ExtendedDescriptor::Pkh(pkh) = xdesc.0 {
+            let path: Vec<ChildNumber> = pkh.into_inner().full_derivation_path().into();
+            let purpose = path.get(0).unwrap();
+            assert_matches!(purpose, Hardened { index: 44 });
+            let coin_type = path.get(1).unwrap();
+            assert_matches!(coin_type, Hardened { index: 0 });
+        }
+
+        let tprvkey = bitcoin::util::bip32::ExtendedPrivKey::from_str("tprv8ZgxMBicQKsPcx5nBGsR63Pe8KnRUqmbJNENAfGftF3yuXoMMoVJJcYeUw5eVkm9WBPjWYt6HMWYJNesB5HaNVBaFc1M6dRjWSYnmewUMYy").unwrap();
+        assert_eq!(Network::Testnet, tprvkey.network);
+        let tdesc = Bip44(tprvkey, KeychainKind::Internal)
+            .build(Network::Testnet)
+            .unwrap();
+
+        if let ExtendedDescriptor::Pkh(pkh) = tdesc.0 {
+            let path: Vec<ChildNumber> = pkh.into_inner().full_derivation_path().into();
+            let purpose = path.get(0).unwrap();
+            assert_matches!(purpose, Hardened { index: 44 });
+            let coin_type = path.get(1).unwrap();
+            assert_matches!(coin_type, Hardened { index: 1 });
+        }
+    }
+
+    // verify template descriptor generates expected address(es)
+    fn check(
+        desc: Result<(Descriptor<DescriptorPublicKey>, KeyMap, ValidNetworks), DescriptorError>,
+        is_witness: bool,
+        is_taproot: bool,
+        is_fixed: bool,
+        network: Network,
+        expected: &[&str],
+    ) {
+        let (desc, _key_map, _networks) = desc.unwrap();
+        assert_eq!(desc.is_witness(), is_witness);
+        assert_eq!(desc.is_taproot(), is_taproot);
+        assert_eq!(!desc.has_wildcard(), is_fixed);
+        for i in 0..expected.len() {
+            let index = i as u32;
+            let child_desc = if !desc.has_wildcard() {
+                desc.at_derivation_index(0)
+            } else {
+                desc.at_derivation_index(index)
+            };
+            let address = child_desc.address(network).unwrap();
+            assert_eq!(address.to_string(), *expected.get(i).unwrap());
+        }
+    }
+
+    // P2PKH
+    #[test]
+    fn test_p2ph_template() {
+        let prvkey =
+            bitcoin::PrivateKey::from_wif("cTc4vURSzdx6QE6KVynWGomDbLaA75dNALMNyfjh3p8DRRar84Um")
+                .unwrap();
+        check(
+            P2Pkh(prvkey).build(Network::Bitcoin),
+            false,
+            false,
+            true,
+            Network::Regtest,
+            &["mwJ8hxFYW19JLuc65RCTaP4v1rzVU8cVMT"],
+        );
+
+        let pubkey = bitcoin::PublicKey::from_str(
+            "03a34b99f22c790c4e36b2b3c2c35a36db06226e41c692fc82b8b56ac1c540c5bd",
+        )
+        .unwrap();
+        check(
+            P2Pkh(pubkey).build(Network::Bitcoin),
+            false,
+            false,
+            true,
+            Network::Regtest,
+            &["muZpTpBYhxmRFuCjLc7C6BBDF32C8XVJUi"],
+        );
+    }
+
+    // P2WPKH-P2SH `sh(wpkh(key))`
+    #[test]
+    fn test_p2wphp2sh_template() {
+        let prvkey =
+            bitcoin::PrivateKey::from_wif("cTc4vURSzdx6QE6KVynWGomDbLaA75dNALMNyfjh3p8DRRar84Um")
+                .unwrap();
+        check(
+            P2Wpkh_P2Sh(prvkey).build(Network::Bitcoin),
+            true,
+            false,
+            true,
+            Network::Regtest,
+            &["2NB4ox5VDRw1ecUv6SnT3VQHPXveYztRqk5"],
+        );
+
+        let pubkey = bitcoin::PublicKey::from_str(
+            "03a34b99f22c790c4e36b2b3c2c35a36db06226e41c692fc82b8b56ac1c540c5bd",
+        )
+        .unwrap();
+        check(
+            P2Wpkh_P2Sh(pubkey).build(Network::Bitcoin),
+            true,
+            false,
+            true,
+            Network::Regtest,
+            &["2N5LiC3CqzxDamRTPG1kiNv1FpNJQ7x28sb"],
+        );
+    }
+
+    // P2WPKH `wpkh(key)`
+    #[test]
+    fn test_p2wph_template() {
+        let prvkey =
+            bitcoin::PrivateKey::from_wif("cTc4vURSzdx6QE6KVynWGomDbLaA75dNALMNyfjh3p8DRRar84Um")
+                .unwrap();
+        check(
+            P2Wpkh(prvkey).build(Network::Bitcoin),
+            true,
+            false,
+            true,
+            Network::Regtest,
+            &["bcrt1q4525hmgw265tl3drrl8jjta7ayffu6jfcwxx9y"],
+        );
+
+        let pubkey = bitcoin::PublicKey::from_str(
+            "03a34b99f22c790c4e36b2b3c2c35a36db06226e41c692fc82b8b56ac1c540c5bd",
+        )
+        .unwrap();
+        check(
+            P2Wpkh(pubkey).build(Network::Bitcoin),
+            true,
+            false,
+            true,
+            Network::Regtest,
+            &["bcrt1qngw83fg8dz0k749cg7k3emc7v98wy0c7azaa6h"],
+        );
+    }
+
+    // P2TR `tr(key)`
+    #[test]
+    fn test_p2tr_template() {
+        let prvkey =
+            bitcoin::PrivateKey::from_wif("cTc4vURSzdx6QE6KVynWGomDbLaA75dNALMNyfjh3p8DRRar84Um")
+                .unwrap();
+        check(
+            P2TR(prvkey).build(Network::Bitcoin),
+            false,
+            true,
+            true,
+            Network::Regtest,
+            &["bcrt1pvjf9t34fznr53u5tqhejz4nr69luzkhlvsdsdfq9pglutrpve2xqnwtkqq"],
+        );
+
+        let pubkey = bitcoin::PublicKey::from_str(
+            "03a34b99f22c790c4e36b2b3c2c35a36db06226e41c692fc82b8b56ac1c540c5bd",
+        )
+        .unwrap();
+        check(
+            P2TR(pubkey).build(Network::Bitcoin),
+            false,
+            true,
+            true,
+            Network::Regtest,
+            &["bcrt1pw74tdcrxlzn5r8z6ku2vztr86fgq0m245s72mjktf4afwzsf8ugs4evwdf"],
+        );
+    }
+
+    // BIP44 `pkh(key/44'/0'/0'/{0,1}/*)`
+    #[test]
+    fn test_bip44_template() {
+        let prvkey = bitcoin::util::bip32::ExtendedPrivKey::from_str("tprv8ZgxMBicQKsPcx5nBGsR63Pe8KnRUqmbJNENAfGftF3yuXoMMoVJJcYeUw5eVkm9WBPjWYt6HMWYJNesB5HaNVBaFc1M6dRjWSYnmewUMYy").unwrap();
+        check(
+            Bip44(prvkey, KeychainKind::External).build(Network::Bitcoin),
+            false,
+            false,
+            false,
+            Network::Regtest,
+            &[
+                "n453VtnjDHPyDt2fDstKSu7A3YCJoHZ5g5",
+                "mvfrrumXgTtwFPWDNUecBBgzuMXhYM7KRP",
+                "mzYvhRAuQqbdSKMVVzXNYyqihgNdRadAUQ",
+            ],
+        );
+        check(
+            Bip44(prvkey, KeychainKind::Internal).build(Network::Bitcoin),
+            false,
+            false,
+            false,
+            Network::Regtest,
+            &[
+                "muHF98X9KxEzdKrnFAX85KeHv96eXopaip",
+                "n4hpyLJE5ub6B5Bymv4eqFxS5KjrewSmYR",
+                "mgvkdv1ffmsXd2B1sRKQ5dByK3SzpG42rA",
+            ],
+        );
+    }
+
+    // BIP44 public `pkh(key/{0,1}/*)`
+    #[test]
+    fn test_bip44_public_template() {
+        let pubkey = bitcoin::util::bip32::ExtendedPubKey::from_str("tpubDDDzQ31JkZB7VxUr9bjvBivDdqoFLrDPyLWtLapArAi51ftfmCb2DPxwLQzX65iNcXz1DGaVvyvo6JQ6rTU73r2gqdEo8uov9QKRb7nKCSU").unwrap();
+        let fingerprint = bitcoin::util::bip32::Fingerprint::from_str("c55b303f").unwrap();
+        check(
+            Bip44Public(pubkey, fingerprint, KeychainKind::External).build(Network::Bitcoin),
+            false,
+            false,
+            false,
+            Network::Regtest,
+            &[
+                "miNG7dJTzJqNbFS19svRdTCisC65dsubtR",
+                "n2UqaDbCjWSFJvpC84m3FjUk5UaeibCzYg",
+                "muCPpS6Ue7nkzeJMWDViw7Lkwr92Yc4K8g",
+            ],
+        );
+        check(
+            Bip44Public(pubkey, fingerprint, KeychainKind::Internal).build(Network::Bitcoin),
+            false,
+            false,
+            false,
+            Network::Regtest,
+            &[
+                "moDr3vJ8wpt5nNxSK55MPq797nXJb2Ru9H",
+                "ms7A1Yt4uTezT2XkefW12AvLoko8WfNJMG",
+                "mhYiyat2rtEnV77cFfQsW32y1m2ceCGHPo",
+            ],
+        );
+    }
+
+    // BIP49 `sh(wpkh(key/49'/0'/0'/{0,1}/*))`
+    #[test]
+    fn test_bip49_template() {
+        let prvkey = bitcoin::util::bip32::ExtendedPrivKey::from_str("tprv8ZgxMBicQKsPcx5nBGsR63Pe8KnRUqmbJNENAfGftF3yuXoMMoVJJcYeUw5eVkm9WBPjWYt6HMWYJNesB5HaNVBaFc1M6dRjWSYnmewUMYy").unwrap();
+        check(
+            Bip49(prvkey, KeychainKind::External).build(Network::Bitcoin),
+            true,
+            false,
+            false,
+            Network::Regtest,
+            &[
+                "2N9bCAJXGm168MjVwpkBdNt6ucka3PKVoUV",
+                "2NDckYkqrYyDMtttEav5hB3Bfw9EGAW5HtS",
+                "2NAFTVtksF9T4a97M7nyCjwUBD24QevZ5Z4",
+            ],
+        );
+        check(
+            Bip49(prvkey, KeychainKind::Internal).build(Network::Bitcoin),
+            true,
+            false,
+            false,
+            Network::Regtest,
+            &[
+                "2NB3pA8PnzJLGV8YEKNDFpbViZv3Bm1K6CG",
+                "2NBiX2Wzxngb5rPiWpUiJQ2uLVB4HBjFD4p",
+                "2NA8ek4CdQ6aMkveYF6AYuEYNrftB47QGTn",
+            ],
+        );
+    }
+
+    // BIP49 public `sh(wpkh(key/{0,1}/*))`
+    #[test]
+    fn test_bip49_public_template() {
+        let pubkey = bitcoin::util::bip32::ExtendedPubKey::from_str("tpubDC49r947KGK52X5rBWS4BLs5m9SRY3pYHnvRrm7HcybZ3BfdEsGFyzCMzayi1u58eT82ZeyFZwH7DD6Q83E3fM9CpfMtmnTygnLfP59jL9L").unwrap();
+        let fingerprint = bitcoin::util::bip32::Fingerprint::from_str("c55b303f").unwrap();
+        check(
+            Bip49Public(pubkey, fingerprint, KeychainKind::External).build(Network::Bitcoin),
+            true,
+            false,
+            false,
+            Network::Regtest,
+            &[
+                "2N3K4xbVAHoiTQSwxkZjWDfKoNC27pLkYnt",
+                "2NCTQfJ1sZa3wQ3pPseYRHbaNEpC3AquEfX",
+                "2MveFxAuC8BYPzTybx7FxSzW8HSd8ATT4z7",
+            ],
+        );
+        check(
+            Bip49Public(pubkey, fingerprint, KeychainKind::Internal).build(Network::Bitcoin),
+            true,
+            false,
+            false,
+            Network::Regtest,
+            &[
+                "2NF2vttKibwyxigxtx95Zw8K7JhDbo5zPVJ",
+                "2Mtmyd8taksxNVWCJ4wVvaiss7QPZGcAJuH",
+                "2NBs3CTVYPr1HCzjB4YFsnWCPCtNg8uMEfp",
+            ],
+        );
+    }
+
+    // BIP84 `wpkh(key/84'/0'/0'/{0,1}/*)`
+    #[test]
+    fn test_bip84_template() {
+        let prvkey = bitcoin::util::bip32::ExtendedPrivKey::from_str("tprv8ZgxMBicQKsPcx5nBGsR63Pe8KnRUqmbJNENAfGftF3yuXoMMoVJJcYeUw5eVkm9WBPjWYt6HMWYJNesB5HaNVBaFc1M6dRjWSYnmewUMYy").unwrap();
+        check(
+            Bip84(prvkey, KeychainKind::External).build(Network::Bitcoin),
+            true,
+            false,
+            false,
+            Network::Regtest,
+            &[
+                "bcrt1qkmvk2nadgplmd57ztld8nf8v2yxkzmdvwtjf8s",
+                "bcrt1qx0v6zgfwe50m4kqc58cqzcyem7ay2sfl3gvqhp",
+                "bcrt1q4h7fq9zhxst6e69p3n882nfj649l7w9g3zccfp",
+            ],
+        );
+        check(
+            Bip84(prvkey, KeychainKind::Internal).build(Network::Bitcoin),
+            true,
+            false,
+            false,
+            Network::Regtest,
+            &[
+                "bcrt1qtrwtz00wxl69e5xex7amy4xzlxkaefg3gfdkxa",
+                "bcrt1qqqasfhxpkkf7zrxqnkr2sfhn74dgsrc3e3ky45",
+                "bcrt1qpks7n0gq74hsgsz3phn5vuazjjq0f5eqhsgyce",
+            ],
+        );
+    }
+
+    // BIP84 public `wpkh(key/{0,1}/*)`
+    #[test]
+    fn test_bip84_public_template() {
+        let pubkey = bitcoin::util::bip32::ExtendedPubKey::from_str("tpubDC2Qwo2TFsaNC4ju8nrUJ9mqVT3eSgdmy1yPqhgkjwmke3PRXutNGRYAUo6RCHTcVQaDR3ohNU9we59brGHuEKPvH1ags2nevW5opEE9Z5Q").unwrap();
+        let fingerprint = bitcoin::util::bip32::Fingerprint::from_str("c55b303f").unwrap();
+        check(
+            Bip84Public(pubkey, fingerprint, KeychainKind::External).build(Network::Bitcoin),
+            true,
+            false,
+            false,
+            Network::Regtest,
+            &[
+                "bcrt1qedg9fdlf8cnnqfd5mks6uz5w4kgpk2prcdvd0h",
+                "bcrt1q3lncdlwq3lgcaaeyruynjnlccr0ve0kakh6ana",
+                "bcrt1qt9800y6xl3922jy3uyl0z33jh5wfpycyhcylr9",
+            ],
+        );
+        check(
+            Bip84Public(pubkey, fingerprint, KeychainKind::Internal).build(Network::Bitcoin),
+            true,
+            false,
+            false,
+            Network::Regtest,
+            &[
+                "bcrt1qm6wqukenh7guu792lj2njgw9n78cmwsy8xy3z2",
+                "bcrt1q694twxtjn4nnrvnyvra769j0a23rllj5c6cgwp",
+                "bcrt1qhlac3c5ranv5w5emlnqs7wxhkxt8maelylcarp",
+            ],
+        );
+    }
+
+    // BIP86 `tr(key/86'/0'/0'/{0,1}/*)`
+    // Used addresses in test vector in https://github.com/bitcoin/bips/blob/master/bip-0086.mediawiki
+    #[test]
+    fn test_bip86_template() {
+        let prvkey = bitcoin::util::bip32::ExtendedPrivKey::from_str("xprv9s21ZrQH143K3GJpoapnV8SFfukcVBSfeCficPSGfubmSFDxo1kuHnLisriDvSnRRuL2Qrg5ggqHKNVpxR86QEC8w35uxmGoggxtQTPvfUu").unwrap();
+        check(
+            Bip86(prvkey, KeychainKind::External).build(Network::Bitcoin),
+            false,
+            true,
+            false,
+            Network::Bitcoin,
+            &[
+                "bc1p5cyxnuxmeuwuvkwfem96lqzszd02n6xdcjrs20cac6yqjjwudpxqkedrcr",
+                "bc1p4qhjn9zdvkux4e44uhx8tc55attvtyu358kutcqkudyccelu0was9fqzwh",
+                "bc1p0d0rhyynq0awa9m8cqrcr8f5nxqx3aw29w4ru5u9my3h0sfygnzs9khxz8",
+            ],
+        );
+        check(
+            Bip86(prvkey, KeychainKind::Internal).build(Network::Bitcoin),
+            false,
+            true,
+            false,
+            Network::Bitcoin,
+            &[
+                "bc1p3qkhfews2uk44qtvauqyr2ttdsw7svhkl9nkm9s9c3x4ax5h60wqwruhk7",
+                "bc1ptdg60grjk9t3qqcqczp4tlyy3z47yrx9nhlrjsmw36q5a72lhdrs9f00nj",
+                "bc1pgcwgsu8naxp7xlp5p7ufzs7emtfza2las7r2e7krzjhe5qj5xz2q88kmk5",
+            ],
+        );
+    }
+
+    // BIP86 public `tr(key/{0,1}/*)`
+    // Used addresses in test vector in https://github.com/bitcoin/bips/blob/master/bip-0086.mediawiki
+    #[test]
+    fn test_bip86_public_template() {
+        let pubkey = bitcoin::util::bip32::ExtendedPubKey::from_str("xpub6BgBgsespWvERF3LHQu6CnqdvfEvtMcQjYrcRzx53QJjSxarj2afYWcLteoGVky7D3UKDP9QyrLprQ3VCECoY49yfdDEHGCtMMj92pReUsQ").unwrap();
+        let fingerprint = bitcoin::util::bip32::Fingerprint::from_str("73c5da0a").unwrap();
+        check(
+            Bip86Public(pubkey, fingerprint, KeychainKind::External).build(Network::Bitcoin),
+            false,
+            true,
+            false,
+            Network::Bitcoin,
+            &[
+                "bc1p5cyxnuxmeuwuvkwfem96lqzszd02n6xdcjrs20cac6yqjjwudpxqkedrcr",
+                "bc1p4qhjn9zdvkux4e44uhx8tc55attvtyu358kutcqkudyccelu0was9fqzwh",
+                "bc1p0d0rhyynq0awa9m8cqrcr8f5nxqx3aw29w4ru5u9my3h0sfygnzs9khxz8",
+            ],
+        );
+        check(
+            Bip86Public(pubkey, fingerprint, KeychainKind::Internal).build(Network::Bitcoin),
+            false,
+            true,
+            false,
+            Network::Bitcoin,
+            &[
+                "bc1p3qkhfews2uk44qtvauqyr2ttdsw7svhkl9nkm9s9c3x4ax5h60wqwruhk7",
+                "bc1ptdg60grjk9t3qqcqczp4tlyy3z47yrx9nhlrjsmw36q5a72lhdrs9f00nj",
+                "bc1pgcwgsu8naxp7xlp5p7ufzs7emtfza2las7r2e7krzjhe5qj5xz2q88kmk5",
+            ],
+        );
+    }
+}

crates/bdk/src/error.rs

@@ -0,0 +1,201 @@
+// Bitcoin Dev Kit
+// Written in 2020 by Alekos Filini <alekos.filini@gmail.com>
+//
+// Copyright (c) 2020-2021 Bitcoin Dev Kit Developers
+//
+// This file is licensed under the Apache License, Version 2.0 <LICENSE-APACHE
+// or http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option.
+// You may not use this file except in accordance with one or both of these
+// licenses.
+
+use crate::bitcoin::Network;
+use crate::{descriptor, wallet};
+use alloc::{string::String, vec::Vec};
+use bitcoin::{OutPoint, Txid};
+use core::fmt;
+
+/// Errors that can be thrown by the [`Wallet`](crate::wallet::Wallet)
+#[derive(Debug)]
+pub enum Error {
+    /// Generic error
+    Generic(String),
+    /// Cannot build a tx without recipients
+    NoRecipients,
+    /// `manually_selected_only` option is selected but no utxo has been passed
+    NoUtxosSelected,
+    /// Output created is under the dust limit, 546 satoshis
+    OutputBelowDustLimit(usize),
+    /// Wallet's UTXO set is not enough to cover recipient's requested plus fee
+    InsufficientFunds {
+        /// Sats needed for some transaction
+        needed: u64,
+        /// Sats available for spending
+        available: u64,
+    },
+    /// Branch and bound coin selection possible attempts with sufficiently big UTXO set could grow
+    /// exponentially, thus a limit is set, and when hit, this error is thrown
+    BnBTotalTriesExceeded,
+    /// Branch and bound coin selection tries to avoid needing a change by finding the right inputs for
+    /// the desired outputs plus fee, if there is not such combination this error is thrown
+    BnBNoExactMatch,
+    /// Happens when trying to spend an UTXO that is not in the internal database
+    UnknownUtxo,
+    /// Thrown when a tx is not found in the internal database
+    TransactionNotFound,
+    /// Happens when trying to bump a transaction that is already confirmed
+    TransactionConfirmed,
+    /// Trying to replace a tx that has a sequence >= `0xFFFFFFFE`
+    IrreplaceableTransaction,
+    /// When bumping a tx the fee rate requested is lower than required
+    FeeRateTooLow {
+        /// Required fee rate (satoshi/vbyte)
+        required: crate::types::FeeRate,
+    },
+    /// When bumping a tx the absolute fee requested is lower than replaced tx absolute fee
+    FeeTooLow {
+        /// Required fee absolute value (satoshi)
+        required: u64,
+    },
+    /// Node doesn't have data to estimate a fee rate
+    FeeRateUnavailable,
+    /// In order to use the [`TxBuilder::add_global_xpubs`] option every extended
+    /// key in the descriptor must either be a master key itself (having depth = 0) or have an
+    /// explicit origin provided
+    ///
+    /// [`TxBuilder::add_global_xpubs`]: crate::wallet::tx_builder::TxBuilder::add_global_xpubs
+    MissingKeyOrigin(String),
+    /// Error while working with [`keys`](crate::keys)
+    Key(crate::keys::KeyError),
+    /// Descriptor checksum mismatch
+    ChecksumMismatch,
+    /// Spending policy is not compatible with this [`KeychainKind`](crate::types::KeychainKind)
+    SpendingPolicyRequired(crate::types::KeychainKind),
+    /// Error while extracting and manipulating policies
+    InvalidPolicyPathError(crate::descriptor::policy::PolicyError),
+    /// Signing error
+    Signer(crate::wallet::signer::SignerError),
+    /// Requested outpoint doesn't exist in the tx (vout greater than available outputs)
+    InvalidOutpoint(OutPoint),
+    /// Error related to the parsing and usage of descriptors
+    Descriptor(crate::descriptor::error::Error),
+    /// Miniscript error
+    Miniscript(miniscript::Error),
+    /// Miniscript PSBT error
+    MiniscriptPsbt(MiniscriptPsbtError),
+    /// BIP32 error
+    Bip32(bitcoin::util::bip32::Error),
+    /// Partially signed bitcoin transaction error
+    Psbt(bitcoin::util::psbt::Error),
+}
+
+/// Errors returned by miniscript when updating inconsistent PSBTs
+#[derive(Debug, Clone)]
+pub enum MiniscriptPsbtError {
+    Conversion(miniscript::descriptor::ConversionError),
+    UtxoUpdate(miniscript::psbt::UtxoUpdateError),
+    OutputUpdate(miniscript::psbt::OutputUpdateError),
+}
+
+impl fmt::Display for MiniscriptPsbtError {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        match self {
+            Self::Conversion(err) => write!(f, "Conversion error: {}", err),
+            Self::UtxoUpdate(err) => write!(f, "UTXO update error: {}", err),
+            Self::OutputUpdate(err) => write!(f, "Output update error: {}", err),
+        }
+    }
+}
+
+#[cfg(feature = "std")]
+impl std::error::Error for MiniscriptPsbtError {}
+
+#[cfg(feature = "std")]
+impl fmt::Display for Error {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        match self {
+            Self::Generic(err) => write!(f, "Generic error: {}", err),
+            Self::NoRecipients => write!(f, "Cannot build tx without recipients"),
+            Self::NoUtxosSelected => write!(f, "No UTXO selected"),
+            Self::OutputBelowDustLimit(limit) => {
+                write!(f, "Output below the dust limit: {}", limit)
+            }
+            Self::InsufficientFunds { needed, available } => write!(
+                f,
+                "Insufficient funds: {} sat available of {} sat needed",
+                available, needed
+            ),
+            Self::BnBTotalTriesExceeded => {
+                write!(f, "Branch and bound coin selection: total tries exceeded")
+            }
+            Self::BnBNoExactMatch => write!(f, "Branch and bound coin selection: not exact match"),
+            Self::UnknownUtxo => write!(f, "UTXO not found in the internal database"),
+            Self::TransactionNotFound => {
+                write!(f, "Transaction not found in the internal database")
+            }
+            Self::TransactionConfirmed => write!(f, "Transaction already confirmed"),
+            Self::IrreplaceableTransaction => write!(f, "Transaction can't be replaced"),
+            Self::FeeRateTooLow { required } => write!(
+                f,
+                "Fee rate too low: required {} sat/vbyte",
+                required.as_sat_per_vb()
+            ),
+            Self::FeeTooLow { required } => write!(f, "Fee to low: required {} sat", required),
+            Self::FeeRateUnavailable => write!(f, "Fee rate unavailable"),
+            Self::MissingKeyOrigin(err) => write!(f, "Missing key origin: {}", err),
+            Self::Key(err) => write!(f, "Key error: {}", err),
+            Self::ChecksumMismatch => write!(f, "Descriptor checksum mismatch"),
+            Self::SpendingPolicyRequired(keychain_kind) => {
+                write!(f, "Spending policy required: {:?}", keychain_kind)
+            }
+            Self::InvalidPolicyPathError(err) => write!(f, "Invalid policy path: {}", err),
+            Self::Signer(err) => write!(f, "Signer error: {}", err),
+            Self::InvalidOutpoint(outpoint) => write!(
+                f,
+                "Requested outpoint doesn't exist in the tx: {}",
+                outpoint
+            ),
+            Self::Descriptor(err) => write!(f, "Descriptor error: {}", err),
+            Self::Miniscript(err) => write!(f, "Miniscript error: {}", err),
+            Self::MiniscriptPsbt(err) => write!(f, "Miniscript PSBT error: {}", err),
+            Self::Bip32(err) => write!(f, "BIP32 error: {}", err),
+            Self::Psbt(err) => write!(f, "PSBT error: {}", err),
+        }
+    }
+}
+
+#[cfg(feature = "std")]
+impl std::error::Error for Error {}
+
+macro_rules! impl_error {
+    ( $from:ty, $to:ident ) => {
+        impl_error!($from, $to, Error);
+    };
+    ( $from:ty, $to:ident, $impl_for:ty ) => {
+        impl core::convert::From<$from> for $impl_for {
+            fn from(err: $from) -> Self {
+                <$impl_for>::$to(err)
+            }
+        }
+    };
+}
+
+impl_error!(descriptor::error::Error, Descriptor);
+impl_error!(descriptor::policy::PolicyError, InvalidPolicyPathError);
+impl_error!(wallet::signer::SignerError, Signer);
+
+impl From<crate::keys::KeyError> for Error {
+    fn from(key_error: crate::keys::KeyError) -> Error {
+        match key_error {
+            crate::keys::KeyError::Miniscript(inner) => Error::Miniscript(inner),
+            crate::keys::KeyError::Bip32(inner) => Error::Bip32(inner),
+            crate::keys::KeyError::InvalidChecksum => Error::ChecksumMismatch,
+            e => Error::Key(e),
+        }
+    }
+}
+
+impl_error!(miniscript::Error, Miniscript);
+impl_error!(MiniscriptPsbtError, MiniscriptPsbt);
+impl_error!(bitcoin::util::bip32::Error, Bip32);
+impl_error!(bitcoin::util::psbt::Error, Psbt);

crates/bdk/src/keys/bip39.rs

@@ -0,0 +1,227 @@
+// Bitcoin Dev Kit
+// Written in 2020 by Alekos Filini <alekos.filini@gmail.com>
+//
+// Copyright (c) 2020-2021 Bitcoin Dev Kit Developers
+//
+// This file is licensed under the Apache License, Version 2.0 <LICENSE-APACHE
+// or http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option.
+// You may not use this file except in accordance with one or both of these
+// licenses.
+
+//! BIP-0039
+
+// TODO: maybe write our own implementation of bip39? Seems stupid to have an extra dependency for
+// something that should be fairly simple to re-implement.
+
+use alloc::string::String;
+use bitcoin::util::bip32;
+use bitcoin::Network;
+
+use miniscript::ScriptContext;
+
+pub use bip39::{Error, Language, Mnemonic};
+
+type Seed = [u8; 64];
+
+/// Type describing entropy length (aka word count) in the mnemonic
+pub enum WordCount {
+    /// 12 words mnemonic (128 bits entropy)
+    Words12 = 128,
+    /// 15 words mnemonic (160 bits entropy)
+    Words15 = 160,
+    /// 18 words mnemonic (192 bits entropy)
+    Words18 = 192,
+    /// 21 words mnemonic (224 bits entropy)
+    Words21 = 224,
+    /// 24 words mnemonic (256 bits entropy)
+    Words24 = 256,
+}
+
+use super::{
+    any_network, DerivableKey, DescriptorKey, ExtendedKey, GeneratableKey, GeneratedKey, KeyError,
+};
+
+fn set_valid_on_any_network<Ctx: ScriptContext>(
+    descriptor_key: DescriptorKey<Ctx>,
+) -> DescriptorKey<Ctx> {
+    // We have to pick one network to build the xprv, but since the bip39 standard doesn't
+    // encode the network, the xprv we create is actually valid everywhere. So we override the
+    // valid networks with `any_network()`.
+    descriptor_key.override_valid_networks(any_network())
+}
+
+/// Type for a BIP39 mnemonic with an optional passphrase
+pub type MnemonicWithPassphrase = (Mnemonic, Option<String>);
+
+#[cfg_attr(docsrs, doc(cfg(feature = "keys-bip39")))]
+impl<Ctx: ScriptContext> DerivableKey<Ctx> for Seed {
+    fn into_extended_key(self) -> Result<ExtendedKey<Ctx>, KeyError> {
+        Ok(bip32::ExtendedPrivKey::new_master(Network::Bitcoin, &self[..])?.into())
+    }
+
+    fn into_descriptor_key(
+        self,
+        source: Option<bip32::KeySource>,
+        derivation_path: bip32::DerivationPath,
+    ) -> Result<DescriptorKey<Ctx>, KeyError> {
+        let descriptor_key = self
+            .into_extended_key()?
+            .into_descriptor_key(source, derivation_path)?;
+
+        Ok(set_valid_on_any_network(descriptor_key))
+    }
+}
+
+#[cfg_attr(docsrs, doc(cfg(feature = "keys-bip39")))]
+impl<Ctx: ScriptContext> DerivableKey<Ctx> for MnemonicWithPassphrase {
+    fn into_extended_key(self) -> Result<ExtendedKey<Ctx>, KeyError> {
+        let (mnemonic, passphrase) = self;
+        let seed: Seed = mnemonic.to_seed(passphrase.as_deref().unwrap_or(""));
+
+        seed.into_extended_key()
+    }
+
+    fn into_descriptor_key(
+        self,
+        source: Option<bip32::KeySource>,
+        derivation_path: bip32::DerivationPath,
+    ) -> Result<DescriptorKey<Ctx>, KeyError> {
+        let descriptor_key = self
+            .into_extended_key()?
+            .into_descriptor_key(source, derivation_path)?;
+
+        Ok(set_valid_on_any_network(descriptor_key))
+    }
+}
+
+#[cfg_attr(docsrs, doc(cfg(feature = "keys-bip39")))]
+impl<Ctx: ScriptContext> DerivableKey<Ctx> for (GeneratedKey<Mnemonic, Ctx>, Option<String>) {
+    fn into_extended_key(self) -> Result<ExtendedKey<Ctx>, KeyError> {
+        let (mnemonic, passphrase) = self;
+        (mnemonic.into_key(), passphrase).into_extended_key()
+    }
+
+    fn into_descriptor_key(
+        self,
+        source: Option<bip32::KeySource>,
+        derivation_path: bip32::DerivationPath,
+    ) -> Result<DescriptorKey<Ctx>, KeyError> {
+        let (mnemonic, passphrase) = self;
+        (mnemonic.into_key(), passphrase).into_descriptor_key(source, derivation_path)
+    }
+}
+
+#[cfg_attr(docsrs, doc(cfg(feature = "keys-bip39")))]
+impl<Ctx: ScriptContext> DerivableKey<Ctx> for Mnemonic {
+    fn into_extended_key(self) -> Result<ExtendedKey<Ctx>, KeyError> {
+        (self, None).into_extended_key()
+    }
+
+    fn into_descriptor_key(
+        self,
+        source: Option<bip32::KeySource>,
+        derivation_path: bip32::DerivationPath,
+    ) -> Result<DescriptorKey<Ctx>, KeyError> {
+        let descriptor_key = self
+            .into_extended_key()?
+            .into_descriptor_key(source, derivation_path)?;
+
+        Ok(set_valid_on_any_network(descriptor_key))
+    }
+}
+
+#[cfg_attr(docsrs, doc(cfg(feature = "keys-bip39")))]
+impl<Ctx: ScriptContext> GeneratableKey<Ctx> for Mnemonic {
+    type Entropy = [u8; 32];
+
+    type Options = (WordCount, Language);
+    type Error = Option<bip39::Error>;
+
+    fn generate_with_entropy(
+        (word_count, language): Self::Options,
+        entropy: Self::Entropy,
+    ) -> Result<GeneratedKey<Self, Ctx>, Self::Error> {
+        let entropy = &entropy.as_ref()[..(word_count as usize / 8)];
+        let mnemonic = Mnemonic::from_entropy_in(language, entropy)?;
+
+        Ok(GeneratedKey::new(mnemonic, any_network()))
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use alloc::string::ToString;
+    use core::str::FromStr;
+
+    use bitcoin::util::bip32;
+
+    use bip39::{Language, Mnemonic};
+
+    use crate::keys::{any_network, GeneratableKey, GeneratedKey};
+
+    use super::WordCount;
+
+    #[test]
+    fn test_keys_bip39_mnemonic() {
+        let mnemonic =
+            "aim bunker wash balance finish force paper analyst cabin spoon stable organ";
+        let mnemonic = Mnemonic::parse_in(Language::English, mnemonic).unwrap();
+        let path = bip32::DerivationPath::from_str("m/44'/0'/0'/0").unwrap();
+
+        let key = (mnemonic, path);
+        let (desc, keys, networks) = crate::descriptor!(wpkh(key)).unwrap();
+        assert_eq!(desc.to_string(), "wpkh([be83839f/44'/0'/0']xpub6DCQ1YcqvZtSwGWMrwHELPehjWV3f2MGZ69yBADTxFEUAoLwb5Mp5GniQK6tTp3AgbngVz9zEFbBJUPVnkG7LFYt8QMTfbrNqs6FNEwAPKA/0/*)#0r8v4nkv");
+        assert_eq!(keys.len(), 1);
+        assert_eq!(networks.len(), 4);
+    }
+
+    #[test]
+    fn test_keys_bip39_mnemonic_passphrase() {
+        let mnemonic =
+            "aim bunker wash balance finish force paper analyst cabin spoon stable organ";
+        let mnemonic = Mnemonic::parse_in(Language::English, mnemonic).unwrap();
+        let path = bip32::DerivationPath::from_str("m/44'/0'/0'/0").unwrap();
+
+        let key = ((mnemonic, Some("passphrase".into())), path);
+        let (desc, keys, networks) = crate::descriptor!(wpkh(key)).unwrap();
+        assert_eq!(desc.to_string(), "wpkh([8f6cb80c/44'/0'/0']xpub6DWYS8bbihFevy29M4cbw4ZR3P5E12jB8R88gBDWCTCNpYiDHhYWNywrCF9VZQYagzPmsZpxXpytzSoxynyeFr4ZyzheVjnpLKuse4fiwZw/0/*)#h0j0tg5m");
+        assert_eq!(keys.len(), 1);
+        assert_eq!(networks.len(), 4);
+    }
+
+    #[test]
+    fn test_keys_generate_bip39() {
+        let generated_mnemonic: GeneratedKey<_, miniscript::Segwitv0> =
+            Mnemonic::generate_with_entropy(
+                (WordCount::Words12, Language::English),
+                crate::keys::test::TEST_ENTROPY,
+            )
+            .unwrap();
+        assert_eq!(generated_mnemonic.valid_networks, any_network());
+        assert_eq!(
+            generated_mnemonic.to_string(),
+            "primary fetch primary fetch primary fetch primary fetch primary fetch primary fever"
+        );
+
+        let generated_mnemonic: GeneratedKey<_, miniscript::Segwitv0> =
+            Mnemonic::generate_with_entropy(
+                (WordCount::Words24, Language::English),
+                crate::keys::test::TEST_ENTROPY,
+            )
+            .unwrap();
+        assert_eq!(generated_mnemonic.valid_networks, any_network());
+        assert_eq!(generated_mnemonic.to_string(), "primary fetch primary fetch primary fetch primary fetch primary fetch primary fetch primary fetch primary fetch primary fetch primary fetch primary fetch primary foster");
+    }
+
+    #[test]
+    fn test_keys_generate_bip39_random() {
+        let generated_mnemonic: GeneratedKey<_, miniscript::Segwitv0> =
+            Mnemonic::generate((WordCount::Words12, Language::English)).unwrap();
+        assert_eq!(generated_mnemonic.valid_networks, any_network());
+
+        let generated_mnemonic: GeneratedKey<_, miniscript::Segwitv0> =
+            Mnemonic::generate((WordCount::Words24, Language::English)).unwrap();
+        assert_eq!(generated_mnemonic.valid_networks, any_network());
+    }
+}

crates/bdk/src/keys/mod.rs

@@ -0,0 +1,997 @@
+// Bitcoin Dev Kit
+// Written in 2020 by Alekos Filini <alekos.filini@gmail.com>
+//
+// Copyright (c) 2020-2021 Bitcoin Dev Kit Developers
+//
+// This file is licensed under the Apache License, Version 2.0 <LICENSE-APACHE
+// or http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option.
+// You may not use this file except in accordance with one or both of these
+// licenses.
+
+//! Key formats
+
+use crate::collections::HashSet;
+use alloc::string::{String, ToString};
+use alloc::vec::Vec;
+use core::any::TypeId;
+use core::fmt;
+use core::marker::PhantomData;
+use core::ops::Deref;
+use core::str::FromStr;
+
+use bitcoin::secp256k1::{self, Secp256k1, Signing};
+
+use bitcoin::util::bip32;
+use bitcoin::{Network, PrivateKey, PublicKey, XOnlyPublicKey};
+
+use miniscript::descriptor::{Descriptor, DescriptorXKey, Wildcard};
+pub use miniscript::descriptor::{
+    DescriptorPublicKey, DescriptorSecretKey, KeyMap, SinglePriv, SinglePub, SinglePubKey,
+    SortedMultiVec,
+};
+pub use miniscript::ScriptContext;
+use miniscript::{Miniscript, Terminal};
+
+use crate::descriptor::{CheckMiniscript, DescriptorError};
+use crate::wallet::utils::SecpCtx;
+
+#[cfg(feature = "keys-bip39")]
+#[cfg_attr(docsrs, doc(cfg(feature = "keys-bip39")))]
+pub mod bip39;
+
+/// Set of valid networks for a key
+pub type ValidNetworks = HashSet<Network>;
+
+/// Create a set containing mainnet, testnet, signet, and regtest
+pub fn any_network() -> ValidNetworks {
+    vec![
+        Network::Bitcoin,
+        Network::Testnet,
+        Network::Regtest,
+        Network::Signet,
+    ]
+    .into_iter()
+    .collect()
+}
+/// Create a set only containing mainnet
+pub fn mainnet_network() -> ValidNetworks {
+    vec![Network::Bitcoin].into_iter().collect()
+}
+/// Create a set containing testnet and regtest
+pub fn test_networks() -> ValidNetworks {
+    vec![Network::Testnet, Network::Regtest, Network::Signet]
+        .into_iter()
+        .collect()
+}
+/// Compute the intersection of two sets
+pub fn merge_networks(a: &ValidNetworks, b: &ValidNetworks) -> ValidNetworks {
+    a.intersection(b).cloned().collect()
+}
+
+/// Container for public or secret keys
+#[derive(Debug)]
+pub enum DescriptorKey<Ctx: ScriptContext> {
+    #[doc(hidden)]
+    Public(DescriptorPublicKey, ValidNetworks, PhantomData<Ctx>),
+    #[doc(hidden)]
+    Secret(DescriptorSecretKey, ValidNetworks, PhantomData<Ctx>),
+}
+
+impl<Ctx: ScriptContext> DescriptorKey<Ctx> {
+    /// Create an instance given a public key and a set of valid networks
+    pub fn from_public(public: DescriptorPublicKey, networks: ValidNetworks) -> Self {
+        DescriptorKey::Public(public, networks, PhantomData)
+    }
+
+    /// Create an instance given a secret key and a set of valid networks
+    pub fn from_secret(secret: DescriptorSecretKey, networks: ValidNetworks) -> Self {
+        DescriptorKey::Secret(secret, networks, PhantomData)
+    }
+
+    /// Override the computed set of valid networks
+    pub fn override_valid_networks(self, networks: ValidNetworks) -> Self {
+        match self {
+            DescriptorKey::Public(key, _, _) => DescriptorKey::Public(key, networks, PhantomData),
+            DescriptorKey::Secret(key, _, _) => DescriptorKey::Secret(key, networks, PhantomData),
+        }
+    }
+
+    // This method is used internally by `bdk::fragment!` and `bdk::descriptor!`. It has to be
+    // public because it is effectively called by external crates once the macros are expanded,
+    // but since it is not meant to be part of the public api we hide it from the docs.
+    #[doc(hidden)]
+    pub fn extract(
+        self,
+        secp: &SecpCtx,
+    ) -> Result<(DescriptorPublicKey, KeyMap, ValidNetworks), KeyError> {
+        match self {
+            DescriptorKey::Public(public, valid_networks, _) => {
+                Ok((public, KeyMap::default(), valid_networks))
+            }
+            DescriptorKey::Secret(secret, valid_networks, _) => {
+                let mut key_map = KeyMap::with_capacity(1);
+
+                let public = secret
+                    .to_public(secp)
+                    .map_err(|e| miniscript::Error::Unexpected(e.to_string()))?;
+                key_map.insert(public.clone(), secret);
+
+                Ok((public, key_map, valid_networks))
+            }
+        }
+    }
+}
+
+/// Enum representation of the known valid [`ScriptContext`]s
+#[derive(Debug, Eq, PartialEq, Copy, Clone)]
+pub enum ScriptContextEnum {
+    /// Legacy scripts
+    Legacy,
+    /// Segwitv0 scripts
+    Segwitv0,
+    /// Taproot scripts
+    Tap,
+}
+
+impl ScriptContextEnum {
+    /// Returns whether the script context is [`ScriptContextEnum::Legacy`]
+    pub fn is_legacy(&self) -> bool {
+        self == &ScriptContextEnum::Legacy
+    }
+
+    /// Returns whether the script context is [`ScriptContextEnum::Segwitv0`]
+    pub fn is_segwit_v0(&self) -> bool {
+        self == &ScriptContextEnum::Segwitv0
+    }
+
+    /// Returns whether the script context is [`ScriptContextEnum::Tap`]
+    pub fn is_taproot(&self) -> bool {
+        self == &ScriptContextEnum::Tap
+    }
+}
+
+/// Trait that adds extra useful methods to [`ScriptContext`]s
+pub trait ExtScriptContext: ScriptContext {
+    /// Returns the [`ScriptContext`] as a [`ScriptContextEnum`]
+    fn as_enum() -> ScriptContextEnum;
+
+    /// Returns whether the script context is [`Legacy`](miniscript::Legacy)
+    fn is_legacy() -> bool {
+        Self::as_enum().is_legacy()
+    }
+
+    /// Returns whether the script context is [`Segwitv0`](miniscript::Segwitv0)
+    fn is_segwit_v0() -> bool {
+        Self::as_enum().is_segwit_v0()
+    }
+
+    /// Returns whether the script context is [`Tap`](miniscript::Tap), aka Taproot or Segwit V1
+    fn is_taproot() -> bool {
+        Self::as_enum().is_taproot()
+    }
+}
+
+impl<Ctx: ScriptContext + 'static> ExtScriptContext for Ctx {
+    fn as_enum() -> ScriptContextEnum {
+        match TypeId::of::<Ctx>() {
+            t if t == TypeId::of::<miniscript::Legacy>() => ScriptContextEnum::Legacy,
+            t if t == TypeId::of::<miniscript::Segwitv0>() => ScriptContextEnum::Segwitv0,
+            t if t == TypeId::of::<miniscript::Tap>() => ScriptContextEnum::Tap,
+            _ => unimplemented!("Unknown ScriptContext type"),
+        }
+    }
+}
+
+/// Trait for objects that can be turned into a public or secret [`DescriptorKey`]
+///
+/// The generic type `Ctx` is used to define the context in which the key is valid: some key
+/// formats, like the mnemonics used by Electrum wallets, encode internally whether the wallet is
+/// legacy or segwit. Thus, trying to turn a valid legacy mnemonic into a `DescriptorKey`
+/// that would become part of a segwit descriptor should fail.
+///
+/// For key types that do care about this, the [`ExtScriptContext`] trait provides some useful
+/// methods that can be used to check at runtime which `Ctx` is being used.
+///
+/// For key types that can do this check statically (because they can only work within a
+/// single `Ctx`), the "specialized" trait can be implemented to make the compiler handle the type
+/// checking.
+///
+/// Keys also have control over the networks they support: constructing the return object with
+/// [`DescriptorKey::from_public`] or [`DescriptorKey::from_secret`] allows to specify a set of
+/// [`ValidNetworks`].
+///
+/// ## Examples
+///
+/// Key type valid in any context:
+///
+/// ```
+/// use bdk::bitcoin::PublicKey;
+///
+/// use bdk::keys::{DescriptorKey, IntoDescriptorKey, KeyError, ScriptContext};
+///
+/// pub struct MyKeyType {
+///     pubkey: PublicKey,
+/// }
+///
+/// impl<Ctx: ScriptContext> IntoDescriptorKey<Ctx> for MyKeyType {
+///     fn into_descriptor_key(self) -> Result<DescriptorKey<Ctx>, KeyError> {
+///         self.pubkey.into_descriptor_key()
+///     }
+/// }
+/// ```
+///
+/// Key type that is only valid on mainnet:
+///
+/// ```
+/// use bdk::bitcoin::PublicKey;
+///
+/// use bdk::keys::{
+///     mainnet_network, DescriptorKey, DescriptorPublicKey, IntoDescriptorKey, KeyError,
+///     ScriptContext, SinglePub, SinglePubKey,
+/// };
+///
+/// pub struct MyKeyType {
+///     pubkey: PublicKey,
+/// }
+///
+/// impl<Ctx: ScriptContext> IntoDescriptorKey<Ctx> for MyKeyType {
+///     fn into_descriptor_key(self) -> Result<DescriptorKey<Ctx>, KeyError> {
+///         Ok(DescriptorKey::from_public(
+///             DescriptorPublicKey::Single(SinglePub {
+///                 origin: None,
+///                 key: SinglePubKey::FullKey(self.pubkey),
+///             }),
+///             mainnet_network(),
+///         ))
+///     }
+/// }
+/// ```
+///
+/// Key type that internally encodes in which context it's valid. The context is checked at runtime:
+///
+/// ```
+/// use bdk::bitcoin::PublicKey;
+///
+/// use bdk::keys::{DescriptorKey, ExtScriptContext, IntoDescriptorKey, KeyError, ScriptContext};
+///
+/// pub struct MyKeyType {
+///     is_legacy: bool,
+///     pubkey: PublicKey,
+/// }
+///
+/// impl<Ctx: ScriptContext + 'static> IntoDescriptorKey<Ctx> for MyKeyType {
+///     fn into_descriptor_key(self) -> Result<DescriptorKey<Ctx>, KeyError> {
+///         if Ctx::is_legacy() == self.is_legacy {
+///             self.pubkey.into_descriptor_key()
+///         } else {
+///             Err(KeyError::InvalidScriptContext)
+///         }
+///     }
+/// }
+/// ```
+///
+/// Key type that can only work within [`miniscript::Segwitv0`] context. Only the specialized version
+/// of the trait is implemented.
+///
+/// This example deliberately fails to compile, to demonstrate how the compiler can catch when keys
+/// are misused. In this case, the "segwit-only" key is used to build a `pkh()` descriptor, which
+/// makes the compiler (correctly) fail.
+///
+/// ```compile_fail
+/// use bdk::bitcoin::PublicKey;
+/// use core::str::FromStr;
+///
+/// use bdk::keys::{DescriptorKey, IntoDescriptorKey, KeyError};
+///
+/// pub struct MySegwitOnlyKeyType {
+///     pubkey: PublicKey,
+/// }
+///
+/// impl IntoDescriptorKey<bdk::miniscript::Segwitv0> for MySegwitOnlyKeyType {
+///     fn into_descriptor_key(self) -> Result<DescriptorKey<bdk::miniscript::Segwitv0>, KeyError> {
+///         self.pubkey.into_descriptor_key()
+///     }
+/// }
+///
+/// let key = MySegwitOnlyKeyType {
+///     pubkey: PublicKey::from_str("...")?,
+/// };
+/// let (descriptor, _, _) = bdk::descriptor!(pkh(key))?;
+/// //                                       ^^^^^ changing this to `wpkh` would make it compile
+///
+/// # Ok::<_, Box<dyn std::error::Error>>(())
+/// ```
+pub trait IntoDescriptorKey<Ctx: ScriptContext>: Sized {
+    /// Turn the key into a [`DescriptorKey`] within the requested [`ScriptContext`]
+    fn into_descriptor_key(self) -> Result<DescriptorKey<Ctx>, KeyError>;
+}
+
+/// Enum for extended keys that can be either `xprv` or `xpub`
+///
+/// An instance of [`ExtendedKey`] can be constructed from an [`ExtendedPrivKey`](bip32::ExtendedPrivKey)
+/// or an [`ExtendedPubKey`](bip32::ExtendedPubKey) by using the `From` trait.
+///
+/// Defaults to the [`Legacy`](miniscript::Legacy) context.
+pub enum ExtendedKey<Ctx: ScriptContext = miniscript::Legacy> {
+    /// A private extended key, aka an `xprv`
+    Private((bip32::ExtendedPrivKey, PhantomData<Ctx>)),
+    /// A public extended key, aka an `xpub`
+    Public((bip32::ExtendedPubKey, PhantomData<Ctx>)),
+}
+
+impl<Ctx: ScriptContext> ExtendedKey<Ctx> {
+    /// Return whether or not the key contains the private data
+    pub fn has_secret(&self) -> bool {
+        match self {
+            ExtendedKey::Private(_) => true,
+            ExtendedKey::Public(_) => false,
+        }
+    }
+
+    /// Transform the [`ExtendedKey`] into an [`ExtendedPrivKey`](bip32::ExtendedPrivKey) for the
+    /// given [`Network`], if the key contains the private data
+    pub fn into_xprv(self, network: Network) -> Option<bip32::ExtendedPrivKey> {
+        match self {
+            ExtendedKey::Private((mut xprv, _)) => {
+                xprv.network = network;
+                Some(xprv)
+            }
+            ExtendedKey::Public(_) => None,
+        }
+    }
+
+    /// Transform the [`ExtendedKey`] into an [`ExtendedPubKey`](bip32::ExtendedPubKey) for the
+    /// given [`Network`]
+    pub fn into_xpub<C: Signing>(
+        self,
+        network: bitcoin::Network,
+        secp: &Secp256k1<C>,
+    ) -> bip32::ExtendedPubKey {
+        let mut xpub = match self {
+            ExtendedKey::Private((xprv, _)) => bip32::ExtendedPubKey::from_priv(secp, &xprv),
+            ExtendedKey::Public((xpub, _)) => xpub,
+        };
+
+        xpub.network = network;
+        xpub
+    }
+}
+
+impl<Ctx: ScriptContext> From<bip32::ExtendedPubKey> for ExtendedKey<Ctx> {
+    fn from(xpub: bip32::ExtendedPubKey) -> Self {
+        ExtendedKey::Public((xpub, PhantomData))
+    }
+}
+
+impl<Ctx: ScriptContext> From<bip32::ExtendedPrivKey> for ExtendedKey<Ctx> {
+    fn from(xprv: bip32::ExtendedPrivKey) -> Self {
+        ExtendedKey::Private((xprv, PhantomData))
+    }
+}
+
+/// Trait for keys that can be derived.
+///
+/// When extra metadata are provided, a [`DerivableKey`] can be transformed into a
+/// [`DescriptorKey`]: the trait [`IntoDescriptorKey`] is automatically implemented
+/// for `(DerivableKey, DerivationPath)` and
+/// `(DerivableKey, KeySource, DerivationPath)` tuples.
+///
+/// For key types that don't encode any indication about the path to use (like bip39), it's
+/// generally recommended to implement this trait instead of [`IntoDescriptorKey`]. The same
+/// rules regarding script context and valid networks apply.
+///
+/// ## Examples
+///
+/// Key types that can be directly converted into an [`ExtendedPrivKey`] or
+/// an [`ExtendedPubKey`] can implement only the required `into_extended_key()` method.
+///
+/// ```
+/// use bdk::bitcoin;
+/// use bdk::bitcoin::util::bip32;
+/// use bdk::keys::{DerivableKey, ExtendedKey, KeyError, ScriptContext};
+///
+/// struct MyCustomKeyType {
+///     key_data: bitcoin::PrivateKey,
+///     chain_code: Vec<u8>,
+///     network: bitcoin::Network,
+/// }
+///
+/// impl<Ctx: ScriptContext> DerivableKey<Ctx> for MyCustomKeyType {
+///     fn into_extended_key(self) -> Result<ExtendedKey<Ctx>, KeyError> {
+///         let xprv = bip32::ExtendedPrivKey {
+///             network: self.network,
+///             depth: 0,
+///             parent_fingerprint: bip32::Fingerprint::default(),
+///             private_key: self.key_data.inner,
+///             chain_code: bip32::ChainCode::from(self.chain_code.as_ref()),
+///             child_number: bip32::ChildNumber::Normal { index: 0 },
+///         };
+///
+///         xprv.into_extended_key()
+///     }
+/// }
+/// ```
+///
+/// Types that don't internally encode the [`Network`](bitcoin::Network) in which they are valid need some extra
+/// steps to override the set of valid networks, otherwise only the network specified in the
+/// [`ExtendedPrivKey`] or [`ExtendedPubKey`] will be considered valid.
+///
+/// ```
+/// use bdk::bitcoin;
+/// use bdk::bitcoin::util::bip32;
+/// use bdk::keys::{
+///     any_network, DerivableKey, DescriptorKey, ExtendedKey, KeyError, ScriptContext,
+/// };
+///
+/// struct MyCustomKeyType {
+///     key_data: bitcoin::PrivateKey,
+///     chain_code: Vec<u8>,
+/// }
+///
+/// impl<Ctx: ScriptContext> DerivableKey<Ctx> for MyCustomKeyType {
+///     fn into_extended_key(self) -> Result<ExtendedKey<Ctx>, KeyError> {
+///         let xprv = bip32::ExtendedPrivKey {
+///             network: bitcoin::Network::Bitcoin, // pick an arbitrary network here
+///             depth: 0,
+///             parent_fingerprint: bip32::Fingerprint::default(),
+///             private_key: self.key_data.inner,
+///             chain_code: bip32::ChainCode::from(self.chain_code.as_ref()),
+///             child_number: bip32::ChildNumber::Normal { index: 0 },
+///         };
+///
+///         xprv.into_extended_key()
+///     }
+///
+///     fn into_descriptor_key(
+///         self,
+///         source: Option<bip32::KeySource>,
+///         derivation_path: bip32::DerivationPath,
+///     ) -> Result<DescriptorKey<Ctx>, KeyError> {
+///         let descriptor_key = self
+///             .into_extended_key()?
+///             .into_descriptor_key(source, derivation_path)?;
+///
+///         // Override the set of valid networks here
+///         Ok(descriptor_key.override_valid_networks(any_network()))
+///     }
+/// }
+/// ```
+///
+/// [`DerivationPath`]: (bip32::DerivationPath)
+/// [`ExtendedPrivKey`]: (bip32::ExtendedPrivKey)
+/// [`ExtendedPubKey`]: (bip32::ExtendedPubKey)
+pub trait DerivableKey<Ctx: ScriptContext = miniscript::Legacy>: Sized {
+    /// Consume `self` and turn it into an [`ExtendedKey`]
+    #[cfg_attr(
+        feature = "keys-bip39",
+        doc = r##"
+This can be used to get direct access to `xprv`s and `xpub`s for types that implement this trait,
+like [`Mnemonic`](bip39::Mnemonic) when the `keys-bip39` feature is enabled.
+```rust
+use bdk::bitcoin::Network;
+use bdk::keys::{DerivableKey, ExtendedKey};
+use bdk::keys::bip39::{Mnemonic, Language};
+
+# fn main() -> Result<(), Box<dyn std::error::Error>> {
+let xkey: ExtendedKey =
+    Mnemonic::parse_in(
+        Language::English,
+        "jelly crash boy whisper mouse ecology tuna soccer memory million news short",
+    )?
+    .into_extended_key()?;
+let xprv = xkey.into_xprv(Network::Bitcoin).unwrap();
+# Ok(()) }
+```
+"##
+    )]
+    fn into_extended_key(self) -> Result<ExtendedKey<Ctx>, KeyError>;
+
+    /// Consume `self` and turn it into a [`DescriptorKey`] by adding the extra metadata, such as
+    /// key origin and derivation path
+    fn into_descriptor_key(
+        self,
+        origin: Option<bip32::KeySource>,
+        derivation_path: bip32::DerivationPath,
+    ) -> Result<DescriptorKey<Ctx>, KeyError> {
+        match self.into_extended_key()? {
+            ExtendedKey::Private((xprv, _)) => DescriptorSecretKey::XPrv(DescriptorXKey {
+                origin,
+                xkey: xprv,
+                derivation_path,
+                wildcard: Wildcard::Unhardened,
+            })
+            .into_descriptor_key(),
+            ExtendedKey::Public((xpub, _)) => DescriptorPublicKey::XPub(DescriptorXKey {
+                origin,
+                xkey: xpub,
+                derivation_path,
+                wildcard: Wildcard::Unhardened,
+            })
+            .into_descriptor_key(),
+        }
+    }
+}
+
+/// Identity conversion
+impl<Ctx: ScriptContext> DerivableKey<Ctx> for ExtendedKey<Ctx> {
+    fn into_extended_key(self) -> Result<ExtendedKey<Ctx>, KeyError> {
+        Ok(self)
+    }
+}
+
+impl<Ctx: ScriptContext> DerivableKey<Ctx> for bip32::ExtendedPubKey {
+    fn into_extended_key(self) -> Result<ExtendedKey<Ctx>, KeyError> {
+        Ok(self.into())
+    }
+}
+
+impl<Ctx: ScriptContext> DerivableKey<Ctx> for bip32::ExtendedPrivKey {
+    fn into_extended_key(self) -> Result<ExtendedKey<Ctx>, KeyError> {
+        Ok(self.into())
+    }
+}
+
+/// Output of a [`GeneratableKey`] key generation
+pub struct GeneratedKey<K, Ctx: ScriptContext> {
+    key: K,
+    valid_networks: ValidNetworks,
+    phantom: PhantomData<Ctx>,
+}
+
+impl<K, Ctx: ScriptContext> GeneratedKey<K, Ctx> {
+    fn new(key: K, valid_networks: ValidNetworks) -> Self {
+        GeneratedKey {
+            key,
+            valid_networks,
+            phantom: PhantomData,
+        }
+    }
+
+    /// Consumes `self` and returns the key
+    pub fn into_key(self) -> K {
+        self.key
+    }
+}
+
+impl<K, Ctx: ScriptContext> Deref for GeneratedKey<K, Ctx> {
+    type Target = K;
+
+    fn deref(&self) -> &Self::Target {
+        &self.key
+    }
+}
+
+impl<K: Clone, Ctx: ScriptContext> Clone for GeneratedKey<K, Ctx> {
+    fn clone(&self) -> GeneratedKey<K, Ctx> {
+        GeneratedKey {
+            key: self.key.clone(),
+            valid_networks: self.valid_networks.clone(),
+            phantom: self.phantom,
+        }
+    }
+}
+
+// Make generated "derivable" keys themselves "derivable". Also make sure they are assigned the
+// right `valid_networks`.
+impl<Ctx, K> DerivableKey<Ctx> for GeneratedKey<K, Ctx>
+where
+    Ctx: ScriptContext,
+    K: DerivableKey<Ctx>,
+{
+    fn into_extended_key(self) -> Result<ExtendedKey<Ctx>, KeyError> {
+        self.key.into_extended_key()
+    }
+
+    fn into_descriptor_key(
+        self,
+        origin: Option<bip32::KeySource>,
+        derivation_path: bip32::DerivationPath,
+    ) -> Result<DescriptorKey<Ctx>, KeyError> {
+        let descriptor_key = self.key.into_descriptor_key(origin, derivation_path)?;
+        Ok(descriptor_key.override_valid_networks(self.valid_networks))
+    }
+}
+
+// Make generated keys directly usable in descriptors, and make sure they get assigned the right
+// `valid_networks`.
+impl<Ctx, K> IntoDescriptorKey<Ctx> for GeneratedKey<K, Ctx>
+where
+    Ctx: ScriptContext,
+    K: IntoDescriptorKey<Ctx>,
+{
+    fn into_descriptor_key(self) -> Result<DescriptorKey<Ctx>, KeyError> {
+        let desc_key = self.key.into_descriptor_key()?;
+        Ok(desc_key.override_valid_networks(self.valid_networks))
+    }
+}
+
+/// Trait for keys that can be generated
+///
+/// The same rules about [`ScriptContext`] and [`ValidNetworks`] from [`IntoDescriptorKey`] apply.
+///
+/// This trait is particularly useful when combined with [`DerivableKey`]: if `Self`
+/// implements it, the returned [`GeneratedKey`] will also implement it. The same is true for
+/// [`IntoDescriptorKey`]: the generated keys can be directly used in descriptors if `Self` is also
+/// [`IntoDescriptorKey`].
+pub trait GeneratableKey<Ctx: ScriptContext>: Sized {
+    /// Type specifying the amount of entropy required e.g. `[u8;32]`
+    type Entropy: AsMut<[u8]> + Default;
+
+    /// Extra options required by the `generate_with_entropy`
+    type Options;
+    /// Returned error in case of failure
+    type Error: core::fmt::Debug;
+
+    /// Generate a key given the extra options and the entropy
+    fn generate_with_entropy(
+        options: Self::Options,
+        entropy: Self::Entropy,
+    ) -> Result<GeneratedKey<Self, Ctx>, Self::Error>;
+
+    /// Generate a key given the options with a random entropy
+    fn generate(options: Self::Options) -> Result<GeneratedKey<Self, Ctx>, Self::Error> {
+        use rand::{thread_rng, Rng};
+
+        let mut entropy = Self::Entropy::default();
+        thread_rng().fill(entropy.as_mut());
+        Self::generate_with_entropy(options, entropy)
+    }
+}
+
+/// Trait that allows generating a key with the default options
+///
+/// This trait is automatically implemented if the [`GeneratableKey::Options`] implements [`Default`].
+pub trait GeneratableDefaultOptions<Ctx>: GeneratableKey<Ctx>
+where
+    Ctx: ScriptContext,
+    <Self as GeneratableKey<Ctx>>::Options: Default,
+{
+    /// Generate a key with the default options and a given entropy
+    fn generate_with_entropy_default(
+        entropy: Self::Entropy,
+    ) -> Result<GeneratedKey<Self, Ctx>, Self::Error> {
+        Self::generate_with_entropy(Default::default(), entropy)
+    }
+
+    /// Generate a key with the default options and a random entropy
+    fn generate_default() -> Result<GeneratedKey<Self, Ctx>, Self::Error> {
+        Self::generate(Default::default())
+    }
+}
+
+/// Automatic implementation of [`GeneratableDefaultOptions`] for [`GeneratableKey`]s where
+/// `Options` implements `Default`
+impl<Ctx, K> GeneratableDefaultOptions<Ctx> for K
+where
+    Ctx: ScriptContext,
+    K: GeneratableKey<Ctx>,
+    <K as GeneratableKey<Ctx>>::Options: Default,
+{
+}
+
+impl<Ctx: ScriptContext> GeneratableKey<Ctx> for bip32::ExtendedPrivKey {
+    type Entropy = [u8; 32];
+
+    type Options = ();
+    type Error = bip32::Error;
+
+    fn generate_with_entropy(
+        _: Self::Options,
+        entropy: Self::Entropy,
+    ) -> Result<GeneratedKey<Self, Ctx>, Self::Error> {
+        // pick a arbitrary network here, but say that we support all of them
+        let xprv = bip32::ExtendedPrivKey::new_master(Network::Bitcoin, entropy.as_ref())?;
+        Ok(GeneratedKey::new(xprv, any_network()))
+    }
+}
+
+/// Options for generating a [`PrivateKey`]
+///
+/// Defaults to creating compressed keys, which save on-chain bytes and fees
+#[derive(Debug, Copy, Clone)]
+pub struct PrivateKeyGenerateOptions {
+    /// Whether the generated key should be "compressed" or not
+    pub compressed: bool,
+}
+
+impl Default for PrivateKeyGenerateOptions {
+    fn default() -> Self {
+        PrivateKeyGenerateOptions { compressed: true }
+    }
+}
+
+impl<Ctx: ScriptContext> GeneratableKey<Ctx> for PrivateKey {
+    type Entropy = [u8; secp256k1::constants::SECRET_KEY_SIZE];
+
+    type Options = PrivateKeyGenerateOptions;
+    type Error = bip32::Error;
+
+    fn generate_with_entropy(
+        options: Self::Options,
+        entropy: Self::Entropy,
+    ) -> Result<GeneratedKey<Self, Ctx>, Self::Error> {
+        // pick a arbitrary network here, but say that we support all of them
+        let inner = secp256k1::SecretKey::from_slice(&entropy)?;
+        let private_key = PrivateKey {
+            compressed: options.compressed,
+            network: Network::Bitcoin,
+            inner,
+        };
+
+        Ok(GeneratedKey::new(private_key, any_network()))
+    }
+}
+
+impl<Ctx: ScriptContext, T: DerivableKey<Ctx>> IntoDescriptorKey<Ctx>
+    for (T, bip32::DerivationPath)
+{
+    fn into_descriptor_key(self) -> Result<DescriptorKey<Ctx>, KeyError> {
+        self.0.into_descriptor_key(None, self.1)
+    }
+}
+
+impl<Ctx: ScriptContext, T: DerivableKey<Ctx>> IntoDescriptorKey<Ctx>
+    for (T, bip32::KeySource, bip32::DerivationPath)
+{
+    fn into_descriptor_key(self) -> Result<DescriptorKey<Ctx>, KeyError> {
+        self.0.into_descriptor_key(Some(self.1), self.2)
+    }
+}
+
+fn expand_multi_keys<Pk: IntoDescriptorKey<Ctx>, Ctx: ScriptContext>(
+    pks: Vec<Pk>,
+    secp: &SecpCtx,
+) -> Result<(Vec<DescriptorPublicKey>, KeyMap, ValidNetworks), KeyError> {
+    let (pks, key_maps_networks): (Vec<_>, Vec<_>) = pks
+        .into_iter()
+        .map(|key| key.into_descriptor_key()?.extract(secp))
+        .collect::<Result<Vec<_>, _>>()?
+        .into_iter()
+        .map(|(a, b, c)| (a, (b, c)))
+        .unzip();
+
+    let (key_map, valid_networks) = key_maps_networks.into_iter().fold(
+        (KeyMap::default(), any_network()),
+        |(mut keys_acc, net_acc), (key, net)| {
+            keys_acc.extend(key.into_iter());
+            let net_acc = merge_networks(&net_acc, &net);
+
+            (keys_acc, net_acc)
+        },
+    );
+
+    Ok((pks, key_map, valid_networks))
+}
+
+// Used internally by `bdk::fragment!` to build `pk_k()` fragments
+#[doc(hidden)]
+pub fn make_pk<Pk: IntoDescriptorKey<Ctx>, Ctx: ScriptContext>(
+    descriptor_key: Pk,
+    secp: &SecpCtx,
+) -> Result<(Miniscript<DescriptorPublicKey, Ctx>, KeyMap, ValidNetworks), DescriptorError> {
+    let (key, key_map, valid_networks) = descriptor_key.into_descriptor_key()?.extract(secp)?;
+    let minisc = Miniscript::from_ast(Terminal::PkK(key))?;
+
+    minisc.check_miniscript()?;
+
+    Ok((minisc, key_map, valid_networks))
+}
+
+// Used internally by `bdk::fragment!` to build `pk_h()` fragments
+#[doc(hidden)]
+pub fn make_pkh<Pk: IntoDescriptorKey<Ctx>, Ctx: ScriptContext>(
+    descriptor_key: Pk,
+    secp: &SecpCtx,
+) -> Result<(Miniscript<DescriptorPublicKey, Ctx>, KeyMap, ValidNetworks), DescriptorError> {
+    let (key, key_map, valid_networks) = descriptor_key.into_descriptor_key()?.extract(secp)?;
+    let minisc = Miniscript::from_ast(Terminal::PkH(key))?;
+
+    minisc.check_miniscript()?;
+
+    Ok((minisc, key_map, valid_networks))
+}
+
+// Used internally by `bdk::fragment!` to build `multi()` fragments
+#[doc(hidden)]
+pub fn make_multi<
+    Pk: IntoDescriptorKey<Ctx>,
+    Ctx: ScriptContext,
+    V: Fn(usize, Vec<DescriptorPublicKey>) -> Terminal<DescriptorPublicKey, Ctx>,
+>(
+    thresh: usize,
+    variant: V,
+    pks: Vec<Pk>,
+    secp: &SecpCtx,
+) -> Result<(Miniscript<DescriptorPublicKey, Ctx>, KeyMap, ValidNetworks), DescriptorError> {
+    let (pks, key_map, valid_networks) = expand_multi_keys(pks, secp)?;
+    let minisc = Miniscript::from_ast(variant(thresh, pks))?;
+
+    minisc.check_miniscript()?;
+
+    Ok((minisc, key_map, valid_networks))
+}
+
+// Used internally by `bdk::descriptor!` to build `sortedmulti()` fragments
+#[doc(hidden)]
+pub fn make_sortedmulti<Pk, Ctx, F>(
+    thresh: usize,
+    pks: Vec<Pk>,
+    build_desc: F,
+    secp: &SecpCtx,
+) -> Result<(Descriptor<DescriptorPublicKey>, KeyMap, ValidNetworks), DescriptorError>
+where
+    Pk: IntoDescriptorKey<Ctx>,
+    Ctx: ScriptContext,
+    F: Fn(
+        usize,
+        Vec<DescriptorPublicKey>,
+    ) -> Result<(Descriptor<DescriptorPublicKey>, PhantomData<Ctx>), DescriptorError>,
+{
+    let (pks, key_map, valid_networks) = expand_multi_keys(pks, secp)?;
+    let descriptor = build_desc(thresh, pks)?.0;
+
+    Ok((descriptor, key_map, valid_networks))
+}
+
+/// The "identity" conversion is used internally by some `bdk::fragment`s
+impl<Ctx: ScriptContext> IntoDescriptorKey<Ctx> for DescriptorKey<Ctx> {
+    fn into_descriptor_key(self) -> Result<DescriptorKey<Ctx>, KeyError> {
+        Ok(self)
+    }
+}
+
+impl<Ctx: ScriptContext> IntoDescriptorKey<Ctx> for DescriptorPublicKey {
+    fn into_descriptor_key(self) -> Result<DescriptorKey<Ctx>, KeyError> {
+        let networks = match self {
+            DescriptorPublicKey::Single(_) => any_network(),
+            DescriptorPublicKey::XPub(DescriptorXKey { xkey, .. })
+                if xkey.network == Network::Bitcoin =>
+            {
+                mainnet_network()
+            }
+            _ => test_networks(),
+        };
+
+        Ok(DescriptorKey::from_public(self, networks))
+    }
+}
+
+impl<Ctx: ScriptContext> IntoDescriptorKey<Ctx> for PublicKey {
+    fn into_descriptor_key(self) -> Result<DescriptorKey<Ctx>, KeyError> {
+        DescriptorPublicKey::Single(SinglePub {
+            key: SinglePubKey::FullKey(self),
+            origin: None,
+        })
+        .into_descriptor_key()
+    }
+}
+
+impl<Ctx: ScriptContext> IntoDescriptorKey<Ctx> for XOnlyPublicKey {
+    fn into_descriptor_key(self) -> Result<DescriptorKey<Ctx>, KeyError> {
+        DescriptorPublicKey::Single(SinglePub {
+            key: SinglePubKey::XOnly(self),
+            origin: None,
+        })
+        .into_descriptor_key()
+    }
+}
+
+impl<Ctx: ScriptContext> IntoDescriptorKey<Ctx> for DescriptorSecretKey {
+    fn into_descriptor_key(self) -> Result<DescriptorKey<Ctx>, KeyError> {
+        let networks = match &self {
+            DescriptorSecretKey::Single(sk) if sk.key.network == Network::Bitcoin => {
+                mainnet_network()
+            }
+            DescriptorSecretKey::XPrv(DescriptorXKey { xkey, .. })
+                if xkey.network == Network::Bitcoin =>
+            {
+                mainnet_network()
+            }
+            _ => test_networks(),
+        };
+
+        Ok(DescriptorKey::from_secret(self, networks))
+    }
+}
+
+impl<Ctx: ScriptContext> IntoDescriptorKey<Ctx> for &'_ str {
+    fn into_descriptor_key(self) -> Result<DescriptorKey<Ctx>, KeyError> {
+        DescriptorSecretKey::from_str(self)
+            .map_err(|e| KeyError::Message(e.to_string()))?
+            .into_descriptor_key()
+    }
+}
+
+impl<Ctx: ScriptContext> IntoDescriptorKey<Ctx> for PrivateKey {
+    fn into_descriptor_key(self) -> Result<DescriptorKey<Ctx>, KeyError> {
+        DescriptorSecretKey::Single(SinglePriv {
+            key: self,
+            origin: None,
+        })
+        .into_descriptor_key()
+    }
+}
+
+/// Errors thrown while working with [`keys`](crate::keys)
+#[derive(Debug)]
+pub enum KeyError {
+    /// The key cannot exist in the given script context
+    InvalidScriptContext,
+    /// The key is not valid for the given network
+    InvalidNetwork,
+    /// The key has an invalid checksum
+    InvalidChecksum,
+
+    /// Custom error message
+    Message(String),
+
+    /// BIP32 error
+    Bip32(bitcoin::util::bip32::Error),
+    /// Miniscript error
+    Miniscript(miniscript::Error),
+}
+
+impl_error!(miniscript::Error, Miniscript, KeyError);
+impl_error!(bitcoin::util::bip32::Error, Bip32, KeyError);
+
+impl fmt::Display for KeyError {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        match self {
+            Self::InvalidScriptContext => write!(f, "Invalid script context"),
+            Self::InvalidNetwork => write!(f, "Invalid network"),
+            Self::InvalidChecksum => write!(f, "Invalid checksum"),
+            Self::Message(err) => write!(f, "{}", err),
+            Self::Bip32(err) => write!(f, "BIP32 error: {}", err),
+            Self::Miniscript(err) => write!(f, "Miniscript error: {}", err),
+        }
+    }
+}
+
+#[cfg(feature = "std")]
+impl std::error::Error for KeyError {}
+
+#[cfg(test)]
+pub mod test {
+    use bitcoin::util::bip32;
+
+    use super::*;
+
+    pub const TEST_ENTROPY: [u8; 32] = [0xAA; 32];
+
+    #[test]
+    fn test_keys_generate_xprv() {
+        let generated_xprv: GeneratedKey<_, miniscript::Segwitv0> =
+            bip32::ExtendedPrivKey::generate_with_entropy_default(TEST_ENTROPY).unwrap();
+
+        assert_eq!(generated_xprv.valid_networks, any_network());
+        assert_eq!(generated_xprv.to_string(), "xprv9s21ZrQH143K4Xr1cJyqTvuL2FWR8eicgY9boWqMBv8MDVUZ65AXHnzBrK1nyomu6wdcabRgmGTaAKawvhAno1V5FowGpTLVx3jxzE5uk3Q");
+    }
+
+    #[test]
+    fn test_keys_generate_wif() {
+        let generated_wif: GeneratedKey<_, miniscript::Segwitv0> =
+            bitcoin::PrivateKey::generate_with_entropy_default(TEST_ENTROPY).unwrap();
+
+        assert_eq!(generated_wif.valid_networks, any_network());
+        assert_eq!(
+            generated_wif.to_string(),
+            "L2wTu6hQrnDMiFNWA5na6jB12ErGQqtXwqpSL7aWquJaZG8Ai3ch"
+        );
+    }
+
+    #[cfg(feature = "keys-bip39")]
+    #[test]
+    fn test_keys_wif_network_bip39() {
+        let xkey: ExtendedKey = bip39::Mnemonic::parse_in(
+            bip39::Language::English,
+            "jelly crash boy whisper mouse ecology tuna soccer memory million news short",
+        )
+        .unwrap()
+        .into_extended_key()
+        .unwrap();
+        let xprv = xkey.into_xprv(Network::Testnet).unwrap();
+
+        assert_eq!(xprv.network, Network::Testnet);
+    }
+}

crates/bdk/src/lib.rs

@@ -0,0 +1,54 @@
+#![doc = include_str!("../README.md")]
+// only enables the `doc_cfg` feature when the `docsrs` configuration attribute is defined
+#![cfg_attr(docsrs, feature(doc_cfg))]
+#![cfg_attr(
+    docsrs,
+    doc(html_logo_url = "https://github.com/bitcoindevkit/bdk/raw/master/static/bdk.png")
+)]
+#![no_std]
+#![warn(missing_docs)]
+
+#[cfg(feature = "std")]
+#[macro_use]
+extern crate std;
+
+#[doc(hidden)]
+#[macro_use]
+pub extern crate alloc;
+
+pub extern crate bitcoin;
+#[cfg(feature = "hardware-signer")]
+pub extern crate hwi;
+extern crate log;
+pub extern crate miniscript;
+extern crate serde;
+extern crate serde_json;
+
+#[cfg(feature = "keys-bip39")]
+extern crate bip39;
+
+#[allow(unused_imports)]
+#[macro_use]
+pub(crate) mod error;
+pub mod descriptor;
+pub mod keys;
+pub mod psbt;
+pub(crate) mod types;
+pub mod wallet;
+
+pub use descriptor::template;
+pub use descriptor::HdKeyPaths;
+pub use error::Error;
+pub use types::*;
+pub use wallet::signer;
+pub use wallet::signer::SignOptions;
+pub use wallet::tx_builder::TxBuilder;
+pub use wallet::Wallet;
+
+/// Get the version of BDK at runtime
+pub fn version() -> &'static str {
+    env!("CARGO_PKG_VERSION", "unknown")
+}
+
+pub use bdk_chain as chain;
+pub(crate) use bdk_chain::collections;

crates/bdk/src/psbt/mod.rs

@@ -0,0 +1,79 @@
+// Bitcoin Dev Kit
+// Written in 2020 by Alekos Filini <alekos.filini@gmail.com>
+//
+// Copyright (c) 2020-2021 Bitcoin Dev Kit Developers
+//
+// This file is licensed under the Apache License, Version 2.0 <LICENSE-APACHE
+// or http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option.
+// You may not use this file except in accordance with one or both of these
+// licenses.
+
+//! Additional functions on the `rust-bitcoin` `PartiallySignedTransaction` structure.
+
+use crate::FeeRate;
+use alloc::vec::Vec;
+use bitcoin::util::psbt::PartiallySignedTransaction as Psbt;
+use bitcoin::TxOut;
+
+// TODO upstream the functions here to `rust-bitcoin`?
+
+/// Trait to add functions to extract utxos and calculate fees.
+pub trait PsbtUtils {
+    /// Get the `TxOut` for the specified input index, if it doesn't exist in the PSBT `None` is returned.
+    fn get_utxo_for(&self, input_index: usize) -> Option<TxOut>;
+
+    /// The total transaction fee amount, sum of input amounts minus sum of output amounts, in sats.
+    /// If the PSBT is missing a TxOut for an input returns None.
+    fn fee_amount(&self) -> Option<u64>;
+
+    /// The transaction's fee rate. This value will only be accurate if calculated AFTER the
+    /// `PartiallySignedTransaction` is finalized and all witness/signature data is added to the
+    /// transaction.
+    /// If the PSBT is missing a TxOut for an input returns None.
+    fn fee_rate(&self) -> Option<FeeRate>;
+}
+
+impl PsbtUtils for Psbt {
+    #[allow(clippy::all)] // We want to allow `manual_map` but it is too new.
+    fn get_utxo_for(&self, input_index: usize) -> Option<TxOut> {
+        let tx = &self.unsigned_tx;
+
+        if input_index >= tx.input.len() {
+            return None;
+        }
+
+        if let Some(input) = self.inputs.get(input_index) {
+            if let Some(wit_utxo) = &input.witness_utxo {
+                Some(wit_utxo.clone())
+            } else if let Some(in_tx) = &input.non_witness_utxo {
+                Some(in_tx.output[tx.input[input_index].previous_output.vout as usize].clone())
+            } else {
+                None
+            }
+        } else {
+            None
+        }
+    }
+
+    fn fee_amount(&self) -> Option<u64> {
+        let tx = &self.unsigned_tx;
+        let utxos: Option<Vec<TxOut>> = (0..tx.input.len()).map(|i| self.get_utxo_for(i)).collect();
+
+        utxos.map(|inputs| {
+            let input_amount: u64 = inputs.iter().map(|i| i.value).sum();
+            let output_amount: u64 = self.unsigned_tx.output.iter().map(|o| o.value).sum();
+            input_amount
+                .checked_sub(output_amount)
+                .expect("input amount must be greater than output amount")
+        })
+    }
+
+    fn fee_rate(&self) -> Option<FeeRate> {
+        let fee_amount = self.fee_amount();
+        fee_amount.map(|fee| {
+            let weight = self.clone().extract_tx().weight();
+            FeeRate::from_wu(fee, weight)
+        })
+    }
+}

crates/bdk/src/types.rs

@@ -0,0 +1,339 @@
+// Bitcoin Dev Kit
+// Written in 2020 by Alekos Filini <alekos.filini@gmail.com>
+//
+// Copyright (c) 2020-2021 Bitcoin Dev Kit Developers
+//
+// This file is licensed under the Apache License, Version 2.0 <LICENSE-APACHE
+// or http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option.
+// You may not use this file except in accordance with one or both of these
+// licenses.
+
+use alloc::boxed::Box;
+use core::convert::AsRef;
+use core::ops::Sub;
+
+use bdk_chain::ConfirmationTime;
+use bitcoin::blockdata::transaction::{OutPoint, Transaction, TxOut};
+use bitcoin::{hash_types::Txid, util::psbt};
+
+use serde::{Deserialize, Serialize};
+
+/// Types of keychains
+#[derive(Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Eq, Hash, Ord, PartialOrd)]
+pub enum KeychainKind {
+    /// External keychain, used for deriving recipient addresses.
+    External = 0,
+    /// Internal keychain, used for deriving change addresses.
+    Internal = 1,
+}
+
+impl KeychainKind {
+    /// Return [`KeychainKind`] as a byte
+    pub fn as_byte(&self) -> u8 {
+        match self {
+            KeychainKind::External => b'e',
+            KeychainKind::Internal => b'i',
+        }
+    }
+}
+
+impl AsRef<[u8]> for KeychainKind {
+    fn as_ref(&self) -> &[u8] {
+        match self {
+            KeychainKind::External => b"e",
+            KeychainKind::Internal => b"i",
+        }
+    }
+}
+
+/// Fee rate
+#[derive(Debug, Copy, Clone, PartialEq, PartialOrd)]
+// Internally stored as satoshi/vbyte
+pub struct FeeRate(f32);
+
+impl FeeRate {
+    /// Create a new instance checking the value provided
+    ///
+    /// ## Panics
+    ///
+    /// Panics if the value is not [normal](https://doc.rust-lang.org/std/primitive.f32.html#method.is_normal) (except if it's a positive zero) or negative.
+    fn new_checked(value: f32) -> Self {
+        assert!(value.is_normal() || value == 0.0);
+        assert!(value.is_sign_positive());
+
+        FeeRate(value)
+    }
+
+    /// Create a new instance of [`FeeRate`] given a float fee rate in sats/kwu
+    pub fn from_sat_per_kwu(sat_per_kwu: f32) -> Self {
+        FeeRate::new_checked(sat_per_kwu / 250.0_f32)
+    }
+
+    /// Create a new instance of [`FeeRate`] given a float fee rate in sats/kvb
+    pub fn from_sat_per_kvb(sat_per_kvb: f32) -> Self {
+        FeeRate::new_checked(sat_per_kvb / 1000.0_f32)
+    }
+
+    /// Create a new instance of [`FeeRate`] given a float fee rate in btc/kvbytes
+    ///
+    /// ## Panics
+    ///
+    /// Panics if the value is not [normal](https://doc.rust-lang.org/std/primitive.f32.html#method.is_normal) (except if it's a positive zero) or negative.
+    pub fn from_btc_per_kvb(btc_per_kvb: f32) -> Self {
+        FeeRate::new_checked(btc_per_kvb * 1e5)
+    }
+
+    /// Create a new instance of [`FeeRate`] given a float fee rate in satoshi/vbyte
+    ///
+    /// ## Panics
+    ///
+    /// Panics if the value is not [normal](https://doc.rust-lang.org/std/primitive.f32.html#method.is_normal) (except if it's a positive zero) or negative.
+    pub fn from_sat_per_vb(sat_per_vb: f32) -> Self {
+        FeeRate::new_checked(sat_per_vb)
+    }
+
+    /// Create a new [`FeeRate`] with the default min relay fee value
+    pub const fn default_min_relay_fee() -> Self {
+        FeeRate(1.0)
+    }
+
+    /// Calculate fee rate from `fee` and weight units (`wu`).
+    pub fn from_wu(fee: u64, wu: usize) -> FeeRate {
+        Self::from_vb(fee, wu.vbytes())
+    }
+
+    /// Calculate fee rate from `fee` and `vbytes`.
+    pub fn from_vb(fee: u64, vbytes: usize) -> FeeRate {
+        let rate = fee as f32 / vbytes as f32;
+        Self::from_sat_per_vb(rate)
+    }
+
+    /// Return the value as satoshi/vbyte
+    pub fn as_sat_per_vb(&self) -> f32 {
+        self.0
+    }
+
+    /// Return the value as satoshi/kwu
+    pub fn sat_per_kwu(&self) -> f32 {
+        self.0 * 250.0_f32
+    }
+
+    /// Calculate absolute fee in Satoshis using size in weight units.
+    pub fn fee_wu(&self, wu: usize) -> u64 {
+        self.fee_vb(wu.vbytes())
+    }
+
+    /// Calculate absolute fee in Satoshis using size in virtual bytes.
+    pub fn fee_vb(&self, vbytes: usize) -> u64 {
+        (self.as_sat_per_vb() * vbytes as f32).ceil() as u64
+    }
+}
+
+impl Default for FeeRate {
+    fn default() -> Self {
+        FeeRate::default_min_relay_fee()
+    }
+}
+
+impl Sub for FeeRate {
+    type Output = Self;
+
+    fn sub(self, other: FeeRate) -> Self::Output {
+        FeeRate(self.0 - other.0)
+    }
+}
+
+/// Trait implemented by types that can be used to measure weight units.
+pub trait Vbytes {
+    /// Convert weight units to virtual bytes.
+    fn vbytes(self) -> usize;
+}
+
+impl Vbytes for usize {
+    fn vbytes(self) -> usize {
+        // ref: https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#transaction-size-calculations
+        (self as f32 / 4.0).ceil() as usize
+    }
+}
+
+/// An unspent output owned by a [`Wallet`].
+///
+/// [`Wallet`]: crate::Wallet
+#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq, Hash)]
+pub struct LocalUtxo {
+    /// Reference to a transaction output
+    pub outpoint: OutPoint,
+    /// Transaction output
+    pub txout: TxOut,
+    /// Type of keychain
+    pub keychain: KeychainKind,
+    /// Whether this UTXO is spent or not
+    pub is_spent: bool,
+    /// The derivation index for the script pubkey in the wallet
+    pub derivation_index: u32,
+    /// The confirmation time for transaction containing this utxo
+    pub confirmation_time: ConfirmationTime,
+}
+
+/// A [`Utxo`] with its `satisfaction_weight`.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct WeightedUtxo {
+    /// The weight of the witness data and `scriptSig` expressed in [weight units]. This is used to
+    /// properly maintain the feerate when adding this input to a transaction during coin selection.
+    ///
+    /// [weight units]: https://en.bitcoin.it/wiki/Weight_units
+    pub satisfaction_weight: usize,
+    /// The UTXO
+    pub utxo: Utxo,
+}
+
+#[derive(Debug, Clone, PartialEq, Eq)]
+/// An unspent transaction output (UTXO).
+pub enum Utxo {
+    /// A UTXO owned by the local wallet.
+    Local(LocalUtxo),
+    /// A UTXO owned by another wallet.
+    Foreign {
+        /// The location of the output.
+        outpoint: OutPoint,
+        /// The information about the input we require to add it to a PSBT.
+        // Box it to stop the type being too big.
+        psbt_input: Box<psbt::Input>,
+    },
+}
+
+impl Utxo {
+    /// Get the location of the UTXO
+    pub fn outpoint(&self) -> OutPoint {
+        match &self {
+            Utxo::Local(local) => local.outpoint,
+            Utxo::Foreign { outpoint, .. } => *outpoint,
+        }
+    }
+
+    /// Get the `TxOut` of the UTXO
+    pub fn txout(&self) -> &TxOut {
+        match &self {
+            Utxo::Local(local) => &local.txout,
+            Utxo::Foreign {
+                outpoint,
+                psbt_input,
+            } => {
+                if let Some(prev_tx) = &psbt_input.non_witness_utxo {
+                    return &prev_tx.output[outpoint.vout as usize];
+                }
+
+                if let Some(txout) = &psbt_input.witness_utxo {
+                    return txout;
+                }
+
+                unreachable!("Foreign UTXOs will always have one of these set")
+            }
+        }
+    }
+}
+
+/// A wallet transaction
+#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
+pub struct TransactionDetails {
+    /// Optional transaction
+    pub transaction: Option<Transaction>,
+    /// Transaction id
+    pub txid: Txid,
+    /// Received value (sats)
+    /// Sum of owned outputs of this transaction.
+    pub received: u64,
+    /// Sent value (sats)
+    /// Sum of owned inputs of this transaction.
+    pub sent: u64,
+    /// Fee value in sats if it was available.
+    pub fee: Option<u64>,
+    /// If the transaction is confirmed, contains height and Unix timestamp of the block containing the
+    /// transaction, unconfirmed transaction contains `None`.
+    pub confirmation_time: ConfirmationTime,
+}
+
+impl PartialOrd for TransactionDetails {
+    fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
+        Some(self.cmp(other))
+    }
+}
+
+impl Ord for TransactionDetails {
+    fn cmp(&self, other: &Self) -> core::cmp::Ordering {
+        self.confirmation_time
+            .cmp(&other.confirmation_time)
+            .then_with(|| self.txid.cmp(&other.txid))
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn can_store_feerate_in_const() {
+        const _MIN_RELAY: FeeRate = FeeRate::default_min_relay_fee();
+    }
+
+    #[test]
+    #[should_panic]
+    fn test_invalid_feerate_neg_zero() {
+        let _ = FeeRate::from_sat_per_vb(-0.0);
+    }
+
+    #[test]
+    #[should_panic]
+    fn test_invalid_feerate_neg_value() {
+        let _ = FeeRate::from_sat_per_vb(-5.0);
+    }
+
+    #[test]
+    #[should_panic]
+    fn test_invalid_feerate_nan() {
+        let _ = FeeRate::from_sat_per_vb(f32::NAN);
+    }
+
+    #[test]
+    #[should_panic]
+    fn test_invalid_feerate_inf() {
+        let _ = FeeRate::from_sat_per_vb(f32::INFINITY);
+    }
+
+    #[test]
+    fn test_valid_feerate_pos_zero() {
+        let _ = FeeRate::from_sat_per_vb(0.0);
+    }
+
+    #[test]
+    fn test_fee_from_btc_per_kvb() {
+        let fee = FeeRate::from_btc_per_kvb(1e-5);
+        assert!((fee.as_sat_per_vb() - 1.0).abs() < f32::EPSILON);
+    }
+
+    #[test]
+    fn test_fee_from_sat_per_vbyte() {
+        let fee = FeeRate::from_sat_per_vb(1.0);
+        assert!((fee.as_sat_per_vb() - 1.0).abs() < f32::EPSILON);
+    }
+
+    #[test]
+    fn test_fee_default_min_relay_fee() {
+        let fee = FeeRate::default_min_relay_fee();
+        assert!((fee.as_sat_per_vb() - 1.0).abs() < f32::EPSILON);
+    }
+
+    #[test]
+    fn test_fee_from_sat_per_kvb() {
+        let fee = FeeRate::from_sat_per_kvb(1000.0);
+        assert!((fee.as_sat_per_vb() - 1.0).abs() < f32::EPSILON);
+    }
+
+    #[test]
+    fn test_fee_from_sat_per_kwu() {
+        let fee = FeeRate::from_sat_per_kwu(250.0);
+        assert!((fee.as_sat_per_vb() - 1.0).abs() < f32::EPSILON);
+        assert_eq!(fee.sat_per_kwu(), 250.0);
+    }
+}

crates/bdk/src/wallet/export.rs

@@ -0,0 +1,341 @@
+// Bitcoin Dev Kit
+// Written in 2020 by Alekos Filini <alekos.filini@gmail.com>
+//
+// Copyright (c) 2020-2021 Bitcoin Dev Kit Developers
+//
+// This file is licensed under the Apache License, Version 2.0 <LICENSE-APACHE
+// or http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option.
+// You may not use this file except in accordance with one or both of these
+// licenses.
+
+//! Wallet export
+//!
+//! This modules implements the wallet export format used by [FullyNoded](https://github.com/Fonta1n3/FullyNoded/blob/10b7808c8b929b171cca537fb50522d015168ac9/Docs/Wallets/Wallet-Export-Spec.md).
+//!
+//! ## Examples
+//!
+//! ### Import from JSON
+//!
+//! ```
+//! # use std::str::FromStr;
+//! # use bitcoin::*;
+//! # use bdk::wallet::export::*;
+//! # use bdk::*;
+//! let import = r#"{
+//!     "descriptor": "wpkh([c258d2e4\/84h\/1h\/0h]tpubDD3ynpHgJQW8VvWRzQ5WFDCrs4jqVFGHB3vLC3r49XHJSqP8bHKdK4AriuUKLccK68zfzowx7YhmDN8SiSkgCDENUFx9qVw65YyqM78vyVe\/0\/*)",
+//!     "blockheight":1782088,
+//!     "label":"testnet"
+//! }"#;
+//!
+//! let import = FullyNodedExport::from_str(import)?;
+//! let wallet = Wallet::new_no_persist(
+//!     &import.descriptor(),
+//!     import.change_descriptor().as_ref(),
+//!     Network::Testnet,
+//! )?;
+//! # Ok::<_, Box<dyn std::error::Error>>(())
+//! ```
+//!
+//! ### Export a `Wallet`
+//! ```
+//! # use bitcoin::*;
+//! # use bdk::wallet::export::*;
+//! # use bdk::*;
+//! let wallet = Wallet::new_no_persist(
+//!     "wpkh([c258d2e4/84h/1h/0h]tpubDD3ynpHgJQW8VvWRzQ5WFDCrs4jqVFGHB3vLC3r49XHJSqP8bHKdK4AriuUKLccK68zfzowx7YhmDN8SiSkgCDENUFx9qVw65YyqM78vyVe/0/*)",
+//!     Some("wpkh([c258d2e4/84h/1h/0h]tpubDD3ynpHgJQW8VvWRzQ5WFDCrs4jqVFGHB3vLC3r49XHJSqP8bHKdK4AriuUKLccK68zfzowx7YhmDN8SiSkgCDENUFx9qVw65YyqM78vyVe/1/*)"),
+//!     Network::Testnet,
+//! )?;
+//! let export = FullyNodedExport::export_wallet(&wallet, "exported wallet", true).unwrap();
+//!
+//! println!("Exported: {}", export.to_string());
+//! # Ok::<_, Box<dyn std::error::Error>>(())
+//! ```
+
+use core::str::FromStr;
+
+use alloc::string::{String, ToString};
+use serde::{Deserialize, Serialize};
+
+use miniscript::descriptor::{ShInner, WshInner};
+use miniscript::{Descriptor, ScriptContext, Terminal};
+
+use crate::types::KeychainKind;
+use crate::wallet::Wallet;
+
+/// Alias for [`FullyNodedExport`]
+#[deprecated(since = "0.18.0", note = "Please use [`FullyNodedExport`] instead")]
+pub type WalletExport = FullyNodedExport;
+
+/// Structure that contains the export of a wallet
+///
+/// For a usage example see [this module](crate::wallet::export)'s documentation.
+#[derive(Debug, Serialize, Deserialize)]
+pub struct FullyNodedExport {
+    descriptor: String,
+    /// Earliest block to rescan when looking for the wallet's transactions
+    pub blockheight: u32,
+    /// Arbitrary label for the wallet
+    pub label: String,
+}
+
+impl ToString for FullyNodedExport {
+    fn to_string(&self) -> String {
+        serde_json::to_string(self).unwrap()
+    }
+}
+
+impl FromStr for FullyNodedExport {
+    type Err = serde_json::Error;
+
+    fn from_str(s: &str) -> Result<Self, Self::Err> {
+        serde_json::from_str(s)
+    }
+}
+
+fn remove_checksum(s: String) -> String {
+    s.split_once('#').map(|(a, _)| String::from(a)).unwrap()
+}
+
+impl FullyNodedExport {
+    /// Export a wallet
+    ///
+    /// This function returns an error if it determines that the `wallet`'s descriptor(s) are not
+    /// supported by Bitcoin Core or don't follow the standard derivation paths defined by BIP44
+    /// and others.
+    ///
+    /// If `include_blockheight` is `true`, this function will look into the `wallet`'s database
+    /// for the oldest transaction it knows and use that as the earliest block to rescan.
+    ///
+    /// If the database is empty or `include_blockheight` is false, the `blockheight` field
+    /// returned will be `0`.
+    pub fn export_wallet<D>(
+        wallet: &Wallet<D>,
+        label: &str,
+        include_blockheight: bool,
+    ) -> Result<Self, &'static str> {
+        let descriptor = wallet
+            .get_descriptor_for_keychain(KeychainKind::External)
+            .to_string_with_secret(
+                &wallet
+                    .get_signers(KeychainKind::External)
+                    .as_key_map(wallet.secp_ctx()),
+            );
+        let descriptor = remove_checksum(descriptor);
+        Self::is_compatible_with_core(&descriptor)?;
+
+        let blockheight = if include_blockheight {
+            wallet
+                .transactions()
+                .next()
+                .map_or(0, |canonical_tx| match canonical_tx.observed_as {
+                    bdk_chain::ChainPosition::Confirmed(a) => a.confirmation_height,
+                    bdk_chain::ChainPosition::Unconfirmed(_) => 0,
+                })
+        } else {
+            0
+        };
+
+        let export = FullyNodedExport {
+            descriptor,
+            label: label.into(),
+            blockheight,
+        };
+
+        let change_descriptor = match wallet.public_descriptor(KeychainKind::Internal).is_some() {
+            false => None,
+            true => {
+                let descriptor = wallet
+                    .get_descriptor_for_keychain(KeychainKind::Internal)
+                    .to_string_with_secret(
+                        &wallet
+                            .get_signers(KeychainKind::Internal)
+                            .as_key_map(wallet.secp_ctx()),
+                    );
+                Some(remove_checksum(descriptor))
+            }
+        };
+        if export.change_descriptor() != change_descriptor {
+            return Err("Incompatible change descriptor");
+        }
+
+        Ok(export)
+    }
+
+    fn is_compatible_with_core(descriptor: &str) -> Result<(), &'static str> {
+        fn check_ms<Ctx: ScriptContext>(
+            terminal: &Terminal<String, Ctx>,
+        ) -> Result<(), &'static str> {
+            if let Terminal::Multi(_, _) = terminal {
+                Ok(())
+            } else {
+                Err("The descriptor contains operators not supported by Bitcoin Core")
+            }
+        }
+
+        // pkh(), wpkh(), sh(wpkh()) are always fine, as well as multi() and sortedmulti()
+        match Descriptor::<String>::from_str(descriptor).map_err(|_| "Invalid descriptor")? {
+            Descriptor::Pkh(_) | Descriptor::Wpkh(_) => Ok(()),
+            Descriptor::Sh(sh) => match sh.as_inner() {
+                ShInner::Wpkh(_) => Ok(()),
+                ShInner::SortedMulti(_) => Ok(()),
+                ShInner::Wsh(wsh) => match wsh.as_inner() {
+                    WshInner::SortedMulti(_) => Ok(()),
+                    WshInner::Ms(ms) => check_ms(&ms.node),
+                },
+                ShInner::Ms(ms) => check_ms(&ms.node),
+            },
+            Descriptor::Wsh(wsh) => match wsh.as_inner() {
+                WshInner::SortedMulti(_) => Ok(()),
+                WshInner::Ms(ms) => check_ms(&ms.node),
+            },
+            _ => Err("The descriptor is not compatible with Bitcoin Core"),
+        }
+    }
+
+    /// Return the external descriptor
+    pub fn descriptor(&self) -> String {
+        self.descriptor.clone()
+    }
+
+    /// Return the internal descriptor, if present
+    pub fn change_descriptor(&self) -> Option<String> {
+        let replaced = self.descriptor.replace("/0/*", "/1/*");
+
+        if replaced != self.descriptor {
+            Some(replaced)
+        } else {
+            None
+        }
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use core::str::FromStr;
+
+    use bdk_chain::{BlockId, ConfirmationTime};
+    use bitcoin::hashes::Hash;
+    use bitcoin::{BlockHash, Network, Transaction};
+
+    use super::*;
+    use crate::wallet::Wallet;
+
+    fn get_test_wallet(
+        descriptor: &str,
+        change_descriptor: Option<&str>,
+        network: Network,
+    ) -> Wallet<()> {
+        let mut wallet = Wallet::new_no_persist(descriptor, change_descriptor, network).unwrap();
+        let transaction = Transaction {
+            input: vec![],
+            output: vec![],
+            version: 0,
+            lock_time: bitcoin::PackedLockTime::ZERO,
+        };
+        wallet
+            .insert_checkpoint(BlockId {
+                height: 5001,
+                hash: BlockHash::all_zeros(),
+            })
+            .unwrap();
+        wallet
+            .insert_tx(
+                transaction,
+                ConfirmationTime::Confirmed {
+                    height: 5000,
+                    time: 0,
+                },
+            )
+            .unwrap();
+        wallet
+    }
+
+    #[test]
+    fn test_export_bip44() {
+        let descriptor = "wpkh(xprv9s21ZrQH143K4CTb63EaMxja1YiTnSEWKMbn23uoEnAzxjdUJRQkazCAtzxGm4LSoTSVTptoV9RbchnKPW9HxKtZumdyxyikZFDLhogJ5Uj/44'/0'/0'/0/*)";
+        let change_descriptor = "wpkh(xprv9s21ZrQH143K4CTb63EaMxja1YiTnSEWKMbn23uoEnAzxjdUJRQkazCAtzxGm4LSoTSVTptoV9RbchnKPW9HxKtZumdyxyikZFDLhogJ5Uj/44'/0'/0'/1/*)";
+
+        let wallet = get_test_wallet(descriptor, Some(change_descriptor), Network::Bitcoin);
+        let export = FullyNodedExport::export_wallet(&wallet, "Test Label", true).unwrap();
+
+        assert_eq!(export.descriptor(), descriptor);
+        assert_eq!(export.change_descriptor(), Some(change_descriptor.into()));
+        assert_eq!(export.blockheight, 5000);
+        assert_eq!(export.label, "Test Label");
+    }
+
+    #[test]
+    #[should_panic(expected = "Incompatible change descriptor")]
+    fn test_export_no_change() {
+        // This wallet explicitly doesn't have a change descriptor. It should be impossible to
+        // export, because exporting this kind of external descriptor normally implies the
+        // existence of an internal descriptor
+
+        let descriptor = "wpkh(xprv9s21ZrQH143K4CTb63EaMxja1YiTnSEWKMbn23uoEnAzxjdUJRQkazCAtzxGm4LSoTSVTptoV9RbchnKPW9HxKtZumdyxyikZFDLhogJ5Uj/44'/0'/0'/0/*)";
+
+        let wallet = get_test_wallet(descriptor, None, Network::Bitcoin);
+        FullyNodedExport::export_wallet(&wallet, "Test Label", true).unwrap();
+    }
+
+    #[test]
+    #[should_panic(expected = "Incompatible change descriptor")]
+    fn test_export_incompatible_change() {
+        // This wallet has a change descriptor, but the derivation path is not in the "standard"
+        // bip44/49/etc format
+
+        let descriptor = "wpkh(xprv9s21ZrQH143K4CTb63EaMxja1YiTnSEWKMbn23uoEnAzxjdUJRQkazCAtzxGm4LSoTSVTptoV9RbchnKPW9HxKtZumdyxyikZFDLhogJ5Uj/44'/0'/0'/0/*)";
+        let change_descriptor = "wpkh(xprv9s21ZrQH143K4CTb63EaMxja1YiTnSEWKMbn23uoEnAzxjdUJRQkazCAtzxGm4LSoTSVTptoV9RbchnKPW9HxKtZumdyxyikZFDLhogJ5Uj/50'/0'/1/*)";
+
+        let wallet = get_test_wallet(descriptor, Some(change_descriptor), Network::Bitcoin);
+        FullyNodedExport::export_wallet(&wallet, "Test Label", true).unwrap();
+    }
+
+    #[test]
+    fn test_export_multi() {
+        let descriptor = "wsh(multi(2,\
+                                [73756c7f/48'/0'/0'/2']tpubDCKxNyM3bLgbEX13Mcd8mYxbVg9ajDkWXMh29hMWBurKfVmBfWAM96QVP3zaUcN51HvkZ3ar4VwP82kC8JZhhux8vFQoJintSpVBwpFvyU3/0/*,\
+                                [f9f62194/48'/0'/0'/2']tpubDDp3ZSH1yCwusRppH7zgSxq2t1VEUyXSeEp8E5aFS8m43MknUjiF1bSLo3CGWAxbDyhF1XowA5ukPzyJZjznYk3kYi6oe7QxtX2euvKWsk4/0/*,\
+                                [c98b1535/48'/0'/0'/2']tpubDCDi5W4sP6zSnzJeowy8rQDVhBdRARaPhK1axABi8V1661wEPeanpEXj4ZLAUEoikVtoWcyK26TKKJSecSfeKxwHCcRrge9k1ybuiL71z4a/0/*\
+                          ))";
+        let change_descriptor = "wsh(multi(2,\
+                                       [73756c7f/48'/0'/0'/2']tpubDCKxNyM3bLgbEX13Mcd8mYxbVg9ajDkWXMh29hMWBurKfVmBfWAM96QVP3zaUcN51HvkZ3ar4VwP82kC8JZhhux8vFQoJintSpVBwpFvyU3/1/*,\
+                                       [f9f62194/48'/0'/0'/2']tpubDDp3ZSH1yCwusRppH7zgSxq2t1VEUyXSeEp8E5aFS8m43MknUjiF1bSLo3CGWAxbDyhF1XowA5ukPzyJZjznYk3kYi6oe7QxtX2euvKWsk4/1/*,\
+                                       [c98b1535/48'/0'/0'/2']tpubDCDi5W4sP6zSnzJeowy8rQDVhBdRARaPhK1axABi8V1661wEPeanpEXj4ZLAUEoikVtoWcyK26TKKJSecSfeKxwHCcRrge9k1ybuiL71z4a/1/*\
+                                 ))";
+
+        let wallet = get_test_wallet(descriptor, Some(change_descriptor), Network::Testnet);
+        let export = FullyNodedExport::export_wallet(&wallet, "Test Label", true).unwrap();
+
+        assert_eq!(export.descriptor(), descriptor);
+        assert_eq!(export.change_descriptor(), Some(change_descriptor.into()));
+        assert_eq!(export.blockheight, 5000);
+        assert_eq!(export.label, "Test Label");
+    }
+
+    #[test]
+    fn test_export_to_json() {
+        let descriptor = "wpkh(xprv9s21ZrQH143K4CTb63EaMxja1YiTnSEWKMbn23uoEnAzxjdUJRQkazCAtzxGm4LSoTSVTptoV9RbchnKPW9HxKtZumdyxyikZFDLhogJ5Uj/44'/0'/0'/0/*)";
+        let change_descriptor = "wpkh(xprv9s21ZrQH143K4CTb63EaMxja1YiTnSEWKMbn23uoEnAzxjdUJRQkazCAtzxGm4LSoTSVTptoV9RbchnKPW9HxKtZumdyxyikZFDLhogJ5Uj/44'/0'/0'/1/*)";
+
+        let wallet = get_test_wallet(descriptor, Some(change_descriptor), Network::Bitcoin);
+        let export = FullyNodedExport::export_wallet(&wallet, "Test Label", true).unwrap();
+
+        assert_eq!(export.to_string(), "{\"descriptor\":\"wpkh(xprv9s21ZrQH143K4CTb63EaMxja1YiTnSEWKMbn23uoEnAzxjdUJRQkazCAtzxGm4LSoTSVTptoV9RbchnKPW9HxKtZumdyxyikZFDLhogJ5Uj/44\'/0\'/0\'/0/*)\",\"blockheight\":5000,\"label\":\"Test Label\"}");
+    }
+
+    #[test]
+    fn test_export_from_json() {
+        let descriptor = "wpkh(xprv9s21ZrQH143K4CTb63EaMxja1YiTnSEWKMbn23uoEnAzxjdUJRQkazCAtzxGm4LSoTSVTptoV9RbchnKPW9HxKtZumdyxyikZFDLhogJ5Uj/44'/0'/0'/0/*)";
+        let change_descriptor = "wpkh(xprv9s21ZrQH143K4CTb63EaMxja1YiTnSEWKMbn23uoEnAzxjdUJRQkazCAtzxGm4LSoTSVTptoV9RbchnKPW9HxKtZumdyxyikZFDLhogJ5Uj/44'/0'/0'/1/*)";
+
+        let import_str = "{\"descriptor\":\"wpkh(xprv9s21ZrQH143K4CTb63EaMxja1YiTnSEWKMbn23uoEnAzxjdUJRQkazCAtzxGm4LSoTSVTptoV9RbchnKPW9HxKtZumdyxyikZFDLhogJ5Uj/44\'/0\'/0\'/0/*)\",\"blockheight\":5000,\"label\":\"Test Label\"}";
+        let export = FullyNodedExport::from_str(import_str).unwrap();
+
+        assert_eq!(export.descriptor(), descriptor);
+        assert_eq!(export.change_descriptor(), Some(change_descriptor.into()));
+        assert_eq!(export.blockheight, 5000);
+        assert_eq!(export.label, "Test Label");
+    }
+}

crates/bdk/src/wallet/hardwaresigner.rs

@@ -0,0 +1,98 @@
+// Bitcoin Dev Kit
+// Written in 2020 by Alekos Filini <alekos.filini@gmail.com>
+//
+// Copyright (c) 2020-2021 Bitcoin Dev Kit Developers
+//
+// This file is licensed under the Apache License, Version 2.0 <LICENSE-APACHE
+// or http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option.
+// You may not use this file except in accordance with one or both of these
+// licenses.
+
+//! HWI Signer
+//!
+//! This module contains HWISigner, an implementation of a [TransactionSigner] to be
+//! used with hardware wallets.
+//! ```no_run
+//! # use bdk::bitcoin::Network;
+//! # use bdk::signer::SignerOrdering;
+//! # use bdk::wallet::hardwaresigner::HWISigner;
+//! # use bdk::wallet::AddressIndex::New;
+//! # use bdk::{FeeRate, KeychainKind, SignOptions, Wallet};
+//! # use hwi::{types::HWIChain, HWIClient};
+//! # use std::sync::Arc;
+//! #
+//! # fn main() -> Result<(), Box<dyn std::error::Error>> {
+//! let mut devices = HWIClient::enumerate()?;
+//! if devices.is_empty() {
+//!     panic!("No devices found!");
+//! }
+//! let first_device = devices.remove(0)?;
+//! let custom_signer = HWISigner::from_device(&first_device, HWIChain::Test)?;
+//!
+//! # let mut wallet = Wallet::new_no_persist(
+//! #     "",
+//! #     None,
+//! #     Network::Testnet,
+//! # )?;
+//! #
+//! // Adding the hardware signer to the BDK wallet
+//! wallet.add_signer(
+//!     KeychainKind::External,
+//!     SignerOrdering(200),
+//!     Arc::new(custom_signer),
+//! );
+//!
+//! # Ok(())
+//! # }
+//! ```
+
+use bitcoin::psbt::PartiallySignedTransaction;
+use bitcoin::secp256k1::{All, Secp256k1};
+use bitcoin::util::bip32::Fingerprint;
+
+use hwi::error::Error;
+use hwi::types::{HWIChain, HWIDevice};
+use hwi::HWIClient;
+
+use crate::signer::{SignerCommon, SignerError, SignerId, TransactionSigner};
+
+#[derive(Debug)]
+/// Custom signer for Hardware Wallets
+///
+/// This ignores `sign_options` and leaves the decisions up to the hardware wallet.
+pub struct HWISigner {
+    fingerprint: Fingerprint,
+    client: HWIClient,
+}
+
+impl HWISigner {
+    /// Create a instance from the specified device and chain
+    pub fn from_device(device: &HWIDevice, chain: HWIChain) -> Result<HWISigner, Error> {
+        let client = HWIClient::get_client(device, false, chain)?;
+        Ok(HWISigner {
+            fingerprint: device.fingerprint,
+            client,
+        })
+    }
+}
+
+impl SignerCommon for HWISigner {
+    fn id(&self, _secp: &Secp256k1<All>) -> SignerId {
+        SignerId::Fingerprint(self.fingerprint)
+    }
+}
+
+/// This implementation ignores `sign_options`
+impl TransactionSigner for HWISigner {
+    fn sign_transaction(
+        &self,
+        psbt: &mut PartiallySignedTransaction,
+        _sign_options: &crate::SignOptions,
+        _secp: &crate::wallet::utils::SecpCtx,
+    ) -> Result<(), SignerError> {
+        psbt.combine(self.client.sign_tx(psbt)?.psbt)
+            .expect("Failed to combine HW signed psbt with passed PSBT");
+        Ok(())
+    }
+}

crates/bdk/src/wallet/tx_builder.rs

@@ -0,0 +1,947 @@
+// Bitcoin Dev Kit
+// Written in 2020 by Alekos Filini <alekos.filini@gmail.com>
+//
+// Copyright (c) 2020-2021 Bitcoin Dev Kit Developers
+//
+// This file is licensed under the Apache License, Version 2.0 <LICENSE-APACHE
+// or http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option.
+// You may not use this file except in accordance with one or both of these
+// licenses.
+
+//! Transaction builder
+//!
+//! ## Example
+//!
+//! ```
+//! # use std::str::FromStr;
+//! # use bitcoin::*;
+//! # use bdk::*;
+//! # use bdk::wallet::tx_builder::CreateTx;
+//! # let to_address = Address::from_str("2N4eQYCbKUHCCTUjBJeHcJp9ok6J2GZsTDt").unwrap();
+//! # let mut wallet = doctest_wallet!();
+//! // create a TxBuilder from a wallet
+//! let mut tx_builder = wallet.build_tx();
+//!
+//! tx_builder
+//!     // Create a transaction with one output to `to_address` of 50_000 satoshi
+//!     .add_recipient(to_address.script_pubkey(), 50_000)
+//!     // With a custom fee rate of 5.0 satoshi/vbyte
+//!     .fee_rate(FeeRate::from_sat_per_vb(5.0))
+//!     // Only spend non-change outputs
+//!     .do_not_spend_change()
+//!     // Turn on RBF signaling
+//!     .enable_rbf();
+//! let (psbt, tx_details) = tx_builder.finish()?;
+//! # Ok::<(), bdk::Error>(())
+//! ```
+
+use crate::collections::BTreeMap;
+use crate::collections::HashSet;
+use alloc::{boxed::Box, rc::Rc, string::String, vec::Vec};
+use bdk_chain::PersistBackend;
+use core::cell::RefCell;
+use core::marker::PhantomData;
+
+use bitcoin::util::psbt::{self, PartiallySignedTransaction as Psbt};
+use bitcoin::{LockTime, OutPoint, Script, Sequence, Transaction};
+
+use super::coin_selection::{CoinSelectionAlgorithm, DefaultCoinSelectionAlgorithm};
+use super::ChangeSet;
+use crate::{
+    types::{FeeRate, KeychainKind, LocalUtxo, WeightedUtxo},
+    TransactionDetails,
+};
+use crate::{Error, Utxo, Wallet};
+/// Context in which the [`TxBuilder`] is valid
+pub trait TxBuilderContext: core::fmt::Debug + Default + Clone {}
+
+/// Marker type to indicate the [`TxBuilder`] is being used to create a new transaction (as opposed
+/// to bumping the fee of an existing one).
+#[derive(Debug, Default, Clone)]
+pub struct CreateTx;
+impl TxBuilderContext for CreateTx {}
+
+/// Marker type to indicate the [`TxBuilder`] is being used to bump the fee of an existing transaction.
+#[derive(Debug, Default, Clone)]
+pub struct BumpFee;
+impl TxBuilderContext for BumpFee {}
+
+/// A transaction builder
+///
+/// A `TxBuilder` is created by calling [`build_tx`] or [`build_fee_bump`] on a wallet. After
+/// assigning it, you set options on it until finally calling [`finish`] to consume the builder and
+/// generate the transaction.
+///
+/// Each option setting method on `TxBuilder` takes and returns `&mut self` so you can chain calls
+/// as in the following example:
+///
+/// ```
+/// # use bdk::*;
+/// # use bdk::wallet::tx_builder::*;
+/// # use bitcoin::*;
+/// # use core::str::FromStr;
+/// # let mut wallet = doctest_wallet!();
+/// # let addr1 = Address::from_str("2N4eQYCbKUHCCTUjBJeHcJp9ok6J2GZsTDt").unwrap();
+/// # let addr2 = addr1.clone();
+/// // chaining
+/// let (psbt1, details) = {
+///     let mut builder = wallet.build_tx();
+///     builder
+///         .ordering(TxOrdering::Untouched)
+///         .add_recipient(addr1.script_pubkey(), 50_000)
+///         .add_recipient(addr2.script_pubkey(), 50_000);
+///     builder.finish()?
+/// };
+///
+/// // non-chaining
+/// let (psbt2, details) = {
+///     let mut builder = wallet.build_tx();
+///     builder.ordering(TxOrdering::Untouched);
+///     for addr in &[addr1, addr2] {
+///         builder.add_recipient(addr.script_pubkey(), 50_000);
+///     }
+///     builder.finish()?
+/// };
+///
+/// assert_eq!(psbt1.unsigned_tx.output[..2], psbt2.unsigned_tx.output[..2]);
+/// # Ok::<(), bdk::Error>(())
+/// ```
+///
+/// At the moment [`coin_selection`] is an exception to the rule as it consumes `self`.
+/// This means it is usually best to call [`coin_selection`] on the return value of `build_tx` before assigning it.
+///
+/// For further examples see [this module](super::tx_builder)'s documentation;
+///
+/// [`build_tx`]: Wallet::build_tx
+/// [`build_fee_bump`]: Wallet::build_fee_bump
+/// [`finish`]: Self::finish
+/// [`coin_selection`]: Self::coin_selection
+#[derive(Debug)]
+pub struct TxBuilder<'a, D, Cs, Ctx> {
+    pub(crate) wallet: Rc<RefCell<&'a mut Wallet<D>>>,
+    pub(crate) params: TxParams,
+    pub(crate) coin_selection: Cs,
+    pub(crate) phantom: PhantomData<Ctx>,
+}
+
+/// The parameters for transaction creation sans coin selection algorithm.
+//TODO: TxParams should eventually be exposed publicly.
+#[derive(Default, Debug, Clone)]
+pub(crate) struct TxParams {
+    pub(crate) recipients: Vec<(Script, u64)>,
+    pub(crate) drain_wallet: bool,
+    pub(crate) drain_to: Option<Script>,
+    pub(crate) fee_policy: Option<FeePolicy>,
+    pub(crate) internal_policy_path: Option<BTreeMap<String, Vec<usize>>>,
+    pub(crate) external_policy_path: Option<BTreeMap<String, Vec<usize>>>,
+    pub(crate) utxos: Vec<WeightedUtxo>,
+    pub(crate) unspendable: HashSet<OutPoint>,
+    pub(crate) manually_selected_only: bool,
+    pub(crate) sighash: Option<psbt::PsbtSighashType>,
+    pub(crate) ordering: TxOrdering,
+    pub(crate) locktime: Option<LockTime>,
+    pub(crate) rbf: Option<RbfValue>,
+    pub(crate) version: Option<Version>,
+    pub(crate) change_policy: ChangeSpendPolicy,
+    pub(crate) only_witness_utxo: bool,
+    pub(crate) add_global_xpubs: bool,
+    pub(crate) include_output_redeem_witness_script: bool,
+    pub(crate) bumping_fee: Option<PreviousFee>,
+    pub(crate) current_height: Option<LockTime>,
+    pub(crate) allow_dust: bool,
+}
+
+#[derive(Clone, Copy, Debug)]
+pub(crate) struct PreviousFee {
+    pub absolute: u64,
+    pub rate: f32,
+}
+
+#[derive(Debug, Clone, Copy)]
+pub(crate) enum FeePolicy {
+    FeeRate(FeeRate),
+    FeeAmount(u64),
+}
+
+impl Default for FeePolicy {
+    fn default() -> Self {
+        FeePolicy::FeeRate(FeeRate::default_min_relay_fee())
+    }
+}
+
+impl<'a, D, Cs: Clone, Ctx> Clone for TxBuilder<'a, D, Cs, Ctx> {
+    fn clone(&self) -> Self {
+        TxBuilder {
+            wallet: self.wallet.clone(),
+            params: self.params.clone(),
+            coin_selection: self.coin_selection.clone(),
+            phantom: PhantomData,
+        }
+    }
+}
+
+// methods supported by both contexts, for any CoinSelectionAlgorithm
+impl<'a, D, Cs: CoinSelectionAlgorithm, Ctx: TxBuilderContext> TxBuilder<'a, D, Cs, Ctx> {
+    /// Set a custom fee rate
+    pub fn fee_rate(&mut self, fee_rate: FeeRate) -> &mut Self {
+        self.params.fee_policy = Some(FeePolicy::FeeRate(fee_rate));
+        self
+    }
+
+    /// Set an absolute fee
+    /// The fee_absolute method refers to the absolute transaction fee in satoshis (sats).
+    /// If anyone sets both the fee_absolute method and the fee_rate method,
+    /// the fee_absolute value will take precedence over the fee_rate.
+    pub fn fee_absolute(&mut self, fee_amount: u64) -> &mut Self {
+        self.params.fee_policy = Some(FeePolicy::FeeAmount(fee_amount));
+        self
+    }
+
+    /// Set the policy path to use while creating the transaction for a given keychain.
+    ///
+    /// This method accepts a map where the key is the policy node id (see
+    /// [`Policy::id`](crate::descriptor::Policy::id)) and the value is the list of the indexes of
+    /// the items that are intended to be satisfied from the policy node (see
+    /// [`SatisfiableItem::Thresh::items`](crate::descriptor::policy::SatisfiableItem::Thresh::items)).
+    ///
+    /// ## Example
+    ///
+    /// An example of when the policy path is needed is the following descriptor:
+    /// `wsh(thresh(2,pk(A),sj:and_v(v:pk(B),n:older(6)),snj:and_v(v:pk(C),after(630000))))`,
+    /// derived from the miniscript policy `thresh(2,pk(A),and(pk(B),older(6)),and(pk(C),after(630000)))`.
+    /// It declares three descriptor fragments, and at the top level it uses `thresh()` to
+    /// ensure that at least two of them are satisfied. The individual fragments are:
+    ///
+    /// 1. `pk(A)`
+    /// 2. `and(pk(B),older(6))`
+    /// 3. `and(pk(C),after(630000))`
+    ///
+    /// When those conditions are combined in pairs, it's clear that the transaction needs to be created
+    /// differently depending on how the user intends to satisfy the policy afterwards:
+    ///
+    /// * If fragments `1` and `2` are used, the transaction will need to use a specific
+    ///   `n_sequence` in order to spend an `OP_CSV` branch.
+    /// * If fragments `1` and `3` are used, the transaction will need to use a specific `locktime`
+    ///   in order to spend an `OP_CLTV` branch.
+    /// * If fragments `2` and `3` are used, the transaction will need both.
+    ///
+    /// When the spending policy is represented as a tree (see
+    /// [`Wallet::policies`](super::Wallet::policies)), every node
+    /// is assigned a unique identifier that can be used in the policy path to specify which of
+    /// the node's children the user intends to satisfy: for instance, assuming the `thresh()`
+    /// root node of this example has an id of `aabbccdd`, the policy path map would look like:
+    ///
+    /// `{ "aabbccdd" => [0, 1] }`
+    ///
+    /// where the key is the node's id, and the value is a list of the children that should be
+    /// used, in no particular order.
+    ///
+    /// If a particularly complex descriptor has multiple ambiguous thresholds in its structure,
+    /// multiple entries can be added to the map, one for each node that requires an explicit path.
+    ///
+    /// ```
+    /// # use std::str::FromStr;
+    /// # use std::collections::BTreeMap;
+    /// # use bitcoin::*;
+    /// # use bdk::*;
+    /// # let to_address = Address::from_str("2N4eQYCbKUHCCTUjBJeHcJp9ok6J2GZsTDt").unwrap();
+    /// # let mut wallet = doctest_wallet!();
+    /// let mut path = BTreeMap::new();
+    /// path.insert("aabbccdd".to_string(), vec![0, 1]);
+    ///
+    /// let builder = wallet
+    ///     .build_tx()
+    ///     .add_recipient(to_address.script_pubkey(), 50_000)
+    ///     .policy_path(path, KeychainKind::External);
+    ///
+    /// # Ok::<(), bdk::Error>(())
+    /// ```
+    pub fn policy_path(
+        &mut self,
+        policy_path: BTreeMap<String, Vec<usize>>,
+        keychain: KeychainKind,
+    ) -> &mut Self {
+        let to_update = match keychain {
+            KeychainKind::Internal => &mut self.params.internal_policy_path,
+            KeychainKind::External => &mut self.params.external_policy_path,
+        };
+
+        *to_update = Some(policy_path);
+        self
+    }
+
+    /// Add the list of outpoints to the internal list of UTXOs that **must** be spent.
+    ///
+    /// If an error occurs while adding any of the UTXOs then none of them are added and the error is returned.
+    ///
+    /// These have priority over the "unspendable" utxos, meaning that if a utxo is present both in
+    /// the "utxos" and the "unspendable" list, it will be spent.
+    pub fn add_utxos(&mut self, outpoints: &[OutPoint]) -> Result<&mut Self, Error> {
+        {
+            let wallet = self.wallet.borrow();
+            let utxos = outpoints
+                .iter()
+                .map(|outpoint| wallet.get_utxo(*outpoint).ok_or(Error::UnknownUtxo))
+                .collect::<Result<Vec<_>, _>>()?;
+
+            for utxo in utxos {
+                let descriptor = wallet.get_descriptor_for_keychain(utxo.keychain);
+                let satisfaction_weight = descriptor.max_satisfaction_weight().unwrap();
+                self.params.utxos.push(WeightedUtxo {
+                    satisfaction_weight,
+                    utxo: Utxo::Local(utxo),
+                });
+            }
+        }
+
+        Ok(self)
+    }
+
+    /// Add a utxo to the internal list of utxos that **must** be spent
+    ///
+    /// These have priority over the "unspendable" utxos, meaning that if a utxo is present both in
+    /// the "utxos" and the "unspendable" list, it will be spent.
+    pub fn add_utxo(&mut self, outpoint: OutPoint) -> Result<&mut Self, Error> {
+        self.add_utxos(&[outpoint])
+    }
+
+    /// Add a foreign UTXO i.e. a UTXO not owned by this wallet.
+    ///
+    /// At a minimum to add a foreign UTXO we need:
+    ///
+    /// 1. `outpoint`: To add it to the raw transaction.
+    /// 2. `psbt_input`: To know the value.
+    /// 3. `satisfaction_weight`: To know how much weight/vbytes the input will add to the transaction for fee calculation.
+    ///
+    /// There are several security concerns about adding foreign UTXOs that application
+    /// developers should consider. First, how do you know the value of the input is correct? If a
+    /// `non_witness_utxo` is provided in the `psbt_input` then this method implicitly verifies the
+    /// value by checking it against the transaction. If only a `witness_utxo` is provided then this
+    /// method doesn't verify the value but just takes it as a given -- it is up to you to check
+    /// that whoever sent you the `input_psbt` was not lying!
+    ///
+    /// Secondly, you must somehow provide `satisfaction_weight` of the input. Depending on your
+    /// application it may be important that this be known precisely. If not, a malicious
+    /// counterparty may fool you into putting in a value that is too low, giving the transaction a
+    /// lower than expected feerate. They could also fool you into putting a value that is too high
+    /// causing you to pay a fee that is too high. The party who is broadcasting the transaction can
+    /// of course check the real input weight matches the expected weight prior to broadcasting.
+    ///
+    /// To guarantee the `satisfaction_weight` is correct, you can require the party providing the
+    /// `psbt_input` provide a miniscript descriptor for the input so you can check it against the
+    /// `script_pubkey` and then ask it for the [`max_satisfaction_weight`].
+    ///
+    /// This is an **EXPERIMENTAL** feature, API and other major changes are expected.
+    ///
+    /// # Errors
+    ///
+    /// This method returns errors in the following circumstances:
+    ///
+    /// 1. The `psbt_input` does not contain a `witness_utxo` or `non_witness_utxo`.
+    /// 2. The data in `non_witness_utxo` does not match what is in `outpoint`.
+    ///
+    /// Note unless you set [`only_witness_utxo`] any non-taproot `psbt_input` you pass to this
+    /// method must have `non_witness_utxo` set otherwise you will get an error when [`finish`]
+    /// is called.
+    ///
+    /// [`only_witness_utxo`]: Self::only_witness_utxo
+    /// [`finish`]: Self::finish
+    /// [`max_satisfaction_weight`]: miniscript::Descriptor::max_satisfaction_weight
+    pub fn add_foreign_utxo(
+        &mut self,
+        outpoint: OutPoint,
+        psbt_input: psbt::Input,
+        satisfaction_weight: usize,
+    ) -> Result<&mut Self, Error> {
+        if psbt_input.witness_utxo.is_none() {
+            match psbt_input.non_witness_utxo.as_ref() {
+                Some(tx) => {
+                    if tx.txid() != outpoint.txid {
+                        return Err(Error::Generic(
+                            "Foreign utxo outpoint does not match PSBT input".into(),
+                        ));
+                    }
+                    if tx.output.len() <= outpoint.vout as usize {
+                        return Err(Error::InvalidOutpoint(outpoint));
+                    }
+                }
+                None => {
+                    return Err(Error::Generic(
+                        "Foreign utxo missing witness_utxo or non_witness_utxo".into(),
+                    ))
+                }
+            }
+        }
+
+        self.params.utxos.push(WeightedUtxo {
+            satisfaction_weight,
+            utxo: Utxo::Foreign {
+                outpoint,
+                psbt_input: Box::new(psbt_input),
+            },
+        });
+
+        Ok(self)
+    }
+
+    /// Only spend utxos added by [`add_utxo`].
+    ///
+    /// The wallet will **not** add additional utxos to the transaction even if they are needed to
+    /// make the transaction valid.
+    ///
+    /// [`add_utxo`]: Self::add_utxo
+    pub fn manually_selected_only(&mut self) -> &mut Self {
+        self.params.manually_selected_only = true;
+        self
+    }
+
+    /// Replace the internal list of unspendable utxos with a new list
+    ///
+    /// It's important to note that the "must-be-spent" utxos added with [`TxBuilder::add_utxo`]
+    /// have priority over these. See the docs of the two linked methods for more details.
+    pub fn unspendable(&mut self, unspendable: Vec<OutPoint>) -> &mut Self {
+        self.params.unspendable = unspendable.into_iter().collect();
+        self
+    }
+
+    /// Add a utxo to the internal list of unspendable utxos
+    ///
+    /// It's important to note that the "must-be-spent" utxos added with [`TxBuilder::add_utxo`]
+    /// have priority over this. See the docs of the two linked methods for more details.
+    pub fn add_unspendable(&mut self, unspendable: OutPoint) -> &mut Self {
+        self.params.unspendable.insert(unspendable);
+        self
+    }
+
+    /// Sign with a specific sig hash
+    ///
+    /// **Use this option very carefully**
+    pub fn sighash(&mut self, sighash: psbt::PsbtSighashType) -> &mut Self {
+        self.params.sighash = Some(sighash);
+        self
+    }
+
+    /// Choose the ordering for inputs and outputs of the transaction
+    pub fn ordering(&mut self, ordering: TxOrdering) -> &mut Self {
+        self.params.ordering = ordering;
+        self
+    }
+
+    /// Use a specific nLockTime while creating the transaction
+    ///
+    /// This can cause conflicts if the wallet's descriptors contain an "after" (OP_CLTV) operator.
+    pub fn nlocktime(&mut self, locktime: LockTime) -> &mut Self {
+        self.params.locktime = Some(locktime);
+        self
+    }
+
+    /// Build a transaction with a specific version
+    ///
+    /// The `version` should always be greater than `0` and greater than `1` if the wallet's
+    /// descriptors contain an "older" (OP_CSV) operator.
+    pub fn version(&mut self, version: i32) -> &mut Self {
+        self.params.version = Some(Version(version));
+        self
+    }
+
+    /// Do not spend change outputs
+    ///
+    /// This effectively adds all the change outputs to the "unspendable" list. See
+    /// [`TxBuilder::unspendable`].
+    pub fn do_not_spend_change(&mut self) -> &mut Self {
+        self.params.change_policy = ChangeSpendPolicy::ChangeForbidden;
+        self
+    }
+
+    /// Only spend change outputs
+    ///
+    /// This effectively adds all the non-change outputs to the "unspendable" list. See
+    /// [`TxBuilder::unspendable`].
+    pub fn only_spend_change(&mut self) -> &mut Self {
+        self.params.change_policy = ChangeSpendPolicy::OnlyChange;
+        self
+    }
+
+    /// Set a specific [`ChangeSpendPolicy`]. See [`TxBuilder::do_not_spend_change`] and
+    /// [`TxBuilder::only_spend_change`] for some shortcuts.
+    pub fn change_policy(&mut self, change_policy: ChangeSpendPolicy) -> &mut Self {
+        self.params.change_policy = change_policy;
+        self
+    }
+
+    /// Only Fill-in the [`psbt::Input::witness_utxo`](bitcoin::util::psbt::Input::witness_utxo) field when spending from
+    /// SegWit descriptors.
+    ///
+    /// This reduces the size of the PSBT, but some signers might reject them due to the lack of
+    /// the `non_witness_utxo`.
+    pub fn only_witness_utxo(&mut self) -> &mut Self {
+        self.params.only_witness_utxo = true;
+        self
+    }
+
+    /// Fill-in the [`psbt::Output::redeem_script`](bitcoin::util::psbt::Output::redeem_script) and
+    /// [`psbt::Output::witness_script`](bitcoin::util::psbt::Output::witness_script) fields.
+    ///
+    /// This is useful for signers which always require it, like ColdCard hardware wallets.
+    pub fn include_output_redeem_witness_script(&mut self) -> &mut Self {
+        self.params.include_output_redeem_witness_script = true;
+        self
+    }
+
+    /// Fill-in the `PSBT_GLOBAL_XPUB` field with the extended keys contained in both the external
+    /// and internal descriptors
+    ///
+    /// This is useful for offline signers that take part to a multisig. Some hardware wallets like
+    /// BitBox and ColdCard are known to require this.
+    pub fn add_global_xpubs(&mut self) -> &mut Self {
+        self.params.add_global_xpubs = true;
+        self
+    }
+
+    /// Spend all the available inputs. This respects filters like [`TxBuilder::unspendable`] and the change policy.
+    pub fn drain_wallet(&mut self) -> &mut Self {
+        self.params.drain_wallet = true;
+        self
+    }
+
+    /// Choose the coin selection algorithm
+    ///
+    /// Overrides the [`DefaultCoinSelectionAlgorithm`](super::coin_selection::DefaultCoinSelectionAlgorithm).
+    ///
+    /// Note that this function consumes the builder and returns it so it is usually best to put this as the first call on the builder.
+    pub fn coin_selection<P: CoinSelectionAlgorithm>(
+        self,
+        coin_selection: P,
+    ) -> TxBuilder<'a, D, P, Ctx> {
+        TxBuilder {
+            wallet: self.wallet,
+            params: self.params,
+            coin_selection,
+            phantom: PhantomData,
+        }
+    }
+
+    /// Finish building the transaction.
+    ///
+    /// Returns the [`BIP174`] "PSBT" and summary details about the transaction.
+    ///
+    /// [`BIP174`]: https://github.com/bitcoin/bips/blob/master/bip-0174.mediawiki
+    pub fn finish(self) -> Result<(Psbt, TransactionDetails), Error>
+    where
+        D: PersistBackend<ChangeSet>,
+    {
+        self.wallet
+            .borrow_mut()
+            .create_tx(self.coin_selection, self.params)
+    }
+
+    /// Enable signaling RBF
+    ///
+    /// This will use the default nSequence value of `0xFFFFFFFD`.
+    pub fn enable_rbf(&mut self) -> &mut Self {
+        self.params.rbf = Some(RbfValue::Default);
+        self
+    }
+
+    /// Enable signaling RBF with a specific nSequence value
+    ///
+    /// This can cause conflicts if the wallet's descriptors contain an "older" (OP_CSV) operator
+    /// and the given `nsequence` is lower than the CSV value.
+    ///
+    /// If the `nsequence` is higher than `0xFFFFFFFD` an error will be thrown, since it would not
+    /// be a valid nSequence to signal RBF.
+    pub fn enable_rbf_with_sequence(&mut self, nsequence: Sequence) -> &mut Self {
+        self.params.rbf = Some(RbfValue::Value(nsequence));
+        self
+    }
+
+    /// Set the current blockchain height.
+    ///
+    /// This will be used to:
+    /// 1. Set the nLockTime for preventing fee sniping.
+    /// **Note**: This will be ignored if you manually specify a nlocktime using [`TxBuilder::nlocktime`].
+    /// 2. Decide whether coinbase outputs are mature or not. If the coinbase outputs are not
+    ///    mature at `current_height`, we ignore them in the coin selection.
+    ///    If you want to create a transaction that spends immature coinbase inputs, manually
+    ///    add them using [`TxBuilder::add_utxos`].
+    ///
+    /// In both cases, if you don't provide a current height, we use the last sync height.
+    pub fn current_height(&mut self, height: u32) -> &mut Self {
+        self.params.current_height = Some(LockTime::from_height(height).expect("Invalid height"));
+        self
+    }
+
+    /// Set whether or not the dust limit is checked.
+    ///
+    /// **Note**: by avoiding a dust limit check you may end up with a transaction that is non-standard.
+    pub fn allow_dust(&mut self, allow_dust: bool) -> &mut Self {
+        self.params.allow_dust = allow_dust;
+        self
+    }
+}
+
+impl<'a, D, Cs: CoinSelectionAlgorithm> TxBuilder<'a, D, Cs, CreateTx> {
+    /// Replace the recipients already added with a new list
+    pub fn set_recipients(&mut self, recipients: Vec<(Script, u64)>) -> &mut Self {
+        self.params.recipients = recipients;
+        self
+    }
+
+    /// Add a recipient to the internal list
+    pub fn add_recipient(&mut self, script_pubkey: Script, amount: u64) -> &mut Self {
+        self.params.recipients.push((script_pubkey, amount));
+        self
+    }
+
+    /// Add data as an output, using OP_RETURN
+    pub fn add_data(&mut self, data: &[u8]) -> &mut Self {
+        let script = Script::new_op_return(data);
+        self.add_recipient(script, 0u64);
+        self
+    }
+
+    /// Sets the address to *drain* excess coins to.
+    ///
+    /// Usually, when there are excess coins they are sent to a change address generated by the
+    /// wallet. This option replaces the usual change address with an arbitrary `script_pubkey` of
+    /// your choosing. Just as with a change output, if the drain output is not needed (the excess
+    /// coins are too small) it will not be included in the resulting transaction. The only
+    /// difference is that it is valid to use `drain_to` without setting any ordinary recipients
+    /// with [`add_recipient`] (but it is perfectly fine to add recipients as well).
+    ///
+    /// If you choose not to set any recipients, you should either provide the utxos that the
+    /// transaction should spend via [`add_utxos`], or set [`drain_wallet`] to spend all of them.
+    ///
+    /// When bumping the fees of a transaction made with this option, you probably want to
+    /// use [`allow_shrinking`] to allow this output to be reduced to pay for the extra fees.
+    ///
+    /// # Example
+    ///
+    /// `drain_to` is very useful for draining all the coins in a wallet with [`drain_wallet`] to a
+    /// single address.
+    ///
+    /// ```
+    /// # use std::str::FromStr;
+    /// # use bitcoin::*;
+    /// # use bdk::*;
+    /// # use bdk::wallet::tx_builder::CreateTx;
+    /// # let to_address = Address::from_str("2N4eQYCbKUHCCTUjBJeHcJp9ok6J2GZsTDt").unwrap();
+    /// # let mut wallet = doctest_wallet!();
+    /// let mut tx_builder = wallet.build_tx();
+    ///
+    /// tx_builder
+    ///     // Spend all outputs in this wallet.
+    ///     .drain_wallet()
+    ///     // Send the excess (which is all the coins minus the fee) to this address.
+    ///     .drain_to(to_address.script_pubkey())
+    ///     .fee_rate(FeeRate::from_sat_per_vb(5.0))
+    ///     .enable_rbf();
+    /// let (psbt, tx_details) = tx_builder.finish()?;
+    /// # Ok::<(), bdk::Error>(())
+    /// ```
+    ///
+    /// [`allow_shrinking`]: Self::allow_shrinking
+    /// [`add_recipient`]: Self::add_recipient
+    /// [`add_utxos`]: Self::add_utxos
+    /// [`drain_wallet`]: Self::drain_wallet
+    pub fn drain_to(&mut self, script_pubkey: Script) -> &mut Self {
+        self.params.drain_to = Some(script_pubkey);
+        self
+    }
+}
+
+// methods supported only by bump_fee
+impl<'a, D> TxBuilder<'a, D, DefaultCoinSelectionAlgorithm, BumpFee> {
+    /// Explicitly tells the wallet that it is allowed to reduce the amount of the output matching this
+    /// `script_pubkey` in order to bump the transaction fee. Without specifying this the wallet
+    /// will attempt to find a change output to shrink instead.
+    ///
+    /// **Note** that the output may shrink to below the dust limit and therefore be removed. If it is
+    /// preserved then it is currently not guaranteed to be in the same position as it was
+    /// originally.
+    ///
+    /// Returns an `Err` if `script_pubkey` can't be found among the recipients of the
+    /// transaction we are bumping.
+    pub fn allow_shrinking(&mut self, script_pubkey: Script) -> Result<&mut Self, Error> {
+        match self
+            .params
+            .recipients
+            .iter()
+            .position(|(recipient_script, _)| *recipient_script == script_pubkey)
+        {
+            Some(position) => {
+                self.params.recipients.remove(position);
+                self.params.drain_to = Some(script_pubkey);
+                Ok(self)
+            }
+            None => Err(Error::Generic(format!(
+                "{} was not in the original transaction",
+                script_pubkey
+            ))),
+        }
+    }
+}
+
+/// Ordering of the transaction's inputs and outputs
+#[derive(Debug, Ord, PartialOrd, Eq, PartialEq, Hash, Clone, Copy)]
+pub enum TxOrdering {
+    /// Randomized (default)
+    Shuffle,
+    /// Unchanged
+    Untouched,
+    /// BIP69 / Lexicographic
+    Bip69Lexicographic,
+}
+
+impl Default for TxOrdering {
+    fn default() -> Self {
+        TxOrdering::Shuffle
+    }
+}
+
+impl TxOrdering {
+    /// Sort transaction inputs and outputs by [`TxOrdering`] variant
+    pub fn sort_tx(&self, tx: &mut Transaction) {
+        match self {
+            TxOrdering::Untouched => {}
+            TxOrdering::Shuffle => {
+                use rand::seq::SliceRandom;
+                let mut rng = rand::thread_rng();
+                tx.input.shuffle(&mut rng);
+                tx.output.shuffle(&mut rng);
+            }
+            TxOrdering::Bip69Lexicographic => {
+                tx.input.sort_unstable_by_key(|txin| {
+                    (txin.previous_output.txid, txin.previous_output.vout)
+                });
+                tx.output
+                    .sort_unstable_by_key(|txout| (txout.value, txout.script_pubkey.clone()));
+            }
+        }
+    }
+}
+
+/// Transaction version
+///
+/// Has a default value of `1`
+#[derive(Debug, Ord, PartialOrd, Eq, PartialEq, Hash, Clone, Copy)]
+pub(crate) struct Version(pub(crate) i32);
+
+impl Default for Version {
+    fn default() -> Self {
+        Version(1)
+    }
+}
+
+/// RBF nSequence value
+///
+/// Has a default value of `0xFFFFFFFD`
+#[derive(Debug, Ord, PartialOrd, Eq, PartialEq, Hash, Clone, Copy)]
+pub(crate) enum RbfValue {
+    Default,
+    Value(Sequence),
+}
+
+impl RbfValue {
+    pub(crate) fn get_value(&self) -> Sequence {
+        match self {
+            RbfValue::Default => Sequence::ENABLE_RBF_NO_LOCKTIME,
+            RbfValue::Value(v) => *v,
+        }
+    }
+}
+
+/// Policy regarding the use of change outputs when creating a transaction
+#[derive(Debug, Ord, PartialOrd, Eq, PartialEq, Hash, Clone, Copy)]
+pub enum ChangeSpendPolicy {
+    /// Use both change and non-change outputs (default)
+    ChangeAllowed,
+    /// Only use change outputs (see [`TxBuilder::only_spend_change`])
+    OnlyChange,
+    /// Only use non-change outputs (see [`TxBuilder::do_not_spend_change`])
+    ChangeForbidden,
+}
+
+impl Default for ChangeSpendPolicy {
+    fn default() -> Self {
+        ChangeSpendPolicy::ChangeAllowed
+    }
+}
+
+impl ChangeSpendPolicy {
+    pub(crate) fn is_satisfied_by(&self, utxo: &LocalUtxo) -> bool {
+        match self {
+            ChangeSpendPolicy::ChangeAllowed => true,
+            ChangeSpendPolicy::OnlyChange => utxo.keychain == KeychainKind::Internal,
+            ChangeSpendPolicy::ChangeForbidden => utxo.keychain == KeychainKind::External,
+        }
+    }
+}
+
+#[cfg(test)]
+mod test {
+    const ORDERING_TEST_TX: &str = "0200000003c26f3eb7932f7acddc5ddd26602b77e7516079b03090a16e2c2f54\
+                                    85d1fd600f0100000000ffffffffc26f3eb7932f7acddc5ddd26602b77e75160\
+                                    79b03090a16e2c2f5485d1fd600f0000000000ffffffff571fb3e02278217852\
+                                    dd5d299947e2b7354a639adc32ec1fa7b82cfb5dec530e0500000000ffffffff\
+                                    03e80300000000000002aaeee80300000000000001aa200300000000000001ff\
+                                    00000000";
+    macro_rules! ordering_test_tx {
+        () => {
+            deserialize::<bitcoin::Transaction>(&Vec::<u8>::from_hex(ORDERING_TEST_TX).unwrap())
+                .unwrap()
+        };
+    }
+
+    use bdk_chain::ConfirmationTime;
+    use bitcoin::consensus::deserialize;
+    use bitcoin::hashes::hex::FromHex;
+
+    use super::*;
+
+    #[test]
+    fn test_output_ordering_default_shuffle() {
+        assert_eq!(TxOrdering::default(), TxOrdering::Shuffle);
+    }
+
+    #[test]
+    fn test_output_ordering_untouched() {
+        let original_tx = ordering_test_tx!();
+        let mut tx = original_tx.clone();
+
+        TxOrdering::Untouched.sort_tx(&mut tx);
+
+        assert_eq!(original_tx, tx);
+    }
+
+    #[test]
+    fn test_output_ordering_shuffle() {
+        let original_tx = ordering_test_tx!();
+        let mut tx = original_tx.clone();
+
+        (0..40)
+            .find(|_| {
+                TxOrdering::Shuffle.sort_tx(&mut tx);
+                original_tx.input != tx.input
+            })
+            .expect("it should have moved the inputs at least once");
+
+        let mut tx = original_tx.clone();
+        (0..40)
+            .find(|_| {
+                TxOrdering::Shuffle.sort_tx(&mut tx);
+                original_tx.output != tx.output
+            })
+            .expect("it should have moved the outputs at least once");
+    }
+
+    #[test]
+    fn test_output_ordering_bip69() {
+        use core::str::FromStr;
+
+        let original_tx = ordering_test_tx!();
+        let mut tx = original_tx;
+
+        TxOrdering::Bip69Lexicographic.sort_tx(&mut tx);
+
+        assert_eq!(
+            tx.input[0].previous_output,
+            bitcoin::OutPoint::from_str(
+                "0e53ec5dfb2cb8a71fec32dc9a634a35b7e24799295ddd5278217822e0b31f57:5"
+            )
+            .unwrap()
+        );
+        assert_eq!(
+            tx.input[1].previous_output,
+            bitcoin::OutPoint::from_str(
+                "0f60fdd185542f2c6ea19030b0796051e7772b6026dd5ddccd7a2f93b73e6fc2:0"
+            )
+            .unwrap()
+        );
+        assert_eq!(
+            tx.input[2].previous_output,
+            bitcoin::OutPoint::from_str(
+                "0f60fdd185542f2c6ea19030b0796051e7772b6026dd5ddccd7a2f93b73e6fc2:1"
+            )
+            .unwrap()
+        );
+
+        assert_eq!(tx.output[0].value, 800);
+        assert_eq!(tx.output[1].script_pubkey, From::from(vec![0xAA]));
+        assert_eq!(tx.output[2].script_pubkey, From::from(vec![0xAA, 0xEE]));
+    }
+
+    fn get_test_utxos() -> Vec<LocalUtxo> {
+        use bitcoin::hashes::Hash;
+
+        vec![
+            LocalUtxo {
+                outpoint: OutPoint {
+                    txid: bitcoin::Txid::from_inner([0; 32]),
+                    vout: 0,
+                },
+                txout: Default::default(),
+                keychain: KeychainKind::External,
+                is_spent: false,
+                confirmation_time: ConfirmationTime::Unconfirmed { last_seen: 0 },
+                derivation_index: 0,
+            },
+            LocalUtxo {
+                outpoint: OutPoint {
+                    txid: bitcoin::Txid::from_inner([0; 32]),
+                    vout: 1,
+                },
+                txout: Default::default(),
+                keychain: KeychainKind::Internal,
+                is_spent: false,
+                confirmation_time: ConfirmationTime::Confirmed {
+                    height: 32,
+                    time: 42,
+                },
+                derivation_index: 1,
+            },
+        ]
+    }
+
+    #[test]
+    fn test_change_spend_policy_default() {
+        let change_spend_policy = ChangeSpendPolicy::default();
+        let filtered = get_test_utxos()
+            .into_iter()
+            .filter(|u| change_spend_policy.is_satisfied_by(u))
+            .count();
+
+        assert_eq!(filtered, 2);
+    }
+
+    #[test]
+    fn test_change_spend_policy_no_internal() {
+        let change_spend_policy = ChangeSpendPolicy::ChangeForbidden;
+        let filtered = get_test_utxos()
+            .into_iter()
+            .filter(|u| change_spend_policy.is_satisfied_by(u))
+            .collect::<Vec<_>>();
+
+        assert_eq!(filtered.len(), 1);
+        assert_eq!(filtered[0].keychain, KeychainKind::External);
+    }
+
+    #[test]
+    fn test_change_spend_policy_only_internal() {
+        let change_spend_policy = ChangeSpendPolicy::OnlyChange;
+        let filtered = get_test_utxos()
+            .into_iter()
+            .filter(|u| change_spend_policy.is_satisfied_by(u))
+            .collect::<Vec<_>>();
+
+        assert_eq!(filtered.len(), 1);
+        assert_eq!(filtered[0].keychain, KeychainKind::Internal);
+    }
+
+    #[test]
+    fn test_default_tx_version_1() {
+        let version = Version::default();
+        assert_eq!(version.0, 1);
+    }
+}

crates/bdk/src/wallet/utils.rs

@@ -0,0 +1,181 @@
+// Bitcoin Dev Kit
+// Written in 2020 by Alekos Filini <alekos.filini@gmail.com>
+//
+// Copyright (c) 2020-2021 Bitcoin Dev Kit Developers
+//
+// This file is licensed under the Apache License, Version 2.0 <LICENSE-APACHE
+// or http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option.
+// You may not use this file except in accordance with one or both of these
+// licenses.
+
+use bitcoin::secp256k1::{All, Secp256k1};
+use bitcoin::{LockTime, Script, Sequence};
+
+use miniscript::{MiniscriptKey, Satisfier, ToPublicKey};
+
+/// Trait to check if a value is below the dust limit.
+/// We are performing dust value calculation for a given script public key using rust-bitcoin to
+/// keep it compatible with network dust rate
+// we implement this trait to make sure we don't mess up the comparison with off-by-one like a <
+// instead of a <= etc.
+pub trait IsDust {
+    /// Check whether or not a value is below dust limit
+    fn is_dust(&self, script: &Script) -> bool;
+}
+
+impl IsDust for u64 {
+    fn is_dust(&self, script: &Script) -> bool {
+        *self < script.dust_value().to_sat()
+    }
+}
+
+pub struct After {
+    pub current_height: Option<u32>,
+    pub assume_height_reached: bool,
+}
+
+impl After {
+    pub(crate) fn new(current_height: Option<u32>, assume_height_reached: bool) -> After {
+        After {
+            current_height,
+            assume_height_reached,
+        }
+    }
+}
+
+pub(crate) fn check_nsequence_rbf(rbf: Sequence, csv: Sequence) -> bool {
+    // The RBF value must enable relative timelocks
+    if !rbf.is_relative_lock_time() {
+        return false;
+    }
+
+    // Both values should be represented in the same unit (either time-based or
+    // block-height based)
+    if rbf.is_time_locked() != csv.is_time_locked() {
+        return false;
+    }
+
+    // The value should be at least `csv`
+    if rbf < csv {
+        return false;
+    }
+
+    true
+}
+
+impl<Pk: MiniscriptKey + ToPublicKey> Satisfier<Pk> for After {
+    fn check_after(&self, n: LockTime) -> bool {
+        if let Some(current_height) = self.current_height {
+            current_height >= n.to_consensus_u32()
+        } else {
+            self.assume_height_reached
+        }
+    }
+}
+
+pub struct Older {
+    pub current_height: Option<u32>,
+    pub create_height: Option<u32>,
+    pub assume_height_reached: bool,
+}
+
+impl Older {
+    pub(crate) fn new(
+        current_height: Option<u32>,
+        create_height: Option<u32>,
+        assume_height_reached: bool,
+    ) -> Older {
+        Older {
+            current_height,
+            create_height,
+            assume_height_reached,
+        }
+    }
+}
+
+impl<Pk: MiniscriptKey + ToPublicKey> Satisfier<Pk> for Older {
+    fn check_older(&self, n: Sequence) -> bool {
+        if let Some(current_height) = self.current_height {
+            // TODO: test >= / >
+            current_height
+                >= self
+                    .create_height
+                    .unwrap_or(0)
+                    .checked_add(n.to_consensus_u32())
+                    .expect("Overflowing addition")
+        } else {
+            self.assume_height_reached
+        }
+    }
+}
+
+pub(crate) type SecpCtx = Secp256k1<All>;
+
+#[cfg(test)]
+mod test {
+    // When nSequence is lower than this flag the timelock is interpreted as block-height-based,
+    // otherwise it's time-based
+    pub(crate) const SEQUENCE_LOCKTIME_TYPE_FLAG: u32 = 1 << 22;
+
+    use super::{check_nsequence_rbf, IsDust};
+    use crate::bitcoin::{Address, Sequence};
+    use core::str::FromStr;
+
+    #[test]
+    fn test_is_dust() {
+        let script_p2pkh = Address::from_str("1GNgwA8JfG7Kc8akJ8opdNWJUihqUztfPe")
+            .unwrap()
+            .script_pubkey();
+        assert!(script_p2pkh.is_p2pkh());
+        assert!(545.is_dust(&script_p2pkh));
+        assert!(!546.is_dust(&script_p2pkh));
+
+        let script_p2wpkh = Address::from_str("bc1qxlh2mnc0yqwas76gqq665qkggee5m98t8yskd8")
+            .unwrap()
+            .script_pubkey();
+        assert!(script_p2wpkh.is_v0_p2wpkh());
+        assert!(293.is_dust(&script_p2wpkh));
+        assert!(!294.is_dust(&script_p2wpkh));
+    }
+
+    #[test]
+    fn test_check_nsequence_rbf_msb_set() {
+        let result = check_nsequence_rbf(Sequence(0x80000000), Sequence(5000));
+        assert!(!result);
+    }
+
+    #[test]
+    fn test_check_nsequence_rbf_lt_csv() {
+        let result = check_nsequence_rbf(Sequence(4000), Sequence(5000));
+        assert!(!result);
+    }
+
+    #[test]
+    fn test_check_nsequence_rbf_different_unit() {
+        let result =
+            check_nsequence_rbf(Sequence(SEQUENCE_LOCKTIME_TYPE_FLAG + 5000), Sequence(5000));
+        assert!(!result);
+    }
+
+    #[test]
+    fn test_check_nsequence_rbf_mask() {
+        let result = check_nsequence_rbf(Sequence(0x3f + 10_000), Sequence(5000));
+        assert!(result);
+    }
+
+    #[test]
+    fn test_check_nsequence_rbf_same_unit_blocks() {
+        let result = check_nsequence_rbf(Sequence(10_000), Sequence(5000));
+        assert!(result);
+    }
+
+    #[test]
+    fn test_check_nsequence_rbf_same_unit_time() {
+        let result = check_nsequence_rbf(
+            Sequence(SEQUENCE_LOCKTIME_TYPE_FLAG + 10_000),
+            Sequence(SEQUENCE_LOCKTIME_TYPE_FLAG + 5000),
+        );
+        assert!(result);
+    }
+}

crates/bdk/tests/common.rs

@@ -0,0 +1,93 @@
+#![allow(unused)]
+use bdk::{wallet::AddressIndex, Wallet};
+use bdk_chain::{BlockId, ConfirmationTime};
+use bitcoin::hashes::Hash;
+use bitcoin::{BlockHash, Network, Transaction, TxOut};
+
+/// Return a fake wallet that appears to be funded for testing.
+pub fn get_funded_wallet_with_change(
+    descriptor: &str,
+    change: Option<&str>,
+) -> (Wallet, bitcoin::Txid) {
+    let mut wallet = Wallet::new_no_persist(descriptor, change, Network::Regtest).unwrap();
+    let address = wallet.get_address(AddressIndex::New).address;
+
+    let tx = Transaction {
+        version: 1,
+        lock_time: bitcoin::PackedLockTime(0),
+        input: vec![],
+        output: vec![TxOut {
+            value: 50_000,
+            script_pubkey: address.script_pubkey(),
+        }],
+    };
+
+    wallet
+        .insert_checkpoint(BlockId {
+            height: 1_000,
+            hash: BlockHash::all_zeros(),
+        })
+        .unwrap();
+    wallet
+        .insert_tx(
+            tx.clone(),
+            ConfirmationTime::Confirmed {
+                height: 1_000,
+                time: 100,
+            },
+        )
+        .unwrap();
+
+    (wallet, tx.txid())
+}
+
+pub fn get_funded_wallet(descriptor: &str) -> (Wallet, bitcoin::Txid) {
+    get_funded_wallet_with_change(descriptor, None)
+}
+
+pub fn get_test_wpkh() -> &'static str {
+    "wpkh(cVpPVruEDdmutPzisEsYvtST1usBR3ntr8pXSyt6D2YYqXRyPcFW)"
+}
+
+pub fn get_test_single_sig_csv() -> &'static str {
+    // and(pk(Alice),older(6))
+    "wsh(and_v(v:pk(cVpPVruEDdmutPzisEsYvtST1usBR3ntr8pXSyt6D2YYqXRyPcFW),older(6)))"
+}
+
+pub fn get_test_a_or_b_plus_csv() -> &'static str {
+    // or(pk(Alice),and(pk(Bob),older(144)))
+    "wsh(or_d(pk(cRjo6jqfVNP33HhSS76UhXETZsGTZYx8FMFvR9kpbtCSV1PmdZdu),and_v(v:pk(cMnkdebixpXMPfkcNEjjGin7s94hiehAH4mLbYkZoh9KSiNNmqC8),older(144))))"
+}
+
+pub fn get_test_single_sig_cltv() -> &'static str {
+    // and(pk(Alice),after(100000))
+    "wsh(and_v(v:pk(cVpPVruEDdmutPzisEsYvtST1usBR3ntr8pXSyt6D2YYqXRyPcFW),after(100000)))"
+}
+
+pub fn get_test_tr_single_sig() -> &'static str {
+    "tr(cNJmN3fH9DDbDt131fQNkVakkpzawJBSeybCUNmP1BovpmGQ45xG)"
+}
+
+pub fn get_test_tr_with_taptree() -> &'static str {
+    "tr(b511bd5771e47ee27558b1765e87b541668304ec567721c7b880edc0a010da55,{pk(cPZzKuNmpuUjD1e8jUU4PVzy2b5LngbSip8mBsxf4e7rSFZVb4Uh),pk(8aee2b8120a5f157f1223f72b5e62b825831a27a9fdf427db7cc697494d4a642)})"
+}
+
+pub fn get_test_tr_with_taptree_both_priv() -> &'static str {
+    "tr(b511bd5771e47ee27558b1765e87b541668304ec567721c7b880edc0a010da55,{pk(cPZzKuNmpuUjD1e8jUU4PVzy2b5LngbSip8mBsxf4e7rSFZVb4Uh),pk(cNaQCDwmmh4dS9LzCgVtyy1e1xjCJ21GUDHe9K98nzb689JvinGV)})"
+}
+
+pub fn get_test_tr_repeated_key() -> &'static str {
+    "tr(b511bd5771e47ee27558b1765e87b541668304ec567721c7b880edc0a010da55,{and_v(v:pk(cVpPVruEDdmutPzisEsYvtST1usBR3ntr8pXSyt6D2YYqXRyPcFW),after(100)),and_v(v:pk(cVpPVruEDdmutPzisEsYvtST1usBR3ntr8pXSyt6D2YYqXRyPcFW),after(200))})"
+}
+
+pub fn get_test_tr_single_sig_xprv() -> &'static str {
+    "tr(tprv8ZgxMBicQKsPdDArR4xSAECuVxeX1jwwSXR4ApKbkYgZiziDc4LdBy2WvJeGDfUSE4UT4hHhbgEwbdq8ajjUHiKDegkwrNU6V55CxcxonVN/*)"
+}
+
+pub fn get_test_tr_with_taptree_xprv() -> &'static str {
+    "tr(cNJmN3fH9DDbDt131fQNkVakkpzawJBSeybCUNmP1BovpmGQ45xG,{pk(tprv8ZgxMBicQKsPdDArR4xSAECuVxeX1jwwSXR4ApKbkYgZiziDc4LdBy2WvJeGDfUSE4UT4hHhbgEwbdq8ajjUHiKDegkwrNU6V55CxcxonVN/*),pk(8aee2b8120a5f157f1223f72b5e62b825831a27a9fdf427db7cc697494d4a642)})"
+}
+
+pub fn get_test_tr_dup_keys() -> &'static str {
+    "tr(cNJmN3fH9DDbDt131fQNkVakkpzawJBSeybCUNmP1BovpmGQ45xG,{pk(8aee2b8120a5f157f1223f72b5e62b825831a27a9fdf427db7cc697494d4a642),pk(8aee2b8120a5f157f1223f72b5e62b825831a27a9fdf427db7cc697494d4a642)})"
+}

crates/bdk/tests/psbt.rs

@@ -0,0 +1,158 @@
+use bdk::bitcoin::TxIn;
+use bdk::wallet::AddressIndex;
+use bdk::wallet::AddressIndex::New;
+use bdk::{psbt, FeeRate, SignOptions};
+use bitcoin::util::psbt::PartiallySignedTransaction as Psbt;
+use core::str::FromStr;
+mod common;
+use common::*;
+
+// from bip 174
+const PSBT_STR: &str = "cHNidP8BAKACAAAAAqsJSaCMWvfEm4IS9Bfi8Vqz9cM9zxU4IagTn4d6W3vkAAAAAAD+////qwlJoIxa98SbghL0F+LxWrP1wz3PFTghqBOfh3pbe+QBAAAAAP7///8CYDvqCwAAAAAZdqkUdopAu9dAy+gdmI5x3ipNXHE5ax2IrI4kAAAAAAAAGXapFG9GILVT+glechue4O/p+gOcykWXiKwAAAAAAAEHakcwRAIgR1lmF5fAGwNrJZKJSGhiGDR9iYZLcZ4ff89X0eURZYcCIFMJ6r9Wqk2Ikf/REf3xM286KdqGbX+EhtdVRs7tr5MZASEDXNxh/HupccC1AaZGoqg7ECy0OIEhfKaC3Ibi1z+ogpIAAQEgAOH1BQAAAAAXqRQ1RebjO4MsRwUPJNPuuTycA5SLx4cBBBYAFIXRNTfy4mVAWjTbr6nj3aAfuCMIAAAA";
+
+#[test]
+#[should_panic(expected = "InputIndexOutOfRange")]
+fn test_psbt_malformed_psbt_input_legacy() {
+    let psbt_bip = Psbt::from_str(PSBT_STR).unwrap();
+    let (mut wallet, _) = get_funded_wallet(get_test_wpkh());
+    let send_to = wallet.get_address(AddressIndex::New);
+    let mut builder = wallet.build_tx();
+    builder.add_recipient(send_to.script_pubkey(), 10_000);
+    let (mut psbt, _) = builder.finish().unwrap();
+    psbt.inputs.push(psbt_bip.inputs[0].clone());
+    let options = SignOptions {
+        trust_witness_utxo: true,
+        ..Default::default()
+    };
+    let _ = wallet.sign(&mut psbt, options).unwrap();
+}
+
+#[test]
+#[should_panic(expected = "InputIndexOutOfRange")]
+fn test_psbt_malformed_psbt_input_segwit() {
+    let psbt_bip = Psbt::from_str(PSBT_STR).unwrap();
+    let (mut wallet, _) = get_funded_wallet(get_test_wpkh());
+    let send_to = wallet.get_address(AddressIndex::New);
+    let mut builder = wallet.build_tx();
+    builder.add_recipient(send_to.script_pubkey(), 10_000);
+    let (mut psbt, _) = builder.finish().unwrap();
+    psbt.inputs.push(psbt_bip.inputs[1].clone());
+    let options = SignOptions {
+        trust_witness_utxo: true,
+        ..Default::default()
+    };
+    let _ = wallet.sign(&mut psbt, options).unwrap();
+}
+
+#[test]
+#[should_panic(expected = "InputIndexOutOfRange")]
+fn test_psbt_malformed_tx_input() {
+    let (mut wallet, _) = get_funded_wallet(get_test_wpkh());
+    let send_to = wallet.get_address(AddressIndex::New);
+    let mut builder = wallet.build_tx();
+    builder.add_recipient(send_to.script_pubkey(), 10_000);
+    let (mut psbt, _) = builder.finish().unwrap();
+    psbt.unsigned_tx.input.push(TxIn::default());
+    let options = SignOptions {
+        trust_witness_utxo: true,
+        ..Default::default()
+    };
+    let _ = wallet.sign(&mut psbt, options).unwrap();
+}
+
+#[test]
+fn test_psbt_sign_with_finalized() {
+    let psbt_bip = Psbt::from_str(PSBT_STR).unwrap();
+    let (mut wallet, _) = get_funded_wallet(get_test_wpkh());
+    let send_to = wallet.get_address(AddressIndex::New);
+    let mut builder = wallet.build_tx();
+    builder.add_recipient(send_to.script_pubkey(), 10_000);
+    let (mut psbt, _) = builder.finish().unwrap();
+
+    // add a finalized input
+    psbt.inputs.push(psbt_bip.inputs[0].clone());
+    psbt.unsigned_tx
+        .input
+        .push(psbt_bip.unsigned_tx.input[0].clone());
+
+    let _ = wallet.sign(&mut psbt, SignOptions::default()).unwrap();
+}
+
+#[test]
+fn test_psbt_fee_rate_with_witness_utxo() {
+    use psbt::PsbtUtils;
+
+    let expected_fee_rate = 1.2345;
+
+    let (mut wallet, _) = get_funded_wallet("wpkh(tprv8ZgxMBicQKsPd3EupYiPRhaMooHKUHJxNsTfYuScep13go8QFfHdtkG9nRkFGb7busX4isf6X9dURGCoKgitaApQ6MupRhZMcELAxTBRJgS/*)");
+    let addr = wallet.get_address(New);
+    let mut builder = wallet.build_tx();
+    builder.drain_to(addr.script_pubkey()).drain_wallet();
+    builder.fee_rate(FeeRate::from_sat_per_vb(expected_fee_rate));
+    let (mut psbt, _) = builder.finish().unwrap();
+    let fee_amount = psbt.fee_amount();
+    assert!(fee_amount.is_some());
+
+    let unfinalized_fee_rate = psbt.fee_rate().unwrap();
+
+    let finalized = wallet.sign(&mut psbt, Default::default()).unwrap();
+    assert!(finalized);
+
+    let finalized_fee_rate = psbt.fee_rate().unwrap();
+    assert!(finalized_fee_rate.as_sat_per_vb() >= expected_fee_rate);
+    assert!(finalized_fee_rate.as_sat_per_vb() < unfinalized_fee_rate.as_sat_per_vb());
+}
+
+#[test]
+fn test_psbt_fee_rate_with_nonwitness_utxo() {
+    use psbt::PsbtUtils;
+
+    let expected_fee_rate = 1.2345;
+
+    let (mut wallet, _) = get_funded_wallet("pkh(tprv8ZgxMBicQKsPd3EupYiPRhaMooHKUHJxNsTfYuScep13go8QFfHdtkG9nRkFGb7busX4isf6X9dURGCoKgitaApQ6MupRhZMcELAxTBRJgS/*)");
+    let addr = wallet.get_address(New);
+    let mut builder = wallet.build_tx();
+    builder.drain_to(addr.script_pubkey()).drain_wallet();
+    builder.fee_rate(FeeRate::from_sat_per_vb(expected_fee_rate));
+    let (mut psbt, _) = builder.finish().unwrap();
+    let fee_amount = psbt.fee_amount();
+    assert!(fee_amount.is_some());
+    let unfinalized_fee_rate = psbt.fee_rate().unwrap();
+
+    let finalized = wallet.sign(&mut psbt, Default::default()).unwrap();
+    assert!(finalized);
+
+    let finalized_fee_rate = psbt.fee_rate().unwrap();
+    assert!(finalized_fee_rate.as_sat_per_vb() >= expected_fee_rate);
+    assert!(finalized_fee_rate.as_sat_per_vb() < unfinalized_fee_rate.as_sat_per_vb());
+}
+
+#[test]
+fn test_psbt_fee_rate_with_missing_txout() {
+    use psbt::PsbtUtils;
+
+    let expected_fee_rate = 1.2345;
+
+    let (mut wpkh_wallet,  _) = get_funded_wallet("wpkh(tprv8ZgxMBicQKsPd3EupYiPRhaMooHKUHJxNsTfYuScep13go8QFfHdtkG9nRkFGb7busX4isf6X9dURGCoKgitaApQ6MupRhZMcELAxTBRJgS/*)");
+    let addr = wpkh_wallet.get_address(New);
+    let mut builder = wpkh_wallet.build_tx();
+    builder.drain_to(addr.script_pubkey()).drain_wallet();
+    builder.fee_rate(FeeRate::from_sat_per_vb(expected_fee_rate));
+    let (mut wpkh_psbt, _) = builder.finish().unwrap();
+
+    wpkh_psbt.inputs[0].witness_utxo = None;
+    wpkh_psbt.inputs[0].non_witness_utxo = None;
+    assert!(wpkh_psbt.fee_amount().is_none());
+    assert!(wpkh_psbt.fee_rate().is_none());
+
+    let (mut pkh_wallet,  _) = get_funded_wallet("pkh(tprv8ZgxMBicQKsPd3EupYiPRhaMooHKUHJxNsTfYuScep13go8QFfHdtkG9nRkFGb7busX4isf6X9dURGCoKgitaApQ6MupRhZMcELAxTBRJgS/*)");
+    let addr = pkh_wallet.get_address(New);
+    let mut builder = pkh_wallet.build_tx();
+    builder.drain_to(addr.script_pubkey()).drain_wallet();
+    builder.fee_rate(FeeRate::from_sat_per_vb(expected_fee_rate));
+    let (mut pkh_psbt, _) = builder.finish().unwrap();
+
+    pkh_psbt.inputs[0].non_witness_utxo = None;
+    assert!(pkh_psbt.fee_amount().is_none());
+    assert!(pkh_psbt.fee_rate().is_none());
+}

crates/chain/Cargo.toml

@@ -0,0 +1,31 @@
+[package]
+name = "bdk_chain"
+version = "0.5.0"
+edition = "2021"
+rust-version = "1.57"
+homepage = "https://bitcoindevkit.org"
+repository = "https://github.com/bitcoindevkit/bdk"
+documentation = "https://docs.rs/bdk_chain"
+description = "Collection of core structures for Bitcoin Dev Kit."
+license = "MIT OR Apache-2.0"
+readme = "README.md"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+# For no-std, remember to enable the bitcoin/no-std feature
+bitcoin = { version = "0.29", default-features = false }
+serde_crate = { package = "serde", version = "1", optional = true, features = ["derive"] }
+
+# Use hashbrown as a feature flag to have HashSet and HashMap from it.
+# note version 0.13 breaks outs MSRV.
+hashbrown = { version = "0.11", optional = true, features = ["serde"] }
+miniscript = { version = "9.0.0", optional = true, default-features = false }
+
+[dev-dependencies]
+rand = "0.8"
+
+[features]
+default = ["std"]
+std = ["bitcoin/std", "miniscript/std"]
+serde = ["serde_crate", "bitcoin/serde"]

crates/chain/README.md

@@ -0,0 +1,3 @@
+# BDK Chain
+
+BDK keychain tracker, tools for storing and indexing chain data.

crates/chain/src/chain_data.rs

@@ -0,0 +1,258 @@
+use bitcoin::{hashes::Hash, BlockHash, OutPoint, TxOut, Txid};
+
+use crate::{Anchor, COINBASE_MATURITY};
+
+/// Represents the observed position of some chain data.
+///
+/// The generic `A` should be a [`Anchor`] implementation.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, core::hash::Hash)]
+pub enum ChainPosition<A> {
+    /// The chain data is seen as confirmed, and in anchored by `A`.
+    Confirmed(A),
+    /// The chain data is seen in mempool at this given timestamp.
+    Unconfirmed(u64),
+}
+
+impl<A> ChainPosition<A> {
+    /// Returns whether [`ChainPosition`] is confirmed or not.
+    pub fn is_confirmed(&self) -> bool {
+        matches!(self, Self::Confirmed(_))
+    }
+}
+
+impl<A: Clone> ChainPosition<&A> {
+    /// Maps a [`ChainPosition<&A>`] into a [`ChainPosition<A>`] by cloning the contents.
+    pub fn cloned(self) -> ChainPosition<A> {
+        match self {
+            ChainPosition::Confirmed(a) => ChainPosition::Confirmed(a.clone()),
+            ChainPosition::Unconfirmed(last_seen) => ChainPosition::Unconfirmed(last_seen),
+        }
+    }
+}
+
+impl<A: Anchor> ChainPosition<A> {
+    /// Determines the upper bound of the confirmation height.
+    pub fn confirmation_height_upper_bound(&self) -> Option<u32> {
+        match self {
+            ChainPosition::Confirmed(a) => Some(a.confirmation_height_upper_bound()),
+            ChainPosition::Unconfirmed(_) => None,
+        }
+    }
+}
+
+/// Block height and timestamp at which a transaction is confirmed.
+#[derive(Debug, Clone, PartialEq, Eq, Copy, PartialOrd, Ord, core::hash::Hash)]
+#[cfg_attr(
+    feature = "serde",
+    derive(serde::Deserialize, serde::Serialize),
+    serde(crate = "serde_crate")
+)]
+pub enum ConfirmationTime {
+    /// The confirmed variant.
+    Confirmed {
+        /// Confirmation height.
+        height: u32,
+        /// Confirmation time in unix seconds.
+        time: u64,
+    },
+    /// The unconfirmed variant.
+    Unconfirmed {
+        /// The last-seen timestamp in unix seconds.
+        last_seen: u64,
+    },
+}
+
+impl ConfirmationTime {
+    /// Construct an unconfirmed variant using the given `last_seen` time in unix seconds.
+    pub fn unconfirmed(last_seen: u64) -> Self {
+        Self::Unconfirmed { last_seen }
+    }
+
+    /// Returns whether [`ConfirmationTime`] is the confirmed variant.
+    pub fn is_confirmed(&self) -> bool {
+        matches!(self, Self::Confirmed { .. })
+    }
+}
+
+impl From<ChainPosition<ConfirmationTimeAnchor>> for ConfirmationTime {
+    fn from(observed_as: ChainPosition<ConfirmationTimeAnchor>) -> Self {
+        match observed_as {
+            ChainPosition::Confirmed(a) => Self::Confirmed {
+                height: a.confirmation_height,
+                time: a.confirmation_time,
+            },
+            ChainPosition::Unconfirmed(_) => Self::Unconfirmed { last_seen: 0 },
+        }
+    }
+}
+
+/// A reference to a block in the canonical chain.
+#[derive(Debug, Clone, PartialEq, Eq, Copy, PartialOrd, Ord, core::hash::Hash)]
+#[cfg_attr(
+    feature = "serde",
+    derive(serde::Deserialize, serde::Serialize),
+    serde(crate = "serde_crate")
+)]
+pub struct BlockId {
+    /// The height of the block.
+    pub height: u32,
+    /// The hash of the block.
+    pub hash: BlockHash,
+}
+
+impl Default for BlockId {
+    fn default() -> Self {
+        Self {
+            height: Default::default(),
+            hash: BlockHash::from_inner([0u8; 32]),
+        }
+    }
+}
+
+impl From<(u32, BlockHash)> for BlockId {
+    fn from((height, hash): (u32, BlockHash)) -> Self {
+        Self { height, hash }
+    }
+}
+
+impl From<BlockId> for (u32, BlockHash) {
+    fn from(block_id: BlockId) -> Self {
+        (block_id.height, block_id.hash)
+    }
+}
+
+impl From<(&u32, &BlockHash)> for BlockId {
+    fn from((height, hash): (&u32, &BlockHash)) -> Self {
+        Self {
+            height: *height,
+            hash: *hash,
+        }
+    }
+}
+
+/// An [`Anchor`] implementation that also records the exact confirmation height of the transaction.
+#[derive(Debug, Default, Clone, PartialEq, Eq, Copy, PartialOrd, Ord, core::hash::Hash)]
+#[cfg_attr(
+    feature = "serde",
+    derive(serde::Deserialize, serde::Serialize),
+    serde(crate = "serde_crate")
+)]
+pub struct ConfirmationHeightAnchor {
+    /// The anchor block.
+    pub anchor_block: BlockId,
+
+    /// The exact confirmation height of the transaction.
+    ///
+    /// It is assumed that this value is never larger than the height of the anchor block.
+    pub confirmation_height: u32,
+}
+
+impl Anchor for ConfirmationHeightAnchor {
+    fn anchor_block(&self) -> BlockId {
+        self.anchor_block
+    }
+
+    fn confirmation_height_upper_bound(&self) -> u32 {
+        self.confirmation_height
+    }
+}
+
+/// An [`Anchor`] implementation that also records the exact confirmation time and height of the
+/// transaction.
+#[derive(Debug, Default, Clone, PartialEq, Eq, Copy, PartialOrd, Ord, core::hash::Hash)]
+#[cfg_attr(
+    feature = "serde",
+    derive(serde::Deserialize, serde::Serialize),
+    serde(crate = "serde_crate")
+)]
+pub struct ConfirmationTimeAnchor {
+    /// The anchor block.
+    pub anchor_block: BlockId,
+    /// The confirmation height of the chain data being anchored.
+    pub confirmation_height: u32,
+    /// The confirmation time of the chain data being anchored.
+    pub confirmation_time: u64,
+}
+
+impl Anchor for ConfirmationTimeAnchor {
+    fn anchor_block(&self) -> BlockId {
+        self.anchor_block
+    }
+
+    fn confirmation_height_upper_bound(&self) -> u32 {
+        self.confirmation_height
+    }
+}
+/// A `TxOut` with as much data as we can retrieve about it
+#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)]
+pub struct FullTxOut<A> {
+    /// The location of the `TxOut`.
+    pub outpoint: OutPoint,
+    /// The `TxOut`.
+    pub txout: TxOut,
+    /// The position of the transaction in `outpoint` in the overall chain.
+    pub chain_position: ChainPosition<A>,
+    /// The txid and chain position of the transaction (if any) that has spent this output.
+    pub spent_by: Option<(ChainPosition<A>, Txid)>,
+    /// Whether this output is on a coinbase transaction.
+    pub is_on_coinbase: bool,
+}
+
+impl<A: Anchor> FullTxOut<A> {
+    /// Whether the `txout` is considered mature.
+    ///
+    /// Depending on the implementation of [`confirmation_height_upper_bound`] in [`Anchor`], this
+    /// method may return false-negatives. In other words, interpretted confirmation count may be
+    /// less than the actual value.
+    ///
+    /// [`confirmation_height_upper_bound`]: Anchor::confirmation_height_upper_bound
+    pub fn is_mature(&self, tip: u32) -> bool {
+        if self.is_on_coinbase {
+            let tx_height = match &self.chain_position {
+                ChainPosition::Confirmed(anchor) => anchor.confirmation_height_upper_bound(),
+                ChainPosition::Unconfirmed(_) => {
+                    debug_assert!(false, "coinbase tx can never be unconfirmed");
+                    return false;
+                }
+            };
+            let age = tip.saturating_sub(tx_height);
+            if age + 1 < COINBASE_MATURITY {
+                return false;
+            }
+        }
+
+        true
+    }
+
+    /// Whether the utxo is/was/will be spendable with chain `tip`.
+    ///
+    /// This method does not take into account the locktime.
+    ///
+    /// Depending on the implementation of [`confirmation_height_upper_bound`] in [`Anchor`], this
+    /// method may return false-negatives. In other words, interpretted confirmation count may be
+    /// less than the actual value.
+    ///
+    /// [`confirmation_height_upper_bound`]: Anchor::confirmation_height_upper_bound
+    pub fn is_confirmed_and_spendable(&self, tip: u32) -> bool {
+        if !self.is_mature(tip) {
+            return false;
+        }
+
+        let confirmation_height = match &self.chain_position {
+            ChainPosition::Confirmed(anchor) => anchor.confirmation_height_upper_bound(),
+            ChainPosition::Unconfirmed(_) => return false,
+        };
+        if confirmation_height > tip {
+            return false;
+        }
+
+        // if the spending tx is confirmed within tip height, the txout is no longer spendable
+        if let Some((ChainPosition::Confirmed(spending_anchor), _)) = &self.spent_by {
+            if spending_anchor.anchor_block().height <= tip {
+                return false;
+            }
+        }
+
+        true
+    }
+}

crates/chain/src/chain_oracle.rs

@@ -0,0 +1,25 @@
+use crate::BlockId;
+
+/// Represents a service that tracks the blockchain.
+///
+/// The main method is [`is_block_in_chain`] which determines whether a given block of [`BlockId`]
+/// is an ancestor of another "static block".
+///
+/// [`is_block_in_chain`]: Self::is_block_in_chain
+pub trait ChainOracle {
+    /// Error type.
+    type Error: core::fmt::Debug;
+
+    /// Determines whether `block` of [`BlockId`] exists as an ancestor of `chain_tip`.
+    ///
+    /// If `None` is returned, it means the implementation cannot determine whether `block` exists
+    /// under `chain_tip`.
+    fn is_block_in_chain(
+        &self,
+        block: BlockId,
+        chain_tip: BlockId,
+    ) -> Result<Option<bool>, Self::Error>;
+
+    /// Get the best chain's chain tip.
+    fn get_chain_tip(&self) -> Result<Option<BlockId>, Self::Error>;
+}

crates/chain/src/descriptor_ext.rs

@@ -0,0 +1,16 @@
+use crate::miniscript::{Descriptor, DescriptorPublicKey};
+
+/// A trait to extend the functionality of a miniscript descriptor.
+pub trait DescriptorExt {
+    /// Returns the minimum value (in satoshis) at which an output is broadcastable.
+    fn dust_value(&self) -> u64;
+}
+
+impl DescriptorExt for Descriptor<DescriptorPublicKey> {
+    fn dust_value(&self) -> u64 {
+        self.at_derivation_index(0)
+            .script_pubkey()
+            .dust_value()
+            .to_sat()
+    }
+}

crates/chain/src/example_utils.rs

@@ -0,0 +1,30 @@
+#![allow(unused)]
+use alloc::vec::Vec;
+use bitcoin::{
+    consensus,
+    hashes::{hex::FromHex, Hash},
+    Transaction,
+};
+
+use crate::BlockId;
+
+pub const RAW_TX_1: &str = "0200000000010116d6174da7183d70d0a7d4dc314d517a7d135db79ad63515028b293a76f4f9d10000000000feffffff023a21fc8350060000160014531c405e1881ef192294b8813631e258bf98ea7a1027000000000000225120a60869f0dbcf1dc659c9cecbaf8050135ea9e8cdc487053f1dc6880949dc684c024730440220591b1a172a122da49ba79a3e79f98aaa03fd7a372f9760da18890b6a327e6010022013e82319231da6c99abf8123d7c07e13cf9bd8d76e113e18dc452e5024db156d012102318a2d558b2936c52e320decd6d92a88d7f530be91b6fe0af5caf41661e77da3ef2e0100";
+pub const RAW_TX_2: &str = "02000000000101a688607020cfae91a61e7c516b5ef1264d5d77f17200c3866826c6c808ebf1620000000000feffffff021027000000000000225120a60869f0dbcf1dc659c9cecbaf8050135ea9e8cdc487053f1dc6880949dc684c20fd48ff530600001600146886c525e41d4522042bd0b159dfbade2504a6bb024730440220740ff7e665cd20565d4296b549df8d26b941be3f1e3af89a0b60e50c0dbeb69a02206213ab7030cf6edc6c90d4ccf33010644261e029950a688dc0b1a9ebe6ddcc5a012102f2ac6b396a97853cb6cd62242c8ae4842024742074475023532a51e9c53194253e760100";
+pub const RAW_TX_3: &str = "0200000000010135d67ee47b557e68b8c6223958f597381965ed719f1207ee2b9e20432a24a5dc0100000000feffffff021027000000000000225120a82f29944d65b86ae6b5e5cc75e294ead6c59391a1edc5e016e3498c67fc7bbb62215a5055060000160014070df7671dea67a50c4799a744b5c9be8f4bac690247304402207ebf8d29f71fd03e7e6977b3ea78ca5fcc5c49a42ae822348fc401862fdd766c02201d7e4ff0684ecb008b6142f36ead1b0b4d615524c4f58c261113d361f4427e25012103e6a75e2fab85e5ecad641afc4ffba7222f998649d9f18cac92f0fcc8618883b3ee760100";
+pub const RAW_TX_4: &str = "02000000000101d00e8f76ed313e19b339ee293c0f52b0325c95e24c8f3966fa353fb2bedbcf580100000000feffffff021027000000000000225120882d74e5d0572d5a816cef0041a96b6c1de832f6f9676d9605c44d5e9a97d3dc9cda55fe53060000160014852b5864b8edd42fab4060c87f818e50780865ff0247304402201dccbb9bed7fba924b6d249c5837cc9b37470c0e3d8fbea77cb59baba3efe6fa0220700cc170916913b9bfc2bc0fefb6af776e8b542c561702f136cddc1c7aa43141012103acec3fc79dbbca745815c2a807dc4e81010c80e308e84913f59cb42a275dad97f3760100";
+
+pub fn tx_from_hex(s: &str) -> Transaction {
+    let raw = Vec::from_hex(s).expect("data must be in hex");
+    consensus::deserialize(raw.as_slice()).expect("must deserialize")
+}
+
+pub fn new_hash<H: Hash>(s: &str) -> H {
+    <H as bitcoin::hashes::Hash>::hash(s.as_bytes())
+}
+
+pub fn new_block_id(height: u32, hash: &str) -> BlockId {
+    BlockId {
+        height,
+        hash: new_hash(hash),
+    }
+}

crates/chain/src/indexed_tx_graph.rs

@@ -0,0 +1,245 @@
+//! Contains the [`IndexedTxGraph`] structure and associated types.
+//!
+//! This is essentially a [`TxGraph`] combined with an indexer.
+
+use alloc::vec::Vec;
+use bitcoin::{OutPoint, Transaction, TxOut};
+
+use crate::{
+    keychain::DerivationAdditions,
+    tx_graph::{Additions, TxGraph},
+    Anchor, Append,
+};
+
+/// A struct that combines [`TxGraph`] and an [`Indexer`] implementation.
+///
+/// This structure ensures that [`TxGraph`] and [`Indexer`] are updated atomically.
+#[derive(Debug)]
+pub struct IndexedTxGraph<A, I> {
+    /// Transaction index.
+    pub index: I,
+    graph: TxGraph<A>,
+}
+
+impl<A, I: Default> Default for IndexedTxGraph<A, I> {
+    fn default() -> Self {
+        Self {
+            graph: Default::default(),
+            index: Default::default(),
+        }
+    }
+}
+
+impl<A, I> IndexedTxGraph<A, I> {
+    /// Construct a new [`IndexedTxGraph`] with a given `index`.
+    pub fn new(index: I) -> Self {
+        Self {
+            index,
+            graph: TxGraph::default(),
+        }
+    }
+
+    /// Get a reference of the internal transaction graph.
+    pub fn graph(&self) -> &TxGraph<A> {
+        &self.graph
+    }
+}
+
+impl<A: Anchor, I: Indexer> IndexedTxGraph<A, I> {
+    /// Applies the [`IndexedAdditions`] to the [`IndexedTxGraph`].
+    pub fn apply_additions(&mut self, additions: IndexedAdditions<A, I::Additions>) {
+        let IndexedAdditions {
+            graph_additions,
+            index_additions,
+        } = additions;
+
+        self.index.apply_additions(index_additions);
+
+        for tx in &graph_additions.txs {
+            self.index.index_tx(tx);
+        }
+        for (&outpoint, txout) in &graph_additions.txouts {
+            self.index.index_txout(outpoint, txout);
+        }
+
+        self.graph.apply_additions(graph_additions);
+    }
+}
+
+impl<A: Anchor, I: Indexer> IndexedTxGraph<A, I>
+where
+    I::Additions: Default + Append,
+{
+    /// Apply an `update` directly.
+    ///
+    /// `update` is a [`TxGraph<A>`] and the resultant changes is returned as [`IndexedAdditions`].
+    pub fn apply_update(&mut self, update: TxGraph<A>) -> IndexedAdditions<A, I::Additions> {
+        let graph_additions = self.graph.apply_update(update);
+
+        let mut index_additions = I::Additions::default();
+        for added_tx in &graph_additions.txs {
+            index_additions.append(self.index.index_tx(added_tx));
+        }
+        for (&added_outpoint, added_txout) in &graph_additions.txouts {
+            index_additions.append(self.index.index_txout(added_outpoint, added_txout));
+        }
+
+        IndexedAdditions {
+            graph_additions,
+            index_additions,
+        }
+    }
+
+    /// Insert a floating `txout` of given `outpoint`.
+    pub fn insert_txout(
+        &mut self,
+        outpoint: OutPoint,
+        txout: &TxOut,
+    ) -> IndexedAdditions<A, I::Additions> {
+        let mut update = TxGraph::<A>::default();
+        let _ = update.insert_txout(outpoint, txout.clone());
+        self.apply_update(update)
+    }
+
+    /// Insert and index a transaction into the graph.
+    ///
+    /// `anchors` can be provided to anchor the transaction to various blocks. `seen_at` is a
+    /// unix timestamp of when the transaction is last seen.
+    pub fn insert_tx(
+        &mut self,
+        tx: &Transaction,
+        anchors: impl IntoIterator<Item = A>,
+        seen_at: Option<u64>,
+    ) -> IndexedAdditions<A, I::Additions> {
+        let txid = tx.txid();
+
+        let mut update = TxGraph::<A>::default();
+        if self.graph.get_tx(txid).is_none() {
+            let _ = update.insert_tx(tx.clone());
+        }
+        for anchor in anchors.into_iter() {
+            let _ = update.insert_anchor(txid, anchor);
+        }
+        if let Some(seen_at) = seen_at {
+            let _ = update.insert_seen_at(txid, seen_at);
+        }
+
+        self.apply_update(update)
+    }
+
+    /// Insert relevant transactions from the given `txs` iterator.
+    ///
+    /// Relevancy is determined by the [`Indexer::is_tx_relevant`] implementation of `I`. Irrelevant
+    /// transactions in `txs` will be ignored. `txs` do not need to be in topological order.
+    ///
+    /// `anchors` can be provided to anchor the transactions to blocks. `seen_at` is a unix
+    /// timestamp of when the transactions are last seen.
+    pub fn insert_relevant_txs<'t>(
+        &mut self,
+        txs: impl IntoIterator<Item = (&'t Transaction, impl IntoIterator<Item = A>)>,
+        seen_at: Option<u64>,
+    ) -> IndexedAdditions<A, I::Additions> {
+        // The algorithm below allows for non-topologically ordered transactions by using two loops.
+        // This is achieved by:
+        // 1. insert all txs into the index. If they are irrelevant then that's fine it will just
+        //    not store anything about them.
+        // 2. decide whether to insert them into the graph depending on whether `is_tx_relevant`
+        //    returns true or not. (in a second loop).
+        let mut additions = IndexedAdditions::<A, I::Additions>::default();
+        let mut transactions = Vec::new();
+        for (tx, anchors) in txs.into_iter() {
+            additions.index_additions.append(self.index.index_tx(tx));
+            transactions.push((tx, anchors));
+        }
+        additions.append(
+            transactions
+                .into_iter()
+                .filter_map(|(tx, anchors)| match self.index.is_tx_relevant(tx) {
+                    true => Some(self.insert_tx(tx, anchors, seen_at)),
+                    false => None,
+                })
+                .fold(Default::default(), |mut acc, other| {
+                    acc.append(other);
+                    acc
+                }),
+        );
+        additions
+    }
+}
+
+/// A structure that represents changes to an [`IndexedTxGraph`].
+#[derive(Clone, Debug, PartialEq)]
+#[cfg_attr(
+    feature = "serde",
+    derive(serde::Deserialize, serde::Serialize),
+    serde(
+        crate = "serde_crate",
+        bound(
+            deserialize = "A: Ord + serde::Deserialize<'de>, IA: serde::Deserialize<'de>",
+            serialize = "A: Ord + serde::Serialize, IA: serde::Serialize"
+        )
+    )
+)]
+#[must_use]
+pub struct IndexedAdditions<A, IA> {
+    /// [`TxGraph`] additions.
+    pub graph_additions: Additions<A>,
+    /// [`Indexer`] additions.
+    pub index_additions: IA,
+}
+
+impl<A, IA: Default> Default for IndexedAdditions<A, IA> {
+    fn default() -> Self {
+        Self {
+            graph_additions: Default::default(),
+            index_additions: Default::default(),
+        }
+    }
+}
+
+impl<A: Anchor, IA: Append> Append for IndexedAdditions<A, IA> {
+    fn append(&mut self, other: Self) {
+        self.graph_additions.append(other.graph_additions);
+        self.index_additions.append(other.index_additions);
+    }
+
+    fn is_empty(&self) -> bool {
+        self.graph_additions.is_empty() && self.index_additions.is_empty()
+    }
+}
+
+impl<A, IA: Default> From<Additions<A>> for IndexedAdditions<A, IA> {
+    fn from(graph_additions: Additions<A>) -> Self {
+        Self {
+            graph_additions,
+            ..Default::default()
+        }
+    }
+}
+
+impl<A, K> From<DerivationAdditions<K>> for IndexedAdditions<A, DerivationAdditions<K>> {
+    fn from(index_additions: DerivationAdditions<K>) -> Self {
+        Self {
+            graph_additions: Default::default(),
+            index_additions,
+        }
+    }
+}
+
+/// Represents a structure that can index transaction data.
+pub trait Indexer {
+    /// The resultant "additions" when new transaction data is indexed.
+    type Additions;
+
+    /// Scan and index the given `outpoint` and `txout`.
+    fn index_txout(&mut self, outpoint: OutPoint, txout: &TxOut) -> Self::Additions;
+
+    /// Scan and index the given transaction.
+    fn index_tx(&mut self, tx: &Transaction) -> Self::Additions;
+
+    /// Apply additions to itself.
+    fn apply_additions(&mut self, additions: Self::Additions);
+
+    /// Determines whether the transaction should be included in the index.
+    fn is_tx_relevant(&self, tx: &Transaction) -> bool;
+}

crates/chain/src/keychain.rs

@@ -0,0 +1,266 @@
+//! Module for keychain related structures.
+//!
+//! A keychain here is a set of application-defined indexes for a miniscript descriptor where we can
+//! derive script pubkeys at a particular derivation index. The application's index is simply
+//! anything that implements `Ord`.
+//!
+//! [`KeychainTxOutIndex`] indexes script pubkeys of keychains and scans in relevant outpoints (that
+//! has a `txout` containing an indexed script pubkey). Internally, this uses [`SpkTxOutIndex`], but
+//! also maintains "revealed" and "lookahead" index counts per keychain.
+//!
+//! [`SpkTxOutIndex`]: crate::SpkTxOutIndex
+
+use crate::{
+    collections::BTreeMap,
+    indexed_tx_graph::IndexedAdditions,
+    local_chain::{self, LocalChain},
+    tx_graph::TxGraph,
+    Anchor, Append,
+};
+
+#[cfg(feature = "miniscript")]
+mod txout_index;
+#[cfg(feature = "miniscript")]
+pub use txout_index::*;
+
+/// Represents updates to the derivation index of a [`KeychainTxOutIndex`].
+///
+/// It can be applied to [`KeychainTxOutIndex`] with [`apply_additions`]. [`DerivationAdditions] are
+/// monotone in that they will never decrease the revealed derivation index.
+///
+/// [`KeychainTxOutIndex`]: crate::keychain::KeychainTxOutIndex
+/// [`apply_additions`]: crate::keychain::KeychainTxOutIndex::apply_additions
+#[derive(Clone, Debug, PartialEq)]
+#[cfg_attr(
+    feature = "serde",
+    derive(serde::Deserialize, serde::Serialize),
+    serde(
+        crate = "serde_crate",
+        bound(
+            deserialize = "K: Ord + serde::Deserialize<'de>",
+            serialize = "K: Ord + serde::Serialize"
+        )
+    )
+)]
+#[must_use]
+pub struct DerivationAdditions<K>(pub BTreeMap<K, u32>);
+
+impl<K> DerivationAdditions<K> {
+    /// Returns whether the additions are empty.
+    pub fn is_empty(&self) -> bool {
+        self.0.is_empty()
+    }
+
+    /// Get the inner map of the keychain to its new derivation index.
+    pub fn as_inner(&self) -> &BTreeMap<K, u32> {
+        &self.0
+    }
+}
+
+impl<K: Ord> Append for DerivationAdditions<K> {
+    /// Append another [`DerivationAdditions`] into self.
+    ///
+    /// If the keychain already exists, increase the index when the other's index > self's index.
+    /// If the keychain did not exist, append the new keychain.
+    fn append(&mut self, mut other: Self) {
+        self.0.iter_mut().for_each(|(key, index)| {
+            if let Some(other_index) = other.0.remove(key) {
+                *index = other_index.max(*index);
+            }
+        });
+
+        self.0.append(&mut other.0);
+    }
+
+    fn is_empty(&self) -> bool {
+        self.0.is_empty()
+    }
+}
+
+impl<K> Default for DerivationAdditions<K> {
+    fn default() -> Self {
+        Self(Default::default())
+    }
+}
+
+impl<K> AsRef<BTreeMap<K, u32>> for DerivationAdditions<K> {
+    fn as_ref(&self) -> &BTreeMap<K, u32> {
+        &self.0
+    }
+}
+
+/// A structure to update [`KeychainTxOutIndex`], [`TxGraph`] and [`LocalChain`]
+/// atomically.
+#[derive(Debug, Clone, PartialEq)]
+pub struct LocalUpdate<K, A> {
+    /// Last active derivation index per keychain (`K`).
+    pub keychain: BTreeMap<K, u32>,
+    /// Update for the [`TxGraph`].
+    pub graph: TxGraph<A>,
+    /// Update for the [`LocalChain`].
+    pub chain: LocalChain,
+}
+
+impl<K, A> Default for LocalUpdate<K, A> {
+    fn default() -> Self {
+        Self {
+            keychain: Default::default(),
+            graph: Default::default(),
+            chain: Default::default(),
+        }
+    }
+}
+
+/// A structure that records the corresponding changes as result of applying an [`LocalUpdate`].
+#[derive(Debug, Clone, PartialEq)]
+#[cfg_attr(
+    feature = "serde",
+    derive(serde::Deserialize, serde::Serialize),
+    serde(
+        crate = "serde_crate",
+        bound(
+            deserialize = "K: Ord + serde::Deserialize<'de>, A: Ord + serde::Deserialize<'de>",
+            serialize = "K: Ord + serde::Serialize, A: Ord + serde::Serialize",
+        )
+    )
+)]
+pub struct LocalChangeSet<K, A> {
+    /// Changes to the [`LocalChain`].
+    pub chain_changeset: local_chain::ChangeSet,
+
+    /// Additions to [`IndexedTxGraph`].
+    ///
+    /// [`IndexedTxGraph`]: crate::indexed_tx_graph::IndexedTxGraph
+    pub indexed_additions: IndexedAdditions<A, DerivationAdditions<K>>,
+}
+
+impl<K, A> Default for LocalChangeSet<K, A> {
+    fn default() -> Self {
+        Self {
+            chain_changeset: Default::default(),
+            indexed_additions: Default::default(),
+        }
+    }
+}
+
+impl<K: Ord, A: Anchor> Append for LocalChangeSet<K, A> {
+    fn append(&mut self, other: Self) {
+        Append::append(&mut self.chain_changeset, other.chain_changeset);
+        Append::append(&mut self.indexed_additions, other.indexed_additions);
+    }
+
+    fn is_empty(&self) -> bool {
+        self.chain_changeset.is_empty() && self.indexed_additions.is_empty()
+    }
+}
+
+impl<K, A> From<local_chain::ChangeSet> for LocalChangeSet<K, A> {
+    fn from(chain_changeset: local_chain::ChangeSet) -> Self {
+        Self {
+            chain_changeset,
+            ..Default::default()
+        }
+    }
+}
+
+impl<K, A> From<IndexedAdditions<A, DerivationAdditions<K>>> for LocalChangeSet<K, A> {
+    fn from(indexed_additions: IndexedAdditions<A, DerivationAdditions<K>>) -> Self {
+        Self {
+            indexed_additions,
+            ..Default::default()
+        }
+    }
+}
+
+/// Balance, differentiated into various categories.
+#[derive(Debug, PartialEq, Eq, Clone, Default)]
+#[cfg_attr(
+    feature = "serde",
+    derive(serde::Deserialize, serde::Serialize),
+    serde(crate = "serde_crate",)
+)]
+pub struct Balance {
+    /// All coinbase outputs not yet matured
+    pub immature: u64,
+    /// Unconfirmed UTXOs generated by a wallet tx
+    pub trusted_pending: u64,
+    /// Unconfirmed UTXOs received from an external wallet
+    pub untrusted_pending: u64,
+    /// Confirmed and immediately spendable balance
+    pub confirmed: u64,
+}
+
+impl Balance {
+    /// Get sum of trusted_pending and confirmed coins.
+    ///
+    /// This is the balance you can spend right now that shouldn't get cancelled via another party
+    /// double spending it.
+    pub fn trusted_spendable(&self) -> u64 {
+        self.confirmed + self.trusted_pending
+    }
+
+    /// Get the whole balance visible to the wallet.
+    pub fn total(&self) -> u64 {
+        self.confirmed + self.trusted_pending + self.untrusted_pending + self.immature
+    }
+}
+
+impl core::fmt::Display for Balance {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        write!(
+            f,
+            "{{ immature: {}, trusted_pending: {}, untrusted_pending: {}, confirmed: {} }}",
+            self.immature, self.trusted_pending, self.untrusted_pending, self.confirmed
+        )
+    }
+}
+
+impl core::ops::Add for Balance {
+    type Output = Self;
+
+    fn add(self, other: Self) -> Self {
+        Self {
+            immature: self.immature + other.immature,
+            trusted_pending: self.trusted_pending + other.trusted_pending,
+            untrusted_pending: self.untrusted_pending + other.untrusted_pending,
+            confirmed: self.confirmed + other.confirmed,
+        }
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use super::*;
+
+    #[test]
+    fn append_keychain_derivation_indices() {
+        #[derive(Ord, PartialOrd, Eq, PartialEq, Clone, Debug)]
+        enum Keychain {
+            One,
+            Two,
+            Three,
+            Four,
+        }
+        let mut lhs_di = BTreeMap::<Keychain, u32>::default();
+        let mut rhs_di = BTreeMap::<Keychain, u32>::default();
+        lhs_di.insert(Keychain::One, 7);
+        lhs_di.insert(Keychain::Two, 0);
+        rhs_di.insert(Keychain::One, 3);
+        rhs_di.insert(Keychain::Two, 5);
+        lhs_di.insert(Keychain::Three, 3);
+        rhs_di.insert(Keychain::Four, 4);
+
+        let mut lhs = DerivationAdditions(lhs_di);
+        let rhs = DerivationAdditions(rhs_di);
+        lhs.append(rhs);
+
+        // Exiting index doesn't update if the new index in `other` is lower than `self`.
+        assert_eq!(lhs.0.get(&Keychain::One), Some(&7));
+        // Existing index updates if the new index in `other` is higher than `self`.
+        assert_eq!(lhs.0.get(&Keychain::Two), Some(&5));
+        // Existing index is unchanged if keychain doesn't exist in `other`.
+        assert_eq!(lhs.0.get(&Keychain::Three), Some(&3));
+        // New keychain gets added if the keychain is in `other` but not in `self`.
+        assert_eq!(lhs.0.get(&Keychain::Four), Some(&4));
+    }
+}

crates/chain/src/keychain/txout_index.rs

@@ -0,0 +1,588 @@
+use crate::{
+    collections::*,
+    indexed_tx_graph::Indexer,
+    miniscript::{Descriptor, DescriptorPublicKey},
+    spk_iter::BIP32_MAX_INDEX,
+    ForEachTxOut, SpkIterator, SpkTxOutIndex,
+};
+use alloc::vec::Vec;
+use bitcoin::{OutPoint, Script, TxOut};
+use core::{fmt::Debug, ops::Deref};
+
+use crate::Append;
+
+use super::DerivationAdditions;
+
+/// A convenient wrapper around [`SpkTxOutIndex`] that relates script pubkeys to miniscript public
+/// [`Descriptor`]s.
+///
+/// Descriptors are referenced by the provided keychain generic (`K`).
+///
+/// Script pubkeys for a descriptor are revealed chronologically from index 0. I.e., If the last
+/// revealed index of a descriptor is 5; scripts of indices 0 to 4 are guaranteed to be already
+/// revealed. In addition to revealed scripts, we have a `lookahead` parameter for each keychain,
+/// which defines the number of script pubkeys to store ahead of the last revealed index.
+///
+/// Methods that could update the last revealed index will return [`DerivationAdditions`] to report
+/// these changes. This can be persisted for future recovery.
+///
+/// ## Synopsis
+///
+/// ```
+/// use bdk_chain::keychain::KeychainTxOutIndex;
+/// # use bdk_chain::{ miniscript::{Descriptor, DescriptorPublicKey} };
+/// # use core::str::FromStr;
+///
+/// // imagine our service has internal and external addresses but also addresses for users
+/// #[derive(Clone, Debug, PartialEq, Eq, Ord, PartialOrd)]
+/// enum MyKeychain {
+///     External,
+///     Internal,
+///     MyAppUser {
+///         user_id: u32
+///     }
+/// }
+///
+/// let mut txout_index = KeychainTxOutIndex::<MyKeychain>::default();
+///
+/// # let secp = bdk_chain::bitcoin::secp256k1::Secp256k1::signing_only();
+/// # let (external_descriptor,_) = Descriptor::<DescriptorPublicKey>::parse_descriptor(&secp, "tr([73c5da0a/86'/0'/0']xprv9xgqHN7yz9MwCkxsBPN5qetuNdQSUttZNKw1dcYTV4mkaAFiBVGQziHs3NRSWMkCzvgjEe3n9xV8oYywvM8at9yRqyaZVz6TYYhX98VjsUk/0/*)").unwrap();
+/// # let (internal_descriptor,_) = Descriptor::<DescriptorPublicKey>::parse_descriptor(&secp, "tr([73c5da0a/86'/0'/0']xprv9xgqHN7yz9MwCkxsBPN5qetuNdQSUttZNKw1dcYTV4mkaAFiBVGQziHs3NRSWMkCzvgjEe3n9xV8oYywvM8at9yRqyaZVz6TYYhX98VjsUk/1/*)").unwrap();
+/// # let descriptor_for_user_42 = external_descriptor.clone();
+/// txout_index.add_keychain(MyKeychain::External, external_descriptor);
+/// txout_index.add_keychain(MyKeychain::Internal, internal_descriptor);
+/// txout_index.add_keychain(MyKeychain::MyAppUser { user_id: 42 }, descriptor_for_user_42);
+///
+/// let new_spk_for_user = txout_index.reveal_next_spk(&MyKeychain::MyAppUser{ user_id: 42 });
+/// ```
+///
+/// [`Ord`]: core::cmp::Ord
+/// [`SpkTxOutIndex`]: crate::spk_txout_index::SpkTxOutIndex
+/// [`Descriptor`]: crate::miniscript::Descriptor
+#[derive(Clone, Debug)]
+pub struct KeychainTxOutIndex<K> {
+    inner: SpkTxOutIndex<(K, u32)>,
+    // descriptors of each keychain
+    keychains: BTreeMap<K, Descriptor<DescriptorPublicKey>>,
+    // last revealed indexes
+    last_revealed: BTreeMap<K, u32>,
+    // lookahead settings for each keychain
+    lookahead: BTreeMap<K, u32>,
+}
+
+impl<K> Default for KeychainTxOutIndex<K> {
+    fn default() -> Self {
+        Self {
+            inner: SpkTxOutIndex::default(),
+            keychains: BTreeMap::default(),
+            last_revealed: BTreeMap::default(),
+            lookahead: BTreeMap::default(),
+        }
+    }
+}
+
+impl<K> Deref for KeychainTxOutIndex<K> {
+    type Target = SpkTxOutIndex<(K, u32)>;
+
+    fn deref(&self) -> &Self::Target {
+        &self.inner
+    }
+}
+
+impl<K: Clone + Ord + Debug> Indexer for KeychainTxOutIndex<K> {
+    type Additions = DerivationAdditions<K>;
+
+    fn index_txout(&mut self, outpoint: OutPoint, txout: &TxOut) -> Self::Additions {
+        self.scan_txout(outpoint, txout)
+    }
+
+    fn index_tx(&mut self, tx: &bitcoin::Transaction) -> Self::Additions {
+        self.scan(tx)
+    }
+
+    fn apply_additions(&mut self, additions: Self::Additions) {
+        self.apply_additions(additions)
+    }
+
+    fn is_tx_relevant(&self, tx: &bitcoin::Transaction) -> bool {
+        self.is_relevant(tx)
+    }
+}
+
+impl<K: Clone + Ord + Debug> KeychainTxOutIndex<K> {
+    /// Scans an object for relevant outpoints, which are stored and indexed internally.
+    ///
+    /// If the matched script pubkey is part of the lookahead, the last stored index is updated for
+    /// the script pubkey's keychain and the [`DerivationAdditions`] returned will reflect the
+    /// change.
+    ///
+    /// Typically, this method is used in two situations:
+    ///
+    /// 1. After loading transaction data from the disk, you may scan over all the txouts to restore all
+    /// your txouts.
+    /// 2. When getting new data from the chain, you usually scan it before incorporating it into
+    /// your chain state (i.e., `SparseChain`, `ChainGraph`).
+    ///
+    /// See [`ForEachTxout`] for the types that support this.
+    ///
+    /// [`ForEachTxout`]: crate::ForEachTxOut
+    pub fn scan(&mut self, txouts: &impl ForEachTxOut) -> DerivationAdditions<K> {
+        let mut additions = DerivationAdditions::<K>::default();
+        txouts.for_each_txout(|(op, txout)| additions.append(self.scan_txout(op, txout)));
+        additions
+    }
+
+    /// Scan a single outpoint for a matching script pubkey.
+    ///
+    /// If it matches, this will store and index it.
+    pub fn scan_txout(&mut self, op: OutPoint, txout: &TxOut) -> DerivationAdditions<K> {
+        match self.inner.scan_txout(op, txout).cloned() {
+            Some((keychain, index)) => self.reveal_to_target(&keychain, index).1,
+            None => DerivationAdditions::default(),
+        }
+    }
+
+    /// Return a reference to the internal [`SpkTxOutIndex`].
+    pub fn inner(&self) -> &SpkTxOutIndex<(K, u32)> {
+        &self.inner
+    }
+
+    /// Get a reference to the set of indexed outpoints.
+    pub fn outpoints(&self) -> &BTreeSet<((K, u32), OutPoint)> {
+        self.inner.outpoints()
+    }
+
+    /// Return a reference to the internal map of the keychain to descriptors.
+    pub fn keychains(&self) -> &BTreeMap<K, Descriptor<DescriptorPublicKey>> {
+        &self.keychains
+    }
+
+    /// Add a keychain to the tracker's `txout_index` with a descriptor to derive addresses.
+    ///
+    /// Adding a keychain means you will be able to derive new script pubkeys under that keychain
+    /// and the txout index will discover transaction outputs with those script pubkeys.
+    ///
+    /// # Panics
+    ///
+    /// This will panic if a different `descriptor` is introduced to the same `keychain`.
+    pub fn add_keychain(&mut self, keychain: K, descriptor: Descriptor<DescriptorPublicKey>) {
+        let old_descriptor = &*self
+            .keychains
+            .entry(keychain)
+            .or_insert_with(|| descriptor.clone());
+        assert_eq!(
+            &descriptor, old_descriptor,
+            "keychain already contains a different descriptor"
+        );
+    }
+
+    /// Return the lookahead setting for each keychain.
+    ///
+    /// Refer to [`set_lookahead`] for a deeper explanation of the `lookahead`.
+    ///
+    /// [`set_lookahead`]: Self::set_lookahead
+    pub fn lookaheads(&self) -> &BTreeMap<K, u32> {
+        &self.lookahead
+    }
+
+    /// Convenience method to call [`set_lookahead`] for all keychains.
+    ///
+    /// [`set_lookahead`]: Self::set_lookahead
+    pub fn set_lookahead_for_all(&mut self, lookahead: u32) {
+        for keychain in &self.keychains.keys().cloned().collect::<Vec<_>>() {
+            self.lookahead.insert(keychain.clone(), lookahead);
+            self.replenish_lookahead(keychain);
+        }
+    }
+
+    /// Set the lookahead count for `keychain`.
+    ///
+    /// The lookahead is the number of scripts to cache ahead of the last stored script index. This
+    /// is useful during a scan via [`scan`] or [`scan_txout`].
+    ///
+    /// # Panics
+    ///
+    /// This will panic if the `keychain` does not exist.
+    ///
+    /// [`scan`]: Self::scan
+    /// [`scan_txout`]: Self::scan_txout
+    pub fn set_lookahead(&mut self, keychain: &K, lookahead: u32) {
+        self.lookahead.insert(keychain.clone(), lookahead);
+        self.replenish_lookahead(keychain);
+    }
+
+    /// Convenience method to call [`lookahead_to_target`] for multiple keychains.
+    ///
+    /// [`lookahead_to_target`]: Self::lookahead_to_target
+    pub fn lookahead_to_target_multi(&mut self, target_indexes: BTreeMap<K, u32>) {
+        for (keychain, target_index) in target_indexes {
+            self.lookahead_to_target(&keychain, target_index)
+        }
+    }
+
+    /// Store lookahead scripts until `target_index`.
+    ///
+    /// This does not change the `lookahead` setting.
+    pub fn lookahead_to_target(&mut self, keychain: &K, target_index: u32) {
+        let next_index = self.next_store_index(keychain);
+        if let Some(temp_lookahead) = target_index.checked_sub(next_index).filter(|&v| v > 0) {
+            let old_lookahead = self.lookahead.insert(keychain.clone(), temp_lookahead);
+            self.replenish_lookahead(keychain);
+
+            // revert
+            match old_lookahead {
+                Some(lookahead) => self.lookahead.insert(keychain.clone(), lookahead),
+                None => self.lookahead.remove(keychain),
+            };
+        }
+    }
+
+    fn replenish_lookahead(&mut self, keychain: &K) {
+        let descriptor = self.keychains.get(keychain).expect("keychain must exist");
+        let next_store_index = self.next_store_index(keychain);
+        let next_reveal_index = self.last_revealed.get(keychain).map_or(0, |v| *v + 1);
+        let lookahead = self.lookahead.get(keychain).map_or(0, |v| *v);
+
+        for (new_index, new_spk) in
+            SpkIterator::new_with_range(descriptor, next_store_index..next_reveal_index + lookahead)
+        {
+            let _inserted = self
+                .inner
+                .insert_spk((keychain.clone(), new_index), new_spk);
+            debug_assert!(_inserted, "replenish lookahead: must not have existing spk: keychain={:?}, lookahead={}, next_store_index={}, next_reveal_index={}", keychain, lookahead, next_store_index, next_reveal_index);
+        }
+    }
+
+    fn next_store_index(&self, keychain: &K) -> u32 {
+        self.inner()
+            .all_spks()
+            .range((keychain.clone(), u32::MIN)..(keychain.clone(), u32::MAX))
+            .last()
+            .map_or(0, |((_, v), _)| *v + 1)
+    }
+
+    /// Generates script pubkey iterators for every `keychain`. The iterators iterate over all
+    /// derivable script pubkeys.
+    pub fn spks_of_all_keychains(
+        &self,
+    ) -> BTreeMap<K, SpkIterator<Descriptor<DescriptorPublicKey>>> {
+        self.keychains
+            .iter()
+            .map(|(keychain, descriptor)| {
+                (
+                    keychain.clone(),
+                    SpkIterator::new_with_range(descriptor.clone(), 0..),
+                )
+            })
+            .collect()
+    }
+
+    /// Generates a script pubkey iterator for the given `keychain`'s descriptor (if it exists). The
+    /// iterator iterates over all derivable scripts of the keychain's descriptor.
+    ///
+    /// # Panics
+    ///
+    /// This will panic if the `keychain` does not exist.
+    pub fn spks_of_keychain(&self, keychain: &K) -> SpkIterator<Descriptor<DescriptorPublicKey>> {
+        let descriptor = self
+            .keychains
+            .get(keychain)
+            .expect("keychain must exist")
+            .clone();
+        SpkIterator::new_with_range(descriptor, 0..)
+    }
+
+    /// Convenience method to get [`revealed_spks_of_keychain`] of all keychains.
+    ///
+    /// [`revealed_spks_of_keychain`]: Self::revealed_spks_of_keychain
+    pub fn revealed_spks_of_all_keychains(
+        &self,
+    ) -> BTreeMap<K, impl Iterator<Item = (u32, &Script)> + Clone> {
+        self.keychains
+            .keys()
+            .map(|keychain| (keychain.clone(), self.revealed_spks_of_keychain(keychain)))
+            .collect()
+    }
+
+    /// Iterates over the script pubkeys revealed by this index under `keychain`.
+    pub fn revealed_spks_of_keychain(
+        &self,
+        keychain: &K,
+    ) -> impl DoubleEndedIterator<Item = (u32, &Script)> + Clone {
+        let next_index = self.last_revealed.get(keychain).map_or(0, |v| *v + 1);
+        self.inner
+            .all_spks()
+            .range((keychain.clone(), u32::MIN)..(keychain.clone(), next_index))
+            .map(|((_, derivation_index), spk)| (*derivation_index, spk))
+    }
+
+    /// Get the next derivation index for `keychain`. The next index is the index after the last revealed
+    /// derivation index.
+    ///
+    /// The second field in the returned tuple represents whether the next derivation index is new.
+    /// There are two scenarios where the next derivation index is reused (not new):
+    ///
+    /// 1. The keychain's descriptor has no wildcard, and a script has already been revealed.
+    /// 2. The number of revealed scripts has already reached 2^31 (refer to BIP-32).
+    ///
+    /// Not checking the second field of the tuple may result in address reuse.
+    ///
+    /// # Panics
+    ///
+    /// Panics if the `keychain` does not exist.
+    pub fn next_index(&self, keychain: &K) -> (u32, bool) {
+        let descriptor = self.keychains.get(keychain).expect("keychain must exist");
+        let last_index = self.last_revealed.get(keychain).cloned();
+
+        // we can only get the next index if the wildcard exists.
+        let has_wildcard = descriptor.has_wildcard();
+
+        match last_index {
+            // if there is no index, next_index is always 0.
+            None => (0, true),
+            // descriptors without wildcards can only have one index.
+            Some(_) if !has_wildcard => (0, false),
+            // derivation index must be < 2^31 (BIP-32).
+            Some(index) if index > BIP32_MAX_INDEX => {
+                unreachable!("index is out of bounds")
+            }
+            Some(index) if index == BIP32_MAX_INDEX => (index, false),
+            // get the next derivation index.
+            Some(index) => (index + 1, true),
+        }
+    }
+
+    /// Get the last derivation index that is revealed for each keychain.
+    ///
+    /// Keychains with no revealed indices will not be included in the returned [`BTreeMap`].
+    pub fn last_revealed_indices(&self) -> &BTreeMap<K, u32> {
+        &self.last_revealed
+    }
+
+    /// Get the last derivation index revealed for `keychain`.
+    pub fn last_revealed_index(&self, keychain: &K) -> Option<u32> {
+        self.last_revealed.get(keychain).cloned()
+    }
+
+    /// Convenience method to call [`Self::reveal_to_target`] on multiple keychains.
+    pub fn reveal_to_target_multi(
+        &mut self,
+        keychains: &BTreeMap<K, u32>,
+    ) -> (
+        BTreeMap<K, SpkIterator<Descriptor<DescriptorPublicKey>>>,
+        DerivationAdditions<K>,
+    ) {
+        let mut additions = DerivationAdditions::default();
+        let mut spks = BTreeMap::new();
+
+        for (keychain, &index) in keychains {
+            let (new_spks, new_additions) = self.reveal_to_target(keychain, index);
+            if !new_additions.is_empty() {
+                spks.insert(keychain.clone(), new_spks);
+                additions.append(new_additions.clone());
+            }
+        }
+
+        (spks, additions)
+    }
+
+    /// Reveals script pubkeys of the `keychain`'s descriptor **up to and including** the
+    /// `target_index`.
+    ///
+    /// If the `target_index` cannot be reached (due to the descriptor having no wildcard and/or
+    /// the `target_index` is in the hardened index range), this method will make a best-effort and
+    /// reveal up to the last possible index.
+    ///
+    /// This returns an iterator of newly revealed indices (alongside their scripts) and a
+    /// [`DerivationAdditions`], which reports updates to the latest revealed index. If no new script
+    /// pubkeys are revealed, then both of these will be empty.
+    ///
+    /// # Panics
+    ///
+    /// Panics if `keychain` does not exist.
+    pub fn reveal_to_target(
+        &mut self,
+        keychain: &K,
+        target_index: u32,
+    ) -> (
+        SpkIterator<Descriptor<DescriptorPublicKey>>,
+        DerivationAdditions<K>,
+    ) {
+        let descriptor = self.keychains.get(keychain).expect("keychain must exist");
+        let has_wildcard = descriptor.has_wildcard();
+
+        let target_index = if has_wildcard { target_index } else { 0 };
+        let next_reveal_index = self.last_revealed.get(keychain).map_or(0, |v| *v + 1);
+        let lookahead = self.lookahead.get(keychain).map_or(0, |v| *v);
+
+        debug_assert_eq!(
+            next_reveal_index + lookahead,
+            self.next_store_index(keychain)
+        );
+
+        // if we need to reveal new indices, the latest revealed index goes here
+        let mut reveal_to_index = None;
+
+        // if the target is not yet revealed, but is already stored (due to lookahead), we need to
+        // set the `reveal_to_index` as target here (as the `for` loop below only updates
+        // `reveal_to_index` for indexes that are NOT stored)
+        if next_reveal_index <= target_index && target_index < next_reveal_index + lookahead {
+            reveal_to_index = Some(target_index);
+        }
+
+        // we range over indexes that are not stored
+        let range = next_reveal_index + lookahead..=target_index + lookahead;
+        for (new_index, new_spk) in SpkIterator::new_with_range(descriptor, range) {
+            let _inserted = self
+                .inner
+                .insert_spk((keychain.clone(), new_index), new_spk);
+            debug_assert!(_inserted, "must not have existing spk",);
+
+            // everything after `target_index` is stored for lookahead only
+            if new_index <= target_index {
+                reveal_to_index = Some(new_index);
+            }
+        }
+
+        match reveal_to_index {
+            Some(index) => {
+                let _old_index = self.last_revealed.insert(keychain.clone(), index);
+                debug_assert!(_old_index < Some(index));
+                (
+                    SpkIterator::new_with_range(descriptor.clone(), next_reveal_index..index + 1),
+                    DerivationAdditions(core::iter::once((keychain.clone(), index)).collect()),
+                )
+            }
+            None => (
+                SpkIterator::new_with_range(
+                    descriptor.clone(),
+                    next_reveal_index..next_reveal_index,
+                ),
+                DerivationAdditions::default(),
+            ),
+        }
+    }
+
+    /// Attempts to reveal the next script pubkey for `keychain`.
+    ///
+    /// Returns the derivation index of the revealed script pubkey, the revealed script pubkey and a
+    /// [`DerivationAdditions`] which represents changes in the last revealed index (if any).
+    ///
+    /// When a new script cannot be revealed, we return the last revealed script and an empty
+    /// [`DerivationAdditions`]. There are two scenarios when a new script pubkey cannot be derived:
+    ///
+    ///  1. The descriptor has no wildcard and already has one script revealed.
+    ///  2. The descriptor has already revealed scripts up to the numeric bound.
+    ///
+    /// # Panics
+    ///
+    /// Panics if the `keychain` does not exist.
+    pub fn reveal_next_spk(&mut self, keychain: &K) -> ((u32, &Script), DerivationAdditions<K>) {
+        let (next_index, _) = self.next_index(keychain);
+        let additions = self.reveal_to_target(keychain, next_index).1;
+        let script = self
+            .inner
+            .spk_at_index(&(keychain.clone(), next_index))
+            .expect("script must already be stored");
+        ((next_index, script), additions)
+    }
+
+    /// Gets the next unused script pubkey in the keychain. I.e., the script pubkey with the lowest
+    /// index that has not been used yet.
+    ///
+    /// This will derive and reveal a new script pubkey if no more unused script pubkeys exist.
+    ///
+    /// If the descriptor has no wildcard and already has a used script pubkey or if a descriptor
+    /// has used all scripts up to the derivation bounds, then the last derived script pubkey will be
+    /// returned.
+    ///
+    /// # Panics
+    ///
+    /// Panics if `keychain` has never been added to the index
+    pub fn next_unused_spk(&mut self, keychain: &K) -> ((u32, &Script), DerivationAdditions<K>) {
+        let need_new = self.unused_spks_of_keychain(keychain).next().is_none();
+        // this rather strange branch is needed because of some lifetime issues
+        if need_new {
+            self.reveal_next_spk(keychain)
+        } else {
+            (
+                self.unused_spks_of_keychain(keychain)
+                    .next()
+                    .expect("we already know next exists"),
+                DerivationAdditions::default(),
+            )
+        }
+    }
+
+    /// Marks the script pubkey at `index` as used even though the tracker hasn't seen an output with it.
+    /// This only has an effect when the `index` had been added to `self` already and was unused.
+    ///
+    /// Returns whether the `index` was initially present as `unused`.
+    ///
+    /// This is useful when you want to reserve a script pubkey for something but don't want to add
+    /// the transaction output using it to the index yet. Other callers will consider `index` on
+    /// `keychain` used until you call [`unmark_used`].
+    ///
+    /// [`unmark_used`]: Self::unmark_used
+    pub fn mark_used(&mut self, keychain: &K, index: u32) -> bool {
+        self.inner.mark_used(&(keychain.clone(), index))
+    }
+
+    /// Undoes the effect of [`mark_used`]. Returns whether the `index` is inserted back into
+    /// `unused`.
+    ///
+    /// Note that if `self` has scanned an output with this script pubkey, then this will have no
+    /// effect.
+    ///
+    /// [`mark_used`]: Self::mark_used
+    pub fn unmark_used(&mut self, keychain: &K, index: u32) -> bool {
+        self.inner.unmark_used(&(keychain.clone(), index))
+    }
+
+    /// Iterates over all unused script pubkeys for a `keychain` stored in the index.
+    pub fn unused_spks_of_keychain(
+        &self,
+        keychain: &K,
+    ) -> impl DoubleEndedIterator<Item = (u32, &Script)> {
+        let next_index = self.last_revealed.get(keychain).map_or(0, |&v| v + 1);
+        let range = (keychain.clone(), u32::MIN)..(keychain.clone(), next_index);
+        self.inner
+            .unused_spks(range)
+            .map(|((_, i), script)| (*i, script))
+    }
+
+    /// Iterates over all the [`OutPoint`] that have a `TxOut` with a script pubkey derived from
+    /// `keychain`.
+    pub fn txouts_of_keychain(
+        &self,
+        keychain: &K,
+    ) -> impl DoubleEndedIterator<Item = (u32, OutPoint)> + '_ {
+        self.inner
+            .outputs_in_range((keychain.clone(), u32::MIN)..(keychain.clone(), u32::MAX))
+            .map(|((_, i), op)| (*i, op))
+    }
+
+    /// Returns the highest derivation index of the `keychain` where [`KeychainTxOutIndex`] has
+    /// found a [`TxOut`] with it's script pubkey.
+    pub fn last_used_index(&self, keychain: &K) -> Option<u32> {
+        self.txouts_of_keychain(keychain).last().map(|(i, _)| i)
+    }
+
+    /// Returns the highest derivation index of each keychain that [`KeychainTxOutIndex`] has found
+    /// a [`TxOut`] with it's script pubkey.
+    pub fn last_used_indices(&self) -> BTreeMap<K, u32> {
+        self.keychains
+            .iter()
+            .filter_map(|(keychain, _)| {
+                self.last_used_index(keychain)
+                    .map(|index| (keychain.clone(), index))
+            })
+            .collect()
+    }
+
+    /// Applies the derivation additions to the [`KeychainTxOutIndex`], extending the number of
+    /// derived scripts per keychain, as specified in the `additions`.
+    pub fn apply_additions(&mut self, additions: DerivationAdditions<K>) {
+        let _ = self.reveal_to_target_multi(&additions.0);
+    }
+}

crates/chain/src/lib.rs

@@ -0,0 +1,102 @@
+//! This crate is a collection of core structures for [Bitcoin Dev Kit] (alpha release).
+//!
+//! The goal of this crate is to give wallets the mechanisms needed to:
+//!
+//! 1. Figure out what data they need to fetch.
+//! 2. Process the data in a way that never leads to inconsistent states.
+//! 3. Fully index that data and expose it to be consumed without friction.
+//!
+//! Our design goals for these mechanisms are:
+//!
+//! 1. Data source agnostic -- nothing in `bdk_chain` cares about where you get data from or whether
+//!    you do it synchronously or asynchronously. If you know a fact about the blockchain, you can just
+//!    tell `bdk_chain`'s APIs about it, and that information will be integrated, if it can be done
+//!    consistently.
+//! 2. Error-free APIs.
+//! 3. Data persistence agnostic -- `bdk_chain` does not care where you cache on-chain data, what you
+//!    cache or how you fetch it.
+//!
+//! [Bitcoin Dev Kit]: https://bitcoindevkit.org/
+
+#![no_std]
+#![warn(missing_docs)]
+
+pub use bitcoin;
+mod spk_txout_index;
+pub use spk_txout_index::*;
+mod chain_data;
+pub use chain_data::*;
+pub mod indexed_tx_graph;
+pub use indexed_tx_graph::IndexedTxGraph;
+pub mod keychain;
+pub mod local_chain;
+mod tx_data_traits;
+pub mod tx_graph;
+pub use tx_data_traits::*;
+pub use tx_graph::TxGraph;
+mod chain_oracle;
+pub use chain_oracle::*;
+mod persist;
+pub use persist::*;
+
+#[doc(hidden)]
+pub mod example_utils;
+
+#[cfg(feature = "miniscript")]
+pub use miniscript;
+#[cfg(feature = "miniscript")]
+mod descriptor_ext;
+#[cfg(feature = "miniscript")]
+pub use descriptor_ext::DescriptorExt;
+#[cfg(feature = "miniscript")]
+mod spk_iter;
+#[cfg(feature = "miniscript")]
+pub use spk_iter::*;
+
+#[allow(unused_imports)]
+#[macro_use]
+extern crate alloc;
+
+#[cfg(feature = "serde")]
+pub extern crate serde_crate as serde;
+
+#[cfg(feature = "bincode")]
+extern crate bincode;
+
+#[cfg(feature = "std")]
+#[macro_use]
+extern crate std;
+
+#[cfg(all(not(feature = "std"), feature = "hashbrown"))]
+extern crate hashbrown;
+
+// When no-std use `alloc`'s Hash collections. This is activated by default
+#[cfg(all(not(feature = "std"), not(feature = "hashbrown")))]
+#[doc(hidden)]
+pub mod collections {
+    #![allow(dead_code)]
+    pub type HashSet<K> = alloc::collections::BTreeSet<K>;
+    pub type HashMap<K, V> = alloc::collections::BTreeMap<K, V>;
+    pub use alloc::collections::{btree_map as hash_map, *};
+}
+
+// When we have std, use `std`'s all collections
+#[cfg(all(feature = "std", not(feature = "hashbrown")))]
+#[doc(hidden)]
+pub mod collections {
+    pub use std::collections::{hash_map, *};
+}
+
+// With this special feature `hashbrown`, use `hashbrown`'s hash collections, and else from `alloc`.
+#[cfg(feature = "hashbrown")]
+#[doc(hidden)]
+pub mod collections {
+    #![allow(dead_code)]
+    pub type HashSet<K> = hashbrown::HashSet<K>;
+    pub type HashMap<K, V> = hashbrown::HashMap<K, V>;
+    pub use alloc::collections::*;
+    pub use hashbrown::hash_map;
+}
+
+/// How many confirmations are needed f or a coinbase output to be spent.
+pub const COINBASE_MATURITY: u32 = 100;

crates/chain/src/local_chain.rs

@@ -0,0 +1,250 @@
+//! The [`LocalChain`] is a local implementation of [`ChainOracle`].
+
+use core::convert::Infallible;
+
+use alloc::collections::BTreeMap;
+use bitcoin::BlockHash;
+
+use crate::{BlockId, ChainOracle};
+
+/// This is a local implementation of [`ChainOracle`].
+#[derive(Debug, Default, Clone, PartialEq, Eq, PartialOrd, Ord)]
+pub struct LocalChain {
+    blocks: BTreeMap<u32, BlockHash>,
+}
+
+impl ChainOracle for LocalChain {
+    type Error = Infallible;
+
+    fn is_block_in_chain(
+        &self,
+        block: BlockId,
+        static_block: BlockId,
+    ) -> Result<Option<bool>, Self::Error> {
+        if block.height > static_block.height {
+            return Ok(None);
+        }
+        Ok(
+            match (
+                self.blocks.get(&block.height),
+                self.blocks.get(&static_block.height),
+            ) {
+                (Some(&hash), Some(&static_hash)) => {
+                    Some(hash == block.hash && static_hash == static_block.hash)
+                }
+                _ => None,
+            },
+        )
+    }
+
+    fn get_chain_tip(&self) -> Result<Option<BlockId>, Self::Error> {
+        Ok(self.tip())
+    }
+}
+
+impl AsRef<BTreeMap<u32, BlockHash>> for LocalChain {
+    fn as_ref(&self) -> &BTreeMap<u32, BlockHash> {
+        &self.blocks
+    }
+}
+
+impl From<LocalChain> for BTreeMap<u32, BlockHash> {
+    fn from(value: LocalChain) -> Self {
+        value.blocks
+    }
+}
+
+impl From<BTreeMap<u32, BlockHash>> for LocalChain {
+    fn from(value: BTreeMap<u32, BlockHash>) -> Self {
+        Self { blocks: value }
+    }
+}
+
+impl LocalChain {
+    /// Contruct a [`LocalChain`] from a list of [`BlockId`]s.
+    pub fn from_blocks<B>(blocks: B) -> Self
+    where
+        B: IntoIterator<Item = BlockId>,
+    {
+        Self {
+            blocks: blocks.into_iter().map(|b| (b.height, b.hash)).collect(),
+        }
+    }
+
+    /// Get a reference to a map of block height to hash.
+    pub fn blocks(&self) -> &BTreeMap<u32, BlockHash> {
+        &self.blocks
+    }
+
+    /// Get the chain tip.
+    pub fn tip(&self) -> Option<BlockId> {
+        self.blocks
+            .iter()
+            .last()
+            .map(|(&height, &hash)| BlockId { height, hash })
+    }
+
+    /// This is like the sparsechain's logic, expect we must guarantee that all invalidated heights
+    /// are to be re-filled.
+    pub fn determine_changeset(&self, update: &Self) -> Result<ChangeSet, UpdateNotConnectedError> {
+        let update = update.as_ref();
+        let update_tip = match update.keys().last().cloned() {
+            Some(tip) => tip,
+            None => return Ok(ChangeSet::default()),
+        };
+
+        // this is the latest height where both the update and local chain has the same block hash
+        let agreement_height = update
+            .iter()
+            .rev()
+            .find(|&(u_height, u_hash)| self.blocks.get(u_height) == Some(u_hash))
+            .map(|(&height, _)| height);
+
+        // the lower bound of the range to invalidate
+        let invalidate_lb = match agreement_height {
+            Some(height) if height == update_tip => u32::MAX,
+            Some(height) => height + 1,
+            None => 0,
+        };
+
+        // the first block's height to invalidate in the local chain
+        let invalidate_from_height = self.blocks.range(invalidate_lb..).next().map(|(&h, _)| h);
+
+        // the first block of height to invalidate (if any) should be represented in the update
+        if let Some(first_invalid_height) = invalidate_from_height {
+            if !update.contains_key(&first_invalid_height) {
+                return Err(UpdateNotConnectedError(first_invalid_height));
+            }
+        }
+
+        let mut changeset: BTreeMap<u32, Option<BlockHash>> = match invalidate_from_height {
+            Some(first_invalid_height) => {
+                // the first block of height to invalidate should be represented in the update
+                if !update.contains_key(&first_invalid_height) {
+                    return Err(UpdateNotConnectedError(first_invalid_height));
+                }
+                self.blocks
+                    .range(first_invalid_height..)
+                    .map(|(height, _)| (*height, None))
+                    .collect()
+            }
+            None => BTreeMap::new(),
+        };
+        for (height, update_hash) in update {
+            let original_hash = self.blocks.get(height);
+            if Some(update_hash) != original_hash {
+                changeset.insert(*height, Some(*update_hash));
+            }
+        }
+
+        Ok(changeset)
+    }
+
+    /// Applies the given `changeset`.
+    pub fn apply_changeset(&mut self, changeset: ChangeSet) {
+        for (height, blockhash) in changeset {
+            match blockhash {
+                Some(blockhash) => self.blocks.insert(height, blockhash),
+                None => self.blocks.remove(&height),
+            };
+        }
+    }
+
+    /// Updates [`LocalChain`] with an update [`LocalChain`].
+    ///
+    /// This is equivalent to calling [`determine_changeset`] and [`apply_changeset`] in sequence.
+    ///
+    /// [`determine_changeset`]: Self::determine_changeset
+    /// [`apply_changeset`]: Self::apply_changeset
+    pub fn apply_update(&mut self, update: Self) -> Result<ChangeSet, UpdateNotConnectedError> {
+        let changeset = self.determine_changeset(&update)?;
+        self.apply_changeset(changeset.clone());
+        Ok(changeset)
+    }
+
+    /// Derives a [`ChangeSet`] that assumes that there are no preceding changesets.
+    ///
+    /// The changeset returned will record additions of all blocks included in [`Self`].
+    pub fn initial_changeset(&self) -> ChangeSet {
+        self.blocks
+            .iter()
+            .map(|(&height, &hash)| (height, Some(hash)))
+            .collect()
+    }
+
+    /// Insert a block of [`BlockId`] into the [`LocalChain`].
+    ///
+    /// # Error
+    ///
+    /// If the insertion height already contains a block, and the block has a different blockhash,
+    /// this will result in an [`InsertBlockNotMatchingError`].
+    pub fn insert_block(
+        &mut self,
+        block_id: BlockId,
+    ) -> Result<ChangeSet, InsertBlockNotMatchingError> {
+        let mut update = Self::from_blocks(self.tip());
+
+        if let Some(original_hash) = update.blocks.insert(block_id.height, block_id.hash) {
+            if original_hash != block_id.hash {
+                return Err(InsertBlockNotMatchingError {
+                    height: block_id.height,
+                    original_hash,
+                    update_hash: block_id.hash,
+                });
+            }
+        }
+
+        Ok(self.apply_update(update).expect("should always connect"))
+    }
+}
+
+/// This is the return value of [`determine_changeset`] and represents changes to [`LocalChain`].
+///
+/// [`determine_changeset`]: LocalChain::determine_changeset
+pub type ChangeSet = BTreeMap<u32, Option<BlockHash>>;
+
+/// Represents an update failure of [`LocalChain`] due to the update not connecting to the original
+/// chain.
+///
+/// The update cannot be applied to the chain because the chain suffix it represents did not
+/// connect to the existing chain. This error case contains the checkpoint height to include so
+/// that the chains can connect.
+#[derive(Clone, Debug, PartialEq)]
+pub struct UpdateNotConnectedError(pub u32);
+
+impl core::fmt::Display for UpdateNotConnectedError {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        write!(
+            f,
+            "the update cannot connect with the chain, try include block at height {}",
+            self.0
+        )
+    }
+}
+
+#[cfg(feature = "std")]
+impl std::error::Error for UpdateNotConnectedError {}
+
+/// Represents a failure when trying to insert a checkpoint into [`LocalChain`].
+#[derive(Clone, Debug, PartialEq)]
+pub struct InsertBlockNotMatchingError {
+    /// The checkpoints' height.
+    pub height: u32,
+    /// Original checkpoint's block hash.
+    pub original_hash: BlockHash,
+    /// Update checkpoint's block hash.
+    pub update_hash: BlockHash,
+}
+
+impl core::fmt::Display for InsertBlockNotMatchingError {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        write!(
+            f,
+            "failed to insert block at height {} as blockhashes conflict: original={}, update={}",
+            self.height, self.original_hash, self.update_hash
+        )
+    }
+}
+
+#[cfg(feature = "std")]
+impl std::error::Error for InsertBlockNotMatchingError {}

crates/chain/src/persist.rs

@@ -0,0 +1,97 @@
+use core::convert::Infallible;
+
+use crate::Append;
+
+/// `Persist` wraps a [`PersistBackend`] (`B`) to create a convenient staging area for changes (`C`)
+/// before they are persisted.
+///
+/// Not all changes to the in-memory representation needs to be written to disk right away, so
+/// [`Persist::stage`] can be used to *stage* changes first and then [`Persist::commit`] can be used
+/// to write changes to disk.
+#[derive(Debug)]
+pub struct Persist<B, C> {
+    backend: B,
+    stage: C,
+}
+
+impl<B, C> Persist<B, C>
+where
+    B: PersistBackend<C>,
+    C: Default + Append,
+{
+    /// Create a new [`Persist`] from [`PersistBackend`].
+    pub fn new(backend: B) -> Self {
+        Self {
+            backend,
+            stage: Default::default(),
+        }
+    }
+
+    /// Stage a `changeset` to be commited later with [`commit`].
+    ///
+    /// [`commit`]: Self::commit
+    pub fn stage(&mut self, changeset: C) {
+        self.stage.append(changeset)
+    }
+
+    /// Get the changes that have not been commited yet.
+    pub fn staged(&self) -> &C {
+        &self.stage
+    }
+
+    /// Commit the staged changes to the underlying persistance backend.
+    ///
+    /// Changes that are committed (if any) are returned.
+    ///
+    /// # Error
+    ///
+    /// Returns a backend-defined error if this fails.
+    pub fn commit(&mut self) -> Result<Option<C>, B::WriteError> {
+        if self.stage.is_empty() {
+            return Ok(None);
+        }
+        self.backend
+            .write_changes(&self.stage)
+            // if written successfully, take and return `self.stage`
+            .map(|_| Some(core::mem::take(&mut self.stage)))
+    }
+}
+
+/// A persistence backend for [`Persist`].
+///
+/// `C` represents the changeset; a datatype that records changes made to in-memory data structures
+/// that are to be persisted, or retrieved from persistence.
+pub trait PersistBackend<C> {
+    /// The error the backend returns when it fails to write.
+    type WriteError: core::fmt::Debug;
+
+    /// The error the backend returns when it fails to load changesets `C`.
+    type LoadError: core::fmt::Debug;
+
+    /// Writes a changeset to the persistence backend.
+    ///
+    /// It is up to the backend what it does with this. It could store every changeset in a list or
+    /// it inserts the actual changes into a more structured database. All it needs to guarantee is
+    /// that [`load_from_persistence`] restores a keychain tracker to what it should be if all
+    /// changesets had been applied sequentially.
+    ///
+    /// [`load_from_persistence`]: Self::load_from_persistence
+    fn write_changes(&mut self, changeset: &C) -> Result<(), Self::WriteError>;
+
+    /// Return the aggregate changeset `C` from persistence.
+    fn load_from_persistence(&mut self) -> Result<C, Self::LoadError>;
+}
+
+impl<C: Default> PersistBackend<C> for () {
+    type WriteError = Infallible;
+
+    type LoadError = Infallible;
+
+    fn write_changes(&mut self, _changeset: &C) -> Result<(), Self::WriteError> {
+        Ok(())
+    }
+
+    fn load_from_persistence(&mut self) -> Result<C, Self::LoadError> {
+        Ok(C::default())
+    }
+}

crates/chain/src/spk_iter.rs

@@ -0,0 +1,215 @@
+use crate::{
+    bitcoin::{secp256k1::Secp256k1, Script},
+    miniscript::{Descriptor, DescriptorPublicKey},
+};
+use core::{borrow::Borrow, ops::Bound, ops::RangeBounds};
+
+/// Maximum [BIP32](https://bips.xyz/32) derivation index.
+pub const BIP32_MAX_INDEX: u32 = (1 << 31) - 1;
+
+/// An iterator for derived script pubkeys.
+///
+/// [`SpkIterator`] is an implementation of the [`Iterator`] trait which possesses its own `next()`
+/// and `nth()` functions, both of which circumvent the unnecessary intermediate derivations required
+/// when using their default implementations.
+///
+/// ## Examples
+///
+/// ```
+/// use bdk_chain::SpkIterator;
+/// # use miniscript::{Descriptor, DescriptorPublicKey};
+/// # use bitcoin::{secp256k1::Secp256k1};
+/// # use std::str::FromStr;
+/// # let secp = bitcoin::secp256k1::Secp256k1::signing_only();
+/// # let (descriptor, _) = Descriptor::<DescriptorPublicKey>::parse_descriptor(&secp, "wpkh([73c5da0a/86'/0'/0']xprv9xgqHN7yz9MwCkxsBPN5qetuNdQSUttZNKw1dcYTV4mkaAFiBVGQziHs3NRSWMkCzvgjEe3n9xV8oYywvM8at9yRqyaZVz6TYYhX98VjsUk/1/0)").unwrap();
+/// # let external_spk_0 = descriptor.at_derivation_index(0).script_pubkey();
+/// # let external_spk_3 = descriptor.at_derivation_index(3).script_pubkey();
+/// # let external_spk_4 = descriptor.at_derivation_index(4).script_pubkey();
+///
+/// // Creates a new script pubkey iterator starting at 0 from a descriptor.
+/// let mut spk_iter = SpkIterator::new(&descriptor);
+/// assert_eq!(spk_iter.next(), Some((0, external_spk_0)));
+/// assert_eq!(spk_iter.next(), None);
+/// ```
+#[derive(Clone)]
+pub struct SpkIterator<D> {
+    next_index: u32,
+    end: u32,
+    descriptor: D,
+    secp: Secp256k1<bitcoin::secp256k1::VerifyOnly>,
+}
+
+impl<D> SpkIterator<D>
+where
+    D: Borrow<Descriptor<DescriptorPublicKey>>,
+{
+    /// Creates a new script pubkey iterator starting at 0 from a descriptor.
+    pub fn new(descriptor: D) -> Self {
+        let end = if descriptor.borrow().has_wildcard() {
+            BIP32_MAX_INDEX
+        } else {
+            0
+        };
+
+        SpkIterator::new_with_range(descriptor, 0..=end)
+    }
+
+    // Creates a new script pubkey iterator from a descriptor with a given range.
+    pub(crate) fn new_with_range<R>(descriptor: D, range: R) -> Self
+    where
+        R: RangeBounds<u32>,
+    {
+        let mut end = match range.end_bound() {
+            Bound::Included(end) => *end + 1,
+            Bound::Excluded(end) => *end,
+            Bound::Unbounded => u32::MAX,
+        };
+        // Because `end` is exclusive, we want the maximum value to be BIP32_MAX_INDEX + 1.
+        end = end.min(BIP32_MAX_INDEX + 1);
+
+        Self {
+            next_index: match range.start_bound() {
+                Bound::Included(start) => *start,
+                Bound::Excluded(start) => *start + 1,
+                Bound::Unbounded => u32::MIN,
+            },
+            end,
+            descriptor,
+            secp: Secp256k1::verification_only(),
+        }
+    }
+}
+
+impl<D> Iterator for SpkIterator<D>
+where
+    D: Borrow<Descriptor<DescriptorPublicKey>>,
+{
+    type Item = (u32, Script);
+
+    fn next(&mut self) -> Option<Self::Item> {
+        // For non-wildcard descriptors, we expect the first element to be Some((0, spk)), then None after.
+        // For wildcard descriptors, we expect it to keep iterating until exhausted.
+        if self.next_index >= self.end {
+            return None;
+        }
+
+        let script = self
+            .descriptor
+            .borrow()
+            .at_derivation_index(self.next_index)
+            .derived_descriptor(&self.secp)
+            .expect("the descriptor cannot need hardened derivation")
+            .script_pubkey();
+        let output = (self.next_index, script);
+
+        self.next_index += 1;
+
+        Some(output)
+    }
+
+    fn nth(&mut self, n: usize) -> Option<Self::Item> {
+        self.next_index = self
+            .next_index
+            .saturating_add(u32::try_from(n).unwrap_or(u32::MAX));
+        self.next()
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use crate::{
+        bitcoin::secp256k1::Secp256k1,
+        keychain::KeychainTxOutIndex,
+        miniscript::{Descriptor, DescriptorPublicKey},
+        spk_iter::{SpkIterator, BIP32_MAX_INDEX},
+    };
+
+    #[derive(Clone, Debug, PartialEq, Eq, Ord, PartialOrd)]
+    enum TestKeychain {
+        External,
+        Internal,
+    }
+
+    fn init_txout_index() -> (
+        KeychainTxOutIndex<TestKeychain>,
+        Descriptor<DescriptorPublicKey>,
+        Descriptor<DescriptorPublicKey>,
+    ) {
+        let mut txout_index = KeychainTxOutIndex::<TestKeychain>::default();
+
+        let secp = Secp256k1::signing_only();
+        let (external_descriptor,_) = Descriptor::<DescriptorPublicKey>::parse_descriptor(&secp, "tr([73c5da0a/86'/0'/0']xprv9xgqHN7yz9MwCkxsBPN5qetuNdQSUttZNKw1dcYTV4mkaAFiBVGQziHs3NRSWMkCzvgjEe3n9xV8oYywvM8at9yRqyaZVz6TYYhX98VjsUk/0/*)").unwrap();
+        let (internal_descriptor,_) = Descriptor::<DescriptorPublicKey>::parse_descriptor(&secp, "tr([73c5da0a/86'/0'/0']xprv9xgqHN7yz9MwCkxsBPN5qetuNdQSUttZNKw1dcYTV4mkaAFiBVGQziHs3NRSWMkCzvgjEe3n9xV8oYywvM8at9yRqyaZVz6TYYhX98VjsUk/1/*)").unwrap();
+
+        txout_index.add_keychain(TestKeychain::External, external_descriptor.clone());
+        txout_index.add_keychain(TestKeychain::Internal, internal_descriptor.clone());
+
+        (txout_index, external_descriptor, internal_descriptor)
+    }
+
+    #[test]
+    #[allow(clippy::iter_nth_zero)]
+    fn test_spkiterator_wildcard() {
+        let (_, external_desc, _) = init_txout_index();
+        let external_spk_0 = external_desc.at_derivation_index(0).script_pubkey();
+        let external_spk_16 = external_desc.at_derivation_index(16).script_pubkey();
+        let external_spk_20 = external_desc.at_derivation_index(20).script_pubkey();
+        let external_spk_21 = external_desc.at_derivation_index(21).script_pubkey();
+        let external_spk_max = external_desc
+            .at_derivation_index(BIP32_MAX_INDEX)
+            .script_pubkey();
+
+        let mut external_spk = SpkIterator::new(&external_desc);
+        let max_index = BIP32_MAX_INDEX - 22;
+
+        assert_eq!(external_spk.next().unwrap(), (0, external_spk_0));
+        assert_eq!(external_spk.nth(15).unwrap(), (16, external_spk_16));
+        assert_eq!(external_spk.nth(3).unwrap(), (20, external_spk_20.clone()));
+        assert_eq!(external_spk.next().unwrap(), (21, external_spk_21));
+        assert_eq!(
+            external_spk.nth(max_index as usize).unwrap(),
+            (BIP32_MAX_INDEX, external_spk_max)
+        );
+        assert_eq!(external_spk.nth(0), None);
+
+        let mut external_spk = SpkIterator::new_with_range(&external_desc, 0..21);
+        assert_eq!(external_spk.nth(20).unwrap(), (20, external_spk_20));
+        assert_eq!(external_spk.next(), None);
+
+        let mut external_spk = SpkIterator::new_with_range(&external_desc, 0..21);
+        assert_eq!(external_spk.nth(21), None);
+    }
+
+    #[test]
+    #[allow(clippy::iter_nth_zero)]
+    fn test_spkiterator_non_wildcard() {
+        let secp = bitcoin::secp256k1::Secp256k1::signing_only();
+        let (no_wildcard_descriptor, _) = Descriptor::<DescriptorPublicKey>::parse_descriptor(&secp, "wpkh([73c5da0a/86'/0'/0']xprv9xgqHN7yz9MwCkxsBPN5qetuNdQSUttZNKw1dcYTV4mkaAFiBVGQziHs3NRSWMkCzvgjEe3n9xV8oYywvM8at9yRqyaZVz6TYYhX98VjsUk/1/0)").unwrap();
+        let external_spk_0 = no_wildcard_descriptor
+            .at_derivation_index(0)
+            .script_pubkey();
+
+        let mut external_spk = SpkIterator::new(&no_wildcard_descriptor);
+
+        assert_eq!(external_spk.next().unwrap(), (0, external_spk_0.clone()));
+        assert_eq!(external_spk.next(), None);
+
+        let mut external_spk = SpkIterator::new(&no_wildcard_descriptor);
+
+        assert_eq!(external_spk.nth(0).unwrap(), (0, external_spk_0));
+        assert_eq!(external_spk.nth(0), None);
+    }
+
+    // The following dummy traits were created to test if SpkIterator is working properly.
+    trait TestSendStatic: Send + 'static {
+        fn test(&self) -> u32 {
+            20
+        }
+    }
+
+    impl TestSendStatic for SpkIterator<Descriptor<DescriptorPublicKey>> {
+        fn test(&self) -> u32 {
+            20
+        }
+    }
+}

crates/chain/src/spk_txout_index.rs

@@ -0,0 +1,337 @@
+use core::ops::RangeBounds;
+
+use crate::{
+    collections::{hash_map::Entry, BTreeMap, BTreeSet, HashMap},
+    indexed_tx_graph::Indexer,
+    ForEachTxOut,
+};
+use bitcoin::{self, OutPoint, Script, Transaction, TxOut, Txid};
+
+/// An index storing [`TxOut`]s that have a script pubkey that matches those in a list.
+///
+/// The basic idea is that you insert script pubkeys you care about into the index with
+/// [`insert_spk`] and then when you call [`scan`], the index will look at any txouts you pass in and
+/// store and index any txouts matching one of its script pubkeys.
+///
+/// Each script pubkey is associated with an application-defined index script index `I`, which must be
+/// [`Ord`]. Usually, this is used to associate the derivation index of the script pubkey or even a
+/// combination of `(keychain, derivation_index)`.
+///
+/// Note there is no harm in scanning transactions that disappear from the blockchain or were never
+/// in there in the first place. `SpkTxOutIndex` is intentionally *monotone* -- you cannot delete or
+/// modify txouts that have been indexed. To find out which txouts from the index are actually in the
+/// chain or unspent, you must use other sources of information like a [`TxGraph`].
+///
+/// [`TxOut`]: bitcoin::TxOut
+/// [`insert_spk`]: Self::insert_spk
+/// [`Ord`]: core::cmp::Ord
+/// [`scan`]: Self::scan
+/// [`TxGraph`]: crate::tx_graph::TxGraph
+#[derive(Clone, Debug)]
+pub struct SpkTxOutIndex<I> {
+    /// script pubkeys ordered by index
+    spks: BTreeMap<I, Script>,
+    /// A reverse lookup from spk to spk index
+    spk_indices: HashMap<Script, I>,
+    /// The set of unused indexes.
+    unused: BTreeSet<I>,
+    /// Lookup index and txout by outpoint.
+    txouts: BTreeMap<OutPoint, (I, TxOut)>,
+    /// Lookup from spk index to outpoints that had that spk
+    spk_txouts: BTreeSet<(I, OutPoint)>,
+}
+
+impl<I> Default for SpkTxOutIndex<I> {
+    fn default() -> Self {
+        Self {
+            txouts: Default::default(),
+            spks: Default::default(),
+            spk_indices: Default::default(),
+            spk_txouts: Default::default(),
+            unused: Default::default(),
+        }
+    }
+}
+
+impl<I: Clone + Ord> Indexer for SpkTxOutIndex<I> {
+    type Additions = ();
+
+    fn index_txout(&mut self, outpoint: OutPoint, txout: &TxOut) -> Self::Additions {
+        self.scan_txout(outpoint, txout);
+        Default::default()
+    }
+
+    fn index_tx(&mut self, tx: &Transaction) -> Self::Additions {
+        self.scan(tx);
+        Default::default()
+    }
+
+    fn apply_additions(&mut self, _additions: Self::Additions) {
+        // This applies nothing.
+    }
+
+    fn is_tx_relevant(&self, tx: &Transaction) -> bool {
+        self.is_relevant(tx)
+    }
+}
+
+/// This macro is used instead of a member function of `SpkTxOutIndex`, which would result in a
+/// compiler error[E0521]: "borrowed data escapes out of closure" when we attempt to take a
+/// reference out of the `ForEachTxOut` closure during scanning.
+macro_rules! scan_txout {
+    ($self:ident, $op:expr, $txout:expr) => {{
+        let spk_i = $self.spk_indices.get(&$txout.script_pubkey);
+        if let Some(spk_i) = spk_i {
+            $self.txouts.insert($op, (spk_i.clone(), $txout.clone()));
+            $self.spk_txouts.insert((spk_i.clone(), $op));
+            $self.unused.remove(&spk_i);
+        }
+        spk_i
+    }};
+}
+
+impl<I: Clone + Ord> SpkTxOutIndex<I> {
+    /// Scans an object containing many txouts.
+    ///
+    /// Typically, this is used in two situations:
+    ///
+    /// 1. After loading transaction data from the disk, you may scan over all the txouts to restore all
+    /// your txouts.
+    /// 2. When getting new data from the chain, you usually scan it before incorporating it into your chain state.
+    ///
+    /// See [`ForEachTxout`] for the types that support this.
+    ///
+    /// [`ForEachTxout`]: crate::ForEachTxOut
+    pub fn scan(&mut self, txouts: &impl ForEachTxOut) -> BTreeSet<I> {
+        let mut scanned_indices = BTreeSet::new();
+
+        txouts.for_each_txout(|(op, txout)| {
+            if let Some(spk_i) = scan_txout!(self, op, txout) {
+                scanned_indices.insert(spk_i.clone());
+            }
+        });
+
+        scanned_indices
+    }
+
+    /// Scan a single `TxOut` for a matching script pubkey and returns the index that matches the
+    /// script pubkey (if any).
+    pub fn scan_txout(&mut self, op: OutPoint, txout: &TxOut) -> Option<&I> {
+        scan_txout!(self, op, txout)
+    }
+
+    /// Get a reference to the set of indexed outpoints.
+    pub fn outpoints(&self) -> &BTreeSet<(I, OutPoint)> {
+        &self.spk_txouts
+    }
+
+    /// Iterate over all known txouts that spend to tracked script pubkeys.
+    pub fn txouts(
+        &self,
+    ) -> impl DoubleEndedIterator<Item = (&I, OutPoint, &TxOut)> + ExactSizeIterator {
+        self.txouts
+            .iter()
+            .map(|(op, (index, txout))| (index, *op, txout))
+    }
+
+    /// Finds all txouts on a transaction that has previously been scanned and indexed.
+    pub fn txouts_in_tx(
+        &self,
+        txid: Txid,
+    ) -> impl DoubleEndedIterator<Item = (&I, OutPoint, &TxOut)> {
+        self.txouts
+            .range(OutPoint::new(txid, u32::MIN)..=OutPoint::new(txid, u32::MAX))
+            .map(|(op, (index, txout))| (index, *op, txout))
+    }
+
+    /// Iterates over all the outputs with script pubkeys in an index range.
+    pub fn outputs_in_range(
+        &self,
+        range: impl RangeBounds<I>,
+    ) -> impl DoubleEndedIterator<Item = (&I, OutPoint)> {
+        use bitcoin::hashes::Hash;
+        use core::ops::Bound::*;
+        let min_op = OutPoint {
+            txid: Txid::from_inner([0x00; 32]),
+            vout: u32::MIN,
+        };
+        let max_op = OutPoint {
+            txid: Txid::from_inner([0xff; 32]),
+            vout: u32::MAX,
+        };
+
+        let start = match range.start_bound() {
+            Included(index) => Included((index.clone(), min_op)),
+            Excluded(index) => Excluded((index.clone(), max_op)),
+            Unbounded => Unbounded,
+        };
+
+        let end = match range.end_bound() {
+            Included(index) => Included((index.clone(), max_op)),
+            Excluded(index) => Excluded((index.clone(), min_op)),
+            Unbounded => Unbounded,
+        };
+
+        self.spk_txouts.range((start, end)).map(|(i, op)| (i, *op))
+    }
+
+    /// Returns the txout and script pubkey index of the `TxOut` at `OutPoint`.
+    ///
+    /// Returns `None` if the `TxOut` hasn't been scanned or if nothing matching was found there.
+    pub fn txout(&self, outpoint: OutPoint) -> Option<(&I, &TxOut)> {
+        self.txouts
+            .get(&outpoint)
+            .map(|(spk_i, txout)| (spk_i, txout))
+    }
+
+    /// Returns the script that has been inserted at the `index`.
+    ///
+    /// If that index hasn't been inserted yet, it will return `None`.
+    pub fn spk_at_index(&self, index: &I) -> Option<&Script> {
+        self.spks.get(index)
+    }
+
+    /// The script pubkeys that are being tracked by the index.
+    pub fn all_spks(&self) -> &BTreeMap<I, Script> {
+        &self.spks
+    }
+
+    /// Adds a script pubkey to scan for. Returns `false` and does nothing if spk already exists in the map
+    ///
+    /// the index will look for outputs spending to this spk whenever it scans new data.
+    pub fn insert_spk(&mut self, index: I, spk: Script) -> bool {
+        match self.spk_indices.entry(spk.clone()) {
+            Entry::Vacant(value) => {
+                value.insert(index.clone());
+                self.spks.insert(index.clone(), spk);
+                self.unused.insert(index);
+                true
+            }
+            Entry::Occupied(_) => false,
+        }
+    }
+
+    /// Iterates over all unused script pubkeys in an index range.
+    ///
+    /// Here, "unused" means that after the script pubkey was stored in the index, the index has
+    /// never scanned a transaction output with it.
+    ///
+    /// # Example
+    ///
+    /// ```rust
+    /// # use bdk_chain::SpkTxOutIndex;
+    ///
+    /// // imagine our spks are indexed like (keychain, derivation_index).
+    /// let txout_index = SpkTxOutIndex::<(u32, u32)>::default();
+    /// let all_unused_spks = txout_index.unused_spks(..);
+    /// let change_index = 1;
+    /// let unused_change_spks =
+    ///     txout_index.unused_spks((change_index, u32::MIN)..(change_index, u32::MAX));
+    /// ```
+    pub fn unused_spks<R>(&self, range: R) -> impl DoubleEndedIterator<Item = (&I, &Script)>
+    where
+        R: RangeBounds<I>,
+    {
+        self.unused
+            .range(range)
+            .map(move |index| (index, self.spk_at_index(index).expect("must exist")))
+    }
+
+    /// Returns whether the script pubkey at `index` has been used or not.
+    ///
+    /// Here, "unused" means that after the script pubkey was stored in the index, the index has
+    /// never scanned a transaction output with it.
+    pub fn is_used(&self, index: &I) -> bool {
+        self.unused.get(index).is_none()
+    }
+
+    /// Marks the script pubkey at `index` as used even though it hasn't seen an output spending to it.
+    /// This only affects when the `index` had already been added to `self` and was unused.
+    ///
+    /// Returns whether the `index` was initially present as `unused`.
+    ///
+    /// This is useful when you want to reserve a script pubkey for something but don't want to add
+    /// the transaction output using it to the index yet. Other callers will consider the `index` used
+    /// until you call [`unmark_used`].
+    ///
+    /// [`unmark_used`]: Self::unmark_used
+    pub fn mark_used(&mut self, index: &I) -> bool {
+        self.unused.remove(index)
+    }
+
+    /// Undoes the effect of [`mark_used`]. Returns whether the `index` is inserted back into
+    /// `unused`.
+    ///
+    /// Note that if `self` has scanned an output with this script pubkey then this will have no
+    /// effect.
+    ///
+    /// [`mark_used`]: Self::mark_used
+    pub fn unmark_used(&mut self, index: &I) -> bool {
+        // we cannot set the index as unused when it does not exist
+        if !self.spks.contains_key(index) {
+            return false;
+        }
+        // we cannot set the index as unused when txouts are indexed under it
+        if self.outputs_in_range(index..=index).next().is_some() {
+            return false;
+        }
+        self.unused.insert(index.clone())
+    }
+
+    /// Returns the index associated with the script pubkey.
+    pub fn index_of_spk(&self, script: &Script) -> Option<&I> {
+        self.spk_indices.get(script)
+    }
+
+    /// Computes total input value going from script pubkeys in the index (sent) and the total output
+    /// value going to script pubkeys in the index (received) in `tx`. For the `sent` to be computed
+    /// correctly, the output being spent must have already been scanned by the index. Calculating
+    /// received just uses the transaction outputs directly, so it will be correct even if it has not
+    /// been scanned.
+    pub fn sent_and_received(&self, tx: &Transaction) -> (u64, u64) {
+        let mut sent = 0;
+        let mut received = 0;
+
+        for txin in &tx.input {
+            if let Some((_, txout)) = self.txout(txin.previous_output) {
+                sent += txout.value;
+            }
+        }
+        for txout in &tx.output {
+            if self.index_of_spk(&txout.script_pubkey).is_some() {
+                received += txout.value;
+            }
+        }
+
+        (sent, received)
+    }
+
+    /// Computes the net value that this transaction gives to the script pubkeys in the index and
+    /// *takes* from the transaction outputs in the index. Shorthand for calling
+    /// [`sent_and_received`] and subtracting sent from received.
+    ///
+    /// [`sent_and_received`]: Self::sent_and_received
+    pub fn net_value(&self, tx: &Transaction) -> i64 {
+        let (sent, received) = self.sent_and_received(tx);
+        received as i64 - sent as i64
+    }
+
+    /// Whether any of the inputs of this transaction spend a txout tracked or whether any output
+    /// matches one of our script pubkeys.
+    ///
+    /// It is easily possible to misuse this method and get false negatives by calling it before you
+    /// have scanned the `TxOut`s the transaction is spending. For example, if you want to filter out
+    /// all the transactions in a block that are irrelevant, you **must first scan all the
+    /// transactions in the block** and only then use this method.
+    pub fn is_relevant(&self, tx: &Transaction) -> bool {
+        let input_matches = tx
+            .input
+            .iter()
+            .any(|input| self.txouts.contains_key(&input.previous_output));
+        let output_matches = tx
+            .output
+            .iter()
+            .any(|output| self.spk_indices.contains_key(&output.script_pubkey));
+        input_matches || output_matches
+    }
+}

crates/chain/src/tx_data_traits.rs

@@ -0,0 +1,120 @@
+use crate::collections::BTreeMap;
+use crate::collections::BTreeSet;
+use crate::BlockId;
+use alloc::vec::Vec;
+use bitcoin::{Block, OutPoint, Transaction, TxOut};
+
+/// Trait to do something with every txout contained in a structure.
+///
+/// We would prefer to just work with things that can give us an `Iterator<Item=(OutPoint, &TxOut)>`
+/// here, but rust's type system makes it extremely hard to do this (without trait objects).
+pub trait ForEachTxOut {
+    /// The provided closure `f` will be called with each `outpoint/txout` pair.
+    fn for_each_txout(&self, f: impl FnMut((OutPoint, &TxOut)));
+}
+
+impl ForEachTxOut for Block {
+    fn for_each_txout(&self, mut f: impl FnMut((OutPoint, &TxOut))) {
+        for tx in self.txdata.iter() {
+            tx.for_each_txout(&mut f)
+        }
+    }
+}
+
+impl ForEachTxOut for Transaction {
+    fn for_each_txout(&self, mut f: impl FnMut((OutPoint, &TxOut))) {
+        let txid = self.txid();
+        for (i, txout) in self.output.iter().enumerate() {
+            f((
+                OutPoint {
+                    txid,
+                    vout: i as u32,
+                },
+                txout,
+            ))
+        }
+    }
+}
+
+/// Trait that "anchors" blockchain data to a specific block of height and hash.
+///
+/// I.e. If transaction A is anchored in block B, then if block B is in the best chain, we can
+/// assume that transaction A is also confirmed in the best chain. This does not necessarily mean
+/// that transaction A is confirmed in block B. It could also mean transaction A is confirmed in a
+/// parent block of B.
+pub trait Anchor: core::fmt::Debug + Clone + Eq + PartialOrd + Ord + core::hash::Hash {
+    /// Returns the [`BlockId`] that the associated blockchain data is "anchored" in.
+    fn anchor_block(&self) -> BlockId;
+
+    /// Get the upper bound of the chain data's confirmation height.
+    ///
+    /// The default definition gives a pessimistic answer. This can be overridden by the `Anchor`
+    /// implementation for a more accurate value.
+    fn confirmation_height_upper_bound(&self) -> u32 {
+        self.anchor_block().height
+    }
+}
+
+impl<A: Anchor> Anchor for &'static A {
+    fn anchor_block(&self) -> BlockId {
+        <A as Anchor>::anchor_block(self)
+    }
+}
+
+/// Trait that makes an object appendable.
+pub trait Append {
+    /// Append another object of the same type onto `self`.
+    fn append(&mut self, other: Self);
+
+    /// Returns whether the structure is considered empty.
+    fn is_empty(&self) -> bool;
+}
+
+impl Append for () {
+    fn append(&mut self, _other: Self) {}
+
+    fn is_empty(&self) -> bool {
+        true
+    }
+}
+
+impl<K: Ord, V> Append for BTreeMap<K, V> {
+    fn append(&mut self, mut other: Self) {
+        BTreeMap::append(self, &mut other)
+    }
+
+    fn is_empty(&self) -> bool {
+        BTreeMap::is_empty(self)
+    }
+}
+
+impl<T: Ord> Append for BTreeSet<T> {
+    fn append(&mut self, mut other: Self) {
+        BTreeSet::append(self, &mut other)
+    }
+
+    fn is_empty(&self) -> bool {
+        BTreeSet::is_empty(self)
+    }
+}
+
+impl<T> Append for Vec<T> {
+    fn append(&mut self, mut other: Self) {
+        Vec::append(self, &mut other)
+    }
+
+    fn is_empty(&self) -> bool {
+        Vec::is_empty(self)
+    }
+}
+
+impl<A: Append, B: Append> Append for (A, B) {
+    fn append(&mut self, other: Self) {
+        Append::append(&mut self.0, other.0);
+        Append::append(&mut self.1, other.1);
+    }
+
+    fn is_empty(&self) -> bool {
+        Append::is_empty(&self.0) && Append::is_empty(&self.1)
+    }
+}

crates/chain/tests/common/mod.rs

@@ -0,0 +1,68 @@
+#[allow(unused_macros)]
+macro_rules! h {
+    ($index:literal) => {{
+        bitcoin::hashes::Hash::hash($index.as_bytes())
+    }};
+}
+
+#[allow(unused_macros)]
+macro_rules! local_chain {
+    [ $(($height:expr, $block_hash:expr)), * ] => {{
+        #[allow(unused_mut)]
+        bdk_chain::local_chain::LocalChain::from_blocks([$(($height, $block_hash).into()),*])
+    }};
+}
+
+#[allow(unused_macros)]
+macro_rules! chain {
+    ($([$($tt:tt)*]),*) => { chain!( checkpoints: [$([$($tt)*]),*] ) };
+    (checkpoints: $($tail:tt)*) => { chain!( index: TxHeight, checkpoints: $($tail)*) };
+    (index: $ind:ty, checkpoints: [ $([$height:expr, $block_hash:expr]),* ] $(,txids: [$(($txid:expr, $tx_height:expr)),*])?) => {{
+        #[allow(unused_mut)]
+        let mut chain = bdk_chain::sparse_chain::SparseChain::<$ind>::from_checkpoints([$(($height, $block_hash).into()),*]);
+
+        $(
+            $(
+                let _ = chain.insert_tx($txid, $tx_height).expect("should succeed");
+            )*
+        )?
+
+        chain
+    }};
+}
+
+#[allow(unused_macros)]
+macro_rules! changeset {
+    (checkpoints: $($tail:tt)*) => { changeset!(index: TxHeight, checkpoints: $($tail)*) };
+    (
+        index: $ind:ty,
+        checkpoints: [ $(( $height:expr, $cp_to:expr )),* ]
+        $(,txids: [ $(( $txid:expr, $tx_to:expr )),* ])?
+    ) => {{
+        use bdk_chain::collections::BTreeMap;
+
+        #[allow(unused_mut)]
+        bdk_chain::sparse_chain::ChangeSet::<$ind> {
+            checkpoints: {
+                let mut changes = BTreeMap::default();
+                $(changes.insert($height, $cp_to);)*
+                changes
+            },
+            txids: {
+                let mut changes = BTreeMap::default();
+                $($(changes.insert($txid, $tx_to.map(|h: TxHeight| h.into()));)*)?
+                changes
+            }
+        }
+    }};
+}
+
+#[allow(unused)]
+pub fn new_tx(lt: u32) -> bitcoin::Transaction {
+    bitcoin::Transaction {
+        version: 0x00,
+        lock_time: bitcoin::PackedLockTime(lt),
+        input: vec![],
+        output: vec![],
+    }
+}

crates/chain/tests/test_indexed_tx_graph.rs

@@ -0,0 +1,467 @@
+#[macro_use]
+mod common;
+
+use std::collections::{BTreeMap, BTreeSet};
+
+use bdk_chain::{
+    indexed_tx_graph::{IndexedAdditions, IndexedTxGraph},
+    keychain::{Balance, DerivationAdditions, KeychainTxOutIndex},
+    local_chain::LocalChain,
+    tx_graph::Additions,
+    BlockId, ChainPosition, ConfirmationHeightAnchor,
+};
+use bitcoin::{secp256k1::Secp256k1, BlockHash, OutPoint, Script, Transaction, TxIn, TxOut};
+use miniscript::Descriptor;
+
+/// Ensure [`IndexedTxGraph::insert_relevant_txs`] can successfully index transactions NOT presented
+/// in topological order.
+///
+/// Given 3 transactions (A, B, C), where A has 2 owned outputs. B and C spends an output each of A.
+/// Typically, we would only know whether B and C are relevant if we have indexed A (A's outpoints
+/// are associated with owned spks in the index). Ensure insertion and indexing is topological-
+/// agnostic.
+#[test]
+fn insert_relevant_txs() {
+    const DESCRIPTOR: &str = "tr([73c5da0a/86'/0'/0']xprv9xgqHN7yz9MwCkxsBPN5qetuNdQSUttZNKw1dcYTV4mkaAFiBVGQziHs3NRSWMkCzvgjEe3n9xV8oYywvM8at9yRqyaZVz6TYYhX98VjsUk/0/*)";
+    let (descriptor, _) = Descriptor::parse_descriptor(&Secp256k1::signing_only(), DESCRIPTOR)
+        .expect("must be valid");
+    let spk_0 = descriptor.at_derivation_index(0).script_pubkey();
+    let spk_1 = descriptor.at_derivation_index(9).script_pubkey();
+
+    let mut graph = IndexedTxGraph::<ConfirmationHeightAnchor, KeychainTxOutIndex<()>>::default();
+    graph.index.add_keychain((), descriptor);
+    graph.index.set_lookahead(&(), 10);
+
+    let tx_a = Transaction {
+        output: vec![
+            TxOut {
+                value: 10_000,
+                script_pubkey: spk_0,
+            },
+            TxOut {
+                value: 20_000,
+                script_pubkey: spk_1,
+            },
+        ],
+        ..common::new_tx(0)
+    };
+
+    let tx_b = Transaction {
+        input: vec![TxIn {
+            previous_output: OutPoint::new(tx_a.txid(), 0),
+            ..Default::default()
+        }],
+        ..common::new_tx(1)
+    };
+
+    let tx_c = Transaction {
+        input: vec![TxIn {
+            previous_output: OutPoint::new(tx_a.txid(), 1),
+            ..Default::default()
+        }],
+        ..common::new_tx(2)
+    };
+
+    let txs = [tx_c, tx_b, tx_a];
+
+    assert_eq!(
+        graph.insert_relevant_txs(txs.iter().map(|tx| (tx, None)), None),
+        IndexedAdditions {
+            graph_additions: Additions {
+                txs: txs.into(),
+                ..Default::default()
+            },
+            index_additions: DerivationAdditions([((), 9_u32)].into()),
+        }
+    )
+}
+
+#[test]
+/// Ensure consistency IndexedTxGraph list_* and balance methods. These methods lists
+/// relevant txouts and utxos from the information fetched from a ChainOracle (here a LocalChain).
+///
+/// Test Setup:
+///
+/// Local Chain =>  <0> ----- <1> ----- <2> ----- <3> ---- ... ---- <150>
+///
+/// Keychains:
+///
+/// keychain_1: Trusted
+/// keychain_2: Untrusted
+///
+/// Transactions:
+///
+/// tx1: A Coinbase, sending 70000 sats to "trusted" address. [Block 0]
+/// tx2: A external Receive, sending 30000 sats to "untrusted" address. [Block 1]
+/// tx3: Internal Spend. Spends tx2 and returns change of 10000 to "trusted" address. [Block 2]
+/// tx4: Mempool tx, sending 20000 sats to "trusted" address.
+/// tx5: Mempool tx, sending 15000 sats to "untested" address.
+/// tx6: Complete unrelated tx. [Block 3]
+///
+/// Different transactions are added via `insert_relevant_txs`.
+/// `list_owned_txout`, `list_owned_utxos` and `balance` method is asserted
+/// with expected values at Block height 0, 1, and 2.
+///
+/// Finally Add more blocks to local chain until tx1 coinbase maturity hits.
+/// Assert maturity at coinbase maturity inflection height. Block height 98 and 99.
+
+fn test_list_owned_txouts() {
+    // Create Local chains
+
+    let local_chain = (0..150)
+        .map(|i| (i as u32, h!("random")))
+        .collect::<BTreeMap<u32, BlockHash>>();
+    let local_chain = LocalChain::from(local_chain);
+
+    // Initiate IndexedTxGraph
+
+    let (desc_1, _) = Descriptor::parse_descriptor(&Secp256k1::signing_only(), "tr(tprv8ZgxMBicQKsPd3krDUsBAmtnRsK3rb8u5yi1zhQgMhF1tR8MW7xfE4rnrbbsrbPR52e7rKapu6ztw1jXveJSCGHEriUGZV7mCe88duLp5pj/86'/1'/0'/0/*)").unwrap();
+    let (desc_2, _) = Descriptor::parse_descriptor(&Secp256k1::signing_only(), "tr(tprv8ZgxMBicQKsPd3krDUsBAmtnRsK3rb8u5yi1zhQgMhF1tR8MW7xfE4rnrbbsrbPR52e7rKapu6ztw1jXveJSCGHEriUGZV7mCe88duLp5pj/86'/1'/0'/1/*)").unwrap();
+
+    let mut graph =
+        IndexedTxGraph::<ConfirmationHeightAnchor, KeychainTxOutIndex<String>>::default();
+
+    graph.index.add_keychain("keychain_1".into(), desc_1);
+    graph.index.add_keychain("keychain_2".into(), desc_2);
+    graph.index.set_lookahead_for_all(10);
+
+    // Get trusted and untrusted addresses
+
+    let mut trusted_spks = Vec::new();
+    let mut untrusted_spks = Vec::new();
+
+    {
+        // we need to scope here to take immutanble reference of the graph
+        for _ in 0..10 {
+            let ((_, script), _) = graph.index.reveal_next_spk(&"keychain_1".to_string());
+            // TODO Assert indexes
+            trusted_spks.push(script.clone());
+        }
+    }
+    {
+        for _ in 0..10 {
+            let ((_, script), _) = graph.index.reveal_next_spk(&"keychain_2".to_string());
+            untrusted_spks.push(script.clone());
+        }
+    }
+
+    // Create test transactions
+
+    // tx1 is the genesis coinbase
+    let tx1 = Transaction {
+        input: vec![TxIn {
+            previous_output: OutPoint::null(),
+            ..Default::default()
+        }],
+        output: vec![TxOut {
+            value: 70000,
+            script_pubkey: trusted_spks[0].clone(),
+        }],
+        ..common::new_tx(0)
+    };
+
+    // tx2 is an incoming transaction received at untrusted keychain at block 1.
+    let tx2 = Transaction {
+        output: vec![TxOut {
+            value: 30000,
+            script_pubkey: untrusted_spks[0].clone(),
+        }],
+        ..common::new_tx(0)
+    };
+
+    // tx3 spends tx2 and gives a change back in trusted keychain. Confirmed at Block 2.
+    let tx3 = Transaction {
+        input: vec![TxIn {
+            previous_output: OutPoint::new(tx2.txid(), 0),
+            ..Default::default()
+        }],
+        output: vec![TxOut {
+            value: 10000,
+            script_pubkey: trusted_spks[1].clone(),
+        }],
+        ..common::new_tx(0)
+    };
+
+    // tx4 is an external transaction receiving at untrusted keychain, unconfirmed.
+    let tx4 = Transaction {
+        output: vec![TxOut {
+            value: 20000,
+            script_pubkey: untrusted_spks[1].clone(),
+        }],
+        ..common::new_tx(0)
+    };
+
+    // tx5 is spending tx3 and receiving change at trusted keychain, unconfirmed.
+    let tx5 = Transaction {
+        output: vec![TxOut {
+            value: 15000,
+            script_pubkey: trusted_spks[2].clone(),
+        }],
+        ..common::new_tx(0)
+    };
+
+    // tx6 is an unrelated transaction confirmed at 3.
+    let tx6 = common::new_tx(0);
+
+    // Insert transactions into graph with respective anchors
+    // For unconfirmed txs we pass in `None`.
+
+    let _ = graph.insert_relevant_txs(
+        [&tx1, &tx2, &tx3, &tx6].iter().enumerate().map(|(i, tx)| {
+            let height = i as u32;
+            (
+                *tx,
+                local_chain
+                    .blocks()
+                    .get(&height)
+                    .map(|&hash| BlockId { height, hash })
+                    .map(|anchor_block| ConfirmationHeightAnchor {
+                        anchor_block,
+                        confirmation_height: anchor_block.height,
+                    }),
+            )
+        }),
+        None,
+    );
+
+    let _ = graph.insert_relevant_txs([&tx4, &tx5].iter().map(|tx| (*tx, None)), Some(100));
+
+    // A helper lambda to extract and filter data from the graph.
+    let fetch =
+        |height: u32,
+         graph: &IndexedTxGraph<ConfirmationHeightAnchor, KeychainTxOutIndex<String>>| {
+            let chain_tip = local_chain
+                .blocks()
+                .get(&height)
+                .map(|&hash| BlockId { height, hash })
+                .expect("block must exist");
+            let txouts = graph
+                .graph()
+                .filter_chain_txouts(
+                    &local_chain,
+                    chain_tip,
+                    graph.index.outpoints().iter().cloned(),
+                )
+                .collect::<Vec<_>>();
+
+            let utxos = graph
+                .graph()
+                .filter_chain_unspents(
+                    &local_chain,
+                    chain_tip,
+                    graph.index.outpoints().iter().cloned(),
+                )
+                .collect::<Vec<_>>();
+
+            let balance = graph.graph().balance(
+                &local_chain,
+                chain_tip,
+                graph.index.outpoints().iter().cloned(),
+                |_, spk: &Script| trusted_spks.contains(spk),
+            );
+
+            assert_eq!(txouts.len(), 5);
+            assert_eq!(utxos.len(), 4);
+
+            let confirmed_txouts_txid = txouts
+                .iter()
+                .filter_map(|(_, full_txout)| {
+                    if matches!(full_txout.chain_position, ChainPosition::Confirmed(_)) {
+                        Some(full_txout.outpoint.txid)
+                    } else {
+                        None
+                    }
+                })
+                .collect::<BTreeSet<_>>();
+
+            let unconfirmed_txouts_txid = txouts
+                .iter()
+                .filter_map(|(_, full_txout)| {
+                    if matches!(full_txout.chain_position, ChainPosition::Unconfirmed(_)) {
+                        Some(full_txout.outpoint.txid)
+                    } else {
+                        None
+                    }
+                })
+                .collect::<BTreeSet<_>>();
+
+            let confirmed_utxos_txid = utxos
+                .iter()
+                .filter_map(|(_, full_txout)| {
+                    if matches!(full_txout.chain_position, ChainPosition::Confirmed(_)) {
+                        Some(full_txout.outpoint.txid)
+                    } else {
+                        None
+                    }
+                })
+                .collect::<BTreeSet<_>>();
+
+            let unconfirmed_utxos_txid = utxos
+                .iter()
+                .filter_map(|(_, full_txout)| {
+                    if matches!(full_txout.chain_position, ChainPosition::Unconfirmed(_)) {
+                        Some(full_txout.outpoint.txid)
+                    } else {
+                        None
+                    }
+                })
+                .collect::<BTreeSet<_>>();
+
+            (
+                confirmed_txouts_txid,
+                unconfirmed_txouts_txid,
+                confirmed_utxos_txid,
+                unconfirmed_utxos_txid,
+                balance,
+            )
+        };
+
+    // ----- TEST BLOCK -----
+
+    // AT Block 0
+    {
+        let (
+            confirmed_txouts_txid,
+            unconfirmed_txouts_txid,
+            confirmed_utxos_txid,
+            unconfirmed_utxos_txid,
+            balance,
+        ) = fetch(0, &graph);
+
+        assert_eq!(confirmed_txouts_txid, [tx1.txid()].into());
+        assert_eq!(
+            unconfirmed_txouts_txid,
+            [tx2.txid(), tx3.txid(), tx4.txid(), tx5.txid()].into()
+        );
+
+        assert_eq!(confirmed_utxos_txid, [tx1.txid()].into());
+        assert_eq!(
+            unconfirmed_utxos_txid,
+            [tx3.txid(), tx4.txid(), tx5.txid()].into()
+        );
+
+        assert_eq!(
+            balance,
+            Balance {
+                immature: 70000,          // immature coinbase
+                trusted_pending: 25000,   // tx3 + tx5
+                untrusted_pending: 20000, // tx4
+                confirmed: 0              // Nothing is confirmed yet
+            }
+        );
+    }
+
+    // AT Block 1
+    {
+        let (
+            confirmed_txouts_txid,
+            unconfirmed_txouts_txid,
+            confirmed_utxos_txid,
+            unconfirmed_utxos_txid,
+            balance,
+        ) = fetch(1, &graph);
+
+        // tx2 gets into confirmed txout set
+        assert_eq!(confirmed_txouts_txid, [tx1.txid(), tx2.txid()].into());
+        assert_eq!(
+            unconfirmed_txouts_txid,
+            [tx3.txid(), tx4.txid(), tx5.txid()].into()
+        );
+
+        // tx2 doesn't get into confirmed utxos set
+        assert_eq!(confirmed_utxos_txid, [tx1.txid()].into());
+        assert_eq!(
+            unconfirmed_utxos_txid,
+            [tx3.txid(), tx4.txid(), tx5.txid()].into()
+        );
+
+        assert_eq!(
+            balance,
+            Balance {
+                immature: 70000,          // immature coinbase
+                trusted_pending: 25000,   // tx3 + tx5
+                untrusted_pending: 20000, // tx4
+                confirmed: 0              // Nothing is confirmed yet
+            }
+        );
+    }
+
+    // AT Block 2
+    {
+        let (
+            confirmed_txouts_txid,
+            unconfirmed_txouts_txid,
+            confirmed_utxos_txid,
+            unconfirmed_utxos_txid,
+            balance,
+        ) = fetch(2, &graph);
+
+        // tx3 now gets into the confirmed txout set
+        assert_eq!(
+            confirmed_txouts_txid,
+            [tx1.txid(), tx2.txid(), tx3.txid()].into()
+        );
+        assert_eq!(unconfirmed_txouts_txid, [tx4.txid(), tx5.txid()].into());
+
+        // tx3 also gets into confirmed utxo set
+        assert_eq!(confirmed_utxos_txid, [tx1.txid(), tx3.txid()].into());
+        assert_eq!(unconfirmed_utxos_txid, [tx4.txid(), tx5.txid()].into());
+
+        assert_eq!(
+            balance,
+            Balance {
+                immature: 70000,          // immature coinbase
+                trusted_pending: 15000,   // tx5
+                untrusted_pending: 20000, // tx4
+                confirmed: 10000          // tx3 got confirmed
+            }
+        );
+    }
+
+    // AT Block 98
+    {
+        let (
+            confirmed_txouts_txid,
+            unconfirmed_txouts_txid,
+            confirmed_utxos_txid,
+            unconfirmed_utxos_txid,
+            balance,
+        ) = fetch(98, &graph);
+
+        assert_eq!(
+            confirmed_txouts_txid,
+            [tx1.txid(), tx2.txid(), tx3.txid()].into()
+        );
+        assert_eq!(unconfirmed_txouts_txid, [tx4.txid(), tx5.txid()].into());
+
+        assert_eq!(confirmed_utxos_txid, [tx1.txid(), tx3.txid()].into());
+        assert_eq!(unconfirmed_utxos_txid, [tx4.txid(), tx5.txid()].into());
+
+        // Coinbase is still immature
+        assert_eq!(
+            balance,
+            Balance {
+                immature: 70000,          // immature coinbase
+                trusted_pending: 15000,   // tx5
+                untrusted_pending: 20000, // tx4
+                confirmed: 10000          // tx1 got matured
+            }
+        );
+    }
+
+    // AT Block 99
+    {
+        let (_, _, _, _, balance) = fetch(100, &graph);
+
+        // Coinbase maturity hits
+        assert_eq!(
+            balance,
+            Balance {
+                immature: 0,              // coinbase matured
+                trusted_pending: 15000,   // tx5
+                untrusted_pending: 20000, // tx4
+                confirmed: 80000          // tx1 + tx3
+            }
+        );
+    }
+}

crates/chain/tests/test_keychain_txout_index.rs

@@ -0,0 +1,368 @@
+#![cfg(feature = "miniscript")]
+
+#[macro_use]
+mod common;
+use bdk_chain::{
+    collections::BTreeMap,
+    keychain::{DerivationAdditions, KeychainTxOutIndex},
+};
+
+use bitcoin::{secp256k1::Secp256k1, OutPoint, Script, Transaction, TxOut};
+use miniscript::{Descriptor, DescriptorPublicKey};
+
+#[derive(Clone, Debug, PartialEq, Eq, Ord, PartialOrd)]
+enum TestKeychain {
+    External,
+    Internal,
+}
+
+fn init_txout_index() -> (
+    bdk_chain::keychain::KeychainTxOutIndex<TestKeychain>,
+    Descriptor<DescriptorPublicKey>,
+    Descriptor<DescriptorPublicKey>,
+) {
+    let mut txout_index = bdk_chain::keychain::KeychainTxOutIndex::<TestKeychain>::default();
+
+    let secp = bdk_chain::bitcoin::secp256k1::Secp256k1::signing_only();
+    let (external_descriptor,_) = Descriptor::<DescriptorPublicKey>::parse_descriptor(&secp, "tr([73c5da0a/86'/0'/0']xprv9xgqHN7yz9MwCkxsBPN5qetuNdQSUttZNKw1dcYTV4mkaAFiBVGQziHs3NRSWMkCzvgjEe3n9xV8oYywvM8at9yRqyaZVz6TYYhX98VjsUk/0/*)").unwrap();
+    let (internal_descriptor,_) = Descriptor::<DescriptorPublicKey>::parse_descriptor(&secp, "tr([73c5da0a/86'/0'/0']xprv9xgqHN7yz9MwCkxsBPN5qetuNdQSUttZNKw1dcYTV4mkaAFiBVGQziHs3NRSWMkCzvgjEe3n9xV8oYywvM8at9yRqyaZVz6TYYhX98VjsUk/1/*)").unwrap();
+
+    txout_index.add_keychain(TestKeychain::External, external_descriptor.clone());
+    txout_index.add_keychain(TestKeychain::Internal, internal_descriptor.clone());
+
+    (txout_index, external_descriptor, internal_descriptor)
+}
+
+fn spk_at_index(descriptor: &Descriptor<DescriptorPublicKey>, index: u32) -> Script {
+    descriptor
+        .derived_descriptor(&Secp256k1::verification_only(), index)
+        .expect("must derive")
+        .script_pubkey()
+}
+
+#[test]
+fn test_set_all_derivation_indices() {
+    let (mut txout_index, _, _) = init_txout_index();
+    let derive_to: BTreeMap<_, _> =
+        [(TestKeychain::External, 12), (TestKeychain::Internal, 24)].into();
+    assert_eq!(
+        txout_index.reveal_to_target_multi(&derive_to).1.as_inner(),
+        &derive_to
+    );
+    assert_eq!(txout_index.last_revealed_indices(), &derive_to);
+    assert_eq!(
+        txout_index.reveal_to_target_multi(&derive_to).1,
+        DerivationAdditions::default(),
+        "no changes if we set to the same thing"
+    );
+}
+
+#[test]
+fn test_lookahead() {
+    let (mut txout_index, external_desc, internal_desc) = init_txout_index();
+
+    // ensure it does not break anything if lookahead is set multiple times
+    (0..=10).for_each(|lookahead| txout_index.set_lookahead(&TestKeychain::External, lookahead));
+    (0..=20)
+        .filter(|v| v % 2 == 0)
+        .for_each(|lookahead| txout_index.set_lookahead(&TestKeychain::Internal, lookahead));
+
+    assert_eq!(txout_index.inner().all_spks().len(), 30);
+
+    // given:
+    // - external lookahead set to 10
+    // - internal lookahead set to 20
+    // when:
+    // - set external derivation index to value higher than last, but within the lookahead value
+    // expect:
+    // - scripts cached in spk_txout_index should increase correctly
+    // - stored scripts of external keychain should be of expected counts
+    for index in (0..20).skip_while(|i| i % 2 == 1) {
+        let (revealed_spks, revealed_additions) =
+            txout_index.reveal_to_target(&TestKeychain::External, index);
+        assert_eq!(
+            revealed_spks.collect::<Vec<_>>(),
+            vec![(index, spk_at_index(&external_desc, index))],
+        );
+        assert_eq!(
+            revealed_additions.as_inner(),
+            &[(TestKeychain::External, index)].into()
+        );
+
+        assert_eq!(
+            txout_index.inner().all_spks().len(),
+            10 /* external lookahead */ +
+            20 /* internal lookahead */ +
+            index as usize + 1 /* `derived` count */
+        );
+        assert_eq!(
+            txout_index
+                .revealed_spks_of_keychain(&TestKeychain::External)
+                .count(),
+            index as usize + 1,
+        );
+        assert_eq!(
+            txout_index
+                .revealed_spks_of_keychain(&TestKeychain::Internal)
+                .count(),
+            0,
+        );
+        assert_eq!(
+            txout_index
+                .unused_spks_of_keychain(&TestKeychain::External)
+                .count(),
+            index as usize + 1,
+        );
+        assert_eq!(
+            txout_index
+                .unused_spks_of_keychain(&TestKeychain::Internal)
+                .count(),
+            0,
+        );
+    }
+
+    // given:
+    // - internal lookahead is 20
+    // - internal derivation index is `None`
+    // when:
+    // - derivation index is set ahead of current derivation index + lookahead
+    // expect:
+    // - scripts cached in spk_txout_index should increase correctly, a.k.a. no scripts are skipped
+    let (revealed_spks, revealed_additions) =
+        txout_index.reveal_to_target(&TestKeychain::Internal, 24);
+    assert_eq!(
+        revealed_spks.collect::<Vec<_>>(),
+        (0..=24)
+            .map(|index| (index, spk_at_index(&internal_desc, index)))
+            .collect::<Vec<_>>(),
+    );
+    assert_eq!(
+        revealed_additions.as_inner(),
+        &[(TestKeychain::Internal, 24)].into()
+    );
+    assert_eq!(
+        txout_index.inner().all_spks().len(),
+        10 /* external lookahead */ +
+        20 /* internal lookahead */ +
+        20 /* external stored index count */ +
+        25 /* internal stored index count */
+    );
+    assert_eq!(
+        txout_index
+            .revealed_spks_of_keychain(&TestKeychain::Internal)
+            .count(),
+        25,
+    );
+
+    // ensure derivation indices are expected for each keychain
+    let last_external_index = txout_index
+        .last_revealed_index(&TestKeychain::External)
+        .expect("already derived");
+    let last_internal_index = txout_index
+        .last_revealed_index(&TestKeychain::Internal)
+        .expect("already derived");
+    assert_eq!(last_external_index, 19);
+    assert_eq!(last_internal_index, 24);
+
+    // when:
+    // - scanning txouts with spks within stored indexes
+    // expect:
+    // - no changes to stored index counts
+    let external_iter = 0..=last_external_index;
+    let internal_iter = last_internal_index - last_external_index..=last_internal_index;
+    for (external_index, internal_index) in external_iter.zip(internal_iter) {
+        let tx = Transaction {
+            output: vec![
+                TxOut {
+                    script_pubkey: external_desc
+                        .at_derivation_index(external_index)
+                        .script_pubkey(),
+                    value: 10_000,
+                },
+                TxOut {
+                    script_pubkey: internal_desc
+                        .at_derivation_index(internal_index)
+                        .script_pubkey(),
+                    value: 10_000,
+                },
+            ],
+            ..common::new_tx(external_index)
+        };
+        assert_eq!(txout_index.scan(&tx), DerivationAdditions::default());
+        assert_eq!(
+            txout_index.last_revealed_index(&TestKeychain::External),
+            Some(last_external_index)
+        );
+        assert_eq!(
+            txout_index.last_revealed_index(&TestKeychain::Internal),
+            Some(last_internal_index)
+        );
+        assert_eq!(
+            txout_index
+                .revealed_spks_of_keychain(&TestKeychain::External)
+                .count(),
+            last_external_index as usize + 1,
+        );
+        assert_eq!(
+            txout_index
+                .revealed_spks_of_keychain(&TestKeychain::Internal)
+                .count(),
+            last_internal_index as usize + 1,
+        );
+    }
+}
+
+// when:
+// - scanning txouts with spks above last stored index
+// expect:
+// - last revealed index should increase as expected
+// - last used index should change as expected
+#[test]
+fn test_scan_with_lookahead() {
+    let (mut txout_index, external_desc, _) = init_txout_index();
+    txout_index.set_lookahead_for_all(10);
+
+    let spks: BTreeMap<u32, Script> = [0, 10, 20, 30]
+        .into_iter()
+        .map(|i| (i, external_desc.at_derivation_index(i).script_pubkey()))
+        .collect();
+
+    for (&spk_i, spk) in &spks {
+        let op = OutPoint::new(h!("fake tx"), spk_i);
+        let txout = TxOut {
+            script_pubkey: spk.clone(),
+            value: 0,
+        };
+
+        let additions = txout_index.scan_txout(op, &txout);
+        assert_eq!(
+            additions.as_inner(),
+            &[(TestKeychain::External, spk_i)].into()
+        );
+        assert_eq!(
+            txout_index.last_revealed_index(&TestKeychain::External),
+            Some(spk_i)
+        );
+        assert_eq!(
+            txout_index.last_used_index(&TestKeychain::External),
+            Some(spk_i)
+        );
+    }
+
+    // now try with index 41 (lookahead surpassed), we expect that the txout to not be indexed
+    let spk_41 = external_desc.at_derivation_index(41).script_pubkey();
+    let op = OutPoint::new(h!("fake tx"), 41);
+    let txout = TxOut {
+        script_pubkey: spk_41,
+        value: 0,
+    };
+    let additions = txout_index.scan_txout(op, &txout);
+    assert!(additions.is_empty());
+}
+
+#[test]
+fn test_wildcard_derivations() {
+    let (mut txout_index, external_desc, _) = init_txout_index();
+    let external_spk_0 = external_desc.at_derivation_index(0).script_pubkey();
+    let external_spk_16 = external_desc.at_derivation_index(16).script_pubkey();
+    let external_spk_26 = external_desc.at_derivation_index(26).script_pubkey();
+    let external_spk_27 = external_desc.at_derivation_index(27).script_pubkey();
+
+    // - nothing is derived
+    // - unused list is also empty
+    //
+    // - next_derivation_index() == (0, true)
+    // - derive_new() == ((0, <spk>), DerivationAdditions)
+    // - next_unused() == ((0, <spk>), DerivationAdditions:is_empty())
+    assert_eq!(txout_index.next_index(&TestKeychain::External), (0, true));
+    let (spk, changeset) = txout_index.reveal_next_spk(&TestKeychain::External);
+    assert_eq!(spk, (0_u32, &external_spk_0));
+    assert_eq!(changeset.as_inner(), &[(TestKeychain::External, 0)].into());
+    let (spk, changeset) = txout_index.next_unused_spk(&TestKeychain::External);
+    assert_eq!(spk, (0_u32, &external_spk_0));
+    assert_eq!(changeset.as_inner(), &[].into());
+
+    // - derived till 25
+    // - used all spks till 15.
+    // - used list : [0..=15, 17, 20, 23]
+    // - unused list: [16, 18, 19, 21, 22, 24, 25]
+
+    // - next_derivation_index() = (26, true)
+    // - derive_new() = ((26, <spk>), DerivationAdditions)
+    // - next_unused() == ((16, <spk>), DerivationAdditions::is_empty())
+    let _ = txout_index.reveal_to_target(&TestKeychain::External, 25);
+
+    (0..=15)
+        .chain(vec![17, 20, 23].into_iter())
+        .for_each(|index| assert!(txout_index.mark_used(&TestKeychain::External, index)));
+
+    assert_eq!(txout_index.next_index(&TestKeychain::External), (26, true));
+
+    let (spk, changeset) = txout_index.reveal_next_spk(&TestKeychain::External);
+    assert_eq!(spk, (26, &external_spk_26));
+
+    assert_eq!(changeset.as_inner(), &[(TestKeychain::External, 26)].into());
+
+    let (spk, changeset) = txout_index.next_unused_spk(&TestKeychain::External);
+    assert_eq!(spk, (16, &external_spk_16));
+    assert_eq!(changeset.as_inner(), &[].into());
+
+    // - Use all the derived till 26.
+    // - next_unused() = ((27, <spk>), DerivationAdditions)
+    (0..=26).for_each(|index| {
+        txout_index.mark_used(&TestKeychain::External, index);
+    });
+
+    let (spk, changeset) = txout_index.next_unused_spk(&TestKeychain::External);
+    assert_eq!(spk, (27, &external_spk_27));
+    assert_eq!(changeset.as_inner(), &[(TestKeychain::External, 27)].into());
+}
+
+#[test]
+fn test_non_wildcard_derivations() {
+    let mut txout_index = KeychainTxOutIndex::<TestKeychain>::default();
+
+    let secp = bitcoin::secp256k1::Secp256k1::signing_only();
+    let (no_wildcard_descriptor, _) = Descriptor::<DescriptorPublicKey>::parse_descriptor(&secp, "wpkh([73c5da0a/86'/0'/0']xprv9xgqHN7yz9MwCkxsBPN5qetuNdQSUttZNKw1dcYTV4mkaAFiBVGQziHs3NRSWMkCzvgjEe3n9xV8oYywvM8at9yRqyaZVz6TYYhX98VjsUk/1/0)").unwrap();
+    let external_spk = no_wildcard_descriptor
+        .at_derivation_index(0)
+        .script_pubkey();
+
+    txout_index.add_keychain(TestKeychain::External, no_wildcard_descriptor);
+
+    // given:
+    // - `txout_index` with no stored scripts
+    // expect:
+    // - next derivation index should be new
+    // - when we derive a new script, script @ index 0
+    // - when we get the next unused script, script @ index 0
+    assert_eq!(txout_index.next_index(&TestKeychain::External), (0, true));
+    let (spk, changeset) = txout_index.reveal_next_spk(&TestKeychain::External);
+    assert_eq!(spk, (0, &external_spk));
+    assert_eq!(changeset.as_inner(), &[(TestKeychain::External, 0)].into());
+
+    let (spk, changeset) = txout_index.next_unused_spk(&TestKeychain::External);
+    assert_eq!(spk, (0, &external_spk));
+    assert_eq!(changeset.as_inner(), &[].into());
+
+    // given:
+    // - the non-wildcard descriptor already has a stored and used script
+    // expect:
+    // - next derivation index should not be new
+    // - derive new and next unused should return the old script
+    // - store_up_to should not panic and return empty additions
+    assert_eq!(txout_index.next_index(&TestKeychain::External), (0, false));
+    txout_index.mark_used(&TestKeychain::External, 0);
+
+    let (spk, changeset) = txout_index.reveal_next_spk(&TestKeychain::External);
+    assert_eq!(spk, (0, &external_spk));
+    assert_eq!(changeset.as_inner(), &[].into());
+
+    let (spk, changeset) = txout_index.next_unused_spk(&TestKeychain::External);
+    assert_eq!(spk, (0, &external_spk));
+    assert_eq!(changeset.as_inner(), &[].into());
+    let (revealed_spks, revealed_additions) =
+        txout_index.reveal_to_target(&TestKeychain::External, 200);
+    assert_eq!(revealed_spks.count(), 0);
+    assert!(revealed_additions.is_empty());
+}

crates/chain/tests/test_local_chain.rs

@@ -0,0 +1,228 @@
+use bdk_chain::local_chain::{
+    ChangeSet, InsertBlockNotMatchingError, LocalChain, UpdateNotConnectedError,
+};
+use bitcoin::BlockHash;
+
+#[macro_use]
+mod common;
+
+#[test]
+fn add_first_tip() {
+    let chain = LocalChain::default();
+    assert_eq!(
+        chain.determine_changeset(&local_chain![(0, h!("A"))]),
+        Ok([(0, Some(h!("A")))].into()),
+        "add first tip"
+    );
+}
+
+#[test]
+fn add_second_tip() {
+    let chain = local_chain![(0, h!("A"))];
+    assert_eq!(
+        chain.determine_changeset(&local_chain![(0, h!("A")), (1, h!("B"))]),
+        Ok([(1, Some(h!("B")))].into())
+    );
+}
+
+#[test]
+fn two_disjoint_chains_cannot_merge() {
+    let chain1 = local_chain![(0, h!("A"))];
+    let chain2 = local_chain![(1, h!("B"))];
+    assert_eq!(
+        chain1.determine_changeset(&chain2),
+        Err(UpdateNotConnectedError(0))
+    );
+}
+
+#[test]
+fn duplicate_chains_should_merge() {
+    let chain1 = local_chain![(0, h!("A"))];
+    let chain2 = local_chain![(0, h!("A"))];
+    assert_eq!(chain1.determine_changeset(&chain2), Ok(Default::default()));
+}
+
+#[test]
+fn can_introduce_older_checkpoints() {
+    let chain1 = local_chain![(2, h!("C")), (3, h!("D"))];
+    let chain2 = local_chain![(1, h!("B")), (2, h!("C"))];
+
+    assert_eq!(
+        chain1.determine_changeset(&chain2),
+        Ok([(1, Some(h!("B")))].into())
+    );
+}
+
+#[test]
+fn fix_blockhash_before_agreement_point() {
+    let chain1 = local_chain![(0, h!("im-wrong")), (1, h!("we-agree"))];
+    let chain2 = local_chain![(0, h!("fix")), (1, h!("we-agree"))];
+
+    assert_eq!(
+        chain1.determine_changeset(&chain2),
+        Ok([(0, Some(h!("fix")))].into())
+    )
+}
+
+/// B and C are in both chain and update
+/// ```
+///        | 0 | 1 | 2 | 3 | 4
+/// chain  |     B   C
+/// update | A   B   C   D
+/// ```
+/// This should succeed with the point of agreement being C and A should be added in addition.
+#[test]
+fn two_points_of_agreement() {
+    let chain1 = local_chain![(1, h!("B")), (2, h!("C"))];
+    let chain2 = local_chain![(0, h!("A")), (1, h!("B")), (2, h!("C")), (3, h!("D"))];
+
+    assert_eq!(
+        chain1.determine_changeset(&chain2),
+        Ok([(0, Some(h!("A"))), (3, Some(h!("D")))].into()),
+    );
+}
+
+/// Update and chain does not connect:
+/// ```
+///        | 0 | 1 | 2 | 3 | 4
+/// chain  |     B   C
+/// update | A   B       D
+/// ```
+/// This should fail as we cannot figure out whether C & D are on the same chain
+#[test]
+fn update_and_chain_does_not_connect() {
+    let chain1 = local_chain![(1, h!("B")), (2, h!("C"))];
+    let chain2 = local_chain![(0, h!("A")), (1, h!("B")), (3, h!("D"))];
+
+    assert_eq!(
+        chain1.determine_changeset(&chain2),
+        Err(UpdateNotConnectedError(2)),
+    );
+}
+
+/// Transient invalidation:
+/// ```
+///        | 0 | 1 | 2 | 3 | 4 | 5
+/// chain  | A       B   C       E
+/// update | A       B'  C'  D
+/// ```
+/// This should succeed and invalidate B,C and E with point of agreement being A.
+#[test]
+fn transitive_invalidation_applies_to_checkpoints_higher_than_invalidation() {
+    let chain1 = local_chain![(0, h!("A")), (2, h!("B")), (3, h!("C")), (5, h!("E"))];
+    let chain2 = local_chain![(0, h!("A")), (2, h!("B'")), (3, h!("C'")), (4, h!("D"))];
+
+    assert_eq!(
+        chain1.determine_changeset(&chain2),
+        Ok([
+            (2, Some(h!("B'"))),
+            (3, Some(h!("C'"))),
+            (4, Some(h!("D"))),
+            (5, None),
+        ]
+        .into())
+    );
+}
+
+/// Transient invalidation:
+/// ```
+///        | 0 | 1 | 2 | 3 | 4
+/// chain  |     B   C       E
+/// update |     B'  C'  D
+/// ```
+///
+/// This should succeed and invalidate B, C and E with no point of agreement
+#[test]
+fn transitive_invalidation_applies_to_checkpoints_higher_than_invalidation_no_point_of_agreement() {
+    let chain1 = local_chain![(1, h!("B")), (2, h!("C")), (4, h!("E"))];
+    let chain2 = local_chain![(1, h!("B'")), (2, h!("C'")), (3, h!("D"))];
+
+    assert_eq!(
+        chain1.determine_changeset(&chain2),
+        Ok([
+            (1, Some(h!("B'"))),
+            (2, Some(h!("C'"))),
+            (3, Some(h!("D"))),
+            (4, None)
+        ]
+        .into())
+    )
+}
+
+/// Transient invalidation:
+/// ```
+///        | 0 | 1 | 2 | 3 | 4
+/// chain  | A   B   C       E
+/// update |     B'  C'  D
+/// ```
+///
+/// This should fail since although it tells us that B and C are invalid it doesn't tell us whether
+/// A was invalid.
+#[test]
+fn invalidation_but_no_connection() {
+    let chain1 = local_chain![(0, h!("A")), (1, h!("B")), (2, h!("C")), (4, h!("E"))];
+    let chain2 = local_chain![(1, h!("B'")), (2, h!("C'")), (3, h!("D"))];
+
+    assert_eq!(
+        chain1.determine_changeset(&chain2),
+        Err(UpdateNotConnectedError(0))
+    )
+}
+
+#[test]
+fn insert_block() {
+    struct TestCase {
+        original: LocalChain,
+        insert: (u32, BlockHash),
+        expected_result: Result<ChangeSet, InsertBlockNotMatchingError>,
+        expected_final: LocalChain,
+    }
+
+    let test_cases = [
+        TestCase {
+            original: local_chain![],
+            insert: (5, h!("block5")),
+            expected_result: Ok([(5, Some(h!("block5")))].into()),
+            expected_final: local_chain![(5, h!("block5"))],
+        },
+        TestCase {
+            original: local_chain![(3, h!("A"))],
+            insert: (4, h!("B")),
+            expected_result: Ok([(4, Some(h!("B")))].into()),
+            expected_final: local_chain![(3, h!("A")), (4, h!("B"))],
+        },
+        TestCase {
+            original: local_chain![(4, h!("B"))],
+            insert: (3, h!("A")),
+            expected_result: Ok([(3, Some(h!("A")))].into()),
+            expected_final: local_chain![(3, h!("A")), (4, h!("B"))],
+        },
+        TestCase {
+            original: local_chain![(2, h!("K"))],
+            insert: (2, h!("K")),
+            expected_result: Ok([].into()),
+            expected_final: local_chain![(2, h!("K"))],
+        },
+        TestCase {
+            original: local_chain![(2, h!("K"))],
+            insert: (2, h!("J")),
+            expected_result: Err(InsertBlockNotMatchingError {
+                height: 2,
+                original_hash: h!("K"),
+                update_hash: h!("J"),
+            }),
+            expected_final: local_chain![(2, h!("K"))],
+        },
+    ];
+
+    for (i, t) in test_cases.into_iter().enumerate() {
+        let mut chain = t.original;
+        assert_eq!(
+            chain.insert_block(t.insert.into()),
+            t.expected_result,
+            "[{}] unexpected result when inserting block",
+            i,
+        );
+        assert_eq!(chain, t.expected_final, "[{}] unexpected final chain", i,);
+    }
+}

crates/chain/tests/test_spk_txout_index.rs

@@ -0,0 +1,100 @@
+use bdk_chain::SpkTxOutIndex;
+use bitcoin::{hashes::hex::FromHex, OutPoint, PackedLockTime, Script, Transaction, TxIn, TxOut};
+
+#[test]
+fn spk_txout_sent_and_received() {
+    let spk1 = Script::from_hex("001404f1e52ce2bab3423c6a8c63b7cd730d8f12542c").unwrap();
+    let spk2 = Script::from_hex("00142b57404ae14f08c3a0c903feb2af7830605eb00f").unwrap();
+
+    let mut index = SpkTxOutIndex::default();
+    index.insert_spk(0, spk1.clone());
+    index.insert_spk(1, spk2.clone());
+
+    let tx1 = Transaction {
+        version: 0x02,
+        lock_time: PackedLockTime(0),
+        input: vec![],
+        output: vec![TxOut {
+            value: 42_000,
+            script_pubkey: spk1.clone(),
+        }],
+    };
+
+    assert_eq!(index.sent_and_received(&tx1), (0, 42_000));
+    assert_eq!(index.net_value(&tx1), 42_000);
+    index.scan(&tx1);
+    assert_eq!(
+        index.sent_and_received(&tx1),
+        (0, 42_000),
+        "shouldn't change after scanning"
+    );
+
+    let tx2 = Transaction {
+        version: 0x1,
+        lock_time: PackedLockTime(0),
+        input: vec![TxIn {
+            previous_output: OutPoint {
+                txid: tx1.txid(),
+                vout: 0,
+            },
+            ..Default::default()
+        }],
+        output: vec![
+            TxOut {
+                value: 20_000,
+                script_pubkey: spk2,
+            },
+            TxOut {
+                script_pubkey: spk1,
+                value: 30_000,
+            },
+        ],
+    };
+
+    assert_eq!(index.sent_and_received(&tx2), (42_000, 50_000));
+    assert_eq!(index.net_value(&tx2), 8_000);
+}
+
+#[test]
+fn mark_used() {
+    let spk1 = Script::from_hex("001404f1e52ce2bab3423c6a8c63b7cd730d8f12542c").unwrap();
+    let spk2 = Script::from_hex("00142b57404ae14f08c3a0c903feb2af7830605eb00f").unwrap();
+
+    let mut spk_index = SpkTxOutIndex::default();
+    spk_index.insert_spk(1, spk1.clone());
+    spk_index.insert_spk(2, spk2);
+
+    assert!(!spk_index.is_used(&1));
+    spk_index.mark_used(&1);
+    assert!(spk_index.is_used(&1));
+    spk_index.unmark_used(&1);
+    assert!(!spk_index.is_used(&1));
+    spk_index.mark_used(&1);
+    assert!(spk_index.is_used(&1));
+
+    let tx1 = Transaction {
+        version: 0x02,
+        lock_time: PackedLockTime(0),
+        input: vec![],
+        output: vec![TxOut {
+            value: 42_000,
+            script_pubkey: spk1,
+        }],
+    };
+
+    spk_index.scan(&tx1);
+    spk_index.unmark_used(&1);
+    assert!(
+        spk_index.is_used(&1),
+        "even though we unmark_used it doesn't matter because there was a tx scanned that used it"
+    );
+}
+
+#[test]
+fn unmark_used_does_not_result_in_invalid_representation() {
+    let mut spk_index = SpkTxOutIndex::default();
+    assert!(!spk_index.unmark_used(&0));
+    assert!(!spk_index.unmark_used(&1));
+    assert!(!spk_index.unmark_used(&2));
+    assert!(spk_index.unused_spks(..).collect::<Vec<_>>().is_empty());
+}

crates/chain/tests/test_tx_graph.rs

@@ -0,0 +1,824 @@
+#[macro_use]
+mod common;
+use bdk_chain::{
+    collections::*,
+    local_chain::LocalChain,
+    tx_graph::{Additions, TxGraph},
+    Append, BlockId, ChainPosition, ConfirmationHeightAnchor,
+};
+use bitcoin::{
+    hashes::Hash, BlockHash, OutPoint, PackedLockTime, Script, Transaction, TxIn, TxOut, Txid,
+};
+use core::iter;
+use std::vec;
+
+#[test]
+fn insert_txouts() {
+    // 2 (Outpoint, TxOut) tupples that denotes original data in the graph, as partial transactions.
+    let original_ops = [
+        (
+            OutPoint::new(h!("tx1"), 1),
+            TxOut {
+                value: 10_000,
+                script_pubkey: Script::new(),
+            },
+        ),
+        (
+            OutPoint::new(h!("tx1"), 2),
+            TxOut {
+                value: 20_000,
+                script_pubkey: Script::new(),
+            },
+        ),
+    ];
+
+    // Another (OutPoint, TxOut) tupple to be used as update as partial transaction.
+    let update_ops = [(
+        OutPoint::new(h!("tx2"), 0),
+        TxOut {
+            value: 20_000,
+            script_pubkey: Script::new(),
+        },
+    )];
+
+    // One full transaction to be included in the update
+    let update_txs = Transaction {
+        version: 0x01,
+        lock_time: PackedLockTime(0),
+        input: vec![TxIn {
+            previous_output: OutPoint::null(),
+            ..Default::default()
+        }],
+        output: vec![TxOut {
+            value: 30_000,
+            script_pubkey: Script::new(),
+        }],
+    };
+
+    // Conf anchor used to mark the full transaction as confirmed.
+    let conf_anchor = ChainPosition::Confirmed(BlockId {
+        height: 100,
+        hash: h!("random blockhash"),
+    });
+
+    // Unconfirmed anchor to mark the partial transactions as unconfirmed
+    let unconf_anchor = ChainPosition::<BlockId>::Unconfirmed(1000000);
+
+    // Make the original graph
+    let mut graph = {
+        let mut graph = TxGraph::<ChainPosition<BlockId>>::default();
+        for (outpoint, txout) in &original_ops {
+            assert_eq!(
+                graph.insert_txout(*outpoint, txout.clone()),
+                Additions {
+                    txouts: [(*outpoint, txout.clone())].into(),
+                    ..Default::default()
+                }
+            );
+        }
+        graph
+    };
+
+    // Make the update graph
+    let update = {
+        let mut graph = TxGraph::default();
+        for (outpoint, txout) in &update_ops {
+            // Insert partials transactions
+            assert_eq!(
+                graph.insert_txout(*outpoint, txout.clone()),
+                Additions {
+                    txouts: [(*outpoint, txout.clone())].into(),
+                    ..Default::default()
+                }
+            );
+            // Mark them unconfirmed.
+            assert_eq!(
+                graph.insert_anchor(outpoint.txid, unconf_anchor),
+                Additions {
+                    txs: [].into(),
+                    txouts: [].into(),
+                    anchors: [(unconf_anchor, outpoint.txid)].into(),
+                    last_seen: [].into()
+                }
+            );
+            // Mark them last seen at.
+            assert_eq!(
+                graph.insert_seen_at(outpoint.txid, 1000000),
+                Additions {
+                    txs: [].into(),
+                    txouts: [].into(),
+                    anchors: [].into(),
+                    last_seen: [(outpoint.txid, 1000000)].into()
+                }
+            );
+        }
+        // Insert the full transaction
+        assert_eq!(
+            graph.insert_tx(update_txs.clone()),
+            Additions {
+                txs: [update_txs.clone()].into(),
+                ..Default::default()
+            }
+        );
+
+        // Mark it as confirmed.
+        assert_eq!(
+            graph.insert_anchor(update_txs.txid(), conf_anchor),
+            Additions {
+                txs: [].into(),
+                txouts: [].into(),
+                anchors: [(conf_anchor, update_txs.txid())].into(),
+                last_seen: [].into()
+            }
+        );
+        graph
+    };
+
+    // Check the resulting addition.
+    let additions = graph.determine_additions(&update);
+
+    assert_eq!(
+        additions,
+        Additions {
+            txs: [update_txs.clone()].into(),
+            txouts: update_ops.into(),
+            anchors: [(conf_anchor, update_txs.txid()), (unconf_anchor, h!("tx2"))].into(),
+            last_seen: [(h!("tx2"), 1000000)].into()
+        }
+    );
+
+    // Apply addition and check the new graph counts.
+    graph.apply_additions(additions);
+    assert_eq!(graph.all_txouts().count(), 4);
+    assert_eq!(graph.full_txs().count(), 1);
+    assert_eq!(graph.floating_txouts().count(), 3);
+
+    // Check TxOuts are fetched correctly from the graph.
+    assert_eq!(
+        graph.tx_outputs(h!("tx1")).expect("should exists"),
+        [
+            (
+                1u32,
+                &TxOut {
+                    value: 10_000,
+                    script_pubkey: Script::new(),
+                }
+            ),
+            (
+                2u32,
+                &TxOut {
+                    value: 20_000,
+                    script_pubkey: Script::new(),
+                }
+            )
+        ]
+        .into()
+    );
+
+    assert_eq!(
+        graph.tx_outputs(update_txs.txid()).expect("should exists"),
+        [(
+            0u32,
+            &TxOut {
+                value: 30_000,
+                script_pubkey: Script::new()
+            }
+        )]
+        .into()
+    );
+}
+
+#[test]
+fn insert_tx_graph_doesnt_count_coinbase_as_spent() {
+    let tx = Transaction {
+        version: 0x01,
+        lock_time: PackedLockTime(0),
+        input: vec![TxIn {
+            previous_output: OutPoint::null(),
+            ..Default::default()
+        }],
+        output: vec![],
+    };
+
+    let mut graph = TxGraph::<()>::default();
+    let _ = graph.insert_tx(tx);
+    assert!(graph.outspends(OutPoint::null()).is_empty());
+    assert!(graph.tx_spends(Txid::all_zeros()).next().is_none());
+}
+
+#[test]
+fn insert_tx_graph_keeps_track_of_spend() {
+    let tx1 = Transaction {
+        version: 0x01,
+        lock_time: PackedLockTime(0),
+        input: vec![],
+        output: vec![TxOut::default()],
+    };
+
+    let op = OutPoint {
+        txid: tx1.txid(),
+        vout: 0,
+    };
+
+    let tx2 = Transaction {
+        version: 0x01,
+        lock_time: PackedLockTime(0),
+        input: vec![TxIn {
+            previous_output: op,
+            ..Default::default()
+        }],
+        output: vec![],
+    };
+
+    let mut graph1 = TxGraph::<()>::default();
+    let mut graph2 = TxGraph::<()>::default();
+
+    // insert in different order
+    let _ = graph1.insert_tx(tx1.clone());
+    let _ = graph1.insert_tx(tx2.clone());
+
+    let _ = graph2.insert_tx(tx2.clone());
+    let _ = graph2.insert_tx(tx1);
+
+    assert_eq!(
+        graph1.outspends(op),
+        &iter::once(tx2.txid()).collect::<HashSet<_>>()
+    );
+    assert_eq!(graph2.outspends(op), graph1.outspends(op));
+}
+
+#[test]
+fn insert_tx_can_retrieve_full_tx_from_graph() {
+    let tx = Transaction {
+        version: 0x01,
+        lock_time: PackedLockTime(0),
+        input: vec![TxIn {
+            previous_output: OutPoint::null(),
+            ..Default::default()
+        }],
+        output: vec![TxOut::default()],
+    };
+
+    let mut graph = TxGraph::<()>::default();
+    let _ = graph.insert_tx(tx.clone());
+    assert_eq!(graph.get_tx(tx.txid()), Some(&tx));
+}
+
+#[test]
+fn insert_tx_displaces_txouts() {
+    let mut tx_graph = TxGraph::<()>::default();
+    let tx = Transaction {
+        version: 0x01,
+        lock_time: PackedLockTime(0),
+        input: vec![],
+        output: vec![TxOut {
+            value: 42_000,
+            script_pubkey: Script::default(),
+        }],
+    };
+
+    let _ = tx_graph.insert_txout(
+        OutPoint {
+            txid: tx.txid(),
+            vout: 0,
+        },
+        TxOut {
+            value: 1_337_000,
+            script_pubkey: Script::default(),
+        },
+    );
+
+    let _ = tx_graph.insert_txout(
+        OutPoint {
+            txid: tx.txid(),
+            vout: 0,
+        },
+        TxOut {
+            value: 1_000_000_000,
+            script_pubkey: Script::default(),
+        },
+    );
+
+    let _additions = tx_graph.insert_tx(tx.clone());
+
+    assert_eq!(
+        tx_graph
+            .get_txout(OutPoint {
+                txid: tx.txid(),
+                vout: 0
+            })
+            .unwrap()
+            .value,
+        42_000
+    );
+    assert_eq!(
+        tx_graph.get_txout(OutPoint {
+            txid: tx.txid(),
+            vout: 1
+        }),
+        None
+    );
+}
+
+#[test]
+fn insert_txout_does_not_displace_tx() {
+    let mut tx_graph = TxGraph::<()>::default();
+    let tx = Transaction {
+        version: 0x01,
+        lock_time: PackedLockTime(0),
+        input: vec![],
+        output: vec![TxOut {
+            value: 42_000,
+            script_pubkey: Script::default(),
+        }],
+    };
+
+    let _additions = tx_graph.insert_tx(tx.clone());
+
+    let _ = tx_graph.insert_txout(
+        OutPoint {
+            txid: tx.txid(),
+            vout: 0,
+        },
+        TxOut {
+            value: 1_337_000,
+            script_pubkey: Script::default(),
+        },
+    );
+
+    let _ = tx_graph.insert_txout(
+        OutPoint {
+            txid: tx.txid(),
+            vout: 0,
+        },
+        TxOut {
+            value: 1_000_000_000,
+            script_pubkey: Script::default(),
+        },
+    );
+
+    assert_eq!(
+        tx_graph
+            .get_txout(OutPoint {
+                txid: tx.txid(),
+                vout: 0
+            })
+            .unwrap()
+            .value,
+        42_000
+    );
+    assert_eq!(
+        tx_graph.get_txout(OutPoint {
+            txid: tx.txid(),
+            vout: 1
+        }),
+        None
+    );
+}
+
+#[test]
+fn test_calculate_fee() {
+    let mut graph = TxGraph::<()>::default();
+    let intx1 = Transaction {
+        version: 0x01,
+        lock_time: PackedLockTime(0),
+        input: vec![],
+        output: vec![TxOut {
+            value: 100,
+            ..Default::default()
+        }],
+    };
+    let intx2 = Transaction {
+        version: 0x02,
+        lock_time: PackedLockTime(0),
+        input: vec![],
+        output: vec![TxOut {
+            value: 200,
+            ..Default::default()
+        }],
+    };
+
+    let intxout1 = (
+        OutPoint {
+            txid: h!("dangling output"),
+            vout: 0,
+        },
+        TxOut {
+            value: 300,
+            ..Default::default()
+        },
+    );
+
+    let _ = graph.insert_tx(intx1.clone());
+    let _ = graph.insert_tx(intx2.clone());
+    let _ = graph.insert_txout(intxout1.0, intxout1.1);
+
+    let mut tx = Transaction {
+        version: 0x01,
+        lock_time: PackedLockTime(0),
+        input: vec![
+            TxIn {
+                previous_output: OutPoint {
+                    txid: intx1.txid(),
+                    vout: 0,
+                },
+                ..Default::default()
+            },
+            TxIn {
+                previous_output: OutPoint {
+                    txid: intx2.txid(),
+                    vout: 0,
+                },
+                ..Default::default()
+            },
+            TxIn {
+                previous_output: intxout1.0,
+                ..Default::default()
+            },
+        ],
+        output: vec![TxOut {
+            value: 500,
+            ..Default::default()
+        }],
+    };
+
+    assert_eq!(graph.calculate_fee(&tx), Some(100));
+
+    tx.input.remove(2);
+
+    // fee would be negative
+    assert_eq!(graph.calculate_fee(&tx), Some(-200));
+
+    // If we have an unknown outpoint, fee should return None.
+    tx.input.push(TxIn {
+        previous_output: OutPoint {
+            txid: h!("unknown_txid"),
+            vout: 0,
+        },
+        ..Default::default()
+    });
+    assert_eq!(graph.calculate_fee(&tx), None);
+}
+
+#[test]
+fn test_calculate_fee_on_coinbase() {
+    let tx = Transaction {
+        version: 0x01,
+        lock_time: PackedLockTime(0),
+        input: vec![TxIn {
+            previous_output: OutPoint::null(),
+            ..Default::default()
+        }],
+        output: vec![TxOut::default()],
+    };
+
+    let graph = TxGraph::<()>::default();
+
+    assert_eq!(graph.calculate_fee(&tx), Some(0));
+}
+
+#[test]
+fn test_conflicting_descendants() {
+    let previous_output = OutPoint::new(h!("op"), 2);
+
+    // tx_a spends previous_output
+    let tx_a = Transaction {
+        input: vec![TxIn {
+            previous_output,
+            ..TxIn::default()
+        }],
+        output: vec![TxOut::default()],
+        ..common::new_tx(0)
+    };
+
+    // tx_a2 spends previous_output and conflicts with tx_a
+    let tx_a2 = Transaction {
+        input: vec![TxIn {
+            previous_output,
+            ..TxIn::default()
+        }],
+        output: vec![TxOut::default(), TxOut::default()],
+        ..common::new_tx(1)
+    };
+
+    // tx_b spends tx_a
+    let tx_b = Transaction {
+        input: vec![TxIn {
+            previous_output: OutPoint::new(tx_a.txid(), 0),
+            ..TxIn::default()
+        }],
+        output: vec![TxOut::default()],
+        ..common::new_tx(2)
+    };
+
+    let txid_a = tx_a.txid();
+    let txid_b = tx_b.txid();
+
+    let mut graph = TxGraph::<()>::default();
+    let _ = graph.insert_tx(tx_a);
+    let _ = graph.insert_tx(tx_b);
+
+    assert_eq!(
+        graph
+            .walk_conflicts(&tx_a2, |depth, txid| Some((depth, txid)))
+            .collect::<Vec<_>>(),
+        vec![(0_usize, txid_a), (1_usize, txid_b),],
+    );
+}
+
+#[test]
+fn test_descendants_no_repeat() {
+    let tx_a = Transaction {
+        output: vec![TxOut::default(), TxOut::default(), TxOut::default()],
+        ..common::new_tx(0)
+    };
+
+    let txs_b = (0..3)
+        .map(|vout| Transaction {
+            input: vec![TxIn {
+                previous_output: OutPoint::new(tx_a.txid(), vout),
+                ..TxIn::default()
+            }],
+            output: vec![TxOut::default()],
+            ..common::new_tx(1)
+        })
+        .collect::<Vec<_>>();
+
+    let txs_c = (0..2)
+        .map(|vout| Transaction {
+            input: vec![TxIn {
+                previous_output: OutPoint::new(txs_b[vout as usize].txid(), vout),
+                ..TxIn::default()
+            }],
+            output: vec![TxOut::default()],
+            ..common::new_tx(2)
+        })
+        .collect::<Vec<_>>();
+
+    let tx_d = Transaction {
+        input: vec![
+            TxIn {
+                previous_output: OutPoint::new(txs_c[0].txid(), 0),
+                ..TxIn::default()
+            },
+            TxIn {
+                previous_output: OutPoint::new(txs_c[1].txid(), 0),
+                ..TxIn::default()
+            },
+        ],
+        output: vec![TxOut::default()],
+        ..common::new_tx(3)
+    };
+
+    let tx_e = Transaction {
+        input: vec![TxIn {
+            previous_output: OutPoint::new(tx_d.txid(), 0),
+            ..TxIn::default()
+        }],
+        output: vec![TxOut::default()],
+        ..common::new_tx(4)
+    };
+
+    let txs_not_connected = (10..20)
+        .map(|v| Transaction {
+            input: vec![TxIn {
+                previous_output: OutPoint::new(h!("tx_does_not_exist"), v),
+                ..TxIn::default()
+            }],
+            output: vec![TxOut::default()],
+            ..common::new_tx(v)
+        })
+        .collect::<Vec<_>>();
+
+    let mut graph = TxGraph::<()>::default();
+    let mut expected_txids = BTreeSet::new();
+
+    // these are NOT descendants of `tx_a`
+    for tx in txs_not_connected {
+        let _ = graph.insert_tx(tx.clone());
+    }
+
+    // these are the expected descendants of `tx_a`
+    for tx in txs_b
+        .iter()
+        .chain(&txs_c)
+        .chain(core::iter::once(&tx_d))
+        .chain(core::iter::once(&tx_e))
+    {
+        let _ = graph.insert_tx(tx.clone());
+        assert!(expected_txids.insert(tx.txid()));
+    }
+
+    let descendants = graph
+        .walk_descendants(tx_a.txid(), |_, txid| Some(txid))
+        .collect::<Vec<_>>();
+
+    assert_eq!(descendants.len(), expected_txids.len());
+
+    for txid in descendants {
+        assert!(expected_txids.remove(&txid));
+    }
+    assert!(expected_txids.is_empty());
+}
+
+#[test]
+fn test_chain_spends() {
+    let local_chain: LocalChain = (0..=100)
+        .map(|ht| (ht, BlockHash::hash(format!("Block Hash {}", ht).as_bytes())))
+        .collect::<BTreeMap<u32, BlockHash>>()
+        .into();
+    let tip = local_chain.tip().expect("must have tip");
+
+    // The parent tx contains 2 outputs. Which are spent by one confirmed and one unconfirmed tx.
+    // The parent tx is confirmed at block 95.
+    let tx_0 = Transaction {
+        input: vec![],
+        output: vec![
+            TxOut {
+                value: 10_000,
+                script_pubkey: Script::new(),
+            },
+            TxOut {
+                value: 20_000,
+                script_pubkey: Script::new(),
+            },
+        ],
+        ..common::new_tx(0)
+    };
+
+    // The first confirmed transaction spends vout: 0. And is confirmed at block 98.
+    let tx_1 = Transaction {
+        input: vec![TxIn {
+            previous_output: OutPoint::new(tx_0.txid(), 0),
+            ..TxIn::default()
+        }],
+        output: vec![
+            TxOut {
+                value: 5_000,
+                script_pubkey: Script::new(),
+            },
+            TxOut {
+                value: 5_000,
+                script_pubkey: Script::new(),
+            },
+        ],
+        ..common::new_tx(0)
+    };
+
+    // The second transactions spends vout:1, and is unconfirmed.
+    let tx_2 = Transaction {
+        input: vec![TxIn {
+            previous_output: OutPoint::new(tx_0.txid(), 1),
+            ..TxIn::default()
+        }],
+        output: vec![
+            TxOut {
+                value: 10_000,
+                script_pubkey: Script::new(),
+            },
+            TxOut {
+                value: 10_000,
+                script_pubkey: Script::new(),
+            },
+        ],
+        ..common::new_tx(0)
+    };
+
+    let mut graph = TxGraph::<ConfirmationHeightAnchor>::default();
+
+    let _ = graph.insert_tx(tx_0.clone());
+    let _ = graph.insert_tx(tx_1.clone());
+    let _ = graph.insert_tx(tx_2.clone());
+
+    [95, 98]
+        .iter()
+        .zip([&tx_0, &tx_1].into_iter())
+        .for_each(|(ht, tx)| {
+            let _ = graph.insert_anchor(
+                tx.txid(),
+                ConfirmationHeightAnchor {
+                    anchor_block: tip,
+                    confirmation_height: *ht,
+                },
+            );
+        });
+
+    // Assert that confirmed spends are returned correctly.
+    assert_eq!(
+        graph.get_chain_spend(&local_chain, tip, OutPoint::new(tx_0.txid(), 0)),
+        Some((
+            ChainPosition::Confirmed(&ConfirmationHeightAnchor {
+                anchor_block: tip,
+                confirmation_height: 98
+            }),
+            tx_1.txid(),
+        )),
+    );
+
+    // Check if chain position is returned correctly.
+    assert_eq!(
+        graph.get_chain_position(&local_chain, tip, tx_0.txid()),
+        // Some(ObservedAs::Confirmed(&local_chain.get_block(95).expect("block expected"))),
+        Some(ChainPosition::Confirmed(&ConfirmationHeightAnchor {
+            anchor_block: tip,
+            confirmation_height: 95
+        }))
+    );
+
+    // Even if unconfirmed tx has a last_seen of 0, it can still be part of a chain spend.
+    assert_eq!(
+        graph.get_chain_spend(&local_chain, tip, OutPoint::new(tx_0.txid(), 1)),
+        Some((ChainPosition::Unconfirmed(0), tx_2.txid())),
+    );
+
+    // Mark the unconfirmed as seen and check correct ObservedAs status is returned.
+    let _ = graph.insert_seen_at(tx_2.txid(), 1234567);
+
+    // Check chain spend returned correctly.
+    assert_eq!(
+        graph
+            .get_chain_spend(&local_chain, tip, OutPoint::new(tx_0.txid(), 1))
+            .unwrap(),
+        (ChainPosition::Unconfirmed(1234567), tx_2.txid())
+    );
+
+    // A conflicting transaction that conflicts with tx_1.
+    let tx_1_conflict = Transaction {
+        input: vec![TxIn {
+            previous_output: OutPoint::new(tx_0.txid(), 0),
+            ..Default::default()
+        }],
+        ..common::new_tx(0)
+    };
+    let _ = graph.insert_tx(tx_1_conflict.clone());
+
+    // Because this tx conflicts with an already confirmed transaction, chain position should return none.
+    assert!(graph
+        .get_chain_position(&local_chain, tip, tx_1_conflict.txid())
+        .is_none());
+
+    // Another conflicting tx that conflicts with tx_2.
+    let tx_2_conflict = Transaction {
+        input: vec![TxIn {
+            previous_output: OutPoint::new(tx_0.txid(), 1),
+            ..Default::default()
+        }],
+        ..common::new_tx(0)
+    };
+
+    // Insert in graph and mark it as seen.
+    let _ = graph.insert_tx(tx_2_conflict.clone());
+    let _ = graph.insert_seen_at(tx_2_conflict.txid(), 1234568);
+
+    // This should return a valid observation with correct last seen.
+    assert_eq!(
+        graph
+            .get_chain_position(&local_chain, tip, tx_2_conflict.txid())
+            .expect("position expected"),
+        ChainPosition::Unconfirmed(1234568)
+    );
+
+    // Chain_spend now catches the new transaction as the spend.
+    assert_eq!(
+        graph
+            .get_chain_spend(&local_chain, tip, OutPoint::new(tx_0.txid(), 1))
+            .expect("expect observation"),
+        (ChainPosition::Unconfirmed(1234568), tx_2_conflict.txid())
+    );
+
+    // Chain position of the `tx_2` is now none, as it is older than `tx_2_conflict`
+    assert!(graph
+        .get_chain_position(&local_chain, tip, tx_2.txid())
+        .is_none());
+}
+
+/// Ensure that `last_seen` values only increase during [`Append::append`].
+#[test]
+fn test_additions_last_seen_append() {
+    let txid: Txid = h!("test txid");
+
+    let test_cases: &[(Option<u64>, Option<u64>)] = &[
+        (Some(5), Some(6)),
+        (Some(5), Some(5)),
+        (Some(6), Some(5)),
+        (None, Some(5)),
+        (Some(5), None),
+    ];
+
+    for (original_ls, update_ls) in test_cases {
+        let mut original = Additions::<()> {
+            last_seen: original_ls.map(|ls| (txid, ls)).into_iter().collect(),
+            ..Default::default()
+        };
+        let update = Additions::<()> {
+            last_seen: update_ls.map(|ls| (txid, ls)).into_iter().collect(),
+            ..Default::default()
+        };
+
+        original.append(update);
+        assert_eq!(
+            &original.last_seen.get(&txid).cloned(),
+            Ord::max(original_ls, update_ls),
+        );
+    }
+}

crates/electrum/Cargo.toml

@@ -0,0 +1,16 @@
+[package]
+name = "bdk_electrum"
+version = "0.3.0"
+edition = "2021"
+homepage = "https://bitcoindevkit.org"
+repository = "https://github.com/bitcoindevkit/bdk"
+documentation = "https://docs.rs/bdk_electrum"
+description = "Fetch data from electrum in the form BDK accepts"
+license = "MIT OR Apache-2.0"
+readme = "README.md"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+bdk_chain = { path = "../chain", version = "0.5.0", features = ["serde", "miniscript"] }
+electrum-client = { version = "0.12" }

crates/electrum/README.md

@@ -0,0 +1,3 @@
+# BDK Electrum
+
+BDK Electrum client library for updating the keychain tracker.

crates/electrum/src/electrum_ext.rs

@@ -0,0 +1,486 @@
+use bdk_chain::{
+    bitcoin::{hashes::hex::FromHex, BlockHash, OutPoint, Script, Transaction, Txid},
+    keychain::LocalUpdate,
+    local_chain::LocalChain,
+    tx_graph::{self, TxGraph},
+    Anchor, BlockId, ConfirmationHeightAnchor, ConfirmationTimeAnchor,
+};
+use electrum_client::{Client, ElectrumApi, Error};
+use std::{
+    collections::{BTreeMap, BTreeSet, HashMap, HashSet},
+    fmt::Debug,
+};
+
+#[derive(Debug, Clone)]
+pub struct ElectrumUpdate<K, A> {
+    pub graph_update: HashMap<Txid, BTreeSet<A>>,
+    pub chain_update: LocalChain,
+    pub keychain_update: BTreeMap<K, u32>,
+}
+
+impl<K, A> Default for ElectrumUpdate<K, A> {
+    fn default() -> Self {
+        Self {
+            graph_update: Default::default(),
+            chain_update: Default::default(),
+            keychain_update: Default::default(),
+        }
+    }
+}
+
+impl<K, A: Anchor> ElectrumUpdate<K, A> {
+    pub fn missing_full_txs<A2>(&self, graph: &TxGraph<A2>) -> Vec<Txid> {
+        self.graph_update
+            .keys()
+            .filter(move |&&txid| graph.as_ref().get_tx(txid).is_none())
+            .cloned()
+            .collect()
+    }
+
+    pub fn finalize(
+        self,
+        client: &Client,
+        seen_at: Option<u64>,
+        missing: Vec<Txid>,
+    ) -> Result<LocalUpdate<K, A>, Error> {
+        let new_txs = client.batch_transaction_get(&missing)?;
+        let mut graph_update = TxGraph::<A>::new(new_txs);
+        for (txid, anchors) in self.graph_update {
+            if let Some(seen_at) = seen_at {
+                let _ = graph_update.insert_seen_at(txid, seen_at);
+            }
+            for anchor in anchors {
+                let _ = graph_update.insert_anchor(txid, anchor);
+            }
+        }
+        Ok(LocalUpdate {
+            keychain: self.keychain_update,
+            graph: graph_update,
+            chain: self.chain_update,
+        })
+    }
+}
+
+impl<K> ElectrumUpdate<K, ConfirmationHeightAnchor> {
+    /// Finalizes the [`ElectrumUpdate`] with `new_txs` and anchors of type
+    /// [`ConfirmationTimeAnchor`].
+    ///
+    /// **Note:** The confirmation time might not be precisely correct if there has been a reorg.
+    /// Electrum's API intends that we use the merkle proof API, we should change `bdk_electrum` to
+    /// use it.
+    pub fn finalize_as_confirmation_time(
+        self,
+        client: &Client,
+        seen_at: Option<u64>,
+        missing: Vec<Txid>,
+    ) -> Result<LocalUpdate<K, ConfirmationTimeAnchor>, Error> {
+        let update = self.finalize(client, seen_at, missing)?;
+
+        let relevant_heights = {
+            let mut visited_heights = HashSet::new();
+            update
+                .graph
+                .all_anchors()
+                .iter()
+                .map(|(a, _)| a.confirmation_height_upper_bound())
+                .filter(move |&h| visited_heights.insert(h))
+                .collect::<Vec<_>>()
+        };
+
+        let height_to_time = relevant_heights
+            .clone()
+            .into_iter()
+            .zip(
+                client
+                    .batch_block_header(relevant_heights)?
+                    .into_iter()
+                    .map(|bh| bh.time as u64),
+            )
+            .collect::<HashMap<u32, u64>>();
+
+        let graph_additions = {
+            let old_additions = TxGraph::default().determine_additions(&update.graph);
+            tx_graph::Additions {
+                txs: old_additions.txs,
+                txouts: old_additions.txouts,
+                last_seen: old_additions.last_seen,
+                anchors: old_additions
+                    .anchors
+                    .into_iter()
+                    .map(|(height_anchor, txid)| {
+                        let confirmation_height = height_anchor.confirmation_height;
+                        let confirmation_time = height_to_time[&confirmation_height];
+                        let time_anchor = ConfirmationTimeAnchor {
+                            anchor_block: height_anchor.anchor_block,
+                            confirmation_height,
+                            confirmation_time,
+                        };
+                        (time_anchor, txid)
+                    })
+                    .collect(),
+            }
+        };
+
+        Ok(LocalUpdate {
+            keychain: update.keychain,
+            graph: {
+                let mut graph = TxGraph::default();
+                graph.apply_additions(graph_additions);
+                graph
+            },
+            chain: update.chain,
+        })
+    }
+}
+
+pub trait ElectrumExt<A> {
+    fn get_tip(&self) -> Result<(u32, BlockHash), Error>;
+
+    fn scan<K: Ord + Clone>(
+        &self,
+        local_chain: &BTreeMap<u32, BlockHash>,
+        keychain_spks: BTreeMap<K, impl IntoIterator<Item = (u32, Script)>>,
+        txids: impl IntoIterator<Item = Txid>,
+        outpoints: impl IntoIterator<Item = OutPoint>,
+        stop_gap: usize,
+        batch_size: usize,
+    ) -> Result<ElectrumUpdate<K, A>, Error>;
+
+    fn scan_without_keychain(
+        &self,
+        local_chain: &BTreeMap<u32, BlockHash>,
+        misc_spks: impl IntoIterator<Item = Script>,
+        txids: impl IntoIterator<Item = Txid>,
+        outpoints: impl IntoIterator<Item = OutPoint>,
+        batch_size: usize,
+    ) -> Result<ElectrumUpdate<(), A>, Error> {
+        let spk_iter = misc_spks
+            .into_iter()
+            .enumerate()
+            .map(|(i, spk)| (i as u32, spk));
+
+        self.scan(
+            local_chain,
+            [((), spk_iter)].into(),
+            txids,
+            outpoints,
+            usize::MAX,
+            batch_size,
+        )
+    }
+}
+
+impl ElectrumExt<ConfirmationHeightAnchor> for Client {
+    fn get_tip(&self) -> Result<(u32, BlockHash), Error> {
+        // TODO: unsubscribe when added to the client, or is there a better call to use here?
+        self.block_headers_subscribe()
+            .map(|data| (data.height as u32, data.header.block_hash()))
+    }
+
+    fn scan<K: Ord + Clone>(
+        &self,
+        local_chain: &BTreeMap<u32, BlockHash>,
+        keychain_spks: BTreeMap<K, impl IntoIterator<Item = (u32, Script)>>,
+        txids: impl IntoIterator<Item = Txid>,
+        outpoints: impl IntoIterator<Item = OutPoint>,
+        stop_gap: usize,
+        batch_size: usize,
+    ) -> Result<ElectrumUpdate<K, ConfirmationHeightAnchor>, Error> {
+        let mut request_spks = keychain_spks
+            .into_iter()
+            .map(|(k, s)| (k, s.into_iter()))
+            .collect::<BTreeMap<K, _>>();
+        let mut scanned_spks = BTreeMap::<(K, u32), (Script, bool)>::new();
+
+        let txids = txids.into_iter().collect::<Vec<_>>();
+        let outpoints = outpoints.into_iter().collect::<Vec<_>>();
+
+        let update = loop {
+            let mut update = ElectrumUpdate::<K, ConfirmationHeightAnchor> {
+                chain_update: prepare_chain_update(self, local_chain)?,
+                ..Default::default()
+            };
+            let anchor_block = update
+                .chain_update
+                .tip()
+                .expect("must have atleast one block");
+
+            if !request_spks.is_empty() {
+                if !scanned_spks.is_empty() {
+                    scanned_spks.append(&mut populate_with_spks(
+                        self,
+                        anchor_block,
+                        &mut update,
+                        &mut scanned_spks
+                            .iter()
+                            .map(|(i, (spk, _))| (i.clone(), spk.clone())),
+                        stop_gap,
+                        batch_size,
+                    )?);
+                }
+                for (keychain, keychain_spks) in &mut request_spks {
+                    scanned_spks.extend(
+                        populate_with_spks(
+                            self,
+                            anchor_block,
+                            &mut update,
+                            keychain_spks,
+                            stop_gap,
+                            batch_size,
+                        )?
+                        .into_iter()
+                        .map(|(spk_i, spk)| ((keychain.clone(), spk_i), spk)),
+                    );
+                }
+            }
+
+            populate_with_txids(self, anchor_block, &mut update, &mut txids.iter().cloned())?;
+
+            let _txs = populate_with_outpoints(
+                self,
+                anchor_block,
+                &mut update,
+                &mut outpoints.iter().cloned(),
+            )?;
+
+            // check for reorgs during scan process
+            let server_blockhash = self
+                .block_header(anchor_block.height as usize)?
+                .block_hash();
+            if anchor_block.hash != server_blockhash {
+                continue; // reorg
+            }
+
+            update.keychain_update = request_spks
+                .into_keys()
+                .filter_map(|k| {
+                    scanned_spks
+                        .range((k.clone(), u32::MIN)..=(k.clone(), u32::MAX))
+                        .rev()
+                        .find(|(_, (_, active))| *active)
+                        .map(|((_, i), _)| (k, *i))
+                })
+                .collect::<BTreeMap<_, _>>();
+            break update;
+        };
+
+        Ok(update)
+    }
+}
+
+/// Prepare an update "template" based on the checkpoints of the `local_chain`.
+fn prepare_chain_update(
+    client: &Client,
+    local_chain: &BTreeMap<u32, BlockHash>,
+) -> Result<LocalChain, Error> {
+    let mut update = LocalChain::default();
+
+    // Find the local chain block that is still there so our update can connect to the local chain.
+    for (&existing_height, &existing_hash) in local_chain.iter().rev() {
+        // TODO: a batch request may be safer, as a reorg that happens when we are obtaining
+        //       `block_header`s will result in inconsistencies
+        let current_hash = client.block_header(existing_height as usize)?.block_hash();
+        let _ = update
+            .insert_block(BlockId {
+                height: existing_height,
+                hash: current_hash,
+            })
+            .expect("This never errors because we are working with a fresh chain");
+
+        if current_hash == existing_hash {
+            break;
+        }
+    }
+
+    // Insert the new tip so new transactions will be accepted into the sparsechain.
+    let tip = {
+        let (height, hash) = crate::get_tip(client)?;
+        BlockId { height, hash }
+    };
+    if update.insert_block(tip).is_err() {
+        // There has been a re-org before we even begin scanning addresses.
+        // Just recursively call (this should never happen).
+        return prepare_chain_update(client, local_chain);
+    }
+
+    Ok(update)
+}
+
+fn determine_tx_anchor(
+    anchor_block: BlockId,
+    raw_height: i32,
+    txid: Txid,
+) -> Option<ConfirmationHeightAnchor> {
+    // The electrum API has a weird quirk where an unconfirmed transaction is presented with a
+    // height of 0. To avoid invalid representation in our data structures, we manually set
+    // transactions residing in the genesis block to have height 0, then interpret a height of 0 as
+    // unconfirmed for all other transactions.
+    if txid
+        == Txid::from_hex("4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b")
+            .expect("must deserialize genesis coinbase txid")
+    {
+        return Some(ConfirmationHeightAnchor {
+            anchor_block,
+            confirmation_height: 0,
+        });
+    }
+    match raw_height {
+        h if h <= 0 => {
+            debug_assert!(h == 0 || h == -1, "unexpected height ({}) from electrum", h);
+            None
+        }
+        h => {
+            let h = h as u32;
+            if h > anchor_block.height {
+                None
+            } else {
+                Some(ConfirmationHeightAnchor {
+                    anchor_block,
+                    confirmation_height: h,
+                })
+            }
+        }
+    }
+}
+
+fn populate_with_outpoints<K>(
+    client: &Client,
+    anchor_block: BlockId,
+    update: &mut ElectrumUpdate<K, ConfirmationHeightAnchor>,
+    outpoints: &mut impl Iterator<Item = OutPoint>,
+) -> Result<HashMap<Txid, Transaction>, Error> {
+    let mut full_txs = HashMap::new();
+    for outpoint in outpoints {
+        let txid = outpoint.txid;
+        let tx = client.transaction_get(&txid)?;
+        debug_assert_eq!(tx.txid(), txid);
+        let txout = match tx.output.get(outpoint.vout as usize) {
+            Some(txout) => txout,
+            None => continue,
+        };
+        // attempt to find the following transactions (alongside their chain positions), and
+        // add to our sparsechain `update`:
+        let mut has_residing = false; // tx in which the outpoint resides
+        let mut has_spending = false; // tx that spends the outpoint
+        for res in client.script_get_history(&txout.script_pubkey)? {
+            if has_residing && has_spending {
+                break;
+            }
+
+            if res.tx_hash == txid {
+                if has_residing {
+                    continue;
+                }
+                has_residing = true;
+                full_txs.insert(res.tx_hash, tx.clone());
+            } else {
+                if has_spending {
+                    continue;
+                }
+                let res_tx = match full_txs.get(&res.tx_hash) {
+                    Some(tx) => tx,
+                    None => {
+                        let res_tx = client.transaction_get(&res.tx_hash)?;
+                        full_txs.insert(res.tx_hash, res_tx);
+                        full_txs.get(&res.tx_hash).expect("just inserted")
+                    }
+                };
+                has_spending = res_tx
+                    .input
+                    .iter()
+                    .any(|txin| txin.previous_output == outpoint);
+                if !has_spending {
+                    continue;
+                }
+            };
+
+            let anchor = determine_tx_anchor(anchor_block, res.height, res.tx_hash);
+
+            let tx_entry = update.graph_update.entry(res.tx_hash).or_default();
+            if let Some(anchor) = anchor {
+                tx_entry.insert(anchor);
+            }
+        }
+    }
+    Ok(full_txs)
+}
+
+fn populate_with_txids<K>(
+    client: &Client,
+    anchor_block: BlockId,
+    update: &mut ElectrumUpdate<K, ConfirmationHeightAnchor>,
+    txids: &mut impl Iterator<Item = Txid>,
+) -> Result<(), Error> {
+    for txid in txids {
+        let tx = match client.transaction_get(&txid) {
+            Ok(tx) => tx,
+            Err(electrum_client::Error::Protocol(_)) => continue,
+            Err(other_err) => return Err(other_err),
+        };
+
+        let spk = tx
+            .output
+            .get(0)
+            .map(|txo| &txo.script_pubkey)
+            .expect("tx must have an output");
+
+        let anchor = match client
+            .script_get_history(spk)?
+            .into_iter()
+            .find(|r| r.tx_hash == txid)
+        {
+            Some(r) => determine_tx_anchor(anchor_block, r.height, txid),
+            None => continue,
+        };
+
+        let tx_entry = update.graph_update.entry(txid).or_default();
+        if let Some(anchor) = anchor {
+            tx_entry.insert(anchor);
+        }
+    }
+    Ok(())
+}
+
+fn populate_with_spks<K, I: Ord + Clone>(
+    client: &Client,
+    anchor_block: BlockId,
+    update: &mut ElectrumUpdate<K, ConfirmationHeightAnchor>,
+    spks: &mut impl Iterator<Item = (I, Script)>,
+    stop_gap: usize,
+    batch_size: usize,
+) -> Result<BTreeMap<I, (Script, bool)>, Error> {
+    let mut unused_spk_count = 0_usize;
+    let mut scanned_spks = BTreeMap::new();
+
+    loop {
+        let spks = (0..batch_size)
+            .map_while(|_| spks.next())
+            .collect::<Vec<_>>();
+        if spks.is_empty() {
+            return Ok(scanned_spks);
+        }
+
+        let spk_histories = client.batch_script_get_history(spks.iter().map(|(_, s)| s))?;
+
+        for ((spk_index, spk), spk_history) in spks.into_iter().zip(spk_histories) {
+            if spk_history.is_empty() {
+                scanned_spks.insert(spk_index, (spk, false));
+                unused_spk_count += 1;
+                if unused_spk_count > stop_gap {
+                    return Ok(scanned_spks);
+                }
+                continue;
+            } else {
+                scanned_spks.insert(spk_index, (spk, true));
+                unused_spk_count = 0;
+            }
+
+            for tx in spk_history {
+                let tx_entry = update.graph_update.entry(tx.tx_hash).or_default();
+                if let Some(anchor) = determine_tx_anchor(anchor_block, tx.height, tx.tx_hash) {
+                    tx_entry.insert(anchor);
+                }
+            }
+        }
+    }
+}

crates/electrum/src/lib.rs

@@ -0,0 +1,35 @@
+//! This crate is used for updating structures of the [`bdk_chain`] crate with data from electrum.
+//!
+//! The star of the show is the [`ElectrumExt::scan`] method, which scans for relevant blockchain
+//! data (via electrum) and outputs an [`ElectrumUpdate`].
+//!
+//! An [`ElectrumUpdate`] only includes `txid`s and no full transactions. The caller is responsible
+//! for obtaining full transactions before applying. This can be done with
+//! these steps:
+//!
+//! 1. Determine which full transactions are missing. The method [`missing_full_txs`] of
+//! [`ElectrumUpdate`] can be used.
+//!
+//! 2. Obtaining the full transactions. To do this via electrum, the method
+//! [`batch_transaction_get`] can be used.
+//!
+//! Refer to [`bdk_electrum_example`] for a complete example.
+//!
+//! [`ElectrumClient::scan`]: ElectrumClient::scan
+//! [`missing_full_txs`]: ElectrumUpdate::missing_full_txs
+//! [`batch_transaction_get`]: ElectrumApi::batch_transaction_get
+//! [`bdk_electrum_example`]: https://github.com/LLFourn/bdk_core_staging/tree/master/bdk_electrum_example
+
+use bdk_chain::bitcoin::BlockHash;
+use electrum_client::{Client, ElectrumApi, Error};
+mod electrum_ext;
+pub use bdk_chain;
+pub use electrum_client;
+pub use electrum_ext::*;
+
+fn get_tip(client: &Client) -> Result<(u32, BlockHash), Error> {
+    // TODO: unsubscribe when added to the client, or is there a better call to use here?
+    client
+        .block_headers_subscribe()
+        .map(|data| (data.height as u32, data.header.block_hash()))
+}

crates/esplora/Cargo.toml

@@ -0,0 +1,29 @@
+[package]
+name = "bdk_esplora"
+version = "0.3.0"
+edition = "2021"
+homepage = "https://bitcoindevkit.org"
+repository = "https://github.com/bitcoindevkit/bdk"
+documentation = "https://docs.rs/bdk_esplora"
+description = "Fetch data from esplora in the form that accepts"
+license = "MIT OR Apache-2.0"
+readme = "README.md"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+bdk_chain = { path = "../chain", version = "0.5.0", default-features = false, features = ["serde", "miniscript"] }
+esplora-client = { version = "0.5", default-features = false }
+async-trait = { version = "0.1.66", optional = true }
+futures = { version = "0.3.26", optional = true }
+
+# use these dependencies if you need to enable their /no-std features
+bitcoin = { version = "0.29", optional = true, default-features = false }
+miniscript = { version = "9.0.0", optional = true, default-features = false }
+
+[features]
+default = ["std", "async-https", "blocking"]
+std = ["bdk_chain/std"]
+async = ["async-trait", "futures", "esplora-client/async"]
+async-https = ["async", "esplora-client/async-https"]
+blocking = ["esplora-client/blocking"]

crates/esplora/README.md

@@ -0,0 +1,33 @@
+# BDK Esplora
+
+BDK Esplora extends [`esplora_client`](crate::esplora_client) to update [`bdk_chain`] structures
+from an Esplora server.
+
+## Usage
+
+There are two versions of the extension trait (blocking and async).
+
+For blocking-only:
+```toml
+bdk_esplora = { version = "0.3", features = ["blocking"] }
+```
+
+For async-only:
+```toml
+bdk_esplora = { version = "0.3", features = ["async"] }
+```
+
+For async-only (with https):
+```toml
+bdk_esplora = { version = "0.3", features = ["async-https"] }
+```
+
+To use the extension traits:
+```rust
+// for blocking
+use bdk_esplora::EsploraExt;
+// for async
+// use bdk_esplora::EsploraAsyncExt;
+```
+
+For full examples, refer to [`example-crates/wallet_esplora`](https://github.com/bitcoindevkit/bdk/tree/master/example-crates/wallet_esplora) (blocking) and [`example-crates/wallet_esplora_async`](https://github.com/bitcoindevkit/bdk/tree/master/example-crates/wallet_esplora_async).

crates/esplora/src/async_ext.rs

@@ -0,0 +1,269 @@
+use async_trait::async_trait;
+use bdk_chain::{
+    bitcoin::{BlockHash, OutPoint, Script, Txid},
+    collections::BTreeMap,
+    keychain::LocalUpdate,
+    BlockId, ConfirmationTimeAnchor,
+};
+use esplora_client::{Error, OutputStatus, TxStatus};
+use futures::{stream::FuturesOrdered, TryStreamExt};
+
+use crate::map_confirmation_time_anchor;
+
+/// Trait to extend [`esplora_client::AsyncClient`] functionality.
+///
+/// This is the async version of [`EsploraExt`]. Refer to
+/// [crate-level documentation] for more.
+///
+/// [`EsploraExt`]: crate::EsploraExt
+/// [crate-level documentation]: crate
+#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
+#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
+pub trait EsploraAsyncExt {
+    /// Scan the blockchain (via esplora) for the data specified and returns a
+    /// [`LocalUpdate<K, ConfirmationTimeAnchor>`].
+    ///
+    /// - `local_chain`: the most recent block hashes present locally
+    /// - `keychain_spks`: keychains that we want to scan transactions for
+    /// - `txids`: transactions for which we want updated [`ConfirmationTimeAnchor`]s
+    /// - `outpoints`: transactions associated with these outpoints (residing, spending) that we
+    ///     want to included in the update
+    ///
+    /// The scan for each keychain stops after a gap of `stop_gap` script pubkeys with no associated
+    /// transactions. `parallel_requests` specifies the max number of HTTP requests to make in
+    /// parallel.
+    #[allow(clippy::result_large_err)] // FIXME
+    async fn scan<K: Ord + Clone + Send>(
+        &self,
+        local_chain: &BTreeMap<u32, BlockHash>,
+        keychain_spks: BTreeMap<
+            K,
+            impl IntoIterator<IntoIter = impl Iterator<Item = (u32, Script)> + Send> + Send,
+        >,
+        txids: impl IntoIterator<IntoIter = impl Iterator<Item = Txid> + Send> + Send,
+        outpoints: impl IntoIterator<IntoIter = impl Iterator<Item = OutPoint> + Send> + Send,
+        stop_gap: usize,
+        parallel_requests: usize,
+    ) -> Result<LocalUpdate<K, ConfirmationTimeAnchor>, Error>;
+
+    /// Convenience method to call [`scan`] without requiring a keychain.
+    ///
+    /// [`scan`]: EsploraAsyncExt::scan
+    #[allow(clippy::result_large_err)] // FIXME
+    async fn scan_without_keychain(
+        &self,
+        local_chain: &BTreeMap<u32, BlockHash>,
+        misc_spks: impl IntoIterator<IntoIter = impl Iterator<Item = Script> + Send> + Send,
+        txids: impl IntoIterator<IntoIter = impl Iterator<Item = Txid> + Send> + Send,
+        outpoints: impl IntoIterator<IntoIter = impl Iterator<Item = OutPoint> + Send> + Send,
+        parallel_requests: usize,
+    ) -> Result<LocalUpdate<(), ConfirmationTimeAnchor>, Error> {
+        self.scan(
+            local_chain,
+            [(
+                (),
+                misc_spks
+                    .into_iter()
+                    .enumerate()
+                    .map(|(i, spk)| (i as u32, spk)),
+            )]
+            .into(),
+            txids,
+            outpoints,
+            usize::MAX,
+            parallel_requests,
+        )
+        .await
+    }
+}
+
+#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
+#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
+impl EsploraAsyncExt for esplora_client::AsyncClient {
+    #[allow(clippy::result_large_err)] // FIXME
+    async fn scan<K: Ord + Clone + Send>(
+        &self,
+        local_chain: &BTreeMap<u32, BlockHash>,
+        keychain_spks: BTreeMap<
+            K,
+            impl IntoIterator<IntoIter = impl Iterator<Item = (u32, Script)> + Send> + Send,
+        >,
+        txids: impl IntoIterator<IntoIter = impl Iterator<Item = Txid> + Send> + Send,
+        outpoints: impl IntoIterator<IntoIter = impl Iterator<Item = OutPoint> + Send> + Send,
+        stop_gap: usize,
+        parallel_requests: usize,
+    ) -> Result<LocalUpdate<K, ConfirmationTimeAnchor>, Error> {
+        let parallel_requests = Ord::max(parallel_requests, 1);
+
+        let (mut update, tip_at_start) = loop {
+            let mut update = LocalUpdate::<K, ConfirmationTimeAnchor>::default();
+
+            for (&height, &original_hash) in local_chain.iter().rev() {
+                let update_block_id = BlockId {
+                    height,
+                    hash: self.get_block_hash(height).await?,
+                };
+                let _ = update
+                    .chain
+                    .insert_block(update_block_id)
+                    .expect("cannot repeat height here");
+                if update_block_id.hash == original_hash {
+                    break;
+                }
+            }
+
+            let tip_at_start = BlockId {
+                height: self.get_height().await?,
+                hash: self.get_tip_hash().await?,
+            };
+
+            if update.chain.insert_block(tip_at_start).is_ok() {
+                break (update, tip_at_start);
+            }
+        };
+
+        for (keychain, spks) in keychain_spks {
+            let mut spks = spks.into_iter();
+            let mut last_active_index = None;
+            let mut empty_scripts = 0;
+            type IndexWithTxs = (u32, Vec<esplora_client::Tx>);
+
+            loop {
+                let futures = (0..parallel_requests)
+                    .filter_map(|_| {
+                        let (index, script) = spks.next()?;
+                        let client = self.clone();
+                        Some(async move {
+                            let mut related_txs = client.scripthash_txs(&script, None).await?;
+
+                            let n_confirmed =
+                                related_txs.iter().filter(|tx| tx.status.confirmed).count();
+                            // esplora pages on 25 confirmed transactions. If there are 25 or more we
+                            // keep requesting to see if there's more.
+                            if n_confirmed >= 25 {
+                                loop {
+                                    let new_related_txs = client
+                                        .scripthash_txs(
+                                            &script,
+                                            Some(related_txs.last().unwrap().txid),
+                                        )
+                                        .await?;
+                                    let n = new_related_txs.len();
+                                    related_txs.extend(new_related_txs);
+                                    // we've reached the end
+                                    if n < 25 {
+                                        break;
+                                    }
+                                }
+                            }
+
+                            Result::<_, esplora_client::Error>::Ok((index, related_txs))
+                        })
+                    })
+                    .collect::<FuturesOrdered<_>>();
+
+                let n_futures = futures.len();
+
+                for (index, related_txs) in futures.try_collect::<Vec<IndexWithTxs>>().await? {
+                    if related_txs.is_empty() {
+                        empty_scripts += 1;
+                    } else {
+                        last_active_index = Some(index);
+                        empty_scripts = 0;
+                    }
+                    for tx in related_txs {
+                        let anchor = map_confirmation_time_anchor(&tx.status, tip_at_start);
+
+                        let _ = update.graph.insert_tx(tx.to_tx());
+                        if let Some(anchor) = anchor {
+                            let _ = update.graph.insert_anchor(tx.txid, anchor);
+                        }
+                    }
+                }
+
+                if n_futures == 0 || empty_scripts >= stop_gap {
+                    break;
+                }
+            }
+
+            if let Some(last_active_index) = last_active_index {
+                update.keychain.insert(keychain, last_active_index);
+            }
+        }
+
+        for txid in txids.into_iter() {
+            if update.graph.get_tx(txid).is_none() {
+                match self.get_tx(&txid).await? {
+                    Some(tx) => {
+                        let _ = update.graph.insert_tx(tx);
+                    }
+                    None => continue,
+                }
+            }
+            match self.get_tx_status(&txid).await? {
+                tx_status if tx_status.confirmed => {
+                    if let Some(anchor) = map_confirmation_time_anchor(&tx_status, tip_at_start) {
+                        let _ = update.graph.insert_anchor(txid, anchor);
+                    }
+                }
+                _ => continue,
+            }
+        }
+
+        for op in outpoints.into_iter() {
+            let mut op_txs = Vec::with_capacity(2);
+            if let (
+                Some(tx),
+                tx_status @ TxStatus {
+                    confirmed: true, ..
+                },
+            ) = (
+                self.get_tx(&op.txid).await?,
+                self.get_tx_status(&op.txid).await?,
+            ) {
+                op_txs.push((tx, tx_status));
+                if let Some(OutputStatus {
+                    txid: Some(txid),
+                    status: Some(spend_status),
+                    ..
+                }) = self.get_output_status(&op.txid, op.vout as _).await?
+                {
+                    if let Some(spend_tx) = self.get_tx(&txid).await? {
+                        op_txs.push((spend_tx, spend_status));
+                    }
+                }
+            }
+
+            for (tx, status) in op_txs {
+                let txid = tx.txid();
+                let anchor = map_confirmation_time_anchor(&status, tip_at_start);
+
+                let _ = update.graph.insert_tx(tx);
+                if let Some(anchor) = anchor {
+                    let _ = update.graph.insert_anchor(txid, anchor);
+                }
+            }
+        }
+
+        if tip_at_start.hash != self.get_block_hash(tip_at_start.height).await? {
+            // A reorg occurred, so let's find out where all the txids we found are now in the chain
+            let txids_found = update
+                .graph
+                .full_txs()
+                .map(|tx_node| tx_node.txid)
+                .collect::<Vec<_>>();
+            update.chain = EsploraAsyncExt::scan_without_keychain(
+                self,
+                local_chain,
+                [],
+                txids_found,
+                [],
+                parallel_requests,
+            )
+            .await?
+            .chain;
+        }
+
+        Ok(update)
+    }
+}

crates/esplora/src/blocking_ext.rs

@@ -0,0 +1,251 @@
+use bdk_chain::bitcoin::{BlockHash, OutPoint, Script, Txid};
+use bdk_chain::collections::BTreeMap;
+use bdk_chain::BlockId;
+use bdk_chain::{keychain::LocalUpdate, ConfirmationTimeAnchor};
+use esplora_client::{Error, OutputStatus, TxStatus};
+
+use crate::map_confirmation_time_anchor;
+
+/// Trait to extend [`esplora_client::BlockingClient`] functionality.
+///
+/// Refer to [crate-level documentation] for more.
+///
+/// [crate-level documentation]: crate
+pub trait EsploraExt {
+    /// Scan the blockchain (via esplora) for the data specified and returns a
+    /// [`LocalUpdate<K, ConfirmationTimeAnchor>`].
+    ///
+    /// - `local_chain`: the most recent block hashes present locally
+    /// - `keychain_spks`: keychains that we want to scan transactions for
+    /// - `txids`: transactions for which we want updated [`ConfirmationTimeAnchor`]s
+    /// - `outpoints`: transactions associated with these outpoints (residing, spending) that we
+    ///     want to included in the update
+    ///
+    /// The scan for each keychain stops after a gap of `stop_gap` script pubkeys with no associated
+    /// transactions. `parallel_requests` specifies the max number of HTTP requests to make in
+    /// parallel.
+    #[allow(clippy::result_large_err)] // FIXME
+    fn scan<K: Ord + Clone>(
+        &self,
+        local_chain: &BTreeMap<u32, BlockHash>,
+        keychain_spks: BTreeMap<K, impl IntoIterator<Item = (u32, Script)>>,
+        txids: impl IntoIterator<Item = Txid>,
+        outpoints: impl IntoIterator<Item = OutPoint>,
+        stop_gap: usize,
+        parallel_requests: usize,
+    ) -> Result<LocalUpdate<K, ConfirmationTimeAnchor>, Error>;
+
+    /// Convenience method to call [`scan`] without requiring a keychain.
+    ///
+    /// [`scan`]: EsploraExt::scan
+    #[allow(clippy::result_large_err)] // FIXME
+    fn scan_without_keychain(
+        &self,
+        local_chain: &BTreeMap<u32, BlockHash>,
+        misc_spks: impl IntoIterator<Item = Script>,
+        txids: impl IntoIterator<Item = Txid>,
+        outpoints: impl IntoIterator<Item = OutPoint>,
+        parallel_requests: usize,
+    ) -> Result<LocalUpdate<(), ConfirmationTimeAnchor>, Error> {
+        self.scan(
+            local_chain,
+            [(
+                (),
+                misc_spks
+                    .into_iter()
+                    .enumerate()
+                    .map(|(i, spk)| (i as u32, spk)),
+            )]
+            .into(),
+            txids,
+            outpoints,
+            usize::MAX,
+            parallel_requests,
+        )
+    }
+}
+
+impl EsploraExt for esplora_client::BlockingClient {
+    fn scan<K: Ord + Clone>(
+        &self,
+        local_chain: &BTreeMap<u32, BlockHash>,
+        keychain_spks: BTreeMap<K, impl IntoIterator<Item = (u32, Script)>>,
+        txids: impl IntoIterator<Item = Txid>,
+        outpoints: impl IntoIterator<Item = OutPoint>,
+        stop_gap: usize,
+        parallel_requests: usize,
+    ) -> Result<LocalUpdate<K, ConfirmationTimeAnchor>, Error> {
+        let parallel_requests = Ord::max(parallel_requests, 1);
+
+        let (mut update, tip_at_start) = loop {
+            let mut update = LocalUpdate::<K, ConfirmationTimeAnchor>::default();
+
+            for (&height, &original_hash) in local_chain.iter().rev() {
+                let update_block_id = BlockId {
+                    height,
+                    hash: self.get_block_hash(height)?,
+                };
+                let _ = update
+                    .chain
+                    .insert_block(update_block_id)
+                    .expect("cannot repeat height here");
+                if update_block_id.hash == original_hash {
+                    break;
+                }
+            }
+
+            let tip_at_start = BlockId {
+                height: self.get_height()?,
+                hash: self.get_tip_hash()?,
+            };
+
+            if update.chain.insert_block(tip_at_start).is_ok() {
+                break (update, tip_at_start);
+            }
+        };
+
+        for (keychain, spks) in keychain_spks {
+            let mut spks = spks.into_iter();
+            let mut last_active_index = None;
+            let mut empty_scripts = 0;
+            type IndexWithTxs = (u32, Vec<esplora_client::Tx>);
+
+            loop {
+                let handles = (0..parallel_requests)
+                    .filter_map(
+                        |_| -> Option<std::thread::JoinHandle<Result<IndexWithTxs, _>>> {
+                            let (index, script) = spks.next()?;
+                            let client = self.clone();
+                            Some(std::thread::spawn(move || {
+                                let mut related_txs = client.scripthash_txs(&script, None)?;
+
+                                let n_confirmed =
+                                    related_txs.iter().filter(|tx| tx.status.confirmed).count();
+                                // esplora pages on 25 confirmed transactions. If there are 25 or more we
+                                // keep requesting to see if there's more.
+                                if n_confirmed >= 25 {
+                                    loop {
+                                        let new_related_txs = client.scripthash_txs(
+                                            &script,
+                                            Some(related_txs.last().unwrap().txid),
+                                        )?;
+                                        let n = new_related_txs.len();
+                                        related_txs.extend(new_related_txs);
+                                        // we've reached the end
+                                        if n < 25 {
+                                            break;
+                                        }
+                                    }
+                                }
+
+                                Result::<_, esplora_client::Error>::Ok((index, related_txs))
+                            }))
+                        },
+                    )
+                    .collect::<Vec<_>>();
+
+                let n_handles = handles.len();
+
+                for handle in handles {
+                    let (index, related_txs) = handle.join().unwrap()?; // TODO: don't unwrap
+                    if related_txs.is_empty() {
+                        empty_scripts += 1;
+                    } else {
+                        last_active_index = Some(index);
+                        empty_scripts = 0;
+                    }
+                    for tx in related_txs {
+                        let anchor = map_confirmation_time_anchor(&tx.status, tip_at_start);
+
+                        let _ = update.graph.insert_tx(tx.to_tx());
+                        if let Some(anchor) = anchor {
+                            let _ = update.graph.insert_anchor(tx.txid, anchor);
+                        }
+                    }
+                }
+
+                if n_handles == 0 || empty_scripts >= stop_gap {
+                    break;
+                }
+            }
+
+            if let Some(last_active_index) = last_active_index {
+                update.keychain.insert(keychain, last_active_index);
+            }
+        }
+
+        for txid in txids.into_iter() {
+            if update.graph.get_tx(txid).is_none() {
+                match self.get_tx(&txid)? {
+                    Some(tx) => {
+                        let _ = update.graph.insert_tx(tx);
+                    }
+                    None => continue,
+                }
+            }
+            match self.get_tx_status(&txid)? {
+                tx_status @ TxStatus {
+                    confirmed: true, ..
+                } => {
+                    if let Some(anchor) = map_confirmation_time_anchor(&tx_status, tip_at_start) {
+                        let _ = update.graph.insert_anchor(txid, anchor);
+                    }
+                }
+                _ => continue,
+            }
+        }
+
+        for op in outpoints.into_iter() {
+            let mut op_txs = Vec::with_capacity(2);
+            if let (
+                Some(tx),
+                tx_status @ TxStatus {
+                    confirmed: true, ..
+                },
+            ) = (self.get_tx(&op.txid)?, self.get_tx_status(&op.txid)?)
+            {
+                op_txs.push((tx, tx_status));
+                if let Some(OutputStatus {
+                    txid: Some(txid),
+                    status: Some(spend_status),
+                    ..
+                }) = self.get_output_status(&op.txid, op.vout as _)?
+                {
+                    if let Some(spend_tx) = self.get_tx(&txid)? {
+                        op_txs.push((spend_tx, spend_status));
+                    }
+                }
+            }
+
+            for (tx, status) in op_txs {
+                let txid = tx.txid();
+                let anchor = map_confirmation_time_anchor(&status, tip_at_start);
+
+                let _ = update.graph.insert_tx(tx);
+                if let Some(anchor) = anchor {
+                    let _ = update.graph.insert_anchor(txid, anchor);
+                }
+            }
+        }
+
+        if tip_at_start.hash != self.get_block_hash(tip_at_start.height)? {
+            // A reorg occurred, so let's find out where all the txids we found are now in the chain
+            let txids_found = update
+                .graph
+                .full_txs()
+                .map(|tx_node| tx_node.txid)
+                .collect::<Vec<_>>();
+            update.chain = EsploraExt::scan_without_keychain(
+                self,
+                local_chain,
+                [],
+                txids_found,
+                [],
+                parallel_requests,
+            )?
+            .chain;
+        }
+
+        Ok(update)
+    }
+}

crates/esplora/src/lib.rs

@@ -0,0 +1,29 @@
+#![doc = include_str!("../README.md")]
+use bdk_chain::{BlockId, ConfirmationTimeAnchor};
+use esplora_client::TxStatus;
+
+pub use esplora_client;
+
+#[cfg(feature = "blocking")]
+mod blocking_ext;
+#[cfg(feature = "blocking")]
+pub use blocking_ext::*;
+
+#[cfg(feature = "async")]
+mod async_ext;
+#[cfg(feature = "async")]
+pub use async_ext::*;
+
+pub(crate) fn map_confirmation_time_anchor(
+    tx_status: &TxStatus,
+    tip_at_start: BlockId,
+) -> Option<ConfirmationTimeAnchor> {
+    match (tx_status.block_time, tx_status.block_height) {
+        (Some(confirmation_time), Some(confirmation_height)) => Some(ConfirmationTimeAnchor {
+            anchor_block: tip_at_start,
+            confirmation_height,
+            confirmation_time,
+        }),
+        _ => None,
+    }
+}

crates/file_store/Cargo.toml

@@ -0,0 +1,19 @@
+[package]
+name = "bdk_file_store"
+version = "0.2.0"
+edition = "2021"
+license = "MIT OR Apache-2.0"
+repository = "https://github.com/bitcoindevkit/bdk"
+documentation = "https://docs.rs/bdk_file_store"
+description = "A simple append-only flat file implementation of Persist for Bitcoin Dev Kit."
+keywords = ["bitcoin", "persist", "persistence", "bdk", "file"]
+authors = ["Bitcoin Dev Kit Developers"]
+readme = "README.md"
+
+[dependencies]
+bdk_chain = { path = "../chain", version = "0.5.0", features = [ "serde", "miniscript" ] }
+bincode = { version = "1" }
+serde = { version = "1", features = ["derive"] }
+
+[dev-dependencies]
+tempfile = "3"

crates/file_store/README.md

@@ -0,0 +1,10 @@
+# BDK File Store
+
+This is a simple append-only flat file implementation of
+[`Persist`](`bdk_chain::Persist`).
+
+The main structure is [`Store`](`crate::Store`), which can be used with [`bdk`]'s
+`Wallet` to persist wallet data into a flat file.
+
+[`bdk`]: https://docs.rs/bdk/latest
+[`bdk_chain`]: https://docs.rs/bdk_chain/latest

crates/file_store/src/entry_iter.rs

@@ -0,0 +1,100 @@
+use bincode::Options;
+use std::{
+    fs::File,
+    io::{self, Seek},
+    marker::PhantomData,
+};
+
+use crate::bincode_options;
+
+/// Iterator over entries in a file store.
+///
+/// Reads and returns an entry each time [`next`] is called. If an error occurs while reading the
+/// iterator will yield a `Result::Err(_)` instead and then `None` for the next call to `next`.
+///
+/// [`next`]: Self::next
+pub struct EntryIter<'t, T> {
+    db_file: Option<&'t mut File>,
+
+    /// The file position for the first read of `db_file`.
+    start_pos: Option<u64>,
+    types: PhantomData<T>,
+}
+
+impl<'t, T> EntryIter<'t, T> {
+    pub fn new(start_pos: u64, db_file: &'t mut File) -> Self {
+        Self {
+            db_file: Some(db_file),
+            start_pos: Some(start_pos),
+            types: PhantomData,
+        }
+    }
+}
+
+impl<'t, T> Iterator for EntryIter<'t, T>
+where
+    T: serde::de::DeserializeOwned,
+{
+    type Item = Result<T, IterError>;
+
+    fn next(&mut self) -> Option<Self::Item> {
+        // closure which reads a single entry starting from `self.pos`
+        let read_one = |f: &mut File, start_pos: Option<u64>| -> Result<Option<T>, IterError> {
+            let pos = match start_pos {
+                Some(pos) => f.seek(io::SeekFrom::Start(pos))?,
+                None => f.stream_position()?,
+            };
+
+            match bincode_options().deserialize_from(&*f) {
+                Ok(changeset) => {
+                    f.stream_position()?;
+                    Ok(Some(changeset))
+                }
+                Err(e) => {
+                    if let bincode::ErrorKind::Io(inner) = &*e {
+                        if inner.kind() == io::ErrorKind::UnexpectedEof {
+                            let eof = f.seek(io::SeekFrom::End(0))?;
+                            if pos == eof {
+                                return Ok(None);
+                            }
+                        }
+                    }
+                    f.seek(io::SeekFrom::Start(pos))?;
+                    Err(IterError::Bincode(*e))
+                }
+            }
+        };
+
+        let result = read_one(self.db_file.as_mut()?, self.start_pos.take());
+        if result.is_err() {
+            self.db_file = None;
+        }
+        result.transpose()
+    }
+}
+
+impl From<io::Error> for IterError {
+    fn from(value: io::Error) -> Self {
+        IterError::Io(value)
+    }
+}
+
+/// Error type for [`EntryIter`].
+#[derive(Debug)]
+pub enum IterError {
+    /// Failure to read from the file.
+    Io(io::Error),
+    /// Failure to decode data from the file.
+    Bincode(bincode::ErrorKind),
+}
+
+impl core::fmt::Display for IterError {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        match self {
+            IterError::Io(e) => write!(f, "io error trying to read entry {}", e),
+            IterError::Bincode(e) => write!(f, "bincode error while reading entry {}", e),
+        }
+    }
+}
+
+impl std::error::Error for IterError {}

crates/file_store/src/lib.rs

@@ -0,0 +1,42 @@
+#![doc = include_str!("../README.md")]
+mod entry_iter;
+mod store;
+use std::io;
+
+use bincode::{DefaultOptions, Options};
+pub use entry_iter::*;
+pub use store::*;
+
+pub(crate) fn bincode_options() -> impl bincode::Options {
+    DefaultOptions::new().with_varint_encoding()
+}
+
+/// Error that occurs due to problems encountered with the file.
+#[derive(Debug)]
+pub enum FileError<'a> {
+    /// IO error, this may mean that the file is too short.
+    Io(io::Error),
+    /// Magic bytes do not match what is expected.
+    InvalidMagicBytes { got: Vec<u8>, expected: &'a [u8] },
+}
+
+impl<'a> core::fmt::Display for FileError<'a> {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        match self {
+            Self::Io(e) => write!(f, "io error trying to read file: {}", e),
+            Self::InvalidMagicBytes { got, expected } => write!(
+                f,
+                "file has invalid magic bytes: expected={:?} got={:?}",
+                expected, got,
+            ),
+        }
+    }
+}
+
+impl<'a> From<io::Error> for FileError<'a> {
+    fn from(value: io::Error) -> Self {
+        Self::Io(value)
+    }
+}
+
+impl<'a> std::error::Error for FileError<'a> {}

crates/file_store/src/store.rs

@@ -0,0 +1,255 @@
+use std::{
+    fmt::Debug,
+    fs::{File, OpenOptions},
+    io::{self, Read, Seek, Write},
+    marker::PhantomData,
+    path::Path,
+};
+
+use bdk_chain::{Append, PersistBackend};
+use bincode::Options;
+
+use crate::{bincode_options, EntryIter, FileError, IterError};
+
+/// Persists an append-only list of changesets (`C`) to a single file.
+///
+/// The changesets are the results of altering a tracker implementation (`T`).
+#[derive(Debug)]
+pub struct Store<'a, C> {
+    magic: &'a [u8],
+    db_file: File,
+    marker: PhantomData<C>,
+}
+
+impl<'a, C> PersistBackend<C> for Store<'a, C>
+where
+    C: Default + Append + serde::Serialize + serde::de::DeserializeOwned,
+{
+    type WriteError = std::io::Error;
+
+    type LoadError = IterError;
+
+    fn write_changes(&mut self, changeset: &C) -> Result<(), Self::WriteError> {
+        self.append_changeset(changeset)
+    }
+
+    fn load_from_persistence(&mut self) -> Result<C, Self::LoadError> {
+        let (changeset, result) = self.aggregate_changesets();
+        result.map(|_| changeset)
+    }
+}
+
+impl<'a, C> Store<'a, C>
+where
+    C: Default + Append + serde::Serialize + serde::de::DeserializeOwned,
+{
+    /// Creates a new store from a [`File`].
+    ///
+    /// The file must have been opened with read and write permissions.
+    ///
+    /// `magic` is the expected prefixed bytes of the file. If this does not match, an error will be
+    /// returned.
+    ///
+    /// [`File`]: std::fs::File
+    pub fn new(magic: &'a [u8], mut db_file: File) -> Result<Self, FileError> {
+        db_file.rewind()?;
+
+        let mut magic_buf = vec![0_u8; magic.len()];
+        db_file.read_exact(magic_buf.as_mut())?;
+
+        if magic_buf != magic {
+            return Err(FileError::InvalidMagicBytes {
+                got: magic_buf,
+                expected: magic,
+            });
+        }
+
+        Ok(Self {
+            magic,
+            db_file,
+            marker: Default::default(),
+        })
+    }
+
+    /// Creates or loads a store from `db_path`.
+    ///
+    /// If no file exists there, it will be created.
+    ///
+    /// Refer to [`new`] for documentation on the `magic` input.
+    ///
+    /// [`new`]: Self::new
+    pub fn new_from_path<P>(magic: &'a [u8], db_path: P) -> Result<Self, FileError>
+    where
+        P: AsRef<Path>,
+    {
+        let already_exists = db_path.as_ref().exists();
+
+        let mut db_file = OpenOptions::new()
+            .read(true)
+            .write(true)
+            .create(true)
+            .open(db_path)?;
+
+        if !already_exists {
+            db_file.write_all(magic)?;
+        }
+
+        Self::new(magic, db_file)
+    }
+
+    /// Iterates over the stored changeset from first to last, changing the seek position at each
+    /// iteration.
+    ///
+    /// The iterator may fail to read an entry and therefore return an error. However, the first time
+    /// it returns an error will be the last. After doing so, the iterator will always yield `None`.
+    ///
+    /// **WARNING**: This method changes the write position in the underlying file. You should
+    /// always iterate over all entries until `None` is returned if you want your next write to go
+    /// at the end; otherwise, you will write over existing entries.
+    pub fn iter_changesets(&mut self) -> EntryIter<C> {
+        EntryIter::new(self.magic.len() as u64, &mut self.db_file)
+    }
+
+    /// Loads all the changesets that have been stored as one giant changeset.
+    ///
+    /// This function returns a tuple of the aggregate changeset and a result that indicates
+    /// whether an error occurred while reading or deserializing one of the entries. If so the
+    /// changeset will consist of all of those it was able to read.
+    ///
+    /// You should usually check the error. In many applications, it may make sense to do a full
+    /// wallet scan with a stop-gap after getting an error, since it is likely that one of the
+    /// changesets it was unable to read changed the derivation indices of the tracker.
+    ///
+    /// **WARNING**: This method changes the write position of the underlying file. The next
+    /// changeset will be written over the erroring entry (or the end of the file if none existed).
+    pub fn aggregate_changesets(&mut self) -> (C, Result<(), IterError>) {
+        let mut changeset = C::default();
+        let result = (|| {
+            for next_changeset in self.iter_changesets() {
+                changeset.append(next_changeset?);
+            }
+            Ok(())
+        })();
+
+        (changeset, result)
+    }
+
+    /// Append a new changeset to the file and truncate the file to the end of the appended
+    /// changeset.
+    ///
+    /// The truncation is to avoid the possibility of having a valid but inconsistent changeset
+    /// directly after the appended changeset.
+    pub fn append_changeset(&mut self, changeset: &C) -> Result<(), io::Error> {
+        // no need to write anything if changeset is empty
+        if changeset.is_empty() {
+            return Ok(());
+        }
+
+        bincode_options()
+            .serialize_into(&mut self.db_file, changeset)
+            .map_err(|e| match *e {
+                bincode::ErrorKind::Io(inner) => inner,
+                unexpected_err => panic!("unexpected bincode error: {}", unexpected_err),
+            })?;
+
+        // truncate file after this changeset addition
+        // if this is not done, data after this changeset may represent valid changesets, however
+        // applying those changesets on top of this one may result in an inconsistent state
+        let pos = self.db_file.stream_position()?;
+        self.db_file.set_len(pos)?;
+
+        Ok(())
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use super::*;
+
+    use bincode::DefaultOptions;
+    use std::{
+        io::{Read, Write},
+        vec::Vec,
+    };
+    use tempfile::NamedTempFile;
+
+    const TEST_MAGIC_BYTES_LEN: usize = 12;
+    const TEST_MAGIC_BYTES: [u8; TEST_MAGIC_BYTES_LEN] =
+        [98, 100, 107, 102, 115, 49, 49, 49, 49, 49, 49, 49];
+
+    type TestChangeSet = Vec<String>;
+
+    #[derive(Debug)]
+    struct TestTracker;
+
+    #[test]
+    fn new_fails_if_file_is_too_short() {
+        let mut file = NamedTempFile::new().unwrap();
+        file.write_all(&TEST_MAGIC_BYTES[..TEST_MAGIC_BYTES_LEN - 1])
+            .expect("should write");
+
+        match Store::<TestChangeSet>::new(&TEST_MAGIC_BYTES, file.reopen().unwrap()) {
+            Err(FileError::Io(e)) => assert_eq!(e.kind(), std::io::ErrorKind::UnexpectedEof),
+            unexpected => panic!("unexpected result: {:?}", unexpected),
+        };
+    }
+
+    #[test]
+    fn new_fails_if_magic_bytes_are_invalid() {
+        let invalid_magic_bytes = "ldkfs0000000";
+
+        let mut file = NamedTempFile::new().unwrap();
+        file.write_all(invalid_magic_bytes.as_bytes())
+            .expect("should write");
+
+        match Store::<TestChangeSet>::new(&TEST_MAGIC_BYTES, file.reopen().unwrap()) {
+            Err(FileError::InvalidMagicBytes { got, .. }) => {
+                assert_eq!(got, invalid_magic_bytes.as_bytes())
+            }
+            unexpected => panic!("unexpected result: {:?}", unexpected),
+        };
+    }
+
+    #[test]
+    fn append_changeset_truncates_invalid_bytes() {
+        // initial data to write to file (magic bytes + invalid data)
+        let mut data = [255_u8; 2000];
+        data[..TEST_MAGIC_BYTES_LEN].copy_from_slice(&TEST_MAGIC_BYTES);
+
+        let changeset = vec!["one".into(), "two".into(), "three!".into()];
+
+        let mut file = NamedTempFile::new().unwrap();
+        file.write_all(&data).expect("should write");
+
+        let mut store = Store::<TestChangeSet>::new(&TEST_MAGIC_BYTES, file.reopen().unwrap())
+            .expect("should open");
+        match store.iter_changesets().next() {
+            Some(Err(IterError::Bincode(_))) => {}
+            unexpected_res => panic!("unexpected result: {:?}", unexpected_res),
+        }
+
+        store.append_changeset(&changeset).expect("should append");
+
+        drop(store);
+
+        let got_bytes = {
+            let mut buf = Vec::new();
+            file.reopen()
+                .unwrap()
+                .read_to_end(&mut buf)
+                .expect("should read");
+            buf
+        };
+
+        let expected_bytes = {
+            let mut buf = TEST_MAGIC_BYTES.to_vec();
+            DefaultOptions::new()
+                .with_varint_encoding()
+                .serialize_into(&mut buf, &changeset)
+                .expect("should encode");
+            buf
+        };
+
+        assert_eq!(got_bytes, expected_bytes);
+    }
+}

crates/file_store/tests/test_file_store.rs

@@ -0,0 +1 @@
+

example-crates/example_cli/Cargo.toml

@@ -0,0 +1,17 @@
+[package]
+name = "example_cli"
+version = "0.2.0"
+edition = "2021"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+bdk_chain = { path = "../../crates/chain", features = ["serde", "miniscript"]}
+bdk_file_store = { path = "../../crates/file_store" }
+bdk_tmp_plan = { path = "../../nursery/tmp_plan" }
+bdk_coin_select = { path = "../../nursery/coin_select" }
+
+clap = { version = "3.2.23", features = ["derive", "env"] }
+anyhow = "1"
+serde = { version = "1", features = ["derive"] }
+serde_json = { version = "^1.0" }

example-crates/example_cli/src/lib.rs

@@ -0,0 +1,736 @@
+pub use anyhow;
+use anyhow::Context;
+use bdk_coin_select::{coin_select_bnb, CoinSelector, CoinSelectorOpt, WeightedValue};
+use bdk_file_store::Store;
+use serde::{de::DeserializeOwned, Serialize};
+use std::{cmp::Reverse, collections::HashMap, path::PathBuf, sync::Mutex, time::Duration};
+
+use bdk_chain::{
+    bitcoin::{
+        psbt::Prevouts, secp256k1::Secp256k1, util::sighash::SighashCache, Address, LockTime,
+        Network, Sequence, Transaction, TxIn, TxOut,
+    },
+    indexed_tx_graph::{IndexedAdditions, IndexedTxGraph},
+    keychain::{DerivationAdditions, KeychainTxOutIndex},
+    miniscript::{
+        descriptor::{DescriptorSecretKey, KeyMap},
+        Descriptor, DescriptorPublicKey,
+    },
+    Anchor, Append, ChainOracle, DescriptorExt, FullTxOut, Persist, PersistBackend,
+};
+pub use bdk_file_store;
+pub use clap;
+
+use clap::{Parser, Subcommand};
+
+pub type KeychainTxGraph<A> = IndexedTxGraph<A, KeychainTxOutIndex<Keychain>>;
+pub type KeychainAdditions<A> = IndexedAdditions<A, DerivationAdditions<Keychain>>;
+pub type Database<'m, C> = Persist<Store<'m, C>, C>;
+
+#[derive(Parser)]
+#[clap(author, version, about, long_about = None)]
+#[clap(propagate_version = true)]
+pub struct Args<S: clap::Subcommand> {
+    #[clap(env = "DESCRIPTOR")]
+    pub descriptor: String,
+    #[clap(env = "CHANGE_DESCRIPTOR")]
+    pub change_descriptor: Option<String>,
+
+    #[clap(env = "BITCOIN_NETWORK", long, default_value = "signet")]
+    pub network: Network,
+
+    #[clap(env = "BDK_DB_PATH", long, default_value = ".bdk_example_db")]
+    pub db_path: PathBuf,
+
+    #[clap(env = "BDK_CP_LIMIT", long, default_value = "20")]
+    pub cp_limit: usize,
+
+    #[clap(subcommand)]
+    pub command: Commands<S>,
+}
+
+#[allow(clippy::almost_swapped)]
+#[derive(Subcommand, Debug, Clone)]
+pub enum Commands<S: clap::Subcommand> {
+    #[clap(flatten)]
+    ChainSpecific(S),
+    /// Address generation and inspection.
+    Address {
+        #[clap(subcommand)]
+        addr_cmd: AddressCmd,
+    },
+    /// Get the wallet balance.
+    Balance,
+    /// TxOut related commands.
+    #[clap(name = "txout")]
+    TxOut {
+        #[clap(subcommand)]
+        txout_cmd: TxOutCmd,
+    },
+    /// Send coins to an address.
+    Send {
+        value: u64,
+        address: Address,
+        #[clap(short, default_value = "bnb")]
+        coin_select: CoinSelectionAlgo,
+    },
+}
+
+#[derive(Clone, Debug)]
+pub enum CoinSelectionAlgo {
+    LargestFirst,
+    SmallestFirst,
+    OldestFirst,
+    NewestFirst,
+    BranchAndBound,
+}
+
+impl Default for CoinSelectionAlgo {
+    fn default() -> Self {
+        Self::LargestFirst
+    }
+}
+
+impl core::str::FromStr for CoinSelectionAlgo {
+    type Err = anyhow::Error;
+
+    fn from_str(s: &str) -> Result<Self, Self::Err> {
+        use CoinSelectionAlgo::*;
+        Ok(match s {
+            "largest-first" => LargestFirst,
+            "smallest-first" => SmallestFirst,
+            "oldest-first" => OldestFirst,
+            "newest-first" => NewestFirst,
+            "bnb" => BranchAndBound,
+            unknown => {
+                return Err(anyhow::anyhow!(
+                    "unknown coin selection algorithm '{}'",
+                    unknown
+                ))
+            }
+        })
+    }
+}
+
+impl core::fmt::Display for CoinSelectionAlgo {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+        use CoinSelectionAlgo::*;
+        write!(
+            f,
+            "{}",
+            match self {
+                LargestFirst => "largest-first",
+                SmallestFirst => "smallest-first",
+                OldestFirst => "oldest-first",
+                NewestFirst => "newest-first",
+                BranchAndBound => "bnb",
+            }
+        )
+    }
+}
+
+#[allow(clippy::almost_swapped)]
+#[derive(Subcommand, Debug, Clone)]
+pub enum AddressCmd {
+    /// Get the next unused address.
+    Next,
+    /// Get a new address regardless of the existing unused addresses.
+    New,
+    /// List all addresses
+    List {
+        #[clap(long)]
+        change: bool,
+    },
+    Index,
+}
+
+#[derive(Subcommand, Debug, Clone)]
+pub enum TxOutCmd {
+    List {
+        /// Return only spent outputs.
+        #[clap(short, long)]
+        spent: bool,
+        /// Return only unspent outputs.
+        #[clap(short, long)]
+        unspent: bool,
+        /// Return only confirmed outputs.
+        #[clap(long)]
+        confirmed: bool,
+        /// Return only unconfirmed outputs.
+        #[clap(long)]
+        unconfirmed: bool,
+    },
+}
+
+#[derive(
+    Debug, Clone, Copy, PartialOrd, Ord, PartialEq, Eq, serde::Deserialize, serde::Serialize,
+)]
+pub enum Keychain {
+    External,
+    Internal,
+}
+
+impl core::fmt::Display for Keychain {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+        match self {
+            Keychain::External => write!(f, "external"),
+            Keychain::Internal => write!(f, "internal"),
+        }
+    }
+}
+
+pub fn run_address_cmd<A, C>(
+    graph: &mut KeychainTxGraph<A>,
+    db: &Mutex<Database<C>>,
+    network: Network,
+    cmd: AddressCmd,
+) -> anyhow::Result<()>
+where
+    C: Default + Append + DeserializeOwned + Serialize + From<KeychainAdditions<A>>,
+{
+    let index = &mut graph.index;
+
+    match cmd {
+        AddressCmd::Next | AddressCmd::New => {
+            let spk_chooser = match cmd {
+                AddressCmd::Next => KeychainTxOutIndex::next_unused_spk,
+                AddressCmd::New => KeychainTxOutIndex::reveal_next_spk,
+                _ => unreachable!("only these two variants exist in match arm"),
+            };
+
+            let ((spk_i, spk), index_additions) = spk_chooser(index, &Keychain::External);
+            let db = &mut *db.lock().unwrap();
+            db.stage(C::from(KeychainAdditions::from(index_additions)));
+            db.commit()?;
+            let addr = Address::from_script(spk, network).context("failed to derive address")?;
+            println!("[address @ {}] {}", spk_i, addr);
+            Ok(())
+        }
+        AddressCmd::Index => {
+            for (keychain, derivation_index) in index.last_revealed_indices() {
+                println!("{:?}: {}", keychain, derivation_index);
+            }
+            Ok(())
+        }
+        AddressCmd::List { change } => {
+            let target_keychain = match change {
+                true => Keychain::Internal,
+                false => Keychain::External,
+            };
+            for (spk_i, spk) in index.revealed_spks_of_keychain(&target_keychain) {
+                let address = Address::from_script(spk, network)
+                    .expect("should always be able to derive address");
+                println!(
+                    "{:?} {} used:{}",
+                    spk_i,
+                    address,
+                    index.is_used(&(target_keychain, spk_i))
+                );
+            }
+            Ok(())
+        }
+    }
+}
+
+pub fn run_balance_cmd<A: Anchor, O: ChainOracle>(
+    graph: &KeychainTxGraph<A>,
+    chain: &O,
+) -> Result<(), O::Error> {
+    fn print_balances<'a>(title_str: &'a str, items: impl IntoIterator<Item = (&'a str, u64)>) {
+        println!("{}:", title_str);
+        for (name, amount) in items.into_iter() {
+            println!("    {:<10} {:>12} sats", name, amount)
+        }
+    }
+
+    let balance = graph.graph().try_balance(
+        chain,
+        chain.get_chain_tip()?.unwrap_or_default(),
+        graph.index.outpoints().iter().cloned(),
+        |(k, _), _| k == &Keychain::Internal,
+    )?;
+
+    let confirmed_total = balance.confirmed + balance.immature;
+    let unconfirmed_total = balance.untrusted_pending + balance.trusted_pending;
+
+    print_balances(
+        "confirmed",
+        [
+            ("total", confirmed_total),
+            ("spendable", balance.confirmed),
+            ("immature", balance.immature),
+        ],
+    );
+    print_balances(
+        "unconfirmed",
+        [
+            ("total", unconfirmed_total),
+            ("trusted", balance.trusted_pending),
+            ("untrusted", balance.untrusted_pending),
+        ],
+    );
+
+    Ok(())
+}
+
+pub fn run_txo_cmd<A: Anchor, O: ChainOracle>(
+    graph: &KeychainTxGraph<A>,
+    chain: &O,
+    network: Network,
+    cmd: TxOutCmd,
+) -> anyhow::Result<()>
+where
+    O::Error: std::error::Error + Send + Sync + 'static,
+{
+    let chain_tip = chain.get_chain_tip()?.unwrap_or_default();
+    let outpoints = graph.index.outpoints().iter().cloned();
+
+    match cmd {
+        TxOutCmd::List {
+            spent,
+            unspent,
+            confirmed,
+            unconfirmed,
+        } => {
+            let txouts = graph
+                .graph()
+                .try_filter_chain_txouts(chain, chain_tip, outpoints)
+                .filter(|r| match r {
+                    Ok((_, full_txo)) => match (spent, unspent) {
+                        (true, false) => full_txo.spent_by.is_some(),
+                        (false, true) => full_txo.spent_by.is_none(),
+                        _ => true,
+                    },
+                    // always keep errored items
+                    Err(_) => true,
+                })
+                .filter(|r| match r {
+                    Ok((_, full_txo)) => match (confirmed, unconfirmed) {
+                        (true, false) => full_txo.chain_position.is_confirmed(),
+                        (false, true) => !full_txo.chain_position.is_confirmed(),
+                        _ => true,
+                    },
+                    // always keep errored items
+                    Err(_) => true,
+                })
+                .collect::<Result<Vec<_>, _>>()?;
+
+            for (spk_i, full_txo) in txouts {
+                let addr = Address::from_script(&full_txo.txout.script_pubkey, network)?;
+                println!(
+                    "{:?} {} {} {} spent:{:?}",
+                    spk_i, full_txo.txout.value, full_txo.outpoint, addr, full_txo.spent_by
+                )
+            }
+            Ok(())
+        }
+    }
+}
+
+#[allow(clippy::too_many_arguments)]
+pub fn run_send_cmd<A: Anchor, O: ChainOracle, C>(
+    graph: &Mutex<KeychainTxGraph<A>>,
+    db: &Mutex<Database<'_, C>>,
+    chain: &O,
+    keymap: &HashMap<DescriptorPublicKey, DescriptorSecretKey>,
+    cs_algorithm: CoinSelectionAlgo,
+    address: Address,
+    value: u64,
+    broadcast: impl FnOnce(&Transaction) -> anyhow::Result<()>,
+) -> anyhow::Result<()>
+where
+    O::Error: std::error::Error + Send + Sync + 'static,
+    C: Default + Append + DeserializeOwned + Serialize + From<KeychainAdditions<A>>,
+{
+    let (transaction, change_index) = {
+        let graph = &mut *graph.lock().unwrap();
+        // take mutable ref to construct tx -- it is only open for a short time while building it.
+        let (tx, change_info) = create_tx(graph, chain, keymap, cs_algorithm, address, value)?;
+
+        if let Some((index_additions, (change_keychain, index))) = change_info {
+            // We must first persist to disk the fact that we've got a new address from the
+            // change keychain so future scans will find the tx we're about to broadcast.
+            // If we're unable to persist this, then we don't want to broadcast.
+            {
+                let db = &mut *db.lock().unwrap();
+                db.stage(C::from(KeychainAdditions::from(index_additions)));
+                db.commit()?;
+            }
+
+            // We don't want other callers/threads to use this address while we're using it
+            // but we also don't want to scan the tx we just created because it's not
+            // technically in the blockchain yet.
+            graph.index.mark_used(&change_keychain, index);
+            (tx, Some((change_keychain, index)))
+        } else {
+            (tx, None)
+        }
+    };
+
+    match (broadcast)(&transaction) {
+        Ok(_) => {
+            println!("Broadcasted Tx : {}", transaction.txid());
+
+            let keychain_additions = graph.lock().unwrap().insert_tx(&transaction, None, None);
+
+            // We know the tx is at least unconfirmed now. Note if persisting here fails,
+            // it's not a big deal since we can always find it again form
+            // blockchain.
+            db.lock().unwrap().stage(C::from(keychain_additions));
+            Ok(())
+        }
+        Err(e) => {
+            if let Some((keychain, index)) = change_index {
+                // We failed to broadcast, so allow our change address to be used in the future
+                graph.lock().unwrap().index.unmark_used(&keychain, index);
+            }
+            Err(e)
+        }
+    }
+}
+
+#[allow(clippy::type_complexity)]
+pub fn create_tx<A: Anchor, O: ChainOracle>(
+    graph: &mut KeychainTxGraph<A>,
+    chain: &O,
+    keymap: &HashMap<DescriptorPublicKey, DescriptorSecretKey>,
+    cs_algorithm: CoinSelectionAlgo,
+    address: Address,
+    value: u64,
+) -> anyhow::Result<(
+    Transaction,
+    Option<(DerivationAdditions<Keychain>, (Keychain, u32))>,
+)>
+where
+    O::Error: std::error::Error + Send + Sync + 'static,
+{
+    let mut additions = DerivationAdditions::default();
+
+    let assets = bdk_tmp_plan::Assets {
+        keys: keymap.iter().map(|(pk, _)| pk.clone()).collect(),
+        ..Default::default()
+    };
+
+    // TODO use planning module
+    let mut candidates = planned_utxos(graph, chain, &assets)?;
+
+    // apply coin selection algorithm
+    match cs_algorithm {
+        CoinSelectionAlgo::LargestFirst => {
+            candidates.sort_by_key(|(_, utxo)| Reverse(utxo.txout.value))
+        }
+        CoinSelectionAlgo::SmallestFirst => candidates.sort_by_key(|(_, utxo)| utxo.txout.value),
+        CoinSelectionAlgo::OldestFirst => {
+            candidates.sort_by_key(|(_, utxo)| utxo.chain_position.clone())
+        }
+        CoinSelectionAlgo::NewestFirst => {
+            candidates.sort_by_key(|(_, utxo)| Reverse(utxo.chain_position.clone()))
+        }
+        CoinSelectionAlgo::BranchAndBound => {}
+    }
+
+    // turn the txos we chose into weight and value
+    let wv_candidates = candidates
+        .iter()
+        .map(|(plan, utxo)| {
+            WeightedValue::new(
+                utxo.txout.value,
+                plan.expected_weight() as _,
+                plan.witness_version().is_some(),
+            )
+        })
+        .collect();
+
+    let mut outputs = vec![TxOut {
+        value,
+        script_pubkey: address.script_pubkey(),
+    }];
+
+    let internal_keychain = if graph.index.keychains().get(&Keychain::Internal).is_some() {
+        Keychain::Internal
+    } else {
+        Keychain::External
+    };
+
+    let ((change_index, change_script), change_additions) =
+        graph.index.next_unused_spk(&internal_keychain);
+    additions.append(change_additions);
+
+    // Clone to drop the immutable reference.
+    let change_script = change_script.clone();
+
+    let change_plan = bdk_tmp_plan::plan_satisfaction(
+        &graph
+            .index
+            .keychains()
+            .get(&internal_keychain)
+            .expect("must exist")
+            .at_derivation_index(change_index),
+        &assets,
+    )
+    .expect("failed to obtain change plan");
+
+    let mut change_output = TxOut {
+        value: 0,
+        script_pubkey: change_script,
+    };
+
+    let cs_opts = CoinSelectorOpt {
+        target_feerate: 0.5,
+        min_drain_value: graph
+            .index
+            .keychains()
+            .get(&internal_keychain)
+            .expect("must exist")
+            .dust_value(),
+        ..CoinSelectorOpt::fund_outputs(
+            &outputs,
+            &change_output,
+            change_plan.expected_weight() as u32,
+        )
+    };
+
+    // TODO: How can we make it easy to shuffle in order of inputs and outputs here?
+    // apply coin selection by saying we need to fund these outputs
+    let mut coin_selector = CoinSelector::new(&wv_candidates, &cs_opts);
+
+    // just select coins in the order provided until we have enough
+    // only use the first result (least waste)
+    let selection = match cs_algorithm {
+        CoinSelectionAlgo::BranchAndBound => {
+            coin_select_bnb(Duration::from_secs(10), coin_selector.clone())
+                .map_or_else(|| coin_selector.select_until_finished(), |cs| cs.finish())?
+        }
+        _ => coin_selector.select_until_finished()?,
+    };
+    let (_, selection_meta) = selection.best_strategy();
+
+    // get the selected utxos
+    let selected_txos = selection.apply_selection(&candidates).collect::<Vec<_>>();
+
+    if let Some(drain_value) = selection_meta.drain_value {
+        change_output.value = drain_value;
+        // if the selection tells us to use change and the change value is sufficient, we add it as an output
+        outputs.push(change_output)
+    }
+
+    let mut transaction = Transaction {
+        version: 0x02,
+        // because the temporary planning module does not support timelocks, we can use the chain
+        // tip as the `lock_time` for anti-fee-sniping purposes
+        lock_time: chain
+            .get_chain_tip()?
+            .and_then(|block_id| LockTime::from_height(block_id.height).ok())
+            .unwrap_or(LockTime::ZERO)
+            .into(),
+        input: selected_txos
+            .iter()
+            .map(|(_, utxo)| TxIn {
+                previous_output: utxo.outpoint,
+                sequence: Sequence::ENABLE_RBF_NO_LOCKTIME,
+                ..Default::default()
+            })
+            .collect(),
+        output: outputs,
+    };
+
+    let prevouts = selected_txos
+        .iter()
+        .map(|(_, utxo)| utxo.txout.clone())
+        .collect::<Vec<_>>();
+    let sighash_prevouts = Prevouts::All(&prevouts);
+
+    // first, set tx values for the plan so that we don't change them while signing
+    for (i, (plan, _)) in selected_txos.iter().enumerate() {
+        if let Some(sequence) = plan.required_sequence() {
+            transaction.input[i].sequence = sequence
+        }
+    }
+
+    // create a short lived transaction
+    let _sighash_tx = transaction.clone();
+    let mut sighash_cache = SighashCache::new(&_sighash_tx);
+
+    for (i, (plan, _)) in selected_txos.iter().enumerate() {
+        let requirements = plan.requirements();
+        let mut auth_data = bdk_tmp_plan::SatisfactionMaterial::default();
+        assert!(
+            !requirements.requires_hash_preimages(),
+            "can't have hash pre-images since we didn't provide any."
+        );
+        assert!(
+            requirements.signatures.sign_with_keymap(
+                i,
+                keymap,
+                &sighash_prevouts,
+                None,
+                None,
+                &mut sighash_cache,
+                &mut auth_data,
+                &Secp256k1::default(),
+            )?,
+            "we should have signed with this input."
+        );
+
+        match plan.try_complete(&auth_data) {
+            bdk_tmp_plan::PlanState::Complete {
+                final_script_sig,
+                final_script_witness,
+            } => {
+                if let Some(witness) = final_script_witness {
+                    transaction.input[i].witness = witness;
+                }
+
+                if let Some(script_sig) = final_script_sig {
+                    transaction.input[i].script_sig = script_sig;
+                }
+            }
+            bdk_tmp_plan::PlanState::Incomplete(_) => {
+                return Err(anyhow::anyhow!(
+                    "we weren't able to complete the plan with our keys."
+                ));
+            }
+        }
+    }
+
+    let change_info = if selection_meta.drain_value.is_some() {
+        Some((additions, (internal_keychain, change_index)))
+    } else {
+        None
+    };
+
+    Ok((transaction, change_info))
+}
+
+#[allow(clippy::type_complexity)]
+pub fn planned_utxos<A: Anchor, O: ChainOracle, K: Clone + bdk_tmp_plan::CanDerive>(
+    graph: &KeychainTxGraph<A>,
+    chain: &O,
+    assets: &bdk_tmp_plan::Assets<K>,
+) -> Result<Vec<(bdk_tmp_plan::Plan<K>, FullTxOut<A>)>, O::Error> {
+    let chain_tip = chain.get_chain_tip()?.unwrap_or_default();
+    let outpoints = graph.index.outpoints().iter().cloned();
+    graph
+        .graph()
+        .try_filter_chain_unspents(chain, chain_tip, outpoints)
+        .filter_map(
+            #[allow(clippy::type_complexity)]
+            |r| -> Option<Result<(bdk_tmp_plan::Plan<K>, FullTxOut<A>), _>> {
+                let (k, i, full_txo) = match r {
+                    Err(err) => return Some(Err(err)),
+                    Ok(((k, i), full_txo)) => (k, i, full_txo),
+                };
+                let desc = graph
+                    .index
+                    .keychains()
+                    .get(&k)
+                    .expect("keychain must exist")
+                    .at_derivation_index(i);
+                let plan = bdk_tmp_plan::plan_satisfaction(&desc, assets)?;
+                Some(Ok((plan, full_txo)))
+            },
+        )
+        .collect()
+}
+
+pub fn handle_commands<S: clap::Subcommand, A: Anchor, O: ChainOracle, C>(
+    graph: &Mutex<KeychainTxGraph<A>>,
+    db: &Mutex<Database<C>>,
+    chain: &Mutex<O>,
+    keymap: &HashMap<DescriptorPublicKey, DescriptorSecretKey>,
+    network: Network,
+    broadcast: impl FnOnce(&Transaction) -> anyhow::Result<()>,
+    cmd: Commands<S>,
+) -> anyhow::Result<()>
+where
+    O::Error: std::error::Error + Send + Sync + 'static,
+    C: Default + Append + DeserializeOwned + Serialize + From<KeychainAdditions<A>>,
+{
+    match cmd {
+        Commands::ChainSpecific(_) => unreachable!("example code should handle this!"),
+        Commands::Address { addr_cmd } => {
+            let graph = &mut *graph.lock().unwrap();
+            run_address_cmd(graph, db, network, addr_cmd)
+        }
+        Commands::Balance => {
+            let graph = &*graph.lock().unwrap();
+            let chain = &*chain.lock().unwrap();
+            run_balance_cmd(graph, chain).map_err(anyhow::Error::from)
+        }
+        Commands::TxOut { txout_cmd } => {
+            let graph = &*graph.lock().unwrap();
+            let chain = &*chain.lock().unwrap();
+            run_txo_cmd(graph, chain, network, txout_cmd)
+        }
+        Commands::Send {
+            value,
+            address,
+            coin_select,
+        } => {
+            let chain = &*chain.lock().unwrap();
+            run_send_cmd(
+                graph,
+                db,
+                chain,
+                keymap,
+                coin_select,
+                address,
+                value,
+                broadcast,
+            )
+        }
+    }
+}
+
+#[allow(clippy::type_complexity)]
+pub fn init<'m, S: clap::Subcommand, C>(
+    db_magic: &'m [u8],
+    db_default_path: &str,
+) -> anyhow::Result<(
+    Args<S>,
+    KeyMap,
+    KeychainTxOutIndex<Keychain>,
+    Mutex<Database<'m, C>>,
+    C,
+)>
+where
+    C: Default + Append + Serialize + DeserializeOwned,
+{
+    if std::env::var("BDK_DB_PATH").is_err() {
+        std::env::set_var("BDK_DB_PATH", db_default_path);
+    }
+    let args = Args::<S>::parse();
+    let secp = Secp256k1::default();
+
+    let mut index = KeychainTxOutIndex::<Keychain>::default();
+
+    let (descriptor, mut keymap) =
+        Descriptor::<DescriptorPublicKey>::parse_descriptor(&secp, &args.descriptor)?;
+    index.add_keychain(Keychain::External, descriptor);
+
+    if let Some((internal_descriptor, internal_keymap)) = args
+        .change_descriptor
+        .as_ref()
+        .map(|desc_str| Descriptor::<DescriptorPublicKey>::parse_descriptor(&secp, desc_str))
+        .transpose()?
+    {
+        keymap.extend(internal_keymap);
+        index.add_keychain(Keychain::Internal, internal_descriptor);
+    }
+
+    let mut db_backend = match Store::<'m, C>::new_from_path(db_magic, &args.db_path) {
+        Ok(db_backend) => db_backend,
+        // we cannot return `err` directly as it has lifetime `'m`
+        Err(err) => return Err(anyhow::anyhow!("failed to init db backend: {:?}", err)),
+    };
+
+    let init_changeset = db_backend.load_from_persistence()?;
+
+    Ok((
+        args,
+        keymap,
+        index,
+        Mutex::new(Database::new(db_backend)),
+        init_changeset,
+    ))
+}

example-crates/example_electrum/Cargo.toml

@@ -0,0 +1,11 @@
+[package]
+name = "example_electrum"
+version = "0.2.0"
+edition = "2021"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+bdk_chain = { path = "../../crates/chain", features = ["serde"] }
+bdk_electrum = { path = "../../crates/electrum" }
+example_cli = { path = "../example_cli" }

example-crates/example_electrum/src/main.rs

@@ -0,0 +1,318 @@
+use std::{
+    collections::BTreeMap,
+    io::{self, Write},
+    sync::Mutex,
+};
+
+use bdk_chain::{
+    bitcoin::{Address, BlockHash, Network, OutPoint, Txid},
+    indexed_tx_graph::{IndexedAdditions, IndexedTxGraph},
+    keychain::LocalChangeSet,
+    local_chain::LocalChain,
+    Append, ConfirmationHeightAnchor,
+};
+use bdk_electrum::{
+    electrum_client::{self, ElectrumApi},
+    ElectrumExt, ElectrumUpdate,
+};
+use example_cli::{
+    anyhow::{self, Context},
+    clap::{self, Parser, Subcommand},
+    Keychain,
+};
+
+const DB_MAGIC: &[u8] = b"bdk_example_electrum";
+const DB_PATH: &str = ".bdk_electrum_example.db";
+const ASSUME_FINAL_DEPTH: usize = 10;
+
+#[derive(Subcommand, Debug, Clone)]
+enum ElectrumCommands {
+    /// Scans the addresses in the wallet using the electrum API.
+    Scan {
+        /// When a gap this large has been found for a keychain, it will stop.
+        #[clap(long, default_value = "5")]
+        stop_gap: usize,
+        #[clap(flatten)]
+        scan_options: ScanOptions,
+    },
+    /// Scans particular addresses using the electrum API.
+    Sync {
+        /// Scan all the unused addresses.
+        #[clap(long)]
+        unused_spks: bool,
+        /// Scan every address that you have derived.
+        #[clap(long)]
+        all_spks: bool,
+        /// Scan unspent outpoints for spends or changes to confirmation status of residing tx.
+        #[clap(long)]
+        utxos: bool,
+        /// Scan unconfirmed transactions for updates.
+        #[clap(long)]
+        unconfirmed: bool,
+        #[clap(flatten)]
+        scan_options: ScanOptions,
+    },
+}
+
+#[derive(Parser, Debug, Clone, PartialEq)]
+pub struct ScanOptions {
+    /// Set batch size for each script_history call to electrum client.
+    #[clap(long, default_value = "25")]
+    pub batch_size: usize,
+}
+
+type ChangeSet = LocalChangeSet<Keychain, ConfirmationHeightAnchor>;
+
+fn main() -> anyhow::Result<()> {
+    let (args, keymap, index, db, init_changeset) =
+        example_cli::init::<ElectrumCommands, ChangeSet>(DB_MAGIC, DB_PATH)?;
+
+    let graph = Mutex::new({
+        let mut graph = IndexedTxGraph::new(index);
+        graph.apply_additions(init_changeset.indexed_additions);
+        graph
+    });
+
+    let chain = Mutex::new({
+        let mut chain = LocalChain::default();
+        chain.apply_changeset(init_changeset.chain_changeset);
+        chain
+    });
+
+    let electrum_url = match args.network {
+        Network::Bitcoin => "ssl://electrum.blockstream.info:50002",
+        Network::Testnet => "ssl://electrum.blockstream.info:60002",
+        Network::Regtest => "tcp://localhost:60401",
+        Network::Signet => "tcp://signet-electrumx.wakiyamap.dev:50001",
+    };
+    let config = electrum_client::Config::builder()
+        .validate_domain(matches!(args.network, Network::Bitcoin))
+        .build();
+
+    let client = electrum_client::Client::from_config(electrum_url, config)?;
+
+    let electrum_cmd = match &args.command {
+        example_cli::Commands::ChainSpecific(electrum_cmd) => electrum_cmd,
+        general_cmd => {
+            let res = example_cli::handle_commands(
+                &graph,
+                &db,
+                &chain,
+                &keymap,
+                args.network,
+                |tx| {
+                    client
+                        .transaction_broadcast(tx)
+                        .map(|_| ())
+                        .map_err(anyhow::Error::from)
+                },
+                general_cmd.clone(),
+            );
+
+            db.lock().unwrap().commit()?;
+            return res;
+        }
+    };
+
+    let response = match electrum_cmd.clone() {
+        ElectrumCommands::Scan {
+            stop_gap,
+            scan_options,
+        } => {
+            let (keychain_spks, local_chain) = {
+                let graph = &*graph.lock().unwrap();
+                let chain = &*chain.lock().unwrap();
+
+                let keychain_spks = graph
+                    .index
+                    .spks_of_all_keychains()
+                    .into_iter()
+                    .map(|(keychain, iter)| {
+                        let mut first = true;
+                        let spk_iter = iter.inspect(move |(i, _)| {
+                            if first {
+                                eprint!("\nscanning {}: ", keychain);
+                                first = false;
+                            }
+
+                            eprint!("{} ", i);
+                            let _ = io::stdout().flush();
+                        });
+                        (keychain, spk_iter)
+                    })
+                    .collect::<BTreeMap<_, _>>();
+
+                let c = chain
+                    .blocks()
+                    .iter()
+                    .rev()
+                    .take(ASSUME_FINAL_DEPTH)
+                    .map(|(k, v)| (*k, *v))
+                    .collect::<BTreeMap<u32, BlockHash>>();
+
+                (keychain_spks, c)
+            };
+
+            client
+                .scan(
+                    &local_chain,
+                    keychain_spks,
+                    core::iter::empty(),
+                    core::iter::empty(),
+                    stop_gap,
+                    scan_options.batch_size,
+                )
+                .context("scanning the blockchain")?
+        }
+        ElectrumCommands::Sync {
+            mut unused_spks,
+            all_spks,
+            mut utxos,
+            mut unconfirmed,
+            scan_options,
+        } => {
+            // Get a short lock on the tracker to get the spks we're interested in
+            let graph = graph.lock().unwrap();
+            let chain = chain.lock().unwrap();
+            let chain_tip = chain.tip().unwrap_or_default();
+
+            if !(all_spks || unused_spks || utxos || unconfirmed) {
+                unused_spks = true;
+                unconfirmed = true;
+                utxos = true;
+            } else if all_spks {
+                unused_spks = false;
+            }
+
+            let mut spks: Box<dyn Iterator<Item = bdk_chain::bitcoin::Script>> =
+                Box::new(core::iter::empty());
+            if all_spks {
+                let all_spks = graph
+                    .index
+                    .all_spks()
+                    .iter()
+                    .map(|(k, v)| (*k, v.clone()))
+                    .collect::<Vec<_>>();
+                spks = Box::new(spks.chain(all_spks.into_iter().map(|(index, script)| {
+                    eprintln!("scanning {:?}", index);
+                    script
+                })));
+            }
+            if unused_spks {
+                let unused_spks = graph
+                    .index
+                    .unused_spks(..)
+                    .map(|(k, v)| (*k, v.clone()))
+                    .collect::<Vec<_>>();
+                spks = Box::new(spks.chain(unused_spks.into_iter().map(|(index, script)| {
+                    eprintln!(
+                        "Checking if address {} {:?} has been used",
+                        Address::from_script(&script, args.network).unwrap(),
+                        index
+                    );
+
+                    script
+                })));
+            }
+
+            let mut outpoints: Box<dyn Iterator<Item = OutPoint>> = Box::new(core::iter::empty());
+
+            if utxos {
+                let init_outpoints = graph.index.outpoints().iter().cloned();
+
+                let utxos = graph
+                    .graph()
+                    .filter_chain_unspents(&*chain, chain_tip, init_outpoints)
+                    .map(|(_, utxo)| utxo)
+                    .collect::<Vec<_>>();
+
+                outpoints = Box::new(
+                    utxos
+                        .into_iter()
+                        .inspect(|utxo| {
+                            eprintln!(
+                                "Checking if outpoint {} (value: {}) has been spent",
+                                utxo.outpoint, utxo.txout.value
+                            );
+                        })
+                        .map(|utxo| utxo.outpoint),
+                );
+            };
+
+            let mut txids: Box<dyn Iterator<Item = Txid>> = Box::new(core::iter::empty());
+
+            if unconfirmed {
+                let unconfirmed_txids = graph
+                    .graph()
+                    .list_chain_txs(&*chain, chain_tip)
+                    .filter(|canonical_tx| !canonical_tx.observed_as.is_confirmed())
+                    .map(|canonical_tx| canonical_tx.node.txid)
+                    .collect::<Vec<Txid>>();
+
+                txids = Box::new(unconfirmed_txids.into_iter().inspect(|txid| {
+                    eprintln!("Checking if {} is confirmed yet", txid);
+                }));
+            }
+
+            let c = chain
+                .blocks()
+                .iter()
+                .rev()
+                .take(ASSUME_FINAL_DEPTH)
+                .map(|(k, v)| (*k, *v))
+                .collect::<BTreeMap<u32, BlockHash>>();
+
+            // drop lock on graph and chain
+            drop((graph, chain));
+
+            let update = client
+                .scan_without_keychain(&c, spks, txids, outpoints, scan_options.batch_size)
+                .context("scanning the blockchain")?;
+            ElectrumUpdate {
+                graph_update: update.graph_update,
+                chain_update: update.chain_update,
+                keychain_update: BTreeMap::new(),
+            }
+        }
+    };
+
+    let missing_txids = {
+        let graph = &*graph.lock().unwrap();
+        response.missing_full_txs(graph.graph())
+    };
+
+    let now = std::time::UNIX_EPOCH
+        .elapsed()
+        .expect("must get time")
+        .as_secs();
+
+    let final_update = response.finalize(&client, Some(now), missing_txids)?;
+
+    let db_changeset = {
+        let mut chain = chain.lock().unwrap();
+        let mut graph = graph.lock().unwrap();
+
+        let chain_changeset = chain.apply_update(final_update.chain)?;
+
+        let indexed_additions = {
+            let mut additions = IndexedAdditions::<ConfirmationHeightAnchor, _>::default();
+            let (_, index_additions) = graph.index.reveal_to_target_multi(&final_update.keychain);
+            additions.append(IndexedAdditions {
+                index_additions,
+                ..Default::default()
+            });
+            additions.append(graph.apply_update(final_update.graph));
+            additions
+        };
+
+        ChangeSet {
+            indexed_additions,
+            chain_changeset,
+        }
+    };
+
+    let mut db = db.lock().unwrap();
+    db.stage(db_changeset);
+    db.commit()?;
+    Ok(())
+}

example-crates/wallet_electrum/Cargo.toml

@@ -0,0 +1,9 @@
+[package]
+name = "wallet_electrum_example"
+version = "0.2.0"
+edition = "2021"
+
+[dependencies]
+bdk = { path = "../../crates/bdk" }
+bdk_electrum = { path = "../../crates/electrum" }
+bdk_file_store = { path = "../../crates/file_store" }

example-crates/wallet_electrum/src/main.rs

@@ -0,0 +1,93 @@
+const DB_MAGIC: &str = "bdk_wallet_electrum_example";
+const SEND_AMOUNT: u64 = 5000;
+const STOP_GAP: usize = 50;
+const BATCH_SIZE: usize = 5;
+
+use std::io::Write;
+use std::str::FromStr;
+
+use bdk::bitcoin::Address;
+use bdk::SignOptions;
+use bdk::{bitcoin::Network, Wallet};
+use bdk_electrum::electrum_client::{self, ElectrumApi};
+use bdk_electrum::ElectrumExt;
+use bdk_file_store::Store;
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    let db_path = std::env::temp_dir().join("bdk-electrum-example");
+    let db = Store::<bdk::wallet::ChangeSet>::new_from_path(DB_MAGIC.as_bytes(), db_path)?;
+    let external_descriptor = "wpkh(tprv8ZgxMBicQKsPdy6LMhUtFHAgpocR8GC6QmwMSFpZs7h6Eziw3SpThFfczTDh5rW2krkqffa11UpX3XkeTTB2FvzZKWXqPY54Y6Rq4AQ5R8L/84'/0'/0'/0/*)";
+    let internal_descriptor = "wpkh(tprv8ZgxMBicQKsPdy6LMhUtFHAgpocR8GC6QmwMSFpZs7h6Eziw3SpThFfczTDh5rW2krkqffa11UpX3XkeTTB2FvzZKWXqPY54Y6Rq4AQ5R8L/84'/0'/0'/1/*)";
+
+    let mut wallet = Wallet::new(
+        external_descriptor,
+        Some(internal_descriptor),
+        db,
+        Network::Testnet,
+    )?;
+
+    let address = wallet.get_address(bdk::wallet::AddressIndex::New);
+    println!("Generated Address: {}", address);
+
+    let balance = wallet.get_balance();
+    println!("Wallet balance before syncing: {} sats", balance.total());
+
+    print!("Syncing...");
+    let client = electrum_client::Client::new("ssl://electrum.blockstream.info:60002")?;
+
+    let local_chain = wallet.checkpoints();
+    let keychain_spks = wallet
+        .spks_of_all_keychains()
+        .into_iter()
+        .map(|(k, k_spks)| {
+            let mut once = Some(());
+            let mut stdout = std::io::stdout();
+            let k_spks = k_spks
+                .inspect(move |(spk_i, _)| match once.take() {
+                    Some(_) => print!("\nScanning keychain [{:?}]", k),
+                    None => print!(" {:<3}", spk_i),
+                })
+                .inspect(move |_| stdout.flush().expect("must flush"));
+            (k, k_spks)
+        })
+        .collect();
+
+    let electrum_update =
+        client.scan(local_chain, keychain_spks, None, None, STOP_GAP, BATCH_SIZE)?;
+
+    println!();
+
+    let missing = electrum_update.missing_full_txs(wallet.as_ref());
+    let update = electrum_update.finalize_as_confirmation_time(&client, None, missing)?;
+
+    wallet.apply_update(update)?;
+    wallet.commit()?;
+
+    let balance = wallet.get_balance();
+    println!("Wallet balance after syncing: {} sats", balance.total());
+
+    if balance.total() < SEND_AMOUNT {
+        println!(
+            "Please send at least {} sats to the receiving address",
+            SEND_AMOUNT
+        );
+        std::process::exit(0);
+    }
+
+    let faucet_address = Address::from_str("mkHS9ne12qx9pS9VojpwU5xtRd4T7X7ZUt")?;
+
+    let mut tx_builder = wallet.build_tx();
+    tx_builder
+        .add_recipient(faucet_address.script_pubkey(), SEND_AMOUNT)
+        .enable_rbf();
+
+    let (mut psbt, _) = tx_builder.finish()?;
+    let finalized = wallet.sign(&mut psbt, SignOptions::default())?;
+    assert!(finalized);
+
+    let tx = psbt.extract_tx();
+    client.transaction_broadcast(&tx)?;
+    println!("Tx broadcasted! Txid: {}", tx.txid());
+
+    Ok(())
+}

example-crates/wallet_esplora/Cargo.toml

@@ -0,0 +1,12 @@
+[package]
+name = "wallet_esplora"
+version = "0.2.0"
+edition = "2021"
+publish = false
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+bdk = { path = "../../crates/bdk" }
+bdk_esplora = { path = "../../crates/esplora", features = ["blocking"] }
+bdk_file_store = { path = "../../crates/file_store" }

example-crates/wallet_esplora/src/main.rs

@@ -0,0 +1,94 @@
+const DB_MAGIC: &str = "bdk_wallet_esplora_example";
+const SEND_AMOUNT: u64 = 5000;
+const STOP_GAP: usize = 50;
+const PARALLEL_REQUESTS: usize = 5;
+
+use std::{io::Write, str::FromStr};
+
+use bdk::{
+    bitcoin::{Address, Network},
+    wallet::AddressIndex,
+    SignOptions, Wallet,
+};
+use bdk_esplora::{esplora_client, EsploraExt};
+use bdk_file_store::Store;
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    let db_path = std::env::temp_dir().join("bdk-esplora-example");
+    let db = Store::<bdk::wallet::ChangeSet>::new_from_path(DB_MAGIC.as_bytes(), db_path)?;
+    let external_descriptor = "wpkh(tprv8ZgxMBicQKsPdy6LMhUtFHAgpocR8GC6QmwMSFpZs7h6Eziw3SpThFfczTDh5rW2krkqffa11UpX3XkeTTB2FvzZKWXqPY54Y6Rq4AQ5R8L/84'/0'/0'/0/*)";
+    let internal_descriptor = "wpkh(tprv8ZgxMBicQKsPdy6LMhUtFHAgpocR8GC6QmwMSFpZs7h6Eziw3SpThFfczTDh5rW2krkqffa11UpX3XkeTTB2FvzZKWXqPY54Y6Rq4AQ5R8L/84'/0'/0'/1/*)";
+
+    let mut wallet = Wallet::new(
+        external_descriptor,
+        Some(internal_descriptor),
+        db,
+        Network::Testnet,
+    )?;
+
+    let address = wallet.get_address(AddressIndex::New);
+    println!("Generated Address: {}", address);
+
+    let balance = wallet.get_balance();
+    println!("Wallet balance before syncing: {} sats", balance.total());
+
+    print!("Syncing...");
+    let client =
+        esplora_client::Builder::new("https://blockstream.info/testnet/api").build_blocking()?;
+
+    let local_chain = wallet.checkpoints();
+    let keychain_spks = wallet
+        .spks_of_all_keychains()
+        .into_iter()
+        .map(|(k, k_spks)| {
+            let mut once = Some(());
+            let mut stdout = std::io::stdout();
+            let k_spks = k_spks
+                .inspect(move |(spk_i, _)| match once.take() {
+                    Some(_) => print!("\nScanning keychain [{:?}]", k),
+                    None => print!(" {:<3}", spk_i),
+                })
+                .inspect(move |_| stdout.flush().expect("must flush"));
+            (k, k_spks)
+        })
+        .collect();
+    let update = client.scan(
+        local_chain,
+        keychain_spks,
+        None,
+        None,
+        STOP_GAP,
+        PARALLEL_REQUESTS,
+    )?;
+    println!();
+    wallet.apply_update(update)?;
+    wallet.commit()?;
+
+    let balance = wallet.get_balance();
+    println!("Wallet balance after syncing: {} sats", balance.total());
+
+    if balance.total() < SEND_AMOUNT {
+        println!(
+            "Please send at least {} sats to the receiving address",
+            SEND_AMOUNT
+        );
+        std::process::exit(0);
+    }
+
+    let faucet_address = Address::from_str("mkHS9ne12qx9pS9VojpwU5xtRd4T7X7ZUt")?;
+
+    let mut tx_builder = wallet.build_tx();
+    tx_builder
+        .add_recipient(faucet_address.script_pubkey(), SEND_AMOUNT)
+        .enable_rbf();
+
+    let (mut psbt, _) = tx_builder.finish()?;
+    let finalized = wallet.sign(&mut psbt, SignOptions::default())?;
+    assert!(finalized);
+
+    let tx = psbt.extract_tx();
+    client.broadcast(&tx)?;
+    println!("Tx broadcasted! Txid: {}", tx.txid());
+
+    Ok(())
+}

example-crates/wallet_esplora_async/Cargo.toml

@@ -0,0 +1,12 @@
+[package]
+name = "wallet_esplora_async"
+version = "0.2.0"
+edition = "2021"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+bdk = { path = "../../crates/bdk" }
+bdk_esplora = { path = "../../crates/esplora", features = ["async-https"] }
+bdk_file_store = { path = "../../crates/file_store" }
+tokio = { version = "1", features = ["rt", "rt-multi-thread", "macros"] }

example-crates/wallet_esplora_async/src/main.rs

@@ -0,0 +1,97 @@
+use std::{io::Write, str::FromStr};
+
+use bdk::{
+    bitcoin::{Address, Network},
+    wallet::AddressIndex,
+    SignOptions, Wallet,
+};
+use bdk_esplora::{esplora_client, EsploraAsyncExt};
+use bdk_file_store::Store;
+
+const DB_MAGIC: &str = "bdk_wallet_esplora_async_example";
+const SEND_AMOUNT: u64 = 5000;
+const STOP_GAP: usize = 50;
+const PARALLEL_REQUESTS: usize = 5;
+
+#[tokio::main]
+async fn main() -> Result<(), Box<dyn std::error::Error>> {
+    let db_path = std::env::temp_dir().join("bdk-esplora-async-example");
+    let db = Store::<bdk::wallet::ChangeSet>::new_from_path(DB_MAGIC.as_bytes(), db_path)?;
+    let external_descriptor = "wpkh(tprv8ZgxMBicQKsPdy6LMhUtFHAgpocR8GC6QmwMSFpZs7h6Eziw3SpThFfczTDh5rW2krkqffa11UpX3XkeTTB2FvzZKWXqPY54Y6Rq4AQ5R8L/84'/0'/0'/0/*)";
+    let internal_descriptor = "wpkh(tprv8ZgxMBicQKsPdy6LMhUtFHAgpocR8GC6QmwMSFpZs7h6Eziw3SpThFfczTDh5rW2krkqffa11UpX3XkeTTB2FvzZKWXqPY54Y6Rq4AQ5R8L/84'/0'/0'/1/*)";
+
+    let mut wallet = Wallet::new(
+        external_descriptor,
+        Some(internal_descriptor),
+        db,
+        Network::Testnet,
+    )?;
+
+    let address = wallet.get_address(AddressIndex::New);
+    println!("Generated Address: {}", address);
+
+    let balance = wallet.get_balance();
+    println!("Wallet balance before syncing: {} sats", balance.total());
+
+    print!("Syncing...");
+    let client =
+        esplora_client::Builder::new("https://blockstream.info/testnet/api").build_async()?;
+
+    let local_chain = wallet.checkpoints();
+    let keychain_spks = wallet
+        .spks_of_all_keychains()
+        .into_iter()
+        .map(|(k, k_spks)| {
+            let mut once = Some(());
+            let mut stdout = std::io::stdout();
+            let k_spks = k_spks
+                .inspect(move |(spk_i, _)| match once.take() {
+                    Some(_) => print!("\nScanning keychain [{:?}]", k),
+                    None => print!(" {:<3}", spk_i),
+                })
+                .inspect(move |_| stdout.flush().expect("must flush"));
+            (k, k_spks)
+        })
+        .collect();
+    let update = client
+        .scan(
+            local_chain,
+            keychain_spks,
+            [],
+            [],
+            STOP_GAP,
+            PARALLEL_REQUESTS,
+        )
+        .await?;
+    println!();
+    wallet.apply_update(update)?;
+    wallet.commit()?;
+
+    let balance = wallet.get_balance();
+    println!("Wallet balance after syncing: {} sats", balance.total());
+
+    if balance.total() < SEND_AMOUNT {
+        println!(
+            "Please send at least {} sats to the receiving address",
+            SEND_AMOUNT
+        );
+        std::process::exit(0);
+    }
+
+    let faucet_address = Address::from_str("mkHS9ne12qx9pS9VojpwU5xtRd4T7X7ZUt")?;
+
+    let mut tx_builder = wallet.build_tx();
+    tx_builder
+        .add_recipient(faucet_address.script_pubkey(), SEND_AMOUNT)
+        .enable_rbf();
+
+    let (mut psbt, _) = tx_builder.finish()?;
+    let finalized = wallet.sign(&mut psbt, SignOptions::default())?;
+    assert!(finalized);
+
+    let tx = psbt.extract_tx();
+    client.broadcast(&tx).await?;
+    println!("Tx broadcasted! Txid: {}", tx.txid());
+
+    Ok(())
+}

examples/compiler.rs

@@ -1,110 +0,0 @@
-extern crate bitcoin;
-extern crate clap;
-extern crate log;
-extern crate magical_bitcoin_wallet;
-extern crate miniscript;
-extern crate rand;
-extern crate serde_json;
-extern crate sled;
-
-use std::str::FromStr;
-
-use log::info;
-
-use rand::distributions::Alphanumeric;
-use rand::{thread_rng, Rng};
-
-use clap::{App, Arg};
-
-use bitcoin::Network;
-use miniscript::policy::Concrete;
-use miniscript::Descriptor;
-
-use magical_bitcoin_wallet::types::ScriptType;
-use magical_bitcoin_wallet::{OfflineWallet, Wallet};
-
-fn main() {
-    env_logger::init_from_env(
-        env_logger::Env::default().filter_or(env_logger::DEFAULT_FILTER_ENV, "info"),
-    );
-
-    let matches = App::new("Miniscript Compiler")
-        .arg(
-            Arg::with_name("POLICY")
-                .help("Sets the spending policy to compile")
-                .required(true)
-                .index(1),
-        )
-        .arg(
-            Arg::with_name("TYPE")
-                .help("Sets the script type used to embed the compiled policy")
-                .required(true)
-                .index(2)
-                .possible_values(&["sh", "wsh", "sh-wsh"]),
-        )
-        .arg(
-            Arg::with_name("parsed_policy")
-                .long("parsed_policy")
-                .short("p")
-                .help("Also return the parsed spending policy in JSON format"),
-        )
-        .arg(
-            Arg::with_name("network")
-                .short("n")
-                .long("network")
-                .help("Sets the network")
-                .takes_value(true)
-                .default_value("testnet")
-                .possible_values(&["testnet", "regtest"]),
-        )
-        .get_matches();
-
-    let policy_str = matches.value_of("POLICY").unwrap();
-    info!("Compiling policy: {}", policy_str);
-
-    let policy = Concrete::<String>::from_str(&policy_str).unwrap();
-    let compiled = policy.compile().unwrap();
-
-    let descriptor = match matches.value_of("TYPE").unwrap() {
-        "sh" => Descriptor::Sh(compiled),
-        "wsh" => Descriptor::Wsh(compiled),
-        "sh-wsh" => Descriptor::ShWsh(compiled),
-        _ => panic!("Invalid type"),
-    };
-
-    info!("... Descriptor: {}", descriptor);
-
-    let temp_db = {
-        let mut temp_db = std::env::temp_dir();
-        let rand_string: String = thread_rng().sample_iter(&Alphanumeric).take(15).collect();
-        temp_db.push(rand_string);
-
-        let database = sled::open(&temp_db).unwrap();
-
-        let network = match matches.value_of("network") {
-            Some("regtest") => Network::Regtest,
-            Some("testnet") | _ => Network::Testnet,
-        };
-        let wallet: OfflineWallet<_> = Wallet::new_offline(
-            &format!("{}", descriptor),
-            None,
-            network,
-            database.open_tree("").unwrap(),
-        )
-        .unwrap();
-
-        info!("... First address: {}", wallet.get_new_address().unwrap());
-
-        if matches.is_present("parsed_policy") {
-            let spending_policy = wallet.policies(ScriptType::External).unwrap();
-            info!(
-                "... Spending policy:\n{}",
-                serde_json::to_string_pretty(&spending_policy).unwrap()
-            );
-        }
-
-        temp_db
-    };
-
-    std::fs::remove_dir_all(temp_db).unwrap();
-}