# BitTool evaluation plan

This is a planned real-device test procedure. Separately completed engine/package checks are documented in [technical_verification_en.md](technical_verification_en.md); they do not populate this device procedure or the compatibility matrix. Record **Not run**, **Blocked**, **Pass**, **Fail**, or **Not applicable** for each result. A missing capability is a result to report, not a reason to silently substitute a different network or product.

## Equipment and recorded baseline

Use two separate devices: one dedicated to Signer and one online companion. Keep a computer available for reading source, verifying published APK information and independently parsing synthetic requests. A camera filming the two devices is preferable to relying on screenshots as evidence of isolation.

Record:

- Device model, OS version, app version/build and installation source for each device.
- Repository tag/commit and published binary hash where applicable. Record the actual downloaded file hash separately; do not copy the publisher's hash into a measured-result field.
- Network, derivation path, script type, address index, request encoding and test-vector revision.
- Date, reviewer and video/log artifact identifiers.

The official site already publishes Android APK downloads, hashes and signing-certificate fingerprints. Compare the acquired package with its own release information. Do not assume Play and direct-download builds have identical binary hashes or identical signing paths. iOS engine-resource comparison is distinct from reproducing an App Store executable.

## 1. Installation and offline preparation

1. Install the selected official build before the Signer device is isolated. Record its source and version.
2. Use a dedicated test device with a screen lock. Complete setup without using an existing funded recovery phrase.
3. Turn on Airplane Mode, then inspect Wi-Fi, cellular data and Bluetooth individually. Record their actual state. A product screenshot saying “no network connections” is not an observation of the test device.
4. Open Signer and perform the planned local operations while connections remain disabled. Record any network-related prompt or feature that fails.
5. Optional independent network instrumentation may strengthen the observation. A short capture with no packets does not prove that an application can never communicate.

## 2. Test identity and pairing

1. Use a disposable identity with no real funds. If loading a published synthetic vector requires its documented public test phrase, use only that fixture in a separate test wallet. Public test phrases must never receive funds.
2. Keep recovery words, recovery QR codes and passwords outside the recording. Public keys and addresses can still reveal account relationships; use test-only identities.
3. Export the documented public pairing data from Signer. Import it into the selected companion using the supported workflow for that exact version.
4. Compare fingerprint, network, account/path, script type and at least two derived public addresses where the format supports derivation. A plain address import cannot establish HD address derivation.
5. Record pairing and watch-only balance capability separately from request construction and signature return.

## 3. Synthetic request and review

1. For Bitcoin, start with the six supplied fixtures in `fixtures/public_psbt_ur_fixtures.json`. They use public test keys, mainnet syntax and fabricated inputs: no real UTXO is being spent. Never fund these keys or broadcast these transactions. The [published test-vector set](https://bittool.com/docs/reference/test-vectors/) is an additional source; record the exact fixture and source-byte hash rather than assuming all sets have the same coverage.
2. Independently decode the request before presenting it to Signer. Write down the expected recipient, amount, fee, network/chain ID, outputs/change and signing key/path as applicable.
3. Present the request by the documented QR/file input. Do not invent a UI command. The inspected Bitcoin release parser accepts mainnet only and explicitly rejects testnet; do not attempt to satisfy this procedure with Bitcoin testnet. If a companion cannot accept the fabricated-input fixture, mark that combination Blocked and keep a protocol-only test separate.
4. On Signer, compare the visible review with the independent decode. Inspect full fields and warnings, not just the headline summary. Record absent or ambiguous fields.
5. Test cancellation first. Confirm that no response is produced. Reopen a fresh copy of the same test-only request, authorize it and record the response.

## 4. Return and independent verification

1. Scan or import the response back into the companion. Confirm that it matches the original request, the intended wallet and the expected public key.
2. Independently verify the signature using a documented reference implementation. For Bitcoin PSBT, record partial signing and reference-library finalization separately. These fixtures contain fabricated previous outputs; finalizing their data structures does not establish a network-valid or spendable transaction.
3. Record response parsing, signature verification and transaction finalization as separate checks.
4. Broadcasting is prohibited for this fixture protocol. Label the result “synthetic mainnet-format request, fabricated inputs, no broadcast.”
5. Do not substitute funded mainnet inputs or present these engine results as an on-chain payment. Any later network exercise requires a separately designed test with documented network support; it is outside this evaluation pack.

## 5. Negative and recovery cases

Use only disposable fixtures. Run one change at a time and record expected versus observed behavior.

| Case | What to inspect | Result |
|---|---|---|
| Cancel before authorization | No new signature/response becomes available | Not run |
| Wrong wallet or derivation context | Request is refused or requires a correctly matching identity; no silent substitution | Not run |
| Bitcoin testnet request | Inspected release engine rejects it with NETWORK; verify native presentation separately | Not run |
| Interrupted multi-frame QR scan | No partial payload is signed; retry behavior is clear | Not run |
| Malformed or unsupported request | Clear refusal without crash or a misleading success message | Not run |
| Unexpected recipient, fee or change | Review exposes the actual request fields and any warning | Not run |
| Approval or authority change fixture | Full semantics remain inspectable; headline does not hide risk | Not run |
| App backgrounding or device relock | Secrets and authorization state behave as documented | Not run |
| Test-only recovery on a clean device | Expected public fingerprint/addresses are recovered from the same test phrase and passphrase | Not run |

Keep fixture results separate from reports about published open findings. Do not mark a finding fixed because its title sounds similar to one passing test. Include the affected release and the exact regression case.

## Evidence and publication

A result is reproducible only when the builds, devices, inputs, steps and observed outputs are recorded. Attach non-secret fixture hashes, decoded fields, return-signature verification, logs and time-coded video. Retain failures and limitations alongside successes.

Suggested compatibility wording after an actual pass:

> Tested with [Signer version/build] on [device/OS] and [companion version/build] on [device/OS], using [network/script/request type]. Pairing, request review and scanned-back response verification passed on [date]. The Bitcoin fixture used mainnet syntax and fabricated inputs; nothing was broadcast. Other combinations remain untested.

Do not shorten this to “works with all [wallet name] versions,” “certified,” or “audited.”

References: [protocol scope](https://github.com/bitbill/bittool-signer-ios/blob/main/docs/Compatibility.md), [integration guide](https://bittool.com/docs/quickstart/), [security status](https://bittool.com/docs/security/audits/).