Direct Answer
An adapter contract test harness generates a structured checklist for verifying a broker adapter's expected behavior, field mapping, schema validation, response contracts, and error handling, organized by criticality level. It turns implicit assumptions about how a broker's API behaves into an explicit, testable contract. Working through it before go-live catches integration bugs that only surface under real market conditions.
Tool
Order Types Your Strategy Uses
Time-in-Force Values Used
Features to Test
Checklist is advisory. Add, remove, or adjust tests based on your broker's specific API surface and your strategy's actual requirements.
How to Use This Tool
Enter the name of the broker or adapter you're testing. Select the primary asset class, API style, and order frequency your strategy uses. Check the order types, TIF values, and features you need to validate. Click Generate Test Checklist to produce a prioritized contract test checklist.
Each test item is labeled Critical (integration won't work without it), Important (integration degrades without it), or Optional (a quality-of-life improvement). Work through Critical items first in your sandbox environment. The "Verify with" hint in each test tells you what observable output to check.
This checklist is a starting point, not an exhaustive test suite. Your broker's API will have unique behaviors, add tests for any endpoint or behavior you discover that isn't covered here. The goal is to have at least one test per contract point before moving to a live environment.
Understanding the Outputs
Each checklist item corresponds to a specific contract point in your broker adapter. A contract test asserts that the adapter produces the correct canonical output for a given broker API input, it doesn't test that the broker itself works, but rather that your adapter correctly translates the broker's response into your system's expected format.
Critical tests cover the minimum viable contract: if any of these fail, your strategy can't function correctly. This includes: submit-and-confirm roundtrip (order reaches broker and fills are returned), idempotency (submitting the same order twice doesn't create two positions), and position sync (account state after fill matches broker's reported state).
Important tests cover behaviors that cause issues under real market conditions but might not surface in happy-path testing: partial fill handling, rate limit behavior at burst traffic, and cancel-and-replace roundtrip. These are the tests most likely to catch bugs that only appear in production.
Optional tests improve integration reliability but are rarely showstoppers: latency profiling, schema snapshot comparisons for non-critical fields, and redundant data validation. Tackle these after Critical and Important tests pass and you're satisfied with basic integration correctness.
Assumptions and Limitations
- Generated tests are structured as natural-language descriptions. Translate them into your test framework's test cases for execution.
- All tests should be run against your broker's sandbox environment first, then repeated against a live account with minimal capital before full deployment.
- This tool doesn't generate code, it generates a specification checklist. Actual test implementation depends on your language, framework, and adapter interface design.
- Broker APIs change. Re-run contract tests after any broker API update, even minor version updates, since field names, value formats, or error codes can change without notice.
FAQ
What is the difference between a contract test and an integration test?
An integration test checks that two real systems communicate correctly end-to-end. A contract test checks that your adapter (the translation layer) correctly converts the broker's API response into your internal canonical format, without necessarily running against a live broker API. Contract tests are faster. Don't require network access, and can be run in CI without real credentials. They use recorded API responses (snapshots) as fixtures instead of live connections. Integration tests are also necessary but run less frequently, typically scheduled daily or pre-deployment.
How do I record API response snapshots for contract tests?
Make a real API call during your initial sandbox integration and capture the raw JSON (or FIX message) response. Store it as a fixture file in your test suite. Your contract test then feeds this fixture to your adapter's parsing function and asserts the canonical output matches expectations. When the broker changes their response format, the snapshot diverges and the test fails, that's the contract break signal. Tools like VCR (Ruby), Betamax (Java), or Nock/Polly (JavaScript) automate HTTP response recording and playback.
How often should I run the contract tests in production?
Run the Critical-tier tests against a live sandbox every day, scheduled 90 minutes before market open. Run the full suite (Critical + Important + Optional) overnight when market impact risk is lowest. Any test failure should trigger an alert and block your strategy from trading until the contract break is investigated and either resolved or acknowledged as an acceptable change requiring adapter update.
What should I do if a Critical test fails in production?
Halt automated trading immediately for the affected strategy. Query the broker's API directly (or check their status page and changelog) to determine whether this is a broker-side change or a transient failure. If the broker changed their API, update the adapter and snapshot to match the new contract, update the test expectation, and re-run the full suite before resuming. If it's a transient failure, wait for the broker to resolve it and verify the test passes before restarting.
Should I include error response tests, not just success responses?
Yes, error contract tests are among the most important. Your adapter must correctly classify broker error responses: a "REJECTED" order that returns HTTP 200 with an error payload must not be treated as a success. Test each error category: insufficient funds, invalid symbol, rate limit exceeded, market closed, and ambiguous state (timeout with no fill confirmation). The test verifies that your adapter maps each error to the correct canonical error type and that your retry policy fires only for retryable errors.
How are recorded snapshots kept from going stale?
A snapshot captures a response at a point in time, so it keeps passing after the broker has changed, which is the failure mode the tests exist to catch. Pairing every snapshot test with a periodic live capture against the sandbox, and diffing the fresh response against the stored fixture, restores the signal. Recording the capture date with each fixture and flagging any that exceed an agreed age also makes staleness visible rather than leaving it to be discovered in production.
Should contract tests assert on fields the adapter does not use?
Asserting on the full response makes every unrelated broker change a test failure, which trains people to ignore failures. Asserting only on consumed fields misses the case where a new field carries information the adapter should now handle. A common compromise is strict assertions on consumed fields plus a separate, lower-severity check that reports schema additions and removals without failing the run, so unexpected changes are surfaced as information rather than as noise.
How should fields that change on every run be handled?
Timestamps, order identifiers, request identifiers, and sequence numbers differ between captures, so comparing them literally makes every test fail. The usual approach is normalizing them before comparison: substituting a placeholder, or asserting on shape and type rather than value, for example that a timestamp parses and falls within a plausible range. What matters is that normalization is applied deliberately to a named list of fields rather than by loosening comparison across the whole payload.
What belongs in a contract test rather than in the adapter unit tests?
A contract test answers whether the external interface still matches what the adapter expects. A unit test answers whether the adapter own logic is correct given a fixed input. Parsing a recorded response into the canonical model sits on the boundary and is usually covered by both: the contract test confirms the response still looks like the fixture, and the unit test confirms the mapping produces the right canonical output. Keeping the distinction clear stops a broker change from failing tests that have nothing to do with it.