Broker Integration Contract Testing

Direct Answer

A broker integration contract test is an automated test that sends a specific request to the broker's API (typically against the sandbox) and asserts that the response conforms to the contract your adapter depends on: the correct HTTP status code, the expected fields in the response body, the expected field types, and the expected behavior for your specific request parameters. When the broker changes its API, adding a required field, renaming an enum value, changing a response schema, the contract test fails before you deploy any code change, giving you advance warning of the breakage.

Contract testing is not end-to-end testing and not unit testing. It's specifically about the interface boundary between your adapter and the broker's API. It answers one question per test: "Does the broker still behave the way my adapter expects?" A comprehensive contract test suite runs on a schedule, daily or weekly, against the sandbox, not just as a one-time setup verification. The difference between a broker API that works today and one that silently changed last Tuesday is only visible if you're watching.

Key Takeaways

• Contract tests run on a schedule against the sandbox: They are not one-time setup tests. Schedule daily runs against the sandbox; a failing test that day means the broker changed something.
• Each test validates one specific behavior: "Submit a limit order and assert the response includes these fields with these types" is a good contract test. "Run a complete trading strategy" is not.
• Test field presence, type, and value separately: A field that exists but changed type (string to number) is a different failure than a field that was removed entirely. Write separate assertions for each dimension.
• Snapshot testing for response schemas: Capture the full response body from the sandbox, store it as a snapshot, and assert future responses match the snapshot structure. Any new field or changed field shows up in the diff.
• Include error contract tests: Test that when you submit an invalid order (bad symbol, negative quantity), you receive the specific error code your adapter handles for that case. If the broker changes the error code, your error handler breaks.
• Test WebSocket event schemas: Submit an order and record the WebSocket event structure for the fill. Assert the event schema matches expectations. Live WebSocket events may include extra fields that break rigid parsers.
• Alert on contract test failures before market open: Schedule contract tests to run 90 minutes before the market opens. A failure gives you time to investigate and disable live trading before the session starts, rather than discovering the failure mid-trade.
• Version contract snapshots: When you intentionally update a contract because the broker released a new API version, commit the new snapshot with a clear commit message. This creates an audit trail of when and why the contract changed.

Core Concepts

Defining a Contract

A contract between your adapter and a broker API is a documented specification of how each endpoint behaves for a specific input. It covers: the HTTP method and endpoint URL, the required and optional fields in the request body, the expected HTTP status code for a valid request, the guaranteed fields in the response body and their types, and the expected error code and structure for specific invalid inputs. A contract is written from your system's perspective, "given this input, I expect this output", not from the broker's marketing perspective of what they claim to support.

A minimal contract for the order submission endpoint might specify: POST /v2/orders with a JSON body containing symbol (string), qty (string representing a decimal), side (enum: "buy" | "sell"), type (enum: "market" | "limit"), and time_in_force; for a valid limit order request, the response is HTTP 200 with a JSON body containing id (string, UUID format), status (string), created_at (string, ISO 8601 datetime), and qty (string). For a request with an invalid symbol, the response is HTTP 422 with a JSON body containing code (number) and message (string).

Write the contract before writing the adapter, if possible. The contract becomes the specification you code against. When the contract changes (broker API update), you update the contract document first, then update the adapter code to match, then update the contract test snapshots. This order ensures you have explicit documentation of what changed and why, rather than chasing an unknown breakage.

Document contracts at the field level. For the status field in an order response, document every possible value you've observed ("pending_new", "new", "partially_filled", "filled", "cancelled", "expired", "rejected") and what each means for your adapter's state machine. When the broker adds a new status value (a real occurrence: Alpaca has added status values in past API updates), your contract test will detect the new value if you assert that the status field only contains known values.

Contract Test Patterns

Field mapping tests verify that your adapter correctly translates between canonical and broker-specific values. Test: create a canonical limit order {orderType: "limit", tif: "gtc"}, serialize it through the adapter's translation function, and assert the output JSON contains type: "limit" and time_in_force: "gtc" (for Alpaca). This is a pure unit test, no API call needed. It verifies the translation table itself, not the broker's behavior. Run these in your main CI pipeline on every commit.

Schema validation tests submit a request to the broker's sandbox and validate the response schema. They verify that the broker's response still contains the fields your adapter expects to parse. Implementation: submit a test order, receive the response JSON, run a JSON schema validator against it (with the schema defined from your contract document), and assert the validation passes. These tests require a live connection to the broker's sandbox and are suitable for a scheduled daily run, not every commit.

Behavior tests verify that the broker performs the expected action in response to your request. Test: submit a limit buy order, assert the response shows status "new", then query the open orders list and assert the order appears. Then cancel the order, assert the cancel response is 200, then query again and assert the order is no longer open. These tests verify the full request-response-state cycle, not just the response schema. They require sandbox access and take longer to run than schema tests.

Error contract tests deliberately submit invalid requests and assert the expected error responses. Test: submit an order with a negative quantity and assert the response is HTTP 422 with error code 40010001 (Alpaca's invalid quantity error). Test: submit an order for a nonexistent symbol and assert a 422 with the unknown symbol error code. When the broker changes an error code, for example, consolidating multiple 422 errors into a different structure, these tests fail immediately, giving you notice before your error handler starts misclassifying the error.

Snapshot Testing for Response Bodies

Snapshot testing captures a complete response body from a passing test and stores it as a "snapshot" file. Future test runs compare the live response against the stored snapshot. Any difference in field names, field types, added fields, or removed fields causes the test to fail with a diff showing exactly what changed. This approach catches breaking changes without requiring you to explicitly assert every field, you only need to write the snapshot capture once.

Implement snapshot testing for broker responses using a snapshot test library (Jest snapshots in JavaScript, pytest-snapshot in Python, or a custom implementation). For each endpoint you test, store a canonical example response: the submit-order response, the cancel-order response, the get-positions response, and the WebSocket order-update event. Run the test daily against the sandbox; when the snapshot diff shows the broker added a new field, investigate whether the new field requires an adapter update or can be safely ignored.

Update snapshots deliberately, not automatically. Snapshot test frameworks often offer a "update snapshots" command that silently updates all snapshots to match current behavior. Never run this command in CI without reviewing the diff, a broken contract that gets snapshot-updated is the same as not having the test at all. Review the diff, determine whether the change is a breaking change or a backward-compatible addition, and update the adapter and the snapshot together in the same commit with an explanatory commit message.

Version the snapshots alongside your code. Store them in your version control repository, not in a separate artifact store. When you checkout a specific commit of your adapter code, you should be able to run the contract tests against the snapshot from that same commit. This allows you to reproduce exactly what the contract expected at any point in history, which is essential for debugging "when did this start failing?" questions.

Running Contract Tests at Scale

A comprehensive contract test suite for a broker integration may include 50-200 individual test cases: one per endpoint × input combination × expected outcome. Running all 200 tests daily against the sandbox takes 5-30 minutes depending on API latency and test parallelism. Structure the test suite to enable selective running: group tests by endpoint (all order-submission tests together), by criticality (critical path tests that run on every deploy vs. exhaustive tests that run daily), and by test type (unit field-mapping tests that run fast vs. live sandbox tests that run slowly).

Schedule critical-path contract tests (order submission, cancellation, status query) to run 90 minutes before the market opens. Schedule exhaustive contract tests (all error cases, all endpoint variations, WebSocket schema tests) to run overnight. Alert on any critical-path test failure before the session starts. Alert on exhaustive test failures with a lower priority, as a signal to investigate before the next session.

For multi-broker systems, each broker gets its own contract test suite. Do not share contract tests across brokers, the whole point is that each broker has a different API contract, and the tests must encode those differences explicitly. Maintaining separate suites per broker also makes it clear when a broker-specific suite is outdated (because that broker released a new API version) while the other broker's suite is still passing.

When a contract test fails in the scheduled run, the investigation process should follow a fixed playbook: (1) Check whether the broker published a change log or API update announcement. (2) Compare the live sandbox response against the stored snapshot to identify exactly what changed. (3) Determine whether the change is breaking (your adapter crashes or produces wrong output) or non-breaking (a new optional field, a documentation update). (4) If breaking: update the adapter, update the contract snapshot, deploy to staging, run contract tests against staging. (5) If non-breaking: update the snapshot to reflect the new field, commit with a note that it's a backward-compatible change.

Worked Scenario

1. Setup. Your Alpaca adapter has a contract test suite with 85 tests. Tests run daily at 07:30 ET (90 minutes before open). Passing all 85 tests is a prerequisite for automated trading to start that session.
2. Failure detected. On a Tuesday, the 07:30 run fails 3 tests. Test names: order.submit.limitBuy.responseSchema, order.submit.marketBuy.responseSchema, order.cancel.responseSchema. These are all schema tests.
3. Diff review. The snapshot diff shows Alpaca added a new field to the order response: source (string, possible values: "api", "web", "mobile"). Your snapshot expected no source field. The diff is a pure addition, no existing fields changed.
4. Impact assessment. Your adapter parses the order response using a strict schema validator that fails on unexpected fields. This new field causes a parsing crash in the current adapter code. Breaking change confirmed.
5. Fix. Update the adapter's response parser to use permissive parsing (ignore unknown fields). Update the contract test snapshot to include the new source field as an optional field. Deploy updated adapter to staging, run full contract test suite against staging: all 85 tests pass.
6. Trading resumes. The fix was deployed by 08:30 ET, 60 minutes before the open. At 09:00 ET, the automated daily contract test re-runs against the updated adapter and all 85 tests pass. Automated trading starts at 09:30 ET without interruption.
7. If not caught. Without contract tests, the breaking change would be discovered when the first live order submission crashes the adapter's response parser at 09:30 ET. The position tracker never receives the OrderAck, the order is in ambiguous state, and the human intervention required may take 30-60 minutes to resolve, missing the session's best liquidity period.

Measurement Framework

Common Failure Modes

Contract Tests Only Run Once During Initial Setup

The most common failure in contract testing programs is treating the initial test run as sufficient validation. A developer writes contract tests, runs them against the sandbox, they pass, and the tests are committed, but never run again on a schedule. Three months later. The broker changes its response schema. The adapter breaks in live trading. The contract tests would have caught it with 24 hours notice if they had been running daily.

Contract tests have zero value without a recurring schedule. The CI pipeline that runs on every commit is not sufficient, most commits don't involve broker API changes. A separate scheduled job that runs the contract suite daily (and 90 minutes before each trading session open) is the minimum operational requirement for contract testing to provide its intended early-warning function.

Strict JSON Parsing Crashes on New Broker Fields

The scenario illustrated in the worked scenario: an adapter that uses strict JSON schema validation or a language's struct deserialization that fails on unexpected keys will crash when the broker adds a new response field. This is a category of production failure that contract tests exist to catch, but it's also preventable by writing more robust parsers from the start.

Use permissive parsing: extract only the fields you need from broker responses and ignore all others. If you use typed structs or models, configure them to ignore unknown fields (additionalProperties: false in JSON Schema validation should be applied to what you send, not what you receive). Document which fields your adapter depends on versus which it ignores, so future developers know the adapter's actual dependencies.

Error Contract Tests Not Written or Not Updated

Most contract test suites focus on the happy path, valid orders succeed. The error path contracts, invalid symbol, negative quantity, rate limit, authentication failure, are often skipped because they require deliberately triggering error states in the sandbox. When the broker changes an error response structure (adding a more specific error code, nesting the error in a different JSON key), the adapter's error classifier may misidentify the new format as a different error category, triggering wrong retry behavior.

Write error contract tests for every error category your adapter handles. Use a dedicated test instrument set: a symbol that always returns an invalid-symbol error (most brokers have one, e.g., INVALID_SYMBOL), a quantity of zero that triggers a minimum-quantity error, and an API key with rate-limit-testing access to the sandbox's rate limit endpoint. These tests are harder to set up but provide the most protection against silent error-handling regressions.

Contract Snapshots Silently Updated in CI

A CI configuration that runs --updateSnapshot automatically on test failure means every contract test failure results in the snapshot being updated to match the new (potentially broken) broker behavior. This defeats the entire purpose of snapshot testing. The tests always pass, they just silently document whatever the broker is currently doing, whether or not your adapter handles it correctly.

Never auto-update snapshots. Configure CI to fail hard on snapshot mismatch and require a human to review the diff, update the adapter if needed, and manually commit the new snapshot. The snapshot update should be a deliberate engineering decision with an audit trail in version control, not an automated response to a test failure.

FAQ

References

• Martin Fowler: Consumer-Driven Contracts: The foundational pattern for contract testing, explaining consumer-driven contracts and why they're more maintainable than provider-driven schemas.
• Pact Documentation: Pact is a widely used contract testing framework; its documentation explains contract testing concepts in depth even if you don't use Pact for broker integration specifically.
• JSON Schema Core Specification: The specification for JSON Schema, which is the natural tool for defining and validating broker API response contracts.
• Alpaca Markets: API Reference: Live broker API documentation serving as the ground truth for contract definitions; note that the API may diverge from documentation, which is exactly what contract tests are designed to detect.

Educational Disclaimer

This guide is for educational and informational purposes only. It does not constitute financial, investment, or legal advice. Broker API contracts change without notice. Contract testing reduces risk from API changes but cannot eliminate it entirely, always monitor live trading behavior alongside automated tests.