Traces, delivery evidence and operations
Connect request traces to business outcomes without exposing credentials or confusing a successful transport with completed work.
On this page
Before you start
- A service ownership map
- Redaction rules
- Business outcome definitions
One operation across several services
Transport traces explain where time was spent. The business action reference explains whether the intended work completed.
Scroll across the diagram to see all four systems.
Propagate trace context
Validate incoming traceparent and establish the local request span.
Link queued work
Carry permitted trace metadata and the business action reference.
Record destination attempt
Measure the dependency call without logging secrets or the full customer payload.
Confirm outcome
Distinguish accepted transport, completed business result and uncertain state.
Publish operation evidence
Record the terminal outcome or the next recovery action.
Expose request reference
Give the caller a safe identifier for support and status inspection.
Propagate a standard trace context
W3C Trace Context defines traceparent and tracestate for correlation across systems. traceparent carries the version, trace identifier, parent span identifier and trace flags in a defined format. Validate incoming values and let the tracing implementation apply its trust policy.
A trace identifier is correlation metadata, not an authentication credential. Never authorize an operation based on a caller supplied trace value. Treat vendor specific tracestate and any additional baggage according to your privacy and propagation policy.
POST /api/v1/campaign-activations HTTP/1.1
Host: api.example.test
traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
Idempotency-Key: tenant_a:campaign_19:revision_4
Content-Type: application/json
{ "campaign_id": "campaign_19", "revision": 4 }Keep trace, event and action identity distinct
A trace groups related execution spans. An event identity describes a source occurrence. A business action key protects the intended effect across attempts. Reusing one value for all three makes a replay difficult to interpret because a new attempt can belong to the same action and source event.
Record links between these identities. A support view should answer which source event triggered the action, which attempt last ran, what the destination confirmed and what the next recovery step requires.
| Identifier | Lifetime | Question it answers |
|---|---|---|
| Request ID | One API request attempt | Which incoming request produced this response? |
| Trace ID | Related execution spans | Where did this execution spend time or fail? |
| Source event ID | One source occurrence under its namespace | Which change was delivered? |
| Business action key | One intended effect across retries | Would repetition create a second business action? |
| Operation or job ID | Durable processing lifecycle | What is the current and final business outcome? |
Measure work age and correctness, not just traffic
A healthy HTTP success rate can coexist with a growing processing backlog. Track the age of the oldest unresolved item, time since the last successful source update, processing lag, destination rejection classes and reconciliation differences.
Choose dimensions that are useful and bounded. Raw contact IDs, request IDs and arbitrary error text make poor metric labels because they create high cardinality. Keep detailed identifiers in logs and traces with appropriate access controls.
Make diagnostic collection safe by construction
Redact authorization headers, signing secrets and session cookies before logs leave the process. Prefer allowlisted diagnostic fields over collecting an entire request and attempting to remove sensitive values afterward.
A failure record can retain schema version, source type, operation class, safe tenant reference, error category and request reference without publishing the customer payload. Apply retention and access policy to diagnostics as well as primary data.
Attach an action to every operational signal
An alert needs an owner, a condition that matters and a recovery procedure. Distinguish loss of source access, temporary capacity, invalid data and uncertain destination outcome. Each requires a different response and should not share one generic retry button.
Exercise the runbook with a controlled fault: pause a destination, revoke a test grant, deliver an invalid event and stop a worker after persistence. Confirm that the evidence identifies the stage and that recovery preserves the original business action identity.
Interpret operation health
Combine transport, backlog and reconciliation signals to decide what an operator should inspect.
/api/v1/operations/operation_42Expected outcome
Outcome confirmed
The business result is confirmed and the reviewed reconciliation scope has no unresolved differences.
{
"operation_id": "operation_42",
"state": "completed",
"request_id": "req_2409",
"next_action": null
}Content-Type: application/jsonDecision path
- Read durable operation state
- Confirm the destination outcome
- Check reconciliation context
Input schema 4 fields
| Field | Type | Meaning |
|---|---|---|
transport_okRequired | boolean | Whether the latest transport attempt succeeded. |
business_completeRequired | boolean | Whether the intended business action is confirmed complete. |
oldest_pending_secondsRequired | number | Age of the oldest unresolved item.Minimum: 0 |
reconciliation_differencesRequired | integer | Unresolved differences in the defined comparison scope.Minimum: 0 |
Standards and documentation
Explore the protocol specifications and platform guidance for this topic.
Bring the contract into your operating environment.
Discuss an implementation