Bulk jobs and partial outcomes
Model imports, exports and large updates as observable operations with item results, cancellation and resumable recovery.
On this page
Before you start
- Item identity
- Job state transitions
- A permission and retention policy
Submit once and inspect completion
The submission response identifies a job. Its operation resource owns processing state and the final item outcomes.
Scroll across the diagram to see all four systems.
Submit manifest
Provide a bounded batch and one submission identity.
Persist job and work
Commit the job record and its eligible work items.
202 + Location
Return the operation resource while work remains incomplete.
Process bounded batches
Apply per item authorization, validation and duplicate protection.
Commit item outcomes
Record successful, failed and skipped items independently.
Read job status
Inspect progress, terminal outcome and downloadable item results.
Make the job a resource with its own lifecycle
Use a submission resource to establish the job and return its identifier. A 202 Accepted response means processing has been accepted; it does not promise that every item will succeed. Provide a Location for the status resource and a documented polling or notification policy.
Separate submission validation from item processing. A malformed manifest can reject the whole submission, while a valid manifest may contain individual records that fail business validation later. Publish that distinction before a client builds its recovery logic.
{
"job_id": "job_104",
"state": "queued",
"submitted": 500,
"processed": 0,
"succeeded": 0,
"failed": 0,
"links": {
"self": "/api/v1/jobs/job_104",
"results": "/api/v1/jobs/job_104/results",
"cancel": "/api/v1/jobs/job_104/cancellation"
}
}Define terminal and intermediate states
Queued, running, cancelling, completed and failed describe different operational conditions. A completed job can still contain failed items if partial success is part of the contract. Include separate counts and a terminal summary so the client does not infer success from the word completed alone.
Keep counts internally consistent: processed should equal the sum of terminal item outcomes under the chosen model. If skipped items are distinct, expose them separately. Do not count an item as successful when it has merely been handed to another queue.
| State | Meaning | Allowed next action |
|---|---|---|
| Queued | Durable submission awaits execution. | Inspect or request cancellation. |
| Running | At least one item is being processed. | Inspect progress; cancellation may be cooperative. |
| Cancelling | The worker is stopping future work. | Wait for confirmed terminal outcomes. |
| Completed | All eligible items have terminal outcomes. | Inspect success and failure counts; download results. |
| Failed | The job cannot continue under its current contract. | Review the failure and retry only eligible work. |
Preserve item identity across batches
Give every submitted item a stable client reference and processing outcome. A result file should identify its source item, destination resource if created, and a machine readable error if rejected. This enables selective correction rather than resubmitting the entire import.
Use item action identities that survive worker retries and job resume. A job level idempotency key protects duplicate submission; it does not automatically protect each destination mutation inside the job.
Cancel future work without pretending to undo committed work
Document whether cancellation is immediate or cooperative. An in flight item may finish after the cancellation request. Report the resulting completed, cancelled and unresolved items rather than claiming that a cancellation rolled back external changes.
For a business operation that needs compensation, model the compensating action explicitly and authorize it separately. A refund, consent restoration or record deletion is not a generic inverse of an earlier write.
Protect the result artifact as carefully as the API
Large exports create a second access surface. Bind the result to its tenant and requester policy, recheck permission at download where appropriate, and expire temporary access according to the data handling requirement. Avoid permanent public URLs for customer data.
Test cancellation during a batch, a worker restart after partial commit, duplicate submission, mixed authorization outcomes, expired result links and a result file whose counts do not match the job summary.
Evaluate a bulk job outcome
Inspect the distinction between accepted work, completed items, partial failure and cooperative cancellation.
/api/v1/jobs/job_104Expected outcome
Job complete
Every submitted item has a successful terminal outcome.
{
"job_id": "job_104",
"state": "completed",
"submitted": 500,
"processed": 500,
"succeeded": 500,
"failed": 0
}Content-Type: application/jsonDecision path
- Load the operation
- Verify terminal item accounting
- Publish the completed summary
Input schema 4 fields
| Field | Type | Meaning |
|---|---|---|
totalRequired | integer | Eligible submitted items.Minimum: 1 |
processedRequired | integer | Items with a terminal outcome.Minimum: 0 |
failedRequired | integer | Processed items that failed.Minimum: 0 |
cancel_requestedRequired | boolean | Whether cancellation has been requested. |
Standards and documentation
Explore the protocol specifications and platform guidance for this topic.
Bring the contract into your operating environment.
Discuss an implementation